how2itsec: Git

Showing posts with label Git. Show all posts

Git pull on subfolders

In order to update your whole IDE (e.g. visual studio code), you can run git pull to all subfolders one by one using:

ls | xargs -I{} git -C {} pull

Or you run it in parallel for multiple subfolders using:

ls | xargs -P10 -I{} git -C {} pull

Example git pull one by one


DEVFLO@SYS40DEVCL001 MINGW64 /c/git

$ 

DEVFLO@SYS40DEVCL001 MINGW64 /c/git

$ 

DEVFLO@SYS40DEVCL001 MINGW64 /c/git

$ cd splunk-apps/ 

DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps

$ ls | xargs -I{} git -C {} pull

Already up to date.

remote: Enumerating objects: 64, done.

remote: Counting objects: 100% (28/28), done.

remote: Compressing objects: 100% (16/16), done.

remote: Total 64 (delta 15), reused 12 (delta 12), pack-reused 36

Unpacking objects: 100% (64/64), 12.38 KiB | 56.00 KiB/s, done.

From https://git.dev.domain.tld/splunk-apps/all_indexes

   21ef148..dc3191d  master     -> origin/master

Updating 21ef148..dc3191d

Fast-forward

 local/indexes.conf | 95 ++++++++++++++++++++++++++++++++++++++++++------------

 1 file changed, 75 insertions(+), 20 deletions(-)

remote: Enumerating objects: 19, done.

remote: Counting objects: 100% (19/19), done.

remote: Compressing objects: 100% (14/14), done.

remote: Total 16 (delta 8), reused 6 (delta 2), pack-reused 0

Unpacking objects: 100% (16/16), 2.00 KiB | 34.00 KiB/s, done.

From https://git.dev.domain.tld/splunk-apps/hf_inputs_http

   45221cc..9ba7459  master     -> origin/master

Updating 45221cc..9ba7459

Fast-forward

 local/inputs.conf | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++--

 1 file changed, 72 insertions(+), 2 deletions(-)

remote: Enumerating objects: 314, done.

remote: Counting objects: 100% (33/33), done.

remote: Compressing objects: 100% (28/28), done.

remote: Total 314 (delta 17), reused 5 (delta 5), pack-reused 281

[...]

Example git pull in parallel


DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps

$

DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps

$

DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps

$ ls | xargs -P10 -I{} git -C {} pull

Already up to date.

Already up to date.

Already up to date.

Already up to date.

Already up to date.

Already up to date.

remote: Enumerating objects: 101, done.

remote: Counting objects: 100% (76/76), done.

remote: Compressing objects: 100% (38/38), done.

remote: Total 101 (delta 46), reused 53 (delta 35), pack-reused 25

Receiving objects: 100% (101/101), 101.59 KiB | 12.70 MiB/s, done.

Resolving deltas: 100% (61/61), completed with 1 local object.

From https://git.dev.domain.tld/splunk-apps/search_securitywork

   7c8b27a..913ed1e  master     -> origin/master

Updating 7c8b27a..913ed1e

Fast-forward

 {local => default}/data/ui/nav/default.xml         |   0

 .../ui/views/security__asset_information.xml   |   4 +-

 .../ui/views/security__dashkpi1__logsource.xml |   4 +-
.../ui/views/security__dashkpi2__usecases.xml  |   4 +-

 .../views/security__dashkpi3__technology.xml   |   2 +-

 .../ui/views/security__dashkpi4_logvolume.xml  |   2 +-
{local => default}/macros.conf                     |   0  

 {local => default}/mlspl.conf                      |   0

 {local => default}/savedsearches.conf              | 160 +++++++++++++++++----

 {local => default}/transforms.conf                 |   0

 lookups/uc_info.csv                            |   8 +-

 14 files changed, 152 insertions(+), 50 deletions(-)

 rename {local => default}/data/ui/nav/default.xml (100%)

 rename {local => default}/data/ui/views/security__asset_information.xml (94%)

 rename {local => default}/data/ui/views/security__dashkpi1__logsource.xml (99%)

 rename {local => default}/data/ui/views/security__dashkpi2__usecases.xml (99%)

 rename {local => default}/data/ui/views/security__dashkpi3__technology.xml (97%)

 rename {local => default}/data/ui/views/security__dashkpi4_logvolume.xml (99%)

 [...]

Fix blocked ldap user in GitLab container using GitLabs shell

If you are running GitLab in a docker container and your are using some directory service, for example ActiveDirectory with LDAPS for authentication, you might face the challenge, that when a user is moved in ActiveDirectory to another ad-group or the ad-group which is used as user-filter is deleted, then GitLab marks the user as "blocked".

Unblock the ldap user in GitLab

Connect to the docker host server
Open a connection to GitLabs Shell using docker exec -it <container-name> gitlab-rails console -e production
Find the user in GitLabs Shell using user = User.find_by_email("someone@e-mail")
Check the Users state using user.state
Unblock the user using user.state = "active"
Save using user.save
Exit

Example:

prdrhel8180:/ #

prdrhel8180:/ # docker exec -it gitlab gitlab-rails console -e production

--------------------------------------------------------------------------------

 Ruby:         ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]

 GitLab:       15.3.1-ee (518311979e3) EE

 GitLab Shell: 14.10.0

 PostgreSQL:   12.10

------------------------------------------------------------[ booted in 37.73s ]

Loading production environment (Rails 6.1.6.1)

irb(main):001:0> user = User.find_by_email("someone@e-mail")

=> nil

irb(main):002:0> user = User.find_by_email("someone@e-mail.com")

=> #<User id:55 @someone>

irb(main):003:0> user.state

=> "ldap_blocked"

irb(main):004:0> user.state = "active"

=> "active"

irb(main):005:0> user.save

=> true

irb(main):006:0> exit

prdrhel8180:/ #

prdrhel8180:/ #

Fix the LDAP user filter

If the user was blocked due to a deleted AD group, which was used as ldap user filter, then you have to fix the LDAP connect from GitLab to ActiveDirectory. GitLab will log this in /var/log/gitlab/gitlab-rails/application.log as:

2023-02-02T01:30:18.098Z: LDAP account "cn=lastname\, firstname,ou=deleted-users,ou=someou,dc=internal,dc=domain,dc=local" does not exist anymore, blocking GitLab user "Lastname, Firstname" (firstname.lastname@domain.local)

prdrhel8180:/ #
prdrhel8180:/ # docker exec -it gitlab cat /etc/gitlab/gitlab.rbgitlab_rails

gitlab_rails['ldap_servers'] = YAML.load <<- br="">
 someldap: #
 
    label: 'LDAP'

    host: 'some-ldaps-vip.internal.domain.local'

    port: 636

    uid: 'sAMAccountName'

[...]

   user_filter: '(|(memberOf=CN=SomeGroup,OU=Groups,OU=SomeOU,DC=internal,DC=domain,DC=local)(memberOf=CN=SomeGroup2,OU=Groups2,OU=SomeOU2,DC=internal,DC=domain,DC=local))'

prdrhel8180:/ #
prdrhel8180:/ #

The user_filter has to be adjusted to the new AD group, which includes the blocked user(s).

Git for Windows - Git uses certificate from internal PKI

If you initially want to git clone a repository from an internal git system from a windows system and the internal git system uses a certificate from a internal PKI, you'll receive an error: SSL certificate problem: self signed certificate in certificate chain

PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk> git clone https://git.domain.tld/ansible-playbooks/carbonblack-threatfeeds.git Cloning into 'carbonblack-threatfeeds'... fatal: unable to access 'https://git.domain.tld/ansible-playbooks/carbonblack-threatfeeds.git/': SSL certificate problem: self signed certificate in certificate chain PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk>

Check your windows git settings with git config -l. In the attribute http.sslcainfo is the ca-cert-bundle-file, which is used by git as certification authority certificate store.

PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk> git config -l diff.astextplain.textconv=astextplain filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.process=git-lfs filter-process filter.lfs.required=true http.sslbackend=openssl http.sslcainfo=C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt core.autocrlf=true core.fscache=true core.symlinks=false pull.rebase=false credential.helper=manager-core credential.https://dev.azure.com.usehttppath=true init.defaultbranch=master user.email=flo@some.mail user.name=flo core.editor="C:\Users\flo\AppData\Local\Programs\Microsoft VS Code\Code.exe" --wait PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk>
So append your CA certificate in the file "C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt". Then it works:

PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk> git clone https://git.domain.tld/ansible-playbooks/carbonblack-threatfeeds.git Cloning into 'carbonblack-threatfeeds'... remote: Enumerating objects: 104, done. remote: Counting objects: 100% (104/104), done. remote: Compressing objects: 100% (58/58), done. remote: Total 104 (delta 50), reused 93 (delta 40), pack-reused 0 eceiving objects: 34% (36/104) Receiving objects: 100% (104/104), 20.03 KiB | 2.86 MiB/s, done. Resolving deltas: 100% (50/50), done. PS C:\Users\flo\Splunk> PS C:\Users\flo\Splunk>

how2itsec