Almost perfect protection for websites and other services - Mutual TLS

Its hard to secure your IT services and applications. The list of possible attacks is long, as shown in the Mitre Att&ck framework, the OWASP lists and so on.

What helps drastically reducing the attack surface is to use mutual TLS (explanations see https://www.cloudflare.com/learning/access-management/what-is-mutual-tls/ or https://en.wikipedia.org/wiki/Mutual_authentication#mTLS or https://www.youtube.com/watch?v=x7B5CwcxCDI or https://www.ietf.org/rfc/rfc5246.txt) for authentication to establish access to your service/application. If you use certificates to authenticate your clients before accessing the first byte to of your service/application, you stop attacks at OSI layer 5. The attackers cant reach OSI layer 6 and 7 which  contain the most vulnerabilites and weaknesses by far.

I've asked ChatGPT to make a list to compare regular web applications with web applications, which use mutual TLS:

Attack TypeRegular Web ApplicationSecured Web Application (Mutual TLS)Mitre ATT&CK Tactic(s)
Cross-Site Scripting (XSS)VulnerableProtectedInitial Access, Execution, Persistence
SQL InjectionVulnerableProtectedCollection, Credential Access, Execution
Cross-Site Request Forgery (CSRF)VulnerableProtectedInitial Access, Collection
Session HijackingVulnerableProtectedCollection, Credential Access
Brute Force AttacksVulnerableProtectedCredential Access, Execution
ClickjackingVulnerableProtectedCollection, Defense Evasion
Directory TraversalVulnerableProtectedCollection, Exfiltration
File Upload VulnerabilitiesVulnerableProtectedExecution, Collection
LDAP InjectionVulnerableProtectedCollection, Credential Access
Man-in-the-Middle (MitM) AttacksVulnerableProtectedCollection, Credential Access, Execution
Remote File InclusionVulnerableProtectedExecution, Collection
XML External Entity (XXE) InjectionVulnerableProtectedCollection, Credential Access
Security MisconfigurationsVulnerableWay less vulnerableDefense Evasion, Discovery
HTTP Header InjectionVulnerableProtectedDefense Evasion, Execution
DDoS AttacksVulnerableWay less vulnerableImpact
API Security IssuesVulnerableProtectedCollection, Credential Access, Execution
HTTP Parameter PollutionVulnerableProtectedDefense Evasion, Execution
Session SidejackingVulnerableProtectedCredential Access, Collection
Social Engineering AttacksVulnerableProtectedCollection, Defense Evasion
Cross-Site Tracing (XST)VulnerableProtectedCredential Access, Collection
Fuzzing AttacksVulnerableProtectedExecution, Defense Evasion
CryptanalysisVulnerableWay less vulnerableCollection, Credential Access
Reverse EngineeringVulnerableProtectedCollection, Credential Access
Subdomain TakeoverVulnerableProtectedInitial Access, Collection
Session Token TheftVulnerableProtectedCredential Access, Collection
Vulnerabilies/0daysVulnerableWay less vulnerableInitial Access
Broken Object Level Authorization (BOLA)VulnerableProtectedCredential Access, Authorization
Improper Rate LimitingVulnerableWay less vulnerableDefense Evasion, Impact
Insecure Direct Object References (IDOR)VulnerableProtectedCollection, Credential Access
Insufficient Logging and MonitoringVulnerableWay less vulnerableDiscovery, Defense Evasion
Insecure DeserializationVulnerableProtectedExecution, Defense Evasion
Lack of Resources and Rate LimitingVulnerableProtectedDefense Evasion, Impact
Mass AssignmentVulnerableProtectedCredential Access, Collection
Insecure Cryptographic StorageVulnerableVulnerableCollection, Credential Access
Use of Components with Known VulnerabilitiesVulnerableWay less vulnerableCollection, Execution
Unvalidated Redirects and ForwardsVulnerableProtectedDefense Evasion, Initial Access
XML External Entity (XXE) InjectionVulnerableProtectedCollection, Credential Access

Caution: This is a simplified point of view - it is only focusing on the inital access. Mutual TLS certificate based authentication make the initial access for attackers almost impossible. Therefore most attacks are not possible anymore without the attackers gets his hands on a client certificate with its private key. However your IT stack is still vulnerable to OSI layer 1-5 attacks as well as e.g. to vulnerabilities within OpenSSL, LibreSSL etc.


Proxmox update error "Sub-process /usr/share/proxmox-ve/pve-apt-hook returned an error code (1)"

Problem

During a proxmox update (e.g. from proxmox version 6 to 7) you receive the following error:

[...]
100% [608 zstd 34.1 kB/630 kB 5%] 1,337 kB/s 0s
100% [Working] 1,337 kB/s 0s

Fetched 255 MB in 2min 30s (1,702 kB/s)
W: (pve-apt-hook) !! WARNING !!
W: (pve-apt-hook) You are attempting to remove the meta-package 'proxmox-ve'!
W: (pve-apt-hook)
W: (pve-apt-hook) If you really want to permanently remove 'proxmox-ve' from your system, run the following command
W: (pve-apt-hook) touch '/please-remove-proxmox-ve'
W: (pve-apt-hook) run apt purge proxmox-ve to remove the meta-package
W: (pve-apt-hook) and repeat your apt invocation.
W: (pve-apt-hook)
W: (pve-apt-hook) If you are unsure why 'proxmox-ve' would be removed, please verify
W: (pve-apt-hook) - your APT repository settings
W: (pve-apt-hook) - that you are using 'apt full-upgrade' to upgrade your system
E: Sub-process /usr/share/proxmox-ve/pve-apt-hook returned an error code (1)
E: Failure running script /usr/share/proxmox-ve/pve-apt-hook
root@proxmox1:~#
root@proxmox1:~# 

 

Solution

 
Proxmox VE 6.x is based on Debian 10.x which is called “buster”.
Proxmox VE 7.x is based on Debian 11.x which is called “bullseye”.  

  1. Check if your /etc/apt/sources.list.d/pve-enterprise.list file and /etc/apt/sources.list file still have "buster" (proxmox version 6) in them and replace it with "bullseye". E.g. with sed -i -e 's/buster/bullseye/g' /etc/apt/sources.list.d/pve-install-repo.list
  2. Run apt update again
  3. Run apt dist-upgrade again

Update Proxmox 6.x to latest 6.4

Update a Proxmox 6.x system to latest 6.4 using the guide https://pve.proxmox.com/wiki/Downloads#Update_a_running_Proxmox_Virtual_Environment_6.x_to_latest_6.4:

Proxmox VE 6.x is based on Debian 10.x which is called “buster”.

  1. Make sure you have a backup of all VMs, Containers, Proxmox itself etc.
  2. Login via SSH/CLI
  3. Check your sources.list file, should look like this:

    cat /etc/apt/sources.list

    deb http://deb.debian.org/debian buster main contrib
    deb http://deb.debian.org/debian buster-updates main contrib
    # security updates
    deb http://security.debian.org buster/updates main contrib

  4. Check the enterprise repository:

    cat /etc/apt/sources.list.d/pve-enterprise.list

    When running Proxmox VE 6.x with No-Subscription use:

    deb http://download.proxmox.com/debian/pve buster pve-no-subscription

    When running Proxmox VE 6.x with a subscription use:

    deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise

  5. Check Proxmox version using:

    pveversion -v


  6. Update your repository and packages:

    apt update

    If you get any errors, your sources.list (or your system or network) has a problem.
  7. Now upgrade the packages:

    apt dist-upgrade

  8. Reboot to activate the new Kernel, to check if you got all packages, run 'pveversion -v' and compare your output (all packages should have equal or higher version numbers): 
  9. Check Proxmox version using

    pveversion -v 


Example:

root@prxmx053b:~#
root@prxmx053b:~# cat /etc/apt/sources.list
deb http://deb.debian.org/debian buster main contrib
deb http://deb.debian.org/debian buster-updates main contrib
# security updates
deb http://security.debian.org buster/updates main contrib
root@prxmx053b:~#
root@prxmx053b:~#
root@prxmx053b:~# apt update
Hit:1 http://security.debian.org buster/updates InRelease
Hit:2 http://download.proxmox.com/debian/pve buster InRelease
Hit:3 http://deb.debian.org/debian buster InRelease
Hit:4 http://deb.debian.org/debian buster-updates InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
242 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@prxmx053b:~#
root@prxmx053b:~#
root@prxmx053b:~# apt dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
[...]
root@prxmx053b:~#
root@prxmx053b:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.73-1-pve)
pve-manager: 6.4-15 (running version: 6.4-15/af7986e6)
pve-kernel-5.4: 6.4-20
pve-kernel-helper: 6.4-20
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+deb10u1
corosync: 3.1.5-pve2~bpo10+1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.22-pve2~bpo10+1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.1.0-1
libpve-access-control: 6.4-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-5
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-5
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.14-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.6-2
pve-cluster: 6.4-1
pve-container: 3.3-6
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.3-2
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-8
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.7-pve1
root@prxmx053b:~#
root@prxmx053b:~# reboot


AKS Security - SIEM UseCase of Get Credentials

A SIEM usecase or forensic security alert for Azure Kubernetes Service AKS should be setup for az aks get-credentials as it reveals sensitive data of AKS.

Example for Azure Cloud Shell Bash:

azureksmoq [ ~ ]$
azureksmoq [ ~ ]$ az aks get-credentials --resource-group rgaks04app23 --name AKS04
Merged "AKS04" as current context in /home/azureksmoq/.kube/config
azureksmoq [ ~ ]$ 
azureksmoq [ ~ ]$
azureksmoq [ ~ ]$ cat /home/azureksmoq/.kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZ[...]RVJUSUZJQ0FURS0tLS0tCg==
    server: https://mykubernetescluster-dns-[...].hcp.eastus.azmk8s.io:443
  name: AKS04
contexts:
- context:
    cluster: AKS04
    user: clusterUser_rgaks04app23_AKS04
  name: AKS04
current-context: AKS04
kind: Config
preferences: {}
users:
- name: clusterUser_rgaks04app23_AKS04
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ[...]RS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJV[...]0VZLS0tLS0K
    token: bl0c8ko2[...]73m4ltf
azureksmoq [ ~ ]$ 
azureksmoq [ ~ ]$ 

This command can be helpful for developers or admins - but it is dual use as it reveals sensitive information. I recommend increasing your SIEM risk score or even make a use case with alerting.

Azure Documentation

Microsoft documentation for az aks get-credentials can be found here: https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials or see here: https://azure.github.io/kubelogin/quick-start.html. See also https://learn.microsoft.com/en-us/azure/aks/control-kubeconfig-access

Get access credentials for a managed Kubernetes cluster.

By default, the credentials are merged into the .kube/config file so kubectl can use them. See -f parameter for details.

az aks get-credentials --name
                       --resource-group
                       [--admin]
                       [--context]
                       [--file]
                       [--format]
                       [--overwrite-existing]
                       [--public-fqdn]

Examples

Get access credentials for a managed Kubernetes cluster. (autogenerated)

az aks get-credentials --name MyManagedCluster --resource-group MyResourceGroup

Required Parameters

--name -n

Name of the managed cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--admin -a

Get cluster administrator credentials. Default: cluster user credentials.

default value: False
--context

If specified, overwrite the default context name. The --admin parameter takes precedence over --context.

--file -f

Kubernetes configuration file to update. Use "-" to print YAML to stdout instead.

default value: ~\.kube\config
--format

Specify the format of the returned credential. Available values are ["exec", "azure"]. Only take effect when requesting clusterUser credential of AAD clusters.

--overwrite-existing

Overwrite any existing cluster entry with the same name.

default value: False
--public-fqdn

Get private cluster credential with server address to be public fqdn.

default value: False
Global Parameters
--debug

Increase logging verbosity to show all debug logs.

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

--output -o

Output format.

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

 

Git pull on subfolders

In order to update your whole IDE (e.g. visual studio code), you can run git pull to all subfolders one by one using:

ls | xargs -I{} git -C {} pull

Or you run it in parallel for multiple subfolders using:

ls | xargs -P10 -I{} git -C {} pull

Example git pull one by one

DEVFLO@SYS40DEVCL001 MINGW64 /c/git
$
DEVFLO@SYS40DEVCL001 MINGW64 /c/git
$
DEVFLO@SYS40DEVCL001 MINGW64 /c/git
$ cd splunk-apps/
DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps
$ ls | xargs -I{} git -C {} pull
Already up to date.
remote: Enumerating objects: 64, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 64 (delta 15), reused 12 (delta 12), pack-reused 36
Unpacking objects: 100% (64/64), 12.38 KiB | 56.00 KiB/s, done.
From https://git.dev.domain.tld/splunk-apps/all_indexes
21ef148..dc3191d master -> origin/master
Updating 21ef148..dc3191d
Fast-forward
local/indexes.conf | 95 ++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 75 insertions(+), 20 deletions(-)
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 16 (delta 8), reused 6 (delta 2), pack-reused 0
Unpacking objects: 100% (16/16), 2.00 KiB | 34.00 KiB/s, done.
From https://git.dev.domain.tld/splunk-apps/hf_inputs_http
45221cc..9ba7459 master -> origin/master
Updating 45221cc..9ba7459
Fast-forward
local/inputs.conf | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 72 insertions(+), 2 deletions(-)
remote: Enumerating objects: 314, done.
remote: Counting objects: 100% (33/33), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 314 (delta 17), reused 5 (delta 5), pack-reused 281
[...]

Example git pull in parallel

DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps
$
DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps
$
DEVFLO@SYS40DEVCL001 MINGW64 /c/git/splunk-apps
$ ls | xargs -P10 -I{} git -C {} pull
Already up to date.
Already up to date.
Already up to date.
Already up to date.
Already up to date.
Already up to date.
remote: Enumerating objects: 101, done.
remote: Counting objects: 100% (76/76), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 101 (delta 46), reused 53 (delta 35), pack-reused 25
Receiving objects: 100% (101/101), 101.59 KiB | 12.70 MiB/s, done.
Resolving deltas: 100% (61/61), completed with 1 local object.
From https://git.dev.domain.tld/splunk-apps/search_securitywork
7c8b27a..913ed1e master -> origin/master
Updating 7c8b27a..913ed1e
Fast-forward
{local => default}/data/ui/nav/default.xml | 0
.../ui/views/security__asset_information.xml | 4 +-
.../ui/views/security__dashkpi1__logsource.xml | 4 +-
.../ui/views/security__dashkpi2__usecases.xml | 4 +-
.../views/security__dashkpi3__technology.xml | 2 +-
.../ui/views/security__dashkpi4_logvolume.xml | 2 +-
{local => default}/macros.conf | 0
{local => default}/mlspl.conf | 0
{local => default}/savedsearches.conf | 160 +++++++++++++++++----
{local => default}/transforms.conf | 0
lookups/uc_info.csv | 8 +-
14 files changed, 152 insertions(+), 50 deletions(-)
rename {local => default}/data/ui/nav/default.xml (100%)
rename {local => default}/data/ui/views/security__asset_information.xml (94%)
rename {local => default}/data/ui/views/security__dashkpi1__logsource.xml (99%)
rename {local => default}/data/ui/views/security__dashkpi2__usecases.xml (99%)
rename {local => default}/data/ui/views/security__dashkpi3__technology.xml (97%)
rename {local => default}/data/ui/views/security__dashkpi4_logvolume.xml (99%)
[...]

Microsoft Portals overview - msportals.io

The website msportals.io is listing a nice overview of Microsofts portals. For example administrator portals:

Microsoft 365 Admin Portals


Many more can be found on
msportals.io.

 

Mitre D3fend Framework - Att&ck from a defenders perspective

Mitre published another awesome framework called d3fend.mitre.org

Mitre D3fend

It is using the att&ck framework but from a defenders perspective :-) 

D3fend Att&ck relationship



Confluence behind LoadBalancer with another domain results in XSRF error

If you have an atlassian confluence running, which is published by a loadbalancer or reverse proxy using another domain, you might run into an XSRF error.

Example

Confluence FQDN: somehostname.domain.tld
LoadBalancer Confluence FQDN: confluence.domain.tld

Some actions like uploading your profile picture (https://confluence.domain.tld/users/profile/editmyprofilepicture.action) do not work. You'll receive an generic error from the confluence page (see red box of the screenshot below). If you check the HTTP Header response, you'll see XSRF check failed. It is caused by the confluence cross site request forgery (CSRF) protection.

Confluence XSRF Error

Solution

Edit confluence server.xml and add the FQDN from the LoadBalancer or reverse proxy.

More information can be found here: https://confluence.atlassian.com/kb/cross-site-request-forgery-csrf-protection-changes-in-atlassian-rest-779294918.html



Splunk Enterprise update plan

Splunk published this awesome Splunk Enterprise update plan: https://docs.splunk.com/images/d/d3/Splunk_upgrade_order_of_ops.pdf 

Regardless if you have a single-site or multi-site splunk installation, if your are running a stand-alone or distributed and/or clustered architecture, if you are using Splunks Universal Forwarder, the Deployment server, a License Master, Search Head cluster or Indexer Cluster master or not - this plan has your environment setup covered.

Step by step it guides you in updating your Splunk Enterprise environment including backuping up every system, checking each systems health and possible connectivity issues as well as the updates itself, may it be a simple upgrade or a rolling upgrade. Additional informations about each step can be found in the PDF as a link to docs.splunk.com.

Splunk Enterprise update plan step by step


 

Phishing using @-URL trick in DNS .zip domains

With Googles release of the DNS top-level-domains .zip and .mov a new phishing (mitre att&ck T1566) trick is possible as bobbyrsec wrote about.

Example 1

https://www.google.com/?q=example.text  <— FQDN = google.com

Example 2

https://www.google.com/example/text/@v1271.zip <— FQDN = google.com right? No, it is v1271.zip. Because the @ character describes e.g. the authentication of the URL.

Example 3

https://www.google.com/example/text/v1271.zip <— FQDN = google.com

Reason



Result

So doublechecking URLs becomes harder. Using Fido2, Passkeys or password-managers (e.g. bitwarden.com) with auto-fill becomes more important because they dont fall for that trick and are more phishing-resistant.

Overview of public interfaces for SOC/IT-Security staff

In case of an IT-security incident, emergency oder if a new critical vulnerability (like log4j in December 2021) arises, it is good to be prepared, so you can quickly answer questions like:

  • "Are we affected?"
  • "Do we use this technology?"
  • "Where do we use this vulnerable protocol?"
  • "To whom is the attack surface exposed to?"
  • "Are there mitigations in place?"
  • "Is is exploitable without authentication in our setup?"
  • "Which is the best place to place a first mitigation?"
  • etc..
An overview like the following can and will be helpful for your IT-security staff or your Security Operations Center SOC:

System Internet Facing Protocol Authentication Security Used Products/Vendors Logs send to SIEM Contact Person Known Weaknesses
Websites Yes, exposed to all public-ip-addresses HTTPS (TCP:443) & HTTP (TCP:80 - HTTP 301 Redirect to HTTPS) None Web Application Firewall F5 BigIP LoadBalancer WAF & Apache Container on OpenShift Yes Link to CMDB Websites may contain 3rd party code, SBOM see CMDB
Managed File Transfer Yes, but limited to dedicated public ip-addresses of partners HTTPS (TCP:443) HTTPS Tokens Web Application Firewall F5 BigIP LoadBalancer WAF IPSwitch Yes Link to CMDB Runs on VM as appliance, OS might not be hardend from vendor
Citrix Yes, exposed to all public-ip-addresses HTTPS (TCP:443) MFA Netscaler WAF Citrix Systems + Okta MFA Yes Link to CMDB NetScaler WAF Ruleset might be out-of-date
Mailserver Yes, exposed to all public-ip-addresses SMTP (TCP:25) None AntiSpam Mailgatway & AV-Sandbox Cisco E-Mail Security Yes Link to CMDB Mailgateways run on Hardware, might not be hardended from vendor
SSLVPN S2E Yes, exposed to all public-ip-addresses HTTPS (TCP:443) Mutual TLS Certbased + MFA Azure DDoS FortiGate SSLVPN Azure VM + Okta MFA Yes Link to CMDB Possible FortiGate FortiOS SSLVPN Vulnerabilities
M365 ActiveSync Yes, exposed to all public-ip-addresses HTTPS (TCP:443) Mutual TLS Certbased Azure DDoS Microsoft 365 + Intunes Yes Link to CMDB Not part of own Vulnerability-Scanner
VPN S2S Yes, but limited to dedicated public ip-addresses of partners IPSec UDP:500 & UDP:4500 & ESP IPsec IKEv2 Certbased Auth Azure DDoS FortiGate SSLVPN Azure VM Link to CMDB -
DMARC SaaS Yes, exposed to all public-ip-addresses DNS (UDP:53), HTTP (TCP:80), HTTPS (TCP:443), SMTP (TCP:25) None - dmarcadvisor.com SaaS No Link to CMDB Not part of own Vulnerability-Scanner
DNS Server Yes, but limited to dedicated public ip-addresses of partners DNS (UDP:53 & TCP:53) None Azure Network Security Groups RHEL Bind Yes Link to CMDB -
ISP Routers Yes, but limited to dedicated public ip-addresses of ISP routers BGP (TCP:179), BFD, Ping (ICMP:0/8) BGP MD5 Auth - Extreme Networks XOS Yes Link to CMDB
etc.. etc.. etc.. etc.. etc.. etc.. etc.. etc.. etc..

 

Of course you can add many more columns like e.g.:

  • "SBOM technologys used" (for example: RHEL, Apache Tomcat, OpenSSL, log4j, puppet, ansible, splunk universal forwarder, appdynamics,..)
  • Direct links to your Firewall Management System, WAF or SIEM
  • "Is it part of our vulnerability scanner?"
  • "Is the vulnerability scanner scanning it authenticated?"
  • "Is the system/application hardended?"
  • and so on :-)
This list will help in case of an IT-security emergency to sort out the first steps in order to mitigate and fix the issue of the public exposed interfaces (like to the internet or to business partners). However this is only one of many steps necessary - always "asume breach" and make sure an attacker controlling a client or server still is unable to spread (unnoticed) in your companies (cloud) network.

Windows fileless persistence local privilege escalation using sc.exe sdset

Alh4zr3d and Grzegorz Tworek wrote about a fileless, local privilege escalation backdoor in order to gain persistence (TA0003) using the following Windows command:

sc.exe sdset scmanager D:(A;;KA;;;WD) 

Windows sc.exe persistence fileless backdoor

Setting the security descriptor on the service manager allows anyone to start SYSTEM services.

To show the security descriptor of service manager:

sc.exe sdshow scmanager

 

The Mitre Att&ck framework has technique T1569.002 for this "System Services: Service Execution", writing: "Adversaries may abuse the Windows service control manager to execute malicious commands or payloads. The Windows service control manager (services.exe) is an interface to manage and manipulate services. The service control manager is accessible to users via GUI components as well as system utilities such as sc.exe and Net."

Microsofts own documentation about sc.exe sdset:

Sets a service's security descriptor, using the Service Descriptor Definition Language (SDDL).

Syntax

sc [<ServerName>] sdset <ServiceName> <ServiceSecurityDescriptor>

Parameters

Parameter

Description

<ServerName>

Specifies the name of the remote server on which the service is located. The name must use the Universal Naming Convention (UNC) format (for example, \\myserver). To run SC.exe locally, omit this parameter.

<ServiceName>

Specifies the service name returned by the getkeyname operation.

<ServiceSecurityDescriptor>

Specifies the service descriptor in SDDL.

/?

Displays help at the command prompt.


To explain the DACL D:(A;;KA;;;WD) gav_gall asked ChatGPT to explain:

ChatGPT DACL Windows persistence backdoor


Fix blocked ldap user in GitLab container using GitLabs shell

If you are running GitLab in a docker container and your are using some directory service, for example ActiveDirectory with LDAPS for authentication, you might face the challenge, that when a user is moved in ActiveDirectory to another ad-group or the ad-group which is used as user-filter is deleted, then GitLab marks the user as "blocked".

Unblock the ldap user in GitLab

  1. Connect to the docker host server
  2. Open a connection to GitLabs Shell using docker exec -it <container-name> gitlab-rails console -e production
  3. Find the user in GitLabs Shell using user = User.find_by_email("someone@e-mail")
  4. Check the Users state using user.state
  5. Unblock the user using user.state = "active"
  6. Save using user.save
  7. Exit

Example:

prdrhel8180:/ #
prdrhel8180:/ # docker exec -it gitlab gitlab-rails console -e production
--------------------------------------------------------------------------------
Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
GitLab: 15.3.1-ee (518311979e3) EE
GitLab Shell: 14.10.0
PostgreSQL: 12.10
------------------------------------------------------------[ booted in 37.73s ]
Loading production environment (Rails 6.1.6.1)
irb(main):001:0> user = User.find_by_email("someone@e-mail")
=> nil
irb(main):002:0> user = User.find_by_email("someone@e-mail.com")
=> #<User id:55 @someone>
irb(main):003:0> user.state
=> "ldap_blocked"
irb(main):004:0> user.state = "active"
=> "active"
irb(main):005:0> user.save
=> true
irb(main):006:0> exit
prdrhel8180:/ #
prdrhel8180:/ #

Fix the LDAP user filter

If the user was blocked due to a deleted AD group, which was used as ldap user filter, then you have to fix the LDAP connect from GitLab to ActiveDirectory. GitLab will log this in /var/log/gitlab/gitlab-rails/application.log as:

2023-02-02T01:30:18.098Z: LDAP account "cn=lastname\, firstname,ou=deleted-users,ou=someou,dc=internal,dc=domain,dc=local" does not exist anymore, blocking GitLab user "Lastname, Firstname" (firstname.lastname@domain.local)

prdrhel8180:/ #
prdrhel8180:/ # docker exec -it gitlab cat /etc/gitlab/gitlab.rbgitlab_rails
gitlab_rails['ldap_servers'] = YAML.load <<- br=""> someldap: #
label: 'LDAP'
host: 'some-ldaps-vip.internal.domain.local'
port: 636
uid: 'sAMAccountName'
[...]
user_filter: '(|(memberOf=CN=SomeGroup,OU=Groups,OU=SomeOU,DC=internal,DC=domain,DC=local)(memberOf=CN=SomeGroup2,OU=Groups2,OU=SomeOU2,DC=internal,DC=domain,DC=local))'
prdrhel8180:/ #
prdrhel8180:/ #

The user_filter has to be adjusted to the new AD group, which includes the blocked user(s).

New LAPS version explained

Microsoft will release a new version of Local Administrator Password Solution (LAPS), which   provides new Azure AD features as well as new Active Directory OnPrem features and some migration features from the old version to the new one.

A video explaining everything in detail can be found here:


This video includes a nice overview showing how LAPS is working internally using CSP (lapscsp.dll), PowerShell (lapspsh.dll) or GPOs and LAPS core logic (laps.dll) which then reads and updates the expiry of accounts as well as updates their password, either in Azure Active Directory or in Windows Server Active Directory on premise:LAPS internal logic and flow architecture

Source: https://learn.microsoft.com/en-us/windows-server/identity/laps/laps-concepts

LAPS can be used as solution against pass-the-hash (https://attack.mitre.org/techniques/T1550/002) and lateral-traversal attacks (https://attack.mitre.org/tactics/TA0008), as well as for securing user help desk access or recover to devices with a fine-grained security model and for RBAC in Azure AD.

 

Paessler PRTG 22.4.81.1532 security fix for Cross Site Scripting XSS

Paessler PRTG released version 22.4.81.1532 (stable) in which a PRTG the tag handling system was fixed regarding tag parameters to avoid the risk of a possible Cross Site Scripting (XSS) attack.

Updates are available for PRTG Preview 22.4.81.1504 or PRTG Stable 22.4.81.1532
  • PRTG Stable 22.4.81.1532 sha256 installer exe 0DD1952B8EE8A56F77FBA968366794BFA58ABEFC38151192D378EAF35607091F
  • PRTG Stable 22.4.81.1532 sha256 installer zip 64F4918CE265ED58EC98996516D59AA97BB7BBC614AAD644E2F90C9DAB22F106
  • PRTG Preview 22.4.81.1504 sha256 installer exe: 28F05875EA31067881B2E04B3557F1BBC3D174D2064CCBA3E93BD4F1EC2E7839
  • PRTG Preview 22.4.81.1504 sha256 installer zip A82725C7C7BBA8E116F31030857BD3DE099FF5B7C40248E04A1DB425A617318E
Source: https://www.paessler.com/prtg/history/stable

Security Testing - Review LAPS

It is a good idea to review your companys Microsoft's Local Administrator Password Solution LAPS installation. Leo Loobeek published a nice powershell-script which helps in finding groups which are specifically delegated by sysadmins and finding users with "All Extended Rights" that can view passwords, and viewing all computers with LAPS enabled.

https://github.com/leoloobeek/LAPSToolkit
https://github.com/leoloobeek/LAPSToolkit/blob/master/LAPSToolkit.ps1

  • Get-LAPSComputers
    Displays all computers with LAPS enabled, password expriation, and password if user has access

  • Find-LAPSDelegatedGroups
    Searches through all OUs to see which AD groups can read the ms-Mcs-AdmPwd attribute

  • Find-AdmPwdExtendedRights
    Parses through ExtendedRights for each AD computer with LAPS enabled and looks for which group has read access and if any user has "All Extended Rights". Sysadmins may not be aware the users with All Extended Rights can view passwords and may be less protected than the users in the delegated groups. An example is the user which adds a computer to the domain automatically receives the "All Extended Rights" permission. Since this function will parse ACLs for each AD computer, this can take very long with a larger domain.
I've seen installations, where LAPS was implemented to improve security. However due to wrong configuration way to many people where able to read passwords in Active Directory (Mitre Att&ck Tactic Credential Access)

Microsoft Windows Defender AntiVirus Performance analysis

When you suspect your Microsoft Defender Antivirus to be a bottleneck for your Windows performance, then you may use Microsofts Defender Antivirus performance analyzer. It helps you with the on-premise Windows Defender Antivirus as well as with the cloud solution Microsoft Defender for Endpoint (Defender ATP).

https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/tune-performance-defender-antivirus?view=o365-worldwide

Especially on developer systems with an IDE Microsoft Defender Antivirus can have a significant performance impact on your system due to the many temporary files, which are not digitally signed but contain exectuable code. Microsofts Defender Antivirus performance can help you to detect:

  • Files with long antivirus scan times
  • Processes with long antivirus scan times
  • File extensions with long antivirus scan times 

Running defender antivirus performance analyzer

  1. Run PowerShell (Admin)
  2. Use the PowerShell command New-MpPerformanceRecording -RecordTo how2itsec-analyze-microsoft-antivirus.etl
  3. Repeate your performance issue, e.g. building your software or opening a programm
  4. Press Enter to stop the trace

Defender Antivirus performance analysis etl

Analysis of the trace 

You can analyze your results using the Get-MpPerformanceReportparameter with one of the following arguments:
Get-MpPerformanceReport    [-Path] <String>
[-TopScans <Int32>]
[-TopFiles  <Int32>
    [-TopScansPerFile <Int32>]
    [-TopProcessesPerFile  <Int32>
        [-TopScansPerProcessPerFile <Int32>]
    ]
]
[-TopExtensions  <Int32>
    [-TopScansPerExtension <Int32>]
    [-TopProcessesPerExtension <Int32>
        [-TopScansPerProcessPerExtension <Int32>]
        ]
    [-TopFilesPerExtension  <Int32>
        [-TopScansPerFilePerExtension <Int32>]
        ]
    ]
]
[-TopProcesses  <Int32>
    [-TopScansPerProcess <Int32>]
    [-TopExtensionsPerProcess <Int32>
        [-TopScansPerExtensionPerProcess <Int32>]
    ]
]
[-TopFilesPerProcess  <Int32>
    [-TopScansPerFilePerProcess <Int32>]
]
[-MinDuration <String>]
[-Raw]

Example Analysis

Get-MpPerformanceReport -Path .\how2itsec-analyze-microsoft-antivirus.etl -TopFiles 10Get-MpPerformanceReport Defender analysis1

Get-MpPerformanceReport -Path .\how2itsec-analyze-microsoft-antivirus.etl -TopFiles 10 -TopScansPerFile 3 Get-MpPerformanceReport Defender analysis files scans per file

Get-MpPerformanceReport -Path .\how2itsec-analyze-microsoft-antivirus.etl -TopExtensions:10 -TopProcesses:10 -TopScans:10Get-MpPerformanceReport Defender analysis2 Top processes top scans per file

Get-MpPerformanceReport -Path .\how2itsec-analyze-microsoft-antivirus.etl -TopScans:100 -MinDuration:100msGet-MpPerformanceReport Defender analysis3 processes scan duration

Get-MpPerformanceReport -Path .\how2itsec-analyze-microsoft-antivirus.etl -TopScans:100 -MinDuration:500ms -Raw | ConvertTo-Js

Debug Windows Defender AntiVirus Performance JSON

Optimize performance 

Based on your analysis results you can carefully set exclusions or adjust parameters in Windows Defender or Defender for Endpoint (Defender ATP) in order to boost performance.

FortiGate admin interface authentication bypass

There are rumors about a vulnerability in Fortinets FortiGate firewalls where you may bypass authentication on their admin interfaces. Affected seem to be FortiOS 7.0.x and FortiOS 7.2.0/1. A fix is included in FortiOS 7.0.7 and FortiOS 7.2.2. It is written the vulnerability has CVE-2022-40684.

It is imperative that you protect your FortiGate interfaces with TrustedHosts AND Local-In-Policies. Only using TrustHosts protects HTTPS, SSH, etc but not other protocols like SIP, IPsec, CAPWAP, BGP, SSLVPN* etc which are also local services running on the FortiGate, which need to be protected, too.

*SSLVPN = Even though SSLVPN might be not configured and therefore seems to be inactive, in some cases for example vulnerability-scanners still trigger the SSLVPN service to log errors in FortiGates log. This only is solved by setting up local-in-polices.

Example for trusthost & local-in-policy:

CLI configuration:
System > Administrators >
config system admin
    edit "admin"
        set trusthost1 172.26.73.48 255.255.255.255
        set accprofile "super_admin"
        set vdom "root"
    next
end

Configuring address and address group as per the trusted hosts:
config firewall address
    edit "trusted-1"
        set type ipmask
        set comment ''
        set visibility enable
        set associated-interface ''
        set color 0
        set allow-routing disable
        set subnet 172.26.73.48 255.255.255.255
    next
end

config firewall addrgrp
    edit "trusted_grp"
        set member "trusted-1"
        set comment ''
        set visibility enable
        set color 0
    next
end
Configuring Firewall local in policies:
config firewall local-in-policy
    edit 2
        set intf "port1"
        set srcaddr "trusted_grp"
        set dstaddr "all"
        set action accept
        set service "PING"
        set schedule "always"
        set status enable
        set comments ''
    next
    edit 1
        set intf "port1"
        set srcaddr "all"
        set dstaddr "all"
        set action deny
        set service "PING"
        set schedule "always"
        set status enable
        set comments ''
    next
end

Remember: This example shows the local-in-policy only for "ping". You want to protect all services (except for example SSLVPN or IPsec if you use them).

Mitre Att&ck Micro Emulations - Test your own security

It is a very good idea to test your own IT-security systems and processes, if they detect something and what level of detail they provide. Mitre Engenuity launched a new project called: Micro Emulation

Mitre Micro Emulations

On GitHub version 4 was already released providing the first set of tools, which contains ActiveDirectory Enumeration, FileAccess, NamedPipes, ProcessInjection, UserExecution ISOBypass, Marcos and Shortcuts, WebShells, WindowsRegistry: https://github.com/center-for-threat-informed-defense/adversary_emulation_library/releases .

Windows Persistence Scheduled Tasks Obfuscation

In the great article „Securonix Threat Labs Security Advisory: Detecting STEEP#MAVERICK“ D. Iuzvyk, T. Peck, O. Kolesnikov reverse engineer an attack campaign where some windows scheduled tasks are used for persistence with some obfuscation for evasion of security detection: https://www.securonix.com/blog/detecting-steepmaverick-new-covert-attack-campaign-targeting-military-contractors/

Quote of the passage:

„Stage (7): Persistence – Scheduled Tasks

The script also attempts to embed itself as a scheduled task on the affected host. The task names itself one of two names depending on the permission level:

  • MicrosoftEdgeUpdateTaskMachine_System
  • MicrosoftEdgeUpdateTaskMachine_User

The task is created using some clever obfuscation to hide the call to “schtasks.exe”. It uses an invoke expression mixed with wildcard matching so “$env:???t??r???\*2\??h???k?*” translates to “$env:SYSTEMROOT\System32\schtasks.exe”.

The task is designed to run the exact same script that we noticed in the registry persistence section, however the invoked script is named “w” instead of “u” and it was hosted on a different C2 URL, however the code was identical and produced matching file hashes.

Figure 17: Persistence – Scheduled Tasks“

Paessler PRTG fixes OpenSSL vulnerability CVE-2022-1292

Paesslers PRTG version 22.3.79 will update its internal OpenSSL libraries to 1.0.2ze in order to address and fix CVE-2022-1292. CVE-2022-1292 is categorized as CWE-78 (Improper Neutralization of Special Elements used in an OS Command 'OS Command Injection').

PRTG release notes of version 22.3.79:
[Security] We updated our OpenSSL libraries to version 1.0.2ze that patches CVE-2022-1292.

CVE-2022-1292 is about the OpenSSL c_rehash script, which does not properly sanitise shell metacharacters to prevent command injection.

Citrix ICA SSO saved credentials in XOR obfuscated readable storage

The Citrix ICA application stores user credentials for its SingleSignOn SSO functionality in readable form using XOR obfuscation with the key „C“, as Benjam Delpy wrote: https://twitter.com/gentilkiwi/status/1570525137962930176

Mimikatz version 3 will be able to reveal this as shown in the following GIF with Windows 11 and Credential Guard enabled: https://video.twimg.com/tweet_video/Fcudz49XoAAfNHO.mp4


Quick win securing Azure AD

An easy quick win for securing Azure Active Directory passwords is the feature "Azure AD Password Protection". This helps you in mitre att&cks tactic credential access, for example in the technique brute force and its sub-techniques password guessing, password spraying, credential stuffing, etc.

Users are recommended to avoide simple passwords and instead should use pass-sentences. Password breaches of the recent past reveal that the majority still chose simple passwords. Azure AD Password Protection finds (audit mode) or enforces (enforce mode) stronger passwords for everybody. There is a hidden global banned password list which is applied to every user in the Azure AD tenant. Additionally you can block custom words like your companys name, your companys slogan, the founders or CEOs name, most used childerens names or your country or famous sport team names, which are often used as weak passwords.

Azure AD security custom banned passwords

Microsoft promises the "password validation algorithm" automatically detects/blocks variants and combinations like "password!1", "!password", "p@ssw0rd" and so on.

This is also available for on-premise Active Directory using an agent:

On-Premise Active Directory security quick win

 

However this needs an additional license, you need to install and agent on your domain controllers, you need to reboot the domain controllers and you need Azure AD. Also you do not see the changing content of Microsofts global password list and there is no enforcement based on Active Directory groups or OUs, so you for example cant just enforce it to priviledged accounts but must enforce it for everybody.

Of course I highly recommend to use Multi Factor Authentication MFA everywhere.

Decrypt TLS sessions of Chrome & Firefox

How to decrypt TLS Sessions of browsers like Chrome and Firefox without Man-in-the-Middle MitM/Adversary-in-the-Middle AitM (Att&ck T1557) like ARP Cache Poisoning, DNS Spoofing, LLMNR/NBT-NS Poisoning and SMB Relay, DHCP Spoofing, Proxy, burp, PAC, WPAD, etc:

This is a silent way to debug issues. However your EDR/SIEM/logings solution should check for SSLKEYLOGFILE entries in your environment variables because this is a silent way to break TLS without informing the user.

Use the SSLKEYLOGFILE in your environmentvariables of your windows, linux or macos system.

Windows

Simply create an environmentvariable called SSLKEYLOGFILE and set the value of the path to the text-file, in which the TLS private keys should be saved:

Windows environmentvariables SSLKEYLOGFILE TLS decrypt
Linux

Simply create an environmentvariable called SSLKEYLOGFILE and set the value of the path to the text-file, in which the TLS private keys should be saved:

export SSLKEYLOGFILE=$HOME/tlsprivatekeys.txt

Linux SSLKEYLOGFILE decrypt TLS

Decrypt TLS sessions with Wireshark

  1. Open the wireshark prefecenses
  2. Go to TLS 
  3. Select your SSLKEYLOGFILE text file as (Pre)-Master-Secret filename as shown in the following screenshot:
Wireshark to decrypt TLS using privatekeys SSLKEYLOGFILE





Almost perfect protection for websites and other services - Mutual TLS

Its hard to secure your IT services and applications. The list of possible attacks is long, as shown in the Mitre Att&ck framework , the...