FortiOS update with versions 5.2.14/5.4.12/5.6.10/6.0.5/6.2.1
Fortinet released two "Customer Support Bulletins" (check fortinet support portal) regarding a possible issue with uninterruptible upgrade in FortiGate HA-clusters running version FortiOS 5.2.14 or 5.4.12 or 5.6.10 or 6.0.5 or 6.2.1. Please check the two Customer Support Bulletins before upgrading and plan a possible short downtime for the update.
Securing Layer 2 / Switing Level
Ideas for securing your layer 2 switching infrastructure
No matter if you use Cisco, Extreme Networks, HP or any other vendor, most of them have the same basic security features, which are rarely used, but actually really should be. Here are some ideas, which you should use for ideal protection:
Securing Switch Management Interfaces
- ACLs --> to limit all management interfaces (SSH, SNMP, etc) to very few dedicated ip-addresses
- Use SSH
- Disable telnet
- SNMPv3 AuthPriv AES+SHA
- Disable SNMPv1/v2c
- Disable HTTP/HTTPS
- Delete default users, groups & communities
- Use radius authentication and two local backup-accounts with long passwords (30+ characters), those should be only useable when the radius-servers are not reachable anymore, otherwise they should not work
- Use syslog (reliable & encrypted if possible) --> send these logs to a SIEM and define alerts/SIEM use cases
- Fail2ban or similar login protection
- Use Out-of-band management interfaces
- Think about using serial-to-ip adapters for out-of-band-management, if you have a dedicated seperated out-of-band-mgmt network
- Disable LLDP/CDP/EDP/etc where possible
- Setup IT-Monitoring (PRTG, Icinga2, Nagios,..) and check CPU, RAM, NIC-Utilization & Errors of critial links like LAGs, Uplinks, Mgmt, etc, 802.1x-Auth-Errors-per-Min, Temperature, Fans, Hardware-Status,
- Setup IT-Monitoring to do negative testing like try to access the switch via Telnet/HTTP/SNMPv1/v2c or SSH/SNMPv3 with default accounts or try to access the switches from a forbidden ip-address (which is not part of the ACL) in order to find possible cfg-errors, holes or cfg mistakes after a switch got replaced, reconfigured or a firmware-update
- Limit Access to VirtualStacking/MultiChassis-Links like MLAG, VSS etc using dedicated ACLs; If possible use internal ip-addresses like 169.254.0.0/16 if you have to use ip-addresses for them
Securing Traffic passing the switches
- Use 802.1x Authentication with certificates from your pki
- Port-Level Traffic Controls and port security mechanisms to provide protection against MAC flooding attacks
- Private VLAN (PVLAN)
- Access Lists on Switches if necessary, normally done on a firewall
- Spanning Tree Protocol Features
- Dynamic Host Configuration Protocol (DHCP) Snooping
- IP Source Guard
- Dynamic ARP Inspection (DAI)
- Advanced Integrated Security Features on High-End Catalyst Switches
- Control Plane Policing (CoPP) Feature
- CPU Rate Limiters
- M/BCast Limitations or Warnings
- Loop Protection as BPDU Guard, Loopguard, ELRP, BCast-Storm-Control, (M/R)STP, Root Guard, etc
- Use VLAN tagging to seperate user-traffic from network traffic like LLDP, (M)STP, OSPF, BGP, BFD, etc. --> Then those protocols are not mixed with user-traffic (example: STP is send untagged while user-traffic is sent tagged)
- Shut down or disable all unused ports on the switch
- Use small subnets, not bigger than /24 for good performance/stability and for small segmentation --> in general: the more segmentation, the better
Extend ZFS zpool of Linux Ubuntu NextCloud VM
If you are running a virtual machine with NextCloud v16 on linux ubuntu and you want to extend /mnt/ncdata, then:
- Make sure your backup is fine, think about creating a vm snapshot
- Extend second virtual harddrive disk in VMWare virtual machine settings (e.g. from 40Gb to 170Gb)
- Login using SSH/VMWare-Console, change to root user using
sudo suorsudo -i - Check the zpool size using
zpool list - Check the /mnt/ncdata size using
df -h - Read the new partition size using
parted -lwith the answer "fix" for the adjustment - You can delete the buffer partition 9 using
parted /dev/sdb rm 9 - Extend the first partition using to 100% of the available size
parted /dev/sdb resizepart 1 100% - Using
zpool online -e ncdata /dev/sdbyou can adjust the partition to the correct size - Check the new zpool size using
zpool list - Check the new /mnt/ncdata size using
df -h
root@nextcloud:/mnt# zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
ncdata 39.8G 15.2M 39.7G - 0% 0% 1.00x ONLINE -
root@nextcloud:/mnt#
root@nextcloud:/mnt#
root@nextcloud:/mnt# df -Th
Filesystem Type Size Used Avail Use% Mounted on
[...]
ncdata zfs 39G 896K 39G 1% /mnt/ncdata
[...]
root@nextcloud:/mnt#
root@nextcloud:/mnt# parted -l
[...]
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 183GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 42.9GB 42.9GB zfs zfs-28cd7
9 42.9GB 42.9GB 8389kB[...]
root@nextcloud:/mnt#
root@nextcloud:/mnt#
root@nextcloud:/mnt# parted /dev/sdb rm 9
Information: You may need to update /etc/fstab.
root@nextcloud:/mnt#
root@nextcloud:/mnt# parted /dev/sdb resizepart 1 100%
Information: You may need to update /etc/fstab.
root@nextcloud:/mnt#
root@nextcloud:/mnt# zpool online -e ncdata /dev/sdb
root@nextcloud:/mnt#
root@nextcloud:/mnt# zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
ncdata 170G 2.51M 170G - 0% 0% 1.00x ONLINE -
root@nextcloud:/mnt#
root@nextcloud:/mnt# df -h
Filesystem Size Used Avail Use% Mounted on[...]
ncdata 165G 896K 165G 1% /mnt/ncdata[...]
root@nextcloud:/mnt#
Hint when VMWare doesnt show a snapshot
If VMWare ESXi/vCenter/your vSphere environment does not show a virtual machine snapshot, however it is there and the virtual machine is using it, here is a little trick which might help you:
- Make sure you have some free space left in your datastore, in which the vm is stored.
- Create another virtual machine snapshot
- Click on "delete all snapshots"
FortiGate default configuration does not verify the LDAP server identity - CVE-2019-5591
I have found a vulnerability in all FortiOS versions, including the current 5.4/5.6/6.0/6.2 branches. The issue has been fixed in 6.0.3/6.2.1 by using the new feature "server-identity-check":
Update to FortiOS 6.0.3+ or 6.2.1+ and set the following option:
config user ldap
edit ldap-server
set server-identity-check enable
The vulnerability is in the LDAPS connection of the FortiGate to a
LDAP-Server. The FortiGate does not properly check the certificate sent from the
LDAP-Server, allthough the correct CA certificate is configured. More details will be published later.
Fortinet PSIRT-team responded quickly, has acknowledged the issue, told me that some one else also reported the issue, assigned CVE-2019-5591 to it and released the following PSIRT advisory: https://fortiguard.com/psirt/FG-IR-19-037
Solution:
Update to FortiOS 6.0.3+ or 6.2.1+ and set the following option:
config user ldap
edit ldap-server
set server-identity-check enable
Migrate nextcloud v15 with mariadb database to nextcloud v16 with postgresql
If you are running nextcloud v15 with mariadb and want to upgrade from nextcloud version 16, then you have to migrate the database from mariadb to postgresql.
This can be done using the following commands, which I adjusted for ubuntu 16.04 and is originally from the following site: https://www.techandme.se/we-migrated-to-postgresql/
#!/bin/bash
## Convert to PostgreSQL ##
# Tested on Ubuntu Server 16.04
# Make sure you have a full backup of your nextcloud installation
# Make sure only root can run our script
if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root, please type sudo -i and run it again." 1>&2
exit 1
fi
service apache2 stop
. <(curl -sL https://raw.githubusercontent.com/nextcloud/vm/master/lib.sh)
NCUSER=pgsql_user_nextcloud
# Install PostgreSQL
apt update
check_command apt install postgresql-9.5
# Create DB
cd /tmp || exit
sudo -u postgres psql <<END
CREATE USER $NCUSER WITH PASSWORD '$PGDB_PASS';
CREATE DATABASE nextcloud_db WITH OWNER $NCUSER TEMPLATE template0 ENCODING 'UTF8';
END
check-command service postgresql restart
# Convert DB
sudo -u www-data php /var/www/nextcloud/occ db:convert-type --all-apps --password "$PGDB_PASS" pgsql $NCUSER 127.0.0.1 nextcloud_db
sudo -u www-data php /var/www/nextcloud/occ maintenance:repair
# Remove MySQL / MariaDB
read -p "Are you sure you want to remove MySQL?" -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]
then
apt clean
apt update
dpkg -r mariadb-client-10.2
dpkg -r mariadb-server-10.2
dpkg -r libmysqlclient20:i386
dpkg -r libmysqlclient20:amd64
dpkg -r libmysqlclient18:amd64
dpkg -r mysql
apt purge mysql\* libmysql\* libmariadb\*
apt autoremove -y
rm -R /var/lib/mysql /var/lib/mysql-files /var/lib/mysql-keyring /var/mysql-upgrade /etc/mysql /var/lib/mysql
fi
# Remove mysql.utf8mb4
if grep -q "mysql.utf8mb4" /var/www/nextcloud/config/config.php
then
sed -i "s|'mysql.utf8mb4' => true,||g" /var/www/nextcloud/config/config.php
sed '/^\s*$/d' /var/www/nextcloud/config/config.php
fi
# Show password
echo "Your new PostgreSQL password is: $PGDB_PASS. It's also written in your Nextcloud config.php file."
# Start Apache2
echo "Apache will start in 30 seconds... Press CTRL+C to abort."
sleep 30
service apache2 start
# Fetch the correct update script
if [ -f "$SCRIPTS"/update.sh ]
then
rm "$SCRIPTS"/update.sh
wget https://raw.githubusercontent.com/nextcloud/vm/master/static/update.sh -P "$SCRIPTS"
chmod +x "$SCRIPTS"/update.sh
fi
exit
This can be done using the following commands, which I adjusted for ubuntu 16.04 and is originally from the following site: https://www.techandme.se/we-migrated-to-postgresql/
#!/bin/bash
## Convert to PostgreSQL ##
# Tested on Ubuntu Server 16.04
# Make sure you have a full backup of your nextcloud installation
# Make sure only root can run our script
if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root, please type sudo -i and run it again." 1>&2
exit 1
fi
service apache2 stop
. <(curl -sL https://raw.githubusercontent.com/nextcloud/vm/master/lib.sh)
NCUSER=pgsql_user_nextcloud
# Install PostgreSQL
apt update
check_command apt install postgresql-9.5
# Create DB
cd /tmp || exit
sudo -u postgres psql <<END
CREATE USER $NCUSER WITH PASSWORD '$PGDB_PASS';
CREATE DATABASE nextcloud_db WITH OWNER $NCUSER TEMPLATE template0 ENCODING 'UTF8';
END
check-command service postgresql restart
# Convert DB
sudo -u www-data php /var/www/nextcloud/occ db:convert-type --all-apps --password "$PGDB_PASS" pgsql $NCUSER 127.0.0.1 nextcloud_db
sudo -u www-data php /var/www/nextcloud/occ maintenance:repair
# Remove MySQL / MariaDB
read -p "Are you sure you want to remove MySQL?" -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]
then
apt clean
apt update
dpkg -r mariadb-client-10.2
dpkg -r mariadb-server-10.2
dpkg -r libmysqlclient20:i386
dpkg -r libmysqlclient20:amd64
dpkg -r libmysqlclient18:amd64
dpkg -r mysql
apt purge mysql\* libmysql\* libmariadb\*
apt autoremove -y
rm -R /var/lib/mysql /var/lib/mysql-files /var/lib/mysql-keyring /var/mysql-upgrade /etc/mysql /var/lib/mysql
fi
# Remove mysql.utf8mb4
if grep -q "mysql.utf8mb4" /var/www/nextcloud/config/config.php
then
sed -i "s|'mysql.utf8mb4' => true,||g" /var/www/nextcloud/config/config.php
sed '/^\s*$/d' /var/www/nextcloud/config/config.php
fi
# Show password
echo "Your new PostgreSQL password is: $PGDB_PASS. It's also written in your Nextcloud config.php file."
# Start Apache2
echo "Apache will start in 30 seconds... Press CTRL+C to abort."
sleep 30
service apache2 start
# Fetch the correct update script
if [ -f "$SCRIPTS"/update.sh ]
then
rm "$SCRIPTS"/update.sh
wget https://raw.githubusercontent.com/nextcloud/vm/master/static/update.sh -P "$SCRIPTS"
chmod +x "$SCRIPTS"/update.sh
fi
exit
Testing FortiGate FortiOS nested adress object groups
Sometimes it is useful to know, if a device really supports nested groups. This little test shows, that a Fortinet FortiGate 60D running FortiOS 5.6 actually supports an address object, which is nested into five different groups:
address object "h-192.168.2.2" is in group "srcgrp05"
address object group "srcgrp05" is in group "srcgrp04"
address object group "srcgrp04" is in group "srcgrp03"
address object group "srcgrp03" is in group "srcgrp02"
address object group "srcgrp02" is in group "srcgrp01"
address object group "srcgrp01" is used in firewall policy with id 10:
edit "h-192.168.2.2"
set subnet 192.168.2.2 255.255.255.255
next
edit "h-172.16.0.182"
set subnet 172.16.0.182 255.255.255.255
next
end
config firewall addrgrp
edit "srcgrp05"
set member "h-192.168.2.2"
next
edit "srcgrp04"
set member "srcgrp05"
next
edit "srcgrp03"
set member "srcgrp04"
next
edit "srcgrp02"
set member "srcgrp03"
next
edit "srcgrp01"
set member "srcgrp02"
next
edit "dstgrp05"
set member "h-172.16.0.182"
next
edit "dstgrp04"
set member "dstgrp05"
next
edit "dstgrp03"
set member "dstgrp04"
next
edit "dstgrp02"
set member "dstgrp03"
next
edit "dstgrp01"
set member "dstgrp02"
next
end
config firewall policy
edit 10
set srcintf "internal3"
set dstintf "wan1"
set srcaddr "srcgrp01"
set dstaddr "dstgrp05"
set action accept
set schedule "always"
set service "SSH"
set logtraffic all
set fsso disable
set nat enable
next
end
id=20085 trace_id=4 func=init_ip_session_common line=5614 msg="allocate a new session-0005ea9f"
id=20085 trace_id=4 func=vf_ip4_route_input line=1604 msg="find a route: flags=00000000 gw-172.16.0.182 via wan1"
id=20085 trace_id=4 func=fw_forward_handler line=746 msg="Allowed by Policy-10: SNAT"
id=20085 trace_id=4 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
id=20085 trace_id=5 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 172.16.0.182:22->172.16.255.254:58871) from wan1. flag [S.], seq 3921820808, ack 1083753678, win 29200"
id=20085 trace_id=5 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, reply direction"
id=20085 trace_id=5 func=__ip_session_run_tuple line=3298 msg="DNAT 172.16.255.254:58871->192.168.2.2:58871"
id=20085 trace_id=5 func=vf_ip4_route_input line=1604 msg="find a route: flags=00000000 gw-192.168.2.2 via internal3"
id=20085 trace_id=5 func=npu_handle_session44 line=919 msg="Trying to offloading session from wan1 to internal3, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x00000000"
id=20085 trace_id=5 func=ip_session_install_npu_session line=270 msg="npu session intallation succeeded"
id=20085 trace_id=6 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 192.168.2.2:58871->172.16.0.182:22) from internal3. flag [.], seq 1083753678, ack 3921820809, win 1026"
id=20085 trace_id=6 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, original direction"
id=20085 trace_id=6 func=npu_handle_session44 line=919 msg="Trying to offloading session from internal3 to wan1, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x00002000"
id=20085 trace_id=6 func=ip_session_install_npu_session line=270 msg="npu session intallation succeeded"
id=20085 trace_id=6 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
id=20085 trace_id=7 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 172.16.0.182:22->172.16.255.254:58871) from wan1. flag [F.], seq 3921820867, ack 1083753680, win 229"
id=20085 trace_id=7 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, reply direction"
id=20085 trace_id=7 func=__ip_session_run_tuple line=3298 msg="DNAT 172.16.255.254:58871->192.168.2.2:58871"
id=20085 trace_id=7 func=npu_handle_session44 line=919 msg="Trying to offloading session from wan1 to internal3, skb.npu_flag=00000000 ses.state=00010204 ses.npu_state=0x00003000"
id=20085 trace_id=8 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 192.168.2.2:58871->172.16.0.182:22) from internal3. flag [F.], seq 1083753680, ack 3921820868, win 1026"
id=20085 trace_id=8 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, original direction"
id=20085 trace_id=8 func=npu_handle_session44 line=919 msg="Trying to offloading session from internal3 to wan1, skb.npu_flag=00000000 ses.state=00010204 ses.npu_state=0x00003000"
id=20085 trace_id=8 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
address object "h-192.168.2.2" is in group "srcgrp05"
address object group "srcgrp05" is in group "srcgrp04"
address object group "srcgrp04" is in group "srcgrp03"
address object group "srcgrp03" is in group "srcgrp02"
address object group "srcgrp02" is in group "srcgrp01"
address object group "srcgrp01" is used in firewall policy with id 10:
srcgrp01/ ├──srcgrp02/ │ └── srcgrp03/ │ └── srcgrp04/ │ └── srcgrp05/ │ └── h-192.168.2.2/
dstgrp01/ ├──dstgrp02/ │ └── dstgrp03/ │ └── dstgrp04/ │ └── dstgrp05/ │ └── h-172.16.0.182/
Firewall policy 10 uses srcgrp1 and dstgrp05 (by mistake, should have been dstgrp01)
Config:
config firewall addressedit "h-192.168.2.2"
set subnet 192.168.2.2 255.255.255.255
next
edit "h-172.16.0.182"
set subnet 172.16.0.182 255.255.255.255
next
end
config firewall addrgrp
edit "srcgrp05"
set member "h-192.168.2.2"
next
edit "srcgrp04"
set member "srcgrp05"
next
edit "srcgrp03"
set member "srcgrp04"
next
edit "srcgrp02"
set member "srcgrp03"
next
edit "srcgrp01"
set member "srcgrp02"
next
edit "dstgrp05"
set member "h-172.16.0.182"
next
edit "dstgrp04"
set member "dstgrp05"
next
edit "dstgrp03"
set member "dstgrp04"
next
edit "dstgrp02"
set member "dstgrp03"
next
edit "dstgrp01"
set member "dstgrp02"
next
end
config firewall policy
edit 10
set srcintf "internal3"
set dstintf "wan1"
set srcaddr "srcgrp01"
set dstaddr "dstgrp05"
set action accept
set schedule "always"
set service "SSH"
set logtraffic all
set fsso disable
set nat enable
next
end
Test using diag debug flow:
FGT60D123456789 # id=20085 trace_id=4 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 192.168.2.2:58871->172.16.0.182:22) from internal3. flag [S], seq 1083753677, ack 0, win 64240"id=20085 trace_id=4 func=init_ip_session_common line=5614 msg="allocate a new session-0005ea9f"
id=20085 trace_id=4 func=vf_ip4_route_input line=1604 msg="find a route: flags=00000000 gw-172.16.0.182 via wan1"
id=20085 trace_id=4 func=fw_forward_handler line=746 msg="Allowed by Policy-10: SNAT"
id=20085 trace_id=4 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
id=20085 trace_id=5 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 172.16.0.182:22->172.16.255.254:58871) from wan1. flag [S.], seq 3921820808, ack 1083753678, win 29200"
id=20085 trace_id=5 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, reply direction"
id=20085 trace_id=5 func=__ip_session_run_tuple line=3298 msg="DNAT 172.16.255.254:58871->192.168.2.2:58871"
id=20085 trace_id=5 func=vf_ip4_route_input line=1604 msg="find a route: flags=00000000 gw-192.168.2.2 via internal3"
id=20085 trace_id=5 func=npu_handle_session44 line=919 msg="Trying to offloading session from wan1 to internal3, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x00000000"
id=20085 trace_id=5 func=ip_session_install_npu_session line=270 msg="npu session intallation succeeded"
id=20085 trace_id=6 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 192.168.2.2:58871->172.16.0.182:22) from internal3. flag [.], seq 1083753678, ack 3921820809, win 1026"
id=20085 trace_id=6 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, original direction"
id=20085 trace_id=6 func=npu_handle_session44 line=919 msg="Trying to offloading session from internal3 to wan1, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x00002000"
id=20085 trace_id=6 func=ip_session_install_npu_session line=270 msg="npu session intallation succeeded"
id=20085 trace_id=6 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
id=20085 trace_id=7 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 172.16.0.182:22->172.16.255.254:58871) from wan1. flag [F.], seq 3921820867, ack 1083753680, win 229"
id=20085 trace_id=7 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, reply direction"
id=20085 trace_id=7 func=__ip_session_run_tuple line=3298 msg="DNAT 172.16.255.254:58871->192.168.2.2:58871"
id=20085 trace_id=7 func=npu_handle_session44 line=919 msg="Trying to offloading session from wan1 to internal3, skb.npu_flag=00000000 ses.state=00010204 ses.npu_state=0x00003000"
id=20085 trace_id=8 func=print_pkt_detail line=5455 msg="vd-root received a packet(proto=6, 192.168.2.2:58871->172.16.0.182:22) from internal3. flag [F.], seq 1083753680, ack 3921820868, win 1026"
id=20085 trace_id=8 func=resolve_ip_tuple_fast line=5530 msg="Find an existing session, id-0005ea9f, original direction"
id=20085 trace_id=8 func=npu_handle_session44 line=919 msg="Trying to offloading session from internal3 to wan1, skb.npu_flag=00000000 ses.state=00010204 ses.npu_state=0x00003000"
id=20085 trace_id=8 func=__ip_session_run_tuple line=3284 msg="SNAT 192.168.2.2->172.16.255.254:58871"
Result
The working with an address object, which is nested in 5 address object groups works.Ideas for your SIEM or Logging-System - Monitoring of critical commands, files and folders
Your are searching for some use cases for your SIEM or your logging/audit/log-alerting system?
How about monitoring some possibly critical commands, files and folders? Here are some examples, which could be helpful. You can use those ideas and adjust them to your environment:
- vi /etc/passwd
or anything that interacts with /etc/passwd - john
When john the ripper is used - msfconsole
When metalspoit console is called - rm –rf /
If someone/something wants to delete "too much" - passwd root
Changing the password of the root user might be interesting to monitor - /etc/init.d/apache2 stop
Stopping, Restarting (with changed cfg/libs/files?) of important services might be something to investigate - vi /etc/postfix/ssl.conf
Editing or looking at your SSL/TLS configuration of your mailserver could be fishy - /var/opt/researchdevelopmentdata or /var/opt/paymenttransactiondata
Folders with critical business, payment, personal, research-and-development or other data could be interesting to watch - /etc/payment-system/configuration.conf
Folders with configuration files of applications with with critical business, payment, personal, research-and-development data should be closly monitored - Windows: useradd Administrator
Also Windows systems of course have lots of critical commands or folders and files, which should be monitored, e.g. when an administrator user is added - Windows Domain Controller: vssadmin create shadow /for=C: (or on which partition NTDS.dit is stored)
Intensivly Monitor NTDS.dit (Active Directory database) and possible attemps to copy/modify it (e.g. using vssadmin)
Which SSL/TLS settings, Algorithms and Methods to use
Recommended Server Encryption Methods
- Hash-Algorithms: SHA3, SHA2-512, SHA2-384, SHA2-256, Poly1305
- Symmetric Encryption Algorithms: AES-GCM-256, AES-GCM-128, ChaCha20
- Key-Exchange-Methods: ECDHE, DHE-4096Bit, Diffie-Hellman group 24 (Modular exponentiation group with a 2048-bit modulus and 256-bit prime order subgroup), 521 bit elliptic curve (DH-Group 21), 384 bit elliptic curve (DH-Group 19), 256 bit elliptic curve (DH-Group 19), DHE-2048Bit (DH-Group 14)
- Certificate-Keys: EC 256Bit, RSA-4096Bit, RSA-2048Bit,
- Certificate Signing: ECDSA, RSA
- Encryption-Protocols: TLS 1.3, TLS 1.2
- Webserver Downgrade attack prevention: TLS_FALLBACK_SCSV, Strict Transport Security (HSTS)
- Password Hashing Algorithms: Argon2, scrypt, bcrypt, PBKDF2
List of Recommended TLS-Ciphers:
Very secure:
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
Secure:
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f)
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)
TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)
High Compatibility but still secure:
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f)
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)
TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)
TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)
FortiConverter 5.5 Python - Installation issue "Internet Explorer" Default Browser
Fortinets configuration migration tool FortiConverter (documentation) has been rewritten using python and has been launched in the past few months. In some special circumstances the installation could show the following error: "FortiConverter can not support Internet Explorer, please use Chrome or Firefox and set them as default browser.":
Problem:
You've changed the default browser of your Windows operating system to Chrome or Firefox - but the error still occurs during the installation.Cause:
Your current Windows user is using Chrome or Firefox as default browser, but the administrator user, which is used to install FortiConverter, has still set Internet Explorer as default Browser.Solution:
Login to Windows with your administrator user, change the Windows Default Browser to Firefox or Chrome. Login again with your normal user, install FortiConverter without any issue.FortiGate HA-Cluster Troubleshooting using Checksums
FortiGate HA-Cluster Troubleshooting using Checksums
Comparing checksums of cluster units
You can use thediagnose sys ha checksum show command to compare the configuration checksums of all cluster units. The output of this command shows checksums labeled global and all as well as checksums for each of the VDOMs including the root VDOM. The get system ha-nonsync-csum command can be used to display similar information; however, this command is intended to be used by FortiManager.The primary unit and subordinate unit checksums should be the same. If they are not you can use the
execute ha synchronize start command to force a synchronization.The following command output is for the primary unit of a cluster that does not have multiple VDOMs enabled:
diagnose sys ha checksum show
is_manage_master()=1, is_root_master()=1
debugzone
global: a0 7f a7 ff ac 00 d5 b6 82 37 cc 13 3e 0b 9b 77
root: 43 72 47 68 7b da 81 17 c8 f5 10 dd fd 6b e9 57
all: c5 90 ed 22 24 3e 96 06 44 35 b6 63 7c 84 88 d5
checksum
global: a0 7f a7 ff ac 00 d5 b6 82 37 cc 13 3e 0b 9b 77
root: 43 72 47 68 7b da 81 17 c8 f5 10 dd fd 6b e9 57
all: c5 90 ed 22 24 3e 96 06 44 35 b6 63 7c 84 88 d5
diagnose sys ha checksum show
is_manage_master()=0, is_root_master()=0
debugzone
global: a0 7f a7 ff ac 00 d5 b6 82 37 cc 13 3e 0b 9b 77
root: 43 72 47 68 7b da 81 17 c8 f5 10 dd fd 6b e9 57
all: c5 90 ed 22 24 3e 96 06 44 35 b6 63 7c 84 88 d5
checksum
global: a0 7f a7 ff ac 00 d5 b6 82 37 cc 13 3e 0b 9b 77
root: 43 72 47 68 7b da 81 17 c8 f5 10 dd fd 6b e9 57
all: c5 90 ed 22 24 3e 96 06 44 35 b6 63 7c 84 88 d5
test and Eng_vdm.From the primary unit:
config global
diagnose sys ha checksum show
is_manage_master()=1, is_root_master()=1
debugzone
global: 65 75 88 97 2d 58 1b bf 38 d3 3d 52 5b 0e 30 a9
test: a5 16 34 8c 7a 46 d6 a4 1e 1f c8 64 ec 1b 53 fe
root: 3c 12 45 98 69 f2 d8 08 24 cf 02 ea 71 57 a7 01
Eng_vdm: 64 51 7c 58 97 79 b1 b3 b3 ed 5c ec cd 07 74 09
all: 30 68 77 82 a1 5d 13 99 d1 42 a3 2f 9f b9 15 53
checksum
global: 65 75 88 97 2d 58 1b bf 38 d3 3d 52 5b 0e 30 a9
test: a5 16 34 8c 7a 46 d6 a4 1e 1f c8 64 ec 1b 53 fe
root: 3c 12 45 98 69 f2 d8 08 24 cf 02 ea 71 57 a7 01
Eng_vdm: 64 51 7c 58 97 79 b1 b3 b3 ed 5c ec cd 07 74 09
all: 30 68 77 82 a1 5d 13 99 d1 42 a3 2f 9f b9 15 53
config global
diagnose sys ha checksum show
is_manage_master()=0, is_root_master()=0
debugzone
global: 65 75 88 97 2d 58 1b bf 38 d3 3d 52 5b 0e 30 a9
test: a5 16 34 8c 7a 46 d6 a4 1e 1f c8 64 ec 1b 53 fe
root: 3c 12 45 98 69 f2 d8 08 24 cf 02 ea 71 57 a7 01
Eng_vdm: 64 51 7c 58 97 79 b1 b3 b3 ed 5c ec cd 07 74 09
all: 30 68 77 82 a1 5d 13 99 d1 42 a3 2f 9f b9 15 53
checksum
global: 65 75 88 97 2d 58 1b bf 38 d3 3d 52 5b 0e 30 a9
test: a5 16 34 8c 7a 46 d6 a4 1e 1f c8 64 ec 1b 53 fe
root: 3c 12 45 98 69 f2 d8 08 24 cf 02 ea 71 57 a7 01
Eng_vdm: 64 51 7c 58 97 79 b1 b3 b3 ed 5c ec cd 07 74 09
all: 30 68 77 82 a1 5d 13 99 d1 42 a3 2f 9f b9 15 53
How to diagnose HA out of sync messages
This section describes how to use thediagnose sys ha checksum show and diagnose debug commands to diagnose the cause of HA out of sync messages.If HA synchronization is not successful, use the following procedures on each cluster unit to find the cause.
To determine why HA synchronization does not occur
- Connect to each cluster unit CLI by connected to the console port.
- Enter the following commands to enable debugging and display HA out of sync messages.
diagnose debug enable
diagnose debug console timestamp enable
diagnose debug application hatalk -1
diagnose debug application hasync -1
Collect the console output and compare the out of sync messages with the information in the table HA out of sync object messages and the configuration objects that they reference.
- Enter the following commands to turn off debugging.
diagnose debug disable
diagnose debug reset
To determine what part of the configuration is causing the problem
If the previous procedure displays messages that include sync object 0x30 (for example,HA_SYNC_SETTING_CONFIGURATION = 0x03)
there is a synchronization problem with the configuration. Use the
following steps to determine the part of the configuration that is
causing the problem.If your cluster consists of two cluster units, use this procedure to capture the configuration checksums for each unit. If your cluster consists of more that two cluster units, repeat this procedure for all cluster units that returned messages that include 0x30 sync object messages.
- Connect to each cluster unit CLI by connected to the console port.
- Enter the following command to turn on terminal capture
diagnose debug enable
- Enter the following command to stop HA synchronization.
execute ha sync stop
- Enter the following command to display configuration checksums.
diagnose sys ha checksum show global
- Copy the output to a text file.
- Repeat for all affected units.
- Compare the text file from the primary unit with the text file from each cluster unit to find the checksums that do not match.
You can use a diff function to compare text files.
- Repeat for the root VDOM:
diagnose sys ha checksum show root
- Repeat for all VDOMS (if multiple VDOM configuration is enabled):
- You can also use the
grepoption to just display checksums for parts of the configuration.
diagnose sys ha checksum show <vdom-name>
For example to display system related configuration checksums in the root VDOM or log-related checksums in the global configuration:
diagnose sys ha checksum root | grep system
diagnose sys ha chechsum global | grep log
Generally it is the first non-matching checksum that is the cause of the synchronization problem.
- Attempt to remove/change the part of the configuration that is causing the problem. You can do this by making configuration changes from the primary unit or subordinate unit CLI.
- Enter the following commands to start HA configuration and stop debugging:
execute ha sync start
diagnose debug disable
diagnose debug reset
Console messages when configuration synchronization fails
If you connect to the console of a subordinate unit that is out of synchronization with the primary unit, messages similar to the following are displayed.
slave is not in sync with master, sequence:0. (type 0x3)
slave is not in sync with master, sequence:1. (type 0x3)
slave is not in sync with master, sequence:2. (type 0x3)
slave is not in sync with master, sequence:3. (type 0x3)
slave is not in sync with master, sequence:4. (type 0x3)
global compared not matched
type 0x3). The type value can help Fortinet Support diagnose the synchronization problem.HA out of sync object messages and the configuration objects that they reference
| Out of Sync Message | Configuration Object |
|---|---|
HA_SYNC_SETTING_CONFIGURATION = 0x03
|
/data/config
|
HA_SYNC_SETTING_AV = 0x10
|
|
HA_SYNC_SETTING_VIR_DB = 0x11
|
/etc/vir
|
HA_SYNC_SETTING_SHARED_LIB = 0x12
|
/data/lib/libav.so
|
HA_SYNC_SETTING_SCAN_UNIT = 0x13
|
/bin/scanunitd
|
HA_SYNC_SETTING_IMAP_PRXY = 0x14
|
/bin/imapd
|
HA_SYNC_SETTING_SMTP_PRXY = 0x15
|
/bin/smtp
|
HA_SYNC_SETTING_POP3_PRXY = 0x16
|
/bin/pop3
|
HA_SYNC_SETTING_HTTP_PRXY = 0x17
|
/bin/thttp
|
HA_SYNC_SETTING_FTP_PRXY = 0x18
|
/bin/ftpd
|
HA_SYNC_SETTING_FCNI = 0x19
|
/etc/fcni.dat
|
HA_SYNC_SETTING_FDNI = 0x1a
|
/etc/fdnservers.dat
|
HA_SYNC_SETTING_FSCI = 0x1b
|
/etc/sci.dat
|
HA_SYNC_SETTING_FSAE = 0x1c
|
/etc/fsae_adgrp.cache
|
HA_SYNC_SETTING_IDS = 0x20
|
/etc/ids.rules
|
HA_SYNC_SETTING_IDSUSER_RULES = 0x21
|
/etc/idsuser.rules
|
HA_SYNC_SETTING_IDSCUSTOM = 0x22
|
|
HA_SYNC_SETTING_IDS_MONITOR = 0x23
|
/bin/ipsmonitor
|
HA_SYNC_SETTING_IDS_SENSOR = 0x24
|
/bin/ipsengine
|
HA_SYNC_SETTING_NIDS_LIB = 0x25
|
/data/lib/libips.so
|
HA_SYNC_SETTING_WEBLISTS = 0x30
|
|
HA_SYNC_SETTING_CONTENTFILTER = 0x31
|
/data/cmdb/webfilter.bword
|
HA_SYNC_SETTING_URLFILTER = 0x32
|
/data/cmdb/webfilter.urlfilter
|
HA_SYNC_SETTING_FTGD_OVRD = 0x33
|
/data/cmdb/webfilter.fgtd-ovrd
|
HA_SYNC_SETTING_FTGD_LRATING = 0x34
|
/data/cmdb/webfilter.fgtd-ovrd
|
HA_SYNC_SETTING_EMAILLISTS = 0x40
|
|
HA_SYNC_SETTING_EMAILCONTENT = 0x41
|
/data/cmdb/spamfilter.bword
|
HA_SYNC_SETTING_EMAILBWLIST = 0x42
|
/data/cmdb/spamfilter.emailbwl
|
HA_SYNC_SETTING_IPBWL = 0x43
|
/data/cmdb/spamfilter.ipbwl
|
HA_SYNC_SETTING_MHEADER = 0x44
|
/data/cmdb/spamfilter.mheader
|
HA_SYNC_SETTING_RBL = 0x45
|
/data/cmdb/spamfilter.rbl
|
HA_SYNC_SETTING_CERT_CONF = 0x50
|
/etc/cert/cert.conf
|
HA_SYNC_SETTING_CERT_CA = 0x51
|
/etc/cert/ca
|
HA_SYNC_SETTING_CERT_LOCAL = 0x52
|
/etc/cert/local
|
HA_SYNC_SETTING_CERT_CRL = 0x53
|
/etc/cert/crl
|
HA_SYNC_SETTING_DB_VER = 0x55
|
|
HA_GET_DETAIL_CSUM = 0x71
|
|
HA_SYNC_CC_SIG = 0x75
|
/etc/cc_sig.dat
|
HA_SYNC_CC_OP = 0x76
|
/etc/cc_op
|
HA_SYNC_CC_MAIN = 0x77
|
/etc/cc_main
|
HA_SYNC_FTGD_CAT_LIST = 0x7a
|
/migadmin/webfilter/ublock/ftgd/ data/ |
Synchronizing the configuration
The FGCP uses a combination of incremental and periodic synchronization to make sure that the configuration of all cluster units is synchronized to that of the primary unit.The following settings are not synchronized between cluster units:
- HA override.
- HA device priority.
- The virtual cluster priority.
- The FortiGate host name.
- The HA priority setting for a ping server (or dead gateway detection) configuration.
- The system interface settings of the HA reserved management interface.
- The HA default route for the reserved management interface, set using the
ha-mgmt-interface-gatewayoption of theconfig system hacommand.
All synchronization activity takes place over the HA heartbeat link using TCP/703 and UDP/703 packets.
Recalculating the checksums to resolve out of sync messages
Sometimes an error can occur when checksums are being calculated by the cluster. As a result of this calculation error the CLI console could display out of sync error messages even though the cluster is otherwise operating normally. You can also sometimes see checksum calculation errors indiagnose sys ha checksum command output when the checksums listed in the debugzone output don’t match the checksums in the checksum part of the output.One solution to this problem could be to re-calculate the checksums. The re-calculated checksums should match and the out of sync error messages should stop appearing.
You can use the following command to re-calculate HA checksums:
diagnose sys ha checksum recalculate [<vdom-name> | global]
Just entering the command
without options recalculates all checksums. You can specify a VDOM name
to just recalculate the checksums for that VDOM. You can also enter global to recalculate the global checksum.Disabling automatic configuration synchronization
In some cases you may want to use the following command to disable automatic synchronization of the primary unit configuration to all cluster units.
config system ha
set sync-config disable
end
When this option is disabled the cluster no longer synchronizes configuration changes. If a device failure occurs, the new primary unit may not have the same configuration as the failed primary unit. As a result, the new primary unit may process sessions differently or may not function on the network in the same way.
In most cases you should not disable automatic configuration synchronization. However, if you have disabled this feature you can use the
execute ha synchronize command to manually synchronize a subordinate unit’s configuration to that of the primary unit.You must enter
execute ha synchronize commands from the subordinate unit that you want to synchronize with the primary unit. Use the execute ha manage command to access a subordinate unit CLI.For example, to access the first subordinate unit and force a synchronization at any time, even if automatic synchronization is disabled enter:
execute ha manage 0
execute ha synchronize start
You can use the following command to stop a synchronization that is in progress.
execute ha synchronize stop
Incremental synchronization
When you log into the cluster GUI or CLI to make configuration changes, you are actually logging into the primary unit. All of your configuration changes are first made to the primary unit. Incremental synchronization then immediately synchronizes these changes to all of the subordinate units.When you log into a subordinate unit CLI (for example using
execute ha manage)
all of the configuration changes that you make to the subordinate unit
are also immediately synchronized to all cluster units, including the
primary unit, using the same process.Incremental synchronization also synchronizes other dynamic configuration information such as the DHCP server address lease database, routing table updates, IPsec SAs, MAC address tables, and so on. See FortiGate HA compatibility with DHCP and PPPoE for more information about DHCP server address lease synchronization and Synchronizing kernel routing tables for information about routing table updates.
Whenever a change is made to a cluster unit configuration, incremental synchronization sends the same configuration change to all other cluster units over the HA heartbeat link. An HA synchronization process running on the each cluster unit receives the configuration change and applies it to the cluster unit. The HA synchronization process makes the configuration change by entering a CLI command that appears to be entered by the administrator who made the configuration change in the first place.
Synchronization takes place silently, and no log messages are recorded about the synchronization activity. However, log messages can be recorded by the cluster units when the synchronization process enters CLI commands. You can see these log messages on the subordinate units if you enable event logging and set the minimum severity level to Information and then check the event log messages written by the cluster units when you make a configuration change.
You can also see these log messages on the primary unit if you make configuration changes from a subordinate unit.
Periodic synchronization
Incremental synchronization makes sure that as an administrator makes configuration changes, the configurations of all cluster units remain the same. However, a number of factors could cause one or more cluster units to go out of sync with the primary unit. For example, if you add a new unit to a functioning cluster, the configuration of this new unit will not match the configuration of the other cluster units. Its not practical to use incremental synchronization to change the configuration of the new unit.Periodic synchronization is a mechanism that looks for synchronization problems and fixes them. Every minute the cluster compares the configuration file checksum of the primary unit with the configuration file checksums of each of the subordinate units. If all subordinate unit checksums are the same as the primary unit checksum, all cluster units are considered synchronized.
If one or more of the subordinate unit checksums is not the same as the primary unit checksum, the subordinate unit configuration is considered out of sync with the primary unit. The checksum of the out of sync subordinate unit is checked again every 15 seconds. This re-checking occurs in case the configurations are out of sync because an incremental configuration sequence has not completed. If the checksums do not match after 5 checks the subordinate unit that is out of sync retrieves the configuration from the primary unit. The subordinate unit then reloads its configuration and resumes operating as a subordinate unit with the same configuration as the primary unit.
The configuration of the subordinate unit is reset in this way because when a subordinate unit configuration gets out of sync with the primary unit configuration there is no efficient way to determine what the configuration differences are and to correct them. Resetting the subordinate unit configuration becomes the most efficient way to resynchronize the subordinate unit.
Synchronization requires that all cluster units run the same FortiOS firmware build. If some cluster units are running different firmware builds, then unstable cluster operation may occur and the cluster units may not be able to synchronize correctly.
|
Re-installing the firmware build running on the primary unit forces the primary unit to upgrade all cluster units to the same firmware build. |
Console messages when configuration synchronization succeeds
When a cluster first forms, or when a new unit is added to a cluster as a subordinate unit, the following messages appear on the CLI console to indicate that the unit joined the cluster and had its configuring synchronized with the primary unit.
slave's configuration is not in sync with master's, sequence:0
slave's configuration is not in sync with master's, sequence:1
slave's configuration is not in sync with master's, sequence:2
slave's configuration is not in sync with master's, sequence:3
slave's configuration is not in sync with master's, sequence:4
slave starts to sync with master
logout all admin users
slave succeeded to sync with master
Subscribe to:
Posts (Atom)
Proxmox Intel NUC crashes - Detected Hardware Unit Hang
Problem If you are running proxmox (e.g. proxmox v 8.4.14) on a Intel NUC and it sporadically crashes with the following entries in the log...
-
You can either use the GUI of the FortiGate to list all certificates, or use the CLI. Either using the commands: Using the "get...
-
The FortiGate uses OFTP (Fortinets proprietary Optimized Fabric Transfer Protocol, not Odette File Transfer Protocol ) to transfer data betw... -
FortiGate HA-Cluster Troubleshooting using Checksums Comparing checksums of cluster units You can use the diagnose sys ha checksum show ...
