Recommendations for OS tuning

Recommended kernel parameters

Changes should be made to /etc/sysctl.conf:

kernel.shmmax = 2473822720
kernel.shmall = 4097152000
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_default = 8388608
net.core.wmem_max = 16777216
net.ipv4.tcp_syncookies = 1
net.ipv4tcp_tw_recycle = 0
net.ipv4tcp_tw_reuse = 0
net.ipv4.tcp_keepalive_time = 10
net.ipv4.tcp_fin_timeout = 5

Then execute a command to apply the changes:

sysctl -p

Additional OS settings with a very large number of simultaneous connections

In the file /etc/security/limits.conf you must prescribe:

* soft nofile 16384
* hard nofile 16384

Recommended file system type and mount options for HDD storage

To ensure maximum storage speed, it is recommended to use XFS with the following options:

mkfs.xfs -f -d su=512k -d sw= -l su=256k /dev/

where:

• swing The number of disks in the RAID array.
• device the name of the device, such as sdc1.

Mounting options in /etc/fstab:

/dev/  xfs async,noatime,nodiratime,attr2,nobarrier,logbufs=8,logbsize=256k,osyncisdsync 0

where:

• device the name of the device, such as sdc1.
• mountpoint - mounting point, for example /opt/storage.

For SSD storage, you also need to add options discard, relatime.

Number of hardware queues on the network map

With a large volume of UDP traffic, you need to use Enterprise-level network cards that have hardware packet queues. However, even in this case, by default, these queues may not be activated:

#ethtool -l enp3s0f1:
Channel parameters for enp3s0f1:
Pre-set maximums:
RX: 0
TX: 0
Other: 1
Combined: 8
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 1

In this example, a total of 8 is available, but 1 is used. To change that:

#ethtool -L enp4s0f1 combined 8
#ethtool -l enp4s0f1
Channel parameters for enp3s0f1:
Pre-set maximums:
RX: 0
TX: 0
Other: 1
Combined: 8
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 8

Modification of processors

Для максимальной производительности CPU необходимо перевести все ядра в режим performance: https://ixnfo.com/izmenenie-cpu-scaling-governor-v-linux.html.

Increase in txqueuelen

The default value of 1000 may not be enough for a packet queue, it is worth putting 10,000:

ifconfig eno1 ifconfig eno1 txqueuelen 10000