How to configure a bare metal server for high-performance computing

Configure your bare metal Linux node for high-performance computing by installing the latest kernel, enabling CPU frequency scaling, and tuning memory parameters. These steps apply to Ubuntu 24.04 and AlmaLinux 9, targeting scientific simulations and data processing. Follow the instructions below to optimize your hardware for maximum throughput.

Prerequisites

• A fresh bare metal installation of Ubuntu 24.04 LTS or AlmaLinux 9.
• Root access via SSH or local terminal.
• At least 32 GB of RAM and a multi-core CPU (6 cores or more).
• Network connectivity to download packages and kernel updates.

Step 1: Update the system and install the latest kernel

Ensure your system has the latest security patches and kernel headers before applying performance tweaks. Run the update command to refresh the package index and install the latest kernel version.

apt update && apt upgrade -y
apt install linux-generic-hwe-24.04 linux-headers-generic-hwe-24.04 -y

For AlmaLinux 9, execute the following commands to update and install the latest kernel from the AppStream repository.

dnf update -y
dnf install kernel kernel-modules-extra kernel-devel -y

Reboot the server to load the new kernel.

reboot

After the reboot, verify the kernel version with the following command.

uname -r

You will see output like 6.8.0-35-generic or 5.14.0-488.27.1.el9_4.x86_64.

Step 2: Configure CPU frequency scaling

Set the CPU governor to performance mode to prevent throttling under load. Install the cpufrequtils package and configure the governor globally.

apt install linux-tools-generic linux-tools-$(uname -r) -y
cpupower frequency-set -g performance

For AlmaLinux 9, use the following command to set the governor.

cpupower frequency-set -g performance

Verify the current governor setting with this command.

cpupower frequency-info | grep "Current"

The output should show "Current: performance". If it shows "ondemand" or "powersave", the change did not take effect.

Step 3: Tune CPU scheduler parameters

Adjust the CPU scheduler to prioritize interactive tasks or real-time workloads. Edit the sysctl configuration file to set the CPU scheduler priority.

echo "kernel.sched_latency_ns = 10000000" >> /etc/sysctl.d/99-hpc.conf
echo "kernel.sched_min_granularity_ns = 10000000" >> /etc/sysctl.d/99-hpc.conf
sysctl -p /etc/sysctl.d/99-hpc.conf

For AlmaLinux 9, create the same file using dnf or direct editing.

cat > /etc/sysctl.d/99-hpc.conf  /etc/sysctl.d/99-hpc.conf
sysctl -p /etc/sysctl.d/99-hpc.conf

For AlmaLinux 9, use the following command to apply the change.

echo "numa_balancing = 0" >> /etc/sysctl.d/99-hpc.conf
sysctl -p /etc/sysctl.d/99-hpc.conf

Verify the NUMA setting with this command.

sysctl numa_balancing

The output should show "numa_balancing = 0" if you disabled it.

Step 5: Optimize memory parameters

Adjust memory allocation to reduce page faults and improve cache efficiency. Set the transparent hugepage settings to optimize memory access patterns.

echo "transparent_hugepage=always" >> /etc/sysctl.d/99-hpc.conf
sysctl -p /etc/sysctl.d/99-hpc.conf

For AlmaLinux 9, use the following command to apply the change.

echo "transparent_hugepage=always" >> /etc/sysctl.d/99-hpc.conf
sysctl -p /etc/sysctl.d/99-hpc.conf

Verify the transparent hugepage setting with this command.

sysctl transparent_hugepage

The output should show "transparent_hugepage=always" if you enabled it.

Verify the installation

Run the following commands to confirm that all optimizations are active and effective. Check the CPU governor, scheduler settings, NUMA configuration, and memory parameters.

cpupower frequency-info | grep "Current"
sysctl kernel.sched_latency_ns
sysctl numa_balancing
sysctl transparent_hugepage

Ensure that all commands return the expected values. If any command fails, review the configuration files and apply the changes again.

Troubleshooting

If the CPU governor does not change, reboot the server and verify that the cpufrequtils package is installed. Check the dmesg output for errors related to CPU frequency scaling.

dmesg | grep -i "cpufreq"

If NUMA balancing is not disabled, check the sysctl configuration file for typos. Ensure that the file path is correct and that the syntax is valid.

cat /etc/sysctl.d/99-hpc.conf

If memory parameters are not applied, reboot the server and verify that the sysctl command is working correctly. Check the kernel logs for any errors related to memory allocation.

journalctl -k | grep -i "hugepage"

If the scheduler settings are not effective, check the CPU model and ensure that the scheduler supports the parameters you set. Review the CPU documentation for supported scheduler options.