How to optimize Nginx worker processes for high concurrency

Configure Nginx worker processes and connection limits to handle thousands of simultaneous requests without crashing your server. These steps apply to Ubuntu 24.04 running Nginx 1.24.0. You will adjust the worker process count based on your CPU cores and set the maximum connections per process to match your hardware limits.

Prerequisites

• Ubuntu 24.04 or compatible Linux distribution (Debian, AlmaLinux, Rocky).
• Nginx 1.24.0 or later installed via package manager.
• Root or sudo privileges to edit configuration files.
• At least 2GB of RAM available for the web server.
• Basic understanding of CPU core counts and network interfaces.

Step 1: Determine the optimal worker process count

Calculate the number of worker processes needed by multiplying the number of CPU cores by a safety factor. The default setting often uses a single process, which limits throughput. You will set the worker_processes directive to match the available CPU cores or slightly higher for multi-core systems.

grep -c ^processor /proc/cpuinfo

You will see a number representing your total CPU cores. For example, a server with 8 cores should use 8 workers. If you have an older CPU with hyperthreading, you might cap this at the physical core count instead. Run the following command to check your current setting:

nginx -t

This checks the current configuration syntax. The output will show "syntax is ok" if the file is valid. If you are unsure, start with the core count. For high-concurrency workloads, setting this to the core count is the standard baseline.

Step 2: Edit the Nginx configuration file

Open the main Nginx configuration file using a text editor. You need to locate the http block to modify the worker settings. Use sudo to gain the necessary permissions to edit system files.

sudo nano /etc/nginx/nginx.conf

Find the line starting with "worker_processes". Change the value from "auto" or "1" to the number of CPU cores you identified earlier. If the line does not exist, add it inside the http block. Ensure there are no spaces around the equals sign if you are setting a specific number.

worker_processes 8;

Save the file and exit the editor. If you used nano, press Ctrl+O, Enter to save, then Ctrl+X to exit. Verify the file is saved correctly by checking the file size or content one more time.

Step 3: Configure worker_connections

Adjust the worker_connections limit to handle the total expected traffic. Each worker process can handle a specific number of connections. The default is often 1024, which is insufficient for high concurrency. Calculate the total capacity by multiplying your worker count by the connections per worker. For 8 workers, a setting of 4096 gives you 32,768 total connections.

worker_connections 4096;

Add this line inside the events block in the configuration file. If the events block is missing, create it near the top of the file. Ensure the value is a power of 2, as Nginx requires this for performance optimization. If you set it too high, the kernel might reject the connection, so check your file descriptor limits.

Step 4: Increase file descriptor limits

Nginx cannot open more file descriptors than your system allows. Check the current limit using the ulimit command. If it is below your worker_connections setting, you must increase it. Edit the PAM limits configuration file to set a higher soft and hard limit.

sudo nano /etc/security/limits.conf

Add the following lines to the end of the file. Replace the username with your system user, usually "www-data" or "nginx".

www-data soft nofile 65535
www-data hard nofile 65535

Save the file and exit. You must log out and log back in for these changes to take effect. Alternatively, edit /etc/security/limits.d/20-nproc.conf to set a global limit.

Step 5: Reload Nginx to apply changes

Apply the new configuration without restarting the service to avoid dropping active connections. Use the nginx reload command to signal the master process to update the worker settings. This ensures the new settings take effect immediately.

sudo nginx -t

This command validates the configuration. If it returns "syntax is ok", proceed to reload. If there is a syntax error, fix the configuration file before reloading. Run the reload command to activate the changes.

sudo nginx -s reload

You will see a message indicating the configuration has been reloaded. Check the status of the service to ensure it is running correctly.

sudo systemctl status nginx

Verify the installation

Confirm that the new worker processes are active and handling connections. Check the process list to see the number of worker processes running. Use the ps command to filter for nginx worker processes.

ps aux | grep nginx | grep -v grep

You should see multiple lines corresponding to the worker count you set. Count the lines to verify the number matches your configuration. Check the active connections using the netstat or ss command to ensure the connection limits are being respected.

ss -an | grep ESTAB | wc -l

This command counts the established connections. If the count exceeds your total limit (workers * connections), the server will start rejecting new connections. Monitor the logs to ensure no errors occur during the transition.

Troubleshooting

If the server rejects connections or crashes, check the error logs for specific messages. The error log is located in /var/log/nginx/error.log. Look for messages about "too many open files" or "worker_connections exceeded". If you see these errors, increase the worker_connections value or the file descriptor limits.

sudo tail -f /var/log/nginx/error.log

Watch the log in real-time while sending traffic to the server. If you see "EMFILE" or "ENFILE" errors, the file descriptor limit is too low. Increase the limit in /etc/security/limits.conf or /etc/sysctl.conf. Reload sysctl settings if you edit /etc/sysctl.conf.

sudo sysctl -w fs.file-max=2097152

Adjust the kernel parameter to allow more open files system-wide. If the worker processes are not starting, check for syntax errors in the configuration file. Run nginx -t again to catch any typos or missing semicolons. Ensure the user running nginx has permission to read the configuration files.