Nginx worker_cpu_affinity

By Default, without setting worker_cpu_affinity directive in nginx.conf, linux kernel will spread all nginx’s worker all over CPUs.

I have 4 logical CPUs on my server, which is CPU0 – CPU3

Cpu0  :  2.9%us,  0.9%sy,  0.0%ni, 88.9%id,  7.2%wa,  0.0%hi,  0.2%si,  0.0%st
Cpu1  :  1.8%us,  0.6%sy,  0.0%ni, 95.3%id,  2.2%wa,  0.0%hi,  0.1%si,  0.0%st
Cpu2  :  2.4%us,  0.7%sy,  0.0%ni, 94.3%id,  2.5%wa,  0.0%hi,  0.1%si,  0.0%st
Cpu3  :  1.9%us,  0.7%sy,  0.0%ni, 96.7%id,  0.6%wa,  0.0%hi,  0.0%si,  0.0%st

Using default setting, nginx’s worker always bind to those 4 logical CPUs. which is has “f” bitmask

# taskset -p 12348
pid 25748's current affinity mask: f
# taskset -p 12349
pid 25749's current affinity mask: f
# taskset -p 12351
pid 25751's current affinity mask: f
# taskset -p 12352
pid 25752's current affinity mask: f
# taskset -p 12353
pid 25753's current affinity mask: f

CPU affinity is represented as a bitmask (given in hexadecimal), with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU.
Examples:

CPU #0: 0x00000001
CPU #1: 0x00000002
CPU #2: 0x00000004
CPU #3: 0x00000008
CPU #0 and CPU #1: 0×00000003
CPU #2 and CPU #3: 0x0000000C
All CPUs: 0xFFFFFFFF

Now, i have worker_processes set to 5 in nginx.conf. I want to bind 4 worker proccess to CPU0 – CPU2 ( 1 CPU per worker proccess), 5th worker proccess to  CPU0 and CPU1. Here’s the config:

worker_processes  5;
worker_cpu_affinity 0001 0010 0100 1000 0011;

Explaination:
0001: mean, set first worker to CPU0 (bitmask 0×00000001)
0010: mean, set second worker to CPU1 (bitmask 0×00000002)
0100: mean, set third worker to CPU2 (bitmask 0×00000004)
1000: mean, set fourth worker to CPU3 (bitmask 0×00000008)
0011: mean, set sixth worker to CPU0 and CPU1 (bitmask 0×00000003)

Restart nginx, verify affinity setting using taskset

25747 ?        S<s    0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
25748 ?        S<     0:00  _ nginx: worker process
25749 ?        S<     0:00  _ nginx: worker process
25751 ?        S<     0:00  _ nginx: worker process
25752 ?        S<     0:00  _ nginx: worker process
25753 ?        S<     0:00  _ nginx: worker process

Here’s new affinity for each worker proccess

# taskset -p 25748
pid 25748's current affinity mask: 1
# taskset -p 25749
pid 25749's current affinity mask: 2
# taskset -p 25751
pid 25751's current affinity mask: 4
# taskset -p 25752
pid 25752's current affinity mask: 8
# taskset -p 25753
pid 25753's current affinity mask: 3

That’s it, i hope this help someone else..

You may also want to read these posts:

• Nginx Configuration SyntaxHighlighting
• How To Graph Nginx Statistics
• Nginx, Strip All Newlines Using nginx-nonewlines Module
• Nginx, limit website visitor bandwidth by country
• Nginx SSL/HTTPS
• Nginx – Customizing 404 page
• Build Nginx RPM With ngx_cache_purge module
• Nginx enabling TLS SNI support on centos 5
• Nginx Memcached module And PHP
• Nginx And Unix User Directories