Greeting, FYI, we noticed a -21.3% regression of netperf.Throughput_tps due to commit: commit: b89fbfbb854c9afc3047e8273cc3a694650b802e ("bpf: Implement minimal BPF perf link") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: netperf on test machine: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: ip: ipv4 runtime: 300s nr_threads: 16 cluster: cs-localhost test: TCP_CRR cpufreq_governor: performance ucode: 0x5003006 test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance. test-url: http://www.netperf.org/netperf/ If you fix the issue, kindly add following tag Reported-by: kernel test robot Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run bin/lkp run generated-yaml-file ========================================================================================= cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode: cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/16/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/TCP_CRR/netperf/0x5003006 commit: 652c1b17b8 ("bpf: Refactor perf_event_set_bpf_prog() to use struct bpf_prog input") b89fbfbb85 ("bpf: Implement minimal BPF perf link") 652c1b17b85b9c19 b89fbfbb854c9afc3047e8273cc ---------------- --------------------------- %stddev %change %stddev \ | \ 382126 -21.3% 300769 netperf.Throughput_total_tps 23882 -21.3% 18798 netperf.Throughput_tps 873.33 -5.1% 828.83 netperf.time.percent_of_cpu_this_job_got 2593 -4.4% 2478 netperf.time.system_time 71.41 ± 6% -28.3% 51.23 netperf.time.user_time 2.26e+08 -22.8% 1.744e+08 netperf.time.voluntary_context_switches 1.146e+08 -21.3% 90230742 netperf.workload 0.26 ± 3% -0.0 0.21 mpstat.cpu.all.usr% 74155969 ± 4% +31.2% 97319249 ± 7% turbostat.C1 2329526 -8.6% 2130170 vmstat.system.cs 83788 ± 32% -40.1% 50201 ± 33% numa-meminfo.node3.Inactive 83788 ± 32% -40.1% 50201 ± 33% numa-meminfo.node3.Inactive(anon) 21155 ± 32% -40.0% 12694 ± 32% numa-vmstat.node3.nr_inactive_anon 21155 ± 32% -40.0% 12694 ± 32% numa-vmstat.node3.nr_zone_inactive_anon 408960 ± 4% -12.5% 357892 meminfo.Inactive 408960 ± 4% -12.5% 357892 meminfo.Inactive(anon) 423396 ± 5% -15.8% 356441 ± 4% meminfo.Mapped 163889 -11.1% 145655 slabinfo.filp.active_objs 2566 -11.2% 2278 slabinfo.filp.active_slabs 164274 -11.2% 145861 slabinfo.filp.num_objs 2566 -11.2% 2278 slabinfo.filp.num_slabs 1046207 -6.0% 983836 proc-vmstat.nr_active_anon 1672962 -4.5% 1598068 proc-vmstat.nr_file_pages 102319 ± 4% -12.5% 89519 proc-vmstat.nr_inactive_anon 106662 ± 5% -15.8% 89786 ± 4% proc-vmstat.nr_mapped 1077486 -7.0% 1002582 proc-vmstat.nr_shmem 1046207 -6.0% 983836 proc-vmstat.nr_zone_active_anon 102319 ± 4% -12.5% 89519 proc-vmstat.nr_zone_inactive_anon 4584675 -13.1% 3982852 proc-vmstat.numa_hit 4351098 -14.4% 3722478 proc-vmstat.numa_local 513996 ± 6% -29.2% 363996 ± 3% proc-vmstat.pgactivate 4595758 -13.1% 3993935 proc-vmstat.pgalloc_normal 1896705 -21.7% 1485852 proc-vmstat.pgfree 2847632 ± 2% -9.6% 2572843 ± 9% interrupts.CAL:Function_call_interrupts 857.83 ± 9% +78.0% 1526 ± 15% interrupts.CPU115.NMI:Non-maskable_interrupts 857.83 ± 9% +78.0% 1526 ± 15% interrupts.CPU115.PMI:Performance_monitoring_interrupts 1744 ± 12% +40.8% 2456 ± 11% interrupts.CPU118.RES:Rescheduling_interrupts 1527 ± 15% +32.8% 2028 ± 16% interrupts.CPU119.RES:Rescheduling_interrupts 29210 ± 29% -37.1% 18383 ± 28% interrupts.CPU120.CAL:Function_call_interrupts 3329 ± 12% -24.2% 2523 ± 18% interrupts.CPU170.RES:Rescheduling_interrupts 9987 ± 9% +96.8% 19656 ± 15% interrupts.CPU176.CAL:Function_call_interrupts 2163 ± 10% +29.7% 2804 ± 11% interrupts.CPU176.RES:Rescheduling_interrupts 2448 ± 14% +24.7% 3054 ± 11% interrupts.CPU187.RES:Rescheduling_interrupts 2229 ± 10% +25.0% 2787 ± 11% interrupts.CPU188.RES:Rescheduling_interrupts 5662 ± 11% -34.8% 3693 ± 16% interrupts.CPU24.RES:Rescheduling_interrupts 10902 ± 27% -32.8% 7331 ± 35% interrupts.CPU37.CAL:Function_call_interrupts 5596 ± 6% -23.0% 4306 ± 17% interrupts.CPU48.RES:Rescheduling_interrupts 6154 ± 10% -22.7% 4760 ± 13% interrupts.CPU49.RES:Rescheduling_interrupts 5253 ± 11% -38.4% 3233 ± 21% interrupts.CPU50.RES:Rescheduling_interrupts 4555 ± 12% -29.9% 3193 ± 19% interrupts.CPU51.RES:Rescheduling_interrupts 3191 ± 11% -31.7% 2179 ± 19% interrupts.CPU53.RES:Rescheduling_interrupts 17323 ± 45% -51.6% 8384 ± 31% interrupts.CPU57.CAL:Function_call_interrupts 1950 ± 19% -42.3% 1126 ± 36% interrupts.CPU57.NMI:Non-maskable_interrupts 1950 ± 19% -42.3% 1126 ± 36% interrupts.CPU57.PMI:Performance_monitoring_interrupts 2412 ± 42% -56.3% 1055 ± 55% interrupts.CPU59.NMI:Non-maskable_interrupts 2412 ± 42% -56.3% 1055 ± 55% interrupts.CPU59.PMI:Performance_monitoring_interrupts 9938 ± 14% -40.5% 5909 ± 28% interrupts.CPU7.CAL:Function_call_interrupts 11392 ± 21% +48.0% 16859 ± 14% interrupts.CPU77.CAL:Function_call_interrupts 1587 ± 11% +26.5% 2007 ± 12% interrupts.CPU91.RES:Rescheduling_interrupts 0.00 -29.2% 0.00 ± 13% perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.tcp_close 0.00 -25.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.aa_sk_perm.security_socket_accept.__sys_accept4_file 0.00 -33.3% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run 0.01 ± 13% +51.6% 0.01 ± 25% perf-sched.sch_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.__unix_dgram_recvmsg.__sys_recvfrom 0.00 -25.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file 0.01 ± 13% +51.6% 0.01 ± 25% perf-sched.sch_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.__unix_dgram_recvmsg.__sys_recvfrom 58.25 ± 7% +44.8% 84.33 ± 3% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.01 +42.9% 0.01 perf-sched.wait_and_delay.avg.ms.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked 98.49 ± 5% -10.8% 87.89 ± 2% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 159.17 ± 7% -31.6% 108.83 ± 3% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 3326971 -19.2% 2688283 perf-sched.wait_and_delay.count.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file 1767 -12.2% 1551 ± 2% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 6.75 ± 17% -25.1% 5.06 ± 7% perf-sched.wait_and_delay.max.ms.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked 0.12 ±117% -84.7% 0.02 ±156% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.02 ± 4% +33.1% 0.03 ± 16% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.inet_accept 0.01 ± 14% +337.8% 0.03 ± 34% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.sk_wait_data 0.02 ± 7% -11.6% 0.02 ± 4% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.tcp_sendmsg 0.02 ± 3% +27.6% 0.03 ± 3% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.inet_accept.__sys_accept4_file 0.02 ± 5% +18.2% 0.03 ± 6% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.inet_csk_accept.inet_accept 0.00 ± 14% +75.0% 0.01 ± 55% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.sk_wait_data.tcp_recvmsg_locked 1.35 ± 49% -70.1% 0.40 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.pagecache_get_page.shmem_getpage_gfp.shmem_write_begin 58.25 ± 7% +44.8% 84.33 ± 3% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.01 +60.0% 0.01 perf-sched.wait_time.avg.ms.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked 98.49 ± 5% -10.8% 87.89 ± 2% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 0.30 ±176% -94.0% 0.02 ±156% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.03 ± 8% +1134.8% 0.38 ±200% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__kmalloc_node_track_caller.kmalloc_reserve.__alloc_skb 0.03 ± 12% +699.5% 0.27 ±188% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.inet_accept 0.03 ± 10% +1738.9% 0.57 ±145% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.__release_sock.release_sock.sk_wait_data 0.03 ± 13% +1386.6% 0.46 ±201% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.kmem_cache_alloc.sock_alloc_inode.alloc_inode 0.03 ± 12% +23.2% 0.04 ± 5% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.inet_csk_accept.inet_accept 0.01 ± 81% +1207.4% 0.18 ±176% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.sk_wait_data.tcp_recvmsg_locked 0.03 ± 6% +31.8% 0.04 ± 6% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.lock_sock_nested.tcp_sendmsg.sock_sendmsg 2.70 ± 28% -64.2% 0.97 ± 76% perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.pagecache_get_page.shmem_getpage_gfp.shmem_write_begin 11.77 ± 2% +31.6% 15.48 ± 3% perf-stat.i.MPKI 1.004e+10 -17.9% 8.241e+09 perf-stat.i.branch-instructions 1.559e+08 -19.3% 1.258e+08 perf-stat.i.branch-misses 11.20 ± 3% +7.0 18.19 perf-stat.i.cache-miss-rate% 65819504 ± 3% +72.8% 1.138e+08 perf-stat.i.cache-misses 5.901e+08 +5.5% 6.228e+08 perf-stat.i.cache-references 2356585 -8.5% 2155457 perf-stat.i.context-switches 1.55 +19.1% 1.84 perf-stat.i.cpi 7.708e+10 -3.0% 7.474e+10 perf-stat.i.cpu-cycles 295.82 -3.6% 285.22 perf-stat.i.cpu-migrations 1235 ± 4% -44.4% 687.56 ± 3% perf-stat.i.cycles-between-cache-misses 1.52e+10 -18.5% 1.239e+10 perf-stat.i.dTLB-loads 242730 ± 15% -48.0% 126135 ± 20% perf-stat.i.dTLB-store-misses 8.874e+09 -18.4% 7.24e+09 perf-stat.i.dTLB-stores 89.72 -4.5 85.18 perf-stat.i.iTLB-load-miss-rate% 1.073e+08 -21.8% 83946753 perf-stat.i.iTLB-load-misses 11647157 +22.3% 14248926 ± 2% perf-stat.i.iTLB-loads 5.082e+10 -18.2% 4.158e+10 perf-stat.i.instructions 486.39 +4.4% 507.91 perf-stat.i.instructions-per-iTLB-miss 0.66 -15.6% 0.55 perf-stat.i.ipc 0.40 -3.0% 0.39 perf-stat.i.metric.GHz 214.40 +21.3% 260.08 perf-stat.i.metric.K/sec 180.75 -17.9% 148.39 perf-stat.i.metric.M/sec 13387607 ± 3% +57.6% 21095835 perf-stat.i.node-load-misses 1252745 ± 3% +66.1% 2081130 perf-stat.i.node-loads 10872702 -21.9% 8486959 perf-stat.i.node-store-misses 550301 -28.0% 396340 ± 2% perf-stat.i.node-stores 11.61 +29.0% 14.98 perf-stat.overall.MPKI 1.55 -0.0 1.53 perf-stat.overall.branch-miss-rate% 11.15 ± 3% +7.1 18.27 perf-stat.overall.cache-miss-rate% 1.52 +18.5% 1.80 perf-stat.overall.cpi 1172 ± 3% -44.0% 657.12 perf-stat.overall.cycles-between-cache-misses 0.00 ± 14% -0.0 0.00 ± 20% perf-stat.overall.dTLB-store-miss-rate% 90.21 -4.7 85.49 perf-stat.overall.iTLB-load-miss-rate% 473.55 +4.6% 495.37 perf-stat.overall.instructions-per-iTLB-miss 0.66 -15.6% 0.56 perf-stat.overall.ipc 134988 +3.9% 140279 perf-stat.overall.path-length 1.001e+10 -17.9% 8.214e+09 perf-stat.ps.branch-instructions 1.554e+08 -19.3% 1.254e+08 perf-stat.ps.branch-misses 65603770 ± 3% +72.8% 1.134e+08 perf-stat.ps.cache-misses 5.882e+08 +5.5% 6.207e+08 perf-stat.ps.cache-references 2348732 -8.5% 2148196 perf-stat.ps.context-switches 7.683e+10 -3.0% 7.449e+10 perf-stat.ps.cpu-cycles 294.87 -3.6% 284.33 perf-stat.ps.cpu-migrations 1.515e+10 -18.5% 1.235e+10 perf-stat.ps.dTLB-loads 241927 ± 15% -48.0% 125735 ± 20% perf-stat.ps.dTLB-store-misses 8.846e+09 -18.4% 7.217e+09 perf-stat.ps.dTLB-stores 1.07e+08 -21.8% 83668098 perf-stat.ps.iTLB-load-misses 11608473 +22.3% 14200885 ± 2% perf-stat.ps.iTLB-loads 5.066e+10 -18.2% 4.145e+10 perf-stat.ps.instructions 13343927 ± 3% +57.6% 21026557 perf-stat.ps.node-load-misses 1248713 ± 3% +66.1% 2074254 perf-stat.ps.node-loads 10837421 -21.9% 8459398 perf-stat.ps.node-store-misses 549221 -28.0% 395635 ± 2% perf-stat.ps.node-stores 1.547e+13 -18.2% 1.266e+13 perf-stat.total.instructions 4014884 ± 13% -27.1% 2926304 ± 12% softirqs.CPU1.NET_RX 258765 ± 12% -24.7% 194744 ± 10% softirqs.CPU1.RCU 1176900 ± 14% -22.9% 907366 ± 18% softirqs.CPU10.NET_RX 2200920 ± 9% -27.9% 1587061 ± 16% softirqs.CPU121.NET_RX 160791 ± 8% -19.3% 129762 ± 11% softirqs.CPU121.RCU 2325401 ± 9% -36.6% 1475333 ± 14% softirqs.CPU122.NET_RX 162209 ± 7% -26.0% 120056 ± 10% softirqs.CPU122.RCU 2439752 ± 15% -39.7% 1470207 ± 13% softirqs.CPU123.NET_RX 172191 ± 11% -31.2% 118529 ± 8% softirqs.CPU123.RCU 1973419 ± 10% -15.6% 1666511 ± 10% softirqs.CPU126.NET_RX 145853 ± 7% -11.8% 128598 ± 6% softirqs.CPU126.RCU 155145 ± 8% -18.8% 125917 ± 12% softirqs.CPU127.RCU 2183325 ± 16% -32.6% 1472632 ± 13% softirqs.CPU128.NET_RX 151893 ± 11% -24.8% 114269 ± 11% softirqs.CPU128.RCU 141163 ± 16% -19.9% 113100 ± 11% softirqs.CPU133.RCU 2137865 ± 10% -21.8% 1672211 ± 15% softirqs.CPU137.NET_RX 2167225 ± 15% -29.4% 1531034 ± 12% softirqs.CPU139.NET_RX 151688 ± 11% -22.5% 117531 ± 8% softirqs.CPU139.RCU 2304600 ± 7% -28.0% 1660132 ± 24% softirqs.CPU145.NET_RX 160072 ± 3% -19.5% 128924 ± 18% softirqs.CPU145.RCU 2196758 ± 18% -34.1% 1447425 ± 14% softirqs.CPU149.NET_RX 155180 ± 11% -26.6% 113886 ± 8% softirqs.CPU149.RCU 2069783 ± 14% -32.2% 1402561 ± 14% softirqs.CPU150.NET_RX 149326 ± 13% -26.7% 109469 ± 11% softirqs.CPU150.RCU 1847061 ± 4% -21.0% 1459388 ± 9% softirqs.CPU151.NET_RX 134926 ± 5% -16.1% 113270 ± 7% softirqs.CPU151.RCU 1951796 ± 13% -29.6% 1373752 ± 20% softirqs.CPU156.NET_RX 142225 ± 9% -23.2% 109249 ± 14% softirqs.CPU156.RCU 1921345 ± 12% -22.2% 1495303 ± 10% softirqs.CPU159.NET_RX 2248959 ± 19% -30.2% 1569394 ± 16% softirqs.CPU161.NET_RX 160768 ± 13% -22.5% 124600 ± 12% softirqs.CPU161.RCU 1862307 ± 12% -22.1% 1449974 ± 15% softirqs.CPU165.NET_RX 1357517 ± 18% -31.9% 923915 ± 26% softirqs.CPU167.NET_RX 2393344 ± 10% -40.6% 1421882 ± 22% softirqs.CPU169.NET_RX 165076 ± 8% -29.3% 116649 ± 17% softirqs.CPU169.RCU 2296035 ± 16% -35.8% 1474994 ± 12% softirqs.CPU170.NET_RX 160311 ± 11% -29.1% 113713 ± 8% softirqs.CPU170.RCU 2232836 ± 19% -28.7% 1592693 ± 18% softirqs.CPU171.NET_RX 159350 ± 15% -23.6% 121777 ± 14% softirqs.CPU171.RCU 2270802 ± 15% -27.7% 1642884 ± 14% softirqs.CPU172.NET_RX 158247 ± 12% -22.4% 122791 ± 10% softirqs.CPU172.RCU 1974994 ± 8% -17.4% 1631109 ± 10% softirqs.CPU173.NET_RX 145079 ± 7% -13.5% 125486 ± 4% softirqs.CPU177.RCU 2023137 ± 17% -24.1% 1535725 ± 7% softirqs.CPU179.NET_RX 140636 ± 11% -16.7% 117143 ± 5% softirqs.CPU179.RCU 142098 ± 7% -17.0% 117946 ± 7% softirqs.CPU180.RCU 3597896 ± 14% -34.5% 2358266 ± 19% softirqs.CPU2.NET_RX 234587 ± 13% -29.1% 166321 ± 14% softirqs.CPU2.RCU 297635 ± 11% -27.7% 215326 ± 13% softirqs.CPU24.RCU 3962994 ± 10% -33.7% 2628479 ± 15% softirqs.CPU25.NET_RX 264305 ± 10% -29.8% 185617 ± 12% softirqs.CPU25.RCU 3147476 ± 13% -30.7% 2180384 ± 23% softirqs.CPU26.NET_RX 213457 ± 11% -26.9% 155950 ± 17% softirqs.CPU26.RCU 2805154 ± 15% -35.9% 1798759 ± 23% softirqs.CPU27.NET_RX 2175695 ± 13% -34.2% 1430966 ± 19% softirqs.CPU28.NET_RX 1903308 ± 12% -37.4% 1192171 ± 23% softirqs.CPU29.NET_RX 145385 ± 9% -28.3% 104230 ± 15% softirqs.CPU29.RCU 200826 ± 14% -28.9% 142832 ± 11% softirqs.CPU3.RCU 1520428 ± 13% -31.1% 1048096 ± 18% softirqs.CPU30.NET_RX 1379814 ± 8% -30.9% 952779 ± 15% softirqs.CPU31.NET_RX 110710 ± 6% -19.1% 89561 ± 9% softirqs.CPU31.RCU 1090621 ± 14% -24.6% 821788 ± 20% softirqs.CPU34.NET_RX 91279 ± 10% -15.5% 77091 ± 11% softirqs.CPU34.RCU 1240050 ± 8% -30.7% 859760 ± 18% softirqs.CPU38.NET_RX 100298 ± 5% -19.7% 80576 ± 10% softirqs.CPU38.RCU 1215230 ± 19% -37.2% 763371 ± 24% softirqs.CPU39.NET_RX 98790 ± 12% -23.3% 75800 ± 12% softirqs.CPU39.RCU 1010316 ± 23% -29.0% 716849 ± 12% softirqs.CPU43.NET_RX 88547 ± 14% -18.0% 72601 ± 6% softirqs.CPU43.RCU 1192624 ± 26% -30.5% 829456 ± 24% softirqs.CPU44.NET_RX 5245236 ± 6% -28.4% 3754659 ± 15% softirqs.CPU48.NET_RX 306421 ± 4% -26.8% 224181 ± 10% softirqs.CPU48.RCU 4267374 ± 8% -38.2% 2638707 ± 9% softirqs.CPU49.NET_RX 271765 ± 8% -33.0% 181951 ± 8% softirqs.CPU49.RCU 3409429 ± 13% -41.4% 1997125 ± 22% softirqs.CPU50.NET_RX 225978 ± 12% -37.1% 142118 ± 18% softirqs.CPU50.RCU 3060626 ± 11% -44.7% 1691740 ± 13% softirqs.CPU51.NET_RX 209935 ± 10% -38.0% 130118 ± 10% softirqs.CPU51.RCU 2490473 ± 14% -44.3% 1386146 ± 17% softirqs.CPU52.NET_RX 174568 ± 12% -36.1% 111632 ± 11% softirqs.CPU52.RCU 2038572 ± 14% -39.3% 1237845 ± 15% softirqs.CPU53.NET_RX 149102 ± 10% -30.6% 103549 ± 9% softirqs.CPU53.RCU 1806889 ± 19% -39.7% 1089972 ± 12% softirqs.CPU54.NET_RX 135107 ± 12% -29.8% 94798 ± 8% softirqs.CPU54.RCU 1724482 ± 13% -35.0% 1120764 ± 17% softirqs.CPU55.NET_RX 130437 ± 10% -26.9% 95317 ± 10% softirqs.CPU55.RCU 1571110 ± 10% -24.8% 1181396 ± 15% softirqs.CPU56.NET_RX 119409 ± 8% -19.4% 96251 ± 9% softirqs.CPU56.RCU 1211612 ± 13% -26.1% 894990 ± 10% softirqs.CPU62.NET_RX 98706 ± 7% -15.2% 83741 ± 5% softirqs.CPU62.RCU 1286365 ± 12% -38.1% 796328 ± 10% softirqs.CPU63.NET_RX 102717 ± 9% -25.6% 76430 ± 6% softirqs.CPU63.RCU 1235222 ± 15% -28.2% 886801 ± 17% softirqs.CPU64.NET_RX 105331 ± 7% -19.0% 85343 ± 8% softirqs.CPU64.RCU 3750481 ± 10% -20.5% 2981716 ± 10% softirqs.CPU73.NET_RX 241578 ± 9% -18.3% 197334 ± 8% softirqs.CPU73.RCU 2995431 ± 11% -17.0% 2485879 ± 6% softirqs.CPU74.NET_RX 203087 ± 9% -15.3% 172016 ± 6% softirqs.CPU74.RCU 2630704 ± 5% -22.1% 2049366 ± 10% softirqs.CPU75.NET_RX 182284 ± 5% -18.3% 148875 ± 7% softirqs.CPU75.RCU 1676900 ± 13% -22.3% 1302832 ± 12% softirqs.CPU78.NET_RX 2085902 ± 8% -33.4% 1389393 ± 24% softirqs.CPU97.NET_RX 148876 ± 8% -22.3% 115711 ± 20% softirqs.CPU97.RCU 2137018 ± 5% -31.5% 1464045 ± 12% softirqs.CPU98.NET_RX 150723 ± 4% -23.5% 115253 ± 8% softirqs.CPU98.RCU 3.505e+08 -21.7% 2.744e+08 softirqs.NET_RX 25635284 -16.0% 21527846 softirqs.RCU 239503 +83.7% 440010 softirqs.TIMER 5.94 ± 5% -1.2 4.76 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.91 ± 5% -1.2 4.73 ± 6% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.31 ± 5% -1.1 4.22 ± 6% perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.28 ± 5% -1.1 4.20 ± 6% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64 3.92 ± 6% -0.7 3.18 ± 6% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.97 ± 6% -0.6 2.32 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.95 ± 6% -0.6 2.31 ± 5% perf-profile.calltrace.cycles-pp.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.92 ± 6% -0.6 2.29 ± 5% perf-profile.calltrace.cycles-pp.__sys_accept4_file.__sys_accept4.__x64_sys_accept.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.37 ± 4% -0.6 3.78 ± 6% perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 1.78 ± 4% -0.5 1.30 ± 9% perf-profile.calltrace.cycles-pp.ret_from_fork 1.78 ± 4% -0.5 1.30 ± 9% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 1.73 ± 4% -0.5 1.26 ± 9% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 1.73 ± 4% -0.5 1.26 ± 10% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 1.75 ± 4% -0.5 1.28 ± 9% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork 1.72 ± 4% -0.5 1.24 ± 9% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread 1.71 ± 4% -0.5 1.24 ± 10% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn 1.63 ± 4% -0.5 1.18 ± 10% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd 1.45 ± 4% -0.4 1.04 ± 9% perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start 1.41 ± 4% -0.4 1.01 ± 10% perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.rcu_do_batch.rcu_core 1.52 ± 7% -0.4 1.15 ± 4% perf-profile.calltrace.cycles-pp.inet_accept.__sys_accept4_file.__sys_accept4.__x64_sys_accept.do_syscall_64 0.61 ± 5% -0.4 0.26 ±100% perf-profile.calltrace.cycles-pp.apparmor_file_free_security.security_file_free.__fput.task_work_run.exit_to_user_mode_prepare 1.08 ± 4% -0.3 0.79 ± 11% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.rcu_do_batch 1.11 ± 6% -0.3 0.83 ± 7% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.61 ± 6% -0.3 0.34 ± 70% perf-profile.calltrace.cycles-pp.security_file_free.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode 1.04 ± 6% -0.3 0.78 ± 7% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 1.21 ± 7% -0.3 0.95 ± 3% perf-profile.calltrace.cycles-pp.inet_csk_accept.inet_accept.__sys_accept4_file.__sys_accept4.__x64_sys_accept 0.97 ± 6% -0.3 0.72 ± 7% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv 0.95 ± 6% -0.2 0.70 ± 7% perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established 0.68 ± 6% -0.2 0.44 ± 44% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file 0.66 ± 6% -0.2 0.43 ± 44% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.inet_csk_accept.inet_accept 0.58 ± 4% -0.2 0.36 ± 70% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.69 ± 5% -0.2 0.48 ± 44% perf-profile.calltrace.cycles-pp.__dentry_kill.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode 1.10 ± 5% -0.2 0.89 ± 4% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 0.95 ± 5% -0.2 0.74 ± 8% perf-profile.calltrace.cycles-pp.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv 1.04 ± 4% -0.2 0.84 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu 0.92 ± 6% -0.2 0.72 ± 8% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue.tcp_rcv_established 0.96 ± 5% -0.2 0.77 ± 4% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process 0.97 ± 5% -0.2 0.78 ± 4% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_child_process.tcp_v4_rcv 0.64 ± 6% -0.2 0.45 ± 44% perf-profile.calltrace.cycles-pp.inet_csk_destroy_sock.tcp_fin.tcp_data_queue.tcp_rcv_state_process.tcp_v4_do_rcv 0.94 ± 5% -0.2 0.76 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable 0.75 ± 6% -0.2 0.57 ± 3% perf-profile.calltrace.cycles-pp.schedule_timeout.inet_csk_accept.inet_accept.__sys_accept4_file.__sys_accept4 0.98 ± 6% -0.2 0.80 ± 8% perf-profile.calltrace.cycles-pp.tcp_fin.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.84 ± 5% -0.2 0.67 ± 8% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_wakeup.tcp_fin.tcp_data_queue 0.80 ± 5% -0.2 0.63 ± 8% perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_wakeup.tcp_fin 0.83 ± 5% -0.2 0.67 ± 5% perf-profile.calltrace.cycles-pp.tcp_send_fin.__tcp_close.tcp_close.inet_release.__sock_release 1.21 ± 4% -0.2 1.06 ± 9% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.78 ± 6% -0.1 0.63 ± 7% perf-profile.calltrace.cycles-pp.sock_alloc_file.__sys_accept4_file.__sys_accept4.__x64_sys_accept.do_syscall_64 0.78 ± 5% -0.1 0.63 ± 7% perf-profile.calltrace.cycles-pp.alloc_file_pseudo.sock_alloc_file.__sys_accept4_file.__sys_accept4.__x64_sys_accept 0.62 ± 3% -0.1 0.48 ± 44% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.92 ± 5% -0.1 0.79 ± 5% perf-profile.calltrace.cycles-pp.inet_csk_clone_lock.tcp_create_openreq_child.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv 0.90 ± 5% -0.1 0.77 ± 5% perf-profile.calltrace.cycles-pp.sk_clone_lock.inet_csk_clone_lock.tcp_create_openreq_child.tcp_v4_syn_recv_sock.tcp_check_req 0.58 ± 4% +0.1 0.72 ± 5% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.58 ± 6% +0.2 0.78 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_send_synack.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv 0.73 ± 6% +0.2 0.94 ± 6% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock 1.08 ± 5% +0.2 1.30 ± 6% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto 1.12 ± 5% +0.2 1.35 ± 6% perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 1.03 ± 5% +0.2 1.26 ± 6% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg 0.57 ± 4% +0.2 0.81 ± 6% perf-profile.calltrace.cycles-pp.tcp_rcv_state_process.tcp_child_process.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 0.81 ± 5% +0.3 1.12 ± 5% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 1.72 ± 5% +0.4 2.07 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_syn_recv_sock.tcp_check_req.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 1.95 ± 5% +0.4 2.35 ± 6% perf-profile.calltrace.cycles-pp.tcp_check_req.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 1.60 ± 5% +0.5 2.14 ± 6% perf-profile.calltrace.cycles-pp.tcp_conn_request.tcp_rcv_state_process.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu 0.00 +0.6 0.57 ± 4% perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv 0.00 +0.6 0.62 ± 6% perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_synsent_state_process.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock 3.18 ± 4% +0.8 3.96 ± 6% perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect 3.15 ± 4% +0.8 3.96 ± 6% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect 3.52 ± 4% +0.9 4.44 ± 5% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect 3.75 ± 4% +1.0 4.75 ± 5% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect 5.98 ± 4% +1.1 7.08 ± 5% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect 6.00 ± 4% +1.1 7.11 ± 6% perf-profile.calltrace.cycles-pp.release_sock.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 5.93 ± 4% +1.1 7.04 ± 6% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.__inet_stream_connect.inet_stream_connect 5.78 ± 4% +1.1 6.91 ± 6% perf-profile.calltrace.cycles-pp.tcp_rcv_synsent_state_process.tcp_rcv_state_process.tcp_v4_do_rcv.__release_sock.release_sock 4.54 ± 4% +1.2 5.74 ± 5% perf-profile.calltrace.cycles-pp.tcp_connect.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect 5.54 ± 4% +1.5 7.04 ± 5% perf-profile.calltrace.cycles-pp.tcp_v4_connect.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect 11.94 ± 4% +2.6 14.51 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.92 ± 4% +2.6 14.50 ± 5% perf-profile.calltrace.cycles-pp.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.72 ± 4% +2.6 14.31 ± 5% perf-profile.calltrace.cycles-pp.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.67 ± 4% +2.6 14.27 ± 5% perf-profile.calltrace.cycles-pp.__inet_stream_connect.inet_stream_connect.__sys_connect.__x64_sys_connect.do_syscall_64 5.95 ± 5% -1.2 4.76 ± 6% perf-profile.children.cycles-pp.__x64_sys_recvfrom 5.91 ± 5% -1.2 4.74 ± 6% perf-profile.children.cycles-pp.__sys_recvfrom 5.31 ± 5% -1.1 4.22 ± 6% perf-profile.children.cycles-pp.inet_recvmsg 5.29 ± 5% -1.1 4.21 ± 6% perf-profile.children.cycles-pp.tcp_recvmsg 3.99 ± 5% -0.7 3.25 ± 5% perf-profile.children.cycles-pp.poll_idle 1.80 ± 6% -0.7 1.14 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_bh 2.97 ± 6% -0.6 2.32 ± 5% perf-profile.children.cycles-pp.__x64_sys_accept 3.18 ± 4% -0.6 2.54 ± 8% perf-profile.children.cycles-pp.kmem_cache_free 2.95 ± 6% -0.6 2.31 ± 5% perf-profile.children.cycles-pp.__sys_accept4 2.92 ± 6% -0.6 2.29 ± 5% perf-profile.children.cycles-pp.__sys_accept4_file 4.38 ± 4% -0.6 3.79 ± 6% perf-profile.children.cycles-pp.tcp_recvmsg_locked 2.45 ± 4% -0.5 1.92 ± 9% perf-profile.children.cycles-pp.rcu_do_batch 2.49 ± 4% -0.5 1.96 ± 9% perf-profile.children.cycles-pp.rcu_core 1.47 ± 6% -0.5 0.96 ± 5% perf-profile.children.cycles-pp.lock_sock_nested 1.22 ± 6% -0.5 0.71 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 2.45 ± 4% -0.5 1.95 ± 9% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages 2.40 ± 4% -0.5 1.91 ± 9% perf-profile.children.cycles-pp.page_counter_uncharge 1.78 ± 4% -0.5 1.30 ± 9% perf-profile.children.cycles-pp.ret_from_fork 1.78 ± 4% -0.5 1.30 ± 9% perf-profile.children.cycles-pp.kthread 1.73 ± 4% -0.5 1.26 ± 10% perf-profile.children.cycles-pp.run_ksoftirqd 1.75 ± 4% -0.5 1.28 ± 9% perf-profile.children.cycles-pp.smpboot_thread_fn 1.83 ± 5% -0.4 1.43 ± 6% perf-profile.children.cycles-pp.__dev_queue_xmit 2.21 ± 5% -0.4 1.82 ± 6% perf-profile.children.cycles-pp._raw_spin_lock 1.88 ± 4% -0.4 1.50 ± 9% perf-profile.children.cycles-pp.page_counter_cancel 1.52 ± 7% -0.4 1.15 ± 4% perf-profile.children.cycles-pp.inet_accept 1.05 ± 6% -0.3 0.76 ± 5% perf-profile.children.cycles-pp.percpu_counter_add_batch 1.50 ± 6% -0.3 1.21 ± 7% perf-profile.children.cycles-pp.kmem_cache_alloc 1.61 ± 6% -0.3 1.33 ± 6% perf-profile.children.cycles-pp.sock_alloc_file 1.60 ± 6% -0.3 1.32 ± 6% perf-profile.children.cycles-pp.alloc_file_pseudo 1.22 ± 5% -0.3 0.95 ± 6% perf-profile.children.cycles-pp.__alloc_skb 1.21 ± 6% -0.3 0.95 ± 3% perf-profile.children.cycles-pp.inet_csk_accept 0.62 ± 5% -0.2 0.38 ± 10% perf-profile.children.cycles-pp.__tcp_get_metrics 1.06 ± 5% -0.2 0.83 ± 8% perf-profile.children.cycles-pp.sock_def_wakeup 0.60 ± 6% -0.2 0.38 ± 4% perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_skb 0.83 ± 4% -0.2 0.63 ± 4% perf-profile.children.cycles-pp.sk_forced_mem_schedule 1.25 ± 6% -0.2 1.05 ± 5% perf-profile.children.cycles-pp.inet_csk_destroy_sock 0.96 ± 5% -0.2 0.77 ± 4% perf-profile.children.cycles-pp.autoremove_wake_function 1.14 ± 6% -0.2 0.94 ± 6% perf-profile.children.cycles-pp.alloc_empty_file 1.15 ± 6% -0.2 0.96 ± 6% perf-profile.children.cycles-pp.alloc_file 1.12 ± 6% -0.2 0.93 ± 6% perf-profile.children.cycles-pp.__alloc_file 0.36 ± 7% -0.2 0.19 ± 3% perf-profile.children.cycles-pp.ip_finish_output 0.78 ± 5% -0.2 0.61 ± 6% perf-profile.children.cycles-pp.sk_stream_alloc_skb 0.90 ± 6% -0.2 0.73 ± 5% perf-profile.children.cycles-pp.dev_hard_start_xmit 0.88 ± 5% -0.2 0.72 ± 6% perf-profile.children.cycles-pp.__sk_mem_reduce_allocated 0.83 ± 5% -0.2 0.67 ± 5% perf-profile.children.cycles-pp.tcp_send_fin 1.23 ± 4% -0.2 1.07 ± 9% perf-profile.children.cycles-pp.menu_select 0.73 ± 6% -0.2 0.57 ± 7% perf-profile.children.cycles-pp.sock_alloc 0.72 ± 6% -0.2 0.56 ± 7% perf-profile.children.cycles-pp.new_inode_pseudo 1.02 ± 5% -0.2 0.86 ± 6% perf-profile.children.cycles-pp.__kfree_skb 0.68 ± 4% -0.2 0.53 ± 8% perf-profile.children.cycles-pp.tcp_get_metrics 0.79 ± 6% -0.1 0.65 ± 4% perf-profile.children.cycles-pp.loopback_xmit 0.93 ± 5% -0.1 0.79 ± 5% perf-profile.children.cycles-pp.inet_csk_clone_lock 0.70 ± 5% -0.1 0.56 ± 6% perf-profile.children.cycles-pp.__dentry_kill 0.68 ± 5% -0.1 0.54 ± 6% perf-profile.children.cycles-pp.sk_reset_timer 0.90 ± 5% -0.1 0.77 ± 6% perf-profile.children.cycles-pp.sk_clone_lock 0.61 ± 6% -0.1 0.49 ± 8% perf-profile.children.cycles-pp.alloc_inode 0.59 ± 5% -0.1 0.47 ± 9% perf-profile.children.cycles-pp.propagate_protected_usage 0.78 ± 5% -0.1 0.66 ± 7% perf-profile.children.cycles-pp.security_file_alloc 0.61 ± 6% -0.1 0.49 ± 5% perf-profile.children.cycles-pp.apparmor_file_free_security 0.46 ± 5% -0.1 0.35 ± 5% perf-profile.children.cycles-pp.lock_timer_base 0.59 ± 5% -0.1 0.48 ± 6% perf-profile.children.cycles-pp.mod_timer 0.61 ± 6% -0.1 0.50 ± 5% perf-profile.children.cycles-pp.security_file_free 0.50 ± 6% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.sock_alloc_inode 0.45 ± 3% -0.1 0.35 ± 6% perf-profile.children.cycles-pp.sk_filter_trim_cap 0.71 ± 5% -0.1 0.61 ± 6% perf-profile.children.cycles-pp.apparmor_file_alloc_security 0.44 ± 7% -0.1 0.34 ± 8% perf-profile.children.cycles-pp.kmalloc_reserve 0.50 ± 6% -0.1 0.40 ± 6% perf-profile.children.cycles-pp.skb_copy_datagram_iter 0.48 ± 6% -0.1 0.40 ± 6% perf-profile.children.cycles-pp.__skb_datagram_iter 0.39 ± 7% -0.1 0.31 ± 9% perf-profile.children.cycles-pp.__kmalloc_node_track_caller 0.48 ± 4% -0.1 0.40 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu 0.46 ± 7% -0.1 0.38 ± 6% perf-profile.children.cycles-pp.sock_recvmsg 0.32 ± 7% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.irqtime_account_irq 0.33 ± 6% -0.1 0.26 ± 6% perf-profile.children.cycles-pp.tcp_write_queue_purge 0.40 ± 5% -0.1 0.33 ± 9% perf-profile.children.cycles-pp.raw_local_deliver 0.30 ± 4% -0.1 0.22 ± 6% perf-profile.children.cycles-pp.__ksize 0.42 ± 5% -0.1 0.34 ± 5% perf-profile.children.cycles-pp.native_sched_clock 0.37 ± 5% -0.1 0.30 ± 6% perf-profile.children.cycles-pp.apparmor_sk_clone_security 0.25 ± 6% -0.1 0.18 ± 8% perf-profile.children.cycles-pp.ip_local_out 0.40 ± 5% -0.1 0.34 ± 5% perf-profile.children.cycles-pp.skb_release_all 0.32 ± 8% -0.1 0.25 ± 6% perf-profile.children.cycles-pp.netif_rx 0.37 ± 5% -0.1 0.30 ± 6% perf-profile.children.cycles-pp.security_sk_clone 0.37 ± 5% -0.1 0.30 ± 5% perf-profile.children.cycles-pp.__tcp_send_ack 0.26 ± 5% -0.1 0.19 ± 5% perf-profile.children.cycles-pp.__skb_clone 0.27 ± 5% -0.1 0.20 ± 10% perf-profile.children.cycles-pp.validate_xmit_skb 0.23 ± 4% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__ip_local_out 0.38 ± 5% -0.1 0.32 ± 4% perf-profile.children.cycles-pp.skb_release_head_state 0.22 ± 6% -0.1 0.15 ± 7% perf-profile.children.cycles-pp.__check_object_size 0.30 ± 8% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.netif_rx_internal 0.46 ± 4% -0.1 0.40 ± 5% perf-profile.children.cycles-pp.read_tsc 0.32 ± 6% -0.1 0.27 ± 4% perf-profile.children.cycles-pp.tcp_mstamp_refresh 0.22 ± 8% -0.1 0.16 ± 11% perf-profile.children.cycles-pp.mod_objcg_state 0.32 ± 5% -0.1 0.26 ± 6% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.26 ± 7% -0.1 0.21 ± 5% perf-profile.children.cycles-pp.enqueue_to_backlog 0.18 ± 4% -0.1 0.13 ± 9% perf-profile.children.cycles-pp.ip_send_check 0.14 ± 9% -0.1 0.08 ± 4% perf-profile.children.cycles-pp.migrate_enable 0.28 ± 6% -0.1 0.22 ± 5% perf-profile.children.cycles-pp.sk_stop_timer 0.25 ± 7% -0.1 0.20 ± 4% perf-profile.children.cycles-pp.inet_csk_clear_xmit_timers 0.20 ± 7% -0.0 0.16 ± 6% perf-profile.children.cycles-pp.kmem_cache_alloc_node 0.32 ± 3% -0.0 0.28 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair 0.22 ± 3% -0.0 0.18 ± 8% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.25 ± 6% -0.0 0.20 ± 8% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.24 ± 5% -0.0 0.19 ± 9% perf-profile.children.cycles-pp.check_preempt_curr 0.12 ± 10% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.sock_put 0.29 ± 5% -0.0 0.24 ± 6% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.19 ± 5% -0.0 0.15 ± 7% perf-profile.children.cycles-pp.del_timer 0.14 ± 7% -0.0 0.10 ± 14% perf-profile.children.cycles-pp.netif_skb_features 0.19 ± 5% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.__entry_text_start 0.17 ± 7% -0.0 0.13 ± 6% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.09 ± 8% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.netdev_core_pick_tx 0.15 ± 5% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.tcp_rcv_space_adjust 0.07 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.minmax_running_min 0.12 ± 4% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.tcp_queue_rcv 0.15 ± 9% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.simple_copy_to_iter 0.18 ± 5% -0.0 0.14 ± 8% perf-profile.children.cycles-pp._copy_to_iter 0.20 ± 6% -0.0 0.17 ± 9% perf-profile.children.cycles-pp.resched_curr 0.18 ± 3% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.sock_getsockopt 0.11 ± 6% -0.0 0.08 ± 7% perf-profile.children.cycles-pp.destroy_inode 0.15 ± 6% -0.0 0.12 ± 8% perf-profile.children.cycles-pp.__netif_receive_skb_core 0.11 ± 8% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.__destroy_inode 0.08 ± 11% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.nf_hook_slow 0.15 ± 2% -0.0 0.12 ± 10% perf-profile.children.cycles-pp.copyout 0.16 ± 7% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.__cond_resched 0.09 ± 10% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.inet_reqsk_alloc 0.09 ± 12% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.inet_csk_complete_hashdance 0.17 ± 4% -0.0 0.14 ± 8% perf-profile.children.cycles-pp.__fget_light 0.12 ± 6% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.cpus_share_cache 0.14 ± 5% -0.0 0.12 ± 10% perf-profile.children.cycles-pp.tcp_wfree 0.09 ± 10% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.migrate_disable 0.14 ± 7% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.sk_setup_caps 0.14 ± 3% -0.0 0.11 ± 5% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.14 ± 7% -0.0 0.11 ± 5% perf-profile.children.cycles-pp.d_instantiate 0.08 ± 8% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.tcp_options_write 0.16 ± 4% -0.0 0.13 ± 4% perf-profile.children.cycles-pp.evict 0.11 ± 8% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.tcp_validate_incoming 0.07 ± 7% -0.0 0.04 ± 45% perf-profile.children.cycles-pp.__d_instantiate 0.11 ± 6% -0.0 0.09 ± 6% perf-profile.children.cycles-pp.__might_fault 0.10 ± 9% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 0.09 ± 5% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.security_sock_rcv_skb 0.08 ± 8% -0.0 0.06 perf-profile.children.cycles-pp.kmem_cache_alloc_trace 0.08 ± 8% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.move_addr_to_user 0.07 ± 10% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.set_next_task_idle 0.08 ± 4% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.pick_next_task_idle 0.06 ± 7% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.prepare_to_wait_exclusive 0.08 ± 6% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.add_wait_queue 0.17 ± 5% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.inet_twsk_deschedule_put 0.10 ± 8% +0.0 0.13 ± 9% perf-profile.children.cycles-pp.inet_twsk_alloc 0.06 ± 11% +0.0 0.10 ± 9% perf-profile.children.cycles-pp.tcp_mtup_init 0.14 ± 5% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.inet_twsk_kill 0.23 ± 8% +0.1 0.29 ± 6% perf-profile.children.cycles-pp.tcp_make_synack 0.06 ± 7% +0.1 0.12 ± 6% perf-profile.children.cycles-pp.tcp_synack_rtt_meas 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.tcp_rack_update_reo_wnd 0.06 ± 9% +0.1 0.12 ± 5% perf-profile.children.cycles-pp.tcp_ca_openreq_child 0.03 ±105% +0.1 0.10 ± 10% perf-profile.children.cycles-pp.inet_lhash2_lookup 0.08 ± 20% +0.1 0.15 ± 7% perf-profile.children.cycles-pp.__inet_lookup_listener 0.13 ± 6% +0.1 0.20 ± 3% perf-profile.children.cycles-pp.__ip_finish_output 0.06 ± 7% +0.1 0.14 ± 8% perf-profile.children.cycles-pp.__ip_dev_find 0.07 ± 5% +0.1 0.15 ± 8% perf-profile.children.cycles-pp.tcp_sync_mss 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.sock_prot_inuse_add 0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.tcp_push 0.42 ± 7% +0.1 0.50 ± 6% perf-profile.children.cycles-pp.tcp_time_wait 0.07 ± 12% +0.1 0.17 ± 5% perf-profile.children.cycles-pp.secure_tcp_ts_off 0.07 ± 10% +0.1 0.17 ± 5% perf-profile.children.cycles-pp.tcp_select_initial_window 0.02 ± 99% +0.1 0.13 ± 7% perf-profile.children.cycles-pp.inet_lookup_ifaddr_rcu 0.08 ± 7% +0.1 0.20 ± 8% perf-profile.children.cycles-pp.skb_entail 0.11 ± 12% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.tcp_openreq_init_rwin 0.00 +0.1 0.13 ± 8% perf-profile.children.cycles-pp.tcp_stream_memory_free 0.14 ± 7% +0.1 0.27 ± 3% perf-profile.children.cycles-pp.tcp_update_pacing_rate 0.14 ± 4% +0.1 0.28 ± 10% perf-profile.children.cycles-pp.tcp_parse_options 0.36 ± 4% +0.1 0.50 ± 7% perf-profile.children.cycles-pp.tcp_finish_connect 0.17 ± 7% +0.1 0.31 ± 4% perf-profile.children.cycles-pp.tcp_connect_init 0.24 ± 6% +0.2 0.41 ± 7% perf-profile.children.cycles-pp.fib_table_lookup 0.09 ± 8% +0.2 0.26 ± 5% perf-profile.children.cycles-pp.tcp_assign_congestion_control 0.00 +0.2 0.17 ± 9% perf-profile.children.cycles-pp.__sk_free 0.34 ± 6% +0.2 0.53 ± 5% perf-profile.children.cycles-pp.tcp_v4_init_sock 0.22 ± 5% +0.2 0.41 ± 6% perf-profile.children.cycles-pp.tcp_schedule_loss_probe 0.33 ± 6% +0.2 0.52 ± 5% perf-profile.children.cycles-pp.tcp_init_sock 0.59 ± 6% +0.2 0.78 ± 5% perf-profile.children.cycles-pp.tcp_v4_send_synack 0.14 ± 7% +0.2 0.34 ± 8% perf-profile.children.cycles-pp.ip_rcv_finish 0.11 ± 8% +0.2 0.31 ± 8% perf-profile.children.cycles-pp.ip_rcv_finish_core 0.07 ± 8% +0.2 0.29 ± 7% perf-profile.children.cycles-pp.fib_lookup_good_nhc 0.20 ± 4% +0.2 0.42 ± 8% perf-profile.children.cycles-pp.inet_csk_route_child_sock 0.18 ± 6% +0.2 0.41 ± 8% perf-profile.children.cycles-pp.inet_csk_route_req 0.04 ± 71% +0.2 0.27 ± 8% perf-profile.children.cycles-pp.tcp_newly_delivered 0.64 ± 4% +0.2 0.88 ± 5% perf-profile.children.cycles-pp.tcp_init_transfer 0.11 ± 7% +0.2 0.36 ± 6% perf-profile.children.cycles-pp.inet_sk_rebuild_header 0.11 ± 6% +0.3 0.38 ± 5% perf-profile.children.cycles-pp.inet_sock_destruct 0.10 ± 6% +0.3 0.39 ± 6% perf-profile.children.cycles-pp.__tcp_select_window 0.21 ± 7% +0.3 0.54 ± 6% perf-profile.children.cycles-pp.tcp_ack_update_rtt 0.14 ± 5% +0.3 0.49 ± 6% perf-profile.children.cycles-pp.inet_ehashfn 1.73 ± 5% +0.3 2.08 ± 5% perf-profile.children.cycles-pp.tcp_v4_syn_recv_sock 0.17 ± 5% +0.4 0.55 ± 6% perf-profile.children.cycles-pp.ip_rcv_core 1.95 ± 5% +0.4 2.36 ± 6% perf-profile.children.cycles-pp.tcp_check_req 1.61 ± 5% +0.5 2.14 ± 6% perf-profile.children.cycles-pp.tcp_conn_request 0.44 ± 6% +0.6 1.03 ± 7% perf-profile.children.cycles-pp.ip_route_output_flow 2.07 ± 5% +0.7 2.77 ± 6% perf-profile.children.cycles-pp.tcp_ack 0.27 ± 3% +0.8 1.06 ± 4% perf-profile.children.cycles-pp.__sk_dst_check 0.62 ± 6% +0.8 1.42 ± 7% perf-profile.children.cycles-pp.ip_route_output_key_hash 0.58 ± 5% +0.8 1.38 ± 6% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu 0.26 ± 4% +1.0 1.22 ± 4% perf-profile.children.cycles-pp.ipv4_dst_check 5.78 ± 4% +1.1 6.92 ± 6% perf-profile.children.cycles-pp.tcp_rcv_synsent_state_process 4.54 ± 4% +1.2 5.74 ± 5% perf-profile.children.cycles-pp.tcp_connect 5.54 ± 4% +1.5 7.04 ± 5% perf-profile.children.cycles-pp.tcp_v4_connect 11.94 ± 4% +2.6 14.51 ± 5% perf-profile.children.cycles-pp.__x64_sys_connect 11.92 ± 4% +2.6 14.50 ± 5% perf-profile.children.cycles-pp.__sys_connect 11.72 ± 4% +2.6 14.31 ± 5% perf-profile.children.cycles-pp.inet_stream_connect 11.67 ± 4% +2.6 14.27 ± 5% perf-profile.children.cycles-pp.__inet_stream_connect 3.85 ± 5% -0.7 3.14 ± 5% perf-profile.self.cycles-pp.poll_idle 1.21 ± 6% -0.5 0.70 ± 7% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 1.87 ± 4% -0.4 1.49 ± 9% perf-profile.self.cycles-pp.page_counter_cancel 1.70 ± 5% -0.3 1.40 ± 6% perf-profile.self.cycles-pp._raw_spin_lock 1.04 ± 6% -0.3 0.74 ± 5% perf-profile.self.cycles-pp.percpu_counter_add_batch 1.14 ± 6% -0.3 0.88 ± 7% perf-profile.self.cycles-pp.__inet_lookup_established 0.62 ± 4% -0.2 0.37 ± 11% perf-profile.self.cycles-pp.__tcp_get_metrics 0.50 ± 9% -0.2 0.27 ± 10% perf-profile.self.cycles-pp.__sk_destruct 1.04 ± 7% -0.2 0.82 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_bh 0.82 ± 4% -0.2 0.62 ± 5% perf-profile.self.cycles-pp.sk_forced_mem_schedule 0.87 ± 6% -0.2 0.71 ± 6% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated 0.36 ± 6% -0.1 0.23 ± 4% perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_skb 0.59 ± 5% -0.1 0.46 ± 8% perf-profile.self.cycles-pp.propagate_protected_usage 0.41 ± 7% -0.1 0.29 ± 7% perf-profile.self.cycles-pp.tcp_recvmsg_locked 0.52 ± 9% -0.1 0.40 ± 8% perf-profile.self.cycles-pp.__dev_queue_xmit 0.60 ± 6% -0.1 0.49 ± 4% perf-profile.self.cycles-pp.apparmor_file_free_security 0.69 ± 5% -0.1 0.59 ± 6% perf-profile.self.cycles-pp.apparmor_file_alloc_security 0.34 ± 7% -0.1 0.24 ± 4% perf-profile.self.cycles-pp.tcp_clean_rtx_queue 0.40 ± 5% -0.1 0.33 ± 5% perf-profile.self.cycles-pp.native_sched_clock 0.39 ± 5% -0.1 0.32 ± 6% perf-profile.self.cycles-pp.kmem_cache_free 0.36 ± 4% -0.1 0.30 ± 6% perf-profile.self.cycles-pp.apparmor_sk_clone_security 0.40 ± 5% -0.1 0.33 ± 10% perf-profile.self.cycles-pp.raw_local_deliver 0.29 ± 4% -0.1 0.22 ± 6% perf-profile.self.cycles-pp.__ksize 0.28 ± 5% -0.1 0.22 ± 6% perf-profile.self.cycles-pp.ip_finish_output2 0.45 ± 4% -0.1 0.38 ± 5% perf-profile.self.cycles-pp.read_tsc 0.21 ± 7% -0.1 0.15 ± 6% perf-profile.self.cycles-pp.__skb_clone 0.32 ± 5% -0.1 0.26 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.26 ± 8% -0.1 0.21 ± 6% perf-profile.self.cycles-pp.tcp_sendmsg_locked 0.17 ± 7% -0.1 0.12 ± 7% perf-profile.self.cycles-pp.fib_table_lookup 0.18 ± 5% -0.1 0.13 ± 9% perf-profile.self.cycles-pp.ip_send_check 0.12 ± 8% -0.1 0.07 ± 12% perf-profile.self.cycles-pp.migrate_enable 0.24 ± 6% -0.1 0.20 ± 6% perf-profile.self.cycles-pp.__alloc_skb 0.19 ± 6% -0.0 0.14 ± 6% perf-profile.self.cycles-pp.process_backlog 0.18 ± 5% -0.0 0.13 ± 5% perf-profile.self.cycles-pp.net_rx_action 0.34 ± 5% -0.0 0.30 ± 4% perf-profile.self.cycles-pp.enqueue_entity 0.07 ± 8% -0.0 0.03 ±100% perf-profile.self.cycles-pp.minmax_running_min 0.12 ± 10% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.sock_put 0.12 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.tcp_queue_rcv 0.23 ± 4% -0.0 0.19 ± 10% perf-profile.self.cycles-pp.__softirqentry_text_start 0.19 ± 5% -0.0 0.15 ± 6% perf-profile.self.cycles-pp.__entry_text_start 0.16 ± 4% -0.0 0.12 ± 10% perf-profile.self.cycles-pp.mod_objcg_state 0.15 ± 9% -0.0 0.11 ± 6% perf-profile.self.cycles-pp.do_syscall_64 0.22 ± 4% -0.0 0.18 ± 7% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.08 ± 10% -0.0 0.04 ± 44% perf-profile.self.cycles-pp.netdev_core_pick_tx 0.07 ± 10% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.inet_reqsk_alloc 0.15 ± 6% -0.0 0.11 ± 8% perf-profile.self.cycles-pp.kmem_cache_alloc_node 0.21 ± 5% -0.0 0.17 ± 5% perf-profile.self.cycles-pp.__local_bh_enable_ip 0.09 ± 5% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.tcp_data_queue 0.20 ± 6% -0.0 0.17 ± 9% perf-profile.self.cycles-pp.resched_curr 0.16 ± 5% -0.0 0.13 ± 5% perf-profile.self.cycles-pp.__tcp_close 0.12 ± 6% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.10 ± 6% -0.0 0.07 ± 8% perf-profile.self.cycles-pp.irqtime_account_irq 0.14 ± 8% -0.0 0.12 ± 11% perf-profile.self.cycles-pp.sk_reset_timer 0.14 ± 4% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.__netif_receive_skb_core 0.08 ± 6% -0.0 0.05 ± 45% perf-profile.self.cycles-pp.tcp_rcv_synsent_state_process 0.08 ± 11% -0.0 0.06 ± 9% perf-profile.self.cycles-pp.migrate_disable 0.12 ± 6% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.cpus_share_cache 0.10 ± 4% -0.0 0.07 ± 11% perf-profile.self.cycles-pp.tcp_rcv_space_adjust 0.14 ± 10% -0.0 0.11 ± 5% perf-profile.self.cycles-pp.enqueue_to_backlog 0.12 ± 6% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.validate_xmit_skb 0.11 ± 8% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.tcp_validate_incoming 0.16 ± 4% -0.0 0.14 ± 9% perf-profile.self.cycles-pp.__fget_light 0.09 ± 4% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.__netif_receive_skb_one_core 0.07 ± 6% -0.0 0.05 perf-profile.self.cycles-pp.tcp_options_write 0.08 ± 10% -0.0 0.06 ± 15% perf-profile.self.cycles-pp.netif_skb_features 0.14 ± 5% -0.0 0.12 ± 6% perf-profile.self.cycles-pp.sk_setup_caps 0.08 ± 8% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.tcp_cleanup_rbuf 0.11 ± 6% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.do_softirq 0.09 ± 4% -0.0 0.07 ± 9% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.06 ± 7% -0.0 0.04 ± 45% perf-profile.self.cycles-pp.tcp_established_options 0.08 ± 4% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.sk_filter_trim_cap 0.08 ± 7% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.10 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.dequeue_task_fair 0.07 ± 10% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__alloc_file 0.08 ± 9% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.schedule 0.06 ± 13% +0.0 0.08 ± 8% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.06 ± 11% +0.0 0.09 ± 7% perf-profile.self.cycles-pp.tcp_init_transfer 0.09 ± 9% +0.0 0.12 ± 7% perf-profile.self.cycles-pp.inet_twsk_alloc 0.06 ± 13% +0.0 0.09 ± 7% perf-profile.self.cycles-pp.tcp_mtup_init 0.07 ± 7% +0.0 0.10 ± 7% perf-profile.self.cycles-pp.inet_csk_get_port 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.__inet_inherit_port 0.11 ± 9% +0.1 0.17 ± 6% perf-profile.self.cycles-pp.sk_clone_lock 0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.inet_put_port 0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.__inet_lookup_listener 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.tcp_rack_update_reo_wnd 0.08 ± 12% +0.1 0.14 ± 6% perf-profile.self.cycles-pp.tcp_connect 0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.inet_twsk_kill 0.02 ±146% +0.1 0.10 ± 10% perf-profile.self.cycles-pp.inet_lhash2_lookup 0.12 ± 6% +0.1 0.20 ± 4% perf-profile.self.cycles-pp.__ip_finish_output 0.07 ± 7% +0.1 0.15 ± 8% perf-profile.self.cycles-pp.tcp_sync_mss 0.06 ± 9% +0.1 0.14 ± 7% perf-profile.self.cycles-pp.tcp_v4_syn_recv_sock 0.00 +0.1 0.08 ± 4% perf-profile.self.cycles-pp.sock_prot_inuse_add 0.06 ± 11% +0.1 0.15 ± 7% perf-profile.self.cycles-pp.tcp_make_synack 0.07 +0.1 0.16 ± 6% perf-profile.self.cycles-pp.__inet_bind 0.00 +0.1 0.09 ± 5% perf-profile.self.cycles-pp.tcp_push 0.04 ± 45% +0.1 0.14 ± 6% perf-profile.self.cycles-pp.tcp_openreq_init_rwin 0.05 +0.1 0.15 ± 6% perf-profile.self.cycles-pp.tcp_create_openreq_child 0.07 ± 9% +0.1 0.17 ± 6% perf-profile.self.cycles-pp.tcp_select_initial_window 0.00 +0.1 0.10 ± 7% perf-profile.self.cycles-pp.tcp_v4_send_synack 0.04 ± 45% +0.1 0.15 ± 6% perf-profile.self.cycles-pp.tcp_get_metrics 0.02 ± 99% +0.1 0.13 ± 7% perf-profile.self.cycles-pp.inet_lookup_ifaddr_rcu 0.06 ± 13% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.tcp_connect_init 0.09 ± 16% +0.1 0.20 ± 7% perf-profile.self.cycles-pp.tcp_update_metrics 0.05 ± 49% +0.1 0.18 ± 7% perf-profile.self.cycles-pp.tcp_init_metrics 0.00 +0.1 0.12 ± 11% perf-profile.self.cycles-pp.tcp_stream_memory_free 0.06 ± 7% +0.1 0.19 ± 8% perf-profile.self.cycles-pp.skb_entail 0.12 ± 8% +0.1 0.24 ± 5% perf-profile.self.cycles-pp.tcp_event_new_data_sent 0.13 ± 7% +0.1 0.27 ± 3% perf-profile.self.cycles-pp.tcp_update_pacing_rate 0.14 ± 4% +0.1 0.28 ± 9% perf-profile.self.cycles-pp.tcp_parse_options 0.01 ±223% +0.1 0.14 ± 5% perf-profile.self.cycles-pp.secure_tcp_ts_off 0.45 ± 7% +0.1 0.60 ± 6% perf-profile.self.cycles-pp.tcp_v4_rcv 0.00 +0.1 0.15 ± 10% perf-profile.self.cycles-pp.__sk_free 0.47 ± 4% +0.2 0.62 ± 8% perf-profile.self.cycles-pp.tcp_ack 0.06 ± 45% +0.2 0.22 ± 7% perf-profile.self.cycles-pp.tcp_set_state 0.08 ± 8% +0.2 0.25 ± 6% perf-profile.self.cycles-pp.tcp_assign_congestion_control 0.10 ± 7% +0.2 0.31 ± 7% perf-profile.self.cycles-pp.ip_rcv_finish_core 0.16 ± 4% +0.2 0.37 ± 7% perf-profile.self.cycles-pp.tcp_schedule_loss_probe 0.07 ± 8% +0.2 0.28 ± 8% perf-profile.self.cycles-pp.fib_lookup_good_nhc 0.02 ±141% +0.2 0.23 ± 4% perf-profile.self.cycles-pp.tcp_init_sock 0.03 ±100% +0.2 0.26 ± 7% perf-profile.self.cycles-pp.tcp_newly_delivered 0.10 ± 5% +0.3 0.38 ± 6% perf-profile.self.cycles-pp.__tcp_select_window 0.28 ± 6% +0.3 0.60 ± 7% perf-profile.self.cycles-pp.tcp_write_xmit 0.14 ± 7% +0.3 0.48 ± 6% perf-profile.self.cycles-pp.inet_ehashfn 0.12 ± 7% +0.4 0.48 ± 6% perf-profile.self.cycles-pp.tcp_ack_update_rtt 0.17 ± 5% +0.4 0.55 ± 7% perf-profile.self.cycles-pp.ip_rcv_core 0.77 ± 4% +0.4 1.20 ± 6% perf-profile.self.cycles-pp.__tcp_transmit_skb 0.09 ± 4% +0.5 0.59 ± 9% perf-profile.self.cycles-pp.ip_protocol_deliver_rcu 0.26 ± 7% +0.6 0.82 ± 6% perf-profile.self.cycles-pp.ip_route_output_key_hash_rcu 0.25 ± 5% +0.6 0.88 ± 6% perf-profile.self.cycles-pp.ip_output 0.25 ± 3% +1.0 1.21 ± 4% perf-profile.self.cycles-pp.ipv4_dst_check netperf.Throughput_tps 26000 +-------------------------------------------------------------------+ | | 25000 |-+ +.++.++.++.++.++.+ | 24000 |+.++.++.++.++.+ ++.+ +.++. | | +.++.++.++.++.++.+ ++.++.+| 23000 |-+ | | | 22000 |-+ | | | 21000 |-+ | 20000 |-+ | |O OO OO OO OO O | 19000 |-+ O OO OO O O | | O OO O O | 18000 +-------------------------------------------------------------------+ netperf.Throughput_total_tps 420000 +------------------------------------------------------------------+ | | 400000 |-+ ++.++.++.++.++.++. | |+.++.++.++.++.+ +++. .++. | 380000 |-+ ++.++.++.++.+++.++ ++.++.+| | | 360000 |-+ | | | 340000 |-+ | | | 320000 |O+O OO O O | | O OO O | 300000 |-+ OO OO OO OO OO OO | | | 280000 +------------------------------------------------------------------+ netperf.workload 1.25e+08 +----------------------------------------------------------------+ | | 1.2e+08 |-+ + +. .+++.++.+++.++.++. | 1.15e+08 |+.++.+ +.+ ++ +++. .++.+ | | ++.+++.++.+++.++ ++.++.+| 1.1e+08 |-+ | | | 1.05e+08 |-+ | | | 1e+08 |-+ | 9.5e+07 |-+ | |O OO OOO OO OO | 9e+07 |-+ OOO OO OOO OO OO | | | 8.5e+07 +----------------------------------------------------------------+ netperf.time.user_time 85 +----------------------------------------------------------------------+ | ++ | 80 |-+ :: | | : : | 75 |-+ : : | | +.+ +. .+. : : | 70 |.++.++.+ +. .+. +.+ +.+ ++ ++.+ +.+.+ .++ ++.++.| | +.+ ++ + +.++.+ +.+ + | 65 |-+ +.+ | | | 60 |-+ | | | 55 |-+O O O OO O | | O OO O O O OO OO O O | 50 +----------------------------------------------------------------------+ netperf.time.system_time 2620 +--------------------------------------------------------------------+ | +.++. + + +.++.++. +.+ .+ ++.+ + | 2600 |-+ + + +.+ + +. : + +.++ +.+ .+ + + +.++.+| 2580 |-+ + + + +.++ +.++ + | | | 2560 |-+ | | | 2540 |-+ | | | 2520 |-+O O O OO | 2500 |-O OO O O O O OO | | O O | 2480 |-+ OO OO | | OO | 2460 +--------------------------------------------------------------------+ netperf.time.percent_of_cpu_this_job_got 890 +---------------------------------------------------------------------+ | | 880 |.++.++. +.++.+ +. .++.++.+ | | ++. +.+ .++. : +.+ ++.+.++. +. + +. .++.| 870 |-+ + + + + ++.++.++.+ ++ | | | 860 |-+ | | | 850 |-+ | | | 840 |-OO OO OO OO OO O OO | | OO O | 830 |-+ O OO OO | | O | 820 +---------------------------------------------------------------------+ netperf.time.voluntary_context_switches 2.4e+08 +-----------------------------------------------------------------+ | .++.++.+++.++.++.+ | 2.3e+08 |+.++.++.+++.++ : | | ++.++.++.+++.++.++.++.+++.++.++.+| 2.2e+08 |-+ | | | 2.1e+08 |-+ | | | 2e+08 |-+ | | | 1.9e+08 |-+ | |O OO OO OOO OO | 1.8e+08 |-+ O | | OO OO OO OO OO O | 1.7e+08 +-----------------------------------------------------------------+ 3.6e+06 +-----------------------------------------------------------------+ 3.5e+06 |-+ +.++ .+ +. | | : + +.+ +++.+ | 3.4e+06 |+.+ +.+++.++.++.+ +.++. +.++.++.+++. | 3.3e+06 |-+ +.+ +++.+ ++.++.+| | | 3.2e+06 |-+ | 3.1e+06 |-+ | 3e+06 |-+ | | | 2.9e+06 |-+ | 2.8e+06 |O+OO OO OOO OO | | OO OO OO O | 2.7e+06 |-+ O O OO O | 2.6e+06 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang