LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: wuqiang <wuqiang.matt@bytedance.com>
To: naveen.n.rao@linux.ibm.com, anil.s.keshavamurthy@intel.com,
	davem@davemloft.net, mhiramat@kernel.org, mingo@kernel.org,
	peterz@infradead.org, linux-kernel@vger.kernel.org,
	wuqiang.matt@bytedance.com
Cc: mattwu@163.com
Subject: [PATCH 0/2] *** kretprobe scalability improvement ***
Date: Sun,  8 Aug 2021 02:54:15 +0800	[thread overview]
Message-ID: <20210807185417.9209-1-wuqiang.matt@bytedance.com> (raw)

kretprobe is using freelist to manage return instances, but freelist as
a LIFO queue based on singly linked list, scales badly and thus lowers
throughput of kretprobed routines, especially for high parallelization.
Here's a typical result (XEON 8260: 2 sockets/48 cores/96 threads):

      1X       2X       4X       6X       8X      12X     16X
10880312 18121228 23214783 13155457 11190217 10991228 9623992
     24X      32X      48X      64X      96X     128X    192X
 8484455  8376786  6766684  5698349  4113405  4528009 4081401

This patch implements a scalabe, lock-less and numa-aware object pool
and as a result improves kretprobe to achieve near-linear scalability.
Tests of kretprobe throughput show the biggest gain as 181.5x of the
original freelist. Tge extreme tests of raw queue throughput can be up
to 282.8 of gain. The comparison results are the followings:

                  1X         2X         4X         8X        16X
freelist:  237911411  163596418   33048459   15506757   10640043
objpool:   234799081  294086132  585290693 1164205947 2334923746
                 24X        32X        48X        64X        96X
freelist:    9025299    7965531    6800225    5507639    4284752
objpool:  3508905695 1106760339 1101385147 1221763856 1211654038

The object pool is a percpu-extended version of original freelist,
with compact memory footprints and balanced performance results for
3 test caess: nonblockable retrieval (most kertprobe cases), bulk
retrieval in a row (multiple-threaded blockable kretprobe), huge
misses (preallocated objects much less than required).

wuqiang (2):
  scalable lock-less object pool implementation
  kretprobe: manage instances with scalable object pool

 include/linux/freelist.h | 521 ++++++++++++++++++++++++++++++++++++---
 include/linux/kprobes.h  |   2 +-
 kernel/kprobes.c         |  83 ++++---
 3 files changed, 536 insertions(+), 70 deletions(-)

-- 
2.25.1


             reply	other threads:[~2021-08-07 18:54 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-07 18:54 wuqiang [this message]
2021-08-07 18:54 ` [PATCH 1/2] scalable lock-less object pool implementation wuqiang
2021-08-07 18:54 ` [PATCH 2/2] kretprobe: manage instances with scalable object pool wuqiang
2021-08-29  9:29 ` [PATCH 0/2] *** kretprobe scalability improvement *** Masami Hiramatsu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210807185417.9209-1-wuqiang.matt@bytedance.com \
    --to=wuqiang.matt@bytedance.com \
    --cc=anil.s.keshavamurthy@intel.com \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mattwu@163.com \
    --cc=mhiramat@kernel.org \
    --cc=mingo@kernel.org \
    --cc=naveen.n.rao@linux.ibm.com \
    --cc=peterz@infradead.org \
    --subject='Re: [PATCH 0/2] *** kretprobe scalability improvement ***' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).