Netdev Archive on
help / color / mirror / Atom feed
From: Jakub Kicinski <>
To: Felix Fietkau <>
Cc:, Eric Dumazet <>,
	Hillf Danton <>
Subject: Re: [PATCH v3 1/2] net: add support for threaded NAPI polling
Date: Fri, 21 Aug 2020 18:49:24 -0700	[thread overview]
Message-ID: <20200821184924.5b5c421c@kicinski-fedora-PC1C0HJN> (raw)
In-Reply-To: <>

On Fri, 21 Aug 2020 21:01:50 +0200 Felix Fietkau wrote:
> For some drivers (especially 802.11 drivers), doing a lot of work in the NAPI
> poll function does not perform well. Since NAPI poll is bound to the CPU it
> was scheduled from, we can easily end up with a few very busy CPUs spending
> most of their time in softirq/ksoftirqd and some idle ones.
> Introduce threaded NAPI for such drivers based on a workqueue. The API is the
> same except for using netif_threaded_napi_add instead of netif_napi_add.
> In my tests with mt76 on MT7621 using threaded NAPI + a thread for tx scheduling
> improves LAN->WLAN bridging throughput by 10-50%. Throughput without threaded
> NAPI is wildly inconsistent, depending on the CPU that runs the tx scheduling
> thread.
> With threaded NAPI, throughput seems stable and consistent (and higher than
> the best results I got without it).
> Based on a patch by Hillf Danton

I've tested this patch on a non-NUMA system with a moderately
high-network workload (roughly 1:6 network to compute cycles)
- and it provides ~2.5% speedup in terms of RPS but 1/6/10% worse
P50/P99/P999 latency.

I started working on a counter-proposal which uses a pool of threads
dedicated to NAPI polling. It's not unlike the workqueue code but
trying to be a little more clever. It gives me ~6.5% more RPS but at
the same time lowers the p99 latency by 35% without impacting other
percentiles. (I only started testing this afternoon, so hopefully the
numbers will improve further).

I'm happy for this patch to be merged, it's quite nice, but I wanted 
to give the heads up that I may have something that would replace it...

The extremely rough PoC, less than half-implemented code which is really
too broken to share:

  parent reply	other threads:[~2020-08-22  1:49 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-21 19:01 Felix Fietkau
2020-08-21 19:01 ` [PATCH v3 2/2] net: add sysfs attribute for enabling threaded NAPI Felix Fietkau
2020-08-22  1:49 ` Jakub Kicinski [this message]
2020-08-22 16:22   ` [PATCH v3 1/2] net: add support for threaded NAPI polling Eric Dumazet
2020-08-30  8:46   ` Sebastian Gottschall
2020-08-25 16:38 ` Wei Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200821184924.5b5c421c@kicinski-fedora-PC1C0HJN \ \ \ \ \ \
    --subject='Re: [PATCH v3 1/2] net: add support for threaded NAPI polling' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).