Netdev Archive on
help / color / mirror / Atom feed
From: Martin Habets <>
To: "Íñigo Huguet" <>
Cc: Edward Cree <>,, Dinan Gunawardena <>
Subject: Re: Bad performance in RX with sfc 40G
Date: Sun, 2 Jan 2022 09:22:07 +0000	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

Hi Íñigo,

On Thu, Dec 23, 2021 at 02:18:03PM +0100, Íñigo Huguet wrote:
> Hi Martin,
> I replied this a few weeks ago, but it seems that, for some reason, I
> didn't CCd you.

I'm just getting back to work after my holidays. Happy new year!

> On Thu, Dec 9, 2021 at 1:06 PM Íñigo Huguet <> wrote:
> >
> > Hi,
> >
> > On Sat, Nov 20, 2021 at 9:31 AM Martin Habets <> wrote:
> > > If you're testing without the IOMMU enabled I suspect the recycle ring
> > > size may be too small. Can your try the patch below?
> >
> > Sorry for the very late reply, but I've had to be out of work for many days.
> >
> > This patch has improved the performance a lot, reaching the same
> > 30Gbps than in TX. However, it seems sometimes a bit erratic, still
> > dropping to 15Gbps sometimes, specially after module remove & probe,
> > or from one iperf call to another. But not being all the times, I
> > didn't found a clear pattern. Anyway, it clearly improves things.

Thanks for the feedback. After module probe the RX cache is cold (empty),
as pages only get recycled as they come in.
The issue you see between iperf calls could be related to the NUMA
locality of the cache. After the 1st run the cache will contain pages for
the NUMA node that iperf ran on. If a subsequent run executes on a
different NUMA node the pages in the cache are further away.
This is where pinning the iperf runs to cores on the same NUMA node will

> > Can this patch be applied as is or it's just a test?

The patch is good for a 40G NIC. But it won't be good enough on a 100G NIC,
and for a 10G NIC the size can be smaller.
I've been puzzling over the way to code this link speed dependency best.
We create this page ring when the interface is brought up, which is
before the driver knows the link speed. So I think it is best to
size it for the maximum speed of a given NIC.
In short, I'll work on a better patch for this.


  reply	other threads:[~2022-01-02  9:22 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-18 15:14 Íñigo Huguet
2021-11-18 17:19 ` Eric Dumazet
2021-12-02 14:26   ` Íñigo Huguet
2021-11-20  8:31 ` Martin Habets
2021-12-09 12:06   ` Íñigo Huguet
2021-12-23 13:18     ` Íñigo Huguet
2022-01-02  9:22       ` Martin Habets [this message]
2022-01-10  8:58         ` [PATCH net-next] sfc: The size of the RX recycle ring should be more flexible Martin Habets
2022-01-10  9:31           ` Íñigo Huguet
2022-01-12  9:05             ` Martin Habets
2022-01-31 11:08             ` Martin Habets
2022-01-10 17:22           ` Jakub Kicinski
2022-01-12  9:08             ` Martin Habets
2022-01-31 11:10           ` [PATCH V2 " Martin Habets
2022-02-02  5:10             ` patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \
    --subject='Re: Bad performance in RX with sfc 40G' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).