LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andrew Lunn <andrew@lunn.ch>
To: "Rafał Miłecki" <zajec5@gmail.com>
Cc: Network Development <netdev@vger.kernel.org>,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-block@vger.kernel.org, John Crispin <john@phrozen.org>,
	Jonas Gorski <jonas.gorski@gmail.com>,
	Jo-Philipp Wich <jo@mein.io>, Felix Fietkau <nbd@nbd.name>
Subject: Re: ARM router NAT performance affected by random/unrelated commits
Date: Tue, 21 May 2019 15:01:01 +0200	[thread overview]
Message-ID: <20190521130101.GC6577@lunn.ch> (raw)
In-Reply-To: <9a9ba4c9-3cb7-eb64-4aac-d43b59224442@gmail.com>

> I also tried running cachestat but didn't get anything interesting:
> Counting cache functions... Output every 1 seconds.
> TIME         HITS   MISSES  DIRTIES    RATIO   BUFFERS_MB   CACHE_MB
> 10:06:59     1020        5        0    99.5%            0          2
> 10:07:00     1029        0        0   100.0%            0          2
> 10:07:01     1013        0        0   100.0%            0          2
> 10:07:02     1029        0        0   100.0%            0          2
> 10:07:03     1029        0        0   100.0%            0          2
> 10:07:04      997        0        0   100.0%            0          2
> 10:07:05     1013        0        0   100.0%            0          2
> (I started iperf at 10:07:00).

Try looking at the L1 cache performance. For this class of device, the
L1 code cache is probably too small to contain the active parts of the
network stack. The less cache thrashing you have, the faster the stack
will go.

Maybe try compiling with -Os so it optimises for size.

Build a custom kernel with everything you don't need turned off.

Look at the work being done to batch process packets. Rather than
passing one packet at a time through the network stack, it passes a
linked list of packets to each stage in the stack. That should result
in less cache misses per packet. But not all layers in the stack
support this batching. See if you can find out where it is being
unbatched, and why. Can you influence this, disable build options, or
work on the code to pass batches further along the stack.

     Andrew

      parent reply	other threads:[~2019-05-21 13:01 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-21 10:28 Rafał Miłecki
2019-05-21 10:45 ` Russell King - ARM Linux admin
2019-05-21 11:16   ` Rafał Miłecki
2019-05-21 11:19     ` Russell King - ARM Linux admin
2019-05-22 11:51   ` Rafał Miłecki
2019-05-22 12:17     ` Russell King - ARM Linux admin
2019-05-22 21:12       ` Rafał Miłecki
2019-05-21 13:01 ` Andrew Lunn [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190521130101.GC6577@lunn.ch \
    --to=andrew@lunn.ch \
    --cc=jo@mein.io \
    --cc=john@phrozen.org \
    --cc=jonas.gorski@gmail.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nbd@nbd.name \
    --cc=netdev@vger.kernel.org \
    --cc=zajec5@gmail.com \
    --subject='Re: ARM router NAT performance affected by random/unrelated commits' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).