Netdev Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Dominique Martinet <asmadeus@codewreck.org>
To: Eric Van Hensbergen <ericvh@gmail.com>
Cc: Christian Schoenebeck <linux_oss@crudebyte.com>,
	Greg Kurz <groug@kaod.org>, Latchesar Ionkov <lucho@ionkov.net>,
	netdev@vger.kernel.org, v9fs-developer@lists.sourceforge.net
Subject: Re: [PATCH 2/2] net/9p: increase default msize to 128k
Date: Mon, 6 Sep 2021 09:07:56 +0900	[thread overview]
Message-ID: <YTVb3K37JxUWUdXN@codewreck.org> (raw)
In-Reply-To: <CAFkjPTkJFrqhCCHgUBsDiEVjpeJoKZ4gRy=G-4DpJo9xanpYaA@mail.gmail.com>

Eric Van Hensbergen wrote on Sun, Sep 05, 2021 at 06:44:13PM -0500:
> there will likely be a tradeoff with tcp in terms of latency to first
> message so while
> absolute bw may be higher processing time may suffer.  8k was default msize
> to more closely match it to jumbo frames on an ethernet.  of course all
> that intuition is close to 30 years out of date….

It's not because the max size is 128k (or 1MB) that this much is sent
over the wire everytime -- if a message used to fit in 8KB, then its
on-the-wire size won't change and speed/latency won't be affected for
these.

For messages that do require more than 8KB (read/write/readdir) then you
can fit more data per message, so for a given userspace request (feed me
xyz amount of data) you'll have less client-server round-trips, and the
final user-reflected latency will be better as well -- that's why
e.g. NFS has been setting a max size of 1MB by default for a while now,
and they allow even more (32MB iirc? not sure)

I've only had done these tests years ago and no longer have access to
the note that was written back then, but TCP also definitely benefits
from > 64k msize as long as there's enough memory available.


The downside (because it's not free) is there though, you need more
memory for 9p with big buffers even if we didn't need so much in the
first place.
The code using a slab now means that the memory is not locked per mount
as it used to, but that also means allocations can fail if there is a
big pressure after not having been released. OTOH as long as it's
consistently used the buffers will be recycled so it's not necessarily
too bad performance-wise in hot periods.

Ideally we'd need to rework transports to allow scatter-gather (iovec or
similar API), and work with smaller allocations batched together on
send, but I don't have time for something like that... If we do that we
can probably get the best of both worlds -- and could consider >1MB, but
that's unrealistic as of now, regardless of the transport.

-- 
Dominique

  parent reply	other threads:[~2021-09-06  0:45 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-04 15:53 [PATCH 0/2] net/9p: increase default msize Christian Schoenebeck
2021-09-04 15:07 ` [PATCH 1/2] net/9p: use macro to define " Christian Schoenebeck
2021-09-04 15:12 ` [PATCH 2/2] net/9p: increase default msize to 128k Christian Schoenebeck
2021-09-04 23:31   ` Dominique Martinet
2021-09-05 13:33     ` Christian Schoenebeck
2021-09-05 21:48       ` Dominique Martinet
     [not found]         ` <CAFkjPTkJFrqhCCHgUBsDiEVjpeJoKZ4gRy=G-4DpJo9xanpYaA@mail.gmail.com>
2021-09-06  0:07           ` Dominique Martinet [this message]
2021-09-10 14:04             ` Christian Schoenebeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YTVb3K37JxUWUdXN@codewreck.org \
    --to=asmadeus@codewreck.org \
    --cc=ericvh@gmail.com \
    --cc=groug@kaod.org \
    --cc=linux_oss@crudebyte.com \
    --cc=lucho@ionkov.net \
    --cc=netdev@vger.kernel.org \
    --cc=v9fs-developer@lists.sourceforge.net \
    --subject='Re: [PATCH 2/2] net/9p: increase default msize to 128k' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).