LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Robin Holt <holt@sgi.com>
Cc: Hugh Dickins <hughd@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org
Subject: Re: shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend).
Date: Sat, 22 Jan 2011 07:59:11 -0800	[thread overview]
Message-ID: <AANLkTikG9ud-hpXaLjLcv6+j9O00guhcZTa=OPs9PX4A@mail.gmail.com> (raw)
In-Reply-To: <20110122153459.GF3306@sgi.com>

On Sat, Jan 22, 2011 at 7:34 AM, Robin Holt <holt@sgi.com> wrote:
> I have a customer system with 12 TB of memory.  The customer is trying
> to do a shmget() call with size of 4TB and it fails due to the check in
> shmem_file_setup() against SHMEM_MAX_BYTES which is 0x4020010000.
>
> I have considered a bunch of options and really do not know which
> direction I should take this.
>
> I could add a third level and fourth level with a similar 1/4 size being
> the current level of indirection, and the next quarter being a next level.
> That would get me closer, but not all the way there.

Ugh.

How about just changing the indexing to use a bigger page allocation?
Right now it uses PAGE_CACHE_SIZE and ENTRIES_PER_PAGE, but as far as
I can tell, the indexing logic is entirely independent from PAGE_SIZE
and PAGE_CACHE_SIZE, and could just use its own SHM_INDEX_PAGE_SIZE or
something.

That would allow increasing the indexing capability fairly easily, no?
No actual change to the (messy) algorithm at all, just make the block
size for the index pages bigger.

Sure, it means that you now require multipage allocations in
shmem_dir_alloc(), but that doesn't sound all that hard. The code is
already set up to try to handle it (because we have that conceptual
difference between PAGE_SIZE and PAGE_CACHE_SIZE, even though the two
end up being the same).

NOTE! I didn't look very closely at the details, there may be some
really basic reason why the above is a completely idiotic idea.

The alternative (and I think it might be a good alternative) is to get
rid of the shmem magic indexing entirely, and rip all the code out and
replace it with something that uses the generic radix tree functions.
Again, I didn't actually look at the code enough to judge whether that
would be the most painful effort ever or even possible at all.

                     Linus

  reply	other threads:[~2011-01-22 15:59 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-22 15:30 shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes Robin Holt
2011-01-22 15:34 ` shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend) Robin Holt
2011-01-22 15:59   ` Linus Torvalds [this message]
2011-01-22 22:22     ` Hugh Dickins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='AANLkTikG9ud-hpXaLjLcv6+j9O00guhcZTa=OPs9PX4A@mail.gmail.com' \
    --to=torvalds@linux-foundation.org \
    --cc=akpm@linux-foundation.org \
    --cc=holt@sgi.com \
    --cc=hughd@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --subject='Re: shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend).' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).