LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Robin Holt <holt@sgi.com>
To: Linus Torvalds <torvalds@linux-foundation.org>,
	Hugh Dickins <hughd@google.com>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: linux-kernel@vger.kernel.org
Subject: shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend).
Date: Sat, 22 Jan 2011 09:34:59 -0600	[thread overview]
Message-ID: <20110122153459.GF3306@sgi.com> (raw)
In-Reply-To: <20110122153033.GR2912@sgi.com>

I have a customer system with 12 TB of memory.  The customer is trying
to do a shmget() call with size of 4TB and it fails due to the check in
shmem_file_setup() against SHMEM_MAX_BYTES which is 0x4020010000.

I have considered a bunch of options and really do not know which
direction I should take this.

I could add a third level and fourth level with a similar 1/4 size being
the current level of indirection, and the next quarter being a next level.
That would get me closer, but not all the way there.

Given the complexity we would be introducing, I really lean towards
having a tree of tables like the page tables instead of the current
half is one level of indirection and the other half is two levels.
It adds complexity which really does not have much value that I can see.


An alternative to the current halves being different levels
of indirection, I considered reworking the info->next_index
increment/decrement to put it inside the same locking as the walk/fill
of the table.  With that, I could resize the table depth based
upon the next_index value.  For next_index from SHMEM_NR_DIRECT
to SHMEM_NR_DIRECT + ENTRIES_PER_PAGE (2MB), it could be direct.
>From there to SHMEM_NR_DIRECT + ENTRIES_PER_PAGE ** 2 (1GB), it could
be one level of indirection.  Then from there to SHMEM_NR_DIRECT +
ENTRIES_PER_PAGE ** 3 (512GB), it could be two levels of indirection.
Finally, from there to SHMEM_NR_DIRECT + ENTRIES_PER_PAGE ** 4 (256TB),
it could be three levels.  That should be enough for a little while.

I am unsure about the value of having the direct entries at the beginning.
Given they have been this way for this long, I would probably leave them
to minimize the chances for a performance impact.

Thanks,
Robin Holt

  reply	other threads:[~2011-01-22 15:35 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-22 15:30 shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes Robin Holt
2011-01-22 15:34 ` Robin Holt [this message]
2011-01-22 15:59   ` shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend) Linus Torvalds
2011-01-22 22:22     ` Hugh Dickins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110122153459.GF3306@sgi.com \
    --to=holt@sgi.com \
    --cc=akpm@linux-foundation.org \
    --cc=hughd@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: shmget limited by SHMEM_MAX_BYTES to 0x4020010000 bytes (Resend).' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).