nommu: handling anonymous mmap clearing in userspace rather than kernel

nommu: handling anonymous mmap clearing in userspace rather than kernel

[Mike Frysinger]

the problem: doing malloc() in userspace on no-mmu systems can take up
an inordinate amount of time merely due to kernel forcing anonymously
mapped memory to zero.  this is because the only way to implement
userspace malloc() on no-mmu systems is via mmap(MAP_ANONYMOUS), and
the kernel has an explicit memset() in mm/nommu.c:do_mmap_private()
when MAP_ANONYMOUS is in the mmap flags.  however, POSIX states that
memory returned from malloc() may be uninitialized, so this zeroing is
not wanted in that case.

the desire: be able to do a mmap(MAP_ANONYMOUS) to grab a chunk of
memory without the memset() call.  this may bring up other arguments
such as information leakage between kernels/processes, but on an
embedded no-mmu system, people are very willing to sacrifice that for
the very measurable performance gain.

a workaround: introduce a new no-mmu-only mmap flag MAP_UNINITIALIZE
to signal to the kernel that it should skip the memset().  this way,
userspace malloc() can do mmap(MAP_ANONYMOUS|MAP_UNINITIALIZE) to get
large chunks of memory without affecting any other anonymous mmap()
call.

another idea: move the memset() from the kernel to the system C
library.  make the kernel memset() controllable via a
no-mmu/embedded-only Kconfig option (with the default being retained:
the memset is called).  now the C library can easily control the
zero-ing of memory as required by POSIX, and you spend less time in
kernel space.  there are many places where the kernel<->user ABI is
not POSIX compliant and instead relies on the C library to provide the
little bit of POSIX glue, so this isnt a big deal.
-mike

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[David Howells]

Mike Frysinger <vapier.adi@gmail.com> wrote:

> a workaround: introduce a new no-mmu-only mmap flag MAP_UNINITIALIZE
> to signal to the kernel that it should skip the memset().  this way,
> userspace malloc() can do mmap(MAP_ANONYMOUS|MAP_UNINITIALIZE) to get
> large chunks of memory without affecting any other anonymous mmap()
> call.

I think that's reasonable for NOMMU.  It's not like the process accessing the
uninitialised memory is prevented from accessing anything it wants to anyway.

I would vote that the memset() should only be skipped if requested as there
may be programs that call mmap(MAP_ANONYMOUS) expecting the memory they're
given to be zeroed out.

David

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[Mike Frysinger]

On Wed, Apr 2, 2008 at 10:20 AM, David Howells <dhowells@redhat.com> wrote:
> Mike Frysinger <vapier.adi@gmail.com> wrote:
>  > a workaround: introduce a new no-mmu-only mmap flag MAP_UNINITIALIZE
>  > to signal to the kernel that it should skip the memset().  this way,
>  > userspace malloc() can do mmap(MAP_ANONYMOUS|MAP_UNINITIALIZE) to get
>  > large chunks of memory without affecting any other anonymous mmap()
>  > call.
>
>  I think that's reasonable for NOMMU.  It's not like the process accessing the
>  uninitialised memory is prevented from accessing anything it wants to anyway.
>
>  I would vote that the memset() should only be skipped if requested as there
>  may be programs that call mmap(MAP_ANONYMOUS) expecting the memory they're
>  given to be zeroed out.

in the second proposal, the C library would be expected to do this, so
no programs would be broken.  but you're right that any program that
invokes the mmap() syscall directly would not get zeroed memory ...
but is anyone doing such a crazy thing, let alone on embedded ?
-mike

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[Bernd Schmidt]

Mike Frysinger wrote:
> On Wed, Apr 2, 2008 at 10:20 AM, David Howells <dhowells@redhat.com> wrote:
>> Mike Frysinger <vapier.adi@gmail.com> wrote:
>>  > a workaround: introduce a new no-mmu-only mmap flag MAP_UNINITIALIZE
>>  > to signal to the kernel that it should skip the memset().  this way,
>>  > userspace malloc() can do mmap(MAP_ANONYMOUS|MAP_UNINITIALIZE) to get
>>  > large chunks of memory without affecting any other anonymous mmap()
>>  > call.
>>
>>  I think that's reasonable for NOMMU.  It's not like the process accessing the
>>  uninitialised memory is prevented from accessing anything it wants to anyway.
>>
>>  I would vote that the memset() should only be skipped if requested as there
>>  may be programs that call mmap(MAP_ANONYMOUS) expecting the memory they're
>>  given to be zeroed out.
> 
> in the second proposal, the C library would be expected to do this, so
> no programs would be broken.  but you're right that any program that
> invokes the mmap() syscall directly would not get zeroed memory ...
> but is anyone doing such a crazy thing, let alone on embedded ?

It's not a guarantee we should break.  What's wrong with just using the 
MAP_UNINITIALIZE code we have?


Bernd
-- 
This footer brought to you by insane German lawmakers.
Analog Devices GmbH      Wilhelm-Wagenfeld-Str. 6      80807 Muenchen
Sitz der Gesellschaft Muenchen, Registergericht Muenchen HRB 40368
Geschaeftsfuehrer Thomas Wessel, William A. Martin, Margaret Seif

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[Mike Frysinger]

On Wed, Apr 2, 2008 at 11:03 AM, Bernd Schmidt <bernds_cb1@t-online.de> wrote:
> Mike Frysinger wrote:
> > On Wed, Apr 2, 2008 at 10:20 AM, David Howells <dhowells@redhat.com>
> > > Mike Frysinger <vapier.adi@gmail.com> wrote:
> > >  > a workaround: introduce a new no-mmu-only mmap flag MAP_UNINITIALIZE
> > >  > to signal to the kernel that it should skip the memset().  this way,
> > >  > userspace malloc() can do mmap(MAP_ANONYMOUS|MAP_UNINITIALIZE) to get
> > >  > large chunks of memory without affecting any other anonymous mmap()
> > >  > call.
> > >
> > >  I think that's reasonable for NOMMU.  It's not like the process
> accessing the
> > >  uninitialised memory is prevented from accessing anything it wants to
> anyway.
> > >
> > >  I would vote that the memset() should only be skipped if requested as
> there
> > >  may be programs that call mmap(MAP_ANONYMOUS) expecting the memory
> they're
> > >  given to be zeroed out.
> >
> > in the second proposal, the C library would be expected to do this, so
> > no programs would be broken.  but you're right that any program that
> > invokes the mmap() syscall directly would not get zeroed memory ...
> > but is anyone doing such a crazy thing, let alone on embedded ?
>
>  It's not a guarantee we should break.  What's wrong with just using the
> MAP_UNINITIALIZE code we have?

i outlined the benefits of doing it in userspace in my first e-mail.
i also expected MAP_UNINITIALIZE to be unacceptable to LKML.  and
afaik, there doesnt seem to be a way to distinguish in the kernel
whether the call is coming from userspace or kernel space, so the
memset() call will still be called for the kernel.  ideally the code
would read:
if (!kernel && !(flags & MAP_UNINITIALIZE))
    memset(base, 0, len);
-mike

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[Bernd Schmidt]

Mike Frysinger wrote:

> i also expected MAP_UNINITIALIZE to be unacceptable to LKML.  and
> afaik, there doesnt seem to be a way to distinguish in the kernel
> whether the call is coming from userspace or kernel space, so the
> memset() call will still be called for the kernel.  ideally the code
> would read:
> if (!kernel && !(flags & MAP_UNINITIALIZE))
>     memset(base, 0, len);

Have you examined all callers in the kernel whether they can take 
uninitialized memory?  (Your code doesn't match your description btw, so 
I'm only guessing what you actually want to achieve.)

Really, if MAP_UNINITIALIZE is unacceptable, then that's too bad and 
we'll carry it as a local patch, but all the other proposed mechanisms 
are non-solutions because they break an established interface.


Bernd
-- 
This footer brought to you by insane German lawmakers.
Analog Devices GmbH      Wilhelm-Wagenfeld-Str. 6      80807 Muenchen
Sitz der Gesellschaft Muenchen, Registergericht Muenchen HRB 40368
Geschaeftsfuehrer Thomas Wessel, William A. Martin, Margaret Seif

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[David Howells]

Bernd Schmidt <bernds_cb1@t-online.de> wrote:

> Really, if MAP_UNINITIALIZE is unacceptable, then that's too bad and we'll
> carry it as a local patch, but all the other proposed mechanisms are
> non-solutions because they break an established interface.

Agreed.  The kernel would not be backwardly compatible with old userspaces.

David

Re: nommu: handling anonymous mmap clearing in userspace rather than kernel

[Mike Frysinger]

On Thu, Apr 3, 2008 at 7:06 AM, Bernd Schmidt <bernds_cb1@t-online.de> wrote:
> Mike Frysinger wrote:
> > i also expected MAP_UNINITIALIZE to be unacceptable to LKML.  and
> > afaik, there doesnt seem to be a way to distinguish in the kernel
> > whether the call is coming from userspace or kernel space, so the
> > memset() call will still be called for the kernel.  ideally the code
> > would read:
> > if (!kernel && !(flags & MAP_UNINITIALIZE))
> >    memset(base, 0, len);
> >
>
>  Have you examined all callers in the kernel whether they can take
> uninitialized memory?

if they cant, they're broken, and need to get fixed.  i see these
crashes then as a good thing :).

> (Your code doesn't match your description btw, so I'm
> only guessing what you actually want to achieve.)

doesnt it ?  the memset should only be called for userspace when it
hasnt given the uninitialize flag.  if it's kernel space, we want to
skip it.  if the uninitialize flag is given, we want to skip it.

>  Really, if MAP_UNINITIALIZE is unacceptable, then that's too bad and we'll
> carry it as a local patch, but all the other proposed mechanisms are
> non-solutions because they break an established interface.

i'm just airing out the different solutions so people get a better
idea of what's going on.  i dont think the userspace <-> kernelspace
breakage is important, as the ABI is maintained at the important
layer: user application <-> C library.
-mike