Re: [patch 1/3 take2] smaps: extract pte walker from smaps code

[Andrew Morton <akpm@linux-foundation.org>]

On Wed, 7 Feb 2007 15:29:34 +0900 Paul Mundt <lethal@linux-sh.org> wrote:

> On Tue, Feb 06, 2007 at 10:15:47PM -0800, David Rientjes wrote:
> > Extracts the page table entry walker from the smaps-specific code in
> > fs/proc/task_mmu.c.  This will be used later for clearing the reference
> > bits on pages to measure the number of pages accessed over a time period
> > through /proc/pid/smaps.
> > 
> I like the general idea of this patch set, however..

David didn't really spell out the rationale.  Userspace people ask "how
much memory is my application using".  For system planning and such.  We
don't know, really.  So the approach taken here is to nuke all the
referenced bits and to then monitor them coming back over time.

Obviously it doesn't work very well if the page scanner is doing things,
but one hopes that when someone is instrumenting their application they'll
ensure that sufficient memory is available to prevent this inaccuracy.

The other part of the question is of course pagecache.  Hopefully /proc/pid/io
will suffice for that, probably in combination with /proc/sys/vm/drop_caches.

I don't really have a sense for how much stuff we want to put into the kernel
to support this sort of thing.  The proposed patches are, I think, minimal.
Perhaps it needs more.  If so, opinions are solicited before we go and add
(and hence be forced to maintain) this new interface.

> > Since the PTE walker is now extracted from the smaps code,
> > smaps_pte_func() is invoked for each PTE in the VMA.  Its behavior is
> > identical to the existing implementation, except it is slightly slower
> > because each PTE now invokes a function call.
> > 
> Perhaps this is something that needs to be looked at more closely and
> made more generic? There are many ranged page table walkers that aren't
> so performance critical that the function call cost would cause too much
> pain. ioremap_page_range() comes to mind, and there's bound to be others.
> This would also help people to get the pte map/unmap right, which seems
> to pop up from time to time as well..

That's what Paul Davies' patches are aimed at:

http://marc.theaimsgroup.com/?l=linux-mm&m=115276500100695&w=2

I promised Paul that I'd take a serious look at those patches next time
they pop up.   It would be good if others could help out in that.