LKML Archive on
help / color / mirror / Atom feed
From: Rusty Russell <>
To: "Eric W. Biederman" <>
Cc: Andrew Morton <>,
	Ingo Molnar <>, Andi Kleen <>,
	lkml - Kernel Mailing List <>
Subject: Re: [PATCH] Allow per-cpu variables to be page-aligned
Date: Thu, 22 Mar 2007 11:09:12 +1100	[thread overview]
Message-ID: <1174522152.2713.73.camel@localhost.localdomain> (raw)
In-Reply-To: <>

On Wed, 2007-03-21 at 10:49 -0600, Eric W. Biederman wrote:
> Rusty Russell <> writes:
> > On Wed, 2007-03-21 at 03:21 -0600, Eric W. Biederman wrote:
> >> Do we really want to allow modules to be able to allocate page sized
> >> per cpu memory.
> >
> > Hi Eric!
> >
> > 	They always could, of course, they just wouldn't get correct alignment.
> > I think the principle of least surprise says that if we support this, it
> > will also work in modules...
> The module load would fail.

Hi again Eric,

	Unfortunately not.  It probably should, though: people ignore printks.
I was probably thinking that large alignment constraints were only for
performance when I wrote this code, but a page-aligned requirement for
hypervisors changes that.

> > Looking at the module per-cpu code again, the rounding up of the memory
> > used by the kernel seems unnecessary though.  I'll try ripping that
> > out...
> I want to say that when dealing with cpu stuff aligning to a cache
> line makes sense as it prevents multiple variables from sharing
> the same cache line.  However we rarely access per cpu variables from
> other cpus (the point) so the extra alignment doesn't seem to have
> a justification in this context.

Um, yes, always good to remember.  I wrote the per-cpu infrastructure,
and I haven't forgotten 8)

> Although I'm not quite certain what this will do to the per cpu
> memory allocator...

It should Just Work.  My only hesitation is that I obviously thought
different when I wrote this code, so am I smarter now, or then?

> After increasing NR_IRQS on x86_64 to (NR_CPUS*32) the per cpu irq
> stats got much bigger especially as NR_CPUS went up.  The only
> reasonable way I could see to fix this at the time was to just make
> PER_CPU_ENOUGH_ROOM do the right thing and change size dynamically
> with the size of the per cpu section.  I added PERCPU_MODULE_RESERVE
> to allocate the amount that we did not have compile information on.
> 8K was roughly what we had left over for modules before I made the
> change so I just preserved that.

This makes a lot of sense.  A fixed constant seemed sensible at the
time, but now we know that the majority of per-cpu vars are in code
which can never be a module.  Reasons are obvious, and seem unlikely to

> > It means the x86 cpu_pda initialization would have to be done in
> > smp_prepare_boot_cpu tho...
> Well that is earlier than trap_init so it shouldn't be a problem...

But it doesn't get called on UP.  Don't know if that matters, but it
wasn't immediately obvious.


      reply	other threads:[~2007-03-22  0:09 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-03-21  6:10 Rusty Russell
2007-03-21  6:30 ` [PATCH 0/4] i386 GDT cleanups Rusty Russell
2007-03-21  6:32   ` [PATCH 1/4] i386 GDT cleanups: Use per-cpu variables for GDT, PDA Rusty Russell
2007-03-21  6:35     ` [PATCH 2/4] i386 GDT cleanups: Use per-cpu GDT immediately upon boot Rusty Russell
2007-03-21  6:36       ` [PATCH 3/4] i386 GDT cleanups: clean up cpu_init() Rusty Russell
2007-03-21  6:53         ` [PATCH 4/4] i386 GDT cleanups: cleanup GDT Access Rusty Russell
2007-03-21  9:31       ` [PATCH 2/4] i386 GDT cleanups: Use per-cpu GDT immediately upon boot Eric W. Biederman
2007-03-21 11:55         ` Rusty Russell
2007-03-21 16:51           ` Eric W. Biederman
2007-03-21 23:58             ` Rusty Russell
2007-03-22  8:10               ` Sébastien Dugué
2007-03-22  9:24                 ` Rusty Russell
2007-03-22 15:59                   ` [PATCH] i386 GDT cleanups: Rename boot_gdt_table to boot_gdt Sébastien Dugué
2007-03-23  7:19                     ` Rusty Russell
2007-03-23 14:15                     ` Andi Kleen
2007-03-21  9:21 ` [PATCH] Allow per-cpu variables to be page-aligned Eric W. Biederman
2007-03-21 11:44   ` Rusty Russell
2007-03-21 16:49     ` Eric W. Biederman
2007-03-22  0:09       ` Rusty Russell [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1174522152.2713.73.camel@localhost.localdomain \ \ \ \ \ \ \
    --subject='Re: [PATCH] Allow per-cpu variables to be page-aligned' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).