LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH 0/4] percpu: Optimize percpu accesses
@ 2008-06-04  0:30 Mike Travis
  2008-06-04  0:30 ` [PATCH 1/4] Zero based percpu: Infrastructure to rebase the per cpu area to zero Mike Travis
                   ` (4 more replies)
  0 siblings, 5 replies; 114+ messages in thread
From: Mike Travis @ 2008-06-04  0:30 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Andrew Morton, Christoph Lameter, David Miller, Eric Dumazet,
	Jeremy Fitzhardinge, linux-kernel


This patchset provides the following:

  * Generic: Percpu infrastructure to rebase the per cpu area to zero

    This provides for the capability of accessing the percpu variables
    using a local register instead of having to go through a table
    on node 0 to find the cpu-specific offsets.  It also would allow
    atomic operations on percpu variables to reduce required locking.
    Uses a new config var HAVE_ZERO_BASED_PER_CPU to indicate to the
    generic code that the arch has this new basing.

  * x86_64: Fold pda into per cpu area

    Declare the pda as a per cpu variable. This will move the pda
    area to an address accessible by the x86_64 per cpu macros.
    Subtraction of __per_cpu_start will make the offset based from
    the beginning of the per cpu area.  Since %gs is pointing to the
    pda, it will then also point to the per cpu variables and can be
    accessed thusly:

	%gs:[&per_cpu_xxxx - __per_cpu_start]

  * x86_64: Rebase per cpu variables to zero

    Take advantage of the zero-based per cpu area provided above.
    Then we can directly use the x86_32 percpu operations. x86_32
    offsets %fs by __per_cpu_start. x86_64 has %gs pointing directly
    to the pda and the per cpu area thereby allowing access to the
    pda with the x86_64 pda operations and access to the per cpu
    variables using x86_32 percpu operations.


Based on linux-2.6.tip

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Mike Travis <travis@sgi.com>
---

-- 

^ permalink raw reply	[flat|nested] 114+ messages in thread
* [PATCH 0/4] x86_64: Optimize percpu accesses
@ 2008-07-25 21:11 Mike Travis
  2008-07-25 21:11 ` [PATCH 3/4] x86_64: Fold pda into per cpu area Mike Travis
  0 siblings, 1 reply; 114+ messages in thread
From: Mike Travis @ 2008-07-25 21:11 UTC (permalink / raw)
  To: Ingo Molnar, Andrew Morton
  Cc: Eric W. Biederman, Hugh Dickins, Jack Steiner,
	Jeremy Fitzhardinge, H. Peter Anvin, linux-kernel


This patchset provides the following:

  * x86_64: Cleanup setup_percpu by fixing some minor potential
    problems as well as add some debugging aids.

  * x86_64: Rebase per cpu variables to zero

    Rebase per cpu variables to zero in preparation for the following
    patch to fold the pda into the per cpu area.

  * x86_64: Fold pda into per cpu area

    Declare the pda as a per cpu variable. This will allow the per cpu
    variables to be accessible on the x86_64 using %gs as the base of
    the percpu areas for each cpu:

	%gs:per_cpu_xxxx

  * x86_64: Reference zero-based percpu variables offset from gs

    Actually implement the above operation for __get_cpu_var() and
    __put_cpu_var().  Since this is now a single instruction, we
    can remove the non-preemptible versions of x86_read_percpu()
    and x86_write_percpu().

Note that the following changes are NOT in this patchset as the plan now
seems to be that the common (to x86) variables that are in the pda should
be made individual per cpu variables, leaving only the stack canary in place.

  * x86_64: Replace cpu_pda ops with percpu ops
  * x86_64: Replace xxx_pda() operations with x86_xxx_percpu().
  * x86_64: Remove xxx_pda() operations
  * x86_64: Remove cpu_pda() macro

Based on linux-2.6.tip/master.

Signed-off-by: Christoph Lameter <cl@linux-foundation.org>
Signed-off-by: Mike Travis <travis@sgi.com>
---

-- 

^ permalink raw reply	[flat|nested] 114+ messages in thread
* [PATCH 0/4] percpu: Optimize percpu accesses
@ 2008-02-01 19:14 travis
  2008-02-01 19:14 ` [PATCH 3/4] x86_64: Fold pda into per cpu area travis
  0 siblings, 1 reply; 114+ messages in thread
From: travis @ 2008-02-01 19:14 UTC (permalink / raw)
  To: Andrew Morton, Andi Kleen, Ingo Molnar, Thomas Gleixner
  Cc: Jeremy Fitzhardinge, Christoph Lameter, Jack Steiner, linux-mm,
	linux-kernel


This patchset provides the following:

  * Generic: Percpu infrastructure to rebase the per cpu area to zero

    This provides for the capability of accessing the percpu variables
    using a local register instead of having to go through a table
    on node 0 to find this cpu specific offsets.  It also would allow
    atomic operations on percpu variables to reduce required locking.

  * Init: Move setup of nr_cpu_ids to as early as possible for usage
    by early boot functions.

  * x86_64: Fold pda into per cpu area

    Declare the pda as a per cpu variable. This will move the pda
    area to an address accessible by the x86_64 per cpu macros.
    Subtraction of __per_cpu_start will make the offset based from
    the beginning of the per cpu area.  Since %gs is pointing to the
    pda, it will then also point to the per cpu variables and can be
    accessed thusly:

	%gs:[&per_cpu_xxxx - __per_cpu_start]

  * x86_64: Rebase per cpu variables to zero

    Take advantage of the zero-based per cpu area provided above.
    Then we can directly use the x86_32 percpu operations. x86_32
    offsets %fs by __per_cpu_start. x86_64 has %gs pointing directly
    to the pda and the per cpu area thereby allowing access to the
    pda with the x86_64 pda operations and access to the per cpu
    variables using x86_32 percpu operations.  After rebasing
    the access now becomes:

	%gs:[&per_cpu_xxxx]

    Introduces a new DEFINE_PER_CPU_FIRST to locate the percpu
    variable (pda in this case) at the beginning of the percpu
    .data section.

  * x86_64: Cleanup non-smp usage of cpu maps

    Cleanup references to the early cpu maps for the non-SMP configuration
    and remove some functions called for SMP configurations only.

Based on linux-2.6.git + x86.git

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Mike Travis <travis@sgi.com>
---
Notes:

(1 - had to disable CONFIG_SIS190 to build)
(2 - no modules)

Configs built and booted:

    x86_64-default
    x86_64-defconfig (2)
    x86_64-nonuma (2)
    x86_64-nosmp (2)
    x86_64-"Ingo Stress Test" (1,2)

Configs built with no errors:

    arm-default
    i386-allyesconfig (1)
    i386-allmodconfig (1)
    i386-defconfig
    i386-nosmp
    ppc-pmac32
    ppc-smp
    sparc64-default
    sparc64-smp
    x86_64-allmodconfig (1)
    x86_64-allyesconfig (1)
    x86_64-maxsmp (NR_CPUS=4k, MAXNODES=512)

Configs with errors prior to patch (preventing full build checkout):

    ia64-sn2: undefined reference to `mem_map' (more)
    ia64-default: (same error)
    ia64-nosmp: `per_cpu__kstat' truncated in .bss (more)
    s390-default: implicit declaration of '__raw_spin_is_contended'
    sparc-default: include/asm/pgtable.h: syntax '___f___swp_entry'

Memory Effects (using x86_64-maxsmp config):

    Note that 1/2MB has been moved from permanent data to
    the init data section, (which is removed after bootup),
    while the per cpu section is only increased by 128 bytes
    per cpu.  Also text size is reduced increasing cache
    performance.

    4k-cpus-before                  4k-cpus-after
       6588928 .data.cacheline_alig     -524288 -7%
	 48072 .data.percpu                +128 +0%
       4804576 .data.read_mostly         -32656 +0%
	854048 .init.data               +557056 +65%
	160382 .init.text                   +62 +0%
       1254214 .rodata                     +274 +0%
       3915552 .text                      -1632 +0%
	 11040 __param                     -272 -2%

       3915552 Text                       -1632 +0%
       1085440 InitData                 +557056 +51%
      11454056 OtherData                -557056 -4%
	 48072 PerCpu                      +128 +0%
      20459748 Total                      -1330 +0%

-- 

^ permalink raw reply	[flat|nested] 114+ messages in thread

end of thread, other threads:[~2008-07-25 21:12 UTC | newest]

Thread overview: 114+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-06-04  0:30 [PATCH 0/4] percpu: Optimize percpu accesses Mike Travis
2008-06-04  0:30 ` [PATCH 1/4] Zero based percpu: Infrastructure to rebase the per cpu area to zero Mike Travis
2008-06-10 10:06   ` Ingo Molnar
2008-06-04  0:30 ` [PATCH 2/4] x86: Extend percpu ops to 64 bit Mike Travis
2008-06-10 10:04   ` Ingo Molnar
2008-06-04  0:30 ` [PATCH 3/4] x86_64: Fold pda into per cpu area Mike Travis
2008-06-04 12:59   ` Jeremy Fitzhardinge
2008-06-04 13:48     ` Mike Travis
2008-06-04 13:58       ` Jeremy Fitzhardinge
2008-06-04 14:17         ` Mike Travis
2008-06-09 23:18     ` Christoph Lameter
2008-06-05 10:22   ` [crash, bisected] " Ingo Molnar
2008-06-05 16:02     ` Mike Travis
2008-06-06  8:29       ` Jeremy Fitzhardinge
2008-06-06 13:15         ` Mike Travis
2008-06-18  5:34           ` Jeremy Fitzhardinge
2008-06-10 21:31     ` Mike Travis
2008-06-18 17:36       ` Jeremy Fitzhardinge
2008-06-18 18:17         ` Mike Travis
2008-06-18 18:33           ` Ingo Molnar
2008-06-18 19:33           ` Jeremy Fitzhardinge
     [not found]             ` <48596893.4040908@sgi.com>
     [not found]               ` <485AADAC.3070301@sgi.com>
     [not found]                 ` <485AB78B.5090904@goop.org>
     [not found]                   ` <485AC120.6010202@sgi.com>
     [not found]                     ` <485AC5D4.6040302@goop.org>
     [not found]                       ` <485ACA8F.10006@sgi.com>
     [not found]                         ` <485ACD92.8050109@sgi.com>
2008-06-19 21:35                           ` Jeremy Fitzhardinge
2008-06-19 21:54                             ` Jeremy Fitzhardinge
2008-06-19 22:13                             ` Mike Travis
2008-06-19 22:21                               ` Jeremy Fitzhardinge
2008-06-30 17:49                                 ` Mike Travis
2008-06-19 22:23                               ` Jeremy Fitzhardinge
     [not found]                                 ` <485BDB04.4090709@sgi.com>
2008-06-20 17:25                                   ` Jeremy Fitzhardinge
2008-06-20 17:48                                     ` Christoph Lameter
2008-06-20 18:30                                       ` Mike Travis
2008-06-20 18:40                                         ` Jeremy Fitzhardinge
2008-06-20 18:37                                       ` Jeremy Fitzhardinge
2008-06-20 18:51                                         ` Christoph Lameter
2008-06-20 19:04                                           ` Jeremy Fitzhardinge
2008-06-20 19:21                                             ` H. Peter Anvin
2008-06-20 19:43                                             ` Eric W. Biederman
2008-06-20 20:04                                               ` Mike Travis
2008-06-20 20:37                                                 ` Christoph Lameter
2008-06-20 19:06                                         ` Mike Travis
2008-06-20 20:25                                           ` Eric W. Biederman
2008-06-20 20:55                                             ` Christoph Lameter
2008-06-23 16:55                                             ` Mike Travis
2008-06-23 17:33                                               ` Jeremy Fitzhardinge
2008-06-23 18:04                                                 ` Mike Travis
2008-06-23 18:36                                                   ` Mike Travis
2008-06-23 19:41                                                     ` Jeremy Fitzhardinge
2008-06-24  0:02                                                       ` Mike Travis
2008-06-30 17:07                                             ` Mike Travis
2008-06-30 17:18                                               ` H. Peter Anvin
2008-06-30 17:57                                                 ` Mike Travis
2008-06-30 20:50                                                   ` Eric W. Biederman
2008-06-30 21:08                                                     ` Jeremy Fitzhardinge
2008-07-01  8:40                                                       ` Eric W. Biederman
2008-07-01 16:27                                                         ` Jeremy Fitzhardinge
2008-07-01 16:55                                                           ` Mike Travis
2008-07-01 16:56                                                         ` H. Peter Anvin
2008-07-01 17:26                                                           ` Jeremy Fitzhardinge
2008-07-01 20:40                                                             ` Eric W. Biederman
2008-07-01 21:10                                                               ` Jeremy Fitzhardinge
2008-07-01 21:39                                                                 ` Eric W. Biederman
2008-07-01 21:52                                                                   ` Jeremy Fitzhardinge
2008-07-02  0:20                                                                     ` H. Peter Anvin
2008-07-02  1:15                                                                       ` Mike Travis
2008-07-02  1:32                                                                         ` Eric W. Biederman
2008-07-02  1:51                                                                           ` Mike Travis
2008-07-02  2:50                                                                             ` Eric W. Biederman
2008-07-02  1:40                                                                         ` H. Peter Anvin
2008-07-02  1:44                                                                         ` Mike Travis
2008-07-02  1:45                                                                           ` H. Peter Anvin
2008-07-02  1:55                                                                             ` Mike Travis
2008-07-02 22:50                                                                             ` Mike Travis
2008-07-03  4:34                                                                               ` Eric W. Biederman
2008-07-07 17:17                                                                                 ` Mike Travis
2008-07-07 19:46                                                                                   ` Eric W. Biederman
2008-07-08 18:21                                                                                     ` Mike Travis
2008-07-08 23:36                                                                                       ` Eric W. Biederman
2008-07-08 23:49                                                                                         ` Jeremy Fitzhardinge
2008-07-09 14:39                                                                                           ` Mike Travis
2008-07-25 20:06                                                                                           ` Mike Travis
2008-07-25 20:12                                                                                             ` Jeremy Fitzhardinge
2008-07-25 20:34                                                                                               ` Mike Travis
2008-07-25 20:43                                                                                                 ` Jeremy Fitzhardinge
2008-07-25 21:05                                                                                                   ` Mike Travis
2008-07-09 14:37                                                                                         ` Mike Travis
2008-07-09 22:38                                                                                           ` Eric W. Biederman
2008-07-09 23:30                                                                                             ` Mike Travis
2008-07-10  0:04                                                                                               ` Eric W. Biederman
2008-07-02  2:01                                                                   ` H. Peter Anvin
2008-07-02  3:08                                                                     ` Eric W. Biederman
2008-07-01 21:11                                                               ` Andi Kleen
2008-07-01 21:42                                                                 ` Eric W. Biederman
2008-07-01 18:41                                                           ` Eric W. Biederman
2008-07-01 12:09                                                       ` Mike Travis
2008-07-01 11:49                                                     ` Mike Travis
2008-06-30 17:43                                               ` Jeremy Fitzhardinge
2008-06-04  0:30 ` [PATCH 4/4] x86: Replace xxx_pda() operations with x86_xx_percpu() Mike Travis
2008-06-09 13:03   ` Ingo Molnar
2008-06-09 16:08     ` Mike Travis
2008-06-09 17:36     ` Mike Travis
2008-06-09 18:20       ` Christoph Lameter
2008-06-09 23:29         ` Jeremy Fitzhardinge
2008-06-10 10:09       ` Ingo Molnar
2008-06-10 15:07         ` Mike Travis
2008-06-04 10:18 ` [PATCH] x86: collapse the various size-dependent percpu accessors together Jeremy Fitzhardinge
2008-06-04 10:45   ` Jeremy Fitzhardinge
2008-06-04 11:29     ` Ingo Molnar
2008-06-04 12:09       ` Jeremy Fitzhardinge
2008-06-10 17:21         ` Christoph Lameter
  -- strict thread matches above, loose matches on Subject: below --
2008-07-25 21:11 [PATCH 0/4] x86_64: Optimize percpu accesses Mike Travis
2008-07-25 21:11 ` [PATCH 3/4] x86_64: Fold pda into per cpu area Mike Travis
2008-02-01 19:14 [PATCH 0/4] percpu: Optimize percpu accesses travis
2008-02-01 19:14 ` [PATCH 3/4] x86_64: Fold pda into per cpu area travis
2008-02-15 20:16   ` Ingo Molnar
2008-02-15 22:43     ` Christoph Lameter
2008-02-17  6:22     ` Yinghai Lu
2008-02-17  7:36       ` Yinghai Lu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).