LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
@ 2007-08-27 15:52 Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486 Mathieu Desnoyers
                   ` (29 more replies)
  0 siblings, 30 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel

Hi Andrew,

Here is the patch series for 2.6.23-rc3-mm1 that adds cmpxchg_local, and now
also cmpxchg64_local, to each architecture.

When the architecture supports it, it also defines cmpxchg64, but is is not
defined for architecture that does not support atomic 64 bits updates.

Following performance testing of the slub allocator with cmpxchg_local, these
patches should prove themselves useful in a near future.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-28  1:40   ` Nick Piggin
  2007-08-27 15:52 ` [patch 02/28] Add cmpxchg64 and cmpxchg64_local to alpha Mathieu Desnoyers
                   ` (28 subsequent siblings)
  29 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, mingo

[-- Attachment #1: i386-cmpxchg64-80386-80486-fallback.patch --]
[-- Type: text/plain, Size: 7517 bytes --]

Actually, on 386, cmpxchg and cmpxchg_local fall back on
cmpxchg_386_u8/16/32: it disables interruptions around non atomic
updates to mimic the cmpxchg behavior.

The comment:
/* Poor man's cmpxchg for 386. Unsuitable for SMP */

already present in cmpxchg_386_u32 tells much about how this cmpxchg
implementation should not be used in a SMP context. However, the cmpxchg_local
can perfectly use this fallback, since it only needs to be atomic wrt the local
cpu.

This patch adds a cmpxchg_486_u64 and uses it as a fallback for cmpxchg64
and cmpxchg64_local on 80386 and 80486.

Q:
but why is it called cmpxchg_486 when the other functions are called

A:
Because the standard cmpxchg is missing only on 386, but cmpxchg8b is
missing both on 386 and 486.

Citing Intel's Instruction set reference:

cmpxchg:
This instruction is not supported on Intel processors earlier than the
Intel486 processors.

cmpxchg8b:
This instruction encoding is not supported on Intel processors earlier
than the Pentium processors.

Q:
What's the reason to have cmpxchg64_local on 32 bit architectures?
Without that need all this would just be a few simple defines.

A:
cmpxchg64_local on 32 bits architectures takes unsigned long long
parameters, but cmpxchg_local only takes longs. Since we have cmpxchg8b
to execute a 8 byte cmpxchg atomically on pentium and +, it makes sense
to provide a flavor of cmpxchg and cmpxchg_local using this instruction.

Also, for 32 bits architectures lacking the 64 bits atomic cmpxchg, it
makes sense _not_ to define cmpxchg64 while cmpxchg could still be
available.

Moreover, the fallback for cmpxchg8b on i386 for 386 and 486 is a
different case than cmpxchg (which is only required for 386). Using
different code makes this easier.

However, cmpxchg64_local will be emulated by disabling interrupts on all
architectures where it is not supported atomically.

Therefore, we *could* turn cmpxchg64_local into a cmpxchg_local, but it
would make the 386/486 fallbacks ugly, make its design different from
cmpxchg/cmpxchg64 (which really depends on atomic operations and cannot
be emulated) and require the __cmpxchg_local to be expressed as a macro
rather than an inline function so the parameters would not be fixed to
unsigned long long in every case.

So I think cmpxchg64_local makes sense there, but I am open to
suggestions.


Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: mingo@redhat.com
---
 arch/i386/kernel/cpu/intel.c |   17 +++++++
 include/asm-i386/cmpxchg.h   |  100 +++++++++++++++++++++++++++++--------------
 2 files changed, 85 insertions(+), 32 deletions(-)

Index: linux-2.6-lttng/arch/i386/kernel/cpu/intel.c
===================================================================
--- linux-2.6-lttng.orig/arch/i386/kernel/cpu/intel.c	2007-08-07 11:00:43.000000000 -0400
+++ linux-2.6-lttng/arch/i386/kernel/cpu/intel.c	2007-08-07 11:11:07.000000000 -0400
@@ -329,5 +329,22 @@ unsigned long cmpxchg_386_u32(volatile v
 EXPORT_SYMBOL(cmpxchg_386_u32);
 #endif
 
+#ifndef CONFIG_X86_CMPXCHG64
+unsigned long long cmpxchg_486_u64(volatile void *ptr, u64 old, u64 new)
+{
+	u64 prev;
+	unsigned long flags;
+
+	/* Poor man's cmpxchg8b for 386 and 486. Unsuitable for SMP */
+	local_irq_save(flags);
+	prev = *(u64 *)ptr;
+	if (prev == old)
+		*(u64 *)ptr = new;
+	local_irq_restore(flags);
+	return prev;
+}
+EXPORT_SYMBOL(cmpxchg_486_u64);
+#endif
+
 // arch_initcall(intel_cpu_init);
 
Index: linux-2.6-lttng/include/asm-i386/cmpxchg.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-i386/cmpxchg.h	2007-08-07 11:01:17.000000000 -0400
+++ linux-2.6-lttng/include/asm-i386/cmpxchg.h	2007-08-07 11:16:42.000000000 -0400
@@ -116,6 +116,15 @@ static inline unsigned long __xchg(unsig
 					(unsigned long)(n),sizeof(*(ptr))))
 #endif
 
+#ifdef CONFIG_X86_CMPXCHG64
+#define cmpxchg64(ptr,o,n)\
+	((__typeof__(*(ptr)))__cmpxchg64((ptr),(unsigned long long)(o),\
+					(unsigned long long)(n)))
+#define cmpxchg64_local(ptr,o,n)\
+	((__typeof__(*(ptr)))__cmpxchg64_local((ptr),(unsigned long long)(o),\
+					(unsigned long long)(n)))
+#endif
+
 static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
 				      unsigned long new, int size)
 {
@@ -203,6 +212,34 @@ static inline unsigned long __cmpxchg_lo
 	return old;
 }
 
+static inline unsigned long long __cmpxchg64(volatile void *ptr,
+			unsigned long long old, unsigned long long new)
+{
+	unsigned long long prev;
+	__asm__ __volatile__(LOCK_PREFIX "cmpxchg8b %3"
+			     : "=A"(prev)
+			     : "b"((unsigned long)new),
+			       "c"((unsigned long)(new >> 32)),
+			       "m"(*__xg(ptr)),
+			       "0"(old)
+			     : "memory");
+	return prev;
+}
+
+static inline unsigned long long __cmpxchg64_local(volatile void *ptr,
+			unsigned long long old, unsigned long long new)
+{
+	unsigned long long prev;
+	__asm__ __volatile__("cmpxchg8b %3"
+			     : "=A"(prev)
+			     : "b"((unsigned long)new),
+			       "c"((unsigned long)(new >> 32)),
+			       "m"(*__xg(ptr)),
+			       "0"(old)
+			     : "memory");
+	return prev;
+}
+
 #ifndef CONFIG_X86_CMPXCHG
 /*
  * Building a kernel capable running on 80386. It may be necessary to
@@ -252,38 +289,37 @@ static inline unsigned long cmpxchg_386(
 })
 #endif
 
-static inline unsigned long long __cmpxchg64(volatile void *ptr, unsigned long long old,
-				      unsigned long long new)
-{
-	unsigned long long prev;
-	__asm__ __volatile__(LOCK_PREFIX "cmpxchg8b %3"
-			     : "=A"(prev)
-			     : "b"((unsigned long)new),
-			       "c"((unsigned long)(new >> 32)),
-			       "m"(*__xg(ptr)),
-			       "0"(old)
-			     : "memory");
-	return prev;
-}
+#ifndef CONFIG_X86_CMPXCHG64
+/*
+ * Building a kernel capable running on 80386 and 80486. It may be necessary
+ * to simulate the cmpxchg8b on the 80386 and 80486 CPU.
+ */
+
+extern unsigned long long cmpxchg_486_u64(volatile void *, u64, u64);
+
+#define cmpxchg64(ptr,o,n)						\
+({									\
+	__typeof__(*(ptr)) __ret;					\
+	if (likely(boot_cpu_data.x86 > 4))				\
+		__ret = __cmpxchg64((ptr), (unsigned long long)(o),	\
+				(unsigned long long)(n));		\
+	else								\
+		__ret = cmpxchg_486_u64((ptr), (unsigned long long)(o),	\
+				(unsigned long long)(n));		\
+	__ret;								\
+})
+#define cmpxchg64_local(ptr,o,n)					\
+({									\
+	__typeof__(*(ptr)) __ret;					\
+	if (likely(boot_cpu_data.x86 > 4))				\
+		__ret = __cmpxchg64_local((ptr), (unsigned long long)(o), \
+				(unsigned long long)(n));		\
+	else								\
+		__ret = cmpxchg_486_u64((ptr), (unsigned long long)(o),	\
+				(unsigned long long)(n));		\
+	__ret;								\
+})
 
-static inline unsigned long long __cmpxchg64_local(volatile void *ptr,
-			unsigned long long old, unsigned long long new)
-{
-	unsigned long long prev;
-	__asm__ __volatile__("cmpxchg8b %3"
-			     : "=A"(prev)
-			     : "b"((unsigned long)new),
-			       "c"((unsigned long)(new >> 32)),
-			       "m"(*__xg(ptr)),
-			       "0"(old)
-			     : "memory");
-	return prev;
-}
+#endif
 
-#define cmpxchg64(ptr,o,n)\
-	((__typeof__(*(ptr)))__cmpxchg64((ptr),(unsigned long long)(o),\
-					(unsigned long long)(n)))
-#define cmpxchg64_local(ptr,o,n)\
-	((__typeof__(*(ptr)))__cmpxchg64_local((ptr),(unsigned long long)(o),\
-					(unsigned long long)(n)))
 #endif

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 02/28] Add cmpxchg64 and cmpxchg64_local to alpha
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips Mathieu Desnoyers
                   ` (27 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, rth, ink

[-- Attachment #1: add-cmpxchg64-to-alpha.patch --]
[-- Type: text/plain, Size: 1315 bytes --]

Make sure that at least cmpxchg64_local is available on all architectures to use
for unsigned long long values.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: rth@twiddle.net
CC: ink@jurassic.park.msu.ru
---
 include/asm-alpha/system.h |    2 ++
 1 file changed, 2 insertions(+)

Index: linux-2.6-lttng/include/asm-alpha/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-alpha/system.h	2007-08-27 11:23:08.000000000 -0400
+++ linux-2.6-lttng/include/asm-alpha/system.h	2007-08-27 11:23:46.000000000 -0400
@@ -687,6 +687,7 @@ __cmpxchg(volatile void *ptr, unsigned l
      (__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,		 \
 				    (unsigned long)_n_, sizeof(*(ptr))); \
   })
+#define cmpxchg64	cmpxchg
 
 static inline unsigned long
 __cmpxchg_u8_local(volatile char *m, long old, long new)
@@ -809,6 +810,7 @@ __cmpxchg_local(volatile void *ptr, unsi
      (__typeof__(*(ptr))) __cmpxchg_local((ptr), (unsigned long)_o_,	 \
 				    (unsigned long)_n_, sizeof(*(ptr))); \
   })
+#define cmpxchg64_local	cmpxchg_local
 
 #endif /* __ASSEMBLY__ */
 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486 Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 02/28] Add cmpxchg64 and cmpxchg64_local to alpha Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-28 13:58   ` Ralf Baechle
  2007-08-27 15:52 ` [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc Mathieu Desnoyers
                   ` (26 subsequent siblings)
  29 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, ralf

[-- Attachment #1: add-cmpxchg64-to-mips.patch --]
[-- Type: text/plain, Size: 1887 bytes --]

Make sure that at least cmpxchg64_local is available on all architectures to use
for unsigned long long values.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: ralf@linux-mips.org
CC linux-mips@linux-mips.org
---
 include/asm-mips/system.h |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Index: linux-2.6-lttng/include/asm-mips/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-mips/system.h	2007-08-27 11:28:22.000000000 -0400
+++ linux-2.6-lttng/include/asm-mips/system.h	2007-08-27 11:38:09.000000000 -0400
@@ -407,13 +407,22 @@ static inline unsigned long __cmpxchg_u6
 	return retval;
 }
 
+#define cmpxchg64	cmpxchg
+#define cmpxchg64_local	cmpxchg_local
+
 #else
+
+#include <asm-generic/cmpxchg-local.h>
+
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
 extern unsigned long __cmpxchg_u64_unsupported_on_32bit_kernels(
 	volatile int * m, unsigned long old, unsigned long new);
 #define __cmpxchg_u64 __cmpxchg_u64_unsupported_on_32bit_kernels
 extern unsigned long __cmpxchg_u64_local_unsupported_on_32bit_kernels(
 	volatile int * m, unsigned long old, unsigned long new);
 #define __cmpxchg_u64_local __cmpxchg_u64_local_unsupported_on_32bit_kernels
+
 #endif
 
 /* This function doesn't exist, so you'll get a linker error
@@ -449,7 +458,6 @@ static inline unsigned long __cmpxchg_lo
 #define cmpxchg(ptr,old,new) \
 	((__typeof__(*(ptr)))__cmpxchg((ptr), \
 		(unsigned long)(old), (unsigned long)(new),sizeof(*(ptr))))
-
 #define cmpxchg_local(ptr,old,new) \
 	((__typeof__(*(ptr)))__cmpxchg_local((ptr), \
 		(unsigned long)(old), (unsigned long)(new),sizeof(*(ptr))))

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (2 preceding siblings ...)
  2007-08-27 15:52 ` [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-09-22  4:46   ` Paul Mackerras
  2007-08-27 15:52 ` [patch 05/28] Add cmpxchg64 and cmpxchg64_local to x86_64 Mathieu Desnoyers
                   ` (25 subsequent siblings)
  29 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, paulus, linuxppc-dev

[-- Attachment #1: add-cmpxchg64-to-powerpc.patch --]
[-- Type: text/plain, Size: 1132 bytes --]

Make sure that at least cmpxchg64_local is available on all architectures to use
for unsigned long long values.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: paulus@samba.org
CC: linuxppc-dev@ozlabs.org
---
 include/asm-powerpc/system.h |    6 ++++++
 1 file changed, 6 insertions(+)

Index: linux-2.6-lttng/include/asm-powerpc/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-powerpc/system.h	2007-08-27 11:42:08.000000000 -0400
+++ linux-2.6-lttng/include/asm-powerpc/system.h	2007-08-27 11:42:43.000000000 -0400
@@ -485,6 +485,12 @@ __cmpxchg_local(volatile void *ptr, unsi
  */
 #define NET_IP_ALIGN	0
 #define NET_SKB_PAD	L1_CACHE_BYTES
+
+#define cmpxchg64	cmpxchg
+#define cmpxchg64_local	cmpxchg_local
+#else
+#include <asm-generic/cmpxchg-local.h>
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 #endif
 
 #define arch_align_stack(x) (x)

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 05/28] Add cmpxchg64 and cmpxchg64_local to x86_64
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (3 preceding siblings ...)
  2007-08-27 15:52 ` [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 06/28] Add cmpxchg_local to asm-generic for per cpu atomic operations Mathieu Desnoyers
                   ` (24 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, Andi Kleen

[-- Attachment #1: add-cmpxchg64-to-x86_64.patch --]
[-- Type: text/plain, Size: 1152 bytes --]

Make sure that at least cmpxchg64_local is available on all architectures to use
for unsigned long long values.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Andi Kleen <ak@muc.de>
---
 include/asm-x86_64/cmpxchg.h |    2 ++
 1 file changed, 2 insertions(+)

Index: linux-2.6-lttng/include/asm-x86_64/cmpxchg.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-x86_64/cmpxchg.h	2007-08-27 11:16:46.000000000 -0400
+++ linux-2.6-lttng/include/asm-x86_64/cmpxchg.h	2007-08-27 11:17:21.000000000 -0400
@@ -127,8 +127,10 @@ static inline unsigned long __cmpxchg_lo
 #define cmpxchg(ptr,o,n)\
 	((__typeof__(*(ptr)))__cmpxchg((ptr),(unsigned long)(o),\
 					(unsigned long)(n),sizeof(*(ptr))))
+#define cmpxchg64	cmpxchg
 #define cmpxchg_local(ptr,o,n)\
 	((__typeof__(*(ptr)))__cmpxchg_local((ptr),(unsigned long)(o),\
 					(unsigned long)(n),sizeof(*(ptr))))
+#define cmpxchg64_local	cmpxchg_local
 
 #endif

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 06/28] Add cmpxchg_local to asm-generic for per cpu atomic operations
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (4 preceding siblings ...)
  2007-08-27 15:52 ` [patch 05/28] Add cmpxchg64 and cmpxchg64_local to x86_64 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 07/28] Add cmpxchg_local to arm Mathieu Desnoyers
                   ` (23 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter

[-- Attachment #1: add-cmpxchg-local-to-generic-for-up.patch --]
[-- Type: text/plain, Size: 3666 bytes --]

Emulates the cmpxchg_local by disabling interrupts around variable modification.
This is not reentrant wrt NMIs and MCEs. It is only protected against normal
interrupts, but this is enough for architectures without such interrupt sources
or if used in a context where the data is not shared with such handlers.

It can be used as a fallback for architectures lacking a real cmpxchg
instruction.

For architectures that have a real cmpxchg but does not have NMIs or MCE,
testing which of the generic vs architecture specific cmpxchg is the fastest
should be done.

asm-generic/cmpxchg.h defines a cmpxchg that uses cmpxchg_local. It is meant to
be used as a cmpxchg fallback for architectures that do not support SMP.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
---
 include/asm-generic/cmpxchg-local.h |   60 ++++++++++++++++++++++++++++++++++++
 include/asm-generic/cmpxchg.h       |   22 +++++++++++++
 2 files changed, 82 insertions(+)

Index: linux-2.6-lttng/include/asm-generic/cmpxchg.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-generic/cmpxchg.h	2007-08-22 10:45:12.000000000 -0400
@@ -0,0 +1,22 @@
+#ifndef __ASM_GENERIC_CMPXCHG_H
+#define __ASM_GENERIC_CMPXCHG_H
+
+/*
+ * Generic cmpxchg
+ *
+ * Uses the local cmpxchg. Does not support SMP.
+ */
+#ifdef CONFIG_SMP
+#error "Cannot use generic cmpxchg on SMP"
+#endif
+
+/*
+ * Atomic compare and exchange.
+ *
+ * Do not define __HAVE_ARCH_CMPXCHG because we want to use it to check whether
+ * a cmpxchg primitive faster than repeated local irq save/restore exists.
+ */
+#define cmpxchg(ptr,o,n)	cmpxchg_local((ptr), (o), (n))
+#define cmpxchg64(ptr,o,n)	cmpxchg64_local((ptr), (o), (n))
+
+#endif
Index: linux-2.6-lttng/include/asm-generic/cmpxchg-local.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-generic/cmpxchg-local.h	2007-08-20 17:54:12.000000000 -0400
@@ -0,0 +1,60 @@
+#ifndef __ASM_GENERIC_CMPXCHG_LOCAL_H
+#define __ASM_GENERIC_CMPXCHG_LOCAL_H
+
+#include <linux/types.h>
+
+extern unsigned long wrong_size_cmpxchg(volatile void *ptr);
+
+/*
+ * Generic version of __cmpxchg_local (disables interrupts). Takes an unsigned
+ * long parameter, supporting various types of architectures.
+ */
+static inline unsigned long __cmpxchg_local_generic(volatile void *ptr,
+				    unsigned long old,
+				    unsigned long new, int size)
+{
+	unsigned long flags, prev;
+
+	/*
+	 * Sanity checking, compile-time.
+	 */
+	if (size == 8 && sizeof(unsigned long) != 8)
+		wrong_size_cmpxchg(ptr);
+
+	local_irq_save(flags);
+	switch (size) {
+	case 1: if ((prev = *(u8*)ptr) == old)
+			*(u8*)ptr = (u8)new;
+		break;
+	case 2: if ((prev = *(u16*)ptr) == old)
+			*(u16*)ptr = (u16)new;
+		break;
+	case 4: if ((prev = *(u32*)ptr) == old)
+			*(u32*)ptr = (u32)new;
+		break;
+	case 8: if ((prev = *(u64*)ptr) == old)
+			*(u64*)ptr = (u64)new;
+		break;
+	default:
+		wrong_size_cmpxchg(ptr);
+	}
+	local_irq_restore(flags);
+	return prev;
+}
+
+/*
+ * Generic version of __cmpxchg64_local. Takes an u64 parameter.
+ */
+static inline u64 __cmpxchg64_local_generic(volatile void *ptr, u64 old, u64 new)
+{
+	u64 prev;
+	unsigned long flags;
+
+	local_irq_save(flags);
+	if ((prev = *(u64*)ptr) == old)
+		*(u64*)ptr = new;
+	local_irq_restore(flags);
+	return prev;
+}
+
+#endif

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 07/28] Add cmpxchg_local to arm
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (5 preceding siblings ...)
  2007-08-27 15:52 ` [patch 06/28] Add cmpxchg_local to asm-generic for per cpu atomic operations Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 08/28] Add cmpxchg_local to avr32 Mathieu Desnoyers
                   ` (22 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, rmk

[-- Attachment #1: add-cmpxchg-local-to-arm.patch --]
[-- Type: text/plain, Size: 2203 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: rmk@arm.linux.org.uk
---
 include/asm-arm/system.h |   41 ++++++++++-------------------------------
 1 file changed, 10 insertions(+), 31 deletions(-)

Index: linux-2.6-lttng/include/asm-arm/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-arm/system.h	2007-08-27 11:16:14.000000000 -0400
+++ linux-2.6-lttng/include/asm-arm/system.h	2007-08-27 11:31:59.000000000 -0400
@@ -350,40 +350,19 @@ static inline unsigned long __xchg(unsig
 extern void disable_hlt(void);
 extern void enable_hlt(void);
 
-#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg-local.h>
+
 /*
- * Atomic compare and exchange.
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
  */
-#define __HAVE_ARCH_CMPXCHG	1
-
-extern unsigned long wrong_size_cmpxchg(volatile void *ptr);
-
-static inline unsigned long __cmpxchg(volatile void *ptr,
-				    unsigned long old,
-				    unsigned long new, int size)
-{
-	unsigned long flags, prev;
-	volatile unsigned long *p = ptr;
-
-	if (size == 4) {
-		local_irq_save(flags);
-		prev = *p;
-		if (prev == old)
-			*p = new;
-		local_irq_restore(flags);
-		return prev;
-	} else
-		return wrong_size_cmpxchg(ptr);
-}
-
-#define cmpxchg(ptr, o, n)					  	\
-({									\
-     __typeof__(*(ptr)) _o_ = (o);					\
-     __typeof__(*(ptr)) _n_ = (n);					\
-     (__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,		\
-		(unsigned long)_n_, sizeof(*(ptr)));	\
-})
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
 #endif
 
 #endif /* __ASSEMBLY__ */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 08/28] Add cmpxchg_local to avr32
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (6 preceding siblings ...)
  2007-08-27 15:52 ` [patch 07/28] Add cmpxchg_local to arm Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 09/28] Add cmpxchg_local to blackfin, replace __cmpxchg by generic cmpxchg Mathieu Desnoyers
                   ` (21 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, Haavard Skinnemoen, clameter

[-- Attachment #1: add-cmpxchg-local-to-avr32.patch --]
[-- Type: text/plain, Size: 1861 bytes --]

Use the new generic cmpxchg_local (disables interrupt) for 8, 16 and 64 bits
cmpxchg_local. Use the __cmpxchg_u32 primitive for 32 bits cmpxchg_local.

Note that cmpxchg only uses the __cmpxchg_u32 or __cmpxchg_u64 and will cause
a linker error if called with 8 or 16 bits argument.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Acked-by: Haavard Skinnemoen <hskinnemoen@atmel.com>
CC: clameter@sgi.com
CC: hskinnemoen@atmel.com
---
 include/asm-avr32/system.h |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

Index: linux-2.6-lttng/include/asm-avr32/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-avr32/system.h	2007-07-20 19:10:57.000000000 -0400
+++ linux-2.6-lttng/include/asm-avr32/system.h	2007-07-20 19:32:36.000000000 -0400
@@ -140,6 +140,29 @@ static inline unsigned long __cmpxchg(vo
 				   (unsigned long)(new),	\
 				   sizeof(*(ptr))))
 
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+		return __cmpxchg_u32(ptr, old, new);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+#define cmpxchg_local(ptr, old, new)					\
+	((typeof(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(old),	\
+				   (unsigned long)(new),		\
+				   sizeof(*(ptr))))
+
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
 struct pt_regs;
 void NORET_TYPE die(const char *str, struct pt_regs *regs, long err);
 void _exception(long signr, struct pt_regs *regs, int code,

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 09/28] Add cmpxchg_local to blackfin, replace __cmpxchg by generic cmpxchg
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (7 preceding siblings ...)
  2007-08-27 15:52 ` [patch 08/28] Add cmpxchg_local to avr32 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 10/28] Add cmpxchg_local to cris Mathieu Desnoyers
                   ` (20 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, michael.frysinger

[-- Attachment #1: add-cmpxchg-local-to-blackfin.patch --]
[-- Type: text/plain, Size: 2856 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set since nobody seems to know why __cmpxchg
has been implemented in assembly in the first place thather than in plain C.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: michael.frysinger@analog.com
---
 include/asm-blackfin/system.h |   59 ++++++++----------------------------------
 1 file changed, 12 insertions(+), 47 deletions(-)

Index: linux-2.6-lttng/include/asm-blackfin/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-blackfin/system.h	2007-08-07 14:31:52.000000000 -0400
+++ linux-2.6-lttng/include/asm-blackfin/system.h	2007-08-07 14:47:31.000000000 -0400
@@ -176,55 +176,20 @@ static inline unsigned long __xchg(unsig
 	return tmp;
 }
 
+#include <asm-generic/cmpxchg-local.h>
+
 /*
- * Atomic compare and exchange.  Compare OLD with MEM, if identical,
- * store NEW in MEM.  Return the initial value in MEM.  Success is
- * indicated by comparing RETURN with OLD.
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
  */
-static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
-				      unsigned long new, int size)
-{
-	unsigned long tmp = 0;
-	unsigned long flags = 0;
-
-	local_irq_save(flags);
-
-	switch (size) {
-	case 1:
-		__asm__ __volatile__
-			("%0 = b%3 (z);\n\t"
-			 "CC = %1 == %0;\n\t"
-			 "IF !CC JUMP 1f;\n\t"
-			 "b%3 = %2;\n\t"
-			 "1:\n\t"
-			 : "=&d" (tmp) : "d" (old), "d" (new), "m" (*__xg(ptr)) : "memory");
-		break;
-	case 2:
-		__asm__ __volatile__
-			("%0 = w%3 (z);\n\t"
-			 "CC = %1 == %0;\n\t"
-			 "IF !CC JUMP 1f;\n\t"
-			 "w%3 = %2;\n\t"
-			 "1:\n\t"
-			 : "=&d" (tmp) : "d" (old), "d" (new), "m" (*__xg(ptr)) : "memory");
-		break;
-	case 4:
-		__asm__ __volatile__
-			("%0 = %3;\n\t"
-			 "CC = %1 == %0;\n\t"
-			 "IF !CC JUMP 1f;\n\t"
-			 "%3 = %2;\n\t"
-			 "1:\n\t"
-			 : "=&d" (tmp) : "d" (old), "d" (new), "m" (*__xg(ptr)) : "memory");
-		break;
-	}
-	local_irq_restore(flags);
-	return tmp;
-}
-
-#define cmpxchg(ptr,o,n)\
-        ((__typeof__(*(ptr)))__cmpxchg((ptr),(unsigned long)(o),\
-                                        (unsigned long)(n),sizeof(*(ptr))))
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
 
 #define prepare_to_switch()     do { } while(0)
 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 10/28] Add cmpxchg_local to cris
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (8 preceding siblings ...)
  2007-08-27 15:52 ` [patch 09/28] Add cmpxchg_local to blackfin, replace __cmpxchg by generic cmpxchg Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 11/28] Add cmpxchg_local to frv Mathieu Desnoyers
                   ` (19 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, starvik

[-- Attachment #1: add-cmpxchg-local-to-cris.patch --]
[-- Type: text/plain, Size: 1365 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: starvik@axis.com
---
 include/asm-cris/system.h |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

Index: linux-2.6-lttng/include/asm-cris/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-cris/system.h	2007-08-07 14:31:52.000000000 -0400
+++ linux-2.6-lttng/include/asm-cris/system.h	2007-08-07 14:48:53.000000000 -0400
@@ -66,6 +66,21 @@ static inline unsigned long __xchg(unsig
   return x;
 }
 
+#include <asm-generic/cmpxchg-local.h>
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
+
 #define arch_align_stack(x) (x)
 
 void default_idle(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 11/28] Add cmpxchg_local to frv
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (9 preceding siblings ...)
  2007-08-27 15:52 ` [patch 10/28] Add cmpxchg_local to cris Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 12/28] Add cmpxchg_local to h8300 Mathieu Desnoyers
                   ` (18 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter

[-- Attachment #1: add-cmpxchg-local-to-frv.patch --]
[-- Type: text/plain, Size: 1577 bytes --]

Use the new generic cmpxchg_local (disables interrupt) for 8, 16 and 64 bits
arguments. Use the 32 bits cmpxchg available on the architecture for 32 bits
arguments.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
---
 include/asm-frv/system.h |   24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

Index: linux-2.6-lttng/include/asm-frv/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-frv/system.h	2007-08-07 14:51:22.000000000 -0400
+++ linux-2.6-lttng/include/asm-frv/system.h	2007-08-07 14:51:39.000000000 -0400
@@ -265,5 +265,29 @@ extern uint32_t __cmpxchg_32(uint32_t *v
 
 #endif
 
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+		return cmpxchg(ptr, old, new);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	\
+     (__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	\
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
 #endif /* _ASM_SYSTEM_H */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 12/28] Add cmpxchg_local to h8300
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (10 preceding siblings ...)
  2007-08-27 15:52 ` [patch 11/28] Add cmpxchg_local to frv Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64 Mathieu Desnoyers
                   ` (17 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter

[-- Attachment #1: add-cmpxchg-local-to-h8300.patch --]
[-- Type: text/plain, Size: 1371 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
---
 include/asm-h8300/system.h |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

Index: linux-2.6-lttng/include/asm-h8300/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-h8300/system.h	2007-07-20 18:36:09.000000000 -0400
+++ linux-2.6-lttng/include/asm-h8300/system.h	2007-07-20 19:27:23.000000000 -0400
@@ -139,6 +139,21 @@ static inline unsigned long __xchg(unsig
         asm("jmp @@0");			\
 })
 
+#include <asm-generic/cmpxchg-local.h>
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
+
 #define arch_align_stack(x) (x)
 
 #endif /* _H8300_SYSTEM_H */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (11 preceding siblings ...)
  2007-08-27 15:52 ` [patch 12/28] Add cmpxchg_local to h8300 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 19:32   ` Christoph Lameter
  2007-08-27 15:52 ` [patch 14/28] New cmpxchg_local (optimized for UP case) for m32r Mathieu Desnoyers
                   ` (16 subsequent siblings)
  29 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, tony.luck, Keith Owens

[-- Attachment #1: add-cmpxchg-local-to-ia64.patch --]
[-- Type: text/plain, Size: 1567 bytes --]

Add the primitives cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64. They
use cmpxchg_acq as underlying macro, just like the already existing ia64
cmpxchg().

Changelog:

ia64 cmpxchg_local coding style fix
Quoting Keith Owens:

As a matter of coding style, I prefer

#define cmpxchg_local   cmpxchg
#define cmpxchg64_local cmpxchg64

Which makes it absolutely clear that they are the same code.  With your
patch, humans have to do a string compare of two defines to see if they
are the same.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: tony.luck@intel.com
CC: Keith Owens <kaos@ocs.com.au>
---
 include/asm-ia64/intrinsics.h |    4 ++++
 1 file changed, 4 insertions(+)

Index: linux-2.6-lttng/include/asm-ia64/intrinsics.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-ia64/intrinsics.h	2007-08-12 09:33:57.000000000 -0400
+++ linux-2.6-lttng/include/asm-ia64/intrinsics.h	2007-08-12 15:09:49.000000000 -0400
@@ -158,6 +158,10 @@ extern long ia64_cmpxchg_called_with_bad
 
 /* for compatibility with other platforms: */
 #define cmpxchg(ptr,o,n)	cmpxchg_acq(ptr,o,n)
+#define cmpxchg64(ptr,o,n)	cmpxchg_acq(ptr,o,n)
+
+#define cmpxchg_local		cmpxchg
+#define cmpxchg64_local		cmpxchg64
 
 #ifdef CONFIG_IA64_DEBUG_CMPXCHG
 # define CMPXCHG_BUGCHECK_DECL	int _cmpxchg_bugcheck_count = 128;

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 14/28] New cmpxchg_local (optimized for UP case) for m32r
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (12 preceding siblings ...)
  2007-08-27 15:52 ` [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 15/28] Fix m32r __xchg Mathieu Desnoyers
                   ` (15 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel
  Cc: Mathieu Desnoyers, Hirokazu Takata, clameter, linux-m32r

[-- Attachment #1: add-cmpxchg-local-to-m32r.patch --]
[-- Type: text/plain, Size: 4367 bytes --]

Add __xchg_local, xchg_local (define), __cmpxchg_local_u32, __cmpxchg_local,
cmpxchg_local(macro).

cmpxchg_local and cmpxchg64_local will use the architecture specific
__cmpxchg_local_u32 for 32 bits arguments, and use the generic
__cmpxchg_local_generic for 8, 16 and 64 bits arguments.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Acked-by: Hirokazu Takata <takata@linux-m32r.org>
CC: clameter@sgi.com
CC: linux-m32r@ml.linux-m32r.org
---
 include/asm-m32r/system.h |  103 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 97 insertions(+), 6 deletions(-)

Index: linux-2.6-lttng/include/asm-m32r/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-m32r/system.h	2007-08-07 14:31:52.000000000 -0400
+++ linux-2.6-lttng/include/asm-m32r/system.h	2007-08-07 14:55:02.000000000 -0400
@@ -123,6 +123,9 @@ static inline void local_irq_disable(voi
 
 #define xchg(ptr,x) \
 	((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))
+#define xchg_local(ptr,x) \
+	((__typeof__(*(ptr)))__xchg_local((unsigned long)(x),(ptr), \
+			sizeof(*(ptr))))
 
 #ifdef CONFIG_SMP
 extern void  __xchg_called_with_bad_pointer(void);
@@ -196,6 +199,42 @@ __xchg(unsigned long x, volatile void * 
 	return (tmp);
 }
 
+static __always_inline unsigned long
+__xchg_local(unsigned long x, volatile void * ptr, int size)
+{
+	unsigned long flags;
+	unsigned long tmp = 0;
+
+	local_irq_save(flags);
+
+	switch (size) {
+	case 1:
+		__asm__ __volatile__ (
+			"ldb	%0, @%2 \n\t"
+			"stb	%1, @%2 \n\t"
+			: "=&r" (tmp) : "r" (x), "r" (ptr) : "memory");
+		break;
+	case 2:
+		__asm__ __volatile__ (
+			"ldh	%0, @%2 \n\t"
+			"sth	%1, @%2 \n\t"
+			: "=&r" (tmp) : "r" (x), "r" (ptr) : "memory");
+		break;
+	case 4:
+		__asm__ __volatile__ (
+			"ld	%0, @%2 \n\t"
+			"st	%1, @%2 \n\t"
+			: "=&r" (tmp) : "r" (x), "r" (ptr) : "memory");
+		break;
+	default:
+		__xchg_called_with_bad_pointer();
+	}
+
+	local_irq_restore(flags);
+
+	return (tmp);
+}
+
 #define __HAVE_ARCH_CMPXCHG	1
 
 static inline unsigned long
@@ -228,6 +267,37 @@ __cmpxchg_u32(volatile unsigned int *p, 
 	return retval;
 }
 
+static inline unsigned long
+__cmpxchg_local_u32(volatile unsigned int *p, unsigned int old,
+			unsigned int new)
+{
+	unsigned long flags;
+	unsigned int retval;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+			DCACHE_CLEAR("%0", "r4", "%1")
+			"ld %0, @%1;		\n"
+		"	bne	%0, %2, 1f;	\n"
+			"st %3, @%1;		\n"
+		"	bra	2f;		\n"
+                "       .fillinsn		\n"
+		"1:"
+			"st %0, @%1;		\n"
+                "       .fillinsn		\n"
+		"2:"
+			: "=&r" (retval)
+			: "r" (p), "r" (old), "r" (new)
+			: "cbit", "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+			, "r4"
+#endif  /* CONFIG_CHIP_M32700_TS1 */
+		);
+	local_irq_restore(flags);
+
+	return retval;
+}
+
 /* This function doesn't exist, so you'll get a linker error
    if something tries to do an invalid cmpxchg().  */
 extern void __cmpxchg_called_with_bad_pointer(void);
@@ -248,12 +318,33 @@ __cmpxchg(volatile void *ptr, unsigned l
 }
 
 #define cmpxchg(ptr,o,n)						 \
-  ({									 \
-     __typeof__(*(ptr)) _o_ = (o);					 \
-     __typeof__(*(ptr)) _n_ = (n);					 \
-     (__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,		 \
-				    (unsigned long)_n_, sizeof(*(ptr))); \
-  })
+     (__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)(o),		 \
+				    (unsigned long)(n), sizeof(*(ptr)))
+
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+		return __cmpxchg_local_u32(ptr, old, new);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	    \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
 #endif  /* __KERNEL__ */
 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 15/28] Fix m32r __xchg
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (13 preceding siblings ...)
  2007-08-27 15:52 ` [patch 14/28] New cmpxchg_local (optimized for UP case) for m32r Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 16/28] m32r: build fix of arch/m32r/kernel/smpboot.c Mathieu Desnoyers
                   ` (14 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel
  Cc: Mathieu Desnoyers, Hirokazu Takata, linux-m32r, Adrian Bunk

[-- Attachment #1: fix-m32r-__xchg.patch --]
[-- Type: text/plain, Size: 1564 bytes --]

the #endif  /* CONFIG_SMP */ should cover the default condition, or it may cause
bad parameter to be silently missed.

To make it work correctly, we have to remove the ifdef CONFIG SMP surrounding 
__xchg_called_with_bad_pointer declaration. Thanks to Adrian Bunk for detecting
this.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Acked-by: Hirokazu Takata <takata@linux-m32r.org>
CC: linux-m32r@ml.linux-m32r.org
CC: Adrian Bunk <bunk@kernel.org>
---
 include/asm-m32r/system.h |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

Index: linux-2.6-lttng/include/asm-m32r/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-m32r/system.h	2007-08-13 18:21:02.000000000 -0400
+++ linux-2.6-lttng/include/asm-m32r/system.h	2007-08-19 07:08:26.000000000 -0400
@@ -127,9 +127,7 @@ static inline void local_irq_disable(voi
 	((__typeof__(*(ptr)))__xchg_local((unsigned long)(x),(ptr), \
 			sizeof(*(ptr))))
 
-#ifdef CONFIG_SMP
 extern void  __xchg_called_with_bad_pointer(void);
-#endif
 
 #ifdef CONFIG_CHIP_M32700_TS1
 #define DCACHE_CLEAR(reg0, reg1, addr)				\
@@ -189,9 +187,9 @@ __xchg(unsigned long x, volatile void * 
 #endif	/* CONFIG_CHIP_M32700_TS1 */
 		);
 		break;
+#endif  /* CONFIG_SMP */
 	default:
 		__xchg_called_with_bad_pointer();
-#endif  /* CONFIG_SMP */
 	}
 
 	local_irq_restore(flags);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 16/28] m32r: build fix of arch/m32r/kernel/smpboot.c
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (14 preceding siblings ...)
  2007-08-27 15:52 ` [patch 15/28] Fix m32r __xchg Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 17/28] local_t m32r use architecture specific cmpxchg_local Mathieu Desnoyers
                   ` (13 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Hirokazu Takata

[-- Attachment #1: fix-m32r-include-sched-h-in-smpboot.patch --]
[-- Type: text/plain, Size: 2246 bytes --]

This patch is for Mathieu Desnoyers's include/asm-m32r/local.h.
Applying the new include/asm-m32r/local.h, inclusion of linux/sched.h
is needed to fix a build error of arch/m32r/kernel/smpboot.c.

<--  snip  -->
  ...
  CC      arch/m32r/kernel/smpboot.o
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c: In function 'do_boot_cpu':
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:279: error: implicit declaration of function 'fork_idle'
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:279: warning: assignment makes pointer from integer without a cast
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:283: error: dereferencing pointer to incomplete type
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:289: error: dereferencing pointer to incomplete type
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:290: error: implicit declaration of function 'task_thread_info'
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:290: error: invalid type argument of '->'
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c: In function 'start_secondary':
/project/m32r-linux/kernel/work/linux-2.6_dev.git/arch/m32r/kernel/smpboot.c:429: error: implicit declaration of function 'cpu_init'
make[2]: *** [arch/m32r/kernel/smpboot.o] Error 1
<--  snip  -->

Signed-off-by: Hirokazu Takata <takata@linux-m32r.org>
---
 arch/m32r/kernel/smpboot.c |    1 +
 1 file changed, 1 insertion(+)

Index: linux-2.6-lttng/arch/m32r/kernel/smpboot.c
===================================================================
--- linux-2.6-lttng.orig/arch/m32r/kernel/smpboot.c	2007-08-21 09:57:48.000000000 -0400
+++ linux-2.6-lttng/arch/m32r/kernel/smpboot.c	2007-08-21 09:58:12.000000000 -0400
@@ -43,6 +43,7 @@
 #include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
+#include <linux/sched.h>
 #include <linux/err.h>
 #include <linux/irq.h>
 #include <linux/bootmem.h>

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 17/28] local_t m32r use architecture specific cmpxchg_local
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (15 preceding siblings ...)
  2007-08-27 15:52 ` [patch 16/28] m32r: build fix of arch/m32r/kernel/smpboot.c Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 18/28] Add cmpxchg_local to m86k Mathieu Desnoyers
                   ` (12 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, Hirokazu Takata, linux-m32r

[-- Attachment #1: local_t_m32r_optimized.patch --]
[-- Type: text/plain, Size: 10263 bytes --]

On m32r, use the new cmpxchg_local as primitive for the local_cmpxchg
operation.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Acked-by: Hirokazu Takata <takata@linux-m32r.org>
CC: linux-m32r@ml.linux-m32r.org
---
 include/asm-m32r/local.h |  362 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 361 insertions(+), 1 deletion(-)

Index: linux-2.6-lttng/include/asm-m32r/local.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-m32r/local.h	2007-07-20 17:29:08.000000000 -0400
+++ linux-2.6-lttng/include/asm-m32r/local.h	2007-07-20 17:43:28.000000000 -0400
@@ -1,6 +1,366 @@
 #ifndef __M32R_LOCAL_H
 #define __M32R_LOCAL_H
 
-#include <asm-generic/local.h>
+/*
+ *  linux/include/asm-m32r/local.h
+ *
+ *  M32R version:
+ *    Copyright (C) 2001, 2002  Hitoshi Yamamoto
+ *    Copyright (C) 2004  Hirokazu Takata <takata at linux-m32r.org>
+ *    Copyright (C) 2007  Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ */
+
+#include <linux/percpu.h>
+#include <asm/assembler.h>
+#include <asm/system.h>
+#include <asm/local.h>
+
+/*
+ * Atomic operations that C can't guarantee us.  Useful for
+ * resource counting etc..
+ */
+
+/*
+ * Make sure gcc doesn't try to be clever and move things around
+ * on us. We need to use _exactly_ the address the user gave us,
+ * not some alias that contains the same information.
+ */
+typedef struct { volatile int counter; } local_t;
+
+#define LOCAL_INIT(i)	{ (i) }
+
+/**
+ * local_read - read local variable
+ * @l: pointer of type local_t
+ *
+ * Atomically reads the value of @l.
+ */
+#define local_read(l)	((l)->counter)
+
+/**
+ * local_set - set local variable
+ * @l: pointer of type local_t
+ * @i: required value
+ *
+ * Atomically sets the value of @l to @i.
+ */
+#define local_set(l,i)	(((l)->counter) = (i))
+
+/**
+ * local_add_return - add long to local variable and return it
+ * @i: long value to add
+ * @l: pointer of type local_t
+ *
+ * Atomically adds @i to @l and return (@i + @l).
+ */
+static __inline__ long local_add_return(long i, local_t *l)
+{
+	unsigned long flags;
+	long result;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_add_return		\n\t"
+		DCACHE_CLEAR("%0", "r4", "%1")
+		"ld %0, @%1;			\n\t"
+		"add	%0, %2;			\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (result)
+		: "r" (&l->counter), "r" (i)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r4"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+
+	return result;
+}
+
+/**
+ * local_sub_return - subtract long from local variable and return it
+ * @i: long value to subtract
+ * @l: pointer of type local_t
+ *
+ * Atomically subtracts @i from @l and return (@l - @i).
+ */
+static __inline__ long local_sub_return(long i, local_t *l)
+{
+	unsigned long flags;
+	long result;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_sub_return		\n\t"
+		DCACHE_CLEAR("%0", "r4", "%1")
+		"ld %0, @%1;			\n\t"
+		"sub	%0, %2;			\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (result)
+		: "r" (&l->counter), "r" (i)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r4"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+
+	return result;
+}
+
+/**
+ * local_add - add long to local variable
+ * @i: long value to add
+ * @l: pointer of type local_t
+ *
+ * Atomically adds @i to @l.
+ */
+#define local_add(i,l) ((void) local_add_return((i), (l)))
+
+/**
+ * local_sub - subtract the local variable
+ * @i: long value to subtract
+ * @l: pointer of type local_t
+ *
+ * Atomically subtracts @i from @l.
+ */
+#define local_sub(i,l) ((void) local_sub_return((i), (l)))
+
+/**
+ * local_sub_and_test - subtract value from variable and test result
+ * @i: integer value to subtract
+ * @l: pointer of type local_t
+ *
+ * Atomically subtracts @i from @l and returns
+ * true if the result is zero, or false for all
+ * other cases.
+ */
+#define local_sub_and_test(i,l) (local_sub_return((i), (l)) == 0)
+
+/**
+ * local_inc_return - increment local variable and return it
+ * @l: pointer of type local_t
+ *
+ * Atomically increments @l by 1 and returns the result.
+ */
+static __inline__ long local_inc_return(local_t *l)
+{
+	unsigned long flags;
+	long result;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_inc_return		\n\t"
+		DCACHE_CLEAR("%0", "r4", "%1")
+		"ld %0, @%1;			\n\t"
+		"addi	%0, #1;			\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (result)
+		: "r" (&l->counter)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r4"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+
+	return result;
+}
+
+/**
+ * local_dec_return - decrement local variable and return it
+ * @l: pointer of type local_t
+ *
+ * Atomically decrements @l by 1 and returns the result.
+ */
+static __inline__ long local_dec_return(local_t *l)
+{
+	unsigned long flags;
+	long result;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_dec_return		\n\t"
+		DCACHE_CLEAR("%0", "r4", "%1")
+		"ld %0, @%1;			\n\t"
+		"addi	%0, #-1;		\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (result)
+		: "r" (&l->counter)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r4"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+
+	return result;
+}
+
+/**
+ * local_inc - increment local variable
+ * @l: pointer of type local_t
+ *
+ * Atomically increments @l by 1.
+ */
+#define local_inc(l) ((void)local_inc_return(l))
+
+/**
+ * local_dec - decrement local variable
+ * @l: pointer of type local_t
+ *
+ * Atomically decrements @l by 1.
+ */
+#define local_dec(l) ((void)local_dec_return(l))
+
+/**
+ * local_inc_and_test - increment and test
+ * @l: pointer of type local_t
+ *
+ * Atomically increments @l by 1
+ * and returns true if the result is zero, or false for all
+ * other cases.
+ */
+#define local_inc_and_test(l) (local_inc_return(l) == 0)
+
+/**
+ * local_dec_and_test - decrement and test
+ * @l: pointer of type local_t
+ *
+ * Atomically decrements @l by 1 and
+ * returns true if the result is 0, or false for all
+ * other cases.
+ */
+#define local_dec_and_test(l) (local_dec_return(l) == 0)
+
+/**
+ * local_add_negative - add and test if negative
+ * @l: pointer of type local_t
+ * @i: integer value to add
+ *
+ * Atomically adds @i to @l and returns true
+ * if the result is negative, or false when
+ * result is greater than or equal to zero.
+ */
+#define local_add_negative(i,l) (local_add_return((i), (l)) < 0)
+
+#define local_cmpxchg(l, o, n) (cmpxchg_local(&((l)->counter), (o), (n)))
+#define local_xchg(v, new) (xchg_local(&((l)->counter), new))
+
+/**
+ * local_add_unless - add unless the number is a given value
+ * @l: pointer of type local_t
+ * @a: the amount to add to l...
+ * @u: ...unless l is equal to u.
+ *
+ * Atomically adds @a to @l, so long as it was not @u.
+ * Returns non-zero if @l was not @u, and zero otherwise.
+ */
+static __inline__ int local_add_unless(local_t *l, long a, long u)
+{
+	long c, old;
+	c = local_read(l);
+	for (;;) {
+		if (unlikely(c == (u)))
+			break;
+		old = local_cmpxchg((l), c, c + (a));
+		if (likely(old == c))
+			break;
+		c = old;
+	}
+	return c != (u);
+}
+
+#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+
+static __inline__ void local_clear_mask(unsigned long  mask, local_t *addr)
+{
+	unsigned long flags;
+	unsigned long tmp;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_clear_mask		\n\t"
+		DCACHE_CLEAR("%0", "r5", "%1")
+		"ld %0, @%1;			\n\t"
+		"and	%0, %2;			\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (tmp)
+		: "r" (addr), "r" (~mask)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r5"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+}
+
+static __inline__ void local_set_mask(unsigned long  mask, local_t *addr)
+{
+	unsigned long flags;
+	unsigned long tmp;
+
+	local_irq_save(flags);
+	__asm__ __volatile__ (
+		"# local_set_mask		\n\t"
+		DCACHE_CLEAR("%0", "r5", "%1")
+		"ld %0, @%1;			\n\t"
+		"or	%0, %2;			\n\t"
+		"st %0, @%1;			\n\t"
+		: "=&r" (tmp)
+		: "r" (addr), "r" (mask)
+		: "memory"
+#ifdef CONFIG_CHIP_M32700_TS1
+		, "r5"
+#endif	/* CONFIG_CHIP_M32700_TS1 */
+	);
+	local_irq_restore(flags);
+}
+
+/* Atomic operations are already serializing on m32r */
+#define smp_mb__before_local_dec()	barrier()
+#define smp_mb__after_local_dec()	barrier()
+#define smp_mb__before_local_inc()	barrier()
+#define smp_mb__after_local_inc()	barrier()
+
+/* Use these for per-cpu local_t variables: on some archs they are
+ * much more efficient than these naive implementations.  Note they take
+ * a variable, not an address.
+ */
+
+#define __local_inc(l)		((l)->a.counter++)
+#define __local_dec(l)		((l)->a.counter++)
+#define __local_add(i,l)	((l)->a.counter+=(i))
+#define __local_sub(i,l)	((l)->a.counter-=(i))
+
+/* Use these for per-cpu local_t variables: on some archs they are
+ * much more efficient than these naive implementations.  Note they take
+ * a variable, not an address.
+ */
+
+/* Need to disable preemption for the cpu local counters otherwise we could
+   still access a variable of a previous CPU in a non local way. */
+#define cpu_local_wrap_v(l)	 	\
+	({ local_t res__;		\
+	   preempt_disable(); 		\
+	   res__ = (l);			\
+	   preempt_enable();		\
+	   res__; })
+#define cpu_local_wrap(l)		\
+	({ preempt_disable();		\
+	   l;				\
+	   preempt_enable(); })		\
+
+#define cpu_local_read(l)    cpu_local_wrap_v(local_read(&__get_cpu_var(l)))
+#define cpu_local_set(l, i)  cpu_local_wrap(local_set(&__get_cpu_var(l), (i)))
+#define cpu_local_inc(l)     cpu_local_wrap(local_inc(&__get_cpu_var(l)))
+#define cpu_local_dec(l)     cpu_local_wrap(local_dec(&__get_cpu_var(l)))
+#define cpu_local_add(i, l)  cpu_local_wrap(local_add((i), &__get_cpu_var(l)))
+#define cpu_local_sub(i, l)  cpu_local_wrap(local_sub((i), &__get_cpu_var(l)))
+
+#define __cpu_local_inc(l)	cpu_local_inc(l)
+#define __cpu_local_dec(l)	cpu_local_dec(l)
+#define __cpu_local_add(i, l)	cpu_local_add((i), (l))
+#define __cpu_local_sub(i, l)	cpu_local_sub((i), (l))
 
 #endif /* __M32R_LOCAL_H */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 18/28] Add cmpxchg_local to m86k
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (16 preceding siblings ...)
  2007-08-27 15:52 ` [patch 17/28] local_t m32r use architecture specific cmpxchg_local Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 19/28] Add cmpxchg_local to m68knommu Mathieu Desnoyers
                   ` (11 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, zippel, linux-m68k

[-- Attachment #1: add-cmpxchg-local-to-m68k.patch --]
[-- Type: text/plain, Size: 1870 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: zippel@linux-m68k.org
CC: linux-m68k@lists.linux-m68k.org
---
 include/asm-m68k/system.h |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

Index: linux-2.6-lttng/include/asm-m68k/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-m68k/system.h	2007-08-07 14:31:51.000000000 -0400
+++ linux-2.6-lttng/include/asm-m68k/system.h	2007-08-07 15:01:41.000000000 -0400
@@ -154,6 +154,10 @@ static inline unsigned long __xchg(unsig
 }
 #endif
 
+#include <asm-generic/cmpxchg-local.h>
+
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
 /*
  * Atomic compare and exchange.  Compare OLD with MEM, if identical,
  * store NEW in MEM.  Return the initial value in MEM.  Success is
@@ -188,6 +192,23 @@ static inline unsigned long __cmpxchg(vo
 #define cmpxchg(ptr,o,n)\
 	((__typeof__(*(ptr)))__cmpxchg((ptr),(unsigned long)(o),\
 					(unsigned long)(n),sizeof(*(ptr))))
+#define cmpxchg_local(ptr,o,n)\
+	((__typeof__(*(ptr)))__cmpxchg((ptr),(unsigned long)(o),\
+					(unsigned long)(n),sizeof(*(ptr))))
+#else
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
+
 #endif
 
 #define arch_align_stack(x) (x)

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 19/28] Add cmpxchg_local to m68knommu
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (17 preceding siblings ...)
  2007-08-27 15:52 ` [patch 18/28] Add cmpxchg_local to m86k Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 20/28] Add cmpxchg_local to parisc Mathieu Desnoyers
                   ` (10 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, gerg

[-- Attachment #1: add-cmpxchg-local-to-m68knommu.patch --]
[-- Type: text/plain, Size: 1849 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: gerg@uclinux.org
---
 include/asm-m68knommu/system.h |   28 +++++++++++-----------------
 1 file changed, 11 insertions(+), 17 deletions(-)

Index: linux-2.6-lttng/include/asm-m68knommu/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-m68knommu/system.h	2007-07-20 18:36:08.000000000 -0400
+++ linux-2.6-lttng/include/asm-m68knommu/system.h	2007-07-20 19:37:02.000000000 -0400
@@ -187,26 +187,20 @@ static inline unsigned long __xchg(unsig
 }
 #endif
 
+#include <asm-generic/cmpxchg-local.h>
+
 /*
- * Atomic compare and exchange.  Compare OLD with MEM, if identical,
- * store NEW in MEM.  Return the initial value in MEM.  Success is
- * indicated by comparing RETURN with OLD.
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
  */
-#define __HAVE_ARCH_CMPXCHG	1
-
-static __inline__ unsigned long
-cmpxchg(volatile int *p, int old, int new)
-{
-	unsigned long flags;
-	int prev;
-
-	local_irq_save(flags);
-	if ((prev = *p) == old)
-		*p = new;
-	local_irq_restore(flags);
-	return(prev);
-}
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
 
 #ifdef CONFIG_M68332
 #define HARD_RESET_NOW() ({		\

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 20/28] Add cmpxchg_local to parisc
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (18 preceding siblings ...)
  2007-08-27 15:52 ` [patch 19/28] Add cmpxchg_local to m68knommu Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 21/28] Add cmpxchg_local to ppc Mathieu Desnoyers
                   ` (9 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, parisc-linux

[-- Attachment #1: add-cmpxchg-local-to-parisc.patch --]
[-- Type: text/plain, Size: 1917 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: parisc-linux@parisc-linux.org
---
 include/asm-parisc/atomic.h |   29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

Index: linux-2.6-lttng/include/asm-parisc/atomic.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-parisc/atomic.h	2007-07-20 19:44:40.000000000 -0400
+++ linux-2.6-lttng/include/asm-parisc/atomic.h	2007-07-20 19:44:47.000000000 -0400
@@ -122,6 +122,35 @@ __cmpxchg(volatile void *ptr, unsigned l
 				    (unsigned long)_n_, sizeof(*(ptr))); \
   })
 
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new_, int size)
+{
+	switch (size) {
+#ifdef CONFIG_64BIT
+	case 8:	return __cmpxchg_u64((unsigned long *)ptr, old, new_);
+#endif
+	case 4:	return __cmpxchg_u32(ptr, old, new_);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new_, size);
+	}
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	\
+     (__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	\
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#ifdef CONFIG_64BIT
+#define cmpxchg64_local(ptr,o,n) cmpxchg_local((ptr), (o), (n))
+#else
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+#endif
+
 /* Note that we need not lock read accesses - aligned word writes/reads
  * are atomic, so a reader never sees unconsistent values.
  *

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 21/28] Add cmpxchg_local to ppc
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (19 preceding siblings ...)
  2007-08-27 15:52 ` [patch 20/28] Add cmpxchg_local to parisc Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 22/28] Add cmpxchg_local to s390 Mathieu Desnoyers
                   ` (8 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Gunnar Larisch, Mathieu Desnoyers, clameter, benh

[-- Attachment #1: add-cmpxchg-local-to-ppc.patch --]
[-- Type: text/plain, Size: 2796 bytes --]

Add a local processor version of cmpxchg for ppc.

Implements __cmpxchg_u32_local and uses it for 32 bits cmpxchg_local.
It uses the non NMI safe cmpxchg_local_generic for 1, 2 and 8 bytes
cmpxchg_local.

From: Gunnar Larisch <gl@denx.de>
From: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Gunnar Larisch <gl@denx.de>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: benh@kernel.crashing.org
---
 include/asm-ppc/system.h |   49 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)

Index: linux-2.6-lttng/include/asm-ppc/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-ppc/system.h	2007-08-07 14:31:51.000000000 -0400
+++ linux-2.6-lttng/include/asm-ppc/system.h	2007-08-07 15:03:51.000000000 -0400
@@ -208,12 +208,34 @@ __cmpxchg_u32(volatile unsigned int *p, 
 	return prev;
 }
 
+static __inline__ unsigned long
+__cmpxchg_u32_local(volatile unsigned int *p, unsigned int old,
+	unsigned int new)
+{
+	unsigned int prev;
+
+	__asm__ __volatile__ ("\n\
+1:	lwarx	%0,0,%2 \n\
+	cmpw	0,%0,%3 \n\
+	bne	2f \n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%4,0,%2 \n\
+	bne-	1b\n"
+"2:"
+	: "=&r" (prev), "=m" (*p)
+	: "r" (p), "r" (old), "r" (new), "m" (*p)
+	: "cc", "memory");
+
+	return prev;
+}
+
 /* This function doesn't exist, so you'll get a linker error
    if something tries to do an invalid cmpxchg().  */
 extern void __cmpxchg_called_with_bad_pointer(void);
 
 static __inline__ unsigned long
-__cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size)
+__cmpxchg(volatile void *ptr, unsigned long old, unsigned long new,
+	unsigned int size)
 {
 	switch (size) {
 	case 4:
@@ -235,6 +257,31 @@ __cmpxchg(volatile void *ptr, unsigned l
 				    (unsigned long)_n_, sizeof(*(ptr))); \
   })
 
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+		return __cmpxchg_u32_local(ptr, old, new);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	\
+     (__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	\
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
 #define arch_align_stack(x) (x)
 
 #endif /* __KERNEL__ */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 22/28] Add cmpxchg_local to s390
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (20 preceding siblings ...)
  2007-08-27 15:52 ` [patch 21/28] Add cmpxchg_local to ppc Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 23/28] Add cmpxchg_local to sh, use generic cmpxchg() instead of cmpxchg_u32 Mathieu Desnoyers
                   ` (7 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, schwidefsky, linux390

[-- Attachment #1: add-cmpxchg-local-to-s390.patch --]
[-- Type: text/plain, Size: 1873 bytes --]

Use the standard __cmpxchg for every type that can be updated atomically.
Use the new generic cmpxchg_local (disables interrupt) for other types.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: schwidefsky@de.ibm.com
CC: linux390@de.ibm.com
---
 include/asm-s390/system.h |   34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

Index: linux-2.6-lttng/include/asm-s390/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-s390/system.h	2007-08-27 11:46:43.000000000 -0400
+++ linux-2.6-lttng/include/asm-s390/system.h	2007-08-27 11:48:38.000000000 -0400
@@ -355,6 +355,40 @@ __cmpxchg(volatile void *ptr, unsigned l
 
 #include <linux/irqflags.h>
 
+#include <asm-generic/cmpxchg-local.h>
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 1:
+	case 2:
+	case 4:
+#ifdef __s390x__
+	case 8:
+#endif
+		return __cmpxchg(ptr, old, new, size);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)						\
+     (__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	\
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#ifdef __s390x__
+#define cmpxchg64_local(ptr,o,n) cmpxchg_local((ptr),(o),(n))
+#else
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+#endif
+
 /*
  * Use to set psw mask except for the first byte which
  * won't be changed by this function.

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 23/28] Add cmpxchg_local to sh, use generic cmpxchg() instead of cmpxchg_u32
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (21 preceding siblings ...)
  2007-08-27 15:52 ` [patch 22/28] Add cmpxchg_local to s390 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 24/28] Add cmpxchg_local to sh64 Mathieu Desnoyers
                   ` (6 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, lethal

[-- Attachment #1: add-cmpxchg-local-to-sh.patch --]
[-- Type: text/plain, Size: 2658 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Since cmpxchg_u32 is _exactly_ the cmpxchg_local generic implementation, remove
it and use the generic version instead.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: lethal@linux-sh.org
---
 include/asm-sh/system.h |   48 ++++++++++++------------------------------------
 1 file changed, 12 insertions(+), 36 deletions(-)

Index: linux-2.6-lttng/include/asm-sh/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-sh/system.h	2007-08-07 14:31:51.000000000 -0400
+++ linux-2.6-lttng/include/asm-sh/system.h	2007-08-07 15:08:10.000000000 -0400
@@ -201,44 +201,20 @@ extern void __xchg_called_with_bad_point
 #define xchg(ptr,x)	\
 	((__typeof__(*(ptr)))__xchg((ptr),(unsigned long)(x), sizeof(*(ptr))))
 
-static inline unsigned long __cmpxchg_u32(volatile int * m, unsigned long old,
-	unsigned long new)
-{
-	__u32 retval;
-	unsigned long flags;
-
-	local_irq_save(flags);
-	retval = *m;
-	if (retval == old)
-		*m = new;
-	local_irq_restore(flags);       /* implies memory barrier  */
-	return retval;
-}
-
-/* This function doesn't exist, so you'll get a linker error
- * if something tries to do an invalid cmpxchg(). */
-extern void __cmpxchg_called_with_bad_pointer(void);
-
-#define __HAVE_ARCH_CMPXCHG 1
+#include <asm-generic/cmpxchg-local.h>
 
-static inline unsigned long __cmpxchg(volatile void * ptr, unsigned long old,
-		unsigned long new, int size)
-{
-	switch (size) {
-	case 4:
-		return __cmpxchg_u32(ptr, old, new);
-	}
-	__cmpxchg_called_with_bad_pointer();
-	return old;
-}
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
-#define cmpxchg(ptr,o,n)						 \
-  ({									 \
-     __typeof__(*(ptr)) _o_ = (o);					 \
-     __typeof__(*(ptr)) _n_ = (n);					 \
-     (__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,		 \
-				    (unsigned long)_n_, sizeof(*(ptr))); \
-  })
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
 
 extern void die(const char *str, struct pt_regs *regs, long err) __attribute__ ((noreturn));
 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 24/28] Add cmpxchg_local to sh64
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (22 preceding siblings ...)
  2007-08-27 15:52 ` [patch 23/28] Add cmpxchg_local to sh, use generic cmpxchg() instead of cmpxchg_u32 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:52 ` [patch 25/28] Add cmpxchg_local to sparc, move __cmpxchg to system.h Mathieu Desnoyers
                   ` (5 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, lethal

[-- Attachment #1: add-cmpxchg-local-to-sh64.patch --]
[-- Type: text/plain, Size: 1376 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: lethal@linux-sh.org
---
 include/asm-sh64/system.h |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

Index: linux-2.6-lttng/include/asm-sh64/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-sh64/system.h	2007-07-23 11:01:16.000000000 -0400
+++ linux-2.6-lttng/include/asm-sh64/system.h	2007-07-23 11:01:20.000000000 -0400
@@ -164,6 +164,21 @@ static __inline__ unsigned long __xchg(u
 	return x;
 }
 
+#include <asm-generic/cmpxchg-local.h>
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
+
 /* XXX
  * disable hlt during certain critical i/o operations
  */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 25/28] Add cmpxchg_local to sparc, move __cmpxchg to system.h
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (23 preceding siblings ...)
  2007-08-27 15:52 ` [patch 24/28] Add cmpxchg_local to sh64 Mathieu Desnoyers
@ 2007-08-27 15:52 ` Mathieu Desnoyers
  2007-08-27 15:53 ` [patch 26/28] Add cmpxchg_local to sparc64 Mathieu Desnoyers
                   ` (4 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:52 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, wli, sparclinux

[-- Attachment #1: add-cmpxchg-local-to-sparc.patch --]
[-- Type: text/plain, Size: 4204 bytes --]

Move cmpxchg and add cmpxchg_local to system.h.
Use the new generic cmpxchg_local (disables interrupt).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: wli@holomorphy.com
CC: sparclinux@vger.kernel.org
---
 include/asm-sparc/atomic.h |   36 ----------------------------------
 include/asm-sparc/system.h |   47 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 36 deletions(-)

Index: linux-2.6-lttng/include/asm-sparc/atomic.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-sparc/atomic.h	2007-08-10 19:36:01.000000000 -0400
+++ linux-2.6-lttng/include/asm-sparc/atomic.h	2007-08-10 19:42:47.000000000 -0400
@@ -17,42 +17,6 @@ typedef struct { volatile int counter; }
 
 #ifdef __KERNEL__
 
-/* Emulate cmpxchg() the same way we emulate atomics,
- * by hashing the object address and indexing into an array
- * of spinlocks to get a bit of performance...
- *
- * See arch/sparc/lib/atomic32.c for implementation.
- *
- * Cribbed from <asm-parisc/atomic.h>
- */
-#define __HAVE_ARCH_CMPXCHG	1
-
-/* bug catcher for when unsupported size is used - won't link */
-extern void __cmpxchg_called_with_bad_pointer(void);
-/* we only need to support cmpxchg of a u32 on sparc */
-extern unsigned long __cmpxchg_u32(volatile u32 *m, u32 old, u32 new_);
-
-/* don't worry...optimizer will get rid of most of this */
-static __inline__ unsigned long
-__cmpxchg(volatile void *ptr, unsigned long old, unsigned long new_, int size)
-{
-	switch(size) {
-	case 4:
-		return __cmpxchg_u32((u32 *)ptr, (u32)old, (u32)new_);
-	default:
-		__cmpxchg_called_with_bad_pointer();
-		break;
-	}
-	return old;
-}
-
-#define cmpxchg(ptr,o,n) ({						\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,	\
-			(unsigned long)_n_, sizeof(*(ptr)));		\
-})
-
 #define ATOMIC_INIT(i)  { (i) }
 
 extern int __atomic_add_return(int, atomic_t *);
Index: linux-2.6-lttng/include/asm-sparc/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-sparc/system.h	2007-08-10 19:43:08.000000000 -0400
+++ linux-2.6-lttng/include/asm-sparc/system.h	2007-08-10 19:43:42.000000000 -0400
@@ -245,6 +245,53 @@ static __inline__ unsigned long __xchg(u
 	return x;
 }
 
+/* Emulate cmpxchg() the same way we emulate atomics,
+ * by hashing the object address and indexing into an array
+ * of spinlocks to get a bit of performance...
+ *
+ * See arch/sparc/lib/atomic32.c for implementation.
+ *
+ * Cribbed from <asm-parisc/atomic.h>
+ */
+#define __HAVE_ARCH_CMPXCHG	1
+
+/* bug catcher for when unsupported size is used - won't link */
+extern void __cmpxchg_called_with_bad_pointer(void);
+/* we only need to support cmpxchg of a u32 on sparc */
+extern unsigned long __cmpxchg_u32(volatile u32 *m, u32 old, u32 new_);
+
+/* don't worry...optimizer will get rid of most of this */
+static __inline__ unsigned long
+__cmpxchg(volatile void *ptr, unsigned long old, unsigned long new_, int size)
+{
+	switch(size) {
+	case 4:
+		return __cmpxchg_u32((u32 *)ptr, (u32)old, (u32)new_);
+	default:
+		__cmpxchg_called_with_bad_pointer();
+		break;
+	}
+	return old;
+}
+
+#define cmpxchg(ptr,o,n) ({						\
+	__typeof__(*(ptr)) _o_ = (o);					\
+	__typeof__(*(ptr)) _n_ = (n);					\
+	(__typeof__(*(ptr))) __cmpxchg((ptr), (unsigned long)_o_,	\
+			(unsigned long)_n_, sizeof(*(ptr)));		\
+})
+
+#include <asm-generic/cmpxchg-local.h>
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
 extern void die_if_kernel(char *str, struct pt_regs *regs) __attribute__ ((noreturn));
 
 #endif /* __KERNEL__ */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 26/28] Add cmpxchg_local to sparc64
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (24 preceding siblings ...)
  2007-08-27 15:52 ` [patch 25/28] Add cmpxchg_local to sparc, move __cmpxchg to system.h Mathieu Desnoyers
@ 2007-08-27 15:53 ` Mathieu Desnoyers
  2007-08-27 15:53 ` [patch 27/28] Add cmpxchg_local to v850 Mathieu Desnoyers
                   ` (3 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:53 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter, sparclinux, wli

[-- Attachment #1: add-cmpxchg-local-to-sparc64.patch --]
[-- Type: text/plain, Size: 2110 bytes --]

Use cmpxchg_u32 and cmpxchg_u64 for cmpxchg_local and cmpxchg64_local. For other
type sizes, use the new generic cmpxchg_local (disables interrupt).

Change:
Since the header depends on local_irqsave/local_irqrestore, it must be
included after their declaration.

Actually, being below the
#include <linux/irqflags.h> should be enough, and on sparc64 it is
included at the beginning of system.h.

So it makes sense to move it up for sparc64.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
CC: sparclinux@vger.kernel.org
CC: wli@holomorphy.com
---
 include/asm-sparc64/system.h |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

Index: linux-2.6-lttng/include/asm-sparc64/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-sparc64/system.h	2007-08-20 17:40:57.000000000 -0400
+++ linux-2.6-lttng/include/asm-sparc64/system.h	2007-08-20 19:42:32.000000000 -0400
@@ -9,6 +9,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/irqflags.h>
+#include <asm-generic/cmpxchg-local.h>
 
 /*
  * Sparc (general) CPU types
@@ -314,6 +315,30 @@ __cmpxchg(volatile void *ptr, unsigned l
 				    (unsigned long)_n_, sizeof(*(ptr))); \
   })
 
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+	case 8:	return __cmpxchg(ptr, old, new, size);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
+
+	return old;
+}
+
+#define cmpxchg_local(ptr,o,n)					  	\
+	(__typeof__(*(ptr)))__cmpxchg_local((ptr), (unsigned long)(o),	\
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) cmpxchg_local((ptr), (o), (n))
+
 #endif /* !(__ASSEMBLY__) */
 
 #define arch_align_stack(x) (x)

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 27/28] Add cmpxchg_local to v850
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (25 preceding siblings ...)
  2007-08-27 15:53 ` [patch 26/28] Add cmpxchg_local to sparc64 Mathieu Desnoyers
@ 2007-08-27 15:53 ` Mathieu Desnoyers
  2007-08-27 15:53 ` [patch 28/28] Add cmpxchg_local to xtensa Mathieu Desnoyers
                   ` (2 subsequent siblings)
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:53 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter

[-- Attachment #1: add-cmpxchg-local-to-v850.patch --]
[-- Type: text/plain, Size: 1353 bytes --]

Use the new generic cmpxchg_local (disables interrupt). Also use the generic
cmpxchg as fallback if SMP is not set.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
---
 include/asm-v850/system.h |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

Index: linux-2.6-lttng/include/asm-v850/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-v850/system.h	2007-07-23 11:28:45.000000000 -0400
+++ linux-2.6-lttng/include/asm-v850/system.h	2007-07-23 11:28:57.000000000 -0400
@@ -104,6 +104,21 @@ static inline unsigned long __xchg (unsi
 	return tmp;
 }
 
+#include <asm-generic/cmpxchg-local.h>
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
+
+#ifndef CONFIG_SMP
+#include <asm-generic/cmpxchg.h>
+#endif
+
 #define arch_align_stack(x) (x)
 
 #endif /* __V850_SYSTEM_H__ */

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* [patch 28/28] Add cmpxchg_local to xtensa
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (26 preceding siblings ...)
  2007-08-27 15:53 ` [patch 27/28] Add cmpxchg_local to v850 Mathieu Desnoyers
@ 2007-08-27 15:53 ` Mathieu Desnoyers
  2007-08-27 16:17 ` [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Andrew Morton
  2007-08-27 19:35 ` Christoph Lameter
  29 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 15:53 UTC (permalink / raw)
  To: akpm, linux-kernel; +Cc: Mathieu Desnoyers, clameter

[-- Attachment #1: add-cmpxchg-local-to-xtensa.patch --]
[-- Type: text/plain, Size: 1631 bytes --]

Use the architecture specific __cmpxchg_u32 for 32 bits cmpxchg)_local. Else,
use the new generic cmpxchg_local (disables interrupt).

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: clameter@sgi.com
chris@zankel.net
---
 include/asm-xtensa/system.h |   22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

Index: linux-2.6-lttng/include/asm-xtensa/system.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-xtensa/system.h	2007-07-23 11:29:18.000000000 -0400
+++ linux-2.6-lttng/include/asm-xtensa/system.h	2007-07-23 11:32:32.000000000 -0400
@@ -156,8 +156,30 @@ __cmpxchg(volatile void *ptr, unsigned l
 	   			        (unsigned long)_n_, sizeof (*(ptr))); \
 	})
 
+#include <asm-generic/cmpxchg-local.h>
 
+static inline unsigned long __cmpxchg_local(volatile void *ptr,
+				      unsigned long old,
+				      unsigned long new, int size)
+{
+	switch (size) {
+	case 4:
+		return __cmpxchg_u32(ptr, old, new);
+	default:
+		return __cmpxchg_local_generic(ptr, old, new, size);
+	}
 
+	return old;
+}
+
+/*
+ * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make
+ * them available.
+ */
+#define cmpxchg_local(ptr,o,n)					  	    \
+     (__typeof__(*(ptr)))__cmpxchg_local_generic((ptr), (unsigned long)(o), \
+			   	 (unsigned long)(n), sizeof(*(ptr)))
+#define cmpxchg64_local(ptr,o,n) __cmpxchg64_local_generic((ptr), (o), (n))
 
 /*
  * xchg_u32

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (27 preceding siblings ...)
  2007-08-27 15:53 ` [patch 28/28] Add cmpxchg_local to xtensa Mathieu Desnoyers
@ 2007-08-27 16:17 ` Andrew Morton
  2007-08-27 17:31   ` Mathieu Desnoyers
  2007-08-27 19:35 ` Christoph Lameter
  29 siblings, 1 reply; 41+ messages in thread
From: Andrew Morton @ 2007-08-27 16:17 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: linux-kernel

On Mon, 27 Aug 2007 11:52:34 -0400 Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:

> Here is the patch series for 2.6.23-rc3-mm1 that adds cmpxchg_local, and now
> also cmpxchg64_local, to each architecture.

How well tested are these on the various architectures?

> When the architecture supports it, it also defines cmpxchg64, but is is not
> defined for architecture that does not support atomic 64 bits updates.
> 
> Following performance testing of the slub allocator with cmpxchg_local, these
> patches should prove themselves useful in a near future.

It would be useful if we could have (numerical) details on these benefits as
part of this patch series description, please.

Also, it would be good to get the slub patch in there at the same time so that
the new code gets a bit of exercise.

Thanks.

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
  2007-08-27 16:17 ` [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Andrew Morton
@ 2007-08-27 17:31   ` Mathieu Desnoyers
  2007-08-27 19:29     ` Christoph Lameter
  0 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 17:31 UTC (permalink / raw)
  To: Andrew Morton, Christoph Lameter; +Cc: linux-kernel

* Andrew Morton (akpm@linux-foundation.org) wrote:
> On Mon, 27 Aug 2007 11:52:34 -0400 Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:
> 
> > Here is the patch series for 2.6.23-rc3-mm1 that adds cmpxchg_local, and now
> > also cmpxchg64_local, to each architecture.
> 
> How well tested are these on the various architectures?
> 

I compile-tested the patchset on:

arm
i686
ia64
m68k
mips/mipsel
powerpc405
powerpc 64
s390
sparc64
sparc
x86_64

With various config options (ALL yes, ALL no, all modules,
CONFIG_MODULES=yes/no...)

Since then, I added the trivial architecture-specific patches:
add-cmpxchg64-to-alpha.patch
add-cmpxchg64-to-mips.patch
add-cmpxchg64-to-powerpc.patch
add-cmpxchg64-to-x86_64.patch

Which I tested before submitting; mips, powerpc405, powerpc64 and x86_64
build fine. (I have no cross-compiler for alpha though)

The rest of the changes, since last thorough architecture-wide compile
test, were either architecture specific comments I got from LKML or
tested since then because they were architecture agnostic.

I must admit though that there are still a few architectures I touch in
the cmpxchg_local patches for which I don't have a cross-compiler. But I
think the changes are trivial enough, and repetitive enough across
architectures, to minimize the build breakages.

For runtime testing, I am a bit limited on hardware: I myself have i686
and AMD64, and rely on the LTTng community (and the kernel community) to
test the other architectures.

Christoph Lameter and I tested the cmpxchg_local with slub on i686 and
x86_64.


> > When the architecture supports it, it also defines cmpxchg64, but is is not
> > defined for architecture that does not support atomic 64 bits updates.
> > 
> > Following performance testing of the slub allocator with cmpxchg_local, these
> > patches should prove themselves useful in a near future.
> 
> It would be useful if we could have (numerical) details on these benefits as
> part of this patch series description, please.
> 

Sure, they follow at the end of this email (will append to the
add-cmpxchg-local-to-generic-for-up.patch description).

> Also, it would be good to get the slub patch in there at the same time so that
> the new code gets a bit of exercise.
> 

Good point. Christoph, could you please prepare a slub cmpxchg_local
patch for the mm tree ?

Mathieu

> Thanks.

Patch add-cmpxchg-local-to-generic-for-up.patch description addendum :

* Patch series comments

Performance improvements of the fast path goes from a 66% speedup on a
Pentium 4 to a 14% speedup on AMD64.

Tested-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Measurements on a Pentium4, 3GHz, Hyperthread.
SLUB Performance testing
========================
1. Kmalloc: Repeatedly allocate then free test

* slub HEAD, test 1
kmalloc(8) = 201 cycles         kfree = 351 cycles
kmalloc(16) = 198 cycles        kfree = 359 cycles
kmalloc(32) = 200 cycles        kfree = 381 cycles
kmalloc(64) = 224 cycles        kfree = 394 cycles
kmalloc(128) = 285 cycles       kfree = 424 cycles
kmalloc(256) = 411 cycles       kfree = 546 cycles
kmalloc(512) = 480 cycles       kfree = 619 cycles
kmalloc(1024) = 623 cycles      kfree = 750 cycles
kmalloc(2048) = 686 cycles      kfree = 811 cycles
kmalloc(4096) = 482 cycles      kfree = 538 cycles
kmalloc(8192) = 680 cycles      kfree = 734 cycles
kmalloc(16384) = 713 cycles     kfree = 843 cycles

* Slub HEAD, test 2
kmalloc(8) = 190 cycles         kfree = 351 cycles
kmalloc(16) = 195 cycles        kfree = 360 cycles
kmalloc(32) = 201 cycles        kfree = 370 cycles
kmalloc(64) = 245 cycles        kfree = 389 cycles
kmalloc(128) = 283 cycles       kfree = 413 cycles
kmalloc(256) = 409 cycles       kfree = 547 cycles
kmalloc(512) = 476 cycles       kfree = 616 cycles
kmalloc(1024) = 628 cycles      kfree = 753 cycles
kmalloc(2048) = 684 cycles      kfree = 811 cycles
kmalloc(4096) = 480 cycles      kfree = 539 cycles
kmalloc(8192) = 661 cycles      kfree = 746 cycles
kmalloc(16384) = 741 cycles     kfree = 856 cycles

* cmpxchg_local Slub test
kmalloc(8) = 83 cycles          kfree = 363 cycles
kmalloc(16) = 85 cycles         kfree = 372 cycles
kmalloc(32) = 92 cycles         kfree = 377 cycles
kmalloc(64) = 115 cycles        kfree = 397 cycles
kmalloc(128) = 179 cycles       kfree = 438 cycles
kmalloc(256) = 314 cycles       kfree = 564 cycles
kmalloc(512) = 398 cycles       kfree = 615 cycles
kmalloc(1024) = 573 cycles      kfree = 745 cycles
kmalloc(2048) = 629 cycles      kfree = 816 cycles
kmalloc(4096) = 473 cycles      kfree = 548 cycles
kmalloc(8192) = 659 cycles      kfree = 745 cycles
kmalloc(16384) = 724 cycles     kfree = 843 cycles


2. Kmalloc: alloc/free test

* slub HEAD, test 1
kmalloc(8)/kfree = 322 cycles
kmalloc(16)/kfree = 318 cycles
kmalloc(32)/kfree = 318 cycles
kmalloc(64)/kfree = 325 cycles
kmalloc(128)/kfree = 318 cycles
kmalloc(256)/kfree = 328 cycles
kmalloc(512)/kfree = 328 cycles
kmalloc(1024)/kfree = 328 cycles
kmalloc(2048)/kfree = 328 cycles
kmalloc(4096)/kfree = 678 cycles
kmalloc(8192)/kfree = 1013 cycles
kmalloc(16384)/kfree = 1157 cycles

* Slub HEAD, test 2
kmalloc(8)/kfree = 323 cycles
kmalloc(16)/kfree = 318 cycles
kmalloc(32)/kfree = 318 cycles
kmalloc(64)/kfree = 318 cycles
kmalloc(128)/kfree = 318 cycles
kmalloc(256)/kfree = 328 cycles
kmalloc(512)/kfree = 328 cycles
kmalloc(1024)/kfree = 328 cycles
kmalloc(2048)/kfree = 328 cycles
kmalloc(4096)/kfree = 648 cycles
kmalloc(8192)/kfree = 1009 cycles
kmalloc(16384)/kfree = 1105 cycles

* cmpxchg_local Slub test
kmalloc(8)/kfree = 112 cycles
kmalloc(16)/kfree = 103 cycles
kmalloc(32)/kfree = 103 cycles
kmalloc(64)/kfree = 103 cycles
kmalloc(128)/kfree = 112 cycles
kmalloc(256)/kfree = 111 cycles
kmalloc(512)/kfree = 111 cycles
kmalloc(1024)/kfree = 111 cycles
kmalloc(2048)/kfree = 121 cycles
kmalloc(4096)/kfree = 650 cycles
kmalloc(8192)/kfree = 1042 cycles
kmalloc(16384)/kfree = 1149 cycles



Tested-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Measurements on a AMD64 2.0 GHz dual-core

In this test, we seem to remove 10 cycles from the kmalloc fast path.
On small allocations, it gives a 14% performance increase. kfree fast
path also seems to have a 10 cycles improvement.

1. Kmalloc: Repeatedly allocate then free test

* cmpxchg_local slub
kmalloc(8) = 63 cycles      kfree = 126 cycles
kmalloc(16) = 66 cycles     kfree = 129 cycles
kmalloc(32) = 76 cycles     kfree = 138 cycles
kmalloc(64) = 100 cycles    kfree = 288 cycles
kmalloc(128) = 128 cycles   kfree = 309 cycles
kmalloc(256) = 170 cycles   kfree = 315 cycles
kmalloc(512) = 221 cycles   kfree = 357 cycles
kmalloc(1024) = 324 cycles  kfree = 393 cycles
kmalloc(2048) = 354 cycles  kfree = 440 cycles
kmalloc(4096) = 394 cycles  kfree = 330 cycles
kmalloc(8192) = 523 cycles  kfree = 481 cycles
kmalloc(16384) = 643 cycles kfree = 649 cycles

* Base
kmalloc(8) = 74 cycles      kfree = 113 cycles
kmalloc(16) = 76 cycles     kfree = 116 cycles
kmalloc(32) = 85 cycles     kfree = 133 cycles
kmalloc(64) = 111 cycles    kfree = 279 cycles
kmalloc(128) = 138 cycles   kfree = 294 cycles
kmalloc(256) = 181 cycles   kfree = 304 cycles
kmalloc(512) = 237 cycles   kfree = 327 cycles
kmalloc(1024) = 340 cycles  kfree = 379 cycles
kmalloc(2048) = 378 cycles  kfree = 433 cycles
kmalloc(4096) = 399 cycles  kfree = 329 cycles
kmalloc(8192) = 528 cycles  kfree = 624 cycles
kmalloc(16384) = 651 cycles kfree = 737 cycles

2. Kmalloc: alloc/free test

* cmpxchg_local slub
kmalloc(8)/kfree = 96 cycles
kmalloc(16)/kfree = 97 cycles
kmalloc(32)/kfree = 97 cycles
kmalloc(64)/kfree = 97 cycles
kmalloc(128)/kfree = 97 cycles
kmalloc(256)/kfree = 105 cycles
kmalloc(512)/kfree = 108 cycles
kmalloc(1024)/kfree = 105 cycles
kmalloc(2048)/kfree = 107 cycles
kmalloc(4096)/kfree = 390 cycles
kmalloc(8192)/kfree = 626 cycles
kmalloc(16384)/kfree = 662 cycles

* Base
kmalloc(8)/kfree = 116 cycles
kmalloc(16)/kfree = 116 cycles
kmalloc(32)/kfree = 116 cycles
kmalloc(64)/kfree = 116 cycles
kmalloc(128)/kfree = 116 cycles
kmalloc(256)/kfree = 126 cycles
kmalloc(512)/kfree = 126 cycles
kmalloc(1024)/kfree = 126 cycles
kmalloc(2048)/kfree = 126 cycles
kmalloc(4096)/kfree = 384 cycles
kmalloc(8192)/kfree = 749 cycles
kmalloc(16384)/kfree = 786 cycles


Tested-by: Christoph Lameter <clameter@sgi.com>
I can confirm Mathieus' measurement now:

Athlon64:

regular NUMA/discontig

1. Kmalloc: Repeatedly allocate then free test
10000 times kmalloc(8) -> 79 cycles kfree -> 92 cycles
10000 times kmalloc(16) -> 79 cycles kfree -> 93 cycles
10000 times kmalloc(32) -> 88 cycles kfree -> 95 cycles
10000 times kmalloc(64) -> 124 cycles kfree -> 132 cycles
10000 times kmalloc(128) -> 157 cycles kfree -> 247 cycles
10000 times kmalloc(256) -> 200 cycles kfree -> 257 cycles
10000 times kmalloc(512) -> 250 cycles kfree -> 277 cycles
10000 times kmalloc(1024) -> 337 cycles kfree -> 314 cycles
10000 times kmalloc(2048) -> 365 cycles kfree -> 330 cycles
10000 times kmalloc(4096) -> 352 cycles kfree -> 240 cycles
10000 times kmalloc(8192) -> 456 cycles kfree -> 340 cycles
10000 times kmalloc(16384) -> 646 cycles kfree -> 471 cycles
2. Kmalloc: alloc/free test
10000 times kmalloc(8)/kfree -> 124 cycles
10000 times kmalloc(16)/kfree -> 124 cycles
10000 times kmalloc(32)/kfree -> 124 cycles
10000 times kmalloc(64)/kfree -> 124 cycles
10000 times kmalloc(128)/kfree -> 124 cycles
10000 times kmalloc(256)/kfree -> 132 cycles
10000 times kmalloc(512)/kfree -> 132 cycles
10000 times kmalloc(1024)/kfree -> 132 cycles
10000 times kmalloc(2048)/kfree -> 132 cycles
10000 times kmalloc(4096)/kfree -> 319 cycles
10000 times kmalloc(8192)/kfree -> 486 cycles
10000 times kmalloc(16384)/kfree -> 539 cycles

cmpxchg_local NUMA/discontig

1. Kmalloc: Repeatedly allocate then free test
10000 times kmalloc(8) -> 55 cycles kfree -> 90 cycles
10000 times kmalloc(16) -> 55 cycles kfree -> 92 cycles
10000 times kmalloc(32) -> 70 cycles kfree -> 91 cycles
10000 times kmalloc(64) -> 100 cycles kfree -> 141 cycles
10000 times kmalloc(128) -> 128 cycles kfree -> 233 cycles
10000 times kmalloc(256) -> 172 cycles kfree -> 251 cycles
10000 times kmalloc(512) -> 225 cycles kfree -> 275 cycles
10000 times kmalloc(1024) -> 325 cycles kfree -> 311 cycles
10000 times kmalloc(2048) -> 346 cycles kfree -> 330 cycles
10000 times kmalloc(4096) -> 351 cycles kfree -> 238 cycles
10000 times kmalloc(8192) -> 450 cycles kfree -> 342 cycles
10000 times kmalloc(16384) -> 630 cycles kfree -> 546 cycles
2. Kmalloc: alloc/free test
10000 times kmalloc(8)/kfree -> 81 cycles
10000 times kmalloc(16)/kfree -> 81 cycles
10000 times kmalloc(32)/kfree -> 81 cycles
10000 times kmalloc(64)/kfree -> 81 cycles
10000 times kmalloc(128)/kfree -> 81 cycles
10000 times kmalloc(256)/kfree -> 91 cycles
10000 times kmalloc(512)/kfree -> 90 cycles
10000 times kmalloc(1024)/kfree -> 91 cycles
10000 times kmalloc(2048)/kfree -> 90 cycles
10000 times kmalloc(4096)/kfree -> 318 cycles
10000 times kmalloc(8192)/kfree -> 483 cycles
10000 times kmalloc(16384)/kfree -> 536 cycles


-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
  2007-08-27 17:31   ` Mathieu Desnoyers
@ 2007-08-27 19:29     ` Christoph Lameter
  0 siblings, 0 replies; 41+ messages in thread
From: Christoph Lameter @ 2007-08-27 19:29 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: Andrew Morton, linux-kernel

On Mon, 27 Aug 2007, Mathieu Desnoyers wrote:

> Good point. Christoph, could you please prepare a slub cmpxchg_local
> patch for the mm tree ?

Patch follows (on top of the per cpu structures patch):

Note that we still have a ways to go before the cmpxchg_local 
implementation for slub is ready to be included. Right now we are surely 
causing a regression on non x86 platforms. There needs to be some sort of 
agreement on how to use cmpxchg_local in a way that does not cause harm 
for other architectures.

Once we have that in place then the same method that is applied to the 
SLUB fastpath can also be applied to the vm statistics. I have a patch 
here that does that. We could then also optimize the page allocator to do 
the same.

---
 include/linux/slub_def.h |   10 +++---
 mm/slub.c                |   76 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 58 insertions(+), 28 deletions(-)

Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c	2007-08-22 21:42:40.000000000 -0700
+++ linux-2.6/mm/slub.c	2007-08-22 21:42:53.000000000 -0700
@@ -1442,13 +1442,18 @@ static void *__slab_alloc(struct kmem_ca
 {
 	void **object;
 	struct page *new;
+	unsigned long flags;
 
+	local_irq_save(flags);
+	put_cpu_no_resched();
 	if (!c->page)
+		/* Slab was flushed */
 		goto new_slab;
 
 	slab_lock(c->page);
 	if (unlikely(!node_match(c, node)))
 		goto another_slab;
+
 load_freelist:
 	object = c->page->freelist;
 	if (unlikely(!object))
@@ -1457,11 +1462,16 @@ load_freelist:
 		goto debug;
 
 	object = c->page->freelist;
-	c->freelist = object[c->offset];
 	c->page->inuse = s->objects;
 	c->page->freelist = NULL;
 	c->node = page_to_nid(c->page);
+	c->freelist = object[c->offset];
+out:
 	slab_unlock(c->page);
+	local_irq_restore(flags);
+	preempt_check_resched();
+	if (unlikely((gfpflags & __GFP_ZERO)))
+		memset(object, 0, c->objsize);
 	return object;
 
 another_slab:
@@ -1502,6 +1512,8 @@ new_slab:
 		c->page = new;
 		goto load_freelist;
 	}
+	local_irq_restore(flags);
+	preempt_check_resched();
 	return NULL;
 debug:
 	object = c->page->freelist;
@@ -1511,8 +1523,7 @@ debug:
 	c->page->inuse++;
 	c->page->freelist = object[c->offset];
 	c->node = -1;
-	slab_unlock(c->page);
-	return object;
+	goto out;
 }
 
 /*
@@ -1529,25 +1540,29 @@ static void __always_inline *slab_alloc(
 		gfp_t gfpflags, int node, void *addr)
 {
 	void **object;
-	unsigned long flags;
 	struct kmem_cache_cpu *c;
 
-	local_irq_save(flags);
-	c = get_cpu_slab(s, smp_processor_id());
-	if (unlikely(!c->freelist || !node_match(c, node)))
+	c = get_cpu_slab(s, get_cpu());
+redo:
+	object = c->freelist;
+	if (unlikely(!object))
+		goto slow;
 
-		object = __slab_alloc(s, gfpflags, node, addr, c);
+	if (unlikely(!node_match(c, node)))
+		goto slow;
 
-	else {
-		object = c->freelist;
-		c->freelist = object[c->offset];
-	}
-	local_irq_restore(flags);
+	if (unlikely(cmpxchg_local(&c->freelist, object,
+		object[c->offset]) != object))
+			goto redo;
 
-	if (unlikely((gfpflags & __GFP_ZERO) && object))
+	put_cpu();
+	if (unlikely((gfpflags & __GFP_ZERO)))
 		memset(object, 0, c->objsize);
 
 	return object;
+slow:
+	return __slab_alloc(s, gfpflags, node, addr, c);
+
 }
 
 void *kmem_cache_alloc(struct kmem_cache *s, gfp_t gfpflags)
@@ -1577,7 +1592,10 @@ static void __slab_free(struct kmem_cach
 {
 	void *prior;
 	void **object = (void *)x;
+	unsigned long flags;
 
+	put_cpu();
+	local_irq_save(flags);
 	slab_lock(page);
 
 	if (unlikely(SlabDebug(page)))
@@ -1603,6 +1621,7 @@ checks_ok:
 
 out_unlock:
 	slab_unlock(page);
+	local_irq_restore(flags);
 	return;
 
 slab_empty:
@@ -1613,6 +1632,7 @@ slab_empty:
 		remove_partial(s, page);
 
 	slab_unlock(page);
+	local_irq_restore(flags);
 	discard_slab(s, page);
 	return;
 
@@ -1637,19 +1657,29 @@ static void __always_inline slab_free(st
 			struct page *page, void *x, void *addr)
 {
 	void **object = (void *)x;
-	unsigned long flags;
+	void **freelist;
 	struct kmem_cache_cpu *c;
 
-	local_irq_save(flags);
 	debug_check_no_locks_freed(object, s->objsize);
-	c = get_cpu_slab(s, smp_processor_id());
-	if (likely(page == c->page && c->node >= 0)) {
-		object[c->offset] = c->freelist;
-		c->freelist = object;
-	} else
-		__slab_free(s, page, x, addr, c->offset);
 
-	local_irq_restore(flags);
+	c = get_cpu_slab(s, get_cpu());
+	if (unlikely(c->node < 0))
+		goto slow;
+redo:
+	freelist = c->freelist;
+	barrier();	/* If interrupt changes c->page -> cmpxchg failure */
+	if (unlikely(page != c->page))
+		goto slow;
+
+	object[c->offset] = freelist;
+	if (unlikely(cmpxchg_local(&c->freelist, freelist, object)
+							!= freelist))
+		goto redo;
+
+	put_cpu();
+	return;
+slow:
+	__slab_free(s, page, x, addr, c->offset);
 }
 
 void kmem_cache_free(struct kmem_cache *s, void *x)
Index: linux-2.6/include/linux/slub_def.h
===================================================================
--- linux-2.6.orig/include/linux/slub_def.h	2007-08-22 21:42:39.000000000 -0700
+++ linux-2.6/include/linux/slub_def.h	2007-08-22 21:42:43.000000000 -0700
@@ -12,11 +12,11 @@
 #include <linux/kobject.h>
 
 struct kmem_cache_cpu {
-	void **freelist;
-	struct page *page;
-	int node;
-	unsigned int offset;
-	unsigned int objsize;
+	void **freelist;	/* Updated through local atomic ops */
+	struct page *page;	/* Updated with interrupts disabled */
+	int node;		/* Updated with interrupts disabled */
+	unsigned int offset;	/* Set up on kmem_cache_create() */
+	unsigned int objsize;	/* Set up on kmem_cache_create() */
 };
 
 struct kmem_cache_node {

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64
  2007-08-27 15:52 ` [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64 Mathieu Desnoyers
@ 2007-08-27 19:32   ` Christoph Lameter
  0 siblings, 0 replies; 41+ messages in thread
From: Christoph Lameter @ 2007-08-27 19:32 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel, tony.luck, Keith Owens

On Mon, 27 Aug 2007, Mathieu Desnoyers wrote:

> Add the primitives cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64. They
> use cmpxchg_acq as underlying macro, just like the already existing ia64
> cmpxchg().

Acked-by: Christoph Lameter <clameter@sgi.com>

Note cmpxchg is *not* a performance win vs interrupt disable / enable on IA64.


^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
  2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
                   ` (28 preceding siblings ...)
  2007-08-27 16:17 ` [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Andrew Morton
@ 2007-08-27 19:35 ` Christoph Lameter
  2007-08-27 20:19   ` Mathieu Desnoyers
  29 siblings, 1 reply; 41+ messages in thread
From: Christoph Lameter @ 2007-08-27 19:35 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel

On Mon, 27 Aug 2007, Mathieu Desnoyers wrote:

> Here is the patch series for 2.6.23-rc3-mm1 that adds cmpxchg_local, and now
> also cmpxchg64_local, to each architecture.

Any planned uses of cmpxchg64_local? 

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture
  2007-08-27 19:35 ` Christoph Lameter
@ 2007-08-27 20:19   ` Mathieu Desnoyers
  0 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-27 20:19 UTC (permalink / raw)
  To: Christoph Lameter; +Cc: akpm, linux-kernel

* Christoph Lameter (clameter@sgi.com) wrote:
> On Mon, 27 Aug 2007, Mathieu Desnoyers wrote:
> 
> > Here is the patch series for 2.6.23-rc3-mm1 that adds cmpxchg_local, and now
> > also cmpxchg64_local, to each architecture.
> 
> Any planned uses of cmpxchg64_local? 

Let's say it could possibly eliminate the xtime read seqlock on x86
(should be investigated) and can be useful to overcome the
non-synchronized TSC handling when taking TSC-based timestamps. All
these operations require locking to play with 64 bits values on 32 bits
architectures, which could be improved by cmpxchg64_local.

But again, it would require some testing to see if we can improve
performances there.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486
  2007-08-27 15:52 ` [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486 Mathieu Desnoyers
@ 2007-08-28  1:40   ` Nick Piggin
  2007-08-28 11:31     ` Mathieu Desnoyers
  0 siblings, 1 reply; 41+ messages in thread
From: Nick Piggin @ 2007-08-28  1:40 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel, clameter, mingo

Mathieu Desnoyers wrote:

> Q:
> What's the reason to have cmpxchg64_local on 32 bit architectures?
> Without that need all this would just be a few simple defines.
> 
> A:
> cmpxchg64_local on 32 bits architectures takes unsigned long long
> parameters, but cmpxchg_local only takes longs. Since we have cmpxchg8b
> to execute a 8 byte cmpxchg atomically on pentium and +, it makes sense
> to provide a flavor of cmpxchg and cmpxchg_local using this instruction.
> 
> Also, for 32 bits architectures lacking the 64 bits atomic cmpxchg, it
> makes sense _not_ to define cmpxchg64 while cmpxchg could still be
> available.
> 
> Moreover, the fallback for cmpxchg8b on i386 for 386 and 486 is a
> different case than cmpxchg (which is only required for 386). Using
> different code makes this easier.
> 
> However, cmpxchg64_local will be emulated by disabling interrupts on all
> architectures where it is not supported atomically.
> 
> Therefore, we *could* turn cmpxchg64_local into a cmpxchg_local, but it
> would make the 386/486 fallbacks ugly, make its design different from
> cmpxchg/cmpxchg64 (which really depends on atomic operations and cannot
> be emulated) and require the __cmpxchg_local to be expressed as a macro
> rather than an inline function so the parameters would not be fixed to
> unsigned long long in every case.
> 
> So I think cmpxchg64_local makes sense there, but I am open to
> suggestions.

Every new thing like this (especially 64 bit operation on 32 bit
architectures) adds a tiny bit more burden for maintainers. Are
there any callers? If not, don't add it. It's simple to add if we
do get a good reason.

-- 
SUSE Labs, Novell Inc.

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486
  2007-08-28  1:40   ` Nick Piggin
@ 2007-08-28 11:31     ` Mathieu Desnoyers
  2007-08-28 23:51       ` Nick Piggin
  0 siblings, 1 reply; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-08-28 11:31 UTC (permalink / raw)
  To: Nick Piggin; +Cc: akpm, linux-kernel, clameter, mingo

* Nick Piggin (nickpiggin@yahoo.com.au) wrote:
> Mathieu Desnoyers wrote:
> 
> >Q:
> >What's the reason to have cmpxchg64_local on 32 bit architectures?
> >Without that need all this would just be a few simple defines.
> >
> >A:
> >cmpxchg64_local on 32 bits architectures takes unsigned long long
> >parameters, but cmpxchg_local only takes longs. Since we have cmpxchg8b
> >to execute a 8 byte cmpxchg atomically on pentium and +, it makes sense
> >to provide a flavor of cmpxchg and cmpxchg_local using this instruction.
> >
> >Also, for 32 bits architectures lacking the 64 bits atomic cmpxchg, it
> >makes sense _not_ to define cmpxchg64 while cmpxchg could still be
> >available.
> >
> >Moreover, the fallback for cmpxchg8b on i386 for 386 and 486 is a
> >different case than cmpxchg (which is only required for 386). Using
> >different code makes this easier.
> >
> >However, cmpxchg64_local will be emulated by disabling interrupts on all
> >architectures where it is not supported atomically.
> >
> >Therefore, we *could* turn cmpxchg64_local into a cmpxchg_local, but it
> >would make the 386/486 fallbacks ugly, make its design different from
> >cmpxchg/cmpxchg64 (which really depends on atomic operations and cannot
> >be emulated) and require the __cmpxchg_local to be expressed as a macro
> >rather than an inline function so the parameters would not be fixed to
> >unsigned long long in every case.
> >
> >So I think cmpxchg64_local makes sense there, but I am open to
> >suggestions.
> 
> Every new thing like this (especially 64 bit operation on 32 bit
> architectures) adds a tiny bit more burden for maintainers. Are
> there any callers? If not, don't add it. It's simple to add if we
> do get a good reason.
> 

I am actually using it in LTTng in my timestamping code. I use it to
work around CPUs with asynchronous TSCs. I need to update 64 bits
values atomically on this 32 bits architecture.

I plan to submit this timestamping code soon.

Mathieu


-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips
  2007-08-27 15:52 ` [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips Mathieu Desnoyers
@ 2007-08-28 13:58   ` Ralf Baechle
  0 siblings, 0 replies; 41+ messages in thread
From: Ralf Baechle @ 2007-08-28 13:58 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel

On Mon, Aug 27, 2007 at 11:52:37AM -0400, Mathieu Desnoyers wrote:

> Make sure that at least cmpxchg64_local is available on all architectures to use
> for unsigned long long values.

Queued for 2.6.24.  Thanks,

  Ralf

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486
  2007-08-28 11:31     ` Mathieu Desnoyers
@ 2007-08-28 23:51       ` Nick Piggin
  0 siblings, 0 replies; 41+ messages in thread
From: Nick Piggin @ 2007-08-28 23:51 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel, clameter, mingo

Mathieu Desnoyers wrote:
> * Nick Piggin (nickpiggin@yahoo.com.au) wrote:
> 
>>Mathieu Desnoyers wrote:
>>
>>
>>>Q:
>>>What's the reason to have cmpxchg64_local on 32 bit architectures?
>>>Without that need all this would just be a few simple defines.
>>>
>>>A:
>>>cmpxchg64_local on 32 bits architectures takes unsigned long long
>>>parameters, but cmpxchg_local only takes longs. Since we have cmpxchg8b
>>>to execute a 8 byte cmpxchg atomically on pentium and +, it makes sense
>>>to provide a flavor of cmpxchg and cmpxchg_local using this instruction.
>>>
>>>Also, for 32 bits architectures lacking the 64 bits atomic cmpxchg, it
>>>makes sense _not_ to define cmpxchg64 while cmpxchg could still be
>>>available.
>>>
>>>Moreover, the fallback for cmpxchg8b on i386 for 386 and 486 is a
>>>different case than cmpxchg (which is only required for 386). Using
>>>different code makes this easier.
>>>
>>>However, cmpxchg64_local will be emulated by disabling interrupts on all
>>>architectures where it is not supported atomically.
>>>
>>>Therefore, we *could* turn cmpxchg64_local into a cmpxchg_local, but it
>>>would make the 386/486 fallbacks ugly, make its design different from
>>>cmpxchg/cmpxchg64 (which really depends on atomic operations and cannot
>>>be emulated) and require the __cmpxchg_local to be expressed as a macro
>>>rather than an inline function so the parameters would not be fixed to
>>>unsigned long long in every case.
>>>
>>>So I think cmpxchg64_local makes sense there, but I am open to
>>>suggestions.
>>
>>Every new thing like this (especially 64 bit operation on 32 bit
>>architectures) adds a tiny bit more burden for maintainers. Are
>>there any callers? If not, don't add it. It's simple to add if we
>>do get a good reason.
>>
> 
> 
> I am actually using it in LTTng in my timestamping code. I use it to
> work around CPUs with asynchronous TSCs. I need to update 64 bits
> values atomically on this 32 bits architecture.
> 
> I plan to submit this timestamping code soon.

OK fair enough. So long as there is a user (and you are sure said
user is going to get upstream -- sometimes it is easier to put
this patchset in with the one that is going to call it, but OTOH
that can turn people off reviewing).

-- 
SUSE Labs, Novell Inc.

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc
  2007-08-27 15:52 ` [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc Mathieu Desnoyers
@ 2007-09-22  4:46   ` Paul Mackerras
  2007-09-22 14:35     ` Mathieu Desnoyers
  0 siblings, 1 reply; 41+ messages in thread
From: Paul Mackerras @ 2007-09-22  4:46 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: akpm, linux-kernel, linuxppc-dev

Mathieu Desnoyers writes:

> Make sure that at least cmpxchg64_local is available on all architectures to use
> for unsigned long long values.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>

Acked-by: Paul Mackerras <paulus@samba.org>

^ permalink raw reply	[flat|nested] 41+ messages in thread

* Re: [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc
  2007-09-22  4:46   ` Paul Mackerras
@ 2007-09-22 14:35     ` Mathieu Desnoyers
  0 siblings, 0 replies; 41+ messages in thread
From: Mathieu Desnoyers @ 2007-09-22 14:35 UTC (permalink / raw)
  To: Paul Mackerras; +Cc: akpm, linux-kernel, linuxppc-dev

* Paul Mackerras (paulus@samba.org) wrote:
> Mathieu Desnoyers writes:
> 
> > Make sure that at least cmpxchg64_local is available on all architectures to use
> > for unsigned long long values.
> > 
> > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> 
> Acked-by: Paul Mackerras <paulus@samba.org>

Thanks Paul,

Just make sure add-cmpxchg-local-to-generic-for-up.patch gets merged
before this one, since it needs asm-generic/cmpxchg-local.h.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 41+ messages in thread

end of thread, other threads:[~2007-09-22 14:41 UTC | newest]

Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-08-27 15:52 [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Mathieu Desnoyers
2007-08-27 15:52 ` [patch 01/28] Fall back on interrupt disable in cmpxchg8b on 80386 and 80486 Mathieu Desnoyers
2007-08-28  1:40   ` Nick Piggin
2007-08-28 11:31     ` Mathieu Desnoyers
2007-08-28 23:51       ` Nick Piggin
2007-08-27 15:52 ` [patch 02/28] Add cmpxchg64 and cmpxchg64_local to alpha Mathieu Desnoyers
2007-08-27 15:52 ` [patch 03/28] Add cmpxchg64 and cmpxchg64_local to mips Mathieu Desnoyers
2007-08-28 13:58   ` Ralf Baechle
2007-08-27 15:52 ` [patch 04/28] Add cmpxchg64 and cmpxchg64_local to powerpc Mathieu Desnoyers
2007-09-22  4:46   ` Paul Mackerras
2007-09-22 14:35     ` Mathieu Desnoyers
2007-08-27 15:52 ` [patch 05/28] Add cmpxchg64 and cmpxchg64_local to x86_64 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 06/28] Add cmpxchg_local to asm-generic for per cpu atomic operations Mathieu Desnoyers
2007-08-27 15:52 ` [patch 07/28] Add cmpxchg_local to arm Mathieu Desnoyers
2007-08-27 15:52 ` [patch 08/28] Add cmpxchg_local to avr32 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 09/28] Add cmpxchg_local to blackfin, replace __cmpxchg by generic cmpxchg Mathieu Desnoyers
2007-08-27 15:52 ` [patch 10/28] Add cmpxchg_local to cris Mathieu Desnoyers
2007-08-27 15:52 ` [patch 11/28] Add cmpxchg_local to frv Mathieu Desnoyers
2007-08-27 15:52 ` [patch 12/28] Add cmpxchg_local to h8300 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 13/28] Add cmpxchg_local, cmpxchg64 and cmpxchg64_local to ia64 Mathieu Desnoyers
2007-08-27 19:32   ` Christoph Lameter
2007-08-27 15:52 ` [patch 14/28] New cmpxchg_local (optimized for UP case) for m32r Mathieu Desnoyers
2007-08-27 15:52 ` [patch 15/28] Fix m32r __xchg Mathieu Desnoyers
2007-08-27 15:52 ` [patch 16/28] m32r: build fix of arch/m32r/kernel/smpboot.c Mathieu Desnoyers
2007-08-27 15:52 ` [patch 17/28] local_t m32r use architecture specific cmpxchg_local Mathieu Desnoyers
2007-08-27 15:52 ` [patch 18/28] Add cmpxchg_local to m86k Mathieu Desnoyers
2007-08-27 15:52 ` [patch 19/28] Add cmpxchg_local to m68knommu Mathieu Desnoyers
2007-08-27 15:52 ` [patch 20/28] Add cmpxchg_local to parisc Mathieu Desnoyers
2007-08-27 15:52 ` [patch 21/28] Add cmpxchg_local to ppc Mathieu Desnoyers
2007-08-27 15:52 ` [patch 22/28] Add cmpxchg_local to s390 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 23/28] Add cmpxchg_local to sh, use generic cmpxchg() instead of cmpxchg_u32 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 24/28] Add cmpxchg_local to sh64 Mathieu Desnoyers
2007-08-27 15:52 ` [patch 25/28] Add cmpxchg_local to sparc, move __cmpxchg to system.h Mathieu Desnoyers
2007-08-27 15:53 ` [patch 26/28] Add cmpxchg_local to sparc64 Mathieu Desnoyers
2007-08-27 15:53 ` [patch 27/28] Add cmpxchg_local to v850 Mathieu Desnoyers
2007-08-27 15:53 ` [patch 28/28] Add cmpxchg_local to xtensa Mathieu Desnoyers
2007-08-27 16:17 ` [patch 00/28] Add cmpxchg64_local and cmpxchg_local to each architecture Andrew Morton
2007-08-27 17:31   ` Mathieu Desnoyers
2007-08-27 19:29     ` Christoph Lameter
2007-08-27 19:35 ` Christoph Lameter
2007-08-27 20:19   ` Mathieu Desnoyers

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).