Re: Linux 2.4.11-pre4

Re: Linux 2.4.11-pre4

[Dieter Nützel]

Hello Linus,

the new and very cool Multiquad NUMA stuff break something...

gcc -D__KERNEL__ -I/usr/src/linux-2.4.11-pre4-preempt/include -Wall 
-Wstrict-prototypes -Wno-trigraphs -O -fomit-frame-pointer 
-fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -mcpu=k6 
-mpreferred-stack-boundary=2 -malign-functions=4 -fschedule-insns2 
-fexpensive-optimizations     -c -o mpparse.o mpparse.c
mpparse.c: In function `MP_processor_info':
mpparse.c:195: `clustered_apic_mode' undeclared (first use in this function)
mpparse.c:195: (Each undeclared identifier is reported only once
mpparse.c:195: for each function it appears in.)
mpparse.c: In function `smp_read_mpc':
mpparse.c:386: `clustered_apic_mode' undeclared (first use in this function)
make[1]: *** [mpparse.o] Error 1
make[1]: Leaving directory 
`/usr/src/linux-2.4.11-pre4-preempt/arch/i386/kernel'
make: *** [_dir_arch/i386/kernel] Error 2
269.520u 29.200s 5:45.26 86.5%  0+0k 0+0io 1007275pf+0w

-Dieter

Re: Linux 2.4.11-pre4

[Martin J. Bligh]

Odd. Compiles for me with and without SMP support turned on.
Does "linux-2.4.11-pre4-preempt" mean you're running with other
patches over the raw kernel? Which ones - just the preempt patch?
Can you point me to them?

Martin

PS. If you had other patches, can you try compiling just raw 2.4.11-pre4, and tell me if that works?

mpparse.c does this:

#include <asm/smp.h>

smp.h does this:

#ifndef clustered_apic_mode
 #ifdef CONFIG_MULTIQUAD
  #define clustered_apic_mode (1)
  #define esr_disable (1)
 #else /* !CONFIG_MULTIQUAD */
  #define clustered_apic_mode (0)
  #define esr_disable (0)
 #endif /* CONFIG_MULTIQUAD */
#endif

Or at least it should .....

--On Friday, October 05, 2001 6:38 AM +0200 Dieter Nützel <Dieter.Nuetzel@hamburg.de> wrote:

> Hello Linus,
> > the new and very cool Multiquad NUMA stuff break something...
> > gcc -D__KERNEL__ -I/usr/src/linux-2.4.11-pre4-preempt/include -Wall > -Wstrict-prototypes -Wno-trigraphs -O -fomit-frame-pointer > -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -mcpu=k6 > -mpreferred-stack-boundary=2 -malign-functions=4 -fschedule-insns2 > -fexpensive-optimizations     -c -o mpparse.o mpparse.c
> mpparse.c: In function `MP_processor_info':
> mpparse.c:195: `clustered_apic_mode' undeclared (first use in this function)
> mpparse.c:195: (Each undeclared identifier is reported only once
> mpparse.c:195: for each function it appears in.)
> mpparse.c: In function `smp_read_mpc':
> mpparse.c:386: `clustered_apic_mode' undeclared (first use in this function)
> make[1]: *** [mpparse.o] Error 1
> make[1]: Leaving directory > `/usr/src/linux-2.4.11-pre4-preempt/arch/i386/kernel'
> make: *** [_dir_arch/i386/kernel] Error 2
> 269.520u 29.200s 5:45.26 86.5%  0+0k 0+0io 1007275pf+0w
> > -Dieter
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Linux 2.4.11-pre4

[Martin J. Bligh]

> Odd. Compiles for me with and without SMP support turned on.

My fault. I'd tested this on SMP and Uniproc, but not uniproc with
IO apic support. Try this patch:

--- smp.h.old	Fri Oct  5 10:46:40 2001
+++ smp.h	Fri Oct  5 10:48:37 2001
@@ -31,9 +31,20 @@
 #  define INT_DELIVERY_MODE 1     /* logical delivery broadcast to all procs */
 # endif
 #else
+# define INT_DELIVERY_MODE 0     /* physical delivery on LOCAL quad */
 # define TARGET_CPUS 0x01
 #endif
 
+#ifndef clustered_apic_mode
+ #ifdef CONFIG_MULTIQUAD
+  #define clustered_apic_mode (1)
+  #define esr_disable (1)
+ #else /* !CONFIG_MULTIQUAD */
+  #define clustered_apic_mode (0)
+  #define esr_disable (0)
+ #endif /* CONFIG_MULTIQUAD */
+#endif 
+
 #ifdef CONFIG_SMP
 #ifndef ASSEMBLY
 
@@ -76,16 +87,6 @@
 extern volatile int physical_apicid_to_cpu[MAX_APICID];
 extern volatile int cpu_to_logical_apicid[NR_CPUS];
 extern volatile int logical_apicid_to_cpu[MAX_APICID];
-
-#ifndef clustered_apic_mode
- #ifdef CONFIG_MULTIQUAD
-  #define clustered_apic_mode (1)
-  #define esr_disable (1)
- #else /* !CONFIG_MULTIQUAD */
-  #define clustered_apic_mode (0)
-  #define esr_disable (0)
- #endif /* CONFIG_MULTIQUAD */
-#endif 
 
 /*
  * General functions that each host system must provide.

Re: Linux 2.4.11-pre4

[Dieter Nützel]

Am Freitag, 5. Oktober 2001 18:25 schrieb Martin J. Bligh:
> Odd. Compiles for me with and without SMP support turned on.

UP only. But with UP IOAPIC.

#
# Processor type and features
#
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
CONFIG_MK7=y
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_USE_3DNOW=y
CONFIG_X86_PGE=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
# CONFIG_TOSHIBA is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_HAVE_DEC_LOCK=y

> Does "linux-2.4.11-pre4-preempt" mean you're running with other
> patches over the raw kernel? Which ones - just the preempt patch?

Yes.

First:
patch-rml-2.4.11-pre2-preempt-kernel-1
It works with pre2 and pre3 (after some clean ups by hand)
http://www.tech9.net/rml/linux/

Second:
Kupdated-patch for ReiserFS but that shouldn't matter.
No link handy. I can send it to you if you need it.

> Can you point me to them?

See above.

>
> Martin
>
> PS. If you had other patches, can you try compiling just raw 2.4.11-pre4,
> and tell me if that works?

Only the two above.
I'll try tomorrow if it's needed.

Ughh.
I have a "dbench 32" loop with 1000 (one thousend) cycles running on an IBM 
DDYS 18GB U160 test disk running. 'cause my own (same but younger, latest 
firmware Rev. S96H) spin down occasionally. I think it have to be replaced:-(
Then it is the third one. My first show "extremely shocked error" code with 
the DFT IBM tool.

System load stays at 29~32 (through put is between 40-45 MB/sec) for some 
hours, now and during I write this text...
Andrea's VM plus preempt-patch is GREAT.

Greetings,
	Dieter

> mpparse.c does this:
>
> #include <asm/smp.h>
>
> smp.h does this:
>
> #ifndef clustered_apic_mode
>  #ifdef CONFIG_MULTIQUAD
>   #define clustered_apic_mode (1)
>   #define esr_disable (1)
>  #else /* !CONFIG_MULTIQUAD */
>   #define clustered_apic_mode (0)
>   #define esr_disable (0)
>  #endif /* CONFIG_MULTIQUAD */
> #endif
>
> Or at least it should .....
>
> --On Friday, October 05, 2001 6:38 AM +0200 Dieter Nützel 
<Dieter.Nuetzel@hamburg.de> wrote:
> > Hello Linus,
> >
> > > the new and very cool Multiquad NUMA stuff break something...
> > > gcc -D__KERNEL__ -I/usr/src/linux-2.4.11-pre4-preempt/include -Wall >
> > > -Wstrict-prototypes -Wno-trigraphs -O -fomit-frame-pointer >
> > > -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2
> > > -mcpu=k6 > -mpreferred-stack-boundary=2 -malign-functions=4
> > > -fschedule-insns2 > -fexpensive-optimizations     -c -o mpparse.o
> > > mpparse.c
> >
> > mpparse.c: In function `MP_processor_info':
> > mpparse.c:195: `clustered_apic_mode' undeclared (first use in this
> > function) mpparse.c:195: (Each undeclared identifier is reported only
> > once mpparse.c:195: for each function it appears in.)
> > mpparse.c: In function `smp_read_mpc':
> > mpparse.c:386: `clustered_apic_mode' undeclared (first use in this
> > function) make[1]: *** [mpparse.o] Error 1
> > make[1]: Leaving directory >
> > `/usr/src/linux-2.4.11-pre4-preempt/arch/i386/kernel' make: ***
> > [_dir_arch/i386/kernel] Error 2
> > 269.520u 29.200s 5:45.26 86.5%  0+0k 0+0io 1007275pf+0w
> >
> > > -Dieter
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> > in the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/

-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
Cognitive Systems Group
Vogt-Kölln-Straße 30
D-22527 Hamburg, Germany

email: nuetzel@kogs.informatik.uni-hamburg.de
@home: Dieter.Nuetzel@hamburg.de

Re: Linux 2.4.11-pre4

[Dieter Nützel]

Am Freitag, 5. Oktober 2001 19:54 schrieb Martin J. Bligh:
> > Odd. Compiles for me with and without SMP support turned on.
>
> My fault. I'd tested this on SMP and Uniproc, but not uniproc with
> IO apic support. Try this patch:

Yes, I have UP with UP_IOAPIC enabled.
Shall I test it or run my disk test over night?

Thanks for your fast fix.
I have some short hiccup during this "test site".
So I can't type that fast...

-Dieter

> --- smp.h.old	Fri Oct  5 10:46:40 2001
> +++ smp.h	Fri Oct  5 10:48:37 2001
> @@ -31,9 +31,20 @@
>  #  define INT_DELIVERY_MODE 1     /* logical delivery broadcast to all
> procs */ # endif
>  #else
> +# define INT_DELIVERY_MODE 0     /* physical delivery on LOCAL quad */
>  # define TARGET_CPUS 0x01
>  #endif
>
> +#ifndef clustered_apic_mode
> + #ifdef CONFIG_MULTIQUAD
> +  #define clustered_apic_mode (1)
> +  #define esr_disable (1)
> + #else /* !CONFIG_MULTIQUAD */
> +  #define clustered_apic_mode (0)
> +  #define esr_disable (0)
> + #endif /* CONFIG_MULTIQUAD */
> +#endif
> +
>  #ifdef CONFIG_SMP
>  #ifndef ASSEMBLY
>
> @@ -76,16 +87,6 @@
>  extern volatile int physical_apicid_to_cpu[MAX_APICID];
>  extern volatile int cpu_to_logical_apicid[NR_CPUS];
>  extern volatile int logical_apicid_to_cpu[MAX_APICID];
> -
> -#ifndef clustered_apic_mode
> - #ifdef CONFIG_MULTIQUAD
> -  #define clustered_apic_mode (1)
> -  #define esr_disable (1)
> - #else /* !CONFIG_MULTIQUAD */
> -  #define clustered_apic_mode (0)
> -  #define esr_disable (0)
> - #endif /* CONFIG_MULTIQUAD */
> -#endif
>
>  /*
>   * General functions that each host system must provide.

-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
Cognitive Systems Group
Vogt-Kölln-Straße 30
D-22527 Hamburg, Germany

email: nuetzel@kogs.informatik.uni-hamburg.de
@home: Dieter.Nuetzel@hamburg.de

Re: Linux 2.4.11-pre4

[Dieter Nützel]

Am Freitag, 5. Oktober 2001 19:54 schrieb Martin J. Bligh:
> > Odd. Compiles for me with and without SMP support turned on.
>
> My fault. I'd tested this on SMP and Uniproc, but not uniproc with
> IO apic support. Try this patch:

1. OK, it fixes the UP UP_IOAPIC compilation problem.
System (with preempt-patch) up and runnig.

2. Woohu. I have 8 CPUs, now...;-)
--- /proc is somewhat broken

[-]
processor       : 7
vendor_id       : 9—U0D'À
cpu family      : 1
model           : 0
model name      : 0D'À
stepping        : unknown
cache size      : 0 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : -1071168528
wp              : yes
flags           : fpu syscall mmxext lm 3dnowext 3dnow cxmmx
bogomips        : 0.00
[-]

3. OOmem deak look.

4. Back in disk stress mode...again.

-Dieter

> --- smp.h.old	Fri Oct  5 10:46:40 2001
> +++ smp.h	Fri Oct  5 10:48:37 2001
> @@ -31,9 +31,20 @@
>  #  define INT_DELIVERY_MODE 1     /* logical delivery broadcast to all
> procs */ # endif
>  #else
> +# define INT_DELIVERY_MODE 0     /* physical delivery on LOCAL quad */
>  # define TARGET_CPUS 0x01
>  #endif
>
> +#ifndef clustered_apic_mode
> + #ifdef CONFIG_MULTIQUAD
> +  #define clustered_apic_mode (1)
> +  #define esr_disable (1)
> + #else /* !CONFIG_MULTIQUAD */
> +  #define clustered_apic_mode (0)
> +  #define esr_disable (0)
> + #endif /* CONFIG_MULTIQUAD */
> +#endif
> +
>  #ifdef CONFIG_SMP
>  #ifndef ASSEMBLY
>
> @@ -76,16 +87,6 @@
>  extern volatile int physical_apicid_to_cpu[MAX_APICID];
>  extern volatile int cpu_to_logical_apicid[NR_CPUS];
>  extern volatile int logical_apicid_to_cpu[MAX_APICID];
> -
> -#ifndef clustered_apic_mode
> - #ifdef CONFIG_MULTIQUAD
> -  #define clustered_apic_mode (1)
> -  #define esr_disable (1)
> - #else /* !CONFIG_MULTIQUAD */
> -  #define clustered_apic_mode (0)
> -  #define esr_disable (0)
> - #endif /* CONFIG_MULTIQUAD */
> -#endif
>
>  /*
>   * General functions that each host system must provide.

Re: Linux 2.4.11-pre4

[Dieter Nützel]

Am Freitag, 5. Oktober 2001 23:35 schrieb Martin J. Bligh:
> > 1. OK, it fixes the UP UP_IOAPIC compilation problem.
> > System (with preempt-patch) up and runnig.
>
> Good.
>
> > 2. Woohu. I have 8 CPUs, now...;-)
> > --- /proc is somewhat broken
>
> Bugger. Didn't realise that cpu_online_map didn't get initialised
> to anything sensible under UP. Should be just cosmetic (it's only
> the output of /proc/cpuinfo, not the sceduler or anything), but
> try this (I haven't tested it yet - if it doesn't work, just change the
> 8 to 1 for a second whilst I fix it properly).

The first version works nice.

Have a nice weekend.

-Dieter


> ===========================
>
> --- setup.c.old	Fri Oct  5 14:20:29 2001
> +++ setup.c	Fri Oct  5 14:28:51 2001
> @@ -2420,7 +2420,7 @@
>  	 * WARNING - nasty evil hack ... if we print > 8, it overflows the
>  	 * page buffer and corrupts memory - this needs fixing properly
>  	 */
> -	for (n = 0; n < 8; n++, c++) {
> +	for (n = 0; n < (clustered_apic_mode ? 8 : NR_CPUS); n++, c++) {
>  	/* for (n = 0; n < NR_CPUS; n++, c++) { */
>  		int fpu_exception;
>  #ifdef CONFIG_SMP
>
> ===========================
>
> The reason for this hackery is that get_cpuinfo writes to a page
> without proper bounds on itself. If you have more than about 8
> cpus, it tramples merrily all over the next page, corrupting page
> tables, etc, etc.
>
> The real fix for this overflow was published here a few weeks ago
> by James Cleverdon (whom I work with). It's in Alan's tree, but not
> Linus' as yet.
>
> M.

Re: Linux 2.4.11-pre4

[Martin J. Bligh]

> 1. OK, it fixes the UP UP_IOAPIC compilation problem.
> System (with preempt-patch) up and runnig.

Good. 
 
> 2. Woohu. I have 8 CPUs, now...;-)
> --- /proc is somewhat broken

Bugger. Didn't realise that cpu_online_map didn't get initialised
to anything sensible under UP. Should be just cosmetic (it's only
the output of /proc/cpuinfo, not the sceduler or anything), but 
try this (I haven't tested it yet - if it doesn't work, just change the
8 to 1 for a second whilst I fix it properly).

===========================

--- setup.c.old	Fri Oct  5 14:20:29 2001
+++ setup.c	Fri Oct  5 14:28:51 2001
@@ -2420,7 +2420,7 @@
 	 * WARNING - nasty evil hack ... if we print > 8, it overflows the
 	 * page buffer and corrupts memory - this needs fixing properly
 	 */
-	for (n = 0; n < 8; n++, c++) {
+	for (n = 0; n < (clustered_apic_mode ? 8 : NR_CPUS); n++, c++) {
 	/* for (n = 0; n < NR_CPUS; n++, c++) { */
 		int fpu_exception;
 #ifdef CONFIG_SMP

===========================

The reason for this hackery is that get_cpuinfo writes to a page
without proper bounds on itself. If you have more than about 8
cpus, it tramples merrily all over the next page, corrupting page
tables, etc, etc. 

The real fix for this overflow was published here a few weeks ago
by James Cleverdon (whom I work with). It's in Alan's tree, but not
Linus' as yet.

M.

Linux 2.4.11-pre4

[Linus Torvalds]

Continued merging from various sources..

		Linus

----

pre4:
 - Al Viro: separate out superblocks and FS namespaces: fs/super.c fathers
   fs/namespace.c
 - David Woodhouse: large MTD and JFFS[2] update
 - Marcelo Tosatti: resurrect oom handling
 - Hugh Dickins: add_to_swap_cache racefix cleanup
 - Jean Tourrilhes: IrDA update
 - Martin Bligh: support clustered logical APIC for >8 CPU x86 boxes
 - Richard Henderson: alpha update

pre3:
 - Al Viro: superblock cleanups, partition handling fixes and cleanups
 - Ben Collins: firewire update
 - Jeff Garzik: network driver updates
 - Urban Widmark: smbfs updates
 - Kai Mäkisara: SCSI tape driver update
 - various: embarrassing lack of error checking in ELF loader
 - Neil Brown: md formatting cleanup.

pre2:
 - me/Al Viro: fix bdget() oops with block device modules that don't
   clean up after they exit
 - Alan Cox: continued merging (drivers, license tags)
 - David Miller: sparc update, network fixes
 - Christoph Hellwig: work around broken drivers that add a gendisk more
   than once
 - Jakub Jelinek: handle more ELF loading special cases
 - Trond Myklebust: NFS client and lockd reclaimer cleanups/fixes
 - Greg KH: USB updates
 - Mikael Pettersson: sparate out local APIC / IO-APIC config options

pre1:
 - Chris Mason: fix ppp race conditions
 - me: buffers-in-pagecache coherency, buffer.c cleanups
 - Al Viro: block device cleanups/fixes
 - Anton Altaparmakov: NTFS 1.1.20 update
 - Andrea Arcangeli: VM tweaks