[2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Tetsuo Handa]

Hello.

I found the CONFIG_DEBUG_LOCKING_API_SELFTESTS hangs
at (A-A deadlock)-(wlock) pair.

2.6.24-rc8 doesn't hang.
So, I think the bug is in 2.6.24-rc8-mm1.bz2 .

The kernel configs are at

  http://I-love.SAKURA.ne.jp/tmp/config-2.6.24-rc8
  http://I-love.SAKURA.ne.jp/tmp/config-2.6.24-rc8-mm1

Regards.

---------- 2.6.24-rc8-mm1 ----------
Linux version 2.6.24-rc8-mm1 (root@tomoyo) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #2 SMP Fri Jan 18 10:13:09 JST 2008
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
 BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
 BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000001fef0000 (usable)
 BIOS-e820: 000000001fef0000 - 000000001feff000 (ACPI data)
 BIOS-e820: 000000001feff000 - 000000001ff00000 (ACPI NVS)
 BIOS-e820: 000000001ff00000 - 0000000020000000 (usable)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
512MB LOWMEM available.
found SMP MP-table at 000f6c90
Zone PFN ranges:
  DMA             0 ->     4096
  Normal       4096 ->   131072
  HighMem    131072 ->   131072
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0:        0 ->   131072
DMI present.
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: INTEL    Product ID: 440BX        APIC at: 0xFEE00000
Processor #0 6:15 APIC version 17
Processor #1 6:15 APIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Enabling APIC mode:  Flat.  Using 1 I/O APICs
Processors: 2
Allocating PCI resources starting at 30000000 (gap: 20000000:dec00000)
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/sda1 ro noapic nolapic console=ttyS0,115200n8
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c0523000 soft=c0521000
PID hash table entries: 2048 (order: 11, 8192 bytes)
Detected 1993.105 MHz processor.
Console: colour VGA+ 80x25
console [ttyS0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          30
... MAX_LOCKDEP_KEYS:        2048
... CLASSHASH_SIZE:           1024
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 1024 kB
 per task-struct memory footprint: 1680 bytes
------------------------
| Locking API testsuite:
----------------------------------------------------------------------------
                                 | spin |wlock |rlock |mutex | wsem | rsem |
  --------------------------------------------------------------------------
                     A-A deadlock:  ok  |

---------- 2.6.24-rc8 ----------
Linux version 2.6.24-rc8 (root@tomoyo) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #1 SMP Fri Jan 18 09:51:40 JST 2008
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
 BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
 BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000001fef0000 (usable)
 BIOS-e820: 000000001fef0000 - 000000001feff000 (ACPI data)
 BIOS-e820: 000000001feff000 - 000000001ff00000 (ACPI NVS)
 BIOS-e820: 000000001ff00000 - 0000000020000000 (usable)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
512MB LOWMEM available.
found SMP MP-table at 000f6c90
Zone PFN ranges:
  DMA             0 ->     4096
  Normal       4096 ->   131072
  HighMem    131072 ->   131072
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0:        0 ->   131072
DMI present.
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: INTEL    Product ID: 440BX        APIC at: 0xFEE00000
Processor #0 6:15 APIC version 17
Processor #1 6:15 APIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Enabling APIC mode:  Flat.  Using 1 I/O APICs
Processors: 2
Allocating PCI resources starting at 30000000 (gap: 20000000:dec00000)
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/sda1 ro noapic nolapic console=ttyS0,115200n8
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c052c000 soft=c052a000
PID hash table entries: 2048 (order: 11, 8192 bytes)
Detected 1994.813 MHz processor.
Console: colour VGA+ 80x25
console [ttyS0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          30
... MAX_LOCKDEP_KEYS:        2048
... CLASSHASH_SIZE:           1024
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 1024 kB
 per task-struct memory footprint: 1680 bytes
------------------------
| Locking API testsuite:
----------------------------------------------------------------------------
                                 | spin |wlock |rlock |mutex | wsem | rsem |
  --------------------------------------------------------------------------
                     A-A deadlock:  ok  |  ok  |  ok  |  ok  |  ok  |  ok  |

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Andrew Morton]

On Fri, 18 Jan 2008 10:20:11 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote:

> Hello.
> 
> I found the CONFIG_DEBUG_LOCKING_API_SELFTESTS hangs
> at (A-A deadlock)-(wlock) pair.
> 
> 2.6.24-rc8 doesn't hang.
> So, I think the bug is in 2.6.24-rc8-mm1.bz2 .
> 
> The kernel configs are at
> 
>   http://I-love.SAKURA.ne.jp/tmp/config-2.6.24-rc8
>   http://I-love.SAKURA.ne.jp/tmp/config-2.6.24-rc8-mm1
> 

Yes, Zan hit the same bug.

> 
> ---------- 2.6.24-rc8-mm1 ----------
> Linux version 2.6.24-rc8-mm1 (root@tomoyo) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #2 SMP Fri Jan 18 10:13:09 JST 2008
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
>  BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
>  BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 000000001fef0000 (usable)
>  BIOS-e820: 000000001fef0000 - 000000001feff000 (ACPI data)
>  BIOS-e820: 000000001feff000 - 000000001ff00000 (ACPI NVS)
>  BIOS-e820: 000000001ff00000 - 0000000020000000 (usable)
>  BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
>  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
>  BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
> 0MB HIGHMEM available.
> 512MB LOWMEM available.
> found SMP MP-table at 000f6c90
> Zone PFN ranges:
>   DMA             0 ->     4096
>   Normal       4096 ->   131072
>   HighMem    131072 ->   131072
> Movable zone start PFN for each node
> early_node_map[1] active PFN ranges
>     0:        0 ->   131072
> DMI present.
> Intel MultiProcessor Specification v1.4
>     Virtual Wire compatibility mode.
> OEM ID: INTEL    Product ID: 440BX        APIC at: 0xFEE00000
> Processor #0 6:15 APIC version 17
> Processor #1 6:15 APIC version 17
> I/O APIC #2 Version 17 at 0xFEC00000.
> Enabling APIC mode:  Flat.  Using 1 I/O APICs
> Processors: 2
> Allocating PCI resources starting at 30000000 (gap: 20000000:dec00000)
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
> Kernel command line: root=/dev/sda1 ro noapic nolapic console=ttyS0,115200n8
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Initializing CPU#0
> CPU 0 irqstacks, hard=c0523000 soft=c0521000
> PID hash table entries: 2048 (order: 11, 8192 bytes)
> Detected 1993.105 MHz processor.
> Console: colour VGA+ 80x25
> console [ttyS0] enabled
> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> ... MAX_LOCKDEP_SUBCLASSES:    8
> ... MAX_LOCK_DEPTH:          30
> ... MAX_LOCKDEP_KEYS:        2048
> ... CLASSHASH_SIZE:           1024
> ... MAX_LOCKDEP_ENTRIES:     8192
> ... MAX_LOCKDEP_CHAINS:      16384
> ... CHAINHASH_SIZE:          8192
>  memory used by lock dependency info: 1024 kB
>  per task-struct memory footprint: 1680 bytes
> ------------------------
> | Locking API testsuite:
> ----------------------------------------------------------------------------
>                                  | spin |wlock |rlock |mutex | wsem | rsem |
>   --------------------------------------------------------------------------
>                      A-A deadlock:  ok  |

So does this mean that we got stuck in the first write_lock() test?

I'd be suspecting git-sched, so in lieu of a full bisection search it would
be great if one of you could apply
http://userweb.kernel.org/~akpm/git-sched.patch to 2.6.24-rc8 and see if it
fails in the same way.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Tetsuo Handa]

Hello.

Andrew Morton wrote:
> I'd be suspecting git-sched, so in lieu of a full bisection search it would
> be great if one of you could apply
> http://userweb.kernel.org/~akpm/git-sched.patch to 2.6.24-rc8 and see if it
> fails in the same way.
Too bad. It didn't fail.

Regards.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Andrew Morton]

On Fri, 18 Jan 2008 12:19:56 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote:

> Hello.
> 
> Andrew Morton wrote:
> > I'd be suspecting git-sched, so in lieu of a full bisection search it would
> > be great if one of you could apply
> > http://userweb.kernel.org/~akpm/git-sched.patch to 2.6.24-rc8 and see if it
> > fails in the same way.
> Too bad. It didn't fail.
> 

I used your exact .config with the current mm lineup (I don't thinlk
anything relevant has changed) and it booted OK on my old PIII machine.

It could be compiler version dependent.  I used gcc-4.1.0.  Which version
were you and Zan using please?

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Tetsuo Handa]

Hello.

Andrew Morton wrote:
> It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> were you and Zan using please?

I'm using gcc 3.3.5 on Debian/Sarge .

By the way, 2.6.24-rc8-mm1 doesn't hang with a kernel config

# make -s allnoconfig
# make -s menuconfig

  Set [*] to all entries in "Kernel hacking  --->" section
  (except "KGDB: kernel debugging with remote gdb").

# make -s && make -s install

Regards.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Dave Young]

On Jan 18, 2008 2:40 PM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
> Hello.
>
> Andrew Morton wrote:
> > It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> > were you and Zan using please?

I have the same problem, gcc 3.4.6

>
> I'm using gcc 3.3.5 on Debian/Sarge .
>
> By the way, 2.6.24-rc8-mm1 doesn't hang with a kernel config
>
> # make -s allnoconfig
> # make -s menuconfig
>
>   Set [*] to all entries in "Kernel hacking  --->" section
>   (except "KGDB: kernel debugging with remote gdb").
>
> # make -s && make -s install
>
> Regards.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Andrew Morton]

On Fri, 18 Jan 2008 14:59:32 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:

> On Jan 18, 2008 2:40 PM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> > Hello.
> >
> > Andrew Morton wrote:
> > > It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> > > were you and Zan using please?
> 
> I have the same problem, gcc 3.4.6
> 
> >
> > I'm using gcc 3.3.5 on Debian/Sarge .
> >
> > By the way, 2.6.24-rc8-mm1 doesn't hang with a kernel config
> >
> > # make -s allnoconfig
> > # make -s menuconfig
> >
> >   Set [*] to all entries in "Kernel hacking  --->" section
> >   (except "KGDB: kernel debugging with remote gdb").
> >
> > # make -s && make -s install

The only machine I have with gcc-3.x takes an amazing amount of time to
compile a quite minimal kernel then barfs in headers_install with

  CHK     include/linux/version.h
make[1]: `scripts/unifdef' is up to date.
make[1]: *** No rule to make target `|', needed by `asm-generic'.  Stop.

which is a mainline bug, too.  ISTR someone else reported it but I don't
recall it getting fixed.

I think I'll give up on this.  Either someone else fixes it or we merge the
bug into mainline.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Sam Ravnborg]

On Fri, Jan 18, 2008 at 12:21:52AM -0800, Andrew Morton wrote:
> On Fri, 18 Jan 2008 14:59:32 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:
> 
> > On Jan 18, 2008 2:40 PM, Tetsuo Handa
> > <penguin-kernel@i-love.sakura.ne.jp> wrote:
> > > Hello.
> > >
> > > Andrew Morton wrote:
> > > > It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> > > > were you and Zan using please?
> > 
> > I have the same problem, gcc 3.4.6
> > 
> > >
> > > I'm using gcc 3.3.5 on Debian/Sarge .
> > >
> > > By the way, 2.6.24-rc8-mm1 doesn't hang with a kernel config
> > >
> > > # make -s allnoconfig
> > > # make -s menuconfig
> > >
> > >   Set [*] to all entries in "Kernel hacking  --->" section
> > >   (except "KGDB: kernel debugging with remote gdb").
> > >
> > > # make -s && make -s install
> 
> The only machine I have with gcc-3.x takes an amazing amount of time to
> compile a quite minimal kernel then barfs in headers_install with
> 
>   CHK     include/linux/version.h
> make[1]: `scripts/unifdef' is up to date.
> make[1]: *** No rule to make target `|', needed by `asm-generic'.  Stop.
> 
> which is a mainline bug, too.  ISTR someone else reported it but I don't
> recall it getting fixed.

I know about this bug and I plan to fix it one day.

I have not considered it important because the bug is only triggered
by following connditions:
1) In kernel hacking the check headers options is selected
2) make is older than version 3.80

Documentation/changes do not require us to use make 3.80.
 
> I think I'll give up on this.  Either someone else fixes it or we merge the
> bug into mainline.

One way to fix this is to disable the config option like this.

	Sam

[PATCH] kbuild: disable headers_check

The headers_check command requires make version 3.80 due to use
of ordered dependencies. But kernel build does otherwise
not require this recent make.
Disable the option until it get properly fixed.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
---
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index a601093..98b2431 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -67,7 +67,7 @@ config DEBUG_FS
 
 config HEADERS_CHECK
 	bool "Run 'make headers_check' when building vmlinux"
-	depends on !UML
+	depends on !UML || BROKEN
 	help
 	  This option will extract the user-visible kernel headers whenever
 	  building the kernel, and will run basic sanity checks on them to

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Ingo Molnar]

* Dave Young <hidave.darkstar@gmail.com> wrote:

> On Jan 18, 2008 2:40 PM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> > Hello.
> >
> > Andrew Morton wrote:
> > > It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> > > were you and Zan using please?
> 
> I have the same problem, gcc 3.4.6

is the patch below already in -rc8-mm1? hmm ... it appears it's not. 
Could you give it a try?

	Ingo

------------->
commit 5a26db5bd25cf4bf32ae9fa9f6136b6b6d5b45c5
Author: Nick Piggin <npiggin@suse.de>
Date:   Wed Jan 16 09:51:58 2008 +0100

    lockdep: fix internal double unlock during self-test
    Lockdep, during self-test (when it was simulating double unlocks) was
    sometimes unconditionally unlocking a spinlock when it had not been
    locked. This won't work for ticket locks.
    
    Signed-off-by: Nick Piggin <npiggin@suse.de>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>
    Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>

diff --git a/kernel/lockdep.c b/kernel/lockdep.c
index 723bd9f..4335f12 100644
--- a/kernel/lockdep.c
+++ b/kernel/lockdep.c
@@ -2943,9 +2943,10 @@ void lockdep_free_key_range(void *start, unsigned long size)
 	struct list_head *head;
 	unsigned long flags;
 	int i;
+	int locked;
 
 	raw_local_irq_save(flags);
-	graph_lock();
+	locked = graph_lock();
 
 	/*
 	 * Unhash all classes that were created by this module:
@@ -2959,7 +2960,8 @@ void lockdep_free_key_range(void *start, unsigned long size)
 				zap_class(class);
 	}
 
-	graph_unlock();
+	if (locked)
+		graph_unlock();
 	raw_local_irq_restore(flags);
 }
 
@@ -2969,6 +2971,7 @@ void lockdep_reset_lock(struct lockdep_map *lock)
 	struct list_head *head;
 	unsigned long flags;
 	int i, j;
+	int locked;
 
 	raw_local_irq_save(flags);
 
@@ -2987,7 +2990,7 @@ void lockdep_reset_lock(struct lockdep_map *lock)
 	 * Debug check: in the end all mapped classes should
 	 * be gone.
 	 */
-	graph_lock();
+	locked = graph_lock();
 	for (i = 0; i < CLASSHASH_SIZE; i++) {
 		head = classhash_table + i;
 		if (list_empty(head))
@@ -3000,7 +3003,8 @@ void lockdep_reset_lock(struct lockdep_map *lock)
 			}
 		}
 	}
-	graph_unlock();
+	if (locked)
+		graph_unlock();
 
 out_restore:
 	raw_local_irq_restore(flags);

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Tetsuo Handa]

Hello.

I tried gcc 4.1.2 on Fedora 8 using
http://I-love.SAKURA.ne.jp/tmp/config-2.6.24-rc8-mm1 .
Same result as gcc 3.3.5 on Debian/Sarge.

It seems kernel config (and possibly hardware) dependent
rather than gcc version dependent.

This is VMware workstation 6.0.0 on Thinkpad X60 (Core 2 Duo).

Regards.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Tetsuo Handa]

Hello.

Ingo Molnar wrote:
> is the patch below already in -rc8-mm1? hmm ... it appears it's not. 
> Could you give it a try?

This patch solved this bug.

Thank you.

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Dave Young]

On Jan 18, 2008 5:38 PM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
> Hello.
>
> Ingo Molnar wrote:
> > is the patch below already in -rc8-mm1? hmm ... it appears it's not.
> > Could you give it a try?

Yes , It fixes that problem.

>
> This patch solved this bug.

So quick :)

>
> Thank you.
>

Re: [2.6.24-rc8-mm1] Locking API boot-time self-tests hangs.

[Zan Lynx]

[-- Attachment #1: Type: text/plain, Size: 2238 bytes --]


On Thu, 2008-01-17 at 20:42 -0800, Andrew Morton wrote:
> On Fri, 18 Jan 2008 12:19:56 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote:
> 
> > Hello.
> > 
> > Andrew Morton wrote:
> > > I'd be suspecting git-sched, so in lieu of a full bisection search it would
> > > be great if one of you could apply
> > > http://userweb.kernel.org/~akpm/git-sched.patch to 2.6.24-rc8 and see if it
> > > fails in the same way.
> > Too bad. It didn't fail.
> > 
> 
> I used your exact .config with the current mm lineup (I don't thinlk
> anything relevant has changed) and it booted OK on my old PIII machine.
> 
> It could be compiler version dependent.  I used gcc-4.1.0.  Which version
> were you and Zan using please?

From my -rc6-mm1 boot.

$ scripts/ver_linux 
If some fields are empty or look unusual you may have an old version.
Compare to the current minimal requirements in Documentation/Changes.
 
Linux zephyr 2.6.24-rc6-mm1 #1 SMP Tue Dec 25 17:53:03 MST 2007 x86_64
AMD Athlon(tm) 64 Processor 3400+ AuthenticAMD GNU/Linux
 
Gnu C                  4.2.2
Gnu make               3.81
binutils               2.18.50.0.1.20070908
util-linux             2.13.1
mount                  2.13.1
module-init-tools      3.4
e2fsprogs              1.40.4
reiserfsprogs          3.6.19
reiser4progs           1.0.6
pcmciautils            014
pcmcia-cs              3.2.9
PPP                    2.4.4
nfs-utils              1.0.7
Linux C Library        Dynamic linker (ldd)   2.7
Procps                 3.2.7
Net-tools              1.60
Kbd                    1.13
oprofile               0.9.3
Sh-utils               6.9
udev                   118
wireless-tools         29
Modules Loaded         nls_cp437 vfat fat nls_iso8859_1 isofs nls_base
nouveau drm snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss
snd_seq_midi_event snd_seq snd_seq_device rfcomm l2cap usbhid hid
hci_usb bluetooth usb_storage libusual joydev nvidiafb fb fb_ddc
i2c_algo_bit cfbcopyarea vgastate i2c_core cfbimgblt cfbfillrect
snd_intel8x0m snd_intel8x0 snd_ac97_codec ehci_hcd psmouse ohci_hcd
serio_raw ac97_bus snd_pcm snd_timer usbcore evdev snd sg snd_page_alloc

-- 
Zan Lynx <zlynx@acm.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]