LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* Re: System crash after "No irq handler for vector" linux 2.6.19
@ 2007-01-23 10:19 l.genoni
  0 siblings, 0 replies; 12+ messages in thread
From: l.genoni @ 2007-01-23 10:19 UTC (permalink / raw)
  To: linux-kernel

reproduced.
it took more or less one hour to reproduce it.  I could reproduce it olny
running also irqbalance 0.55 and commenting out the sleep 1. The message 
in
syslog is the same and then, after a few seconds I think, KABOM! system 
crash
and reboot.

I tested also a similar system that has 4 dual core CPU Opteron 2600MHZ. 
On
this system (linux sees 8 CPU, but it is the same kernel, same gcc, same
config, same glibc, same active services) I could not reproduce it even
running irqbalance 0.55 in almost 1 hour. Maybe I could reproduce it 
waiting
for more time, but my users need to do their work, so I could not have a
longer test window. So on 16 CPU I had the crash, on 8 CPU I had no crash.

I need to give back the system to the users, so if you need other tests,
please, tell me as soon.

thanx

Luigi Genoni

On Monday 22 January 2007 18:14, Eric W. Biederman wrote:
> "Luigi Genoni" <luigi.genoni@pirelli.com> writes:
> > (e-mail resent because not delivered using my other e-mail account)
> >
> > Hi,
> > this night a linux server 8 dual core CPU Optern 2600Mhz crashed just
> > after giving this message
> >
> > Jan 22 04:48:28 frey kernel: do_IRQ: 1.98 No irq handler for vector
>
> Ok.  This indicates that the hardware is doing something we didn't 
expect.
> We don't know which irq the hardware was trying to deliver when it
> sent vector 0x98 to cpu 1.
>
> > I have no other logs, and I eventually lost the OOPS since I have no 
net
> > console setled up.
>
> If you had an oops it may have meant the above message was a secondary
> symptom.  Groan.  If it stayed up long enough to give an OOPS then
> there is a chance the above message appearing only once had nothing
> to do with the actual crash.
>
> How long had the system been up?
>
> > As I said sistem is running linux 2.6.19 compiled with gcc 4.1.1 for 
AMD
> > Opteron (attached see .config), no kernel preemption excepted the BKL
> > preemption. glibc 2.4.
> >
> > System has 16 GB RAM and 8 dual core Opteron 2600Mhz.
> >
> > I am running irqbalance 0.55.
> >
> > any hints on what has happened?
>
> Three guesses.
>
> - A race triggered by irq migration (but I would expect more people to 
be
> yelling). The code path where that message comes from is new in 2.6.19 
so
> it may not have had all of the bugs found yet :(
> - A weird hardware or BIOS setup.
> - A secondary symptom triggered by some other bug.
>
> If this winds up being reproducible we should be able to track it down.
> If not this may end up in the files of crap something bad happened that
> we don't understand.
>
> The one condition I know how to test for (if you are willing) is an
> irq migration race.  Simply by triggering irq migration much more often,
> and thus increasing our chances of hitting a problem.
>
> Stopping irqbalance and running something like:
> for irq in 0 24 28 29 44 45 60 68 ; do
>       while :; do
>               for mask in 1 2 4 8 10 20 40 80 100 200 400 800 1000 2000 
4000 8000 ; do
>                       echo mask > /proc/irq/$irq/smp_affinity
>                       sleep 1
>               done
>       done &
> done
>
> Should force every irq to migrate once a second, and removing the sleep 
1
> is even harsher, although we max at one irq migration by irq received.
>
> If some variation of the above loop does not trigger the do_IRQ ??? No 
irq
> handler for vector message chances are it isn't a race in irq migration.
>
> If we can rule out the race scenario it will at least put us in the 
right
> direction for guessing what went wrong with your box.
>
> Eric

-- 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]                   ` <200702021905.39922.luigi.genoni@pirelli.com>
  2007-02-02 18:32                     ` Eric W. Biederman
@ 2007-02-03  0:40                     ` Eric W. Biederman
  1 sibling, 0 replies; 12+ messages in thread
From: Eric W. Biederman @ 2007-02-03  0:40 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel


Luigi,

Unless you have a completely different cause I believe the patches I
just posted will fix the issue.  If you can test and confirm this that
would be great.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]                   ` <200702021905.39922.luigi.genoni@pirelli.com>
@ 2007-02-02 18:32                     ` Eric W. Biederman
  2007-02-03  0:40                     ` Eric W. Biederman
  1 sibling, 0 replies; 12+ messages in thread
From: Eric W. Biederman @ 2007-02-02 18:32 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> the message appeared just once, but no crash.
> anyway the load average was really abnormal.

Good. You tested it and it worked!

High load average is interesting, because it has similar causes
as the "No irq handler for vector", but technically they are
completely independent.

In practice surviving a high load average is a very useful property
though.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]               ` <200702021848.55921.luigi.genoni@pirelli.com>
@ 2007-02-02 18:02                 ` Eric W. Biederman
       [not found]                   ` <200702021905.39922.luigi.genoni@pirelli.com>
  0 siblings, 1 reply; 12+ messages in thread
From: Eric W. Biederman @ 2007-02-02 18:02 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> I tested the patch, but I could not really stress the HW.
> anyway no crash, but load average is somehow abnormal, higher than it should 
> be.

Thanks.  Did you get any nasty messages about "No irq handler for vector?"
If not then you never even hit the problem condition.

Given how rare the trigger condition is it would probably take force
irq migration to even trigger the "No irq handler for vector message".

That patch I'm not really worried about, I've tested it and it meets
the obviously correct condition :)  But it should enable people to
not be afraid of IRQ migration.

I'm slowly working my way towards a real fix.  I know what I have to code,
now I just have to figure out how. :)

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
@ 2007-02-01 13:33 Chris Rankin
  0 siblings, 0 replies; 12+ messages in thread
From: Chris Rankin @ 2007-02-01 13:33 UTC (permalink / raw)
  To: linux-kernel

> Ok. I've finally figured out what is going on. The code is race free but the programmer was an
> idiot.

Hi,

Could this IRQ problem account for this bug as well, please? Or is yours strictly a 2.6.19.x
issue?

http://bugzilla.kernel.org/show_bug.cgi?id=7847

I have a dual P4 Xeon box (HT enabled) so there is a lot of scope for IRQ migration. I was playing
WoW when this bug occurred, so there would have been a lot of IRQs needing handling between both
the video and sound cards.

Cheers,
Chris



		
___________________________________________________________ 
What kind of emailer are you? Find out today - get a free analysis of your email personality. Take the quiz at the Yahoo! Mail Championship. 
http://uk.rd.yahoo.com/evt=44106/*http://mail.yahoo.net/uk 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]           ` <200701311549.22512.luigi.genoni@pirelli.com>
  2007-02-01  5:59             ` Eric W. Biederman
@ 2007-02-01  7:20             ` Eric W. Biederman
       [not found]               ` <200702021848.55921.luigi.genoni@pirelli.com>
  1 sibling, 1 reply; 12+ messages in thread
From: Eric W. Biederman @ 2007-02-01  7:20 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> OK,
> willing to test any patch.

Ok. I've finally figured out what is going on.  The code is
race free but the programmer was an idiot.

In the local apic there are two relevant registers.
ISR (in service register) describing all of the
interrupts that the cpu in the process of handling.
IRR (intrerupt request register) which lists all of
the interrupts that are currently pending.

Well it happens that IRR is used to catch the case
when we are servicing an interrupt and that same interrupt
comes in again.  When that happens as soon as we are
done service the interrupt that same interrupt fires again.

We perform interrupt migration in an interrupt handler, so
we can be race free.

It turns out that if I'm performing migration (updating all
of the data structures and hardware registers) while IRR
is set the interrupt will happen in the old location immediate after
my migration work is complete.  And since the kernel is not
setup to deal with it we get an ugly error message.

Anyway now that I know what is going on I'm going to have to think
about this a little bit more to figure out how to fix this.  My hunch
is the easy fix will be simply not to migrate until I have an
interrupt instance when IRR is clear.  

Anyway with a little luck tomorrow I will be able to figure it out,
it's to bed with me now.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]           ` <200701311549.22512.luigi.genoni@pirelli.com>
@ 2007-02-01  5:59             ` Eric W. Biederman
  2007-02-01  7:20             ` Eric W. Biederman
  1 sibling, 0 replies; 12+ messages in thread
From: Eric W. Biederman @ 2007-02-01  5:59 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> OK,
> willing to test any patch.

Sure.  After I get things working on this end I will copy you,
on any fixes so you can confirm they work for you.

I am still root causing this but I have found a small fix that should
keep the system from going down when this problem occurs.

If you could confirm that it keeps your system from going down I'd
appreciate it.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]       ` <Pine.LNX.4.64.0701232052330.32111@baldios.it.pirelli.com>
@ 2007-01-31  8:39         ` Eric W. Biederman
       [not found]           ` <200701311549.22512.luigi.genoni@pirelli.com>
  0 siblings, 1 reply; 12+ messages in thread
From: Eric W. Biederman @ 2007-01-31  8:39 UTC (permalink / raw)
  To: luigi.genoni; +Cc: akpm, linux-kernel

<luigi.genoni@pirelli.com> writes:

> I have in interesting update, at less I suppose I have.

It was, at least as another data point.

> I do not know very well what happens with irq stuff migrating shared irq, but I
> suppose this has something to do with this crash.

The fact the irq was shared should have no bearing on this crash scenario.
A shared irq is not at all helpful in a performance sense, but this problem
is low enough that a shared irq should have made not difference at all, except
for the frequency the interrupt fired, and was migrated.  And a high interrupt
frequency, and a high migration rate tend to cause this problem.

> Right now I stopped irqbalance and puff! load average is back to normal, and
> under the same workload notthing similar is happening for the moment.

Yes.  That sounds like a good work around until this problem is sorted out.

> Lesson number one I learnt: avoid shared IRQ on this systems (but to reconfigure
> HW cabling right now is not so easy).

Right.   Because the only sharing should be because the traces on the
motherboard are shared.

> I hope this helps

It has all helped.

I have been tracking some easier problems, keeping this one on my back burner.
The good/bad news is that by restricting my set of vectors I can choose from
in the kernel, and running ping -f to another machine.  And migrating the
single irq for my NIC.  I have been able to reproduce this in about 5 minutes.

I haven't root caused it yet, but the fact I can reproduce this on dual socket
motherboard suggest the reason it took you an hour to reproduce the problem is
simply because you had so few irqs on your system, and that the extreme
latency of 8-socket Opterons is not to blame.

Hopefully now that I can reproduce this I will be able to root cause
this and then fix this bug.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found]   ` <200701231051.32945.luigi.genoni@pirelli.com>
@ 2007-01-23 12:18     ` Eric W. Biederman
       [not found]       ` <Pine.LNX.4.64.0701232052330.32111@baldios.it.pirelli.com>
  0 siblings, 1 reply; 12+ messages in thread
From: Eric W. Biederman @ 2007-01-23 12:18 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> reproduced.
> it took more or less one hour to reproduce it.  I could reproduce it olny 
> running also irqbalance 0.55 and commenting out the sleep 1. The message in 
> syslog is the same and then, after a few seconds I think, KABOM! system crash 
> and reboot.
>
> I tested also a similar system that has 4 dual core CPU Opteron 2600MHZ. On 
> this system (linux sees 8 CPU, but it is the same kernel, same gcc, same 
> config, same glibc, same active services) I could not reproduce it even 
> running irqbalance 0.55 in almost 1 hour. Maybe I could reproduce it waiting 
> for more time, but my users need to do their work, so I could not have a 
> longer test window. So on 16 CPU I had the crash, on 8 CPU I had no crash. 
>
> I need to give back the system to the users, so if you need other tests, 
> please, tell me as soon.

Ok.  Since it seems to be the irq code I'm going to need to get a dump
of the state of the apics, basically the output print_IO_APIC from the time
this happens. 

I'm not really out of bed yet so no patch but a heads up.  Once I've finished
sleeping I'll look at putting a debugging patch together.

Getting a bootlog of the identical system that would not crash at 8 cpus
would be interesting.  In part this is because the number of apic modes
that are available for use are much fewer when we have 8 or more cpus,
so it would be interesting to see if it is actually using the same code
for interrupt delivery.

If you have a few minutes to try it, it would be interesting to know
if forcing the migration from the command line (as I was suggesting)
would reproduce this faster than irqbalance.

Hopefully I can think up things to fix this when I wake up.

The time zone difference is going to be a pain :(

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
@ 2007-01-23 10:35 l.genoni
  0 siblings, 0 replies; 12+ messages in thread
From: l.genoni @ 2007-01-23 10:35 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: TEXT/PLAIN, Size: 6149 bytes --]

On Mon, 22 Jan 2007, Eric W. Biederman wrote:

> "Luigi Genoni" <luigi.genoni@pirelli.com> writes:
>
>> Hi,
>> this night a linux server 8 dual core CPU Optern 2600Mhz crashed just 
after
>> giving this message
>>
>> Jan 22 04:48:28 frey kernel: do_IRQ: 1.98 No irq handler for vector
>
> Ok.  This indicates that the hardware is doing something we didn't 
expect.
> We don't know which irq the hardware was trying to deliver when it
> sent vector 0x98 to cpu 1.
>
>> I have no other logs, and I eventually lost the OOPS since I have no 
net
>> console setled up.
>
> If you had an oops it may have meant the above message was a secondary
> symptom.  Groan.  If it stayed up long enough to give an OOPS then
> there is a chance the above message appearing only once had nothing
> to do with the actual crash.
>
>
> How long had the system been up?

Sorry, my english is so bad, so I could not espress really what I wanted 
to
say.

I didn't get an OOPS. I could not see it on the console (nor logs). I do 
not
have netconsole

The system was up and running since 52 days.

>
>> As I said sistem is running linux 2.6.19 compiled with gcc 4.1.1 for 
AMD
>> Opteron (attached see .config), no kernel preemption excepted the BKL
>> preemption. glibc 2.4.
>>
>> System has 16 GB RAM and 8 dual core Opteron 2600Mhz.
>>
>> I am running irqbalance 0.55.
>>
>> any hints on what has happened?
>
> Three guesses.
>
> - A race triggered by irq migration (but I would expect more people to 
be
> yelling).
>  The code path where that message comes from is new in 2.6.19 so it may 
not
> have
>  had all of the bugs found yet :(
> - A weird hardware or BIOS setup.
> - A secondary symptom triggered by some other bug.
>
> If this winds up being reproducible we should be able to track it down.
> If not this may end up in the files of crap something bad happened that
> we don't understand.
>
> The one condition I know how to test for (if you are willing) is an
> irq migration race.  Simply by triggering irq migration much more often,
> and thus increasing our chances of hitting a problem.

OK, will try tomorrow morning (I am at home right now, in Italy is night).

>
> Stopping irqbalance and running something like:
> for irq in 0 24 28 29 44 45 60 68 ; do
>       while :; do
>               for mask in 1 2 4 8 10 20 40 80 100 200 400 800 1000 2000 
4000
> 8000 ; do
>                       echo mask > /proc/irq/$irq/smp_affinity
>                       sleep 1
>               done
>       done &
> done
>
> Should force every irq to migrate once a second, and removing the sleep 
1
> is even harsher, although we max at one irq migration by irq received.
>
> If some variation of the above loop does not trigger the do_IRQ ??? No 
irq
> handler
> for vector message chances are it isn't a race in irq migration.
>
> If we can rule out the race scenario it will at least put us in the 
right
> direction
> for guessing what went wrong with your box.

btw, the server is running tibco (bwengine, imse and adr3, all tibco 
software
is multithread) which use multicast and stress the net cards continuosly
sending out and receiving multicast and UDP packets talking with a remote
oracle DB on eth2 (gigabit ethernet). tibco continuously creates and 
deletes
small files (small often less than 1 KB, thousands per minute when it has 
a lot
of work to do) on three SAN luns (IBM DS8000)  of 33 GB each one, accessed
trought 2 path fibre channels 2Gbit (linux dm multipath) with LVM2 volumes
(reiserFS, since gives us the best performance with all those small files, 
and
never gave us trobles).

In reality the system sees 21 luns on two paths, and those luns are also 
seen
by other 4 linux servers identical to this one clustered togheter with 
this
system using service guard (which is only userspace cluster software, no 
kernel
modules, eth0 and eth4 are HB and eth6 is the card on the public LAN ).
The Volume Group accessed by this server is reserved using LVM2 tags.

Tibco also does a normal I/O (about 2 MB/s) on a TCP NFS3 mounted volume 
using
eth2. Never had troubles with this NFS mount

Client nfs v3 (after 10 hour of uptime):
null       getattr    setattr    lookup     access     readlink
0       0% 2770320 78% 122     0% 546401 15% 19672   0% 133     0%
read       write      create     mkdir      symlink    mknod
40673   1% 3338    0% 58      0% 0       0% 0       0% 0       0%
remove     rmdir      rename     link       readdir    readdirplus
32      0% 0       0% 0       0% 0       0% 144213  4% 552     0%
fsstat     fsinfo     pathconf   commit
316     0% 872     0% 0       0% 16      0%


so network cards and SAN cards do work a lot. I include the boot messages,
maybe could help you to figure out how this server is configured.

what more... ah, usually the system during the sunday night is not working 
a
lot (not working at all), and it crashed a sunday night. Loocking the sar 
I see
strange statistics...

sar -I SUM        INTR    intr/s
04:40:01          sum    839.40
05:00:01          sum     62.10 <---
05:10:01          sum    245.63


sar -a
                    CPU     %user     %nice   %system   %iowait     %idle
04:40:01          all      1.14      0.00      0.44      0.06     98.36
05:00:01          all    287.15    288.31    287.73    288.17      0.00 <---
05:10:01          all      0.00      0.00      0.00      0.01     99.98

but normally during the day:
10:30:01          all     88.41      0.00      0.81      0.05     10.73)

sar -W        cswch/s
04:40:01      7738.69
05:00:01    478986467937.98 <---
05:10:01       194.36

(but normally during the day:
10:20:01     61264.85)

sar -b            tps      rtps      wtps   bread/s   bwrtn/s
04:40:01       187.05      0.02    187.03      0.27   1777.81
05:00:01       106.24    111.47    106.29    108.40     59.48 <---
05:10:01         2.48      0.02      2.47      0.27     28.40

sar -c           proc/s
04:40:01         0.34
05:00:01    478986468210.56 <--- this is absolutelly abnormal
05:10:01         0.03

That's all I can say about the race scenario. As you need more tests to be 
done, please tell me.

>
> Eric
>

Thanx

Luigi Genoni

[-- Attachment #2: Type: TEXT/PLAIN, Size: 46905 bytes --]

Inspecting /boot/System.map-2.6.19-smp
Loaded 22489 symbols from /boot/System.map-2.6.19-smp.
Symbols match kernel version 2.6.19.
No module symbols loaded - kernel modules not enabled.

klogd 1.4.1, log source = ksyslog started.
OS-provided physical RAM map:
<4> BIOS-e820: 0000000000000000 - 000000000009a000 (usable)
<4> BIOS-e820: 000000000009a000 - 00000000000a0000 (reserved)
<4> BIOS-e820: 00000000000ce000 - 0000000000100000 (reserved)
<4> BIOS-e820: 0000000000100000 - 00000000dfef0000 (usable)
<4> BIOS-e820: 00000000dfef0000 - 00000000dfefb000 (ACPI data)
<4> BIOS-e820: 00000000dfefb000 - 00000000dff00000 (ACPI NVS)
<4> BIOS-e820: 00000000dff00000 - 00000000e0000000 (reserved)
<4> BIOS-e820: 00000000fec00000 - 00000000fec00400 (reserved)
<4> BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
<4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
<4> BIOS-e820: 0000000100000000 - 0000000420000000 (usable)
<7>Entering add_active_range(0, 0, 154) 0 entries of 3200 used
<7>Entering add_active_range(0, 256, 917232) 1 entries of 3200 used
<7>Entering add_active_range(0, 1048576, 4325376) 2 entries of 3200 used
<4>end_pfn_map = 4325376
<6>DMI present.
<7>ACPI: RSDP (v002 PTLTD                                 ) @ 0x00000000000f7480
<7>ACPI: XSDT (v001 PTLTD  	 XSDT   0x06040000  LTP 0x00000000) @ 0x00000000dfef5719
<7>ACPI: FADT (v003 FSC    D2003    0x06040000 PTEC 0x000f4240) @ 0x00000000dfef9754
<7>ACPI: SSDT (v001 PTLTD  POWERNOW 0x06040000  LTP 0x00000001) @ 0x00000000dfef9848
<7>ACPI: SRAT (v001 AMD    HAMMER   0x06040000 AMD  0x00000001) @ 0x00000000dfefaa0c
<7>ACPI: SSDT (v001 AMD-K8 AMD-ACPI 0x06040000  AMD 0x00000001) @ 0x00000000dfefaccc
<7>ACPI: SSDT (v001 AMD-K8 AMD-ACPI 0x06040000  AMD 0x00000001) @ 0x00000000dfefad69
<7>ACPI: MADT (v001 PTLTD  	 APIC   0x06040000  LTP 0x00000000) @ 0x00000000dfefae06
<7>ACPI: BOOT (v001 PTLTD  $SBFTBL$ 0x06040000  LTP 0x00000001) @ 0x00000000dfefaf88
<7>ACPI: SPCR (v001 PTLTD  $UCRTBL$ 0x06040000 PTL  0x00000001) @ 0x00000000dfefafb0
<7>ACPI: DSDT (v001    FSC    D2003 0x06040000 MSFT 0x02000002) @ 0x0000000000000000
<6>SRAT: PXM 0 -> APIC 0 -> Node 0
<6>SRAT: PXM 0 -> APIC 1 -> Node 0
<6>SRAT: PXM 1 -> APIC 2 -> Node 1
<6>SRAT: PXM 1 -> APIC 3 -> Node 1
<6>SRAT: PXM 2 -> APIC 4 -> Node 2
<6>SRAT: PXM 2 -> APIC 5 -> Node 2
<6>SRAT: PXM 3 -> APIC 6 -> Node 3
<6>SRAT: PXM 3 -> APIC 7 -> Node 3
<6>SRAT: PXM 4 -> APIC 8 -> Node 4
<6>SRAT: PXM 4 -> APIC 9 -> Node 4
<6>SRAT: PXM 5 -> APIC 10 -> Node 5
<6>SRAT: PXM 5 -> APIC 11 -> Node 5
<6>SRAT: PXM 6 -> APIC 12 -> Node 6
<6>SRAT: PXM 6 -> APIC 13 -> Node 6
<6>SRAT: PXM 7 -> APIC 14 -> Node 7
<6>SRAT: PXM 7 -> APIC 15 -> Node 7
<6>SRAT: Node 0 PXM 0 0-a0000
<7>Entering add_active_range(0, 0, 154) 0 entries of 3200 used
<6>SRAT: Node 0 PXM 0 0-80000000
<7>Entering add_active_range(0, 0, 154) 1 entries of 3200 used
<7>Entering add_active_range(0, 256, 524288) 1 entries of 3200 used
<6>SRAT: Node 1 PXM 1 80000000-e0000000
<7>Entering add_active_range(1, 524288, 917232) 2 entries of 3200 used
<6>SRAT: Node 1 PXM 1 80000000-120000000
<7>Entering add_active_range(1, 524288, 917232) 3 entries of 3200 used
<7>Entering add_active_range(1, 1048576, 1179648) 3 entries of 3200 used
<6>SRAT: Node 2 PXM 2 120000000-1a0000000
<7>Entering add_active_range(2, 1179648, 1703936) 4 entries of 3200 used
<6>SRAT: Node 3 PXM 3 1a0000000-220000000
<7>Entering add_active_range(3, 1703936, 2228224) 5 entries of 3200 used
<6>SRAT: Node 4 PXM 4 220000000-2a0000000
<7>Entering add_active_range(4, 2228224, 2752512) 6 entries of 3200 used
<6>SRAT: Node 5 PXM 5 2a0000000-320000000
<7>Entering add_active_range(5, 2752512, 3276800) 7 entries of 3200 used
<6>SRAT: Node 6 PXM 6 320000000-3a0000000
<7>Entering add_active_range(6, 3276800, 3801088) 8 entries of 3200 used
<6>SRAT: Node 7 PXM 7 3a0000000-420000000
<7>Entering add_active_range(7, 3801088, 4325376) 9 entries of 3200 used
<7>NUMA: Using 29 for the hash shift.
<6>Bootmem setup node 0 0000000000000000-0000000080000000
<6>Bootmem setup node 1 0000000080000000-0000000120000000
<6>Bootmem setup node 2 0000000120000000-00000001a0000000
<6>Bootmem setup node 3 00000001a0000000-0000000220000000
<6>Bootmem setup node 4 0000000220000000-00000002a0000000
<6>Bootmem setup node 5 00000002a0000000-0000000320000000
<6>Bootmem setup node 6 0000000320000000-00000003a0000000
<6>Bootmem setup node 7 00000003a0000000-0000000420000000
<4>Zone PFN ranges:
<4>  DMA             0 ->     4096
<4>  DMA32        4096 ->  1048576
<4>  Normal    1048576 ->  4325376
<4>early_node_map[10] active PFN ranges
<4>    0:        0 ->      154
<4>    0:      256 ->   524288
<4>    1:   524288 ->   917232
<4>    1:  1048576 ->  1179648
<4>    2:  1179648 ->  1703936
<4>    3:  1703936 ->  2228224
<4>    4:  2228224 ->  2752512
<4>    5:  2752512 ->  3276800
<4>    6:  3276800 ->  3801088
<4>    7:  3801088 ->  4325376
<7>On node 0 totalpages: 524186
<7>  DMA zone: 56 pages used for memmap
<7>  DMA zone: 1070 pages reserved
<7>  DMA zone: 2868 pages, LIFO batch:0
<7>  DMA32 zone: 7112 pages used for memmap
<7>  DMA32 zone: 513080 pages, LIFO batch:31
<7>  Normal zone: 0 pages used for memmap
<7>On node 1 totalpages: 524016
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 7168 pages used for memmap
<7>  DMA32 zone: 385776 pages, LIFO batch:31
<7>  Normal zone: 1792 pages used for memmap
<7>  Normal zone: 129280 pages, LIFO batch:31
<7>On node 2 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<7>On node 3 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<7>On node 4 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<7>On node 5 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<7>On node 6 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<7>On node 7 totalpages: 524288
<7>  DMA zone: 0 pages used for memmap
<7>  DMA32 zone: 0 pages used for memmap
<7>  Normal zone: 7168 pages used for memmap
<7>  Normal zone: 517120 pages, LIFO batch:31
<6>ACPI: PM-Timer IO Port: 0xc008
<7>ACPI: Local APIC address 0xfee00000
<6>ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
<6>Processor #0 (Bootup-CPU)
<6>ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
<6>Processor #1
<6>ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
<6>Processor #2
<6>ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
<6>Processor #3
<6>ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
<6>Processor #4
<6>ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
<6>Processor #5
<6>ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
<6>Processor #6
<6>ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
<6>Processor #7
<6>ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] enabled)
<6>Processor #8
<6>ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] enabled)
<6>Processor #9
<6>ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] enabled)
<6>Processor #10
<6>ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] enabled)
<6>Processor #11
<6>ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] enabled)
<6>Processor #12
<6>ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] enabled)
<6>Processor #13
<6>ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] enabled)
<6>Processor #14
<6>ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] enabled)
<6>Processor #15
<6>ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x09] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0a] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0b] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0c] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0d] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0e] high edge lint[0x1])
<6>ACPI: LAPIC_NMI (acpi_id[0x0f] high edge lint[0x1])
<6>ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
<6>IOAPIC[0]: apic_id 1, address 0xfec00000, GSI 0-23
<6>ACPI: IOAPIC (id[0x02] address[0xe8000000] gsi_base[24])
<6>IOAPIC[1]: apic_id 2, address 0xe8000000, GSI 24-27
<6>ACPI: IOAPIC (id[0x03] address[0xe8001000] gsi_base[28])
<6>IOAPIC[2]: apic_id 3, address 0xe8001000, GSI 28-31
<6>ACPI: IOAPIC (id[0x06] address[0xf8010000] gsi_base[40])
<6>IOAPIC[3]: apic_id 6, address 0xf8010000, GSI 40-43
<6>ACPI: IOAPIC (id[0x07] address[0xf8011000] gsi_base[44])
<6>IOAPIC[4]: apic_id 7, address 0xf8011000, GSI 44-47
<6>ACPI: IOAPIC (id[0x0a] address[0xf8410000] gsi_base[56])
<6>IOAPIC[5]: apic_id 10, address 0xf8410000, GSI 56-59
<6>ACPI: IOAPIC (id[0x0b] address[0xf8411000] gsi_base[60])
<6>IOAPIC[6]: apic_id 11, address 0xf8411000, GSI 60-63
<6>ACPI: IOAPIC (id[0x0c] address[0xf8800000] gsi_base[64])
<6>IOAPIC[7]: apic_id 12, address 0xf8800000, GSI 64-67
<6>ACPI: IOAPIC (id[0x0d] address[0xf8801000] gsi_base[68])
<6>IOAPIC[8]: apic_id 13, address 0xf8801000, GSI 68-71
<6>ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
<7>ACPI: IRQ0 used by override.
<7>ACPI: IRQ2 used by override.
<7>ACPI: IRQ9 used by override.
<6>Setting APIC routing to physical flat
<6>Using ACPI (MADT) for SMP configuration information
<4>Nosave address range: 000000000009a000 - 00000000000a0000
<4>Nosave address range: 00000000000a0000 - 00000000000ce000
<4>Nosave address range: 00000000000ce000 - 0000000000100000
<4>Nosave address range: 00000000dfef0000 - 00000000dfefb000
<4>Nosave address range: 00000000dfefb000 - 00000000dff00000
<4>Nosave address range: 00000000dff00000 - 00000000e0000000
<4>Nosave address range: 00000000e0000000 - 00000000fec00000
<4>Nosave address range: 00000000fec00000 - 00000000fee00000
<4>Nosave address range: 00000000fee00000 - 00000000fee01000
<4>Nosave address range: 00000000fee01000 - 00000000fff80000
<4>Nosave address range: 00000000fff80000 - 0000000100000000
<6>Allocating PCI resources starting at e2000000 (gap: e0000000:1ec00000)
<6>PERCPU: Allocating 33664 bytes of per cpu data
<4>Built 8 zonelists.  Total pages: 4133724
<5>Kernel command line: auto BOOT_IMAGE=Linux ro root=801
<4>Initializing CPU#0
<4>PID hash table entries: 4096 (order: 12, 32768 bytes)
<4>Console: colour VGA+ 80x25
<4>Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
<4>Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
<4>Checking aperture...
<4>CPU 0: aperture @ 0 size 32 MB
<4>No AGP bridge found
<4>Your BIOS doesn't leave a aperture memory hole
<4>Please enable the IOMMU option in the BIOS setup
<4>This costs you 64 MB of RAM
<4>Mapping aperture over 65536 KB of RAM @ 4000000
<4>Memory: 16443736k/17301504k available (2426k kernel code, 331984k reserved, 952k data, 316k init)
<4>Calibrating delay using timer specific routine.. 5197.21 BogoMIPS (lpj=25986093)
<6>Security Framework v1.0.0 initialized
<6>Capability LSM initialized
<4>Mount-cache hash table entries: 256
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 0/0 -> Node 0
<6>CPU: Physical Processor ID: 0
<6>CPU: Processor Core ID: 0
<6>Freeing SMP alternatives: 28k freed
<6>ACPI: Core revision 20060707
<6>Using local APIC timer interrupts.
<4>result 12483928
<6>Detected 12.483 MHz APIC timer.
<6>Booting processor 1/16 APIC 0x1
<4>Initializing CPU#1
<4>Calibrating delay using timer specific routine.. 5193.42 BogoMIPS (lpj=25967116)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 1/1 -> Node 0
<6>CPU: Physical Processor ID: 0
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 1: Syncing TSC to CPU 0.
<6>CPU 1: synchronized TSC with CPU 0 (last diff 2 cycles, maxerr 1685 cycles)
<6>Booting processor 2/16 APIC 0x2
<4>Initializing CPU#2
<4>Calibrating delay using timer specific routine.. 5193.42 BogoMIPS (lpj=25967140)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 2/2 -> Node 1
<6>CPU: Physical Processor ID: 1
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 2: Syncing TSC to CPU 0.
<6>CPU 2: synchronized TSC with CPU 0 (last diff 3 cycles, maxerr 1682 cycles)
<6>Booting processor 3/16 APIC 0x3
<4>Initializing CPU#3
<4>Calibrating delay using timer specific routine.. 5193.42 BogoMIPS (lpj=25967137)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 3/3 -> Node 1
<6>CPU: Physical Processor ID: 1
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 3: Syncing TSC to CPU 0.
<6>CPU 3: synchronized TSC with CPU 0 (last diff -1 cycles, maxerr 1687 cycles)
<6>Booting processor 4/16 APIC 0x4
<4>Initializing CPU#4
<4>Calibrating delay using timer specific routine.. 5193.42 BogoMIPS (lpj=25967144)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 4/4 -> Node 2
<6>CPU: Physical Processor ID: 2
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 4: Syncing TSC to CPU 0.
<6>CPU 4: synchronized TSC with CPU 0 (last diff -3 cycles, maxerr 1682 cycles)
<6>Booting processor 5/16 APIC 0x5
<4>Initializing CPU#5
<4>Calibrating delay using timer specific routine.. 5193.42 BogoMIPS (lpj=25967142)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 5/5 -> Node 2
<6>CPU: Physical Processor ID: 2
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 5: Syncing TSC to CPU 0.
<6>CPU 5: synchronized TSC with CPU 0 (last diff 12 cycles, maxerr 1685 cycles)
<6>Booting processor 6/16 APIC 0x6
<4>Initializing CPU#6
<4>Calibrating delay using timer specific routine.. 5193.41 BogoMIPS (lpj=25967061)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 6/6 -> Node 3
<6>CPU: Physical Processor ID: 3
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 6: Syncing TSC to CPU 0.
<6>CPU 6: synchronized TSC with CPU 0 (last diff -263 cycles, maxerr 1688 cycles)
<6>Booting processor 7/16 APIC 0x7
<4>Initializing CPU#7
<4>Calibrating delay using timer specific routine.. 5193.41 BogoMIPS (lpj=25967062)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 7/7 -> Node 3
<6>CPU: Physical Processor ID: 3
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 7: Syncing TSC to CPU 0.
<6>CPU 7: synchronized TSC with CPU 0 (last diff -262 cycles, maxerr 1686 cycles)
<6>Booting processor 8/16 APIC 0x8
<4>Initializing CPU#8
<4>Calibrating delay using timer specific routine.. 5193.41 BogoMIPS (lpj=25967055)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 8/8 -> Node 4
<6>CPU: Physical Processor ID: 4
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 8: Syncing TSC to CPU 0.
<6>CPU 8: synchronized TSC with CPU 0 (last diff -264 cycles, maxerr 1682 cycles)
<6>Booting processor 9/16 APIC 0x9
<4>Initializing CPU#9
<4>Calibrating delay using timer specific routine.. 5193.40 BogoMIPS (lpj=25967048)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 9/9 -> Node 4
<6>CPU: Physical Processor ID: 4
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 9: Syncing TSC to CPU 0.
<6>CPU 9: synchronized TSC with CPU 0 (last diff -251 cycles, maxerr 1662 cycles)
<6>Booting processor 10/16 APIC 0xa
<4>Initializing CPU#10
<4>Calibrating delay using timer specific routine.. 5193.40 BogoMIPS (lpj=25967008)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 10/a -> Node 5
<6>CPU: Physical Processor ID: 5
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 10: Syncing TSC to CPU 0.
<6>CPU 10: synchronized TSC with CPU 0 (last diff -269 cycles, maxerr 1708 cycles)
<6>Booting processor 11/16 APIC 0xb
<4>Initializing CPU#11
<4>Calibrating delay using timer specific routine.. 5193.39 BogoMIPS (lpj=25966996)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 11/b -> Node 5
<6>CPU: Physical Processor ID: 5
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 11: Syncing TSC to CPU 0.
<6>CPU 11: synchronized TSC with CPU 0 (last diff -269 cycles, maxerr 1706 cycles)
<6>Booting processor 12/16 APIC 0xc
<4>Initializing CPU#12
<4>Calibrating delay using timer specific routine.. 5193.40 BogoMIPS (lpj=25967008)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 12/c -> Node 6
<6>CPU: Physical Processor ID: 6
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 12: Syncing TSC to CPU 0.
<6>CPU 12: synchronized TSC with CPU 0 (last diff -269 cycles, maxerr 1697 cycles)
<6>Booting processor 13/16 APIC 0xd
<4>Initializing CPU#13
<4>Calibrating delay using timer specific routine.. 5193.40 BogoMIPS (lpj=25967000)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 13/d -> Node 6
<6>CPU: Physical Processor ID: 6
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 13: Syncing TSC to CPU 0.
<6>CPU 13: synchronized TSC with CPU 0 (last diff -272 cycles, maxerr 1700 cycles)
<6>Booting processor 14/16 APIC 0xe
<4>Initializing CPU#14
<4>Calibrating delay using timer specific routine.. 5193.47 BogoMIPS (lpj=25967367)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 14/e -> Node 7
<6>CPU: Physical Processor ID: 7
<6>CPU: Processor Core ID: 0
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 14: Syncing TSC to CPU 0.
<6>CPU 14: synchronized TSC with CPU 0 (last diff -284 cycles, maxerr 2789 cycles)
<6>Booting processor 15/16 APIC 0xf
<4>Initializing CPU#15
<4>Calibrating delay using timer specific routine.. 5193.47 BogoMIPS (lpj=25967386)
<6>CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
<6>CPU: L2 Cache: 1024K (64 bytes/line)
<6>CPU 15/f -> Node 7
<6>CPU: Physical Processor ID: 7
<6>CPU: Processor Core ID: 1
<4>Dual Core AMD Opteron(tm) Processor 885 stepping 02
<6>CPU 15: Syncing TSC to CPU 0.
<6>CPU 15: synchronized TSC with CPU 0 (last diff -297 cycles, maxerr 2802 cycles)
<6>Brought up 16 CPUs
<6>testing NMI watchdog ... OK.
<6>Disabling vsyscall due to use of PM timer
<6>time.c: Using 3.579545 MHz WALL PM GTOD PM timer.
<6>time.c: Detected 2596.653 MHz processor.
<4>migration_cost=644,695
<6>NET: Registered protocol family 16
<6>ACPI: bus type pci registered
<6>PCI: Using configuration type 1
<4>mtrr: your CPUs had inconsistent fixed MTRR settings
<6>mtrr: probably your BIOS does not setup all CPUs.
<6>mtrr: corrected configuration.
<6>ACPI: Interpreter enabled
<6>ACPI: Using IOAPIC for interrupt routing
<6>ACPI: PCI Root Bridge [PCI0] (0000:00)
<7>PCI: Probing PCI hardware (bus 00)
<7>Boot video device is 0000:01:05.0
<4>ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11)
<4>ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 *10 11)
<4>ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 10 11) *0, disabled.
<4>ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11)
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.TP2P._PRT]
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.G0PA._PRT]
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.G0PB._PRT]
<6>ACPI: PCI Root Bridge [PCI2] (0000:10)
<7>PCI: Probing PCI hardware (bus 10)
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI2.G2PA._PRT]
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI2.G2PB._PRT]
<6>ACPI: PCI Root Bridge [PCI5] (0000:28)
<7>PCI: Probing PCI hardware (bus 28)
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI5.G5PA._PRT]
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI5.G5PB._PRT]
<6>ACPI: PCI Root Bridge [PCI7] (0000:38)
<7>PCI: Probing PCI hardware (bus 38)
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI7.G7PA._PRT]
<7>ACPI: PCI Interrupt Routing Table [\_SB_.PCI7.G7PB._PRT]
<6>Linux Plug and Play Support v0.97 (c) Adam Belay
<6>pnp: PnP ACPI init
<6>pnp: PnP ACPI: found 14 devices
<6>AMD768 RNG detected
<5>SCSI subsystem initialized
<6>PCI: Using ACPI for IRQ routing
<6>PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
<6>PCI-DMA: Disabling AGP.
<4>PCI-DMA: aperture base @ 4000000 size 65536 KB
<6>PCI-DMA: using GART IOMMU.
<6>PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
<6>pnp: 00:05: ioport range 0x4d0-0x4d1 has been reserved
<6>pnp: 00:05: ioport range 0x1100-0x117f has been reserved
<6>pnp: 00:05: ioport range 0x1180-0x11ff has been reserved
<6>pnp: 00:07: ioport range 0xca2-0xca5 has been reserved
<6>PCI: Bridge: 0000:00:06.0
<6>  IO window: 2000-2fff
<6>  MEM window: e8100000-e81fffff
<6>  PREFETCH window: f0000000-f7ffffff
<6>PCI: Bridge: 0000:00:0a.0
<6>  IO window: 3000-3fff
<6>  MEM window: e8200000-e82fffff
<6>  PREFETCH window: e2000000-e20fffff
<6>PCI: Bridge: 0000:00:0b.0
<6>  IO window: 4000-4fff
<6>  MEM window: e8300000-e83fffff
<6>  PREFETCH window: e2100000-e21fffff
<6>PCI: Bridge: 0000:10:0c.0
<6>  IO window: disabled.
<6>  MEM window: disabled.
<6>  PREFETCH window: disabled.
<6>PCI: Bridge: 0000:10:0d.0
<6>  IO window: 5000-5fff
<6>  MEM window: f8100000-f81fffff
<6>  PREFETCH window: e2200000-e22fffff
<6>PCI: Bridge: 0000:28:05.0
<6>  IO window: disabled.
<6>  MEM window: disabled.
<6>  PREFETCH window: disabled.
<6>PCI: Bridge: 0000:28:06.0
<6>  IO window: disabled.
<6>  MEM window: f8500000-f85fffff
<6>  PREFETCH window: e2300000-e23fffff
<6>PCI: Bridge: 0000:38:07.0
<6>  IO window: disabled.
<6>  MEM window: disabled.
<6>  PREFETCH window: disabled.
<6>PCI: Bridge: 0000:38:08.0
<6>  IO window: disabled.
<6>  MEM window: f8900000-f89fffff
<6>  PREFETCH window: e2400000-e24fffff
<6>NET: Registered protocol family 2
<4>IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
<4>TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
<4>TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
<6>TCP: Hash tables configured (established 262144 bind 65536)
<6>TCP reno registered
<6>Simple Boot Flag at 0x37 set to 0x1
<3>microcode: CPU0 not a capable Intel processor
<3>microcode: CPU1 not a capable Intel processor
<3>microcode: CPU2 not a capable Intel processor
<3>microcode: CPU3 not a capable Intel processor
<3>microcode: CPU4 not a capable Intel processor
<3>microcode: CPU5 not a capable Intel processor
<3>microcode: CPU6 not a capable Intel processor
<3>microcode: CPU7 not a capable Intel processor
<3>microcode: CPU8 not a capable Intel processor
<3>microcode: CPU9 not a capable Intel processor
<3>microcode: CPU10 not a capable Intel processor
<3>microcode: CPU11 not a capable Intel processor
<3>microcode: CPU12 not a capable Intel processor
<3>microcode: CPU13 not a capable Intel processor
<3>microcode: CPU14 not a capable Intel processor
<3>microcode: CPU15 not a capable Intel processor
<6>IA-32 Microcode Update Driver: v1.14a <tigran@veritas.com>
<4>Total HugeTLB memory allocated, 0
<5>VFS: Disk quotas dquot_6.5.1
<4>Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
<6>io scheduler noop registered
<6>io scheduler cfq registered (default)
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:00:0a.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:00:0b.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:10:0c.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:10:0d.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:28:05.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:28:06.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:38:07.0 subordinate bus.
<4>PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:38:08.0 subordinate bus.
<6>ACPI: Power Button (FF) [PWRF]
<6>ACPI: Power Button (CM) [PWRB]
<4>Using specific hotkey driver
<6>Real Time Clock Driver v1.12ac
<6>Linux agpgart interface v0.101 (c) Dave Jones
<6>tg3.c:v3.69 (November 15, 2006)
<6>ACPI: PCI Interrupt 0000:03:04.0[A] -> GSI 28 (level, low) -> IRQ 28
<6>eth0: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c3:15
<6>eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth0: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:03:04.1[B] -> GSI 29 (level, low) -> IRQ 29
<6>eth1: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c3:16
<6>eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth1: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:12:04.0[A] -> GSI 44 (level, low) -> IRQ 44
<6>eth2: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c1:33
<6>eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth2: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:12:04.1[B] -> GSI 45 (level, low) -> IRQ 45
<6>eth3: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c1:34
<6>eth3: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth3: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:2a:04.0[A] -> GSI 60 (level, low) -> IRQ 60
<6>eth4: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c2:48
<6>eth4: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth4: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:2a:04.1[B] -> GSI 61 (level, low) -> IRQ 61
<6>eth5: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c2:49
<6>eth5: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth5: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:3a:04.0[A] -> GSI 68 (level, low) -> IRQ 68
<6>eth6: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c2:5a
<6>eth6: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth6: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>ACPI: PCI Interrupt 0000:3a:04.1[B] -> GSI 69 (level, low) -> IRQ 69
<6>eth7: Tigon3 [partno(BCM95704A41) rev 2100 PHY(serdes)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:30:05:71:c2:5b
<6>eth7: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[0] TSOcap[1] 
<6>eth7: dma_rwctrl[769f0000] dma_mask[64-bit]
<6>Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
<6>ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
<6>megasas: 00.00.03.05 Mon Oct 02 11:21:32 PDT 2006
<6>Fusion MPT base driver 3.04.02
<6>Copyright (c) 1999-2005 LSI Logic Corporation
<6>Fusion MPT SPI Host driver 3.04.02
<6>Fusion MPT SAS Host driver 3.04.02
<6>ACPI: PCI Interrupt 0000:02:04.0[A] -> GSI 24 (level, low) -> IRQ 24
<6>mptbase: Initiating ioc0 bringup
<6>ioc0: SAS1068: Capabilities={Initiator}
<6>scsi0 : ioc0: LSISAS1068, FwRev=010a0100h, Ports=1, MaxQ=511, IRQ=24
<5>scsi 0:0:0:0: Direct-Access     SEAGATE  ST3146854SS      1503 PQ: 0 ANSI: 3
<5>scsi 0:0:0:0: Attached scsi generic sg0 type 0
<5>scsi 0:0:1:0: Direct-Access     SEAGATE  ST3146854SS      1503 PQ: 0 ANSI: 3
<5>scsi 0:0:1:0: Attached scsi generic sg1 type 0
<5>scsi 0:1:0:0: Direct-Access     LSILOGIC Logical Volume   3000 PQ: 0 ANSI: 2
<5>SCSI device sda: 285155328 512-byte hdwr sectors (146000 MB)
<5>sda: Write Protect is off
<7>sda: Mode Sense: 03 00 00 08
<5>SCSI device sda: drive cache: write through
<5>SCSI device sda: 285155328 512-byte hdwr sectors (146000 MB)
<5>sda: Write Protect is off
<7>sda: Mode Sense: 03 00 00 08
<5>SCSI device sda: drive cache: write through
<6> sda: sda1 sda2 sda3
<5>sd 0:1:0:0: Attached scsi disk sda
<5>sd 0:1:0:0: Attached scsi generic sg2 type 0
<6>Fusion MPT misc device (ioctl) driver 3.04.02
<6>mptctl: Registered with Fusion MPT base driver
<6>mptctl: /dev/mptctl @ (major,minor=10,220)
<6>PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
<6>serio: i8042 KBD port at 0x60,0x64 irq 1
<6>serio: i8042 AUX port at 0x60,0x64 irq 12
<6>mice: PS/2 mouse device common for all mice
<6>input: PC Speaker as /class/input/input0
<6>input: AT Translated Set 2 keyboard as /class/input/input1
<6>device-mapper: ioctl: 4.10.0-ioctl (2006-09-14) initialised: dm-devel@redhat.com
<6>EDAC MC: Ver: 2.0.1 Nov 30 2006
<6>TCP cubic registered
<6>NET: Registered protocol family 1
<6>NET: Registered protocol family 17
<4>drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
<6>input: ImExPS/2 Generic Explorer Mouse as /class/input/input2
<5>ReiserFS: sda1: found reiserfs format "3.6" with standard journal
<5>ReiserFS: sda1: using ordered data mode
<5>ReiserFS: sda1: journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
<5>ReiserFS: sda1: checking transaction log (sda1)
<5>ReiserFS: sda1: replayed 1501 transactions in 15 seconds
<5>ReiserFS: sda1: Using r5 hash to sort names
<4>VFS: Mounted root (reiserfs filesystem) readonly.
<6>Freeing unused kernel memory: 316k freed
<5>ReiserFS: sda1: Removing [2 473 0x0 SD]..done
<5>ReiserFS: sda1: There were 1 uncompleted unlinks/truncates. Completed
<6>Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
<6>serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
<6>00:0a: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
<6>loop: loaded (max 8 devices)
<5>ReiserFS: dm-0: found reiserfs format "3.6" with standard journal
<5>ReiserFS: dm-0: using ordered data mode
<5>ReiserFS: dm-0: journal params: device dm-0, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
<5>ReiserFS: dm-0: checking transaction log (dm-0)
<5>ReiserFS: dm-0: replayed 114 transactions in 7 seconds
<5>ReiserFS: dm-0: Using r5 hash to sort names
<5>ReiserFS: dm-1: found reiserfs format "3.6" with standard journal
<5>ReiserFS: dm-1: using ordered data mode
<5>ReiserFS: dm-1: journal params: device dm-1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
<5>ReiserFS: dm-1: checking transaction log (dm-1)
<5>ReiserFS: dm-1: replayed 285 transactions in 15 seconds
<5>ReiserFS: dm-1: Using r5 hash to sort names
<5>ReiserFS: dm-2: found reiserfs format "3.6" with standard journal
<5>ReiserFS: dm-2: using ordered data mode
<5>ReiserFS: dm-2: journal params: device dm-2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
<5>ReiserFS: dm-2: checking transaction log (dm-2)
<5>ReiserFS: dm-2: replayed 870 transactions in 23 seconds
<5>ReiserFS: dm-2: Using r5 hash to sort names
<5>ReiserFS: dm-2: Removing [127 1895 0x0 SD]..done
<5>ReiserFS: dm-2: Removing [127 1893 0x0 SD]..done
<5>ReiserFS: dm-2: Removing [4 1889 0x0 SD]..done
<5>ReiserFS: dm-2: There were 3 uncompleted unlinks/truncates. Completed
<5>ReiserFS: dm-6: found reiserfs format "3.6" with standard journal
<5>ReiserFS: dm-6: using ordered data mode
<5>ReiserFS: dm-6: journal params: device dm-6, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
<5>ReiserFS: dm-6: checking transaction log (dm-6)
<5>ReiserFS: dm-6: replayed 152 transactions in 6 seconds
Kernel logging (ksyslog) stopped.
Kernel log daemon terminating.

Boot logging started on /dev/tty1(/dev/console) at Sat Jan 20 04:44:47 2007

Master Resource Control: previous runlevel: N, switching to runlevel: 3
<notice>startproc: execve (/usr/bin/dbus-daemon) [ /usr/bin/dbus-daemon --system ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/bin/dbus-daemon ]
<notice>startproc: execve (/sbin/resmgrd) [ /sbin/resmgrd ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/sbin/resmgrd ]
modprobe: FATAL: Module ac not found.


modprobe: FATAL: Module button not found.


modprobe: FATAL: Module fan not found.


modprobe: FATAL: Module processor not found.


modprobe: FATAL: Module thermal not found.


<notice>startproc: execve (/sbin/acpid) [ /sbin/acpid ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/sbin/acpid ]
[acpid]: starting up


[acpid]: 0 rules loaded


<notice>startproc: execve (/usr/sbin/irqbalance) [ /usr/sbin/irqbalance ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/irqbalance ]
/bin/mknod: `/dev/deadman': File exists
Created /dev/deadman c 10 62
raw driver installed.
Tivoli_lcf1: No such file or directory
acpid: loading ACPI modules ( battery ) failed
acpid: will skip unsupported modules
acpid: loading ACPI laptop modules ( ) done
Starting acpid done
FATAL: Module apparmor not found.
Loading AppArmor module failed
- could not start AppArmorfailed
cmcluster: No such file or directory
Initializing random number generatordone
Starting resource managerdone
Starting D-BUS daemondone
Starting irqbalance done
Starting HAL daemon<notice>startproc: execve (/usr/sbin/hald) [ /usr/sbin/hald --daemon=yes --retain-privileges ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/hald ]
done
Setting up network interfaces:
[acpid]: client connected from 3592[0:0]


[acpid]: 1 client rule loaded


    lo        
    lo        IP address: 127.0.0.1/8   
done    eth0      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
    eth0      configuration: eth-id-00:30:05:71:c3:15
    eth0      IP address: 10.201.25.204/22   
Warning: Could not set up default route via interface 
 Command ip route replace to default via 10.201.11.1 returned:
 . RTNETLINK answers: Network is unreachable
 Configuration line: default 10.201.11.1 - - 
 This needs NOT to be AN ERROR if you set up multiple interfaces.
 See man 5 routes how to avoid this warning.

done    eth1      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
              No configuration found for eth1
unused    eth2      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
    eth2      configuration: eth-id-00:30:05:71:c1:33
    eth2      IP address: 10.201.21.204/22   
Warning: Could not set up default route via interface 
 Command ip route replace to default via 10.201.11.1 returned:
 . RTNETLINK answers: Network is unreachable
 Configuration line: default 10.201.11.1 - - 
 This needs NOT to be AN ERROR if you set up multiple interfaces.
 See man 5 routes how to avoid this warning.

done    eth3      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
              No configuration found for eth3
unused    eth4      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
    eth4      configuration: eth-id-00:30:05:71:c2:48
    eth4      IP address: 10.201.10.204/24   
Warning: Could not set up default route via interface 
 Command ip route replace to default via 10.201.11.1 returned:
 . RTNETLINK answers: Network is unreachable
 Configuration line: default 10.201.11.1 - - 
 This needs NOT to be AN ERROR if you set up multiple interfaces.
 See man 5 routes how to avoid this warning.

done    eth5      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
              No configuration found for eth5
unused    eth6      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
    eth6      configuration: eth-id-00:30:05:71:c2:5a
    eth6      IP address: 10.201.11.204/24   
done    eth7      device: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
              No configuration found for eth7
unusedSetting up service network  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .done
Starting syslog services<notice>startproc: execve (/sbin/syslogd) [ /sbin/syslogd -a /var/lib/dhcp/dev/log -a /var/lib/named/dev/log ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/sbin/syslogd ]
<notice>startproc: execve (/sbin/klogd) [ /sbin/klogd -c 1 -x -x ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/sbin/klogd ]
done
<notice>startproc: execve (/sbin/portmap) [ /sbin/portmap ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/sbin/portmap ]
Starting RPC portmap daemondone
Importing Net File System (NFS)unused
Checking/updating CPU microcode unused
Starting nfsboot (sm-notify) done
Starting ZENworks Management Daemondone
<notice>startproc: execve (/usr/sbin/sshd) [ /usr/sbin/sshd -o PidFile=/var/run/sshd.init.pid ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/sshd ]
<notice>checkproc: /sbin/portmap 4298
<notice>startproc: execve (/usr/sbin/ypbind) [ /usr/sbin/ypbind ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/ypbind ]
Starting SSH daemondone
Jan 22 04:56:38 frey ntpdate[4397]: step time server 10.201.10.68 offset 173505.656746 sec

<notice>startproc: execve (/usr/sbin/ntpd) [ /usr/sbin/ntpd -p /var/lib/ntp/var/run/ntp/ntpd.pid -u ntp -i /var/lib/ntp ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/ntpd ]
Starting multipathdLoading keymap i386/qwerty/it.map.gz
doneStarting ypbind . .Jan 22 04:56:38 frey ntpd[4589]: ntpd 4.2.0a@1.1196-r Thu Jun 29 17:48:00 UTC 2006 (1)

<notice>startproc: execve (/usr/sbin/nscd) [ /usr/sbin/nscd ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/nscd ]
<notice>startproc: execve (/usr/sbin/atd) [ /usr/sbin/atd ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/atd ]
<notice>startproc: execve (/usr/sbin/famd) [ /usr/sbin/famd -t 4 -T 0 -L ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/famd ]
<notice>startproc: execve (/usr/sbin/sendmail) [ /usr/sbin/sendmail -L sendmail -Am -bd -q15m -om ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/sendmail ]
<notice>startproc: execve (/usr/sbin/sendmail) [ /usr/sbin/sendmail -L sendmail-client -Ac -qp1m ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/sendmail ]
<notice>startproc: execve (/usr/sbin/cron) [ /usr/sbin/cron ], [ CONSOLE=/dev/console SHELL=/bin/sh TERM=linux LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=25 SHLVL=2 HOME=/ BOOT_IMAGE=Linux SPLASH=no _=/sbin/startproc DAEMON=/usr/sbin/cron ]
No NIS server and no -broadcast option specified.
Add a NIS server to the /etc/yp.conf configuration file,
or start ypbind with the -broadcast option.
done
Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd000000000000171d: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 65:160 1000 8:17

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd000000000000171e: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 65:176 1000 8:19

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd000000000000171f: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 65:192 1000 8:20

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001720: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 65:208 1000 8:22

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001721: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 65:224 1000 8:24

Jan 22 04:57:52 freyJan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001714: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:32 1000 65:16 

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001715: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:48 1000 65:32 

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001716: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:64 1000 65:48 

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001717: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:80 1000 65:64 

Jan 22 04:57:52 frey multipathd: 36005076303ffc4bd0000000000001718: load table [0 69206016 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:96 1000 65:80 

..dead
Try to get initial date and time via NTP from  10.201.10.68 10.201.10.69done
Starting network time protocol daemon (NTPD)done
Starting Name Service Cache Daemondone
Starting service at daemondone
Starting File Access Monitoring Daemondone
Loading compose table latin1.adddone
Start Unicode mode
doneLoading console font lat9w-16.psfu  -m trivial G0:loadable
done . . . . . . . . . . . . . No NIS server founddone
Starting service automounter ("files nis" does not provide any mounts)skipped
Initializing SMTP port (sendmail)done
Starting CRON daemondone
Master Resource Control: runlevel 3 has been reached
Failed services in runlevel 3: Tivoli_lcf1 acpid cmcluster
Skipped services in runlevel 3: nfs microcode autofs
<notice>killproc: kill(3308,3)

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: System crash after "No irq handler for vector" linux 2.6.19
       [not found] <200701221116.13154.luigi.genoni@pirelli.com>
@ 2007-01-22 17:14 ` Eric W. Biederman
       [not found]   ` <200701231051.32945.luigi.genoni@pirelli.com>
  0 siblings, 1 reply; 12+ messages in thread
From: Eric W. Biederman @ 2007-01-22 17:14 UTC (permalink / raw)
  To: Luigi Genoni; +Cc: akpm, ebiederm, linux-kernel

"Luigi Genoni" <luigi.genoni@pirelli.com> writes:

> (e-mail resent because not delivered using my other e-mail account)
>
> Hi,
> this night a linux server 8 dual core CPU Optern 2600Mhz crashed just after
> giving this message
>
> Jan 22 04:48:28 frey kernel: do_IRQ: 1.98 No irq handler for vector

Ok.  This indicates that the hardware is doing something we didn't expect.
We don't know which irq the hardware was trying to deliver when it
sent vector 0x98 to cpu 1.

> I have no other logs, and I eventually lost the OOPS since I have no net
> console setled up.

If you had an oops it may have meant the above message was a secondary
symptom.  Groan.  If it stayed up long enough to give an OOPS then
there is a chance the above message appearing only once had nothing
to do with the actual crash.

How long had the system been up?

> As I said sistem is running linux 2.6.19 compiled with gcc 4.1.1 for AMD
> Opteron (attached see .config), no kernel preemption excepted the BKL
> preemption. glibc 2.4.
>
> System has 16 GB RAM and 8 dual core Opteron 2600Mhz.
>
> I am running irqbalance 0.55.
>
> any hints on what has happened?

Three guesses.

- A race triggered by irq migration (but I would expect more people to be yelling).
  The code path where that message comes from is new in 2.6.19 so it may not have
  had all of the bugs found yet :(
- A weird hardware or BIOS setup.
- A secondary symptom triggered by some other bug.

If this winds up being reproducible we should be able to track it down.
If not this may end up in the files of crap something bad happened that
we don't understand.

The one condition I know how to test for (if you are willing) is an
irq migration race.  Simply by triggering irq migration much more often,
and thus increasing our chances of hitting a problem.

Stopping irqbalance and running something like:
for irq in 0 24 28 29 44 45 60 68 ; do
	while :; do
		for mask in 1 2 4 8 10 20 40 80 100 200 400 800 1000 2000 4000 8000 ; do
        		echo mask > /proc/irq/$irq/smp_affinity
	                sleep 1
	        done
	done &
done

Should force every irq to migrate once a second, and removing the sleep 1
is even harsher, although we max at one irq migration by irq received.

If some variation of the above loop does not trigger the do_IRQ ??? No irq handler
for vector message chances are it isn't a race in irq migration.

If we can rule out the race scenario it will at least put us in the right direction
for guessing what went wrong with your box.

Eric

^ permalink raw reply	[flat|nested] 12+ messages in thread

* System crash after "No irq handler for vector" linux 2.6.19
@ 2007-01-22 10:06 l.genoni
  0 siblings, 0 replies; 12+ messages in thread
From: l.genoni @ 2007-01-22 10:06 UTC (permalink / raw)
  To: akpm, ebiederm; +Cc: linux-kernel, ezio.bombardelli, gianni.minoia

[-- Attachment #1: Type: TEXT/PLAIN, Size: 581 bytes --]

Hi,
this night a linux server 8 dual core CPU Optern 2600Mhz crashed just 
after giving this message

Jan 22 04:48:28 frey kernel: do_IRQ: 1.98 No irq handler for vector

I have no other logs, and I eventually lost the OOPS since I have no net 
console setled up.

As I said sistem is running linux 2.6.19 compiled with gcc 4.1.1 for AMD 
Opteron (attached see .config), no kernel preemption excepted the BKL 
preemption. glibc 2.4.

System has 16 GB RAM and 8 dual core Opteron 2600Mhz.

I am running irqbalance 0.55.

any hints on what has happened?

thanx

regards

Luigi Genoni

[-- Attachment #2: Type: TEXT/PLAIN, Size: 10502 bytes --]

processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5197.21
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 1
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.42
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 2
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 1
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.42
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 3
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 1
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.42
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 4
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 2
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.42
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 5
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 2
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.42
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 6
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.41
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 7
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.41
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 8
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 4
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.41
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 9
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 4
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.40
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 10
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 5
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.40
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 11
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 5
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.39
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 12
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 6
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.40
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 13
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 6
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.40
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 14
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 7
siblings	: 2
core id		: 0
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.47
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor	: 15
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 33
model name	: Dual Core AMD Opteron(tm) Processor 885
stepping	: 2
cpu MHz		: 2596.653
cache size	: 1024 KB
physical id	: 7
siblings	: 2
core id		: 1
cpu cores	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips	: 5193.47
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: ts fid vid ttp


[-- Attachment #3: Type: TEXT/PLAIN, Size: 20310 bytes --]

00:06.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8111 PCI (rev 07) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 115
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=68
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: e8100000-e81fffff
	Prefetchable memory behind bridge: f0000000-f7ffffff
	Capabilities: [c0] HyperTransport: Slave or Primary Interface
	Capabilities: [f0] HyperTransport: Interrupt Discovery and Configuration

00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-8111 LPC (rev 05)
	Flags: bus master, 66MHz, medium devsel, latency 0

00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-8111 ACPI (rev 05)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: medium devsel

00:0a.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=64
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: e8200000-e82fffff
	Prefetchable memory behind bridge: 00000000e2000000-00000000e2000000
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

00:0a.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at e8000000 (64-bit, non-prefetchable) [size=4K]

00:0b.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=64
	I/O behind bridge: 00004000-00004fff
	Memory behind bridge: e8300000-e83fffff
	Prefetchable memory behind bridge: 00000000e2100000-00000000e2100000
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at e8001000 (64-bit, non-prefetchable) [size=4K]

00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:19.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:19.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:19.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:19.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1a.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1a.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1a.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1a.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1b.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1b.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1b.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1b.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1c.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1c.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1c.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1c.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1d.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1d.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1d.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1d.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1e.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1e.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1e.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1e.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:1f.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:1f.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:1f.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:1f.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

01:00.0 USB Controller: Advanced Micro Devices [AMD] AMD-8111 USB (rev 0b) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, medium devsel, latency 64, IRQ 11
	Memory at e8110000 (32-bit, non-prefetchable) [size=4K]

01:00.1 USB Controller: Advanced Micro Devices [AMD] AMD-8111 USB (rev 0b) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, medium devsel, latency 64, IRQ 11
	Memory at e8111000 (32-bit, non-prefetchable) [size=4K]

01:05.0 VGA compatible controller: ATI Technologies Inc Radeon RV100 QY [Radeon 7000/VE] (prog-if 00 [VGA])
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, stepping, medium devsel, latency 66, IRQ 10
	Memory at f0000000 (32-bit, prefetchable) [size=128M]
	I/O ports at 2000 [size=256]
	Memory at e8100000 (32-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e8120000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 2

02:04.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS (rev 01)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 10ba
	Flags: bus master, 66MHz, medium devsel, latency 72, IRQ 24
	I/O ports at 3000 [disabled] [size=256]
	Memory at e8210000 (64-bit, non-prefetchable) [size=16K]
	Memory at e8200000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2000000 [disabled] [size=1M]
	Capabilities: [50] Power Management version 2
	Capabilities: [98] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
	Capabilities: [68] PCI-X non-bridge device
	Capabilities: [b0] MSI-X: Enable- Mask- TabSize=1

03:02.0 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1051
	Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 28
	I/O ports at 4000 [size=256]
	Memory at e8320000 (64-bit, non-prefetchable) [size=4K]
	[virtual] Expansion ROM at e2100000 [disabled] [size=128K]
	Capabilities: [44] Power Management version 2
	Capabilities: [4c] PCI-X non-bridge device
	Capabilities: [54] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
	Capabilities: [64] #06 [0080]

03:02.1 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1051
	Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 29
	I/O ports at 4400 [size=256]
	Memory at e8321000 (64-bit, non-prefetchable) [size=4K]
	[virtual] Expansion ROM at e2120000 [disabled] [size=128K]
	Capabilities: [44] Power Management version 2
	Capabilities: [4c] PCI-X non-bridge device
	Capabilities: [54] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
	Capabilities: [64] #06 [0080]

03:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 28
	Memory at e8300000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2140000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

03:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 29
	Memory at e8310000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2150000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

10:0c.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=10, secondary=11, subordinate=11, sec-latency=64
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

10:0c.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8010000 (64-bit, non-prefetchable) [size=4K]

10:0d.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=10, secondary=12, subordinate=12, sec-latency=64
	I/O behind bridge: 00005000-00005fff
	Memory behind bridge: f8100000-f81fffff
	Prefetchable memory behind bridge: 00000000e2200000-00000000e2200000
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

10:0d.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8011000 (64-bit, non-prefetchable) [size=4K]

12:02.0 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1051
	Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 44
	I/O ports at 5000 [size=256]
	Memory at f8120000 (64-bit, non-prefetchable) [size=4K]
	[virtual] Expansion ROM at e2200000 [disabled] [size=128K]
	Capabilities: [44] Power Management version 2
	Capabilities: [4c] PCI-X non-bridge device
	Capabilities: [54] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
	Capabilities: [64] #06 [0080]

12:02.1 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1051
	Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 45
	I/O ports at 5400 [size=256]
	Memory at f8121000 (64-bit, non-prefetchable) [size=4K]
	[virtual] Expansion ROM at e2220000 [disabled] [size=128K]
	Capabilities: [44] Power Management version 2
	Capabilities: [4c] PCI-X non-bridge device
	Capabilities: [54] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-
	Capabilities: [64] #06 [0080]

12:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 44
	Memory at f8100000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2240000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

12:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 45
	Memory at f8110000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2250000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

28:05.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=28, secondary=29, subordinate=29, sec-latency=64
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

28:05.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8410000 (64-bit, non-prefetchable) [size=4K]

28:06.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=28, secondary=2a, subordinate=2a, sec-latency=64
	Memory behind bridge: f8500000-f85fffff
	Prefetchable memory behind bridge: 00000000e2300000-00000000e2300000
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

28:06.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8411000 (64-bit, non-prefetchable) [size=4K]

2a:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 60
	Memory at f8500000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2300000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

2a:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 61
	Memory at f8510000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2310000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

38:07.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=38, secondary=39, subordinate=39, sec-latency=64
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

38:07.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8800000 (64-bit, non-prefetchable) [size=4K]

38:08.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=38, secondary=3a, subordinate=3a, sec-latency=64
	Memory behind bridge: f8900000-f89fffff
	Prefetchable memory behind bridge: 00000000e2400000-00000000e2400000
	Capabilities: [a0] PCI-X bridge device
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

38:08.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
	Flags: bus master, medium devsel, latency 0
	Memory at f8801000 (64-bit, non-prefetchable) [size=4K]

3a:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 68
	Memory at f8900000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2400000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

3a:04.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10)
	Subsystem: Fujitsu Siemens Computer GmbH Unknown device 1086
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 69
	Memory at f8910000 (64-bit, non-prefetchable) [size=64K]
	[virtual] Expansion ROM at e2410000 [disabled] [size=64K]
	Capabilities: [40] PCI-X non-bridge device
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-


[-- Attachment #4: Type: TEXT/PLAIN, Size: 3082 bytes --]

           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15      
  0:    2199854          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      timer
  1:         21        120          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      i8042
  8:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      rtc
  9:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   acpi
 12:        112          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      i8042
 24:      36392      20462       1673       1677          0          0          0          0          0       1421          0       5004          0       1190          0          0   IO-APIC-fasteoi   ioc0
 28:     327502     613799      86142      64687          0          0          0          0          0          0          0          0          0      20779          0     115682   IO-APIC-fasteoi   eth0, qla2xxx
 29:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   qla2xxx
 44:     233568    2065753          0     242794          0    1151595          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   eth2, qla2xxx
 45:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   qla2xxx
 60:     314328     592572      57878      33951          0          0          0          0          0          0          0      65143          0       1933          0          0   IO-APIC-fasteoi   eth4
 68:      22746     275530     178481     224048          0          0          0          0          0          0          0          0          0     110545          0     118910   IO-APIC-fasteoi   eth6
NMI:       3004       1448       1731       1948       2469       1947       2517       1705       1919       2307       1988       2468       2024       2236       2354       1410 
LOC:    2199784    2199767    2199750    2199733    2199716    2199698    2199682    2199665    2199648    2199631    2199614    2199597    2199580    2199563    2199546    2199529 
ERR:          0

[-- Attachment #5: Type: APPLICATION/octet-stream, Size: 9304 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2007-02-03  0:40 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-01-23 10:19 System crash after "No irq handler for vector" linux 2.6.19 l.genoni
  -- strict thread matches above, loose matches on Subject: below --
2007-02-01 13:33 Chris Rankin
2007-01-23 10:35 l.genoni
     [not found] <200701221116.13154.luigi.genoni@pirelli.com>
2007-01-22 17:14 ` Eric W. Biederman
     [not found]   ` <200701231051.32945.luigi.genoni@pirelli.com>
2007-01-23 12:18     ` Eric W. Biederman
     [not found]       ` <Pine.LNX.4.64.0701232052330.32111@baldios.it.pirelli.com>
2007-01-31  8:39         ` Eric W. Biederman
     [not found]           ` <200701311549.22512.luigi.genoni@pirelli.com>
2007-02-01  5:59             ` Eric W. Biederman
2007-02-01  7:20             ` Eric W. Biederman
     [not found]               ` <200702021848.55921.luigi.genoni@pirelli.com>
2007-02-02 18:02                 ` Eric W. Biederman
     [not found]                   ` <200702021905.39922.luigi.genoni@pirelli.com>
2007-02-02 18:32                     ` Eric W. Biederman
2007-02-03  0:40                     ` Eric W. Biederman
2007-01-22 10:06 l.genoni

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).