LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* A note on APIC bus latency
@ 2001-10-08 21:51 Jonathan Lundell
  2001-10-08 22:08 ` Alan Cox
  0 siblings, 1 reply; 2+ messages in thread
From: Jonathan Lundell @ 2001-10-08 21:51 UTC (permalink / raw)
  To: linux-kernel

We recently ran into some issues caused by APIC bus latency. I was 
reminded of that by the recent discussion of NAPI and related 
interrupt-performance matters.

Intel processors that predate Pentium 4 but use an APIC transmit APIC 
messages over a serial APIC bus, typically at 16.7 MHz. (Pentium 4 
uses the system bus for APIC messages.)

A message exchange (IO-APIC sends an interrupt message; CPU sends 
back an EOI message) requires from 35 to 48 APIC bus clocks, or 2-3 
microseconds. That gets to be pretty significant compared to packet 
times, especially at Gbit speeds, but even at 100 MHz, and is the 
time required to burst a thousand bytes or more at faster PCI rates.

It's also likely to be significant for inter-processor interrupts, 
though I don't know what the implications are here.
-- 
/Jonathan Lundell.

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: A note on APIC bus latency
  2001-10-08 21:51 A note on APIC bus latency Jonathan Lundell
@ 2001-10-08 22:08 ` Alan Cox
  0 siblings, 0 replies; 2+ messages in thread
From: Alan Cox @ 2001-10-08 22:08 UTC (permalink / raw)
  To: Jonathan Lundell; +Cc: linux-kernel

> A message exchange (IO-APIC sends an interrupt message; CPU sends 
> back an EOI message) requires from 35 to 48 APIC bus clocks, or 2-3 
> microseconds. That gets to be pretty significant compared to packet 
> times, especially at Gbit speeds, but even at 100 MHz, and is the 
> time required to burst a thousand bytes or more at faster PCI rates.
> 
> It's also likely to be significant for inter-processor interrupts, 
> though I don't know what the implications are here.

The big implication so far has been some extremely horrible to debug 
irq handling bugs where drivers such as the i810 audio assumed that the
disable of an irq on the pci device was immediate once then pci write
and a pci read to force posting completed. 

There are impacts on things like TLB shootdowns where the latency impacts
an SMP crosscall. I'm not sure how bad the impact is on the bigger numa
boxes as I notice Martin uses multiple sends for that

Alan

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2001-10-08 22:02 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-10-08 21:51 A note on APIC bus latency Jonathan Lundell
2001-10-08 22:08 ` Alan Cox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).