Re: 2.6.24-git: kmap_atomic() WARN_ON()

[Andrew Morton <akpm@linux-foundation.org>]

On Mon, 25 Feb 2008 15:08:35 -0500 Jeff Garzik <jgarzik@pobox.com> wrote:

> Bj__rn Steinbrink wrote:
> > On 2008.02.07 00:58:42 +0100, Thomas Gleixner wrote:
> >> current mainline triggers:
> >>
> >> WARNING: at /home/tglx/work/kernel/x86/linux-2.6/arch/x86/mm/highmem_32.c:52 kmap_atomic_prot+0xe5/0x19b()
> >> Modules linked in: ahci(+) sata_sil libata sd_mod scsi_mod raid1 ext3 jbd ehci_hcd ohci_hcd uhci_hcd
> >> Pid: 0, comm: swapper Not tainted 2.6.24 #173
> >>  [<c0126b60>] warn_on_slowpath+0x41/0x51
> >>  [<c011c5eb>] ? __enqueue_entity+0x9c/0xa4
> >>  [<c011c717>] ? enqueue_entity+0x124/0x13b
> >>  [<c011cedb>] ? enqueue_task_fair+0x41/0x4c
> >>  [<c032ce88>] ? _spin_lock_irqsave+0x14/0x2e
> >>  [<c012e885>] ? lock_timer_base+0x1f/0x3e
> >>  [<c011ad6d>] kmap_atomic_prot+0xe5/0x19b
> >>  [<c011ae37>] kmap_atomic+0x14/0x16
> >>  [<f88ea218>] ata_scsi_rbuf_get+0x1e/0x2c [libata]
> >>  [<f88ea821>] atapi_qc_complete+0x23f/0x289 [libata]
> >>  [<f88e3d00>] __ata_qc_complete+0x8e/0x93 [libata]
> >>  [<f88e3fc7>] ata_qc_complete+0x115/0x128 [libata]
> >>  [<f88e8d47>] ata_qc_complete_multiple+0x86/0xa0 [libata]
> >>  [<f8841a5d>] ahci_interrupt+0x370/0x40d [ahci]
> >>  [<c01512c6>] handle_IRQ_event+0x21/0x48
> >>  [<c01521ca>] handle_edge_irq+0xc9/0x10a
> >>  [<c0152101>] ? handle_edge_irq+0x0/0x10a
> >>  [<c0107518>] do_IRQ+0x8b/0xb7
> >>  [<c01055db>] common_interrupt+0x23/0x28
> >>  [<c032007b>] ? init_chipset_cmd64x+0xb/0x93
> >>  [<c010316e>] ? mwait_idle_with_hints+0x39/0x3d
> >>  [<c0103172>] ? mwait_idle+0x0/0xf
> >>  [<c010317f>] mwait_idle+0xd/0xf
> >>  [<c0103761>] cpu_idle+0xb0/0xe4
> >>  [<c031b509>] rest_init+0x5d/0x5f
> >>
> >> This is not a new problem. It was pointed out some time ago already,
> >> but now the WARN_ON() finally made it into mainline :)
> >>
> >> The fix is not obvious, as this code seems to be called from various
> >> call sites.
> > 
> > Hm, do you have lockdep enabled? If not, does lockdep make this go away?
> > Because lockdep will set IRQF_DISABLED for all interrupt handlers, and
> > unless that flag is set, handle_IRQ_event will reenable interrupts while
> > the handler is running. And ahci_interrupt only uses a plain spin_lock,
> > so interrupts keep being enabled. The patch below should help with that.
> > 
> > Hmhm, maybe that also solves the deadlock you saw? Dunno...
> > 
> > I can't come up with an useful commit message right now, but I'll resend
> > in suitable form for submission if that thing actually works.
> > 
> > Bj__rn
> > 
> > 
> > diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
> > index 1db93b6..ae3dbc8 100644
> > --- a/drivers/ata/ahci.c
> > +++ b/drivers/ata/ahci.c
> > @@ -1739,6 +1739,7 @@ static irqreturn_t ahci_interrupt(int irq, void *dev_instance)
> >  	unsigned int i, handled = 0;
> >  	void __iomem *mmio;
> >  	u32 irq_stat, irq_ack = 0;
> > +	unsigned long flags;
> >  
> >  	VPRINTK("ENTER\n");
> >  
> > @@ -1751,7 +1752,7 @@ static irqreturn_t ahci_interrupt(int irq, void *dev_instance)
> >  	if (!irq_stat)
> >  		return IRQ_NONE;
> >  
> > -	spin_lock(&host->lock);
> > +	spin_lock_irqsave(&host->lock, flags);
> >  
> >  	for (i = 0; i < host->n_ports; i++) {
> >  		struct ata_port *ap;
> > @@ -1778,7 +1779,7 @@ static irqreturn_t ahci_interrupt(int irq, void *dev_instance)
> >  		handled = 1;
> >  	}
> >  
> > -	spin_unlock(&host->lock);
> > +	spin_unlock_irqrestore(&host->lock, flags);
> 
> If this truly fixes the problem, then lockdep is definitely the problem 
> source.
> 
> There are plenty of drivers that do the same thing that ahci does, in 
> terms of interrupt handler locking...  and I will definitely push back 
> on efforts to convert otherwise-100%-safe spin_lock() into 
> spin_lock_irqsave() just to quiet lockdep.
> 
> Very interesting email, thanks...
> 

I suspect this is a bug in my old kmap_atomic debugging patch.  It doesn't
know about the implicit irq-disablememnt which interrupt handlers enjoy.  I
don't think...