LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* pci-ftpci100: race condition in masking/unmasking interrupts
@ 2021-08-18 11:47 Pali Rohár
  2021-09-07 11:22 ` Linus Walleij
  0 siblings, 1 reply; 4+ messages in thread
From: Pali Rohár @ 2021-08-18 11:47 UTC (permalink / raw)
  To: Linus Walleij, Lorenzo Pieralisi, Rob Herring
  Cc: Marc Zyngier, linux-pci, linux-kernel

Hello!

I do not see any entry in MAINTAINERS file for pci-ftpci100.c driver, so
I'm not sure to whom should I address this issue...

During pci-aardvark review, Marc pointed one issue which is currently
available also in pci-ftpci100.c driver.

When masking or unmasking interrupts there is read-modify-write sequence
for FARADAY_PCI_CTRL2 register without any locking and is not atomic:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-ftpci100.c?h=v5.13#n270

So there is race condition when masking/unmasking more interrupts at the
same time.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: pci-ftpci100: race condition in masking/unmasking interrupts
  2021-08-18 11:47 pci-ftpci100: race condition in masking/unmasking interrupts Pali Rohár
@ 2021-09-07 11:22 ` Linus Walleij
  2021-09-07 12:20   ` Pali Rohár
  2021-09-07 16:53   ` Marc Zyngier
  0 siblings, 2 replies; 4+ messages in thread
From: Linus Walleij @ 2021-09-07 11:22 UTC (permalink / raw)
  To: Pali Rohár
  Cc: Lorenzo Pieralisi, Rob Herring, Marc Zyngier, linux-pci, linux-kernel

On Wed, Aug 18, 2021 at 1:47 PM Pali Rohár <pali@kernel.org> wrote:

> I do not see any entry in MAINTAINERS file for pci-ftpci100.c driver, so
> I'm not sure to whom should I address this issue...

It's me.

> During pci-aardvark review, Marc pointed one issue which is currently
> available also in pci-ftpci100.c driver.
>
> When masking or unmasking interrupts there is read-modify-write sequence
> for FARADAY_PCI_CTRL2 register without any locking and is not atomic:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-ftpci100.c?h=v5.13#n270
>
> So there is race condition when masking/unmasking more interrupts at the
> same time.

I thought those operations were called in atomic context.
How did you fix it?

Yours,
Linus Walleij

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: pci-ftpci100: race condition in masking/unmasking interrupts
  2021-09-07 11:22 ` Linus Walleij
@ 2021-09-07 12:20   ` Pali Rohár
  2021-09-07 16:53   ` Marc Zyngier
  1 sibling, 0 replies; 4+ messages in thread
From: Pali Rohár @ 2021-09-07 12:20 UTC (permalink / raw)
  To: Linus Walleij
  Cc: Lorenzo Pieralisi, Rob Herring, Marc Zyngier, linux-pci, linux-kernel

On Tuesday 07 September 2021 13:22:37 Linus Walleij wrote:
> On Wed, Aug 18, 2021 at 1:47 PM Pali Rohár <pali@kernel.org> wrote:
> 
> > I do not see any entry in MAINTAINERS file for pci-ftpci100.c driver, so
> > I'm not sure to whom should I address this issue...
> 
> It's me.

Ok! So could you send update for MAINTAINERS file for this driver?

> > During pci-aardvark review, Marc pointed one issue which is currently
> > available also in pci-ftpci100.c driver.
> >
> > When masking or unmasking interrupts there is read-modify-write sequence
> > for FARADAY_PCI_CTRL2 register without any locking and is not atomic:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-ftpci100.c?h=v5.13#n270
> >
> > So there is race condition when masking/unmasking more interrupts at the
> > same time.
> 
> I thought those operations were called in atomic context.

I guess that they cannot be as for performance reasons you could want to
mask or unmask more interrupts in parallel on more CPUs.

> How did you fix it?

Guarding all read-modify-write operations on register by raw spin lock. See:
https://lore.kernel.org/linux-pci/20210820155020.3000-1-pali@kernel.org/

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: pci-ftpci100: race condition in masking/unmasking interrupts
  2021-09-07 11:22 ` Linus Walleij
  2021-09-07 12:20   ` Pali Rohár
@ 2021-09-07 16:53   ` Marc Zyngier
  1 sibling, 0 replies; 4+ messages in thread
From: Marc Zyngier @ 2021-09-07 16:53 UTC (permalink / raw)
  To: Linus Walleij
  Cc: Pali Rohár, Lorenzo Pieralisi, Rob Herring, linux-pci, linux-kernel

Hi Linus,

On Tue, 07 Sep 2021 12:22:37 +0100,
Linus Walleij <linus.walleij@linaro.org> wrote:
> 
> On Wed, Aug 18, 2021 at 1:47 PM Pali Rohár <pali@kernel.org> wrote:
> 
> > I do not see any entry in MAINTAINERS file for pci-ftpci100.c driver, so
> > I'm not sure to whom should I address this issue...
> 
> It's me.
> 
> > During pci-aardvark review, Marc pointed one issue which is currently
> > available also in pci-ftpci100.c driver.
> >
> > When masking or unmasking interrupts there is read-modify-write sequence
> > for FARADAY_PCI_CTRL2 register without any locking and is not atomic:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-ftpci100.c?h=v5.13#n270
> >
> > So there is race condition when masking/unmasking more interrupts at the
> > same time.
> 
> I thought those operations were called in atomic context.
> How did you fix it?

They are.

But that doesn't mean that you cannot have two CPUs dealing with two
different interrupts at the same time (using disable_irq(), for
example). When that happens, your interrupt masking becomes a bit
soup. irq_ack() also gets in the way, as it does a RMW of the same
register. If the underlying HW is strictly UP, you're safe. But even
in this case, you could have some locking that gets elided at compile
time.

I also don't understand why you always clear the interrupt status
every time you mask/unmask an interrupt.

I came up with the following patchlet, which is completely untested
(not even compile-tested).

Thanks,

	M.

diff --git a/drivers/pci/controller/pci-ftpci100.c b/drivers/pci/controller/pci-ftpci100.c
index 88980a44461d..dd1697e61206 100644
--- a/drivers/pci/controller/pci-ftpci100.c
+++ b/drivers/pci/controller/pci-ftpci100.c
@@ -120,6 +120,7 @@ struct faraday_pci_variant {
 };
 
 struct faraday_pci {
+	raw_spinlock_t lock
 	struct device *dev;
 	void __iomem *base;
 	struct irq_domain *irqdomain;
@@ -270,34 +271,41 @@ static struct pci_ops faraday_pci_ops = {
 static void faraday_pci_ack_irq(struct irq_data *d)
 {
 	struct faraday_pci *p = irq_data_get_irq_chip_data(d);
+	unsigned long flags;
 	unsigned int reg;
 
+	raw_spin_lock_irqsave(&p->lock, flags);
 	faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, &reg);
 	reg &= ~(0xF << PCI_CTRL2_INTSTS_SHIFT);
 	reg |= BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTSTS_SHIFT);
 	faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg);
+	raw_spin_unlock_irqrestore(&p->lock, flags);
 }
 
 static void faraday_pci_mask_irq(struct irq_data *d)
 {
 	struct faraday_pci *p = irq_data_get_irq_chip_data(d);
+	unsigned long flags;
 	unsigned int reg;
 
+	raw_spin_lock_irqsave(&p->lock, flags);
 	faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, &reg);
-	reg &= ~((0xF << PCI_CTRL2_INTSTS_SHIFT)
-		 | BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT));
+	reg &= ~BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT);
 	faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg);
+	raw_spin_unlock_irqrestore(&p->lock, flags);
 }
 
 static void faraday_pci_unmask_irq(struct irq_data *d)
 {
 	struct faraday_pci *p = irq_data_get_irq_chip_data(d);
+	unsigned long flags;
 	unsigned int reg;
 
+	raw_spin_lock_irqsave(&p->lock, flags);
 	faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, &reg);
-	reg &= ~(0xF << PCI_CTRL2_INTSTS_SHIFT);
 	reg |= BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT);
 	faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg);
+	raw_spin_unlock_irqrestore(&p->lock, flags);
 }
 
 static void faraday_pci_irq_handler(struct irq_desc *desc)
@@ -441,6 +449,8 @@ static int faraday_pci_probe(struct platform_device *pdev)
 	host->sysdata = p;
 	p->dev = dev;
 
+	raw_spin_lock_init(&p->lock);
+
 	/* Retrieve and enable optional clocks */
 	clk = devm_clk_get(dev, "PCLK");
 	if (IS_ERR(clk))

-- 
Without deviation from the norm, progress is not possible.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-09-07 16:54 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-18 11:47 pci-ftpci100: race condition in masking/unmasking interrupts Pali Rohár
2021-09-07 11:22 ` Linus Walleij
2021-09-07 12:20   ` Pali Rohár
2021-09-07 16:53   ` Marc Zyngier

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).