LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Will Deacon <will@kernel.org>
To: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Alan Stern <stern@rowland.harvard.edu>,
	Marco Elver <elver@google.com>, Boqun Feng <boqun.feng@gmail.com>,
	Andrea Parri <parri.andrea@gmail.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Dmitry Vyukov <dvyukov@google.com>,
	kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org
Subject: Re: LKMM: Read dependencies of writes ordered by dma_wmb()?
Date: Wed, 18 Aug 2021 12:39:36 +0100	[thread overview]
Message-ID: <20210818113935.GA14107@willie-the-truck> (raw)
In-Reply-To: <20210817135308.GO4126399@paulmck-ThinkPad-P17-Gen-1>

Hi Paul.

On Tue, Aug 17, 2021 at 06:53:08AM -0700, Paul E. McKenney wrote:
> On Tue, Aug 17, 2021 at 01:28:16PM +0100, Will Deacon wrote:
> > Just on this bit...
> > 
> > On Mon, Aug 16, 2021 at 01:50:57PM -0700, Paul E. McKenney wrote:
> > > 5.	The dma_mb(), dma_rmb(), and dma_wmb() appear to be specific
> > > 	to ARMv8.
> > 
> > These are useful on other architectures too! IIRC, they were added by x86 in
> > the first place. They're designed to be used with dma_alloc_coherent()
> > allocations where you're sharing something like a ring buffer with a device
> > and they guarantee accesses won't be reordered before they become visible
> > to the device. They _also_ provide the same ordering to other CPUs.
> > 
> > I gave a talk at LPC about some of this, which might help (or might make
> > things worse...):
> > 
> > https://www.youtube.com/watch?v=i6DayghhA8Q
> 
> The slides are here, correct?  Nice summary and examples!
> 
> https://elinux.org/images/a/a8/Uh-oh-Its-IO-Ordering-Will-Deacon-Arm.pdf

Yes, that looks like them. I've also put them up here:

https://mirrors.edge.kernel.org/pub/linux/kernel/people/will/slides/elce-2018.pdf

(turns out it was ELCE not LPC!)

> And this is all I see for dma_mb():
> 
> arch/arm64/include/asm/barrier.h:#define dma_mb()	dmb(osh)
> arch/arm64/include/asm/io.h:#define __iomb()		dma_mb()
> 
> And then for __iomb():
> 
> arch/arm64/include/asm/io.h:#define __iomb()		dma_mb()
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c:	__iomb();
> 
> But yes, dma_rmb() and dma_wmb() do look to have a few hundred uses
> between them, and not just within ARMv8.  I gave up too soon, so
> thank you!

No problem, and yes, dma_mb() is an arm64-internal thing which we should
probably rename.

> > Ignore the bits about mmiowb() as we got rid of that.
> 
> Should the leftovers in current mainline be replaced by wmb()?  Or are
> patches to that effect on their way in somewhere?

I already got rid of the non-arch usage of mmiowb(), but I wasn't bravei
enough to change the arch code as it may well be that they're relying on
some specific instruction semantics.

Despite my earlier comment, mmiowb() still exists, but only as a part of
ARCH_HAS_MMIOWB where it is used to add additional spinlock ordering so
that the rest of the kernel doesn't need to use mmiowb() at all.

So I suppose for these:

> arch/mips/kernel/gpio_txx9.c:	mmiowb();
> arch/mips/kernel/gpio_txx9.c:	mmiowb();
> arch/mips/kernel/gpio_txx9.c:	mmiowb();
> arch/mips/kernel/irq_txx9.c:	mmiowb();
> arch/mips/loongson2ef/common/bonito-irq.c:	mmiowb();
> arch/mips/loongson2ef/common/bonito-irq.c:	mmiowb();
> arch/mips/loongson2ef/common/mem.c:		mmiowb();
> arch/mips/loongson2ef/common/pm.c:	mmiowb();
> arch/mips/loongson2ef/lemote-2f/reset.c:	mmiowb();
> arch/mips/loongson2ef/lemote-2f/reset.c:	mmiowb();
> arch/mips/loongson2ef/lemote-2f/reset.c:	mmiowb();
> arch/mips/loongson2ef/lemote-2f/reset.c:	mmiowb();
> arch/mips/loongson2ef/lemote-2f/reset.c:	mmiowb();
> arch/mips/pci/ops-bonito64.c:	mmiowb();
> arch/mips/pci/ops-loongson2.c:	mmiowb();
> arch/mips/txx9/generic/irq_tx4939.c:	mmiowb();
> arch/mips/txx9/generic/setup.c:	mmiowb();
> arch/mips/txx9/rbtx4927/irq.c:	mmiowb();
> arch/mips/txx9/rbtx4938/irq.c:	mmiowb();
> arch/mips/txx9/rbtx4938/irq.c:	mmiowb();
> arch/mips/txx9/rbtx4938/setup.c:	mmiowb();
> arch/mips/txx9/rbtx4939/irq.c:	mmiowb();

we could replace mmiowb() with iobarrier_w().

Will

  reply	other threads:[~2021-08-18 11:39 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-16 10:12 Marco Elver
2021-08-16 14:59 ` Alan Stern
2021-08-16 17:23   ` Marco Elver
2021-08-16 19:21     ` Alan Stern
2021-08-16 20:50       ` Paul E. McKenney
2021-08-17 12:14         ` Marco Elver
2021-08-17 12:28         ` Will Deacon
2021-08-17 13:27           ` Marco Elver
2021-08-17 13:53           ` Paul E. McKenney
2021-08-18 11:39             ` Will Deacon [this message]
2021-08-18 23:17               ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210818113935.GA14107@willie-the-truck \
    --to=will@kernel.org \
    --cc=boqun.feng@gmail.com \
    --cc=dvyukov@google.com \
    --cc=elver@google.com \
    --cc=kasan-dev@googlegroups.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=parri.andrea@gmail.com \
    --cc=paulmck@kernel.org \
    --cc=stern@rowland.harvard.edu \
    --subject='Re: LKMM: Read dependencies of writes ordered by dma_wmb()?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).