LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Guo Ren <guoren@kernel.org>
To: Atish Patra <atishp@atishpatra.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>,
	devicetree <devicetree@vger.kernel.org>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Tobias Klauser <tklauser@distanz.ch>,
	Robin Murphy <robin.murphy@arm.com>,
	"linux-kernel@vger.kernel.org List"
	<linux-kernel@vger.kernel.org>, Rob Herring <robh+dt@kernel.org>,
	Atish Patra <Atish.Patra@wdc.com>,
	iommu@lists.linux-foundation.org,
	Guo Ren <guoren@linux.alibaba.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	linux-riscv <linux-riscv@lists.infradead.org>,
	Frank Rowand <frowand.list@gmail.com>,
	Christoph Hellwig <hch@lst.de>,
	Dmitry Vyukov <dvyukov@google.com>
Subject: Re: [RFC 0/5] Support non-coherent DMA on RISC-V using a global pool
Date: Tue, 17 Aug 2021 14:42:44 +0800	[thread overview]
Message-ID: <CAJF2gTQcGiudC6X=xN-0U=wuVLE2jthjuY8R7BDo1CZTTZbuVA@mail.gmail.com> (raw)
In-Reply-To: <CAOnJCUKPitLD1g1LocTw8H+G7hg04-=Dd3biHGgDPMwer2cFpw@mail.gmail.com>

On Tue, Aug 17, 2021 at 11:28 AM Atish Patra <atishp@atishpatra.org> wrote:
>
> On Mon, Aug 16, 2021 at 6:37 PM Guo Ren <guoren@kernel.org> wrote:
> >
> > 1
> >
> > On Thu, Jul 29, 2021 at 2:19 PM Atish Patra <atishp@atishpatra.org> wrote:
> > >
> > > On Wed, Jul 28, 2021 at 9:30 PM Palmer Dabbelt <palmer@dabbelt.com> wrote:
> > > >
> > > > On Fri, 23 Jul 2021 14:40:26 PDT (-0700), Atish Patra wrote:
> > > > > RISC-V privilege specification doesn't define an IOMMU or any method modify
> > > > > PMA attributes or PTE entries to allow non-coherent mappings yet. In
> > > > > the beginning, this approach was adopted assuming that most of the RISC-V
> > > > > platforms would support full cache-coherent IO. Here is excerpt from the
> > > > > priv spec section 3.6.5
> > > > >
> > > > > "In RISC-V platforms, the use of hardware-incoherent regions is discouraged
> > > > > due to software complexity, performance, and energy impacts."
> > > > >
> > > > > While some of the RISC-V platforms adhere to the above suggestion, not all
> > > > > platforms can afford to build to fully cache-coherent I/O devices. To
> > > > > address DMA for non-coherent I/O devices, we need to mark a region of memory
> > > > > as non-cacheable. Some of the platforms rely on a fixed region of uncached
> > > > > memory that is remapped to the system memory while some other modify
> > > > > the PTE entries to achieve that.
> > > > >
> > > > > The patch3 solves the issue for the fist use case by using a global
> > > > > pool of memory that is reserved for DMA. The device access the reserved area
> > > > > of the region while corresponding CPU address will be from uncached region
> > > > > As the uncached region is remapped to the beginning of the system ram, both
> > > > > CPU and device get the same view.
> > > > >
> > > > > To facilitate streaming DMA APIs, patch 1 introduces a set of generic
> > > > > cache operations. Any platform can use the generic ops to provide platform
> > > > > specific cache management operations. Once the standard RISC-V CMO extension
> > > > > is available, it will also use these generic ops.
> > > > >
> > > > > To address the second use case, Page Based Memory Attribute (PBMT) extension
> > > > > is proposed. Once the extension is in good shape, we can leverage that
> > > > > using CONFIG_DIRECT_REMAP. Currently, it is selected via a compile time config
> > > > > option. We will probably need another arch specific hooks to know if the
> > > > > platform supports direct remap at runtime. For RISC-V, it will check the
> > > > > presence of PBMT extension while other arch function will simply return true
> > > >
> > > > IIUC this is another extension that's not yet frozen or implemented in
> > > > hardware?  Is this one compatible with what's in the c906, or is it
> > > > doing things its own way?
> > >
> > > Hi Palmer,
> > > This series doesn't implement the PBMT extension which is still in discussion.
> > > It simply reuse the existing non-coherent dma mapping support in
> > > upstream kernel and enable
> > > it for RISC-V. The current version uses a non-coherent global pool. I
> > > will update it to use arch_set_uncached
> > > as per Christoph's suggestion. It solves the non-coherent DMA problem
> > > for beagleV and not c906.
> > >
> > > I briefly mentioned the PBMT extension just to provide an idea how the
> > > RISC-V Linux kernel
> > > can support both unached window and PBMT extension. PBMT extension is
> > > planned to be frozen
> > > by the end of this year and none of the hardware has implemented it.
> > >
> > > The implementation in c906 is a non-standard one and will not be
> > > supported by the default PBMT
> > > extension implementation.
> > The default PBMT & c908 PBMT are similar, only BIT definitions are
> > different. I propose to support default PBMT first and give the back
> > door to modify the PBMT definition during boot.
> > The "protection_map[] = (__P000, __P001 ..__S000, __S001)" in
> > mm/mmap.c has been modified by arch/mips, arm, sparc, x86, so I think
> > it's proper solution direction.
> >
> > The reset problem is how to passing custom PBMT at the very early boot
> > stage. Now I'm trying to use the DTS parsing instead of boot param hdr
> > which Anup objected to.
> >
>
> IIRC, c906 has a compatible mode that has the compliant PTE bit modifications.
> Can you use that mode iOn the Allwinner D1 board to boot Linux ? I am
> not sure if you have any fallback method for non-coherent DMA
> if custom DMA_COHERENT bits are not enabled through enhanced mode ?
We need custom PBMT(enhanced mode) to enable the dma driver on D1
(GMAC, USB, EMMC) or these drivers couldn't work.
D1 hasn't any uncached region in SOC design.

>
> > ref: https://lore.kernel.org/linux-riscv/1623693067-53886-1-git-send-email-guoren@kernel.org/
> >
> > Any comments are welcome.
> >
> > >
> > >
> > > >
> > > > > if DIRECT_REMAP is enabled. This is required as arious different config
> > > > > (DIRECT_REMAP, GLOBAL_POOL) will be enabled in the defconfig so that a
> > > > > unified kernel image can boot on all RISC-V platforms.
> > > > >
> > > > > This patch series depends on Christoph's global pool support series[1].
> > > > > Tested on Qemu, HiFive unleashed and beagleV. This series is also available
> > > > > at [2].
> > > > > This series also solves the non-coherent DMA support on beagleV but requires
> > > > > additional beagleV specific patches[3] which will be upstreamed soon.
> > > > >
> > > > >
> > > > > [1] https://lists.linuxfoundation.org/pipermail/iommu/2021-July/057266.html
> > > > > [2] https://github.com/atishp04/linux/tree/riscv_nc_global_pool
> > > > > [3] https://github.com/atishp04/linux/tree/wip_beaglev_dma_nc_global
> > > > >
> > > > > Atish Patra (5):
> > > > > RISC-V: Implement arch_sync_dma* functions
> > > > > of: Move of_dma_get_range to of_address.h
> > > > > dma-mapping: Enable global non-coherent pool support for RISC-V
> > > > > dma-direct: Allocate dma pages directly if global pool allocation
> > > > > fails
> > > > > RISC-V: Support a new config option for non-coherent DMA
> > > > >
> > > > > arch/riscv/Kconfig                       |  8 +++
> > > > > arch/riscv/include/asm/dma-noncoherent.h | 19 +++++++
> > > > > arch/riscv/mm/Makefile                   |  1 +
> > > > > arch/riscv/mm/dma-noncoherent.c          | 66 ++++++++++++++++++++++++
> > > > > drivers/of/of_private.h                  | 10 ----
> > > > > include/linux/of_address.h               | 12 +++++
> > > > > kernel/dma/coherent.c                    | 49 +++++++++++++++---
> > > > > kernel/dma/direct.c                      |  7 ++-
> > > > > 8 files changed, 152 insertions(+), 20 deletions(-)
> > > > > create mode 100644 arch/riscv/include/asm/dma-noncoherent.h
> > > > > create mode 100644 arch/riscv/mm/dma-noncoherent.c
> > > >
> > > > Can you guys please make up your minds about how this is going to be
> > > > supported at the ISA level?  I get a different answer every day here:
> > > > sometimes it's that these systems are not compliant, sometimes that
> > > > Linux is the compliance suite, sometimes that we're doing an ISA
> > > > extension, and sometimes that we're doing the SBI stuff.
> > > >
> > >
> > > I am not sure whom you have talked to. I would be happy to set up a
> > > meeting so that everybody is on
> > > the same page if you are getting different answers every time.
> > >
> > > > I don't really care all that much about what the decision is, but it's
> > > > impossible to move forward without some semblance of a plan.
> > > >
> > > > _______________________________________________
> > > > linux-riscv mailing list
> > > > linux-riscv@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Atish
> > > _______________________________________________
> > > iommu mailing list
> > > iommu@lists.linux-foundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> >
> >
> >
> > --
> > Best Regards
> >  Guo Ren
> >
> > ML: https://lore.kernel.org/linux-csky/
>
>
>
> --
> Regards,
> Atish



-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

      reply	other threads:[~2021-08-17  6:42 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-23 21:40 Atish Patra
2021-07-23 21:40 ` [RFC 1/5] RISC-V: Implement arch_sync_dma* functions Atish Patra
2021-07-26  6:56   ` Christoph Hellwig
2021-07-26 21:52     ` Atish Patra
2021-09-11  9:37       ` Guo Ren
2021-08-17  1:48   ` Guo Ren
2021-08-17  3:24     ` Atish Patra
2021-08-17  6:28       ` Guo Ren
2021-07-23 21:40 ` [RFC 2/5] of: Move of_dma_get_range to of_address.h Atish Patra
2021-07-23 21:40 ` [RFC 3/5] dma-mapping: Enable global non-coherent pool support for RISC-V Atish Patra
2021-07-25 22:29   ` Rob Herring
2021-07-26  7:00   ` Christoph Hellwig
2021-07-26 22:47     ` Atish Patra
2021-07-27  8:52       ` Christoph Hellwig
2021-08-02 18:22         ` Atish Patra
2021-07-23 21:40 ` [RFC 4/5] dma-direct: Allocate dma pages directly if global pool allocation fails Atish Patra
2021-07-26  7:01   ` Christoph Hellwig
2021-07-23 21:40 ` [RFC 5/5] RISC-V: Support a new config option for non-coherent DMA Atish Patra
2021-07-29  4:30 ` [RFC 0/5] Support non-coherent DMA on RISC-V using a global pool Palmer Dabbelt
2021-07-29  6:19   ` Atish Patra
2021-08-17  1:37     ` Guo Ren
2021-08-17  3:28       ` Atish Patra
2021-08-17  6:42         ` Guo Ren [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJF2gTQcGiudC6X=xN-0U=wuVLE2jthjuY8R7BDo1CZTTZbuVA@mail.gmail.com' \
    --to=guoren@kernel.org \
    --cc=Atish.Patra@wdc.com \
    --cc=aou@eecs.berkeley.edu \
    --cc=atishp@atishpatra.org \
    --cc=devicetree@vger.kernel.org \
    --cc=dvyukov@google.com \
    --cc=frowand.list@gmail.com \
    --cc=guoren@linux.alibaba.com \
    --cc=hch@lst.de \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=palmer@dabbelt.com \
    --cc=paul.walmsley@sifive.com \
    --cc=robh+dt@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=tklauser@distanz.ch \
    --subject='Re: [RFC 0/5] Support non-coherent DMA on RISC-V using a global pool' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).