LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Sunil Muthuswamy <sunilmut@microsoft.com>
To: Marc Zyngier <maz@kernel.org>, Robin Murphy <robin.murphy@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
	"will@kernel.org" <will@kernel.org>,
	Michael Kelley <mikelley@microsoft.com>,
	Boqun Feng <Boqun.Feng@microsoft.com>,
	KY Srinivasan <kys@microsoft.com>, Arnd Bergmann <arnd@arndb.de>
Subject: RE: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
Date: Wed, 4 Aug 2021 20:10:43 +0000	[thread overview]
Message-ID: <MW4PR21MB2002860BAED850B2B52D6E2BC0F19@MW4PR21MB2002.namprd21.prod.outlook.com> (raw)
In-Reply-To: <87r1f9wooc.wl-maz@kernel.org>

On Wednesday, August 4, 2021 2:21 AM,
Marc Zyngier <maz@kernel.org> wrote:
>
> On Tue, 03 Aug 2021 09:35:12 +0100,
> Robin Murphy <robin.murphy@arm.com> wrote:
> >
> > On 2021-08-03 03:11, Sunil Muthuswamy wrote:
> > >   On Saturday, July 31, 2021 2:52 AM,
> > >   Marc Zyngier <maz@kernel.org> wrote:
> > >>
> > >> [...]
> > >>
> > >>>> I also want to understand *how* you are going to plumb this into both
> > >>>> ACPI and DT, given that neither understand how to link a PCI endpoint
> > >>>> to a set of RDs.
> > >>>>
> > >>>> 	M.
> > >>>
> > >>> One way to do this for NUMA-aware systems would be to use the NUMA
> > >>> related information that is available with PCI endpoints or root complex, to
> > >>> pick a Redistributor/CPU that is in the NUMA node, as specified by the PCI
> > >>> endpoint/root complex. In DT PCI devices can specify this using
> > >>> 'numa-node-id' and in ACPI using the '_PXM (Proximity)'. For systems that
> > >>> are not NUMA-aware, we can go with *any* Redistributor/CPU.
> > >>
> > >> This makes zero sense. From the point of view of a device, all the RDs
> > >> should be reachable, and firmware has no say in it. Dealing with
> > >> interrupt affinity is the responsibility of the endpoint driver, and
> > >> NUMA affinity is only a performance optimisation.
> > >>
> > >>> Is there any additional information we would be able to gather from ACPI
> > >>> or DT that's not there currently, that would be useful here?
> > >>
> > >> You will need some firmware information describing that a given set of
> > >> devices must use the RDs for their MSIs. Just like we currently
> > >> describe it in IORT for the ITS. You cannot /assume/ things. At the
> > >> moment, there is nothing at all, because no-one (including Microsoft)
> > >> thought it would be a good idea not to have an ITS, which is also why
> > >> ACPI doesn't describe MBIs as a potential MSI provider.
> > >>
> > > I am a little bit confused by your above comment. Maybe you can help me
> > > understand the ask. You indicate that from the point of the view of the
> > > device, all the RDs should be reachable. But, then if we define a mapping
> > > between PCI endpoint and RD in the firmware, we would be doing exactly
> > > the opposite. i.e. restricting the RDs that are reachable by the device. Can
> > > you please clarify?
> 
> Not at all. What I am saying is that you need to describe that the
> MSIs have to be routed to the RDs, and there is no way to express this
> at the moment.
> 
> > >
> > > Is your concern that the device should be able to only DMA to a subset of
> > > GIC Redistributor, for the MSIs? If so, in the IORT, there is "memory address
> > > size limit" for both device and root complex nodes. In the implementation,
> > > we can enforce that the GICR is within that range. And, if a device deviates
> > > further than that (ex: by having accessibility gaps within the GICR range),
> > > then that is out of scope for support.
> >
> > No, please don't try to abuse the Memory Address Size Limit - that has
> > far more chance of adversely affecting normal DMA operation than of
> > being any use here.
> >
> > I believe the point Marc was trying to make is that firmware should
> > not associate a device with any one *specific* redistributor, however
> > ACPI currently has no way to describe that MSIs can target
> > redistributors *at all*, only ITS groups - there is no such concept as
> > a "redistributor group".
> 
> Thanks Robin.
> 
> That is exactly my point. There is no linkage whatsoever between a
> device generating MSIs and the redistributors. In that respect, this
> is the same issue we have with DT, as none of the firmware interfaces
> can currently describe MSIs directly targeting the RDs. The only way
> to describe MSIs with GICv3 in ACPI is to describe an ITS, and you
> cannot use the *absence* of an ITS to decide and use DirectLPIs.
> 
> Given the state of things, using DirectLPI means that you need to
> extend the firmware interfaces. This means both an extension to the DT
> binding, and an update to the ACPI spec. There is no way around this.
> 
> Thanks,
> 
> 	M.
> 
Thanks Marc and Robin for clarifying. I see and understand the point
about having explicit MSI mappings in the firmware specification for
Direct LPIs for generic hardware support.

Hey Mark, would you be willing to consider a scoped down implementation of
GIC Direct LPI with just an IRQ chip implementation and no
Direct LPI PCI-MSI IRQ chip. This will allow a MSI provider (such as Hyper-V vPCI) to
provide a PCI-MSI IRQ chip on top of the Direct LPI IRQ chip and enable
PCI-MSI scenarios, and avoid building in assumptions in other cases (like PCI) where
firmware specification is not available.
- This scoped down implementation would allow Microsoft to build virtual
PCI on top, to enable our business needs.
- If there's a need for a generic support for a Direct LPI PCI MSI IRQ, that could
drive firmware revision efforts, and we are happy to assist there.

Looking forward to hearing back.

Thanks,
Sunil

  reply	other threads:[~2021-08-04 20:10 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-08 19:36 Sunil Muthuswamy
2021-07-11 11:09 ` Marc Zyngier
2021-07-26 15:33   ` [EXTERNAL] " Sunil Muthuswamy
2021-07-31  9:52     ` Marc Zyngier
2021-08-03  2:11       ` Sunil Muthuswamy
2021-08-03  8:35         ` Robin Murphy
2021-08-04  9:21           ` Marc Zyngier
2021-08-04 20:10             ` Sunil Muthuswamy [this message]
2021-08-05  8:35               ` Marc Zyngier
2021-08-06 19:14                 ` Sunil Muthuswamy
2021-08-08 10:19                   ` Marc Zyngier
2021-08-09  2:35                     ` Sunil Muthuswamy
2021-08-09  9:15                       ` Marc Zyngier
2021-08-10  1:10                         ` Sunil Muthuswamy
2021-08-10 13:57                           ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MW4PR21MB2002860BAED850B2B52D6E2BC0F19@MW4PR21MB2002.namprd21.prod.outlook.com \
    --to=sunilmut@microsoft.com \
    --cc=Boqun.Feng@microsoft.com \
    --cc=arnd@arndb.de \
    --cc=catalin.marinas@arm.com \
    --cc=kys@microsoft.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mikelley@microsoft.com \
    --cc=robin.murphy@arm.com \
    --cc=tglx@linutronix.de \
    --cc=will@kernel.org \
    --subject='RE: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).