LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Bjorn Helgaas <helgaas@kernel.org>
To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: "Kishon Vijay Abraham I" <kishon@ti.com>,
	bhelgaas@google.com, "Rob Herring" <robh@kernel.org>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	devicetree@vger.kernel.org
Subject: Re: [PATCH v2 4/5] PCI: keystone: Add quirk to mark AM654 RC BAR flag as IORESOURCE_UNSET
Date: Tue, 8 Feb 2022 10:20:39 -0600	[thread overview]
Message-ID: <20220208162039.GA477897@bhelgaas> (raw)
In-Reply-To: <20220208115344.GB6233@lpieralisi>

On Tue, Feb 08, 2022 at 11:53:44AM +0000, Lorenzo Pieralisi wrote:
> On Fri, Feb 04, 2022 at 08:38:46PM +0530, Kishon Vijay Abraham I wrote:
> > On 11/01/22 11:53 am, Kishon Vijay Abraham I wrote:
> > > On 04/01/22 9:27 pm, Lorenzo Pieralisi wrote:
> > >> On Fri, Nov 26, 2021 at 02:01:18PM +0530, Kishon Vijay Abraham I wrote:
> > >>> AM654 RootComplex has a hard coded 64 bit BAR of size 1MB and
> > >>> also has both MSI and MSI-X capability in it's config space.
> > >>> If PCIEPORTBUS is enabled, it tries to configure MSI-X and
> > >>> msix_mask_all() adds about 10 Second boot up delay when it
> > >>> tries to write to undefined location.

s/AM654 RootComplex/The AM654 Root Complex/

But Root Complexes are not normally materialized as PCI devices with
their own bus/device/function address, config space, BARs, etc.
Sounds like this might really be a Root *Port*, not a Root Complex?

s/it's config/its config/
s/10 Second/10 second/

> > >>> Add quirk to mark AM654 RC BAR flag as IORESOURCE_UNSET so
> > >>> that msix_map_region() returns NULL for Root Complex and avoid
> > >>> un-desirable writes to MSI-X table.
> > >>
> > >> I don't think this is the right fix (it is not even a fix, just
> > >> a plaster to workaround an issue).
> > >>
> > >> What do you mean by "writing to an undefined location" ?
> > >>
> > >> What does "a hard coded BAR" mean ?
> > >>
> > >> What happens if we _rightly_ write into it (ie to size it) ?
> > > 
> > > There are two parts w.r.t setting the BAR; one is during the
> > > configuration and the other is during the enumeration.
> > > i) During the configuration, the size of the BAR is configured
> > >    and the inbound ATU is configured to map the BAR to a
> > >    physical memory.
> > > ii) During the enumeration, the size of the BAR is obtained and
> > >     an address is allocated and programmed in the BAR.
> > > 
> > > In the case of RC, for (i) above, the BAR size is configured as '0'
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-designware-host.c#n556
> > > and the inbound ATU is not programmed at all.
> > > 
> > > However, in the case of AM654, the HW configures BAR0 for a
> > > fixed size of 1MB (irrespective of what SW programmed in [i]).
> > > While this was done more for a endpoint usecase, since the same
> > > IP is configured for both RC mode and EP mode, the fixed BAR
> > > size is seen with RC mode as well.

This doesn't seem to quite answer Lorenzo's question.  On AM654, does
__pci_read_base() discover the 1MB size correctly for both RC and EP
mode?

What value should BAR0 contain in RC mode?  Does the device respond at
that address?  Do we still need to ensure that 1MB address space is
not assigned to any other device?

> > > AM654 also has MSI-X capability for RC mode (the IP should have
> > > been ideally configured to have MSI-X capability for EP mode).
> > > This results in PCIEPORTBUS doing some undesired access in
> > > msix_mask_all().

We have several quirks that set dev->no_msi.  Maybe that would be a
better way to prevent use of MSI-X (it would also prevent use of MSI;
not sure whether that needs to be avoided as well)?

> > > Here I configure IORESOURCE_UNSET so that memory is not
> > > allocated for RC BAR.

I guess this implies that this device (RP?  I don't think RCs have
BARs) never responds to PCI address space described by BAR0?

> > Do you need further clarifications on this?
> 
> There are two things here:
> 
> 1) As Rob mentioned, you can write it as a quirk applying only to
>    the bridge _only_
> 2) What you want is that the BAR should not be visible to the OS
>    since it is not an actual resource. What I am questioning is
>    whether your way of doing that complies with how this is done in
>    the kernel for other devices/bridges. I need Bjorn's input on
>    this since he knows better (especially wrt IORESOURCE_UNSET
>    usage). I don't want to add any other IORESOURCE_UNSET usage that
>    deviates from what's expected from it

If BAR0 should not exist at all as far as the OS is concerned, I think
you should just set r->flags = 0.  But that assumes the device never
uses the value in the BAR, and it never responds using whatever
address is there, even when PCI_COMMAND_MEMORY is set.

> > >>> Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
> > >>> ---
> > >>>  drivers/pci/controller/dwc/pci-keystone.c | 8 +++++++-
> > >>>  1 file changed, 7 insertions(+), 1 deletion(-)
> > >>>
> > >>> diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
> > >>> index 52d20fe17ee9..73e6626a0d8f 100644
> > >>> --- a/drivers/pci/controller/dwc/pci-keystone.c
> > >>> +++ b/drivers/pci/controller/dwc/pci-keystone.c
> > >>> @@ -557,8 +557,14 @@ static void ks_pcie_quirk(struct pci_dev *dev)
> > >>>  		{ 0, },
> > >>>  	};
> > >>>  
> > >>> -	if (pci_is_root_bus(bus))
> > >>> +	if (pci_is_root_bus(bus)) {
> > >>>  		bridge = dev;
> > >>> +		if (pci_match_id(am6_pci_devids, bridge)) {
> > >>> +			struct resource *r = &dev->resource[0];
> > >>> +
> > >>> +			r->flags |= IORESOURCE_UNSET;
> > >>> +		}
> > >>> +	}
> > >>>  
> > >>>  	/* look for the host bridge */
> > >>>  	while (!pci_is_root_bus(bus)) {
> > >>> -- 
> > >>> 2.17.1
> > >>>

  reply	other threads:[~2022-02-08 16:20 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-26  8:31 [PATCH v2 0/5] PCI: Keystone: Misc fixes for TI's AM65x PCIe Kishon Vijay Abraham I
2021-11-26  8:31 ` [PATCH v2 1/5] dt-bindings: PCI: ti,am65: Fix "ti,syscon-pcie-id"/"ti,syscon-pcie-mode" to take argument Kishon Vijay Abraham I
2021-11-27 23:13   ` Rob Herring
2021-11-29  4:03     ` Kishon Vijay Abraham I
2021-12-01 22:55   ` Rob Herring
2021-11-26  8:31 ` [PATCH v2 2/5] PCI: keystone: Use phandle argument from "ti,syscon-pcie-id"/"ti,syscon-pcie-mode" Kishon Vijay Abraham I
2021-11-26  8:31 ` [PATCH v2 3/5] PCI: keystone: Add workaround for Errata #i2037 (AM65x SR 1.0) Kishon Vijay Abraham I
2021-11-26  8:31 ` [PATCH v2 4/5] PCI: keystone: Add quirk to mark AM654 RC BAR flag as IORESOURCE_UNSET Kishon Vijay Abraham I
2022-01-04 15:57   ` Lorenzo Pieralisi
2022-01-11  6:23     ` Kishon Vijay Abraham I
2022-02-04 15:08       ` Kishon Vijay Abraham I
2022-02-08 11:53         ` Lorenzo Pieralisi
2022-02-08 16:20           ` Bjorn Helgaas [this message]
2022-01-04 18:25   ` Rob Herring
2021-11-26  8:31 ` [PATCH v2 5/5] PCI: keystone: Set DMA mask and coherent DMA mask Kishon Vijay Abraham I
2022-02-28 11:38   ` Christian Gmeiner
2022-01-04  9:02 ` [PATCH v2 0/5] PCI: Keystone: Misc fixes for TI's AM65x PCIe Kishon Vijay Abraham I
2022-01-07 10:56 ` (subset) " Lorenzo Pieralisi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220208162039.GA477897@bhelgaas \
    --to=helgaas@kernel.org \
    --cc=bhelgaas@google.com \
    --cc=devicetree@vger.kernel.org \
    --cc=kishon@ti.com \
    --cc=kw@linux.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=robh@kernel.org \
    --subject='Re: [PATCH v2 4/5] PCI: keystone: Add quirk to mark AM654 RC BAR flag as IORESOURCE_UNSET' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).