Re: [PATCH V7 6/9] PCI: Enable 10-Bit Tag support for PCIe RP devices

[Bjorn Helgaas <helgaas@kernel.org>]

On Wed, Aug 04, 2021 at 09:47:05PM +0800, Dongdong Liu wrote:
> PCIe spec 5.0r1.0 section 2.2.6.2 implementation note, In configurations
> where a Requester with 10-Bit Tag Requester capability needs to target
> multiple Completers, one needs to ensure that the Requester sends 10-Bit
> Tag Requests only to Completers that have 10-Bit Tag Completer capability.
> So we enable 10-Bit Tag Requester for root port only when the devices
> under the root port support 10-Bit Tag Completer.

Fix quoting.  I can't tell what is from the spec and what you wrote.

> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
> ---
>  drivers/pci/pcie/portdrv_pci.c | 69 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
> 
> diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c
> index c7ff1ee..2382cd2 100644
> --- a/drivers/pci/pcie/portdrv_pci.c
> +++ b/drivers/pci/pcie/portdrv_pci.c
> @@ -90,6 +90,72 @@ static const struct dev_pm_ops pcie_portdrv_pm_ops = {
>  #define PCIE_PORTDRV_PM_OPS	NULL
>  #endif /* !PM */
>  
> +static int pci_10bit_tag_comp_support(struct pci_dev *dev, void *data)
> +{
> +	bool *support = (bool *)data;
> +
> +	if (!pci_is_pcie(dev)) {
> +		*support = false;
> +		return 1;
> +	}
> +
> +	/*
> +	 * PCIe spec 5.0r1.0 section 2.2.6.2 implementation note.
> +	 * For configurations where a Requester with 10-Bit Tag Requester
> +	 * capability targets Completers where some do and some do not have
> +	 * 10-Bit Tag Completer capability, how the Requester determines which
> +	 * NPRs include 10-Bit Tags is outside the scope of this specification.
> +	 * So we do not consider hotplug scenario.
> +	 */
> +	if (dev->is_hotplug_bridge) {
> +		*support = false;
> +		return 1;
> +	}
> +
> +	if (!(dev->pcie_devcap2 & PCI_EXP_DEVCAP2_10BIT_TAG_COMP)) {
> +		*support = false;
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +static void pci_configure_rp_10bit_tag(struct pci_dev *dev)
> +{
> +	bool support = true;
> +
> +	if (dev->subordinate == NULL)
> +		return;
> +
> +	/* If no devices under the root port, no need to enable 10-Bit Tag. */
> +	if (list_empty(&dev->subordinate->devices))
> +		return;
> +
> +	pci_10bit_tag_comp_support(dev, &support);
> +	if (!support)
> +		return;
> +
> +	/*
> +	 * PCIe spec 5.0r1.0 section 2.2.6.2 implementation note.
> +	 * In configurations where a Requester with 10-Bit Tag Requester
> +	 * capability needs to target multiple Completers, one needs to ensure
> +	 * that the Requester sends 10-Bit Tag Requests only to Completers
> +	 * that have 10-Bit Tag Completer capability. So we enable 10-Bit Tag
> +	 * Requester for root port only when the devices under the root port
> +	 * support 10-Bit Tag Completer.
> +	 */
> +	pci_walk_bus(dev->subordinate, pci_10bit_tag_comp_support, &support);
> +	if (!support)
> +		return;
> +
> +	if (!(dev->pcie_devcap2 & PCI_EXP_DEVCAP2_10BIT_TAG_REQ))
> +		return;
> +
> +	pci_dbg(dev, "enabling 10-Bit Tag Requester\n");
> +	pcie_capability_set_word(dev, PCI_EXP_DEVCTL2,
> +				 PCI_EXP_DEVCTL2_10BIT_TAG_REQ_EN);
> +}
> +
>  /*
>   * pcie_portdrv_probe - Probe PCI-Express port devices
>   * @dev: PCI-Express port device being probed
> @@ -111,6 +177,9 @@ static int pcie_portdrv_probe(struct pci_dev *dev,
>  	     (type != PCI_EXP_TYPE_RC_EC)))
>  		return -ENODEV;
>  
> +	if (type == PCI_EXP_TYPE_ROOT_PORT)
> +		pci_configure_rp_10bit_tag(dev);

I don't think this has anything to do with the portdrv, so all this
should go somewhere else.

Out of curiosity, IIUC this enables 10-bit tags for MMIO transactions
from the root port toward the device, i.e., traffic that originates
from a CPU.  Is that a significant benefit?  I would expect high-speed
devices would primarily operate via DMA with relatively little MMIO
traffic.

>  	if (type == PCI_EXP_TYPE_RC_EC)
>  		pcie_link_rcec(dev);
>  
> -- 
> 2.7.4
>