Netdev Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Leon Romanovsky <leon@kernel.org>
To: Logan Gunthorpe <logang@deltatee.com>
Cc: Dongdong Liu <liudongdong3@huawei.com>,
	helgaas@kernel.org, hch@infradead.org, kw@linux.com,
	linux-pci@vger.kernel.org, rajur@chelsio.com,
	hverkuil-cisco@xs4all.nl, linux-media@vger.kernel.org,
	netdev@vger.kernel.org
Subject: Re: [PATCH V6 7/8] PCI: Add "pci=disable_10bit_tag=" parameter for peer-to-peer support
Date: Sun, 25 Jul 2021 09:39:53 +0300	[thread overview]
Message-ID: <YP0HOf7kE1aOkqjV@unreal> (raw)
In-Reply-To: <a8a8ffee-67e8-c899-3d04-1e28fb72560a@deltatee.com>

On Fri, Jul 23, 2021 at 10:20:50AM -0600, Logan Gunthorpe wrote:
> 
> 
> 
> On 2021-07-23 5:32 a.m., Leon Romanovsky wrote:
> > On Fri, Jul 23, 2021 at 07:06:41PM +0800, Dongdong Liu wrote:
> >> PCIe spec 5.0 r1.0 section 2.2.6.2 says that if an Endpoint supports
> >> sending Requests to other Endpoints (as opposed to host memory), the
> >> Endpoint must not send 10-Bit Tag Requests to another given Endpoint
> >> unless an implementation-specific mechanism determines that the Endpoint
> >> supports 10-Bit Tag Completer capability. Add "pci=disable_10bit_tag="
> >> parameter to disable 10-Bit Tag Requester if the peer device does not
> >> support the 10-Bit Tag Completer. This will make P2P traffic safe.
> >>
> >> Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
> >> ---
> >>  Documentation/admin-guide/kernel-parameters.txt |  7 ++++
> >>  drivers/pci/pci.c                               | 56 +++++++++++++++++++++++++
> >>  drivers/pci/pci.h                               |  1 +
> >>  drivers/pci/pcie/portdrv_pci.c                  | 13 +++---
> >>  drivers/pci/probe.c                             |  9 ++--
> >>  5 files changed, 78 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> >> index bdb2200..c2c4585 100644
> >> --- a/Documentation/admin-guide/kernel-parameters.txt
> >> +++ b/Documentation/admin-guide/kernel-parameters.txt
> >> @@ -4019,6 +4019,13 @@
> >>  				bridges without forcing it upstream. Note:
> >>  				this removes isolation between devices and
> >>  				may put more devices in an IOMMU group.
> >> +		disable_10bit_tag=<pci_dev>[; ...]
> >> +				  Specify one or more PCI devices (in the format
> >> +				  specified above) separated by semicolons.
> >> +				  Disable 10-Bit Tag Requester if the peer
> >> +				  device does not support the 10-Bit Tag
> >> +				  Completer.This will make P2P traffic safe.
> > 
> > I can't imagine more awkward user experience than such kernel parameter.
> > 
> > As a user, I will need to boot the system, hope for the best that system
> > works, write down all PCI device numbers, guess which one doesn't work
> > properly, update grub with new command line argument and reboot the
> > system. Any HW change and this dance should be repeated.
> 
> There are already two such PCI parameters with this pattern and they are
> not that awkward. pci_dev may be specified with either vendor/device IDS
> or with a path of BDFs (which protects against renumbering).

Unfortunately, in the real world, BDF is not so stable. It changes with
addition of new hardware, BIOS upgrades and even broken servers.

Vendor/device IDs doesn't work if you have multiple devices of same
vendor in the system.

> 
> This flag is only useful in P2PDMA traffic, and if the user attempts
> such a transfer, it prints a warning (see the next patch) with the exact
> parameter that needs to be added to the command line.

Dongdong citied PCI spec and it was very clear - don't enable this
feature unless you clearly know that it is safe to enable. This is
completely opposite to the proposal here - always enable and disable
if something is printed to the dmesg.

> 
> This has worked well for disable_acs_redir and was used for
> resource_alignment before that for quite some time. So save a better
> suggestion I think this is more than acceptable.

I don't know about other parameters and their history, but we are not in
90s anymore and addition of modules parameters (for the PCI it is kernel
cmdline arguments) are better to be changed to some configuration tool/sysfs.

Even FW upgrade with such kernel parameter can be problematic.

Thanks

> 
> Logan

  reply	other threads:[~2021-07-25  6:40 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-23 11:06 [PATCH V6 0/8] PCI: Enable 10-Bit tag support for PCIe devices Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 1/8] PCI: Use cached Device Capabilities Register Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 2/8] PCI: Use cached Device Capabilities 2 Register Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 3/8] PCI: Add 10-Bit Tag register definitions Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 4/8] PCI: Enable 10-Bit tag support for PCIe Endpoint devices Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 5/8] PCI/IOV: Enable 10-Bit tag support for PCIe VF devices Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 6/8] PCI: Enable 10-Bit tag support for PCIe RP devices Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 7/8] PCI: Add "pci=disable_10bit_tag=" parameter for peer-to-peer support Dongdong Liu
2021-07-23 11:32   ` Leon Romanovsky
2021-07-23 16:20     ` Logan Gunthorpe
2021-07-25  6:39       ` Leon Romanovsky [this message]
2021-07-26 15:48         ` Logan Gunthorpe
2021-07-27 11:05           ` Leon Romanovsky
2021-07-27 14:30             ` Dongdong Liu
2021-07-27 15:41               ` Leon Romanovsky
2021-07-27 14:00           ` Dongdong Liu
2021-07-23 16:58   ` kernel test robot
2021-07-24 10:35     ` Dongdong Liu
2021-07-23 11:06 ` [PATCH V6 8/8] PCI/P2PDMA: Add a 10-bit tag check in P2PDMA Dongdong Liu
2021-07-23 16:25   ` Logan Gunthorpe
2021-07-24 10:36     ` Dongdong Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YP0HOf7kE1aOkqjV@unreal \
    --to=leon@kernel.org \
    --cc=hch@infradead.org \
    --cc=helgaas@kernel.org \
    --cc=hverkuil-cisco@xs4all.nl \
    --cc=kw@linux.com \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=liudongdong3@huawei.com \
    --cc=logang@deltatee.com \
    --cc=netdev@vger.kernel.org \
    --cc=rajur@chelsio.com \
    --subject='Re: [PATCH V6 7/8] PCI: Add "pci=disable_10bit_tag=" parameter for peer-to-peer support' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).