RE: [PATCH net-next 2/3] devlink: Consider other controller while building phys_port_name

[Parav Pandit <parav@nvidia.com>]

> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Thursday, August 27, 2020 1:38 AM
> 
> On Wed, 26 Aug 2020 04:27:35 +0000 Parav Pandit wrote:
> > > On Tue, 25 Aug 2020 16:58:38 +0300 Parav Pandit wrote:
> > > > A devlink port may be for a controller consist of PCI device.
> > > > A devlink instance holds ports of two types of controllers.
> > > > (1) controller discovered on same system where eswitch resides
> > > > This is the case where PCI PF/VF of a controller and devlink
> > > > eswitch instance both are located on a single system.
> > > > (2) controller located on other system.
> > > > This is the case where a controller is located in one system and
> > > > its devlink eswitch ports are located in a different system. In
> > > > this case devlink instance of the eswitch only have access to
> > > > ports of the controller.
> > > >
> > > > When a devlink eswitch instance serves the devlink ports of both
> > > > controllers together, PCI PF/VF numbers may overlap.
> > > > Due to this a unique phys_port_name cannot be constructed.
> > >
> > > This description is clear as mud to me. Is it just me? Can someone
> > > understand this?
> >
> > I would like to improve this description.
> > Do you have an input to describe these two different controllers, each
> > has same PF and VF numbers?
> 
> Not yet, I'm just trying to figure out how things come together.
> 
> Are some VFs of the same PF under one controller and other ones under a
> different controller?
> 
Correct.

> > $ devlink port show looks like below without a controller annotation.
> > pci/0000:00:08.0/0: type eth netdev eth5 flavour physical
> > pci/0000:00:08.0/1: type eth netdev eth6 flavour pcipf pfnum 0
> > pci/0000:00:08.0/2: type eth netdev eth7 flavour pcipf pfnum 0
> 
> How can you have two PF 0? Aaah - by controller you mean hardware IP, not
> whoever is controlling the switching! So the chip has multiple HW controllers,
> each of which can have multiple PFs?
> 
Hardware IP is one. This IP is plugged into two PCI root complexes.
One is eswitch PF, this PF has its own VFs/SFs.
Other PF(s) plugged into an second PCI Root complex serving the server system.
So you are right there are multiple PFs. Both the PFs have same PCI BDF.

> Definitely please make that more clear.
> 
Ok. yeah.
I should add above details.
I think a text diagram might be more useful.

> Why is @controller_num not under PCI port attrs, but a separate field
> without even a mention of PCI? Are some of the controllers a different bus?
I think It can be added under PCI attribute, but than we need to add it for PF and VF and in future for SF too.
So added it as generic. But yes, keeping it as part of PCI port attributes is good too.