Re: Failing to attach bond(created with two interfaces from different NICs) to a bridge

[Vasundhara Volam <vasundhara-v.volam@broadcom.com>]

On Mon, Sep 7, 2020 at 1:06 PM Ido Schimmel <idosch@idosch.org> wrote:
>
> On Sun, Sep 06, 2020 at 12:23:23PM -0700, Florian Fainelli wrote:
> >
> >
> > On 9/6/2020 10:13 AM, Jakub Kicinski wrote:
> > > On Sun, 6 Sep 2020 14:12:49 +0300 Ido Schimmel wrote:
> > > > On Thu, Sep 03, 2020 at 12:14:28PM -0700, Jakub Kicinski wrote:
> > > > > On Thu, 3 Sep 2020 12:52:25 +0530 Vasundhara Volam wrote:
> > > > > > Hello Jiri,
> > > > > >
> > > > > > After the following set of upstream commits, the user fails to attach
> > > > > > a bond to the bridge, if the user creates the bond with two interfaces
> > > > > > from different bnxt_en NICs. Previously bnxt_en driver does not
> > > > > > advertise the switch_id for legacy mode as part of
> > > > > > ndo_get_port_parent_id cb but with the following patches, switch_id is
> > > > > > returned even in legacy mode which is causing the failure.
> > > > > >
> > > > > > ---------------
> > > > > > 7e1146e8c10c00f859843817da8ecc5d902ea409 net: devlink: introduce
> > > > > > devlink_compat_switch_id_get() helper
> > > > > > 6605a226781eb1224c2dcf974a39eea11862b864 bnxt: pass switch ID through
> > > > > > devlink_port_attrs_set()
> > > > > > 56d9f4e8f70e6f47ad4da7640753cf95ae51a356 bnxt: remove
> > > > > > ndo_get_port_parent_id implementation for physical ports
> > > > > > ----------------
> > > > > >
> > > > > > As there is a plan to get rid of ndo_get_port_parent_id in future, I
> > > > > > think there is a need to fix devlink_compat_switch_id_get() to return
> > > > > > the switch_id only when device is in SWITCHDEV mode and this effects
> > > > > > all the NICs.
> > > > > >
> > > > > > Please let me know your thoughts. Thank you.
> > > > >
> > > > > I'm not Jiri, but I'd think that hiding switch_id from devices should
> > > > > not be the solution here. Especially that no NICs offload bridging
> > > > > today.
> > > > >
> > > > > Could you describe the team/bridge failure in detail, I'm not that
> > > > > familiar with this code.
> > > >
> > > > Maybe:
> > > >
> > > > br_add_slave()
> > > >   br_add_if()
> > > >           nbp_switchdev_mark_set()
> > > >                   dev_get_port_parent_id()
> > > >
> > > > I believe the last call will return '-ENODATA' because the two bnxt
> > > > netdevs member in the bond have different switch IDs. Perhaps the
> > > > function can be changed to return '-EOPNOTSUPP' when it's called for an
> > > > upper device that have multiple parent IDs beneath it:
> > > >
> > > > diff --git a/net/core/dev.c b/net/core/dev.c
> > > > index d42c9ea0c3c0..7932594ca437 100644
> > > > --- a/net/core/dev.c
> > > > +++ b/net/core/dev.c
> > > > @@ -8646,7 +8646,7 @@ int dev_get_port_parent_id(struct net_device *dev,
> > > >                  if (!first.id_len)
> > > >                          first = *ppid;
> > > >                  else if (memcmp(&first, ppid, sizeof(*ppid)))
> > > > -                       return -ENODATA;
> > > > +                       return -EOPNOTSUPP;
> > > >          }
> > > >          return err;
> > >
> > > LGTM, or we could make bridge ignore ENODATA (in case the distinctions
> > > is useful?)
> > >
> > > I was searching for the early versions of Florian's patch set but
> > > I can't find it :( Florian, do you remember if there was a reason to
> > > fail bridge in this case?
> >
> > v3: https://patchwork.kernel.org/patch/10798697/
> > v2: https://lore.kernel.org/patchwork/patch/1038907/
> > v1:
> > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1921358.html
> >
> > I went back to check the tree before
> > d6abc5969463359c366d459247b90366fcd6f5c5 and the logic for return -ENODATA
> > was copied from switchdev_port_attr_get():
> >
> > ...
> >           /* Switch device port(s) may be stacked under
> >            * bond/team/vlan dev, so recurse down to get attr on
> >            * each port.  Return -ENODATA if attr values don't
> >            * compare across ports.
> >            */
> >
> >           netdev_for_each_lower_dev(dev, lower_dev, iter) {
> >                   err = switchdev_port_attr_get(lower_dev, attr);
> >                   if (err)
> >                           break;
> >                   if (first.id == SWITCHDEV_ATTR_ID_UNDEFINED)
> >                           first = *attr;
> >                   else if (memcmp(&first, attr, sizeof(*attr)))
> >                           return -ENODATA;
> >           }
> >
> >           return err;
> >   }
> >   EXPORT_SYMBOL_GPL(switchdev_port_attr_get);
> >
> > the bridge code would specifically treat -EOPNOTSUPP as success and return
> > early, whereas other error code would be treated as a failure.
> >
> > Jiri or Ido, do you remember the reason for return -ENODATA here?
>
> I don't know about the past, but I checked all the current callers of
> dev_get_port_parent_id() and I think the proposed change should be OK:
>
> 1. nbp_switchdev_mark_set(): Current use case. Does not seem to be a
> problem
>
> 2. dev_get_port_parent_id(): Recursive call
>
> 3. netdev_port_same_parent_id(): Unaffected by this change
>
> 4. phys_switch_id_show(): Likewise. Does not recurse
>
> 5. rtnl_phys_switch_id_fill(): Likewise
>
> 6. vif_add: Does not check the error code
>
> I can test the patch in our regression and submit later this week unless
> you have a better suggestion. Please let me know.
Thanks, sounds good.

>
> Thanks