Netdev Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andrew Lunn <andrew@lunn.ch>
To: Dario Alcocer <dalcocer@helixd.com>
Cc: netdev@vger.kernel.org
Subject: Re: Marvell switch port shows LOWERLAYERDOWN, ping fails
Date: Fri, 23 Jul 2021 15:41:57 +0200	[thread overview]
Message-ID: <YPrHJe+zJGJ7oezW@lunn.ch> (raw)
In-Reply-To: <6a70869d-d8d5-4647-0640-4e95866a0392@helixd.com>

On Thu, Jul 22, 2021 at 03:55:04PM -0700, Dario Alcocer wrote:
> After configuring a DSA tree for two Marvell 88E6176 chips, attempts to
> ping a host connected to the first user port fail. Running "ip link" shows
> the user port state as LOWERLAYERDOWN, and no packets make it out the port.
> 
> It's not clear why the state is LOWERLAYERDOWN. The port has a connected
> network device, and the port LED is on.
> 
> I'd welcome any ideas on how to figure out why the port is not working.
> 
> The Marvell chips are configured for multi-chip mode, and are connected
> to a single MDIO bus:
> 
> * chip at PHY address 0x1E is connected to eth0 of the host processor,
>   an Altera Cyclone V
> * chip at PHY address 0x1A is connected over SERDES to the other chip,
>   using port 4
> 
> Here's a console capture showing the port set-up and the failed ping:
> 
> root@dali:~# uname -a
> Linux dali 5.4.114-altera #1 SMP Thu Jul 22 20:07:57 UTC 2021 armv7l armv7l armv7l GNU/Linux
> root@dali:~# ip link
> 
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: can0: <NOARP,ECHO> mtu 16 qdisc noop state DOWN mode DEFAULT group default qlen 10
>     link/can
> 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1508 qdisc mq state UP mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/sit 0.0.0.0 brd 0.0.0.0
> 5: lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 6: lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 7: lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 8: lan4@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 9: dmz@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> root@dali:~# ip addr add 192.0.2.1/24 dev lan1
> root@dali:~# ip link set lan1 up
> [  902.623835] mv88e6085 stmmac-0:1a lan1: configuring for phy/gmii link mode
> [  902.633092] 8021q: adding VLAN 0 to HW filter on device lan1
> root@dali:~# ping -c 4 192.0.2.2
> PING 192.0.2.2 (192.0.2.2): 56 data bytes
> 
> --- 192.0.2.2 ping statistics ---
> 4 packets transmitted, 0 packets received, 100% packet loss
> root@dali:~# ip link
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: can0: <NOARP,ECHO> mtu 16 qdisc noop state DOWN mode DEFAULT group default qlen 10
>     link/can
> 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1508 qdisc mq state UP mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/sit 0.0.0.0 brd 0.0.0.0
> 5: lan1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff

NO-CARRIER suggests a PHY problem. You should see it report link up if
it really is up.

When then switch probes the PHYs should also probe. What PHY driver is
being used? You want the Marvell PHY driver, not genphy.

> 6: lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 7: lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 8: lan4@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> 9: dmz@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
>     link/ether aa:10:22:58:38:77 brd ff:ff:ff:ff:ff:ff
> root@dali:~# ip route
> default via 192.168.0.1 dev eth0

That is not correct. eth0 just acts as a pipe towards the switch. It
does not have an IP address, and there should not be any routes using
it. It needs to be configured up, but that is all.

> 192.0.2.0/24 dev lan1 proto kernel scope link src 192.0.2.1 linkdown
> 192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.3
> 224.0.0.0/4 dev eth0 scope link src 192.168.0.3
> root@dali:~#
> 
> The DSA switch tree is configured using this device tree fragment:
> 
> gmac0 {
>     status = "okay";
>     phy-mode = "gmii";

gmii? That is a bit unusual. You normally see rgmii.

>     txc-skew-ps = <0xbb8>;
>     rxc-skew-ps = <0xbb8>;
>     txen-skew-ps = <0>;
>     rxdv-skew-ps = <0>;
>     rxd0-skew-ps = <0>;
>     rxd1-skew-ps = <0>;
>     rxd2-skew-ps = <0>;
>     rxd3-skew-ps = <0>;
>     txd0-skew-ps = <0>;
>     txd1-skew-ps = <0>;
>     txd2-skew-ps = <0>;
>     txd3-skew-ps = <0>;
>     max-frame-size = <3800>;
> 
>     fixed-link {
>     speed = <0x3e8>;

decimal would be much more readable. Or is this not the source .dts
file, but you have decompiled the DTB back to DTS?

>         full-duplex;
>         pause;
>     };
> 
>     mdio {
>         compatible = "snps,dwmac-mdio";
>         #address-cells = <0x1>;
>         #size-cells = <0x0>;
> 
>         switch0: switch0@1a {
>             compatible = "marvell,mv88e6085";
>             reg = <0x1a>;
>             dsa,member = <0 0>;
>             ports {
>                 #address-cells = <1>;
>                 #size-cells = <0>;
>                 port@0 {
>                     reg = <0>;
>                     label = "lan1";
>                 };
>                 port@1 {
>                     reg = <1>;
>                     label = "lan2";
>                 };
>                 port@2 {
>                     reg = <2>;
>                     label = "lan3";
>                 };
>                 switch0port4: port@4 {
>                     reg = <4>;
>                     phy-mode = "rgmii-id";
>                     link = <&switch1port4>;
>                     fixed-link {
>                         speed = <1000>;
>                         full-duplex;
>                     };
>                 };
>                 port@6 {
>                     reg = <6>;
>                     ethernet = <&gmac0>;
>                     label = "cpu";
>                 };
>             };
>         };
>         switch1: switch1@1e {
>             compatible = "marvell,mv88e6085";
>             reg = <0x1e>;
>             dsa,member = <0 1>;
>             ports {
>                 #address-cells = <1>;
>                 #size-cells = <0>;
>                 port@0 {
>                     reg = <0>;
>                     label = "lan4";
>                 };
>                 port@1 {
>                     reg = <1>;
>                     label = "dmz";
>                 };
>                 switch1port4: port@4 {
>                     reg = <4>;
>                     link = <&switch0port4>;
>                     phy-mode = "rgmii-id";

You probably don't want both ends of the link in rgmii-id mode. That
will give you twice the delay.

    Andrew

  reply	other threads:[~2021-07-23 13:42 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-22 22:55 Dario Alcocer
2021-07-23 13:41 ` Andrew Lunn [this message]
2021-07-23 18:21   ` Dario Alcocer
2021-07-23 18:25     ` Andrew Lunn
2021-07-23 18:36       ` Dario Alcocer
2021-07-23 21:58         ` Dario Alcocer
2021-07-24 17:34           ` Andrew Lunn
2021-07-25  2:26             ` Dario Alcocer
2021-07-25  2:36               ` Dario Alcocer
2021-07-27  1:39                 ` Dario Alcocer
2021-07-28 18:07                   ` Dario Alcocer
2021-07-28 18:23                     ` Andrew Lunn
2021-07-28 18:33                       ` Dario Alcocer
2021-07-28 19:24                         ` Andrew Lunn
2021-07-28 19:37                           ` Dario Alcocer
2021-07-28 20:47                             ` Andrew Lunn
2021-07-28 20:54                               ` Dario Alcocer
2021-07-28 21:09                             ` Andrew Lunn
2021-08-05 21:44                             ` Dario Alcocer
2021-08-06 16:03                               ` Dario Alcocer
2021-08-06 23:46                                 ` Dario Alcocer
2021-08-07 18:57                                   ` Andrew Lunn
2021-08-09 16:28                                     ` Dario Alcocer
2021-08-10 20:58                                       ` Dario Alcocer
2021-08-10 22:13                                         ` Andrew Lunn
2021-08-11 13:16                                           ` Dario Alcocer
2021-08-07 17:44                               ` Andrew Lunn

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YPrHJe+zJGJ7oezW@lunn.ch \
    --to=andrew@lunn.ch \
    --cc=dalcocer@helixd.com \
    --cc=netdev@vger.kernel.org \
    --subject='Re: Marvell switch port shows LOWERLAYERDOWN, ping fails' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).