Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Adriana Kobylak]

[-- Attachment #1: Type: text/html, Size: 2376 bytes --]

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Rob Lippert]

On Fri, Jun 9, 2017 at 10:03 AM, Adriana Kobylak <anoo@us.ibm.com> wrote:
> Looping in the community for feedback.
>
>
>
> ----- Original message -----
> From: Adi Gangidi <adi.gangidi@rackspace.com>
> To: "anoo@us.ibm.com" <anoo@us.ibm.com>, Kenneth Wilke
> <kenneth.wilke@RACKSPACE.COM>, Brad DeBauche <brad.debauche@RACKSPACE.COM>,
> Chad Somerlot <chad.somerlot@RACKSPACE.COM>, "abarrera@us.ibm.com"
> <abarrera@us.ibm.com>, "ryan.dc.yu@foxconn.com" <ryan.dc.yu@foxconn.com>,
> "Rob Lippert" <rlippert@google.com>
> Cc:
> Subject: Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. (
> Zaius )
> Date: Fri, Jun 9, 2017 10:30 AM
>
> Sorry for couple of typo below, corrected them now.
>
> Please read again if it didn't make sense before.
>
> -Adi
> ________________________________________
> From: Adi Gangidi
> Sent: Friday, June 9, 2017 10:06 AM
> To: anoo@us.ibm.com; Kenneth Wilke; Brad DeBauche; Chad Somerlot;
> abarrera@us.ibm.com; ryan.dc.yu@foxconn.com; Rob Lippert
> Subject: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )
>
> Hi Adriana /.  Ryan ( Foxconn )
>
> Right now on openBMC
> BMC Shared      = eth0
> BMC Dedicated = eth1
>
>
> Instead, Is there a way we can make sure the BMC dedicated comes up as eth0
> and BMC shared link comes up as eth1, in Zaius / Barreleye G2 ?
>
>
> In how it works currently . you could have BMC dedicated be eth0 or eth1
> depending upon the hardware config ( Whether or not Broadcom 5719A is popped
> OR de-popped ) . That could confusing to support operators who don't
> understand the nuances .

The eth0 and eth1 on the BMC map to the BMC hardware MAC unit #.  On
Zaius MAC1 is connected to various NC-SI stuff and MAC2 is connected
to the dedicated BMC MAC.

So the eth numbers should not move around even if there is nothing
connected on the other end of eth0.  The only way the dedicated BMC
MAC MAC2 would become eth0 should be if the devtree entry for BMC MAC1
was removed.

-Rob

>
> If you guys are busy with other things to make the change  , can I have your
> thoughts on if this can be changed ? I'm guessing this is a u-boot change .
> Any thoughts on how we could make this change ourselves ?
>
> -Adi
>
>
>
>
>
>

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Adi Gangidi]

Thanks Rob, this helps.

We're working with Broadcom on Link flap caused by the NC-SI interface on both the interfaces. We tested bringing down the NC-SI interface in BMC (with ip command) and link flap won't occur at that point on dedicated connection. 

And if it turns out that Broadcom can't resolve this, we will consider disabling that NC-SI interface via device tree. 

-Adi

________________________________________
From: Rob Lippert <rlippert@google.com>
Sent: Friday, June 9, 2017 12:13 PM
To: Adriana Kobylak
Cc: Adi Gangidi; OpenBMC Maillist; Adrian Barrera; Brad DeBauche; Chad Somerlot; Kenneth Wilke; ryan.dc.yu@foxconn.com
Subject: Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

On Fri, Jun 9, 2017 at 10:03 AM, Adriana Kobylak <anoo@us.ibm.com> wrote:
> Looping in the community for feedback.
>
>
>
> ----- Original message -----
> From: Adi Gangidi <adi.gangidi@rackspace.com>
> To: "anoo@us.ibm.com" <anoo@us.ibm.com>, Kenneth Wilke
> <kenneth.wilke@RACKSPACE.COM>, Brad DeBauche <brad.debauche@RACKSPACE.COM>,
> Chad Somerlot <chad.somerlot@RACKSPACE.COM>, "abarrera@us.ibm.com"
> <abarrera@us.ibm.com>, "ryan.dc.yu@foxconn.com" <ryan.dc.yu@foxconn.com>,
> "Rob Lippert" <rlippert@google.com>
> Cc:
> Subject: Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. (
> Zaius )
> Date: Fri, Jun 9, 2017 10:30 AM
>
> Sorry for couple of typo below, corrected them now.
>
> Please read again if it didn't make sense before.
>
> -Adi
> ________________________________________
> From: Adi Gangidi
> Sent: Friday, June 9, 2017 10:06 AM
> To: anoo@us.ibm.com; Kenneth Wilke; Brad DeBauche; Chad Somerlot;
> abarrera@us.ibm.com; ryan.dc.yu@foxconn.com; Rob Lippert
> Subject: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )
>
> Hi Adriana /.  Ryan ( Foxconn )
>
> Right now on openBMC
> BMC Shared      = eth0
> BMC Dedicated = eth1
>
>
> Instead, Is there a way we can make sure the BMC dedicated comes up as eth0
> and BMC shared link comes up as eth1, in Zaius / Barreleye G2 ?
>
>
> In how it works currently . you could have BMC dedicated be eth0 or eth1
> depending upon the hardware config ( Whether or not Broadcom 5719A is popped
> OR de-popped ) . That could confusing to support operators who don't
> understand the nuances .

The eth0 and eth1 on the BMC map to the BMC hardware MAC unit #.  On
Zaius MAC1 is connected to various NC-SI stuff and MAC2 is connected
to the dedicated BMC MAC.

So the eth numbers should not move around even if there is nothing
connected on the other end of eth0.  The only way the dedicated BMC
MAC MAC2 would become eth0 should be if the devtree entry for BMC MAC1
was removed.

-Rob

>
> If you guys are busy with other things to make the change  , can I have your
> thoughts on if this can be changed ? I'm guessing this is a u-boot change .
> Any thoughts on how we could make this change ourselves ?
>
> -Adi
>
>
>
>
>
>

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Joel Stanley]

On Sat, Jun 10, 2017 at 2:33 AM, Adriana Kobylak <anoo@us.ibm.com> wrote:
> From: Adi Gangidi
> Sent: Friday, June 9, 2017 10:06 AM
> To: anoo@us.ibm.com; Kenneth Wilke; Brad DeBauche; Chad Somerlot;
> abarrera@us.ibm.com; ryan.dc.yu@foxconn.com; Rob Lippert
> Subject: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )
>
> Hi Adriana /.  Ryan ( Foxconn )
>
> Right now on openBMC
> BMC Shared      = eth0
> BMC Dedicated = eth1
>
>
> Instead, Is there a way we can make sure the BMC dedicated comes up as eth0
> and BMC shared link comes up as eth1, in Zaius / Barreleye G2 ?
>
>
> In how it works currently . you could have BMC dedicated be eth0 or eth1
> depending upon the hardware config ( Whether or not Broadcom 5719A is popped
> OR de-popped ) . That could confusing to support operators who don't
> understand the nuances .
>
> If you guys are busy with other things to make the change  , can I have your
> thoughts on if this can be changed ? I'm guessing this is a u-boot change .
> Any thoughts on how we could make this change ourselves ?

You can use a udev rule to ensure the devices are named as you expect.

Cheers,

Joel

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 2408 bytes --]

On Thu, Jun 15, 2017 at 03:00:30PM +0930, Joel Stanley wrote:
> On Sat, Jun 10, 2017 at 2:33 AM, Adriana Kobylak <anoo@us.ibm.com> wrote:
> > From: Adi Gangidi
> > Sent: Friday, June 9, 2017 10:06 AM
> > To: anoo@us.ibm.com; Kenneth Wilke; Brad DeBauche; Chad Somerlot;
> > abarrera@us.ibm.com; ryan.dc.yu@foxconn.com; Rob Lippert
> > Subject: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )
> >
> > Hi Adriana /.  Ryan ( Foxconn )
> >
> > Right now on openBMC
> > BMC Shared      = eth0
> > BMC Dedicated = eth1
> >
> >
> > Instead, Is there a way we can make sure the BMC dedicated comes up as eth0
> > and BMC shared link comes up as eth1, in Zaius / Barreleye G2 ?
> >
> >
> > In how it works currently . you could have BMC dedicated be eth0 or eth1
> > depending upon the hardware config ( Whether or not Broadcom 5719A is popped
> > OR de-popped ) . That could confusing to support operators who don't
> > understand the nuances .
> >
> > If you guys are busy with other things to make the change  , can I have your
> > thoughts on if this can be changed ? I'm guessing this is a u-boot change .
> > Any thoughts on how we could make this change ourselves ?
> 
> You can use a udev rule to ensure the devices are named as you expect.

Most Linux distros have moved away from 'ethN' style naming and to
"something else".  My laptop's network interfaces are named 'enp0s31f6'
and 'wlp4s0' for the ethernet and wireless interfaces respectively.

https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/

According to that, the default order of naming we should be getting is something
like this:

1. Names incorporating Firmware/BIOS provided index numbers for on-board devices (example: eno1)
2. Names incorporating Firmware/BIOS provided PCI Express hotplug slot index numbers (example: ens1)
3. Names incorporating physical/geographical location of the connector of the hardware (example: enp2s0)
4. Classic, unpredictable kernel-native ethX naming (example: eth0)

Any idea why we are ending up with #4?  Is this due to device tree
entries / naming?  It seems like using something like #3 would get well
defined and consistent names no matter which network interface is
enabled.  It should also, in theory, make all AST2400 chips arrive at
the same names for NC-SI vs direct PHY.

-- 
Patrick Williams

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Switch up eth0 and eth1 to BMC shared and dedicated resp. ( Zaius )

[Milton Miller II]

On 06/26/2017 about 06:06PM, Patrick Williams wrote:
>On Thu, Jun 15, 2017 at 03:00:30PM +0930, Joel Stanley wrote:
>> On Sat, Jun 10, 2017 at 2:33 AM, Adriana Kobylak <anoo@us.ibm.com>
>wrote:
>> > From: Adi Gangidi
>> > Sent: Friday, June 9, 2017 10:06 AM
>> > To: anoo@us.ibm.com; Kenneth Wilke; Brad DeBauche; Chad Somerlot;
>> > abarrera@us.ibm.com; ryan.dc.yu@foxconn.com; Rob Lippert
>> > Subject: Switch up eth0 and eth1 to BMC shared and dedicated
>resp. ( Zaius )
>> >
>> > Hi Adriana /.  Ryan ( Foxconn )
>> >
>> > Right now on openBMC
>> > BMC Shared      = eth0
>> > BMC Dedicated = eth1
>> >
>> >
>> > Instead, Is there a way we can make sure the BMC dedicated comes
>up as eth0
>> > and BMC shared link comes up as eth1, in Zaius / Barreleye G2 ?
>> >
>> >
>> > In how it works currently . you could have BMC dedicated be eth0
>or eth1
>> > depending upon the hardware config ( Whether or not Broadcom
>5719A is popped
>> > OR de-popped ) . That could confusing to support operators who
>don't
>> > understand the nuances .
>> >
>> > If you guys are busy with other things to make the change  , can
>I have your
>> > thoughts on if this can be changed ? I'm guessing this is a
>u-boot change .
>> > Any thoughts on how we could make this change ourselves ?
>> 
>> You can use a udev rule to ensure the devices are named as you
>expect.
>
>Most Linux distros have moved away from 'ethN' style naming and to
>"something else".  My laptop's network interfaces are named
>'enp0s31f6'
>and 'wlp4s0' for the ethernet and wireless interfaces respectively.
>
>https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkI
>nterfaceNames/
>
>According to that, the default order of naming we should be getting
>is something
>like this:
>
>1. Names incorporating Firmware/BIOS provided index numbers for
>on-board devices (example: eno1)
>2. Names incorporating Firmware/BIOS provided PCI Express hotplug
>slot index numbers (example: ens1)
>3. Names incorporating physical/geographical location of the
>connector of the hardware (example: enp2s0)
>4. Classic, unpredictable kernel-native ethX naming (example: eth0)
>
>Any idea why we are ending up with #4?  Is this due to device tree
>entries / naming?  It seems like using something like #3 would get
>well
>defined and consistent names no matter which network interface is
>enabled.  It should also, in theory, make all AST2400 chips arrive at
>the same names for NC-SI vs direct PHY.
>

Upstream has udev rules to name PCI devices.  Our NICs are not PCI
devices.  We don't have any BIOS or interface to firmware and I am
not aware of any standard for device tree labeling of connectors[1].

I don't think it makes sense to name the pins on the chip, I would
want to follow the phy(s) for the mac(s).


[1] other than something like ibm,location-code in PAPR.

milton