on the topic of alternate MAC addresses

on the topic of alternate MAC addresses

[David Miller]

I've been meaning to bring this up.

A lot of cards that support multiple MAC addresses
in hardware provide pre-cooked lists of alternate
MAC addresses.  This is either done via EEPROM,
NVRAM, or OpenFirmware device properties.

For example, the Sun Neptune cards can provide an
array of up to 16 alternate MAC addresses per-port.

We should provide a way to export these lists to
userspace.

Perhaps an ethtool thing, Jeff?

Once we have this, virtualization implementations can probe for and
use them.  Currently they have no way to discover these things so
probably they just generate their own using some algorithm.

Re: on the topic of alternate MAC addresses

[Jeff Garzik]

David Miller wrote:
> I've been meaning to bring this up.
> 
> A lot of cards that support multiple MAC addresses
> in hardware provide pre-cooked lists of alternate
> MAC addresses.  This is either done via EEPROM,
> NVRAM, or OpenFirmware device properties.
> 
> For example, the Sun Neptune cards can provide an
> array of up to 16 alternate MAC addresses per-port.
> 
> We should provide a way to export these lists to
> userspace.
> 
> Perhaps an ethtool thing, Jeff?
> 
> Once we have this, virtualization implementations can probe for and
> use them.  Currently they have no way to discover these things so
> probably they just generate their own using some algorithm.

hmmmm.  Using ethtool isn't a big deal, but IMO you probably want more 
than just an exported list for the usage you described...  it sounds 
like some sort of reservation system should be used, to note which MAC 
addresses are [not] in use?

Then a virt client -- or anyone who wants multiple unicast addresses for 
whatever reason -- can let other clients to avoid MAC addresses 1, 7, or 
13 (random examples).

	Jeff

Re: on the topic of alternate MAC addresses

[Jeff Garzik]

Jeff Garzik wrote:
> hmmmm.  Using ethtool isn't a big deal, but IMO you probably want more 
> than just an exported list for the usage you described...  it sounds 
> like some sort of reservation system should be used, to note which MAC 
> addresses are [not] in use?
> 
> Then a virt client -- or anyone who wants multiple unicast addresses for 
> whatever reason -- can let other clients to avoid MAC addresses 1, 7, or 
> 13 (random examples).

er, "can let other clients know to"....

Re: on the topic of alternate MAC addresses

[David Miller]

From: Jeff Garzik <jeff@garzik.org>
Date: Tue, 23 Oct 2007 22:25:05 -0400

> hmmmm.  Using ethtool isn't a big deal, but IMO you probably want more 
> than just an exported list for the usage you described...  it sounds 
> like some sort of reservation system should be used, to note which MAC 
> addresses are [not] in use?
> 
> Then a virt client -- or anyone who wants multiple unicast addresses for 
> whatever reason -- can let other clients to avoid MAC addresses 1, 7, or 
> 13 (random examples).

I see your point.

However, it's not the virt clients that do this, it's the control
node (aka: domain 0) which has to manage these things.

It has to manage all of the global hardware resources and allocate
them out to itself and the clients anyways.

And this is why I think it's sufficient to just publish the list of
MAC addresses from the driver, and leave the allocation and policy
to the userland virtualizatin daemon running on the control node.

Let me know if you still disagree.

Re: on the topic of alternate MAC addresses

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On Tue, 2007-10-23 at 20:22 -0700, David Miller wrote:

> However, it's not the virt clients that do this, it's the control
> node (aka: domain 0) which has to manage these things.
> 
> It has to manage all of the global hardware resources and allocate
> them out to itself and the clients anyways.
> 
> And this is why I think it's sufficient to just publish the list of
> MAC addresses from the driver, and leave the allocation and policy
> to the userland virtualizatin daemon running on the control node.

From a wireless angle, however, it's not sufficient. It appears that
there are some wireless cards that have multiple MAC addresses in their
EEPROM (or a way to generate multiple, by e.g. the vendor assigning only
even addresses and reserving odd ones). Then, bringing up a second,
third, ... virtual wireless interface should for best usability choose
an alternate address if the same one cannot be used due to restrictions.

We can probably manage this issue in userspace, in fact, for AP mode we
require proper configuration in hostapd, but it seems that some sort of
reservation system would be easier for multiple virtual station
interfaces when supported (currently no driver does).

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: on the topic of alternate MAC addresses

[Eric W. Biederman]

David Miller <davem@davemloft.net> writes:

> From: Jeff Garzik <jeff@garzik.org>
> Date: Tue, 23 Oct 2007 22:25:05 -0400
>
>> hmmmm.  Using ethtool isn't a big deal, but IMO you probably want more 
>> than just an exported list for the usage you described...  it sounds 
>> like some sort of reservation system should be used, to note which MAC 
>> addresses are [not] in use?
>> 
>> Then a virt client -- or anyone who wants multiple unicast addresses for 
>> whatever reason -- can let other clients to avoid MAC addresses 1, 7, or 
>> 13 (random examples).
>
> I see your point.
>
> However, it's not the virt clients that do this, it's the control
> node (aka: domain 0) which has to manage these things.
>
> It has to manage all of the global hardware resources and allocate
> them out to itself and the clients anyways.
>
> And this is why I think it's sufficient to just publish the list of
> MAC addresses from the driver, and leave the allocation and policy
> to the userland virtualizatin daemon running on the control node.
>
> Let me know if you still disagree.

On the per interface level we know the set of used mac addresses in
the uc_list.

At least at a per interface level we should already have this
information, and this is where we care because duplicates at
that could cause problems.

Duplicate mac addresses across interfaces on the same machine
should generally be a don't care.  Although there may some
cases we don't mind.

Currently drivers like macvlan that care today call
random_ether_addr().  I think it would make sense to convenience
kernel function that picked the next available unicast address
for an interface that was not on the uc_list, (called
random_ether_adder if there was not such address) and called
dev_unicast_add to let the driver receive it and to keep other people
from using it.

Exporting that information with ethtool to handle the strange cases
makes sense.  But it looks like there are easy cases that don't
need that help.

Eric

Re: on the topic of alternate MAC addresses

[Rick Jones]

Eric W. Biederman wrote:
> Duplicate mac addresses across interfaces on the same machine
> should generally be a don't care.  Although there may some
> cases we don't mind.

What might the switches think of that?  Outside of the context of a 
link-aggregate I would think that could do some unhappy things when multiple 
interfaces are connected to the same broadcast domain.

That the "all interfaces have one MAC" default behaviour on SPARC systems often 
ran into troubles is probably cautionary here.

rick jones

Re: on the topic of alternate MAC addresses

[Eric W. Biederman]

Rick Jones <rick.jones2@hp.com> writes:

> Eric W. Biederman wrote:
>> Duplicate mac addresses across interfaces on the same machine
>> should generally be a don't care.  Although there may some
>> cases we don't mind.
>
> What might the switches think of that?  Outside of the context of a
> link-aggregate I would think that could do some unhappy things when multiple
> interfaces are connected to the same broadcast domain.
>
> That the "all interfaces have one MAC" default behaviour on SPARC systems often
> ran into troubles is probably cautionary here.

Sure we don't want to duplicate mac addresses globally by default.

The case that is always wrong is duplicate mac addresses going
in and out the same interface, and skimming through the code
it looks like we already have the data structures necessary to
prevent duplicate use of unicast mac addresses already.

Eric