[Qemu-devel] -net interface association behavior change in current -git.

[Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

In 1.14.0, if I did this:

  qemu -net nic,blah -net user -net nic,blah -net tun,blah

Then the first nic would be -net user, and the second nic would be -net
tun.    In current -git, -net user attaches to the second interface and
-net tun attaches to the first, I.E. the order is reversed.

Either way the first -nic becomes eth0 in Linux and the second becomes
eth1 (I can manually assign mac addresses in order to confirm which is
which), but eth0 used to be the -net user interface and now eth1 is the
-net user interface.

I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
changed the behavior, and I can't find _documentation_ on having
multiple interfaces transports hooked up to the same qemu instance
anyway.  (It used to work, but possibly that was an accident?)

Any ideas?

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Vincent Palatin]

Hi,

On Wed, May 11, 2011 at 22:39, Rob Landley <rob@landley.net> wrote:
> In 1.14.0, if I did this:
>
>  qemu -net nic,blah -net user -net nic,blah -net tun,blah
>
> Then the first nic would be -net user, and the second nic would be -net
> tun.    In current -git, -net user attaches to the second interface and
> -net tun attaches to the first, I.E. the order is reversed.
>
> Either way the first -nic becomes eth0 in Linux and the second becomes
> eth1 (I can manually assign mac addresses in order to confirm which is
> which), but eth0 used to be the -net user interface and now eth1 is the
> -net user interface.
>
> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
> changed the behavior, and I can't find _documentation_ on having
> multiple interfaces transports hooked up to the same qemu instance
> anyway.  (It used to work, but possibly that was an accident?)
>
> Any ideas?

First of all, as you have 2 totally separated subnets in your setup, I
think your command-line should use "vlan=" parameter to isolate them,
else you will end up with some random routing/broadcasting (and random
tends to change over time).
I'm not using setup with multiple NICs but I would have written something like :
qemu -net nic,vlan=1,blah -net user,vlan=1 -net nic,vlan=2,blah -net
tun,vlan=2,blah

In addition to this, which type of NIC are you using ?
In my understanding, the Linux kernel might assign interface number
depending on the order the interfaces are appearing.
Without my change, when a packet arrives and should be distributed to
multiple interfaces (that seems to be the case in your setup even
though it is not intended) if one of the interface is not ready, the
packet is only forwarded to the ready interface (and the other never
receives it). This produces interesting timing effects where packets
are routed according to obscure race conditions, but in your former
setup, that might cause the packet to be routed to the interface you
want.

-- 
Vincent

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/12/2011 08:20 AM, Vincent Palatin wrote:
> Hi,
> 
> On Wed, May 11, 2011 at 22:39, Rob Landley <rob@landley.net> wrote:
>> In 1.14.0, if I did this:
>>
>>  qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>
>> Then the first nic would be -net user, and the second nic would be -net
>> tun.    In current -git, -net user attaches to the second interface and
>> -net tun attaches to the first, I.E. the order is reversed.
>>
>> Either way the first -nic becomes eth0 in Linux and the second becomes
>> eth1 (I can manually assign mac addresses in order to confirm which is
>> which), but eth0 used to be the -net user interface and now eth1 is the
>> -net user interface.
>>
>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>> changed the behavior, and I can't find _documentation_ on having
>> multiple interfaces transports hooked up to the same qemu instance
>> anyway.  (It used to work, but possibly that was an accident?)
>>
>> Any ideas?
> 
> First of all, as you have 2 totally separated subnets in your setup, I
> think your command-line should use "vlan=" parameter to isolate them,
> else you will end up with some random routing/broadcasting (and random
> tends to change over time).

Does the kernel need some sort of vlan support compiled into it for this
to work, or can the kernel not care?

> I'm not using setup with multiple NICs but I would have written something like :
> qemu -net nic,vlan=1,blah -net user,vlan=1 -net nic,vlan=2,blah -net
> tun,vlan=2,blah
> 
> In addition to this, which type of NIC are you using ?

The actual command line is various permutations of:

~/qemu/qemu/x86_64-softmmu/qemu-system-x86_64 -m 512 -kernel
~/linux/linux/arch/x86/boot/bzImage -no-reboot -hda squeeze.ext3 -append
"root=/dev/hda rw" -net nic,model=e1000,macaddr=52:54:00:11:11:11 -net
user -redir tcp:9876::22 -net nic,model=e1000,macaddr=52:54:00:22:22:22
-net tap,ifname=kvm0,script=no,downscript=no

With the parameters reordered to try to beat some controllable behavior
out of it.  (It had a deterministic behavior before the above commit,
going back at least 2 years.  The behavior was changed by the commit.)

> In my understanding, the Linux kernel might assign interface number
> depending on the order the interfaces are appearing.

It's going in PCI bus order.  And the _interfaces_ are still happily in
PCI bus order both before and after this commit.  What's changing is the
association between interface and -net user or -net tap.

The first one, with macaddr 11:11:11, is always eth0.  But before the
patch, eth0 is -net user, and after the patch eth0 is -net tap.

> Without my change, when a packet arrives and should be distributed to
> multiple interfaces (that seems to be the case in your setup even
> though it is not intended)

That that I know of.  DHCP was doing broadcast out of eth0, but ignoring
eth1.  (It's still attempting to do that, it's just not getting a
response unless I hook the tun/tap plumbing up to a dhcp server on the
host.)

> if one of the interface is not ready, the
> packet is only forwarded to the ready interface (and the other never
> receives it). This produces interesting timing effects where packets
> are routed according to obscure race conditions, but in your former
> setup, that might cause the packet to be routed to the interface you
> want.

That does not seem to match what I'm seeing.

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Vincent Palatin]

>> First of all, as you have 2 totally separated subnets in your setup, I
>> think your command-line should use "vlan=" parameter to isolate them,
>> else you will end up with some random routing/broadcasting (and random
>> tends to change over time).
>
> Does the kernel need some sort of vlan support compiled into it for this
> to work, or can the kernel not care?

You don't need anything in your kernel, this is for Qemu network layer
configuration.

>> I'm not using setup with multiple NICs but I would have written something like :
>> qemu -net nic,vlan=1,blah -net user,vlan=1 -net nic,vlan=2,blah -net
>> tun,vlan=2,blah
>>
>> In addition to this, which type of NIC are you using ?
>
> The actual command line is various permutations of:
>
> ~/qemu/qemu/x86_64-softmmu/qemu-system-x86_64 -m 512 -kernel
> ~/linux/linux/arch/x86/boot/bzImage -no-reboot -hda squeeze.ext3 -append
> "root=/dev/hda rw" -net nic,model=e1000,macaddr=52:54:00:11:11:11 -net
> user -redir tcp:9876::22 -net nic,model=e1000,macaddr=52:54:00:22:22:22
> -net tap,ifname=kvm0,script=no,downscript=no
>
> With the parameters reordered to try to beat some controllable behavior
> out of it.  (It had a deterministic behavior before the above commit,
> going back at least 2 years.  The behavior was changed by the commit.)
>
>> In my understanding, the Linux kernel might assign interface number
>> depending on the order the interfaces are appearing.
>
> It's going in PCI bus order.  And the _interfaces_ are still happily in
> PCI bus order both before and after this commit.  What's changing is the
> association between interface and -net user or -net tap.
>
> The first one, with macaddr 11:11:11, is always eth0.  But before the
> patch, eth0 is -net user, and after the patch eth0 is -net tap.

So, did you try the vlan parameter which is supposed to associate each
interface with the proper thing (IMO the command line order should not
have anything to do with it) ?

-- 
Vincent

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/12/2011 08:19 PM, Vincent Palatin wrote:
>>> First of all, as you have 2 totally separated subnets in your setup, I
>>> think your command-line should use "vlan=" parameter to isolate them,
>>> else you will end up with some random routing/broadcasting (and random
>>> tends to change over time).
>>
>> Does the kernel need some sort of vlan support compiled into it for this
>> to work, or can the kernel not care?
> 
> You don't need anything in your kernel, this is for Qemu network layer
> configuration.

Oh good.  (The kernel has a couple different VLAN config options, plus
cisco has vlan stuff in its routers...  I've been trying not to get any
of this complexity on me.)

>>> In my understanding, the Linux kernel might assign interface number
>>> depending on the order the interfaces are appearing.
>>
>> It's going in PCI bus order.  And the _interfaces_ are still happily in
>> PCI bus order both before and after this commit.  What's changing is the
>> association between interface and -net user or -net tap.
>>
>> The first one, with macaddr 11:11:11, is always eth0.  But before the
>> patch, eth0 is -net user, and after the patch eth0 is -net tap.
> 
> So, did you try the vlan parameter which is supposed to associate each
> interface with the proper thing (IMO the command line order should not
> have anything to do with it) ?

Apparently you cannot stick a vlan on -redir, and if you do the error
message is a bit funky (cannot initialize -net user).  Good to know...

Yup, that fixed it.  Thanks.

By the way, does -redir only apply to -net user?  Actually, reading qemu
--help it looks like I should be using hostfwd= but the documentation on
that is REALLY WEIRD:

  This option can not be given multiple times, but multiple rules may
  be combined.
...
  To combine two or more hostfwd rules, simply use a comma as a
  delimiter. For example, to combine the two rules mentioned in
  the examples above, use the following:

  #on the host
  qemu -net user,hostfwd=tcp:127.0.0.1:6001-:6000,hostfwd=tcp:5555-::23

Define "may not be given multiple times"...?  Oh well, specifying one of
'em seems to work.

Thanks.  As you can tell, confusion is sort of my ground state,

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2340 bytes --]

On 2011-05-13 03:49, Rob Landley wrote:
> On 05/12/2011 08:19 PM, Vincent Palatin wrote:
>>>> First of all, as you have 2 totally separated subnets in your setup, I
>>>> think your command-line should use "vlan=" parameter to isolate them,
>>>> else you will end up with some random routing/broadcasting (and random
>>>> tends to change over time).
>>>
>>> Does the kernel need some sort of vlan support compiled into it for this
>>> to work, or can the kernel not care?
>>
>> You don't need anything in your kernel, this is for Qemu network layer
>> configuration.
> 
> Oh good.  (The kernel has a couple different VLAN config options, plus
> cisco has vlan stuff in its routers...  I've been trying not to get any
> of this complexity on me.)
> 
>>>> In my understanding, the Linux kernel might assign interface number
>>>> depending on the order the interfaces are appearing.
>>>
>>> It's going in PCI bus order.  And the _interfaces_ are still happily in
>>> PCI bus order both before and after this commit.  What's changing is the
>>> association between interface and -net user or -net tap.
>>>
>>> The first one, with macaddr 11:11:11, is always eth0.  But before the
>>> patch, eth0 is -net user, and after the patch eth0 is -net tap.
>>
>> So, did you try the vlan parameter which is supposed to associate each
>> interface with the proper thing (IMO the command line order should not
>> have anything to do with it) ?
> 
> Apparently you cannot stick a vlan on -redir, and if you do the error
> message is a bit funky (cannot initialize -net user).  Good to know...
> 
> Yup, that fixed it.  Thanks.
> 
> By the way, does -redir only apply to -net user?

-redir (like all the other free-standing usernet switches) is legacy as
it can only be applied to a single usernet instance. And it's highly
confusing to which it refers. That's also documented in the man page.

>  Actually, reading qemu
> --help it looks like I should be using hostfwd= but the documentation on
> that is REALLY WEIRD:
> 
>   This option can not be given multiple times, but multiple rules may
>   be combined.

Where did you find this? A quick grep did not reveal the location to me.
qemu-options.hx states it clearly:

@item hostfwd=[tcp|udp]:
[...]
This option can be given multiple times.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/13/2011 01:39 AM, Jan Kiszka wrote:
>>  Actually, reading qemu
>> --help it looks like I should be using hostfwd= but the documentation on
>> that is REALLY WEIRD:
>>
>>   This option can not be given multiple times, but multiple rules may
>>   be combined.
> 
> Where did you find this? A quick grep did not reveal the location to me.
> qemu-options.hx states it clearly:

I did a "man qemu" and searched for hostfwd, it was at the end of the
first paragraph:

   hostfwd=[tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport
       Redirect incoming TCP or UDP connections to the host port
       hostport to the guest IP address guestaddr on guest port
       guestport. If guestaddr is not specified, its value is x.x.x.15
       (default first address given by the built-in DHCP server). By
       specifying hostaddr, the rule can be bound to a specific host
       interface. If no connection type is set, TCP is used. This
       option can not be given multiple times, but multiple rules may
       be combined.

Looks like it grabbed the xubuntu 10.04 (current LTS) man page, which is
wrong.  The one in the qemu source is correct, but wasn't built by
"make" or installed by "make install".  (It built when I explicitlly
said make qemu.1 though.  Go figure.)

*shrug*  Never underestimate ubuntu's ability to subtly break qemu, this
is the... fourth different way I've noticed them doing that?  (I dunno
why they keep fiddling with it.  Thought installing qemu from source
also installed the man page, which would be in the search path before
the distro version.  My bad...)

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Jan Kiszka]

On 2011-05-13 12:31, Rob Landley wrote:
> On 05/13/2011 01:39 AM, Jan Kiszka wrote:
>>>  Actually, reading qemu
>>> --help it looks like I should be using hostfwd= but the documentation on
>>> that is REALLY WEIRD:
>>>
>>>   This option can not be given multiple times, but multiple rules may
>>>   be combined.
>>
>> Where did you find this? A quick grep did not reveal the location to me.
>> qemu-options.hx states it clearly:
> 
> I did a "man qemu" and searched for hostfwd, it was at the end of the
> first paragraph:
> 
>    hostfwd=[tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport
>        Redirect incoming TCP or UDP connections to the host port
>        hostport to the guest IP address guestaddr on guest port
>        guestport. If guestaddr is not specified, its value is x.x.x.15
>        (default first address given by the built-in DHCP server). By
>        specifying hostaddr, the rule can be bound to a specific host
>        interface. If no connection type is set, TCP is used. This
>        option can not be given multiple times, but multiple rules may
>        be combined.
> 
> Looks like it grabbed the xubuntu 10.04 (current LTS) man page, which is
> wrong.  The one in the qemu source is correct, but wasn't built by
> "make" or installed by "make install".  (It built when I explicitlly
> said make qemu.1 though.  Go figure.)
> 
> *shrug*  Never underestimate ubuntu's ability to subtly break qemu, this
> is the... fourth different way I've noticed them doing that?  (I dunno
> why they keep fiddling with it.  Thought installing qemu from source
> also installed the man page, which would be in the search path before
> the distro version.  My bad...)

Did they also "improve" the code accordingly? Indeed very strange.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Brad Hards]

On Fri, 13 May 2011 03:19:42 AM Vincent Palatin wrote:
> > Does the kernel need some sort of vlan support compiled into it for this
> > to work, or can the kernel not care?
> 
> You don't need anything in your kernel, this is for Qemu network layer
> configuration.
Just to explain a bit further: qemu "vlan" is not related to 803.1q vlan 
tagging. vlan is virtual lan in the sense of a virtual machine - something 
we're emulating in qemu.

Brad
 

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Markus Armbruster]

Rob Landley <rob@landley.net> writes:

> In 1.14.0, if I did this:
>
>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>
> Then the first nic would be -net user, and the second nic would be -net
> tun.    In current -git, -net user attaches to the second interface and
> -net tun attaches to the first, I.E. the order is reversed.
>
> Either way the first -nic becomes eth0 in Linux and the second becomes
> eth1 (I can manually assign mac addresses in order to confirm which is
> which), but eth0 used to be the -net user interface and now eth1 is the
> -net user interface.
>
> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
> changed the behavior, and I can't find _documentation_ on having
> multiple interfaces transports hooked up to the same qemu instance
> anyway.  (It used to work, but possibly that was an accident?)
>
> Any ideas?

Does it happen with -device and -netdev as well?

See docs/qdev-device-use.txt for how to go from -net to -device.

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/12/2011 09:10 AM, Markus Armbruster wrote:
> Rob Landley <rob@landley.net> writes:
> 
>> In 1.14.0, if I did this:
>>
>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>
>> Then the first nic would be -net user, and the second nic would be -net
>> tun.    In current -git, -net user attaches to the second interface and
>> -net tun attaches to the first, I.E. the order is reversed.
>>
>> Either way the first -nic becomes eth0 in Linux and the second becomes
>> eth1 (I can manually assign mac addresses in order to confirm which is
>> which), but eth0 used to be the -net user interface and now eth1 is the
>> -net user interface.
>>
>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>> changed the behavior, and I can't find _documentation_ on having
>> multiple interfaces transports hooked up to the same qemu instance
>> anyway.  (It used to work, but possibly that was an accident?)
>>
>> Any ideas?
> 
> Does it happen with -device and -netdev as well?
> 
> See docs/qdev-device-use.txt for how to go from -net to -device.

Read read read...

That seems to be micromanaging PCI bus slot assignment, which isn't
changed by this patch.  The cards don't move around, nor does the
association between cards and Linux eth0/eth1.  What changes is which
virtual LAN each virtual ethernet card is plugged into.  (The virtual
cat5 cable coming out of the card moves to a different switch.)

(The fix was to tag everything with vlan arguments and manually manage
the association.)

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Markus Armbruster]

Rob Landley <rob@landley.net> writes:

> On 05/12/2011 09:10 AM, Markus Armbruster wrote:
>> Rob Landley <rob@landley.net> writes:
>> 
>>> In 1.14.0, if I did this:
>>>
>>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>>
>>> Then the first nic would be -net user, and the second nic would be -net
>>> tun.    In current -git, -net user attaches to the second interface and
>>> -net tun attaches to the first, I.E. the order is reversed.
>>>
>>> Either way the first -nic becomes eth0 in Linux and the second becomes
>>> eth1 (I can manually assign mac addresses in order to confirm which is
>>> which), but eth0 used to be the -net user interface and now eth1 is the
>>> -net user interface.
>>>
>>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>>> changed the behavior, and I can't find _documentation_ on having
>>> multiple interfaces transports hooked up to the same qemu instance
>>> anyway.  (It used to work, but possibly that was an accident?)
>>>
>>> Any ideas?
>> 
>> Does it happen with -device and -netdev as well?
>> 
>> See docs/qdev-device-use.txt for how to go from -net to -device.
>
> Read read read...
>
> That seems to be micromanaging PCI bus slot assignment, which isn't
> changed by this patch.  The cards don't move around, nor does the
> association between cards and Linux eth0/eth1.  What changes is which
> virtual LAN each virtual ethernet card is plugged into.  (The virtual
> cat5 cable coming out of the card moves to a different switch.)

I didn't mean to tell you "try using -device to juggle PCI addresses".
I meant to steer you away from QEMU VLANs, to find out whether they're a
factor in your problem.  Possible, because non-VLAN uses a few different
code paths in QEMU.  Sorry if I was too terse.

In general, my advice is stay away from QEMU VLANs.

> (The fix was to tag everything with vlan arguments and manually manage
> the association.)

Glad you got your problem solved.

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/13/2011 01:54 AM, Markus Armbruster wrote:
> Rob Landley <rob@landley.net> writes:
> 
>> On 05/12/2011 09:10 AM, Markus Armbruster wrote:
>>> Rob Landley <rob@landley.net> writes:
>>>
>>>> In 1.14.0, if I did this:
>>>>
>>>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>>>
>>>> Then the first nic would be -net user, and the second nic would be -net
>>>> tun.    In current -git, -net user attaches to the second interface and
>>>> -net tun attaches to the first, I.E. the order is reversed.
>>>>
>>>> Either way the first -nic becomes eth0 in Linux and the second becomes
>>>> eth1 (I can manually assign mac addresses in order to confirm which is
>>>> which), but eth0 used to be the -net user interface and now eth1 is the
>>>> -net user interface.
>>>>
>>>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>>>> changed the behavior, and I can't find _documentation_ on having
>>>> multiple interfaces transports hooked up to the same qemu instance
>>>> anyway.  (It used to work, but possibly that was an accident?)
>>>>
>>>> Any ideas?
>>>
>>> Does it happen with -device and -netdev as well?
>>>
>>> See docs/qdev-device-use.txt for how to go from -net to -device.
>>
>> Read read read...
>>
>> That seems to be micromanaging PCI bus slot assignment, which isn't
>> changed by this patch.  The cards don't move around, nor does the
>> association between cards and Linux eth0/eth1.  What changes is which
>> virtual LAN each virtual ethernet card is plugged into.  (The virtual
>> cat5 cable coming out of the card moves to a different switch.)
> 
> I didn't mean to tell you "try using -device to juggle PCI addresses".
> I meant to steer you away from QEMU VLANs, to find out whether they're a
> factor in your problem.  Possible, because non-VLAN uses a few different
> code paths in QEMU.  Sorry if I was too terse.
> 
> In general, my advice is stay away from QEMU VLANs.

Ok, now I'm confused.

Before this, I wasn't using them.  Now I am.  What's the reason for
avoiding them?  (Also, I didn't see a way in -device to specify a
network transport, just cards and their properties.  Quite possibly I
missed it...)

>> (The fix was to tag everything with vlan arguments and manually manage
>> the association.)
> 
> Glad you got your problem solved.

Solved, yes.  Understood... less so than I thought, apparently?

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Markus Armbruster]

Rob Landley <rob@landley.net> writes:

> On 05/13/2011 01:54 AM, Markus Armbruster wrote:
>> Rob Landley <rob@landley.net> writes:
>> 
>>> On 05/12/2011 09:10 AM, Markus Armbruster wrote:
>>>> Rob Landley <rob@landley.net> writes:
>>>>
>>>>> In 1.14.0, if I did this:
>>>>>
>>>>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>>>>
>>>>> Then the first nic would be -net user, and the second nic would be -net
>>>>> tun.    In current -git, -net user attaches to the second interface and
>>>>> -net tun attaches to the first, I.E. the order is reversed.
>>>>>
>>>>> Either way the first -nic becomes eth0 in Linux and the second becomes
>>>>> eth1 (I can manually assign mac addresses in order to confirm which is
>>>>> which), but eth0 used to be the -net user interface and now eth1 is the
>>>>> -net user interface.
>>>>>
>>>>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>>>>> changed the behavior, and I can't find _documentation_ on having
>>>>> multiple interfaces transports hooked up to the same qemu instance
>>>>> anyway.  (It used to work, but possibly that was an accident?)
>>>>>
>>>>> Any ideas?
>>>>
>>>> Does it happen with -device and -netdev as well?
>>>>
>>>> See docs/qdev-device-use.txt for how to go from -net to -device.
>>>
>>> Read read read...
>>>
>>> That seems to be micromanaging PCI bus slot assignment, which isn't
>>> changed by this patch.  The cards don't move around, nor does the
>>> association between cards and Linux eth0/eth1.  What changes is which
>>> virtual LAN each virtual ethernet card is plugged into.  (The virtual
>>> cat5 cable coming out of the card moves to a different switch.)
>> 
>> I didn't mean to tell you "try using -device to juggle PCI addresses".
>> I meant to steer you away from QEMU VLANs, to find out whether they're a
>> factor in your problem.  Possible, because non-VLAN uses a few different
>> code paths in QEMU.  Sorry if I was too terse.
>> 
>> In general, my advice is stay away from QEMU VLANs.
>
> Ok, now I'm confused.

Sorry :)

> Before this, I wasn't using them.  Now I am.  What's the reason for
> avoiding them?  (Also, I didn't see a way in -device to specify a
> network transport, just cards and their properties.  Quite possibly I
> missed it...)

A virtual network device has a host and a guest part.

The modern way to configure host and guest part is -netdev for host and
-device for guest part.  Example:

    -netdev user,id=net0 -device e1000,netdev=net0

    (qemu) info network
    Devices not on any VLAN:
      net0: net=10.0.2.0, restricted=n peer=e1000.0
      e1000.0: model=e1000,macaddr=52:54:00:12:34:56 peer=net0

One guest part connected to one host part.

For backward compatibility, the older way still works:

    -netdev user,id=net0 -net nic,model=rtl8139,netdev=net0

There's also a completely different kind of host part, called VLAN (not
related to 802.1q VLANs).  Example:

    -net user,vlan=0 -device rtl8139,vlan=0

    (qemu) info network
    VLAN 0 devices:
      user.0: net=10.0.2.0, restricted=n
      rtl8139.0: model=rtl8139,macaddr=52:54:00:12:34:56

You can connect multiple guest parts to it.  This is rarely needed, and
when you need it, you're often better off with a real ethernet bridge,
see brctl(8).  Best stick to -netdev.

For backward compatibility, the older way still works:

    -net user,vlan=0 -net nic,model=rtl8139,vlan=0

You can omit vlan=0, because it's the default.

>>> (The fix was to tag everything with vlan arguments and manually manage
>>> the association.)
>> 
>> Glad you got your problem solved.
>
> Solved, yes.  Understood... less so than I thought, apparently?

Hope the above helps a bit.

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Rob Landley]

On 05/13/2011 07:19 AM, Markus Armbruster wrote:
> Rob Landley <rob@landley.net> writes:
> 
>> On 05/13/2011 01:54 AM, Markus Armbruster wrote:
>>> Rob Landley <rob@landley.net> writes:
>>>
>>>> On 05/12/2011 09:10 AM, Markus Armbruster wrote:
>>>>> Rob Landley <rob@landley.net> writes:
>>>>>
>>>>>> In 1.14.0, if I did this:
>>>>>>
>>>>>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>>>>>
>>>>>> Then the first nic would be -net user, and the second nic would be -net
>>>>>> tun.    In current -git, -net user attaches to the second interface and
>>>>>> -net tun attaches to the first, I.E. the order is reversed.
>>>>>>
>>>>>> Either way the first -nic becomes eth0 in Linux and the second becomes
>>>>>> eth1 (I can manually assign mac addresses in order to confirm which is
>>>>>> which), but eth0 used to be the -net user interface and now eth1 is the
>>>>>> -net user interface.
>>>>>>
>>>>>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>>>>>> changed the behavior, and I can't find _documentation_ on having
>>>>>> multiple interfaces transports hooked up to the same qemu instance
>>>>>> anyway.  (It used to work, but possibly that was an accident?)
>>>>>>
>>>>>> Any ideas?
>>>>>
>>>>> Does it happen with -device and -netdev as well?
>>>>>
>>>>> See docs/qdev-device-use.txt for how to go from -net to -device.
>>>>
>>>> Read read read...
>>>>
>>>> That seems to be micromanaging PCI bus slot assignment, which isn't
>>>> changed by this patch.  The cards don't move around, nor does the
>>>> association between cards and Linux eth0/eth1.  What changes is which
>>>> virtual LAN each virtual ethernet card is plugged into.  (The virtual
>>>> cat5 cable coming out of the card moves to a different switch.)
>>>
>>> I didn't mean to tell you "try using -device to juggle PCI addresses".
>>> I meant to steer you away from QEMU VLANs, to find out whether they're a
>>> factor in your problem.  Possible, because non-VLAN uses a few different
>>> code paths in QEMU.  Sorry if I was too terse.
>>>
>>> In general, my advice is stay away from QEMU VLANs.
>>
>> Ok, now I'm confused.
> 
> Sorry :)
> 
>> Before this, I wasn't using them.  Now I am.  What's the reason for
>> avoiding them?  (Also, I didn't see a way in -device to specify a
>> network transport, just cards and their properties.  Quite possibly I
>> missed it...)
> 
> A virtual network device has a host and a guest part.
> 
> The modern way to configure host and guest part is -netdev for host and
> -device for guest part.  Example:
> 
>     -netdev user,id=net0 -device e1000,netdev=net0
> 
>     (qemu) info network
>     Devices not on any VLAN:
>       net0: net=10.0.2.0, restricted=n peer=e1000.0
>       e1000.0: model=e1000,macaddr=52:54:00:12:34:56 peer=net0
> 
> One guest part connected to one host part.

Well, not calling it a "VLAN" which already has at least two other
meanings is less confusing, so I'll give it a shot.  (I'm trying to
update my container documentation at http://landley.net/lxc so I don't
just want to get this to work but I want to be able to EXPLAIN it.)

The name "netdev" is not self-explanatory.  FYI.  Let's see...

Doesn't like "macaddr".  It got gratuitously renamed to "mac=".  Eh,
remove that and try again.

qemu-system-x86_64: -netdev id=lan0,user,hostfwd=tcp:127.0.0.1:9876-:22:
Parameter 'type' is missing

Um, no it isn't?  Ok, that argument magically has to be first...

So my third wild guess at redoing my command line worked:

qemu-system-x86_64 -m 512 -kernel ~/linux/linux/arch/x86/boot/bzImage
-no-reboot -hda squeeze.ext3 -append "root=/dev/hda rw" -netdev
user,id=lan0,hostfwd=tcp:127.0.0.1:9876-:22 -device e1000,netdev=lan0
-netdev tap,id=lan1,ifname=kvm0,script=no,downscript=no -device
e1000,netdev=lan1
Warning: more nics requested than this machine supports; some have been
ignored

I have no idea where that warning came from, but I got both my
interfaces so it's at least working.

> For backward compatibility, the older way still works:
> 
>     -netdev user,id=net0 -net nic,model=rtl8139,netdev=net0
> 
> There's also a completely different kind of host part, called VLAN (not
> related to 802.1q VLANs).  Example:
> 
>     -net user,vlan=0 -device rtl8139,vlan=0
> 
>     (qemu) info network
>     VLAN 0 devices:
>       user.0: net=10.0.2.0, restricted=n
>       rtl8139.0: model=rtl8139,macaddr=52:54:00:12:34:56
> 
> You can connect multiple guest parts to it.  This is rarely needed, and
> when you need it, you're often better off with a real ethernet bridge,
> see brctl(8).  Best stick to -netdev.

Um, are you saying I CAN'T connect multiple guest parts to -netdev user?

> For backward compatibility, the older way still works:
> 
>     -net user,vlan=0 -net nic,model=rtl8139,vlan=0
> 
> You can omit vlan=0, because it's the default.

Is the old mechanism ever going to get removed?  Does the new one
actually do anything the old one couldn't?  Why are there two redundant
mechanisms here anyway?

>>>> (The fix was to tag everything with vlan arguments and manually manage
>>>> the association.)
>>>
>>> Glad you got your problem solved.
>>
>> Solved, yes.  Understood... less so than I thought, apparently?
> 
> Hope the above helps a bit.

I'm now confused about different things, which is progress.  (Possibly
lateral progress, but eh, still new territory.)

Rob

Re: [Qemu-devel] -net interface association behavior change in current -git.

[Markus Armbruster]

Rob Landley <rob@landley.net> writes:

> On 05/13/2011 07:19 AM, Markus Armbruster wrote:
>> Rob Landley <rob@landley.net> writes:
>> 
>>> On 05/13/2011 01:54 AM, Markus Armbruster wrote:
>>>> Rob Landley <rob@landley.net> writes:
>>>>
>>>>> On 05/12/2011 09:10 AM, Markus Armbruster wrote:
>>>>>> Rob Landley <rob@landley.net> writes:
>>>>>>
>>>>>>> In 1.14.0, if I did this:
>>>>>>>
>>>>>>>   qemu -net nic,blah -net user -net nic,blah -net tun,blah
>>>>>>>
>>>>>>> Then the first nic would be -net user, and the second nic would be -net
>>>>>>> tun.    In current -git, -net user attaches to the second interface and
>>>>>>> -net tun attaches to the first, I.E. the order is reversed.
>>>>>>>
>>>>>>> Either way the first -nic becomes eth0 in Linux and the second becomes
>>>>>>> eth1 (I can manually assign mac addresses in order to confirm which is
>>>>>>> which), but eth0 used to be the -net user interface and now eth1 is the
>>>>>>> -net user interface.
>>>>>>>
>>>>>>> I bisected this to commit 60c07d933c66c4b30a83b but I don't know why it
>>>>>>> changed the behavior, and I can't find _documentation_ on having
>>>>>>> multiple interfaces transports hooked up to the same qemu instance
>>>>>>> anyway.  (It used to work, but possibly that was an accident?)
>>>>>>>
>>>>>>> Any ideas?
>>>>>>
>>>>>> Does it happen with -device and -netdev as well?
>>>>>>
>>>>>> See docs/qdev-device-use.txt for how to go from -net to -device.
>>>>>
>>>>> Read read read...
>>>>>
>>>>> That seems to be micromanaging PCI bus slot assignment, which isn't
>>>>> changed by this patch.  The cards don't move around, nor does the
>>>>> association between cards and Linux eth0/eth1.  What changes is which
>>>>> virtual LAN each virtual ethernet card is plugged into.  (The virtual
>>>>> cat5 cable coming out of the card moves to a different switch.)
>>>>
>>>> I didn't mean to tell you "try using -device to juggle PCI addresses".
>>>> I meant to steer you away from QEMU VLANs, to find out whether they're a
>>>> factor in your problem.  Possible, because non-VLAN uses a few different
>>>> code paths in QEMU.  Sorry if I was too terse.
>>>>
>>>> In general, my advice is stay away from QEMU VLANs.
>>>
>>> Ok, now I'm confused.
>> 
>> Sorry :)
>> 
>>> Before this, I wasn't using them.  Now I am.  What's the reason for
>>> avoiding them?  (Also, I didn't see a way in -device to specify a
>>> network transport, just cards and their properties.  Quite possibly I
>>> missed it...)
>> 
>> A virtual network device has a host and a guest part.
>> 
>> The modern way to configure host and guest part is -netdev for host and
>> -device for guest part.  Example:
>> 
>>     -netdev user,id=net0 -device e1000,netdev=net0
>> 
>>     (qemu) info network
>>     Devices not on any VLAN:
>>       net0: net=10.0.2.0, restricted=n peer=e1000.0
>>       e1000.0: model=e1000,macaddr=52:54:00:12:34:56 peer=net0
>> 
>> One guest part connected to one host part.
>
> Well, not calling it a "VLAN" which already has at least two other
> meanings is less confusing, so I'll give it a shot.  (I'm trying to
> update my container documentation at http://landley.net/lxc so I don't
> just want to get this to work but I want to be able to EXPLAIN it.)
>
> The name "netdev" is not self-explanatory.  FYI.  Let's see...

I guess it was chosen for similarity to -chardev.  Both create a device
host part.

> Doesn't like "macaddr".  It got gratuitously renamed to "mac=".  Eh,
> remove that and try again.

Documented in docs/qdev-device-use.txt.

> qemu-system-x86_64: -netdev id=lan0,user,hostfwd=tcp:127.0.0.1:9876-:22:
> Parameter 'type' is missing
>
> Um, no it isn't?  Ok, that argument magically has to be first...

"user" is shorthand for "type=user".  The short form is only accepted
when it comes first.  Many option arguments of the form NAME=VALUE...
work that way, including -net.

> So my third wild guess at redoing my command line worked:
>
> qemu-system-x86_64 -m 512 -kernel ~/linux/linux/arch/x86/boot/bzImage
> -no-reboot -hda squeeze.ext3 -append "root=/dev/hda rw" -netdev
> user,id=lan0,hostfwd=tcp:127.0.0.1:9876-:22 -device e1000,netdev=lan0
> -netdev tap,id=lan1,ifname=kvm0,script=no,downscript=no -device
> e1000,netdev=lan1
> Warning: more nics requested than this machine supports; some have been
> ignored
>
> I have no idea where that warning came from, but I got both my
> interfaces so it's at least working.

Relatively recent regression, suspecting commit f68b9d67.  Ignore the
warning for now.

>> For backward compatibility, the older way still works:
>> 
>>     -netdev user,id=net0 -net nic,model=rtl8139,netdev=net0
>> 
>> There's also a completely different kind of host part, called VLAN (not
>> related to 802.1q VLANs).  Example:
>> 
>>     -net user,vlan=0 -device rtl8139,vlan=0
>> 
>>     (qemu) info network
>>     VLAN 0 devices:
>>       user.0: net=10.0.2.0, restricted=n
>>       rtl8139.0: model=rtl8139,macaddr=52:54:00:12:34:56
>> 
>> You can connect multiple guest parts to it.  This is rarely needed, and
>> when you need it, you're often better off with a real ethernet bridge,
>> see brctl(8).  Best stick to -netdev.
>
> Um, are you saying I CAN'T connect multiple guest parts to -netdev user?

Yes.

If you want multiple guest NICs on the same ethernet, use a bridge.

>> For backward compatibility, the older way still works:
>> 
>>     -net user,vlan=0 -net nic,model=rtl8139,vlan=0
>> 
>> You can omit vlan=0, because it's the default.
>
> Is the old mechanism ever going to get removed?  Does the new one
> actually do anything the old one couldn't?  Why are there two redundant
> mechanisms here anyway?

Important optimizations like GSO or vhost-net aren't feasible with
-net's VLANs.  That's why -netdev got added[*].  VLANs are still around
for backward compatibility.  I hope we can get of them eventually.

>>>>> (The fix was to tag everything with vlan arguments and manually manage
>>>>> the association.)
>>>>
>>>> Glad you got your problem solved.
>>>
>>> Solved, yes.  Understood... less so than I thought, apparently?
>> 
>> Hope the above helps a bit.
>
> I'm now confused about different things, which is progress.  (Possibly
> lateral progress, but eh, still new territory.)

Let me know whether I've found more ways to confuse you.


[*] http://lists.gnu.org/archive/html/qemu-devel/2009-10/msg00858.html