[Qemu-devel] socket,mcast looping back frames -> IPv6 broken

[Qemu-devel] socket,mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Hello,

The reason why IPv6 does not work when using -net socket,mcast=foo is
that since qemu explicitly sets IP_MULTICAST_LOOP to 1, it receives its
own frames.  When the IPv6 stack performs duplicate addresse detection
(DAD) through a multicasted announce, it receives its own announcement,
and thus believes another machine has the same address.

AIUI, on a real physical network network boards do not receive the
multicasts they send, so the issue does not happen. Perhaps some boards
even filter out any frame with its own MAC as source, eliminating the
issue altogether.

As a result, we should probably perform this kind of dropping, I'm just
wondering at which level that would be preferable.

- We could do that in qemu_send_packet_async_with_flags, thus fixing the
issue for all kinds of frame transporters.

- Or we could do that for the only case that matters, mcast, in
net_socket_send_dgram (which will thus do it for the unicast udp case
too).

What do people think about it?

Samuel

Re: [Qemu-devel] socket,mcast looping back frames -> IPv6 broken

[Stefan Hajnoczi]

On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> The reason why IPv6 does not work when using -net socket,mcast=foo is
> that since qemu explicitly sets IP_MULTICAST_LOOP to 1, it receives its
> own frames.  When the IPv6 stack performs duplicate addresse detection
> (DAD) through a multicasted announce, it receives its own announcement,
> and thus believes another machine has the same address.

The reason for IP_MULTICAST_LOOP is to allow QEMU processes running on
the same host to communicate with each other.

> AIUI, on a real physical network network boards do not receive the
> multicasts they send, so the issue does not happen. Perhaps some boards
> even filter out any frame with its own MAC as source, eliminating the
> issue altogether.
> 
> As a result, we should probably perform this kind of dropping, I'm just
> wondering at which level that would be preferable.
> 
> - We could do that in qemu_send_packet_async_with_flags, thus fixing the
> issue for all kinds of frame transporters.
> 
> - Or we could do that for the only case that matters, mcast, in
> net_socket_send_dgram (which will thus do it for the unicast udp case
> too).
> 
> What do people think about it?

We should fix the layer that introduces the problem.  Therefore I think
the fix needs to be net/socket.c.

Unfortunately net/socket.c does not have the concept of a link-layer
address, so we cannot easily filter out multicast packets coming from
our NIC's address.

Are you aware of a way to filter out just the packets sent by *this*
process?

Stefan

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> > The reason why IPv6 does not work when using -net socket,mcast=foo is
> > that since qemu explicitly sets IP_MULTICAST_LOOP to 1, it receives its
> > own frames.  When the IPv6 stack performs duplicate addresse detection
> > (DAD) through a multicasted announce, it receives its own announcement,
> > and thus believes another machine has the same address.
> 
> The reason for IP_MULTICAST_LOOP is to allow QEMU processes running on
> the same host to communicate with each other.

Sure, I've seen the comment, I wasn't suggesting to drop that :)

> > AIUI, on a real physical network network boards do not receive the
> > multicasts they send, so the issue does not happen. Perhaps some boards
> > even filter out any frame with its own MAC as source, eliminating the
> > issue altogether.
> > 
> > As a result, we should probably perform this kind of dropping, I'm just
> > wondering at which level that would be preferable.
> > 
> > - We could do that in qemu_send_packet_async_with_flags, thus fixing the
> > issue for all kinds of frame transporters.
> > 
> > - Or we could do that for the only case that matters, mcast, in
> > net_socket_send_dgram (which will thus do it for the unicast udp case
> > too).
> > 
> > What do people think about it?
> 
> We should fix the layer that introduces the problem.  Therefore I think
> the fix needs to be net/socket.c.

Ok.

> Unfortunately net/socket.c does not have the concept of a link-layer
> address, so we cannot easily filter out multicast packets coming from
> our NIC's address.
> 
> Are you aware of a way to filter out just the packets sent by *this*
> process?

I haven't seen any in the Linux source code.  One thing that should
work, however, is to use recvfrom, and drop whatever comes from our
sockname.

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Stefan Hajnoczi]

On Wed, Mar 06, 2013 at 02:15:25PM +0100, Samuel Thibault wrote:
> Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> > Unfortunately net/socket.c does not have the concept of a link-layer
> > address, so we cannot easily filter out multicast packets coming from
> > our NIC's address.
> > 
> > Are you aware of a way to filter out just the packets sent by *this*
> > process?
> 
> I haven't seen any in the Linux source code.  One thing that should
> work, however, is to use recvfrom, and drop whatever comes from our
> sockname.

Sounds like a plan :).

Stefan

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Stefan Hajnoczi, le Thu 07 Mar 2013 10:38:26 +0100, a écrit :
> On Wed, Mar 06, 2013 at 02:15:25PM +0100, Samuel Thibault wrote:
> > Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > > On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> > > Unfortunately net/socket.c does not have the concept of a link-layer
> > > address, so we cannot easily filter out multicast packets coming from
> > > our NIC's address.
> > > 
> > > Are you aware of a way to filter out just the packets sent by *this*
> > > process?
> > 
> > I haven't seen any in the Linux source code.  One thing that should
> > work, however, is to use recvfrom, and drop whatever comes from our
> > sockname.
> 
> Sounds like a plan :).

Except that the sockname is the multicast address itself...  I'll have a
closer look.

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Stefan Hajnoczi]

On Thu, Mar 07, 2013 at 06:14:28PM +0100, Samuel Thibault wrote:
> Stefan Hajnoczi, le Thu 07 Mar 2013 10:38:26 +0100, a écrit :
> > On Wed, Mar 06, 2013 at 02:15:25PM +0100, Samuel Thibault wrote:
> > > Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > > > On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> > > > Unfortunately net/socket.c does not have the concept of a link-layer
> > > > address, so we cannot easily filter out multicast packets coming from
> > > > our NIC's address.
> > > > 
> > > > Are you aware of a way to filter out just the packets sent by *this*
> > > > process?
> > > 
> > > I haven't seen any in the Linux source code.  One thing that should
> > > work, however, is to use recvfrom, and drop whatever comes from our
> > > sockname.
> > 
> > Sounds like a plan :).
> 
> Except that the sockname is the multicast address itself...  I'll have a
> closer look.

What about the port number?

Stefan

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Stefan Hajnoczi, le Fri 08 Mar 2013 09:43:18 +0100, a écrit :
> On Thu, Mar 07, 2013 at 06:14:28PM +0100, Samuel Thibault wrote:
> > Stefan Hajnoczi, le Thu 07 Mar 2013 10:38:26 +0100, a écrit :
> > > On Wed, Mar 06, 2013 at 02:15:25PM +0100, Samuel Thibault wrote:
> > > > Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > > > > On Tue, Mar 05, 2013 at 05:35:10PM +0100, Samuel Thibault wrote:
> > > > > Unfortunately net/socket.c does not have the concept of a link-layer
> > > > > address, so we cannot easily filter out multicast packets coming from
> > > > > our NIC's address.
> > > > > 
> > > > > Are you aware of a way to filter out just the packets sent by *this*
> > > > > process?
> > > > 
> > > > I haven't seen any in the Linux source code.  One thing that should
> > > > work, however, is to use recvfrom, and drop whatever comes from our
> > > > sockname.
> > > 
> > > Sounds like a plan :).
> > 
> > Except that the sockname is the multicast address itself...  I'll have a
> > closer look.
> 
> What about the port number?

It needs to be the same.

There does exist some unique address, which is returned by recvfrom,
I'll have a look at how to get access to it.

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Samuel Thibault, le Fri 08 Mar 2013 10:08:55 +0100, a écrit :
> There does exist some unique address, which is returned by recvfrom,
> I'll have a look at how to get access to it.

Ah, no, it's not unique... It's just the host IP address and the same
port as the multicast address, so it'll be the same for all qemus on the
same host.  I've checked how Linux bounces the datagram, it's through
the loopback interface, and thus dispatched over all listeners without
distinction. I don't see any way to get the information that the packet
comes from us, except using the ethernet content.

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Mike Lovell]

On 03/08/2013 05:47 AM, Samuel Thibault wrote:
> Samuel Thibault, le Fri 08 Mar 2013 10:08:55 +0100, a écrit :
>> There does exist some unique address, which is returned by recvfrom,
>> I'll have a look at how to get access to it.
> Ah, no, it's not unique... It's just the host IP address and the same
> port as the multicast address, so it'll be the same for all qemus on the
> same host.  I've checked how Linux bounces the datagram, it's through
> the loopback interface, and thus dispatched over all listeners without
> distinction. I don't see any way to get the information that the packet
> comes from us, except using the ethernet content.

this is actually a problem that i dealt with when i was building the 
switched multicast backend i did last year. ( 
http://lists.nongnu.org/archive/html/qemu-devel/2012-06/msg04082.html )

one solution is to actually use two sockets. one that is bound to the 
multicast address, which receives the multicast packets, and another 
that is just bound to any ephemeral udp port, which is used for sending 
packets. when a packet is to be sent out to the multicast address, call 
sendto on the ephemeral socket with a destination of the multicast 
address. then, using recv_from on the multicast socket, packets being 
received can be compared to the local ephemeral address. if the address 
on the recv_from matches the address for the local ephemeral socket, the 
packet can just be dropped. no inspection of the packet being passed 
around is needed in this case.

if the group is interested is a solution like this, i can probably make 
some time over the next couple days to cook up a patch. thoughts?

mike

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Mike Lovell, le Mon 01 Apr 2013 00:35:03 -0600, a écrit :
> On 03/08/2013 05:47 AM, Samuel Thibault wrote:
> >Samuel Thibault, le Fri 08 Mar 2013 10:08:55 +0100, a écrit :
> >>There does exist some unique address, which is returned by recvfrom,
> >>I'll have a look at how to get access to it.
> >Ah, no, it's not unique... It's just the host IP address and the same
> >port as the multicast address, so it'll be the same for all qemus on the
> >same host.  I've checked how Linux bounces the datagram, it's through
> >the loopback interface, and thus dispatched over all listeners without
> >distinction. I don't see any way to get the information that the packet
> >comes from us, except using the ethernet content.
> 
> one solution is to actually use two sockets. one that is bound to the
> multicast address, which receives the multicast packets, and another that is
> just bound to any ephemeral udp port, which is used for sending packets.

Mmm, that should work indeed.

> if the group is interested is a solution like this, i can probably make some
> time over the next couple days to cook up a patch. thoughts?

Yes, please.

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Stefan Hajnoczi]

On Mon, Apr 01, 2013 at 11:32:29AM +0200, Samuel Thibault wrote:
> Mike Lovell, le Mon 01 Apr 2013 00:35:03 -0600, a écrit :
> > On 03/08/2013 05:47 AM, Samuel Thibault wrote:
> > >Samuel Thibault, le Fri 08 Mar 2013 10:08:55 +0100, a écrit :
> > >>There does exist some unique address, which is returned by recvfrom,
> > >>I'll have a look at how to get access to it.
> > >Ah, no, it's not unique... It's just the host IP address and the same
> > >port as the multicast address, so it'll be the same for all qemus on the
> > >same host.  I've checked how Linux bounces the datagram, it's through
> > >the loopback interface, and thus dispatched over all listeners without
> > >distinction. I don't see any way to get the information that the packet
> > >comes from us, except using the ethernet content.
> > 
> > one solution is to actually use two sockets. one that is bound to the
> > multicast address, which receives the multicast packets, and another that is
> > just bound to any ephemeral udp port, which is used for sending packets.
> 
> Mmm, that should work indeed.
> 
> > if the group is interested is a solution like this, i can probably make some
> > time over the next couple days to cook up a patch. thoughts?
> 
> Yes, please.

Yeah, great.  It's a real problem that people are hitting so it would be
worth fixing.

Stefan

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Samuel Thibault]

Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > What do people think about it?
> 
> We should fix the layer that introduces the problem.  Therefore I think
> the fix needs to be net/socket.c.
> 
> Unfortunately net/socket.c does not have the concept of a link-layer
> address, so we cannot easily filter out multicast packets coming from
> our NIC's address.
> 
> Are you aware of a way to filter out just the packets sent by *this*
> process?

So in the end I couldn't find any way (even Linuxish) to distinguish
packets coming from self or from other qemus running on the same host,
event with recvmsg etc.

So, it seems if we want to fix the issue we'll have to filter by source
MAC address.  Since the guest can actually change its own MACs, we'd
have to learn them on the fly.  We can also implement some aging on the
entries in case the user migrates a MAC from a VM to another.

Otherwise we may just document that one has to disable Duplicate Address
Detection to get IPv6 working :/

Samuel

Re: [Qemu-devel] [PER] Re: socket, mcast looping back frames -> IPv6 broken

[Stefan Hajnoczi]

On Sun, Mar 10, 2013 at 09:01:52PM +0100, Samuel Thibault wrote:
> Stefan Hajnoczi, le Wed 06 Mar 2013 13:29:37 +0100, a écrit :
> > > What do people think about it?
> > 
> > We should fix the layer that introduces the problem.  Therefore I think
> > the fix needs to be net/socket.c.
> > 
> > Unfortunately net/socket.c does not have the concept of a link-layer
> > address, so we cannot easily filter out multicast packets coming from
> > our NIC's address.
> > 
> > Are you aware of a way to filter out just the packets sent by *this*
> > process?
> 
> So in the end I couldn't find any way (even Linuxish) to distinguish
> packets coming from self or from other qemus running on the same host,
> event with recvmsg etc.
> 
> So, it seems if we want to fix the issue we'll have to filter by source
> MAC address.  Since the guest can actually change its own MACs, we'd
> have to learn them on the fly.  We can also implement some aging on the
> entries in case the user migrates a MAC from a VM to another.

Yes, the guest may use multiple MACs at the same time (e.g. nested
virtualization).

> Otherwise we may just document that one has to disable Duplicate Address
> Detection to get IPv6 working :/

Seems like this might be the only way for now.

Stefan

[Qemu-devel] [PATCH] Document mcast+ipv6 (Was: Re: socket, mcast looping back frames -> IPv6 broken)

[Samuel Thibault]

Stefan Hajnoczi, le Mon 11 Mar 2013 09:36:14 +0100, a écrit :
> > Otherwise we may just document that one has to disable Duplicate Address
> > Detection to get IPv6 working :/
> 
> Seems like this might be the only way for now.

Here is a patch

Samuel


Document how to get IPv6 working with mcast socket

Document that IPv6 can not work in mcast mode unless disabling DAD in
the guest.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

diff --git a/qemu-options.hx b/qemu-options.hx
index c40ba55..9074a24 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1683,6 +1683,12 @@ mcast support is compatible with User Mode Linux (argument @option{eth@var{N}=mc
 @url{http://user-mode-linux.sf.net}.
 @item
 Use @option{fd=h} to specify an already opened UDP multicast socket.
+@item
+Since with multicast diffusion the guest will receive its own frames, its IPv6
+Duplicate Address Detection (DAD) will erroneously detect a duplicate, thus
+preventing IPv6 from working.  One has to disable DAD to get IPv6 working, for
+instance on Linux by setting the sysctl @code{net.ipv6.conf.all.accept_dad} to
+0.
 @end enumerate
 
 Example:

Re: [Qemu-devel] [PATCH] Document mcast+ipv6 (Was: Re: socket, mcast looping back frames -> IPv6 broken)

[Stefan Hajnoczi]

On Mon, Apr 01, 2013 at 02:12:51AM +0200, Samuel Thibault wrote:
> Stefan Hajnoczi, le Mon 11 Mar 2013 09:36:14 +0100, a écrit :
> > > Otherwise we may just document that one has to disable Duplicate Address
> > > Detection to get IPv6 working :/
> > 
> > Seems like this might be the only way for now.
> 
> Here is a patch

Let's check out the 2 socket patch first.  If that doesn't work out I'll
merge the documentation warning.

Stefan

Re: [Qemu-devel] [PATCH] Document mcast+ipv6 (Was: Re: socket, mcast looping back frames -> IPv6 broken)

[Samuel Thibault]

Stefan Hajnoczi, le Mon 08 Apr 2013 13:13:19 +0200, a écrit :
> On Mon, Apr 01, 2013 at 02:12:51AM +0200, Samuel Thibault wrote:
> > Stefan Hajnoczi, le Mon 11 Mar 2013 09:36:14 +0100, a écrit :
> > > > Otherwise we may just document that one has to disable Duplicate Address
> > > > Detection to get IPv6 working :/
> > > 
> > > Seems like this might be the only way for now.
> > 
> > Here is a patch
> 
> Let's check out the 2 socket patch first.  If that doesn't work out I'll
> merge the documentation warning.

Sure.

Samuel