Network questions

Network questions

[Xiaofang Chen]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

Hi,

I wonder if there is some way to control the network traffic of new domains(other than Dom0) in Xen.
I mean although we can allocate an external IP address for each new domain, and the new domains can
access the WAN as they wish, there is only one physical network card.
So if each new domain has a lot of network traffic, the interrupts will keep the whole system very busy. 

I know we can control the CPU time, memory, and disk for new domains.
So can we do something similar to that kind of scheduling for network?
Or can someone tell me which part of the source code relates to this?

I know little about network. So if my understanding is wrong, please point it out.
Thanks.

Xiaofang


[-- Attachment #2: Type: text/html, Size: 1222 bytes --]

Re: Network questions

[Ian Pratt]

> I wonder if there is some way to control the network traffic of new domains(other than Dom0) in Xen.
> I mean although we can allocate an external IP address for each new domain, and the new domains can
> access the WAN as they wish, there is only one physical network card.
> So if each new domain has a lot of network traffic, the interrupts will keep the whole system very busy. 
> 
> I know we can control the CPU time, memory, and disk for new domains.
> So can we do something similar to that kind of scheduling for network?
> Or can someone tell me which part of the source code relates to this?

See "xc_dom_control.py vif_setsched". You can use it to set rate
limit for a domain's transmit bandwidth in bytes per microsecond.

As regards packets received, Xen is careful to do "early discard"
on incoming packets for domains that are unable to keep up with
packet arrivals, so only minimal resources are wasted. 

Under some deployment scenarios I guess it might be useful to be
able to rate limit incoming traffic as well as out going. We
currently don't do this, but it might be possible to configure
Linux tc to do this under the NGIO model in the unstable tree.

Ian


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click

Network questions

[Xiaofang Chen]

[-- Attachment #1: Type: text/plain, Size: 550 bytes --]

Hi,

Can someone tell me what is the functionality of "backend" in XenoLinux?

Take the network for example, it seems that all the virtual network interfaces corresponding to 
guest OS's are created in Xen. And on the receiving side, interrupt VIRQ_NET is raised from 
Xen upward to XenoLinux. However, it seems that this kind of interrupts are not processed by 
"backend". 

So what does "backend" do for networking? For example, functions like 'netif_ctrlif_rx()', 
'net_rx_action()', 'netif_be_start_xmit()', etc. 

Thanks!
Xiaofang

[-- Attachment #2: Type: text/html, Size: 2916 bytes --]

Network questions

[Xiaofang Chen]

Hi,

Can someone tell me what is the functionality of "backend" in XenoLinux?
It seems that virtual network interfaces corresponding to guest OS's
are created in Xen, including 'net_get_target_vif()', etc. And the virtual
interrupt VIRQ_NET raised by Xen is handled by other parts than "backend"
in XenoLinux.

So what is the rough flow of network in the system? Take the receiving
side for example, how does Xen cooperate with the "backend"?

Thanks for your help!
Xiaofang



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

Re: Network questions

[Mark Williamson]

Are you using Xen 1.2 or unstable?  There have been big changes to networking 
in the unstable tree.  In my previous e-mail I was describing the current 
state of the unstable tree.

The old IO world: (as used by Xen 1.2 and as described in the SOSP paper)
* Device drivers for physical network devices are in Xen.
* All domains (including 0) use a "virtual" network driver, which talks to 
Xen.  Xen then routes packets between domains or out onto the physical network.
* VIRQ_NET is the IRQ associated with a virtual network device.  Xen uses it 
to signal a domain when it has been sent data by another domain or from the 
physical network.
* There's no "backend", just Xen.

The new IO world: (as used by the Unstable tree and the forthcoming 2.0 
release)
* Xen does not contain device drivers anymore.
* Instead, the device drivers for physical devices run in domains.  Usually, 
all the drivers for your hardware run in dom0.
* A domain that's controlling physical devices can export virtual devices to 
other domains.  The driver which exports a device to other domains is called 
the "backend".
* The driver which imports a virtual device from the backend is called the 
"frontend".  This driver runs in unprivileged domains to give them their 
virtual devices.  It's a lot like the old (1.2) virtual network interface 
driver except that it talks to another domain instead of to Xen.
* VIRQ_NET does not exist anymore.  Instead, the backend driver (in the domain 
that controls the hardware) can send an interrupt to the frontend driver (in 
the domain using the virtual devices) using the event channel mechanism I 
described in my last e-mail.

Does that make sense?

Mark



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

Re: Network questions

[Xiaofang Chen]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

Sorry for the spam.
This message is because of our mail server problem.

Xiaofang

  ----- Original Message ----- 
  From: Xiaofang Chen 
  To: xen-devel@lists.sourceforge.net 
  Sent: Monday, June 28, 2004 9:50 PM
  Subject: [Xen-devel] Network questions


  Hi,

  Can someone tell me what is the functionality of "backend" in XenoLinux?

  Take the network for example, it seems that all the virtual network interfaces corresponding to 
  guest OS's are created in Xen. And on the receiving side, interrupt VIRQ_NET is raised from 
  Xen upward to XenoLinux. However, it seems that this kind of interrupts are not processed by 
  "backend". 

  So what does "backend" do for networking? For example, functions like 'netif_ctrlif_rx()', 
  'net_rx_action()', 'netif_be_start_xmit()', etc. 

  Thanks!
  Xiaofang

[-- Attachment #2: Type: text/html, Size: 4081 bytes --]

Re: Network questions

[Xiaofang Chen]

Thanks a lot for the explainations!
The idea seems clear to me now.

I guess the code I was reading was a "middle" version between the ealy
unstable and latest unstable.
I got such a version by using "bk pull" based on the early unstable version,
so that the code included both VIRQ_NET and "backend".

For the latest unstable version, I am still not clear on how the "backend"
controls the physical network device.
For example, if I use a 3Com 3c59x.c driver, when packets come,  after
'vortex_rx()' is called, '/net/core/dev.c: netif_rx()' will be called.
Along this way up, it seems that no functions in "backend" will be called.

Also in /arch/xen/drivers/netif/backend/control.c: netif_ctrlif_init(),
after the cmsg(CMSG_NETIF_BE_DRIVER_STATUS_CHANGED + NETIF_DRIVER_STATUS_UP)
is put into ctrl_if tx_ring, seems no corresponding handlers are called
because of id=0xFF. Confused here.

Am I missing something?

Thanks.
Xiaofang





> Are you using Xen 1.2 or unstable?  There have been big changes to
networking
> in the unstable tree.  In my previous e-mail I was describing the current
> state of the unstable tree.
>
> The old IO world: (as used by Xen 1.2 and as described in the SOSP paper)
> * Device drivers for physical network devices are in Xen.
> * All domains (including 0) use a "virtual" network driver, which talks to
> Xen.  Xen then routes packets between domains or out onto the physical
network.
> * VIRQ_NET is the IRQ associated with a virtual network device.  Xen uses
it
> to signal a domain when it has been sent data by another domain or from
the
> physical network.
> * There's no "backend", just Xen.
>
> The new IO world: (as used by the Unstable tree and the forthcoming 2.0
> release)
> * Xen does not contain device drivers anymore.
> * Instead, the device drivers for physical devices run in domains.
Usually,
> all the drivers for your hardware run in dom0.
> * A domain that's controlling physical devices can export virtual devices
to
> other domains.  The driver which exports a device to other domains is
called
> the "backend".
> * The driver which imports a virtual device from the backend is called the
> "frontend".  This driver runs in unprivileged domains to give them their
> virtual devices.  It's a lot like the old (1.2) virtual network interface
> driver except that it talks to another domain instead of to Xen.
> * VIRQ_NET does not exist anymore.  Instead, the backend driver (in the
domain
> that controls the hardware) can send an interrupt to the frontend driver
(in
> the domain using the virtual devices) using the event channel mechanism I
> described in my last e-mail.
>
> Does that make sense?
>
> Mark
>
>




-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

Re: Network questions

[Ian Pratt]

> I guess the code I was reading was a "middle" version between the ealy
> unstable and latest unstable.
> I got such a version by using "bk pull" based on the early unstable version,
> so that the code included both VIRQ_NET and "backend".
> 
> For the latest unstable version, I am still not clear on how the "backend"
> controls the physical network device.
> For example, if I use a 3Com 3c59x.c driver, when packets come,  after
> 'vortex_rx()' is called, '/net/core/dev.c: netif_rx()' will be called.
> Along this way up, it seems that no functions in "backend" will be called.

There is no net/core/dev.c in current Xen. I think your src repo
is in a broken state. Reclone the repository.

The backend netif driver appears in domain 0 Linux as just
another network interface (vifX.Y). Packets tx'ed by the frontend
appear as received packets here and vice versa.

As to how packets get to the physical network, that's down to how
domain 0 is configured. You can either create an L2 bridge, or
enable forwarding and configure IP routing appropriately. In both
cases you can use iptables to provide firewalling functionality.
 
Ian


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com

Re: Network questions

[Mark Williamson]

The until a few weeks ago, the unstable tree had a compile-time choice between 
"old" and "new" IO world.  Recently, the old IO world has been removed, so if 
you have pulled recently, the VIRQ_NET and the drivers in Xen, etc should 
disappear.

The backend driver creates an virtual "ethernet interface" for each 
unprivileged domain whose frontend connects to it.  The kernel that is running 
the backend driver thinks that it's connected to all the other domains by 
ethernet interfaces as well as to the physical network by the physical 
ethernet hardware.

Getting packets between virtual interfaces of domains and the physical network 
is then just a task of bridging or routing, which is dealt with by the same 
code that would transfer packets between multiple ethernet interfaces in any 
Linux system.  Currently, we use bridging by default.  See the code in 
linux/net/bridge/ for more details about how the Linux bridging code works.

In your example, when the packet is received on the Vortex card, it eventually 
gets passed to the bridging code, which identifies that it should be passed 
out on one of the virtual interfaces connected to another domain.  It then 
passes the packet to the backend driver, which transfers it to the frontend 
driver of the destination domain and notifies that domain of the arrival.

Since this stuff happens in the bridging code, you wouldn't be able to find an 
obvious call path.

As for the control messages, you should remember that the control interface is 
only used for a domain to talk to xend.  The backend driver uses the message 
you described to tell Xend that it is UP and ready to accept connections from 
other domains.  When other domains want to connect their network interfaces, 
they use their control interface to tell Xend.  Xend co-ordinates the process 
of connecting, then once the connection is established, the two domains can 
talk directly using shared memory.  You can find relevant code in 
tools/python/xen/xend/server/netif.py (I think) but you'll need to read other 
bits of Xend to understand how this all fits together.

It would be nice to have some documentation for this stuff (and we probably 
need some diagrams) in the interface manual...

HTH,

Mark



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com