nat does not work with hvm

nat does not work with hvm

[Erdem Bayer]

Hello all

I am using open source xen v 3.0.4. When network-nat and vif-nat is 
activated in the config file, modified guests boot ok but hvm guests 
does not have network connection. Here is the config files:

xend-config.sxp
-------------------------------------
#(network-script network-bridge)
#(vif-script vif-bridge)

(network-script network-nat)
(vif-script vif-nat)

-------------------------------------
I have commented the following lines from qemu-ifup script as I don't 
want to use bridge.
-------------------------------------
#ifconfig $1 0.0.0.0 up
#brctl addif $2 $1
-------------------------------------
I have also try commenting only the second line from the script. Still 
no luck. In that case an interface named tap is created also.

And the config file for the domain:
-------------------------------------
import os, re
arch = os.uname()[4]
if re.search('64', arch):
   arch_libdir = 'lib64'
else:
   arch_libdir = 'lib'
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 512
shadow_memory = 8
name = "winxp"
acpi=0
vif = [ 'ip=192.168.3.1, mac=aa:00:00:50:02:f0, type=ioemu, 
model=rtl8139' ]
disk = [ 'file:/mnt/sda5/xen/images/winxp1.img,hda,w',
'phy:/dev/cdrom,hdb:cdrom,r' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
sdl=1
vnc=0
vncpasswd=''
stdvga=0
serial='pty'
-------------------------------------

When this domain is booted, it boots ok. Here is the network-nat status 
output.
============================================================
eth0      Link encap:Ethernet  HWaddr 00:17:08:37:F1:B7
         inet addr:10.0.0.5  Bcast:10.255.255.255  Mask:255.0.0.0
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:30 errors:0 dropped:0 overruns:0 frame:0
         TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:4530 (4.4 Kb)  TX bytes:2217 (2.1 Kb)
         Interrupt:20

lo        Link encap:Local Loopback
         inet addr:127.0.0.1  Mask:255.0.0.0
         UP LOOPBACK RUNNING  MTU:16436  Metric:1
         RX packets:6 errors:0 dropped:0 overruns:0 frame:0
         TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:328 (328.0 b)  TX bytes:328 (328.0 b)

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
         inet addr:192.168.3.128  Bcast:0.0.0.0  Mask:255.255.255.255
         UP BROADCAST MULTICAST  MTU:1500  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:32
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)


192.168.3.1 dev vif1.0  scope link  src 192.168.3.128
10.0.0.0/8 dev eth0  proto kernel  scope link  src 10.0.0.5
127.0.0.0/8 via 127.0.0.1 dev lo  scope link
default via 10.0.0.2 dev eth0

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
192.168.3.1     0.0.0.0         255.255.255.255 UH    0      0        0 
vif1.0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 
eth0
127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo
0.0.0.0         10.0.0.2        0.0.0.0         UG    0      0        0 
eth0
============================================================

When this domain is booted, it cannot connect anywhere. It can not even 
ping 192.168.3.128 !!!!
And when the dom0 pings 192.168.3.1 Destination host unreachable error 
is get. The tcpdump in dom0 does not show any packets flowing.

I have tried the same config with an ubuntu dapper hvm guest and still 
no connections.
But when a modified guest is booted with the same network config, there 
is no problem. Everyone can ping each other and domU does have an 
internet connection. Here is the network-nat status output of the 
modified guest.

============================================================
eth0      Link encap:Ethernet  HWaddr 00:17:08:37:F1:B7
          inet addr:10.0.0.5  Bcast:10.255.255.255  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1706 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1546 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1333224 (1.2 Mb)  TX bytes:159280 (155.5 Kb)
          Interrupt:20

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:9 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:500 (500.0 b)  TX bytes:500 (500.0 b)

vif10.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet addr:192.168.3.128  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1242 (1.2 Kb)  TX bytes:3118 (3.0 Kb)


192.168.3.1 dev vif10.0  scope link  src 192.168.3.128
10.0.0.0/8 dev eth0  proto kernel  scope link  src 10.0.0.5
127.0.0.0/8 via 127.0.0.1 dev lo  scope link
default via 10.0.0.2 dev eth0

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
192.168.3.1     0.0.0.0         255.255.255.255 UH    0      0        0 
vif10.0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth0
127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo
0.0.0.0         10.0.0.2        0.0.0.0         UG    0      0        0 eth0
============================================================


I have also tried dhcp instead of statically giving an ip address. Still 
the modified guest takes an ip address and have network, the hvm guest 
can not even reach the dhcp server in dom0.

Any ideas?
Erdem

Re: nat does not work with hvm

[Keir Fraser]

On 14/4/07 07:24, "Erdem Bayer" <ebayer@ttnet.net.tr> wrote:

> -------------------------------------
> #ifconfig $1 0.0.0.0 up
> #brctl addif $2 $1
> -------------------------------------
> I have also try commenting only the second line from the script. Still
> no luck. In that case an interface named tap is created also.

If you comment out those two lines and don't provide an alternative then the
network backend interface will not even be brought up. It's then no surprise
that no traffic can get to or from your HVM guest.

If you want to route traffic rather than bridging it then you need to add
commands to set up a route to your VM. And you almost certainly want to keep
the first line that you commented out. That's the one that actually brings
the interface online. If routing you'll want to put an actual IP address
rather than 0.0.0.0.

I don't think an IP address can currently be specified to that script, by
the way. At least not directly.

 -- Keir

Re: nat does not work with hvm

[Erdem Bayer]

But the network-nat script still runs in that case right? So what should 
be in the qemu-ifup script? Also should there also be a qemu-ifdown 
script to undo the changes?

I think in the qemu-ifup I should write a rule to switch the vifX.0 
created by the network-nat script and the tap interface created by the 
qemu-ifup script? Am I right about this?

Thanks for help

Erdem

Keir Fraser wrote On 14-04-2007 12:37:
> On 14/4/07 07:24, "Erdem Bayer" <ebayer@ttnet.net.tr> wrote:
>
>   
>> -------------------------------------
>> #ifconfig $1 0.0.0.0 up
>> #brctl addif $2 $1
>> -------------------------------------
>> I have also try commenting only the second line from the script. Still
>> no luck. In that case an interface named tap is created also.
>>     
>
> If you comment out those two lines and don't provide an alternative then the
> network backend interface will not even be brought up. It's then no surprise
> that no traffic can get to or from your HVM guest.
>
> If you want to route traffic rather than bridging it then you need to add
> commands to set up a route to your VM. And you almost certainly want to keep
> the first line that you commented out. That's the one that actually brings
> the interface online. If routing you'll want to put an actual IP address
> rather than 0.0.0.0.
>
> I don't think an IP address can currently be specified to that script, by
> the way. At least not directly.
>
>  -- Keir
>
>
>
>   

Re: nat does not work with hvm

[Keir Fraser]

On 14/4/07 07:56, "Erdem Bayer" <ebayer@ttnet.net.tr> wrote:

> But the network-nat script still runs in that case right?

Network-nat will run once, when xend is started. It's not run per-vif.

> So what should be in the qemu-ifup script?

Something a bit like what's in vif-nat, perhaps? That's the script that gets
run per-vif for non-hvm domains.

> Also should there also be a qemu-ifdown script to undo the changes?

Not needed. Everything should tear down automatically when the qemu process
dies.

 -- Keir

> I think in the qemu-ifup I should write a rule to switch the vifX.0
> created by the network-nat script and the tap interface created by the
> qemu-ifup script? Am I right about this?

Adding PTE Flag during Page Fault

[Joe Laws]

I altered the method, "__spurious_page_fault", in "xen/arch/x86/traps.c"
to change a page to present when it is marked as not present.  I am
attempting to do this using the "l1e_add_flags(l1e, _PAGE_PRESENT);" 
definition call.  I then set it to return EXCRET_fault_fixed, so the
trap should have the client retry what it was doing with the changes I
made.  Unfortunately after it returns from the page fault to the user,
it immediately faults again on the same error at the same memory
location.  This means my "l1e_add_flags(l1e, _PAGE_PRESENT)" is not
actually updating the correct location in memory or  if it is, the
changes get discarded immediately.  Does anyone who is familiar with the
page_fault trap code have any idea why my change to the PTE is not
sticking.  I tried doing a TLB flush for the memory address, but the
same thing happens.  My only other guess is that the l1e defined in the
function is just a copy of the actual table entry, so I need to look
back further where l1e was defined.

Thanks,
Joe

Re: Adding PTE Flag during Page Fault

[Joe Laws]

Tried changing the l1e by doing this:

l1e_write_atomic(&l1t[l1_table_offset(addr)],l1e);
unmap_domain_page(l1t);

Now it just causes Xen to crash and my computer to reboot. 
Unfortunately I can only see the output to the log by doing xm dmesg,
which gets cleared on every reboot.  Are there any efficient techniques
to get the latest xm dmesg output to file before the system crashes?

-Joe

Joe Laws wrote:
> I altered the method, "__spurious_page_fault", in "xen/arch/x86/traps.c"
> to change a page to present when it is marked as not present.  I am
> attempting to do this using the "l1e_add_flags(l1e, _PAGE_PRESENT);" 
> definition call.  I then set it to return EXCRET_fault_fixed, so the
> trap should have the client retry what it was doing with the changes I
> made.  Unfortunately after it returns from the page fault to the user,
> it immediately faults again on the same error at the same memory
> location.  This means my "l1e_add_flags(l1e, _PAGE_PRESENT)" is not
> actually updating the correct location in memory or  if it is, the
> changes get discarded immediately.  Does anyone who is familiar with the
> page_fault trap code have any idea why my change to the PTE is not
> sticking.  I tried doing a TLB flush for the memory address, but the
> same thing happens.  My only other guess is that the l1e defined in the
> function is just a copy of the actual table entry, so I need to look
> back further where l1e was defined.
>
> Thanks,
> Joe
>
>
>   

Re: Re: Adding PTE Flag during Page Fault

[Keir Fraser]

On 14/4/07 22:29, "Joe Laws" <jlaws@andrew.cmu.edu> wrote:

> Now it just causes Xen to crash and my computer to reboot.
> Unfortunately I can only see the output to the log by doing xm dmesg,
> which gets cleared on every reboot.  Are there any efficient techniques
> to get the latest xm dmesg output to file before the system crashes?

You can use kdump (basically chain boot into a special crash-dump Linux
kernel with block or net drivers which will scrape memory for things like
Xen heap and Xen logging output).

 -- Keir

Re: Re: Adding PTE Flag during Page Fault

[Joe Laws]

Thanks for the suggestion Keir, unfortunately my kernel lacks the option
for kexec, despite the fact that all the source files are included. 
Anyway, I was able to get the information I need by passing certain args
to the xen kernel via grub:

kernel /xen.gz noreboot console=vga vga=keep xencons=tty

Using this output I managed to get my stuff working this morning also. 
I was surprised to learn that of the ~90K pages saved (for a 380MB
memory image) only ~1500 are needed for the system to start up and
continue running.  Granted every time something is done another 10-20
pages get loaded, the fact that less than 2% only need to be present to
start the VM again is crazy.  This means that only ~6MB of the memory
image needs to be present to resume. 

-Joe

Keir Fraser wrote:
>
> On 14/4/07 22:29, "Joe Laws" <jlaws@andrew.cmu.edu> wrote:
>
>   
>> Now it just causes Xen to crash and my computer to reboot.
>> Unfortunately I can only see the output to the log by doing xm dmesg,
>> which gets cleared on every reboot.  Are there any efficient techniques
>> to get the latest xm dmesg output to file before the system crashes?
>>     
>
> You can use kdump (basically chain boot into a special crash-dump Linux
> kernel with block or net drivers which will scrape memory for things like
> Xen heap and Xen logging output).
>
>  -- Keir
>
>
>
>   

Re: Adding PTE Flag during Page Fault

[Keir Fraser]

On 14/4/07 21:50, "Joe Laws" <jlaws@andrew.cmu.edu> wrote:

> My only other guess is that the l1e defined in the
> function is just a copy of the actual table entry, so I need to look
> back further where l1e was defined.

Well, l1e is a local variable in that function. *l1t points at the actual
page table entry. So you want something like:
 l1e_add_flags(l1e, _PAGE_PRESENT);
 l1e_write_atomic(l1t, l1e);

 -- Keir