Masquerading problems - XenU 3.0 on x86_64

Masquerading problems - XenU 3.0 on x86_64

[Jim Pick]

Hi,

I'm trying to migrate my Xen sessions installed on 32-bit Xen 2.0 server 
to a 64-bit Xen 3.0 server.

On the Xen 2.0 server (32-bit), I built a DomU kernel with masquerading, 
and I use that to do NAT for some private networks running on the same 
box.

When I tried to do it with Xen 3.0 (64-bit), I couldn't get it to work. 
  I had to build a custom DomU kernel (from xen-3.0-testing.hg, 2.6.16, 
2 days ago) in order to include the netfilter/iptables code.  ICMP 
works.  TCP doesn't.  Non-masquerading traffic is OK.  I had the same 
problems with the 2.6.12 kernel from Xen 3.0.1.

I captured some of the traffic, and ethereal is showing that the 
masqueraded traffic being output has bad TCP checksums.

I'm going to have to do some debugging to try to figure out what's going 
wrong.

Has anybody else encountered this?  Also, if it's already been fixed 
somewhere, I'd love to know.  Any Netfilter debugging tips would also be 
appreciated.

Cheers,

  - Jim

Re: Masquerading problems - XenU 3.0 on x86_64

[Keir Fraser]

On 9 Apr 2006, at 01:01, Jim Pick wrote:

> I'm trying to migrate my Xen sessions installed on 32-bit Xen 2.0 
> server to a 64-bit Xen 3.0 server.
>
> On the Xen 2.0 server (32-bit), I built a DomU kernel with 
> masquerading, and I use that to do NAT for some private networks 
> running on the same box.
>
> When I tried to do it with Xen 3.0 (64-bit), I couldn't get it to 
> work.  I had to build a custom DomU kernel (from xen-3.0-testing.hg, 
> 2.6.16, 2 days ago) in order to include the netfilter/iptables code.  
> ICMP works.  TCP doesn't.  Non-masquerading traffic is OK.  I had the 
> same problems with the 2.6.12 kernel from Xen 3.0.1.
>
> I captured some of the traffic, and ethereal is showing that the 
> masqueraded traffic being output has bad TCP checksums.
>
> I'm going to have to do some debugging to try to figure out what's 
> going wrong.
>
> Has anybody else encountered this?  Also, if it's already been fixed 
> somewhere, I'd love to know.  Any Netfilter debugging tips would also 
> be appreciated.

Turn off tx checksum offload in your domU's using ethtool. We had fixed 
some forms of NAT with our checksum offload, but maybe not for your 
type of setup.

  -- Keir

Re: Masquerading problems - XenU 3.0 on x86_64

[Jim Pick]

Keir Fraser wrote:
> 
> On 9 Apr 2006, at 01:01, Jim Pick wrote:
> 
>> I'm trying to migrate my Xen sessions installed on 32-bit Xen 2.0 
>> server to a 64-bit Xen 3.0 server.
>>
>> On the Xen 2.0 server (32-bit), I built a DomU kernel with 
>> masquerading, and I use that to do NAT for some private networks 
>> running on the same box.
>>
>> When I tried to do it with Xen 3.0 (64-bit), I couldn't get it to 
>> work.  I had to build a custom DomU kernel (from xen-3.0-testing.hg, 
>> 2.6.16, 2 days ago) in order to include the netfilter/iptables code.  
>> ICMP works.  TCP doesn't.  Non-masquerading traffic is OK.  I had the 
>> same problems with the 2.6.12 kernel from Xen 3.0.1.
>>
>> I captured some of the traffic, and ethereal is showing that the 
>> masqueraded traffic being output has bad TCP checksums.
>>
>> I'm going to have to do some debugging to try to figure out what's 
>> going wrong.
>>
>> Has anybody else encountered this?  Also, if it's already been fixed 
>> somewhere, I'd love to know.  Any Netfilter debugging tips would also 
>> be appreciated.
> 
> 
> Turn off tx checksum offload in your domU's using ethtool. We had fixed 
> some forms of NAT with our checksum offload, but maybe not for your type 
> of setup.

That fixed it.  Thanks!

Cheers,

  - Jim

Re: Masquerading problems - XenU 3.0 on x86_64

[Keir Fraser]

On 9 Apr 2006, at 21:18, Jim Pick wrote:

>>> Has anybody else encountered this?  Also, if it's already been fixed 
>>> somewhere, I'd love to know.  Any Netfilter debugging tips would 
>>> also be appreciated.
>> Turn off tx checksum offload in your domU's using ethtool. We had 
>> fixed some forms of NAT with our checksum offload, but maybe not for 
>> your type of setup.
>
> That fixed it.  Thanks!

It would be interesting to know more about your configuration, and also 
what Xen version you were using. Changeset 9579 fixed some bugs, so if 
your repository is older than that then you might well see problems.

  -- Keir

Re: Masquerading problems - XenU 3.0 on x86_64

[Jim Pick]

[-- Attachment #1: Type: text/plain, Size: 1381 bytes --]

The last change I had on xen-3.0-testing.hg was:

changeset:   9612:32b22f5286be
user:        kaf24@firebug.cl.cam.ac.uk
date:        Thu Apr  6 18:34:32 2006 +0100
summary:     Fix another blkback kernel thread I introduced. :-( The 
kernel thread

(ignore the changeset number - I checked some stuff into my own hg
repository)

It looks like I had changeset 9579.

I built it from within a Domain 0 running the Debian unstable amd64 
packages from amd64.debian.net.

$ gcc --version
gcc (GCC) 4.0.3 (Debian 4.0.3-1)

I'll attach my gzipped kernel config.

Anything else that would be useful?  I can also grant you access to the 
machine if that would be of any benefit.

Cheers,

  - Jim


Keir Fraser wrote:
> 
> On 9 Apr 2006, at 21:18, Jim Pick wrote:
> 
>>>> Has anybody else encountered this?  Also, if it's already been fixed 
>>>> somewhere, I'd love to know.  Any Netfilter debugging tips would 
>>>> also be appreciated.
>>>
>>> Turn off tx checksum offload in your domU's using ethtool. We had 
>>> fixed some forms of NAT with our checksum offload, but maybe not for 
>>> your type of setup.
>>
>>
>> That fixed it.  Thanks!
> 
> 
> It would be interesting to know more about your configuration, and also 
> what Xen version you were using. Changeset 9579 fixed some bugs, so if 
> your repository is older than that then you might well see problems.
> 
>  -- Keir
> 

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 5311 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel