Fix core dumps of guests > 2 GB in size on i386

Fix core dumps of guests > 2 GB in size on i386

[Daniel P. Berrange]

[-- Attachment #1: Type: text/plain, Size: 2411 bytes --]

During testing of Xen 3.0.3 tree we discovered that core dumps of guests
with more than 2 GB of RAM would fail on i386 platforms. A quick look at
the libxc code for generating core files shows that it does not use the
O_LARGEFILE flag or explicit open64 function when opening the file. Thus
the file is limited to 2^31-1 bytes.

The attached patch adds the O_LARGEFILE flag when opening the file. It
is also neccessary to add additional CFLAGS & LDFLAGS to enable use of
the POSIX Largefile  standard.  There are two ways to enable LFS support,
in implicit mode all types, functions & macros are automatically changed
to the 64 bit variants at compile time - this however changes the ABI
contract - eg size_t is now 64-bits instead of 32. The alternate is the
explicit mode, which merely makes 64-bit variants of the calls available
and thus preserves ABI. Thus in this patch I use the explicit LFS mode
and then pass O_LARGEFILE to the open() call flags.

Without this patch you see:

  # xm list
  xm Name                                      ID Mem(MiB) VCPUs State   Time(s)
  Domain-0                                   0     1485     4 r-----   4905.3
  coretest                                   9     2556     1 -b----     15.0
  # xm dump-core coretest
  Dumping core of domain: coretest ...
  Error: Failed to dump core: (27, 'File too large')
  Usage: xm dump-core [-L|--live] [-C|--crash] <Domain> [Filename]
  # ls -l /var/xen/dump/
  total 4721672
  -rw------- 1 root root 2147483647 Nov 15 16:33 2006-1115-1632.46-coretest.9.core-incomplete


With this patch applied:

  # xm list
  Name                                      ID Mem(MiB) VCPUs State   Time(s)
  Domain-0                                   0     1485     4 r-----     29.6
  coretest                                   1     2556     1 r-----      3.1
  # xm dump-core coretest
  Dumping core of domain: coretest ...
  # ls -l /var/xen/dump/
  total 4721672
  -rw------- 1 root root 2682781696 Nov 15 16:58 2006-1115-1657.33-coretest.1.core



  Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

[-- Attachment #2: xen-core-2gb.patch --]
[-- Type: text/plain, Size: 1630 bytes --]

diff -rc xen-3.0.3_0-src/tools/libxc/Makefile xen-3.0.3_0-src.new/tools/libxc/Makefile
*** xen-3.0.3_0-src/tools/libxc/Makefile	2006-10-15 08:22:03.000000000 -0400
--- xen-3.0.3_0-src.new/tools/libxc/Makefile	2006-11-15 16:08:47.000000000 -0500
***************
*** 39,44 ****
--- 39,48 ----
  CFLAGS   += -fno-strict-aliasing
  CFLAGS   += $(INCLUDES) -I.
  
+ # Enable large file support.
+ CFLAGS   += $(shell getconf LFS64_CFLAGS)
+ LDFLAGS  += $(shell getconf LFS64_LDFLAGS) $(shell getconf LFS64_LIBS)
+ 
  # Define this to make it possible to run valgrind on code linked with these
  # libraries.
  #CFLAGS   += -DVALGRIND -O0 -ggdb3
diff -rc xen-3.0.3_0-src/tools/libxc/xc_core.c xen-3.0.3_0-src.new/tools/libxc/xc_core.c
*** xen-3.0.3_0-src/tools/libxc/xc_core.c	2006-10-15 08:22:03.000000000 -0400
--- xen-3.0.3_0-src.new/tools/libxc/xc_core.c	2006-11-15 16:07:02.000000000 -0500
***************
*** 1,6 ****
--- 1,7 ----
  #include "xg_private.h"
  #include <stdlib.h>
  #include <unistd.h>
+ #include <fcntl.h>
  
  /* number of pages to write at a time */
  #define DUMP_INCREMENT (4 * 1024)
***************
*** 156,162 ****
      struct dump_args da;
      int sts;
  
!     if ( (da.fd = open(corename, O_CREAT|O_RDWR, S_IWUSR|S_IRUSR)) < 0 )
      {
          PERROR("Could not open corefile %s: %s", corename, strerror(errno));
          return -errno;
--- 157,163 ----
      struct dump_args da;
      int sts;
  
!     if ( (da.fd = open(corename, O_CREAT|O_RDWR|O_LARGEFILE, S_IWUSR|S_IRUSR)) < 0 )
      {
          PERROR("Could not open corefile %s: %s", corename, strerror(errno));
          return -errno;

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Fix core dumps of guests > 2 GB in size on i386

[Keir Fraser]

On 15/11/06 22:02, "Daniel P. Berrange" <berrange@redhat.com> wrote:

> The attached patch adds the O_LARGEFILE flag when opening the file. It
> is also neccessary to add additional CFLAGS & LDFLAGS to enable use of
> the POSIX Largefile  standard.  There are two ways to enable LFS support,
> in implicit mode all types, functions & macros are automatically changed
> to the 64 bit variants at compile time - this however changes the ABI
> contract - eg size_t is now 64-bits instead of 32.

I like the implicit method. I think it's unlikely to bite us. A lot of tools
subdirs are defining _FILE_OFFSET_BITS=64 anyway (and not using getconf!).
I'll provide a portable common CFLAGS/LDFLAGS addition in tools/Rules.mk.

 -- Keir

Re: Fix core dumps of guests > 2 GB in size on i386

[Daniel P. Berrange]

On Fri, Nov 17, 2006 at 09:52:15AM +0000, Keir Fraser wrote:
> On 15/11/06 22:02, "Daniel P. Berrange" <berrange@redhat.com> wrote:
> 
> > The attached patch adds the O_LARGEFILE flag when opening the file. It
> > is also neccessary to add additional CFLAGS & LDFLAGS to enable use of
> > the POSIX Largefile  standard.  There are two ways to enable LFS support,
> > in implicit mode all types, functions & macros are automatically changed
> > to the 64 bit variants at compile time - this however changes the ABI
> > contract - eg size_t is now 64-bits instead of 32.
> 
> I like the implicit method. I think it's unlikely to bite us. A lot of tools
> subdirs are defining _FILE_OFFSET_BITS=64 anyway (and not using getconf!).
> I'll provide a portable common CFLAGS/LDFLAGS addition in tools/Rules.mk.

Well there are two methods in libxc which expose a size_t argument so the 
implicit method will break ABI changing these from 32 to 64 bits.

xenctrl.h:int xc_tbuf_enable(int xc_handle, size_t cnt, unsigned long *mfn,
xenctrl.h:int xc_acm_op(int xc_handle, int cmd, void *arg, size_t arg_size);

They are probably only called from other apps / libraries in the tools/
dir, so perhaps we don't care about ABI breakage in these two functions.

Must make sure the python binding is still working correctly though because
a 64-bit size_t on 32-bit arch will no longer fit in Python's native Int
type - will have to switch to Long

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Re: Fix core dumps of guests > 2 GB in size on i386

[Keir Fraser]

On 17/11/06 12:12, "Daniel P. Berrange" <berrange@redhat.com> wrote:

> Well there are two methods in libxc which expose a size_t argument so the
> implicit method will break ABI changing these from 32 to 64 bits.
> 
> xenctrl.h:int xc_tbuf_enable(int xc_handle, size_t cnt, unsigned long *mfn,
> xenctrl.h:int xc_acm_op(int xc_handle, int cmd, void *arg, size_t arg_size);

It'd be better probably to replace with explicit types and avoid
off_t/size_t confusion at all. Maybe uint64_t.

 -- Keir

Re: Fix core dumps of guests > 2 GB in size on i386

[Daniel P. Berrange]

On Fri, Nov 17, 2006 at 01:47:12PM +0000, Keir Fraser wrote:
> On 17/11/06 12:12, "Daniel P. Berrange" <berrange@redhat.com> wrote:
> 
> > Well there are two methods in libxc which expose a size_t argument so the
> > implicit method will break ABI changing these from 32 to 64 bits.
> > 
> > xenctrl.h:int xc_tbuf_enable(int xc_handle, size_t cnt, unsigned long *mfn,
> > xenctrl.h:int xc_acm_op(int xc_handle, int cmd, void *arg, size_t arg_size);
> 
> It'd be better probably to replace with explicit types and avoid
> off_t/size_t confusion at all. Maybe uint64_t.

Looking more closely at the python code, it appears the size_t parameters
do not get directly exposed to the python, so there should be no data
type size issues here to worry about - the size_t param is merely used
in the raw C code doing sizeof() on one of the other params. So the patch
you've already committed to put LFS flags in the top level makefile looks
like it'll be sufficient.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|