From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Daniel P. Berrange" Subject: Re: Re: Getting xen to recognise large disks Date: Tue, 21 Nov 2006 22:41:41 +0000 Message-ID: <20061121224141.GC3438@redhat.com> References: <20061121211118.GB3438@redhat.com> Reply-To: "Daniel P. Berrange" Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="yrj/dFKFPuw6o+aM" Return-path: Content-Disposition: inline In-Reply-To: <20061121211118.GB3438@redhat.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser Cc: xen-devel@lists.xensource.com, Robin Bowes List-Id: xen-devel@lists.xenproject.org --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Nov 21, 2006 at 09:11:18PM +0000, Daniel P. Berrange wrote: > On Tue, Nov 21, 2006 at 11:34:45AM +0000, Keir Fraser wrote: > > On 21/11/06 11:21, "Robin Bowes" wrote: > > > > > Keir Fraser wrote: > > >> On 21/11/06 2:13 am, "Robin Bowes" wrote: > > >> > > >> I'll make a patch today. > > >> > > > > > > Thanks Keir, looking forward to testing it. > > > > If you don't mind using the xen-unstable source repository, it's changeset > > 12496:0c0ef61de06b. It probably hasn't reached the public repository just > > yet (should very shortly though). > > I've tested that changeset with the following > > - phy: against a 5 TB partition > - file: against a 7.3 TB file > > In both cases the # of sectors matches in Dom0 vs DomU. For good measure > I also ran Stephen Tweedie's verify-data tool in the DomU to verify no > data I/O wraparound issues elsewhere in the code & it passed without > trouble. > > Blktap, however, is a different story - it is showing wraparound for disk > size at the 2 TB size mark stil. The userspace blktap tools have totally > inconsistent data types. Sometimes using int, sometimes long, sometimes > unsigned long & sometimes uint64. I'm working on a patch which makes it > > - 'unsigned long long' for # sectors > - 'unsigned long' for sector size > - 'unsigned int' for info > > This makes it match the data types used in blkfront/blkback exactly. > With this patch applied, the DomU sees correct disk size, however, > the verify-data tool is showing nasty data consistency issues when > writing/reading to such a disk. So I think there is 32-bit wrap > around somewhere in the I/O codepath for blktap. I'll get back when > I've found out more info... It turns out that blktap wasn't (directly) at fault here. I was storing my file based disk images on an XFS formatted partition in the host. Well it appears that XFS doesn't play nice with the async I/O + O_DIRECT options that blktap likes so all your data goes to /dev/null :-) I re-tested blktap + large file backed disks on ext3 & GFS and everything is working as expected. So stay away from a XFS+blktap combo if you like your data :-) Attaching the patch to blktap to fix 32-bit wraparound of sector counts. Signed-off-by: Daniel P. Berrange Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="blktap-2tb-2.patch" diff -r 00ed59a6f043 tools/blktap/drivers/blktapctrl.c --- a/tools/blktap/drivers/blktapctrl.c Tue Nov 21 10:22:19 2006 +0000 +++ b/tools/blktap/drivers/blktapctrl.c Tue Nov 21 14:53:39 2006 -0500 @@ -420,7 +420,7 @@ static int read_msg(int fd, int msgtype, image->secsize = img->secsize; image->info = img->info; - DPRINTF("Received CTLMSG_IMG: %lu, %lu, %lu\n", + DPRINTF("Received CTLMSG_IMG: %llu, %lu, %u\n", image->size, image->secsize, image->info); if(msgtype != CTLMSG_IMG) ret = 0; break; diff -r 00ed59a6f043 tools/blktap/drivers/blktapctrl.h --- a/tools/blktap/drivers/blktapctrl.h Tue Nov 21 10:22:19 2006 +0000 +++ b/tools/blktap/drivers/blktapctrl.h Tue Nov 21 14:53:39 2006 -0500 @@ -30,19 +30,19 @@ */ -static inline long int tapdisk_get_size(blkif_t *blkif) +static inline unsigned long long tapdisk_get_size(blkif_t *blkif) { image_t *img = (image_t *)blkif->prv; return img->size; } -static inline long int tapdisk_get_secsize(blkif_t *blkif) +static inline unsigned long tapdisk_get_secsize(blkif_t *blkif) { image_t *img = (image_t *)blkif->prv; return img->secsize; } -static inline unsigned tapdisk_get_info(blkif_t *blkif) +static inline unsigned int tapdisk_get_info(blkif_t *blkif) { image_t *img = (image_t *)blkif->prv; return img->info; diff -r 00ed59a6f043 tools/blktap/drivers/tapdisk.h --- a/tools/blktap/drivers/tapdisk.h Tue Nov 21 10:22:19 2006 +0000 +++ b/tools/blktap/drivers/tapdisk.h Tue Nov 21 14:53:39 2006 -0500 @@ -74,9 +74,9 @@ struct td_state { void *ring_info; void *fd_entry; char backing_file[1024]; /*Used by differencing disks, e.g. qcow*/ - long int sector_size; - uint64_t size; - long int info; + unsigned long sector_size; + unsigned long long size; + unsigned int info; }; /* Prototype of the callback to activate as requests complete. */ diff -r 00ed59a6f043 tools/blktap/lib/blktaplib.h --- a/tools/blktap/lib/blktaplib.h Tue Nov 21 10:22:19 2006 +0000 +++ b/tools/blktap/lib/blktaplib.h Tue Nov 21 14:54:21 2006 -0500 @@ -97,9 +97,9 @@ typedef struct { } pending_req_t; struct blkif_ops { - long int (*get_size)(struct blkif *blkif); - long int (*get_secsize)(struct blkif *blkif); - unsigned (*get_info)(struct blkif *blkif); + unsigned long long (*get_size)(struct blkif *blkif); + unsigned long (*get_secsize)(struct blkif *blkif); + unsigned int (*get_info)(struct blkif *blkif); }; typedef struct blkif { @@ -156,9 +156,9 @@ typedef struct domid_translate { } domid_translate_t ; typedef struct image { - long int size; - long int secsize; - long int info; + unsigned long long size; + unsigned long secsize; + unsigned int info; } image_t; typedef struct msg_hdr { diff -r 00ed59a6f043 tools/blktap/lib/xenbus.c --- a/tools/blktap/lib/xenbus.c Tue Nov 21 10:22:19 2006 +0000 +++ b/tools/blktap/lib/xenbus.c Tue Nov 21 14:53:58 2006 -0500 @@ -219,7 +219,7 @@ static void ueblktap_setup(struct xs_han } /* Supply the information about the device to xenstore */ - er = xs_printf(h, be->backpath, "sectors", "%lu", + er = xs_printf(h, be->backpath, "sectors", "%llu", be->blkif->ops->get_size(be->blkif)); if (er == 0) { --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --yrj/dFKFPuw6o+aM--