Re: Re: Getting xen to recognise large disks

["Daniel P. Berrange" <berrange@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2685 bytes --]

On Tue, Nov 21, 2006 at 09:11:18PM +0000, Daniel P. Berrange wrote:
> On Tue, Nov 21, 2006 at 11:34:45AM +0000, Keir Fraser wrote:
> > On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote:
> > 
> > > Keir Fraser wrote:
> > >> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote:
> > >> 
> > >> I'll make a patch today.
> > >> 
> > > 
> > > Thanks Keir, looking forward to testing it.
> > 
> > If you don't mind using the xen-unstable source repository, it's changeset
> > 12496:0c0ef61de06b. It probably hasn't reached the public repository just
> > yet (should very shortly though).
> 
> I've tested that changeset with the following
> 
>  - phy:  against a 5 TB partition
>  - file: against a 7.3 TB file
> 
> In both cases the # of sectors matches in Dom0 vs DomU. For good measure
> I also ran Stephen Tweedie's verify-data tool in the DomU to verify no
> data I/O wraparound issues elsewhere in the code & it passed without
> trouble.
> 
> Blktap, however, is a different story - it is showing wraparound for disk
> size at the 2 TB size mark stil. The userspace blktap tools have totally
> inconsistent data types. Sometimes using int, sometimes long, sometimes
> unsigned long & sometimes uint64. I'm working on a patch which makes it 
> 
>  - 'unsigned long long'  for # sectors
>  - 'unsigned long'       for sector size
>  - 'unsigned int'        for info
> 
> This makes it match the data types used in blkfront/blkback exactly.
> With this patch applied, the DomU sees correct disk size, however,
> the verify-data tool is showing nasty data consistency issues when
> writing/reading to such a disk. So I think there is 32-bit wrap
> around somewhere in the I/O codepath for blktap. I'll get back when
> I've found out more info...

It turns out that blktap wasn't (directly) at fault here. I was storing my
file based disk images on an XFS formatted partition in the host. Well it
appears that XFS doesn't play nice with the async I/O + O_DIRECT options
that blktap likes so all your data goes to /dev/null :-)

I re-tested blktap + large file backed disks on ext3 & GFS and everything
is working as expected. So stay away from a XFS+blktap combo if you like 
your data :-)


Attaching the patch to blktap to fix 32-bit wraparound of sector counts.

 Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

[-- Attachment #2: blktap-2tb-2.patch --]
[-- Type: text/plain, Size: 3206 bytes --]

diff -r 00ed59a6f043 tools/blktap/drivers/blktapctrl.c
--- a/tools/blktap/drivers/blktapctrl.c	Tue Nov 21 10:22:19 2006 +0000
+++ b/tools/blktap/drivers/blktapctrl.c	Tue Nov 21 14:53:39 2006 -0500
@@ -420,7 +420,7 @@ static int read_msg(int fd, int msgtype,
 			image->secsize = img->secsize;
 			image->info = img->info;
 
-			DPRINTF("Received CTLMSG_IMG: %lu, %lu, %lu\n",
+			DPRINTF("Received CTLMSG_IMG: %llu, %lu, %u\n",
 				image->size, image->secsize, image->info);
 			if(msgtype != CTLMSG_IMG) ret = 0;
 			break;
diff -r 00ed59a6f043 tools/blktap/drivers/blktapctrl.h
--- a/tools/blktap/drivers/blktapctrl.h	Tue Nov 21 10:22:19 2006 +0000
+++ b/tools/blktap/drivers/blktapctrl.h	Tue Nov 21 14:53:39 2006 -0500
@@ -30,19 +30,19 @@
  */
 
 
-static inline long int tapdisk_get_size(blkif_t *blkif)
+static inline unsigned long long tapdisk_get_size(blkif_t *blkif)
 {
 	image_t *img = (image_t *)blkif->prv;
 	return img->size;
 }
 
-static inline long int tapdisk_get_secsize(blkif_t *blkif)
+static inline unsigned long tapdisk_get_secsize(blkif_t *blkif)
 {
 	image_t *img = (image_t *)blkif->prv;
 	return img->secsize;
 }
 
-static inline unsigned tapdisk_get_info(blkif_t *blkif)
+static inline unsigned int tapdisk_get_info(blkif_t *blkif)
 {
 	image_t *img = (image_t *)blkif->prv;
 	return img->info;
diff -r 00ed59a6f043 tools/blktap/drivers/tapdisk.h
--- a/tools/blktap/drivers/tapdisk.h	Tue Nov 21 10:22:19 2006 +0000
+++ b/tools/blktap/drivers/tapdisk.h	Tue Nov 21 14:53:39 2006 -0500
@@ -74,9 +74,9 @@ struct td_state {
 	void *ring_info;
 	void *fd_entry;
 	char backing_file[1024]; /*Used by differencing disks, e.g. qcow*/
-	long int   sector_size;
-	uint64_t   size;
-	long int   info;
+	unsigned long      sector_size;
+	unsigned long long size;
+	unsigned int       info;
 };
 
 /* Prototype of the callback to activate as requests complete.              */
diff -r 00ed59a6f043 tools/blktap/lib/blktaplib.h
--- a/tools/blktap/lib/blktaplib.h	Tue Nov 21 10:22:19 2006 +0000
+++ b/tools/blktap/lib/blktaplib.h	Tue Nov 21 14:54:21 2006 -0500
@@ -97,9 +97,9 @@ typedef struct {
 } pending_req_t;
 
 struct blkif_ops {
-	long int (*get_size)(struct blkif *blkif);
-	long int (*get_secsize)(struct blkif *blkif);
-	unsigned (*get_info)(struct blkif *blkif);
+	unsigned long long (*get_size)(struct blkif *blkif);
+	unsigned long (*get_secsize)(struct blkif *blkif);
+	unsigned int (*get_info)(struct blkif *blkif);
 };
 
 typedef struct blkif {
@@ -156,9 +156,9 @@ typedef struct domid_translate {
 } domid_translate_t ;
 
 typedef struct image {
-	long int size;
-	long int secsize;
-	long int info;
+	unsigned long long size;
+	unsigned long secsize;
+	unsigned int info;
 } image_t;
 
 typedef struct msg_hdr {
diff -r 00ed59a6f043 tools/blktap/lib/xenbus.c
--- a/tools/blktap/lib/xenbus.c	Tue Nov 21 10:22:19 2006 +0000
+++ b/tools/blktap/lib/xenbus.c	Tue Nov 21 14:53:58 2006 -0500
@@ -219,7 +219,7 @@ static void ueblktap_setup(struct xs_han
 	}
 
 	/* Supply the information about the device to xenstore */
-	er = xs_printf(h, be->backpath, "sectors", "%lu",
+	er = xs_printf(h, be->backpath, "sectors", "%llu",
 			be->blkif->ops->get_size(be->blkif));
 
 	if (er == 0) {

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel