[uml-devel] ubd and write barriers

[uml-devel] ubd and write barriers

[Chris Lightfoot]

The current ubd implementation doesn't support write
barriers (blk_queue_ordered etc.), and the reccomendation
is to use synchronous IO to ensure data integrity. In
principle, adding barrier request support ought to get
around this problem, allowing writes without O_SYNC with
occasional calls to fdatasync on the underlying data (the
motivation for write barrier support was safety of
journalling filesystems on disks with write caches, an
analogous problem).

ext3 and reiserfs support the write barrier stuff if
mounted with appropriate options (barrier=1 and
barrier=flush respectively).

Implementing this ought to give performance somewhere
between ubdXs and ubdX, but with adequate guarantees of
data integrity for use with modern filesystems. That said,
I haven't measured this yet.

A patch for this is pretty simple (see below). Has there
been discussion of this before? I couldn't find any in the
list archives....

--- ubd_kern.c.orig     2005-10-04 10:17:50.000000000 +0100
+++ ubd_kern.c  2005-10-04 10:22:59.000000000 +0100
@@ -68,6 +68,7 @@
        unsigned long long cow_offset;
        unsigned long bitmap_words[2];
        int error;
+        int barrier;
 };
 
 extern int open_ubd_file(char *file, struct openflags *openflags,
@@ -825,6 +826,8 @@
                unregister_blkdev(MAJOR_NR, "ubd");
                return -1;
        }
+
+        blk_queue_ordered(ubd_queue, QUEUE_ORDERED_TAG);
 
        if (fake_major != MAJOR_NR) {
                char name[sizeof("ubd_nnn\0")];
@@ -1003,6 +1006,8 @@
        io_req->buffer = req->buffer;
        io_req->sectorsize = 1 << 9;
 
+        io_req->barrier = blk_barrier_rq(req);
+
        if(dev->cow.file != NULL)
                cowify_req(io_req, dev->cow.bitmap, dev->cow.bitmap_offset,
                           dev->cow.bitmap_len);
@@ -1334,6 +1339,8 @@
                start = end;
        } while(start < nsectors);
 
+        if (req->barrier) fdatasync(req->fds[bit]); /* XXX also sync before this write? */
+
        req->error = update_bitmap(req);
 }
 


-- 
``Is there no beginning to your talents?''
  (Clive Anderson, to Jeffrey Archer)


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] ubd and write barriers

[Blaisorblade]

On Tuesday 04 October 2005 11:31, Chris Lightfoot wrote:
> The current ubd implementation doesn't support write
> barriers (blk_queue_ordered etc.), and the reccomendation
> is to use synchronous IO to ensure data integrity. In
> principle, adding barrier request support ought to get
> around this problem, allowing writes without O_SYNC with
> occasional calls to fdatasync on the underlying data (the
> motivation for write barrier support was safety of
> journalling filesystems on disks with write caches, an
> analogous problem).
Yes, I more or less remembered that was the idea (from readings on lwn.net), 
thanks for confirming it.
> ext3 and reiserfs support the write barrier stuff if
> mounted with appropriate options (barrier=1 and
> barrier=flush respectively).
So they don't auto-enable this when support exists? I.e. everybody is running 
slower just because we don't add those params?
> Implementing this ought to give performance somewhere
> between ubdXs and ubdX, but with adequate guarantees of
> data integrity for use with modern filesystems. That said,
> I haven't measured this yet.

> A patch for this is pretty simple (see below). Has there
> been discussion of this before? I couldn't find any in the
> list archives....
This was discussed between me and Jeff recently, even if maybe that was 
off-list.
-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894)
http://www.user-mode-linux.org/~blaisorblade

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it



-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] ubd and write barriers

[Chris Lightfoot]

On Tue, Oct 04, 2005 at 12:31:56PM +0200, Blaisorblade wrote:
> On Tuesday 04 October 2005 11:31, Chris Lightfoot wrote:
> > The current ubd implementation doesn't support write
> > barriers (blk_queue_ordered etc.), and the reccomendation
> > is to use synchronous IO to ensure data integrity. In
> > principle, adding barrier request support ought to get
> > around this problem, allowing writes without O_SYNC with
> > occasional calls to fdatasync on the underlying data (the
> > motivation for write barrier support was safety of
> > journalling filesystems on disks with write caches, an
> > analogous problem).
> Yes, I more or less remembered that was the idea (from readings on lwn.net), 
> thanks for confirming it.

ok -- q.v. the discussion in here:
    http://lwn.net/images/pdf/LDD3/ch16.pdf

> > ext3 and reiserfs support the write barrier stuff if
> > mounted with appropriate options (barrier=1 and
> > barrier=flush respectively).
> So they don't auto-enable this when support exists? I.e. everybody is running 
> slower just because we don't add those params?

This bit I'm not sure about. I found one reference which
said that barrier=1 is enabled by default in ext3, but it
certainly isn't in the current kernel, and I suspect it's
because some real disks lie about cache flushes(?).
There's a compatibility chart here which tells you which
combinations are safe on real hardware:
    http://www.dt.e-technik.uni-dortmund.de/~ma/linux/kernel/safe-write-caches.html
There was also some discussion somewhere about having a
blacklist or whitelist of supported drives. Still, I don't
understand why barrier=0 is any safer: if the write
barriers aren't supported by the driver or hardware,
there's still no other way of ensuring changes are
committed to disk -- it's up to the user to test and
configure appropriately.

Anyway, none of that applies to the UML/ubd case, but if
barriers are implemented in ubd it's pretty easy to add
the barrier=1 mount option when starting each UML
instance.

> > between ubdXs and ubdX, but with adequate guarantees of
> > data integrity for use with modern filesystems. That said,
> > I haven't measured this yet.
> 
> > A patch for this is pretty simple (see below). Has there
> > been discussion of this before? I couldn't find any in the
> > list archives....
> This was discussed between me and Jeff recently, even if maybe that was 
> off-list.

-- 
``Nobody ever forgets where he buried the hatchet.'' (Hubbard)


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] ubd and write barriers

[Adam Heath]

On Tue, 4 Oct 2005, Chris Lightfoot wrote:

> --- ubd_kern.c.orig     2005-10-04 10:17:50.000000000 +0100
> +++ ubd_kern.c  2005-10-04 10:22:59.000000000 +0100
> @@ -68,6 +68,7 @@
>         unsigned long long cow_offset;
>         unsigned long bitmap_words[2];
>         int error;
> +        int barrier;
>  };
>
>  extern int open_ubd_file(char *file, struct openflags *openflags,
> @@ -825,6 +826,8 @@
>                 unregister_blkdev(MAJOR_NR, "ubd");
>                 return -1;
>         }
> +
> +        blk_queue_ordered(ubd_queue, QUEUE_ORDERED_TAG);
>
>         if (fake_major != MAJOR_NR) {
>                 char name[sizeof("ubd_nnn\0")];
> @@ -1003,6 +1006,8 @@
>         io_req->buffer = req->buffer;
>         io_req->sectorsize = 1 << 9;
>
> +        io_req->barrier = blk_barrier_rq(req);
> +
>         if(dev->cow.file != NULL)
>                 cowify_req(io_req, dev->cow.bitmap, dev->cow.bitmap_offset,
>                            dev->cow.bitmap_len);
> @@ -1334,6 +1339,8 @@
>                 start = end;
>         } while(start < nsectors);
>
> +        if (req->barrier) fdatasync(req->fds[bit]); /* XXX also sync before this write? */
> +
>         req->error = update_bitmap(req);
>  }

Fix your tabbing.


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel