[uml-devel] Integer overflows in new AIO ubd driver

[uml-devel] Integer overflows in new AIO ubd driver

[Gerd Stolpmann]

Hi,

I recently tried to benchmark the new AIO-based ubd driver (including
O_DIRECT), and ran into serious problems with filesystems > 4GB. I think
I found the problem; two overflows in ubd_kern.c (see patch).

My test system was 2.6.16 plus all patches from user-mode-linux.sf.net
minus no_cow_odirect. Symptoms were that ext3 complained that it was
tried to write blocks into the system zone, and switched the filesystem
to r/o.

I also experienced (hard) lockups when doing lots of I/O. But this seems
to be unrelated to this problem (but who knows).

Gerd

--- linux-2.6.16/arch/um/drivers/ubd_kern.c.orig        2006-03-26 16:57:11.000000000 +0200
+++ linux-2.6.16/arch/um/drivers/ubd_kern.c     2006-03-26 18:16:02.000000000 +0200
@@ -1228,7 +1228,7 @@ static void do_ubd_request(request_queue
                while(dev->start_sg < dev->end_sg){
                        struct scatterlist *sg = &dev->sg[dev->start_sg];

-                       prepare_request(req, &io_req, req->sector << 9,
+                       prepare_request(req, &io_req, (unsigned long long) req->sector << 9,
                                        sg->offset, sg->length, sg->page);

                        if(do_io(&io_req, req, dev) == -EAGAIN)
@@ -1573,7 +1573,7 @@ static int do_io(struct io_thread_req *r
                }

                off = req->offsets[bit] + req->offset +
-                       start * req->sectorsize;
+                       (unsigned long long) start * req->sectorsize;
                len = (end - start) * req->sectorsize;
                buf = &req->buffer[start * req->sectorsize];




-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Jeff Dike]

On Sun, Mar 26, 2006 at 08:49:25PM +0200, Gerd Stolpmann wrote:
> I recently tried to benchmark the new AIO-based ubd driver (including
> O_DIRECT), and ran into serious problems with filesystems > 4GB. I think
> I found the problem; two overflows in ubd_kern.c (see patch).

The first bit looks reasonable.  I would change start to an unsigned
long long instead of casting, though.

> I also experienced (hard) lockups when doing lots of I/O. But this seems
> to be unrelated to this problem (but who knows).

What was your workload?

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Gerd Stolpmann]

Am Montag, den 27.03.2006, 10:38 -0500 schrieb Jeff Dike:
> On Sun, Mar 26, 2006 at 08:49:25PM +0200, Gerd Stolpmann wrote:
> > I recently tried to benchmark the new AIO-based ubd driver (including
> > O_DIRECT), and ran into serious problems with filesystems > 4GB. I think
> > I found the problem; two overflows in ubd_kern.c (see patch).
> 
> The first bit looks reasonable.  I would change start to an unsigned
> long long instead of casting, though.
> 
> > I also experienced (hard) lockups when doing lots of I/O. But this seems
> > to be unrelated to this problem (but who knows).
> 
> What was your workload?

I used iozone in various ways. The simplest workload leading to problems
was:

- Write a 5GB file
- Write it again
- Read it back
- Read it back

This benchmark ran fine the first time I started it. When I started it
again, UML locked up in the middle. This is reproducible.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Jeff Dike]

On Mon, Mar 27, 2006 at 05:47:31PM +0200, Gerd Stolpmann wrote:
> I used iozone in various ways. The simplest workload leading to problems
> was:
> 
> - Write a 5GB file
> - Write it again
> - Read it back
> - Read it back
> 
> This benchmark ran fine the first time I started it. When I started it
> again, UML locked up in the middle. This is reproducible.

I'm interpretting that description as being this command:
	iozone -s 5G -i 0 -i 0 -i 1 -i 1

Yes?

And it runs fine here.

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Gerd Stolpmann]

Am Mittwoch, den 29.03.2006, 14:33 -0500 schrieb Jeff Dike:
> On Mon, Mar 27, 2006 at 05:47:31PM +0200, Gerd Stolpmann wrote:
> > I used iozone in various ways. The simplest workload leading to problems
> > was:
> > 
> > - Write a 5GB file
> > - Write it again
> > - Read it back
> > - Read it back
> > 
> > This benchmark ran fine the first time I started it. When I started it
> > again, UML locked up in the middle. This is reproducible.
> 
> I'm interpretting that description as being this command:
> 	iozone -s 5G -i 0 -i 0 -i 1 -i 1
> 
> Yes?

Almost:

iozone -i 0 -i 1 -r 64k -s 5G -e -f /Data/testfile -U /Data

i.e. record size is 64K, and there is a umount/mount between the
runs. /Data is an ext3 filesystem with 4k block size, on a 20 G
partition.

> And it runs fine here.

For me it runs fine only the first time I call it. The next time UML
locks up.

Can I do something to find out more details?

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Jeff Dike]

On Wed, Mar 29, 2006 at 11:16:27PM +0200, Gerd Stolpmann wrote:
> Almost:
> 
> iozone -i 0 -i 1 -r 64k -s 5G -e -f /Data/testfile -U /Data
> 
> i.e. record size is 64K, and there is a umount/mount between the
> runs. /Data is an ext3 filesystem with 4k block size, on a 20 G
> partition.

OK, this doesn't run fine for me even the first time, so I have something
I can chew on.

BTW, your earlier fixes are now in my patchset.

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Integer overflows in new AIO ubd driver

[Jeff Dike]

On Wed, Mar 29, 2006 at 11:16:27PM +0200, Gerd Stolpmann wrote:
> Almost:
> 
> iozone -i 0 -i 1 -r 64k -s 5G -e -f /Data/testfile -U /Data
> 
> i.e. record size is 64K, and there is a umount/mount between the
> runs. /Data is an ext3 filesystem with 4k block size, on a 20 G
> partition.
> 
> > And it runs fine here.
> 
> For me it runs fine only the first time I call it. The next time UML
> locks up.
> 
> Can I do something to find out more details?

I've been playing with this, and iozone can't make UML do anything
bad.  Is this the only scenario you have?

				Jeff


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel