vfs whitebox test

vfs whitebox test

[Pat LaVarre]

Anybody already pull together some more closely interactive way of
trying creative patterns of calls thru the vfs layer?

I ask because I'm still hung up on my demo that udf.ko can't reliably
write more than one file at a time:

http://marc.theaimsgroup.com/?l=linux-fsdevel&m=106676214111422

List: linux-fsdevel
Subject: Re: zeroes read back more often than appended
Date: 2003-10-21 18:44:54

I'm guessing what makes that demo tough to analyse is that I wrote it up
at the high level of fopen/ fwrite/ fclose/ hexdump.  I'm guessing
translating that demo down to the more verbose level of open/ write/
read/ close actually won't make the demo significantly easier to
analyse.  I'm thinking I have to drop down to the vfs level.

Can I somehow drop down to the vfs level?

Maybe I could run the fs as a .so?

Can I help in any way quicker than deciphering the source of udf.ko?

Pat LaVarre

Re: vfs whitebox test

[Pat LaVarre]

> Anybody already pull together some more closely interactive way of
> trying creative patterns of calls thru the vfs layer?

>From the linux-fsdevel 2003-10-07 archives "Re: fs test suite", we know
possibilities, perhaps sorted now from most to least accessible,
include:

http://ltp.sourceforge.net/
http://www.iozone.org
http://www.connectathon.org/

http://www.codemonkey.org.uk/cruft/
http://www.zip.com.au/~akpm/linux/patches/stuff/

http://www.google.com/search?q=postmark+linux
http://www.google.com/search?q=bonnie+linux

By now I can testify that at a glance these particular links do Not say
if they point only to ways of running up in the stratospheric layer of
open/ read/ write/ close, or not.

Pat LaVarre

Re: vfs whitebox test

[Pat LaVarre]

> Anybody already pull together some more
> closely interactive way of trying creative
> patterns of calls thru the vfs layer?

I read contents/ index/ pages at random and then sequentially the first
two-hundred ninety-two pages of the German-flavoured English (e.g.
"smaller than zero") of "Linux Kernel Internals" and then I saw:

gdb /usr/src/linux/vmlinux /proc/kcore

... "all the structures in the kernel to be read, but no local variables
[also] not possible to change values or call kernel functions ..."

Also I saw:

- CFLAGS= -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
+ CFLAGS= -Wall -Wstrict-prototypes -O2 -g

CFLAGS -g I find in linux-2.6.0-test9/Makefile near CONFIG_DEBUG_INFO.

grep CONFIG_DEBUG_INFO `find . -name 'Kconfig'`
then flunks, but that question I can pursue in ?l=kernelnewbies.

Pat LaVarre

Re: vfs whitebox test

[Randy.Dunlap]

On 25 Nov 2003 10:04:56 -0700 Pat LaVarre <p.lavarre@ieee.org> wrote:

| > Anybody already pull together some more
| > closely interactive way of trying creative
| > patterns of calls thru the vfs layer?
| 
| I read contents/ index/ pages at random and then sequentially the first
| two-hundred ninety-two pages of the German-flavoured English (e.g.
| "smaller than zero") of "Linux Kernel Internals" and then I saw:
| 
| gdb /usr/src/linux/vmlinux /proc/kcore
| 
| ... "all the structures in the kernel to be read, but no local variables
| [also] not possible to change values or call kernel functions ..."
| 
| Also I saw:
| 
| - CFLAGS= -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
| + CFLAGS= -Wall -Wstrict-prototypes -O2 -g
| 
| CFLAGS -g I find in linux-2.6.0-test9/Makefile near CONFIG_DEBUG_INFO.
| 
| grep CONFIG_DEBUG_INFO `find . -name 'Kconfig'`
| then flunks, but that question I can pursue in ?l=kernelnewbies.

search for DEBUG_INFO in all Kconfig files.  They don't include
the CONFIG_ prefix.

--
~Randy
MOTD:  Always include version info.

Re: vfs whitebox test

[Randy.Dunlap]

On 25 Nov 2003 08:46:22 -0700 Pat LaVarre <p.lavarre@ieee.org> wrote:

| > Anybody already pull together some more closely interactive way of
| > trying creative patterns of calls thru the vfs layer?
| 
| >From the linux-fsdevel 2003-10-07 archives "Re: fs test suite", we know
| possibilities, perhaps sorted now from most to least accessible,
| include:
| 
| http://ltp.sourceforge.net/

fsstress e.g.

| http://www.iozone.org

Lots of file I/O, little VFS metadata activity.

| http://www.connectathon.org/
| 
| http://www.codemonkey.org.uk/cruft/

fsx-linux in particular

| http://www.zip.com.au/~akpm/linux/patches/stuff/
| 
| http://www.google.com/search?q=postmark+linux

postmark typically works with many small files and does more VFS
metadata operations than a workload like IOzone or tiobench (sf.net).

| http://www.google.com/search?q=bonnie+linux
| 
| By now I can testify that at a glance these particular links do Not say
| if they point only to ways of running up in the stratospheric layer of
| open/ read/ write/ close, or not.


--
~Randy
MOTD:  Always include version info.

Re: vfs whitebox test

[Pat LaVarre]

> gdb /usr/src/linux/vmlinux /proc/kcore
> 
> ... "all the structures in the kernel to be read, but no local variables
> [also] not possible to change values or call kernel functions ..."
> ...
> 
> grep CONFIG_DEBUG_INFO `find . -name 'Kconfig'`
> then flunks, but that question I can pursue in ?l=kernelnewbies.

Whoops, sorry, by now I see what works is such slightly different syntax
as:

grep -i CONFIG.SMP  `find . -name 'Kconfig'`
grep -i CONFIG.PREEMPT `find . -name 'Kconfig'`
grep -i CONFIG.DEBUG_INFO `find . -name 'Kconfig'`

For example:

$ grep -i CONFIG.USB_STORAGE_DEBUG  `find . -name 'Kconfig'`
./drivers/usb/storage/Kconfig:config USB_STORAGE_DEBUG
$

Pat LaVarre

Re: vfs whitebox test

[Pat LaVarre]

> ... ltp ... fsstress ...
> ... iozone ... Lots of file I/O, little VFS metadata activity.
> ... cruft ... fsx-linux in particular
> ... postmark ... many small files... more ... metadata ...

Guidance!  Thank you.  In particular, as I carry these links forward
hopefully now instead I will remember to say:

http://www.codemonkey.org.uk/cruft/fsx-linux.c

At first glance in fsx-linux.c I think I see command line "usage"
defined to produce open/ close/ read/ write/ lseek/ mmap/ munmap
sequences.

Pat LaVarre

P.S. Also thank you I picked up:

http://www.google.com/search?q=tiobench+sf.net
http://tiobench.sourceforge.net/
http://sourceforge.net/projects/tiobench/

self-described as: "portable, robust, fully-threaded I/O benchmark
program".

Re: vfs whitebox test

[Pat LaVarre]

> > gdb /usr/src/linux/vmlinux /proc/kcore
> > 
> > ... "all the structures in the kernel to be read, but no local variables
> > [also] not possible to change values or call kernel functions ..."
> ...
> grep -i CONFIG.DEBUG_INFO `find . -name 'Kconfig'`

`make xconfig` --> "Kernel hacking" forces the pair:

+ CONFIG_DEBUG_KERNEL=y
+ CONFIG_DEBUG_INFO=y

Also I accepted the suggestions:

+ CONFIG_DEBUG_SLAB=y
+ CONFIG_MAGIC_SYSRQ=y

While still testing at random in hope of discovering something relevant,
I should also try the slow DEBUG_PAGEALLOC, also DEBUG_STACKOVERFLOW.

Pat LaVarre

Re: vfs whitebox test

[Pat LaVarre]

kdb and kgdb exist

Re: vfs whitebox test

[Pat LaVarre]

Kindly offline I'm reminded the man mount example:

mount /tmp/fdimage /mnt -t msdos -o loop=/dev/loop3,blocksize=1024

is a hint that we have some control over virtual block size.

Not yet enough.

Despite mkudffs -b $bs (syntax from fs/udf/super.c) and mount -o
loop,bs=$bs (syntax from man mount) still we get 512 from `sudo blockdev
--getss` e.g.

$ udfbs 0 10 sudo blockdev --getss /dev/loop0
...
+ sudo blockdev --getss /dev/loop0
512
+ sudo umount /mnt/loop0
$

Pat LaVarre

--- /dev/null	2003-01-30 03:24:37.000000000 -0700
+++ bin/udfbs	2003-12-05 11:07:13.199344360 -0700
@@ -0,0 +1,21 @@
+#!/bin/bash -x
+
+mkbs='-b 2048'
+mobs=',bs=2048'
+
+cmd="dd if=/dev/zero of=dd.bin bs=1M seek=$1 count=$2"
+echo "$cmd 2>/dev/null"
+$cmd 2>/dev/null
+shift 2
+
+sudo losetup /dev/loop0 dd.bin
+sudo mkudffs $mkbs /dev/loop0 >/dev/null
+sudo losetup -d /dev/loop0
+sudo mount -t udf dd.bin /mnt/loop0 -o loop$mobs
+sudo chown `id -g`:`id -u` /mnt/loop0/.
+
+export PS1="udfbs \$ "
+ls -l `which sh`
+"$@"
+
+sudo umount /mnt/loop0