mkfs.xfs doesn't detect size of storage correctly

mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

Hi


Yesterday i bought a 750GB HDD.
I encrypt nearly everything with loop-aes, so i also did it with this 
HDD.

I create a "fake" partition table and:
losetup -e aes256 -p 0 -o 4096 /dev/loop6 /dev/sdb < key

This creates a loop with everything except the first 4KB, i.e. it leaves 
out the MBR and another 3,5KB.

/proc/partions shows the correct(tm) size informations for the HDD and 
the loop:
- snip -
   7     6  732574580 loop6
   8    16  732574584 sdb
   8    17  732572001 sdb1
- snip -

But when i mkfs.xfs the loop
#> mkfs.xfs /dev/loop6
meta-data=/dev/loop6             isize=256    agcount=3, agsize=45785911 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=137357733, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=32768, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

And mount it:
mount /dev/loop6 /mnt

And least but not least df it:
#> df -m /mnt
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/loop6              536426         5    536422   1% /mnt

There is roughly 1/3 missing.

What can i do to fix this?


#> mkfs.xfs -V
mkfs.xfs version 2.9.5
(Package is from: Debian SID/unstable)

Kernel is 2.6.24 (vanilla, plus of course the external module: loop-aes)


Btw. I have 50 other HDDs in more or less exactly the same setup, (49 
HDDs >= 200 GB and <= 500GB and one 1TB, of couse created with my system 
at an older stage from Feb 2007 onwoard, the 750GB is the first with 
kernel 2.6.24.) Up to now i can't remember ever "missing" space.





Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[nscott]

> Hi
>
>
> Yesterday i bought a 750GB HDD.
> I encrypt nearly everything with loop-aes, so i also did it with this
> HDD.
>
> I create a "fake" partition table and:
> losetup -e aes256 -p 0 -o 4096 /dev/loop6 /dev/sdb < key
>
> This creates a loop with everything except the first 4KB, i.e. it leaves
> out the MBR and another 3,5KB.
>
> /proc/partions shows the correct(tm) size informations for the HDD and
> the loop:
> - snip -
>    7     6  732574580 loop6
>    8    16  732574584 sdb
>    8    17  732572001 sdb1
> - snip -
>
> But when i mkfs.xfs the loop
> #> mkfs.xfs /dev/loop6
> meta-data=/dev/loop6             isize=256    agcount=3, agsize=45785911
> blks
>          =                       sectsz=512   attr=2
> data     =                       bsize=4096   blocks=137357733, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096
> log      =internal log           bsize=4096   blocks=32768, version=2
>          =                       sectsz=512   sunit=0 blks, lazy-count=0
> realtime =none                   extsz=4096   blocks=0, rtextents=0
>
> And mount it:
> mount /dev/loop6 /mnt
>
> And least but not least df it:
> #> df -m /mnt
> Filesystem           1M-blocks      Used Available Use% Mounted on
> /dev/loop6              536426         5    536422   1% /mnt
>
> There is roughly 1/3 missing.
>
> What can i do to fix this?

mkfs.xfs uses the BLKGETSIZE64 ioctl to extract the device size, so
the problem is likely in the loop device driver (just a guess).  You
can use the test program xfs-cmds/xfstests/src/getdevicesize.c to
test what that device returns as its size (no XFS-specific code in the
test program, so if it returns bad data we've narrowed down the root
cause a whole lot).

What does that program produce for your device?

cheers.

--
Nathan

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

On 29.01.2008 21:15, nscott@aconex.com wrote:
> > Hi
> >
> >
> > Yesterday i bought a 750GB HDD.
> > I encrypt nearly everything with loop-aes, so i also did it with this
> > HDD.
> >
> > I create a "fake" partition table and:
> > losetup -e aes256 -p 0 -o 4096 /dev/loop6 /dev/sdb < key
> >
> > This creates a loop with everything except the first 4KB, i.e. it leaves
> > out the MBR and another 3,5KB.
> >
> > /proc/partions shows the correct(tm) size informations for the HDD and
> > the loop:
> > - snip -
> >    7     6  732574580 loop6
> >    8    16  732574584 sdb
> >    8    17  732572001 sdb1
> > - snip -
> >
> > But when i mkfs.xfs the loop
> > #> mkfs.xfs /dev/loop6
> > meta-data=/dev/loop6             isize=256    agcount=3, agsize=45785911
> > blks
> >          =                       sectsz=512   attr=2
> > data     =                       bsize=4096   blocks=137357733, imaxpct=25
> >          =                       sunit=0      swidth=0 blks
> > naming   =version 2              bsize=4096
> > log      =internal log           bsize=4096   blocks=32768, version=2
> >          =                       sectsz=512   sunit=0 blks, lazy-count=0
> > realtime =none                   extsz=4096   blocks=0, rtextents=0
> >
> > And mount it:
> > mount /dev/loop6 /mnt
> >
> > And least but not least df it:
> > #> df -m /mnt
> > Filesystem           1M-blocks      Used Available Use% Mounted on
> > /dev/loop6              536426         5    536422   1% /mnt
> >
> > There is roughly 1/3 missing.
> >
> > What can i do to fix this?
> 
> mkfs.xfs uses the BLKGETSIZE64 ioctl to extract the device size, so
> the problem is likely in the loop device driver (just a guess).  You
> can use the test program xfs-cmds/xfstests/src/getdevicesize.c to
> test what that device returns as its size (no XFS-specific code in the
> test program, so if it returns bad data we've narrowed down the root
> cause a whole lot).
> 
> What does that program produce for your device?

After a little odyssey to actually finding getdevicesize.c :-)

#> wget 'http://oss.sgi.com/cgi-bin/cvsweb.cgi/~checkout~/xfs-cmds/xfstests/src/getdevicesize.c?rev=1.3;content-type=text%2Fplain' -O getdevicesize.c
#> gcc -o getdevicesize getdevicesize.c
#> ./getdevicesize /dev/loop6
1465149160 512 byte blocks (BLKGETSIZE64)

Which is 750156369920 bytes or 732574580 KBytes, IOW EXACTLY the number 
reported in /proc/partions for the loop.



A maybe important detail, i forgot to mention in the original mail is:
The machine has 8GB of RAM, so i compiled the kernel with "64bit=yes" 
(formaly x86_64), BUT(!) the userspace is 32bit or plain old i386.

I only need the RAM for a large tmpfs. So i didn't reinstall the machine 
with a 64bit userspace when i switched the hardware from a Dual P3 with 
3GB to a Core2Duo with 8GB.




Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Matthias Schniedermeyer wrote:

> A maybe important detail, i forgot to mention in the original mail is:
> The machine has 8GB of RAM, so i compiled the kernel with "64bit=yes" 
> (formaly x86_64), BUT(!) the userspace is 32bit or plain old i386.

If you think that's the issue, I'd try compiling the above as a 32-bit
binary and re-run.

Although, for a simple sparse-file-backed loopback device w/o loop-aes,
that doesn't cause any problem for me:

[root@inode tmp]# ls -l loopfile
-rw-r--r-- 1 root root 750156369920 2008-01-29 08:53 loopfile

[root@inode tmp]# losetup /dev/loop0 loopfile

[root@inode tmp]# ./gds32 /dev/loop0
1465149160 512 byte blocks (BLKGETSIZE64)
[root@inode tmp]# ./gds64 /dev/loop0
1465149160 512 byte blocks (BLKGETSIZE64)

-Eric

> Bis denn
> 

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

On 29.01.2008 09:19, Eric Sandeen wrote:
> Matthias Schniedermeyer wrote:
> 
> > A maybe important detail, i forgot to mention in the original mail is:
> > The machine has 8GB of RAM, so i compiled the kernel with "64bit=yes" 
> > (formaly x86_64), BUT(!) the userspace is 32bit or plain old i386.
> 
> If you think that's the issue, I'd try compiling the above as a 32-bit
> binary and re-run.

I don't think it's an issue, i just wanted to mention it.

But as soon as i get home i will connect the HDD to an all 32bit system 
and see if it makes a difference. (I'm still undecided if i should hope 
that it makes a difference. ;-) )




Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Matthias Schniedermeyer wrote:
> On 29.01.2008 09:19, Eric Sandeen wrote:
>> Matthias Schniedermeyer wrote:
>>
>>> A maybe important detail, i forgot to mention in the original mail is:
>>> The machine has 8GB of RAM, so i compiled the kernel with "64bit=yes" 
>>> (formaly x86_64), BUT(!) the userspace is 32bit or plain old i386.
>> If you think that's the issue, I'd try compiling the above as a 32-bit
>> binary and re-run.
> 
> I don't think it's an issue, i just wanted to mention it.
> 
> But as soon as i get home i will connect the HDD to an all 32bit system 
> and see if it makes a difference. (I'm still undecided if i should hope 
> that it makes a difference. ;-) )
> 

well, this can't be hard to find, with a bit of investigation.  :)

As Nathan said, mkfs.xfs just uses standard ioctls to get device size,
so something is odd... you could sprinkle some printfs into the mkfs
code, too, to further debug.

-Eric

> 
> 
> Bis denn
> 

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

On 29.01.2008 09:42, Eric Sandeen wrote:
> Matthias Schniedermeyer wrote:
> > On 29.01.2008 09:19, Eric Sandeen wrote:
> >> Matthias Schniedermeyer wrote:
> >>
> >>> A maybe important detail, i forgot to mention in the original mail is:
> >>> The machine has 8GB of RAM, so i compiled the kernel with "64bit=yes" 
> >>> (formaly x86_64), BUT(!) the userspace is 32bit or plain old i386.
> >> If you think that's the issue, I'd try compiling the above as a 32-bit
> >> binary and re-run.
> > 
> > I don't think it's an issue, i just wanted to mention it.
> > 
> > But as soon as i get home i will connect the HDD to an all 32bit system 
> > and see if it makes a difference. (I'm still undecided if i should hope 
> > that it makes a difference. ;-) )

I'm back home, and i just connected the HDD to my all 32bit system.
No difference.






Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

On 29.01.2008 10:32, Matthias Schniedermeyer wrote:
> Hi
> 
> 
> Yesterday i bought a 750GB HDD.
> I encrypt nearly everything with loop-aes, so i also did it with this 
> HDD.
> 
> I create a "fake" partition table and:
> losetup -e aes256 -p 0 -o 4096 /dev/loop6 /dev/sdb < key
> 
> This creates a loop with everything except the first 4KB, i.e. it leaves 
> out the MBR and another 3,5KB.
> 
> /proc/partions shows the correct(tm) size informations for the HDD and 
> the loop:
> - snip -
>    7     6  732574580 loop6
>    8    16  732574584 sdb
>    8    17  732572001 sdb1
> - snip -
> 
> But when i mkfs.xfs the loop
> #> mkfs.xfs /dev/loop6
> meta-data=/dev/loop6             isize=256    agcount=3, agsize=45785911 blks
>          =                       sectsz=512   attr=2
> data     =                       bsize=4096   blocks=137357733, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096
> log      =internal log           bsize=4096   blocks=32768, version=2
>          =                       sectsz=512   sunit=0 blks, lazy-count=0
> realtime =none                   extsz=4096   blocks=0, rtextents=0

I just found a workaround. :-)
As can be seen above the agcount is 3.

For a reason that lays years in the past, when there were issues with 
the agcount (actually agsize), that to the best of my knowledge are 
fixed years ago (but still cause a weird feeling whenever i see that 
word) i just tried '-d agcount=4'

#> mkfs.xfs -l size=1024b -d agcount=4 /dev/loop6 -f
meta-data=/dev/loop6             isize=256    agcount=4, agsize=45785912 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=183143645, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=32768, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

As can be seen the "blocks"-number is much higher.

#> df -k /mnt
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/loop6           732570484      4256 732566228   1% /mnt

matches(tm) with the loop-size.


So i'm "burned" by agcount/agsize AGAIN, seems my weird feeling about 
those two words are still with reason. ;-)






Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Matthias Schniedermeyer wrote:
> I just found a workaround. :-)
> As can be seen above the agcount is 3.
> 
> For a reason that lays years in the past, when there were issues with 
> the agcount (actually agsize), that to the best of my knowledge are 
> fixed years ago (but still cause a weird feeling whenever i see that 
> word) i just tried '-d agcount=4'
> 
> #> mkfs.xfs -l size=1024b -d agcount=4 /dev/loop6 -f
> meta-data=/dev/loop6             isize=256    agcount=4, agsize=45785912 blks
>          =                       sectsz=512   attr=2
> data     =                       bsize=4096   blocks=183143645, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096
> log      =internal log           bsize=4096   blocks=32768, version=2
>          =                       sectsz=512   sunit=0 blks, lazy-count=0
> realtime =none                   extsz=4096   blocks=0, rtextents=0
> 
> As can be seen the "blocks"-number is much higher.
> 
> #> df -k /mnt
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/loop6           732570484      4256 732566228   1% /mnt
> 
> matches(tm) with the loop-size.
> 

Ohh.. um... Barry?  problems :)

[root@magnesium tmp]# mkfs.xfs -dfile,name=xfsfile,size=750156369920
meta-data=xfsfile                isize=256    agcount=3, agsize=45785911
blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=137357733, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=32768, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@magnesium tmp]# bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
137357733*4096
562617274368

ouch... that's, um, not the right size :)

-Eric

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Chris Wedgwood wrote:
> On Tue, Jan 29, 2008 at 02:14:50PM -0600, Eric Sandeen wrote:
>
>   
>> [root@magnesium tmp]# mkfs.xfs -dfile,name=xfsfile,size=750156369920
>>     
>
> I see this too except I'm pre-creating a file of the right length:
>
> -rw-r--r-- 1 root root 750156369920 Jan 29 12:35 xfsfile
>
> [...]
>
> /root/xfsfile  xfs    524G  4.2M  524G   1% /mnt/tmp
>
>   
I think this fixes it, but it was just a quickie so pls double check :)

if agcount * agsize <= dblocks, then the last ag shouldn't be "small"

-Eric

Index: xfsprogs-2.9.5/mkfs/xfs_mkfs.c
===================================================================
--- xfsprogs-2.9.5.orig/mkfs/xfs_mkfs.c
+++ xfsprogs-2.9.5/mkfs/xfs_mkfs.c
@@ -523,7 +523,8 @@ validate_ag_geometry(
 	 * If the last AG is too small, reduce the filesystem size
 	 * and drop the blocks.
 	 */
-	if ( dblocks % agsize != 0 &&
+	if ( agcount * agsize > dblocks &&
+	     dblocks % agsize != 0 &&
 	     (dblocks % agsize < XFS_AG_MIN_BLOCKS(blocklog))) {
 		fprintf(stderr,
 	_("last AG size %lld blocks too small, minimum size is %lld blocks\n"),
@@ -1808,7 +1809,8 @@ _("size %s specified for log subvolume i
 	 * If the last AG is too small, reduce the filesystem size
 	 * and drop the blocks.
 	 */
-	if ( dblocks % agsize != 0 &&
+	if ( agcount * agsize > dblocks &&
+	     dblocks % agsize != 0 &&
 	     (dblocks % agsize < XFS_AG_MIN_BLOCKS(blocklog))) {
 		dblocks = (xfs_drfsbno_t)((agcount - 1) * agsize);
 		agcount--;

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

A little followup.

Today i bought another 750GB HDD, and a 500GB.

Initializing the encryption of the 500GB finished a minute ago and the 
same bug, of course with a different size (357702MB), was observable.

Workaround also worked in this case (Now its 476937MB). This also proves 
that the bug is of recent nature, the last 500GB HDD i bought was just 
over a month ago and this bug didn't show up then.






Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Matthias Schniedermeyer wrote:
> A little followup.
> 
> Today i bought another 750GB HDD, and a 500GB.
> 
> Initializing the encryption of the 500GB finished a minute ago and the 
> same bug, of course with a different size (357702MB), was observable.
> 
> Workaround also worked in this case (Now its 476937MB). This also proves 
> that the bug is of recent nature, the last 500GB HDD i bought was just 
> over a month ago and this bug didn't show up then.

Yes, it's a regression in 2.9.5.

http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfsprogs/mkfs/xfs_mkfs.c.diff?r1=1.82;r2=1.83

-Eric

Re: mkfs.xfs doesn't detect size of storage correctly

[Eric Sandeen]

Matthias Schniedermeyer wrote:

> There is roughly 1/3 missing.

SGI, have you settled on a fix for this?  Latest released xfsprogs,
2.9.5 has this problem, and I've put it in rawhide; it's also in the F9
alpha spin.  Should I revert to 2.9.4 if you don't have plans to fix it
soon?

Nothing indicates to the user that their fs was misformatted, and there
will be a growing number fileystems missing a substantial portion of
their space out there....

-Eric

Re: mkfs.xfs doesn't detect size of storage correctly

[Matthias Schniedermeyer]

On 05.02.2008 08:53, Eric Sandeen wrote:
> Matthias Schniedermeyer wrote:
> 
> > There is roughly 1/3 missing.
> 
> SGI, have you settled on a fix for this?  Latest released xfsprogs,
> 2.9.5 has this problem, and I've put it in rawhide; it's also in the F9
> alpha spin.  Should I revert to 2.9.4 if you don't have plans to fix it
> soon?
> 
> Nothing indicates to the user that their fs was misformatted, and there
> will be a growing number fileystems missing a substantial portion of
> their space out there....

The big question here is if you can grow the filesystem to fix it.

Before i found the workaround with the agcount i tested to grow the fs, 
but it didn't work.

So: A correctly(tm) fixed package i think it must include a xfs_grows 
that can fix a damaged(tm) fs, otherwise people who use such a 
filesystem (and can't backup/restore) are stuck with missing a large 
chunk of space.




Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Re: mkfs.xfs doesn't detect size of storage correctly

[Mark Goodwin]

Eric Sandeen wrote:
> Matthias Schniedermeyer wrote:
> 
>> There is roughly 1/3 missing.
> 
> SGI, have you settled on a fix for this? 

Yes, Barry is reviewing and testing your proposed fix. The delay was caused
by most of the XFS team attending LCA last week. .. sorry.

We'll get a new package out onto OSS asap.

Cheers
-- Mark


> Latest released xfsprogs,
> 2.9.5 has this problem, and I've put it in rawhide; it's also in the F9
> alpha spin.  Should I revert to 2.9.4 if you don't have plans to fix it
> soon?
> 
> Nothing indicates to the user that their fs was misformatted, and there
> will be a growing number fileystems missing a substantial portion of
> their space out there....
> 
> -Eric
> 
>