Volume too big

Volume too big

[Jan Engelhardt]

Hi,



16:51 localhost:~ # fdisk -l /dev/md0

Disk /dev/md0: 60183.2 GB, 60183225368576 bytes
2 heads, 4 sectors/track, -1 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

16:51 localhost:~ # mkfs.xfs -ff /dev/md0
meta-data=/dev/md0               isize=256    agcount=55, agsize=268435455 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=14693170256, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal log           bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
list too long)
mount: Argument list too long

16:51 localhost:~ # uname -a
Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
i686 athlon i386 GNU/Linux

CONFIG_LBD=y. Do I need an extra flag for mkfs?

Re: Volume too big

[Eric Sandeen]

Jan Engelhardt wrote:
> Hi,
> 
> 
> 
> 16:51 localhost:~ # fdisk -l /dev/md0
> 
> Disk /dev/md0: 60183.2 GB, 60183225368576 bytes
> 2 heads, 4 sectors/track, -1 cylinders
> Units = cylinders of 8 * 512 = 4096 bytes
> Disk identifier: 0x00000000
> 
> Disk /dev/md0 doesn't contain a valid partition table
> 
> 16:51 localhost:~ # mkfs.xfs -ff /dev/md0
> meta-data=/dev/md0               isize=256    agcount=55, agsize=268435455 blks
>          =                       sectsz=512   attr=0
> data     =                       bsize=4096   blocks=14693170256, imaxpct=25
>          =                       sunit=0      swidth=0 blks, unwritten=1
> naming   =version 2              bsize=4096  
> log      =internal log           bsize=4096   blocks=32768, version=1
>          =                       sectsz=512   sunit=0 blks, lazy-count=0
> realtime =none                   extsz=4096   blocks=0, rtextents=0
> 
> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
> list too long)
> mount: Argument list too long
> 
> 16:51 localhost:~ # uname -a
> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
> i686 athlon i386 GNU/Linux
> 
> CONFIG_LBD=y. Do I need an extra flag for mkfs?

Nope; this is probably that you can't do > 16T on a 32 bit box (core
linux restriction, pretty much)

dmesg would also yield more information about the cause of the mount
failure.

(Hm, argument list too long is an interesting result tho...)

get an x86_64, I think :)

-Eric

Re: Volume too big

[Justin Piszcz]

On Sat, 19 Jan 2008, Eric Sandeen wrote:

> Jan Engelhardt wrote:
>> Hi,
>>
>>
>>
>> 16:51 localhost:~ # fdisk -l /dev/md0
>>
>> Disk /dev/md0: 60183.2 GB, 60183225368576 bytes
>> 2 heads, 4 sectors/track, -1 cylinders
>> Units = cylinders of 8 * 512 = 4096 bytes
>> Disk identifier: 0x00000000
>>
>> Disk /dev/md0 doesn't contain a valid partition table
>>
>> 16:51 localhost:~ # mkfs.xfs -ff /dev/md0
>> meta-data=/dev/md0               isize=256    agcount=55, agsize=268435455 blks
>>          =                       sectsz=512   attr=0
>> data     =                       bsize=4096   blocks=14693170256, imaxpct=25
>>          =                       sunit=0      swidth=0 blks, unwritten=1
>> naming   =version 2              bsize=4096
>> log      =internal log           bsize=4096   blocks=32768, version=1
>>          =                       sectsz=512   sunit=0 blks, lazy-count=0
>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>>
>> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
>> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument
>> list too long)
>> mount: Argument list too long
>>
>> 16:51 localhost:~ # uname -a
>> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC
>> i686 athlon i386 GNU/Linux
>>
>> CONFIG_LBD=y. Do I need an extra flag for mkfs?
>
> Nope; this is probably that you can't do > 16T on a 32 bit box (core
> linux restriction, pretty much)
>
> dmesg would also yield more information about the cause of the mount
> failure.
>
> (Hm, argument list too long is an interesting result tho...)
>
> get an x86_64, I think :)
>
> -Eric
>
>

Is that on one of those thumper boxes?

What type of HW?

Justin.

Re: Volume too big

[Jan Engelhardt]

On Jan 19 2008 12:29, Justin Piszcz wrote:
>>
>> Disk /dev/md0: 60183.2 GB, 60183225368576 bytes
>
> Is that on one of those thumper boxes?
> What type of HW?

A bunch of "VMware Virtual SCSI Disk" in sparse mode.

Re: Volume too big

[Jan Engelhardt]

On Jan 19 2008 10:15, Eric Sandeen wrote:
>> 
>> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
>> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
>> list too long)
>> mount: Argument list too long
>> 
>> 16:51 localhost:~ # uname -a
>> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
>> i686 athlon i386 GNU/Linux
>> 
>> CONFIG_LBD=y. Do I need an extra flag for mkfs?
>
>Nope; this is probably that you can't do > 16T on a 32 bit box (core
>linux restriction, pretty much)
>get an x86_64, I think :)

Hm, JFS fails too.
But btrfs 0.11 can mount it.

>dmesg would also yield more information about the cause of the mount
>failure.

XFS: file system too large to be mounted on this system
XFS: SB validate fialed

>(Hm, argument list too long is an interesting result tho...)

I suppose it's right here:

#if XFS_BIG_BLKNOS     /* Limited by ULONG_MAX of page cache index */ 
        if (nblocks >> (PAGE_CACHE_SHIFT - sbp->sb_blocklog) > ULONG_MAX)
                return E2BIG;
#else                  /* Limited by UINT_MAX of sectors */ 

Re: Volume too big

[Eric Sandeen]

Jan Engelhardt wrote:

>> (Hm, argument list too long is an interesting result tho...)
> 
> I suppose it's right here:
> 
> #if XFS_BIG_BLKNOS     /* Limited by ULONG_MAX of page cache index */ 
>         if (nblocks >> (PAGE_CACHE_SHIFT - sbp->sb_blocklog) > ULONG_MAX)
>                 return E2BIG;
> #else                  /* Limited by UINT_MAX of sectors */ 
> 

Yeah, I really think the E2BIGs in xfs should probably be swapped for
EFBIG; more helpful error message.

-Eric

Re: Volume too big

[Eric Sandeen]

Jan Engelhardt wrote:
> On Jan 19 2008 10:15, Eric Sandeen wrote:
>>> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
>>> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
>>> list too long)
>>> mount: Argument list too long
>>>
>>> 16:51 localhost:~ # uname -a
>>> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
>>> i686 athlon i386 GNU/Linux
>>>
>>> CONFIG_LBD=y. Do I need an extra flag for mkfs?
>> Nope; this is probably that you can't do > 16T on a 32 bit box (core
>> linux restriction, pretty much)
>> get an x86_64, I think :)
> 
> Hm, JFS fails too.
> But btrfs 0.11 can mount it.

It's possible that btrfs can cope with this somehow - but also quite
possible that it's just missing the right checks :)

-Eric

Re: Volume too big

[Jan Engelhardt]

On Jan 19 2008 14:41, Eric Sandeen wrote:
>Jan Engelhardt wrote:
>> On Jan 19 2008 10:15, Eric Sandeen wrote:
>>>> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
>>>> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
>>>> list too long)
>>>> mount: Argument list too long
>>>>
>>>> 16:51 localhost:~ # uname -a
>>>> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
>>>> i686 athlon i386 GNU/Linux
>>>>
>>>> CONFIG_LBD=y. Do I need an extra flag for mkfs?
>>> Nope; this is probably that you can't do > 16T on a 32 bit box (core
>>> linux restriction, pretty much)
>>> get an x86_64, I think :)
>> 
>> Hm, JFS fails too.
>> But btrfs 0.11 can mount it.
>
>It's possible that btrfs can cope with this somehow - but also quite
>possible that it's just missing the right checks :)
>
I am not sure why Linux would be limited to 16 TB. If LBD is on,
things are 64 bit, so I would expect to have at least access to
2 exabyte.

Re: Volume too big

[Eric Sandeen]

Jan Engelhardt wrote:
> On Jan 19 2008 14:41, Eric Sandeen wrote:

>> It's possible that btrfs can cope with this somehow - but also quite
>> possible that it's just missing the right checks :)
>>
> I am not sure why Linux would be limited to 16 TB. If LBD is on,
> things are 64 bit, so I would expect to have at least access to
> 2 exabyte.

64-bit sector addressing, but there is a 32-bit index into the (4k)
pagecache.

2^32 * 4096 is 16T

So an address space has a 16T limit.

Even mkfs, if it needs to write past 16T (and I think mkfs.btrfs doesn't
need that...) will have trouble, if the device is > 16T - unless mkfs
uses direct IO.

-Eric

Re: Volume too big

[Eric Sandeen]

Eric Sandeen wrote:
> Jan Engelhardt wrote:
>> On Jan 19 2008 10:15, Eric Sandeen wrote:
>>>> 16:51 localhost:~ # strace -e mount mount /dev/md0 /mnt
>>>> mount("/dev/md0", "/mnt", "xfs", MS_MGC_VAL, NULL) = -1 E2BIG (Argument 
>>>> list too long)
>>>> mount: Argument list too long
>>>>
>>>> 16:51 localhost:~ # uname -a
>>>> Linux localhost 2.6.23.14-ccj63-regular #1 SMP 2007/10/26 14:17:15 UTC 
>>>> i686 athlon i386 GNU/Linux
>>>>
>>>> CONFIG_LBD=y. Do I need an extra flag for mkfs?
>>> Nope; this is probably that you can't do > 16T on a 32 bit box (core
>>> linux restriction, pretty much)
>>> get an x86_64, I think :)
>> Hm, JFS fails too.
>> But btrfs 0.11 can mount it.
> 
> It's possible that btrfs can cope with this somehow - but also quite
> possible that it's just missing the right checks :)

re: btrfs...

[15:50]  <mason> sandeen_: mkfs will work on >16TB, but the limiting
factor is the 32 bit radix tree ops on the page cache
[15:50]  <mason> so yes, it is missing a check ;)

:)

-Eric