create very large file system

create very large file system

[Mark F]

Suse Linux Enterprise Server 9 SP3

I've tried to create a large 5TB file system using both reiserfs and ext3 and both have failed.

I end up with only a 1.5TB file system.  Does anyone know why this doesn't work, what to do to fix it?

Others have suggested that  only XFS or JFS will work.  Is this so?

Thanks,
-Mark

Re: create very large file system

[Alexander Zarochentsev]

On Wednesday 19 July 2006 16:10, Mark F wrote:
> Suse Linux Enterprise Server 9 SP3
>
> I've tried to create a large 5TB file system using both reiserfs and
> ext3 and both have failed.

how did they fail?

>
> I end up with only a 1.5TB file system.  Does anyone know why this
> doesn't work, what to do to fix it?

you have a single 5TB device? h/w raid, I think ?

you might need to convert the partition table to GPT format for 
supporting 2TB+ partitions.  it can be done by the gnu parted tool.

> Others have suggested that  only XFS or JFS will work.  Is this so?
>
> Thanks,
> -Mark


-- 
Alex.

Re: create very large file system

[Mark F]

Alexander Zarochentsev wrote:
> On Wednesday 19 July 2006 16:10, Mark F wrote:
>> Suse Linux Enterprise Server 9 SP3
>>
>> I've tried to create a large 5TB file system using both reiserfs and
>> ext3 and both have failed.
> 
> how did they fail?
> 
When I create the partition using cfdisk it appeared to be created correctly, now I have discovered that it is not.

I thought the problem was with the mkfs.<filesystem> commands but it looks like it is a bit deeper than that.

>> I end up with only a 1.5TB file system.  Does anyone know why this
>> doesn't work, what to do to fix it?
> 
> you have a single 5TB device? h/w raid, I think ?

Yes, Orion 12S RAID Array.  12x500GB drives

> 
> you might need to convert the partition table to GPT format for 
> supporting 2TB+ partitions.  it can be done by the gnu parted tool.
> 

Thanks, I don't know what GPT format is but I'll try to figure it out, I'm compiling/installing gparted right now.

-Mark

>> Others have suggested that  only XFS or JFS will work.  Is this so?
>>
>> Thanks,
>> -Mark
> 
> 

Re: create very large file system

[Andreas Dilger]

On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
> On Wednesday 19 July 2006 16:10, Mark F wrote:
> > I've tried to create a large 5TB file system using both reiserfs and
> > ext3 and both have failed.
> 
> you might need to convert the partition table to GPT format for 
> supporting 2TB+ partitions.  it can be done by the gnu parted tool.

Or, for that matter, don't use a partition table at all, since this
adds an unhelpful offset to all the filesystem structures and can
hurt performance on RAID where the filesystem is trying to align IO
to RAID stripe boundaries.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: create very large file system

[Clay Barnes]

On 02:26 Thu 20 Jul     , Andreas Dilger wrote:
> On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
> > On Wednesday 19 July 2006 16:10, Mark F wrote:
> > > I've tried to create a large 5TB file system using both reiserfs and
> > > ext3 and both have failed.
> > 
> > you might need to convert the partition table to GPT format for 
> > supporting 2TB+ partitions.  it can be done by the gnu parted tool.
> 
> Or, for that matter, don't use a partition table at all, since this
> adds an unhelpful offset to all the filesystem structures and can
> hurt performance on RAID where the filesystem is trying to align IO
> to RAID stripe boundaries.
Well, damn.  Too late for that to help me.  I've a 700GB that's...
rather too full to put on DVDs or my other drives to fix that.  There
isn't some "safe" way to remove a partition table from a SW RAID array,
is there?

--Clay
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.

Re: create very large file system

[David Masover]

Clay Barnes wrote:
> On 02:26 Thu 20 Jul     , Andreas Dilger wrote:
>> On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
>>> On Wednesday 19 July 2006 16:10, Mark F wrote:
>>>> I've tried to create a large 5TB file system using both reiserfs and
>>>> ext3 and both have failed.
>>> you might need to convert the partition table to GPT format for 
>>> supporting 2TB+ partitions.  it can be done by the gnu parted tool.
>> Or, for that matter, don't use a partition table at all, since this
>> adds an unhelpful offset to all the filesystem structures and can
>> hurt performance on RAID where the filesystem is trying to align IO
>> to RAID stripe boundaries.
> Well, damn.  Too late for that to help me.  I've a 700GB that's...
> rather too full to put on DVDs or my other drives to fix that.  There
> isn't some "safe" way to remove a partition table from a SW RAID array,
> is there?

Define "safe".  You could probably do it non-destructively with dd, if 
you set the block size properly.  That doesn't make it safe, though, 
you're basically screwed if you lose power.

And even once you do that, I don't think the Reiser4 tools would let you 
resize.

Anyway, I haven't tried this myself, as my SW RAID does actually have 
multiple partitions, for good reason.

Re: create very large file system

[Christian Iversen]

On Thursday 20 July 2006 08:26, Andreas Dilger wrote:
> On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
> > On Wednesday 19 July 2006 16:10, Mark F wrote:
> > > I've tried to create a large 5TB file system using both reiserfs and
> > > ext3 and both have failed.
> >
> > you might need to convert the partition table to GPT format for
> > supporting 2TB+ partitions.  it can be done by the gnu parted tool.
>
> Or, for that matter, don't use a partition table at all, since this
> adds an unhelpful offset to all the filesystem structures and can
> hurt performance on RAID where the filesystem is trying to align IO
> to RAID stripe boundaries.

Can linux still auto-detect raid volumes if there's no partition table?

-- 
Regards,
Christian Iversen

Re: create very large file system

[Andreas Dilger]

On Jul 20, 2006  13:17 +0200, Christian Iversen wrote:
> On Thursday 20 July 2006 08:26, Andreas Dilger wrote:
> > On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
> > > On Wednesday 19 July 2006 16:10, Mark F wrote:
> > > > I've tried to create a large 5TB file system using both reiserfs and
> > > > ext3 and both have failed.
> > >
> > > you might need to convert the partition table to GPT format for
> > > supporting 2TB+ partitions.  it can be done by the gnu parted tool.
> >
> > Or, for that matter, don't use a partition table at all, since this
> > adds an unhelpful offset to all the filesystem structures and can
> > hurt performance on RAID where the filesystem is trying to align IO
> > to RAID stripe boundaries.
> 
> Can linux still auto-detect raid volumes if there's no partition table?

Hmm, that I'm not sure of - we mostly deal with external RAID devices.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: create very large file system

[Jonathan Briggs]

[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]

On Thu, 2006-07-20 at 13:17 +0200, Christian Iversen wrote:
> On Thursday 20 July 2006 08:26, Andreas Dilger wrote:
> > On Jul 19, 2006  16:57 +0400, Alexander Zarochentsev wrote:
> > > On Wednesday 19 July 2006 16:10, Mark F wrote:
> > > > I've tried to create a large 5TB file system using both reiserfs and
> > > > ext3 and both have failed.
> > >
> > > you might need to convert the partition table to GPT format for
> > > supporting 2TB+ partitions.  it can be done by the gnu parted tool.
> >
> > Or, for that matter, don't use a partition table at all, since this
> > adds an unhelpful offset to all the filesystem structures and can
> > hurt performance on RAID where the filesystem is trying to align IO
> > to RAID stripe boundaries.
> 
> Can linux still auto-detect raid volumes if there's no partition table?

You're not supposed to be doing it that way these days.  RAID autodetect
is getting tossed out of the kernel in the future (probably still many
versions away though), and RAID, DM, LVM, and maybe even regular
partition setup is going to be done in initramfs / initrd.

At least, that is what I read.
-- 
Jonathan Briggs <jbriggs@esoft.com>
eSoft, Inc.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: create very large file system

[Avuton Olrich]

On 7/20/06, Jonathan Briggs <jbriggs@esoft.com> wrote:
> You're not supposed to be doing it that way these days.  RAID autodetect
> is getting tossed out of the kernel in the future (probably still many

Bit OT, but is there something that is supposed to replace RAID
autodetect, or we're just supposed to make initscripts to run mdadm?
-- 
avuton
--
 Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: create very large file system

[David Masover]

Avuton Olrich wrote:
> On 7/20/06, Jonathan Briggs <jbriggs@esoft.com> wrote:
>> You're not supposed to be doing it that way these days.  RAID autodetect
>> is getting tossed out of the kernel in the future (probably still many
> 
> Bit OT, but is there something that is supposed to replace RAID
> autodetect, or we're just supposed to make initscripts to run mdadm?

I'm not sure, but I suspect it may have something to do with the 
device-mapper.  I know that's what I use for a striped array -- and yes, 
I have to make an initscript -- initrd, because I boot off the thing -- 
to set them up.  The good news is, the detection can be done in userland 
now -- since mine is nvidia raid, I can use dmraid, which runs entirely 
in userland, autodetects from BIOS settings, then sets up a proper 
device-mapper table in the kernel.

Device-mapper doesn't support everything yet (as far as I know), but the 
principle should be about the same for more complex RAID -- some 
userland tool.  There really is no reason for this autodetection to be 
in the kernel.

In the meantime, there may be rough edges...

Re: create very large file system

[Stephen Tweedie]

Hi,

On Wed, Jul 19, 2006 at 07:10:56AM -0500, Mark F wrote:
> Suse Linux Enterprise Server 9 SP3
> 
> I've tried to create a large 5TB file system using both reiserfs and ext3 
> and both have failed.
> 
> I end up with only a 1.5TB file system.  Does anyone know why this doesn't 
> work, what to do to fix it?

I fixed a bug in mke2fs that had this result over a year ago, so a
recent e2fsprogs should fix it.

Failing that, there's a workaround: use "mke2fs -b 4096" to prevent
mke2fs from trying to work out the device size in units of 1k blocks.
Counting in 4k blocks prevents a 32-bit overflow.

--Stephen

Re: create very large file system

[Mark F]

Stephen Tweedie wrote:
> Hi,
> 
> On Wed, Jul 19, 2006 at 07:10:56AM -0500, Mark F wrote:
>> Suse Linux Enterprise Server 9 SP3
>>
>> I've tried to create a large 5TB file system using both reiserfs and ext3 
>> and both have failed.
>>
>> I end up with only a 1.5TB file system.  Does anyone know why this doesn't 
>> work, what to do to fix it?
> 
> I fixed a bug in mke2fs that had this result over a year ago, so a
> recent e2fsprogs should fix it.
> 
> Failing that, there's a workaround: use "mke2fs -b 4096" to prevent
> mke2fs from trying to work out the device size in units of 1k blocks.
> Counting in 4k blocks prevents a 32-bit overflow.
> 
> --Stephen

Thanks,

I finally got it formated using the GPT label with parted.  I formatted it 
reiserfs, it takes a few seconds to mount but seems to work fine and shows up at 
full size.

-Mark