Minimum device size of 256 MiB?

Minimum device size of 256 MiB?

[Berke Durak]

Hello,

There seems to be a 256 MiB lower limit on device size : mkfs.btrfs
refuses to create a filesystem
on a device that is smaller than that.

I'm interested in using the btrfs RAID facility to provide redundancy
for an embedded system, but I can't afford
two 256 MiB partitions.

What are the structural reasons for this limit?  Could I reduce this
limit easily with a couple small code changes
or would it take a major effort?

Thanks,
-- 
Berke Durak

Re: Minimum device size of 256 MiB?

[Aaron Toponce]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

On Fri, May 11, 2012 at 03:57:24PM -0400, Berke Durak wrote:
> There seems to be a 256 MiB lower limit on device size : mkfs.btrfs
> refuses to create a filesystem on a device that is smaller than that.

I've noticed the same. I'm interested in researching the patterns the
filesystem puts down on an encrypted container, but would like to use 1MB
files as the block device for the filesystem. Looking for patterns in 256MB
files is too expensive.

--
. o .   o . o   . . o   o . .   . o .
. . o   . o o   o . o   . o o   . . o
o o o   . o .   . o o   o o .   o o o

[-- Attachment #2: Type: application/pgp-signature, Size: 519 bytes --]

Re: Minimum device size of 256 MiB?

[Berke Durak]

I think I may have found a workaround.

> One very interesting use of dm-zero is for creating "sparse" devices =
in
conjunction with dm-snapshot. A sparse device reports a device-size
larger than the amount of actual storage space
available for that device. A user can write data anywhere within the
sparse device and read it back like a normal
device. Reads to previously unwritten areas will return a zero'd
buffer. When enough data has been written to fill up
the actual storage space, the sparse device is deactivated. This can
be very useful for testing device and filesystem
limitations.

I don't know if that would be useful for your crypto signature analysis=
=2E

On Fri, May 11, 2012 at 4:50 PM, Aaron Toponce <aaron.toponce@gmail.com=
> wrote:
> On Fri, May 11, 2012 at 03:57:24PM -0400, Berke Durak wrote:
>> There seems to be a 256 MiB lower limit on device size : mkfs.btrfs
>> refuses to create a filesystem on a device that is smaller than that=
=2E
>
> I've noticed the same. I'm interested in researching the patterns the
> filesystem puts down on an encrypted container, but would like to use=
 1MB
> files as the block device for the filesystem. Looking for patterns in=
 256MB
> files is too expensive.
>
> --
> . o . =C2=A0 o . o =C2=A0 . . o =C2=A0 o . . =C2=A0 . o .
> . . o =C2=A0 . o o =C2=A0 o . o =C2=A0 . o o =C2=A0 . . o
> o o o =C2=A0 . o . =C2=A0 . o o =C2=A0 o o . =C2=A0 o o o
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Minimum device size of 256 MiB?

[Berke Durak]

On Fri, May 11, 2012 at 4:50 PM, Aaron Toponce
<aaron.toponce@gmail.com> wrote:
> I've noticed the same. I'm interested in researching the patterns
> the filesystem puts down on an encrypted container, but would like
> to use 1MB files as the block device for the filesystem. Looking for
> patterns in 256MB files is too expensive.

I think I may have found a workaround to my problem : use dm-zero to
create a sparse 256 MB device backed by a smaller partition.

I don't know if that would be useful for your crypto signature
analysis.

Of course it would be better if btrfs simply allowed smaller
partitions.

(Sorry for the previous mail it did not come out as intended.)
-- 
Berke Durak

Re: Minimum device size of 256 MiB?

[Berke Durak]

Unfortunately nothing guarantees that the footprint of the filesystem
will stay bounded.

In other words, even if the number of blocks occupied by the
filesystem at any time is less than the capacity of the physical
device backing the sparse device, the total number of blocks ever
occupied by the filesystem since its creation may exceed the capacity
of the physical device.
-- 
Berke Durak

Re: Minimum device size of 256 MiB?

[Duncan]

Aaron Toponce posted on Fri, 11 May 2012 14:50:32 -0600 as excerpted:

> On Fri, May 11, 2012 at 03:57:24PM -0400, Berke Durak wrote:
>> There seems to be a 256 MiB lower limit on device size : mkfs.btrfs
>> refuses to create a filesystem on a device that is smaller than that.
> 
> I've noticed the same. I'm interested in researching the patterns the
> filesystem puts down on an encrypted container, but would like to use
> 1MB files as the block device for the filesystem. Looking for patterns
> in 256MB files is too expensive.

You don't mention what kernels or how recent your btrfs-tools builds are, 
but that has been fixed recently, I believe for the 3.3 kernels but 
certainly for 3.4-rcs, as I have a 128 MB /boot on btrfs, here.

You can read about it in the wiki at btrfs.wiki.kernel.org, but 
basically, btrfs' normal allocation pattern is separate data and metadata 
"chunks", which on large enough filesystems are a gig each.  As the 
filesystem shrinks, so does the chunk, but the chunksize minimum used to 
be 256 MiB (or possibly 128 MiB, but you needed one data and one metadata 
chunk, totaling 256 MiB but only half of it could be used).

With current kernels and tools, a filesystem below a gig in size defaults 
to "mixed" mode, data and metadata mixed in the same chunk, and the 
minimum size of the chunk has been eliminated or at least significantly 
reduced, as well.  (I'm not sure if the minimum is tiny, now, say a MiB, 
or eliminated entirely, but a 128 MiB filesystem works, I know that.)

Meanwhile, as the wiki states repeatedly, btrfs is still an experimental 
filesystem, with enough changes going in that if you're not at least 
using the latest stable kernel if not the -rcs, you're using old and 
stale code that has known bugs compared to current.  Obviously, that 
applies here, since you're obviously using stale code if you're still 
running into this problem...

Unless of course you're running a new kernel and have run into a 
different bug.  The mixed-chunk code /is/ quite new, after all, and could 
have some bugs to work out.  But if you're running current and still 
seeing a 256 MiB minimum, then you're definitely seeing a bug, as that's 
supposed to be gone!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Minimum device size of 256 MiB?

[David Sterba]

On Fri, May 11, 2012 at 03:57:24PM -0400, Berke Durak wrote:
> There seems to be a 256 MiB lower limit on device size : mkfs.btrfs
> refuses to create a filesystem
> on a device that is smaller than that.

I was experimenting with the mkfs and sizes, and I found that:

* mkfs crashes for sizes < 4M
* mkfs sizes 4-10M is ok, mount passes, but the fs is unwritable and
  reports 'no space left', this is due to some hardcoded numbers in the
  allocator paths
* mkfs 11M+ ok, mount ok, fs is writable and an exhaustive write filled
  the fs to SIZE-4M of data

> What are the structural reasons for this limit?  Could I reduce this
> limit easily with a couple small code changes
> or would it take a major effort?

It should be possible to push the limit downwards, to like 2MB (current
allocation 'cluster' size), or even to 1.44 so btrfs can support floppy
disks as well. It's not a major work, but one has to make sure not to
break assumptions made on various levels through the allocator(s).


david