Issue building a file based rootfs image with mkfs.btrfs

Issue building a file based rootfs image with mkfs.btrfs

[Saul Wold]

Hi there,

I am attempting to build a rootfs image from an existing rootfs 
directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.

The couple problem I saw was that the target image file needed to exist, 
although I think I can patch that then the FS size was much larger than 
the actual size, I tracked this to the usage of ftw not accounting for 
symlinks, I have a patch for that which I will send once I finish 
getting the other issues resolved.

Next issue I hit was an assertion failure after getting "not enough free 
space" message:

not enough free space
add_file_items failed
unable to traverse_directory
Making image is aborted.
mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.

I am kind of stuck on this one, took it as far as I can right now. 
Would I be better off dropping back to 0.19 or can we move forward 
fixing this?

Thanks

-- 
     Sau!

Saul Wold
Yocto Component Wrangler @ Intel
Yocto Project / Poky Build System

Re: Issue building a file based rootfs image with mkfs.btrfs

[Saul Wold]

On 09/19/2013 11:19 AM, Saul Wold wrote:
>
> Hi there,
>
> I am attempting to build a rootfs image from an existing rootfs
> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
>
> The couple problem I saw was that the target image file needed to exist,
> although I think I can patch that then the FS size was much larger than
> the actual size, I tracked this to the usage of ftw not accounting for
> symlinks, I have a patch for that which I will send once I finish
> getting the other issues resolved.
>
> Next issue I hit was an assertion failure after getting "not enough free
> space" message:
>
> not enough free space
> add_file_items failed
> unable to traverse_directory
> Making image is aborted.
> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
>
> I am kind of stuck on this one, took it as far as I can right now. Would
> I be better off dropping back to 0.19 or can we move forward fixing this?
>
Anybody have any ideas on this?  I can reproduce this at will on Fedora 
19 (with btrfs-progs-0.20), Ubuntu 12.10 (with btrfs-profs-0.19) and 
with HEAD from Chris's repo above (is this the right place these days)?

Thanks your your help.

Sau!

> Thanks
>

Re: Issue building a file based rootfs image with mkfs.btrfs

[Chris Mason]

Quoting Saul Wold (2013-09-19 14:19:34)
> Hi there,
> 
> I am attempting to build a rootfs image from an existing rootfs 
> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
> 
> The couple problem I saw was that the target image file needed to exist, 
> although I think I can patch that then the FS size was much larger than 
> the actual size, I tracked this to the usage of ftw not accounting for 
> symlinks, I have a patch for that which I will send once I finish 
> getting the other issues resolved.
> 
> Next issue I hit was an assertion failure after getting "not enough free 
> space" message:
> 
> not enough free space
> add_file_items failed
> unable to traverse_directory
> Making image is aborted.
> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
> 
> I am kind of stuck on this one, took it as far as I can right now. 
> Would I be better off dropping back to 0.19 or can we move forward 
> fixing this?

Hi Saul,

Update on my end, the problem is the image code expects every file to
fit inside a single chunk.  It's only creating 8MB chunks, so any file
over 8MB in size is causing problems.

I'm fixing it up here, I should have a patch for you on Monday.

Thanks!

-chris

Re: Issue building a file based rootfs image with mkfs.btrfs

[Saul Wold]

On 09/28/2013 05:29 AM, Chris Mason wrote:
> Quoting Saul Wold (2013-09-19 14:19:34)
>> Hi there,
>>
>> I am attempting to build a rootfs image from an existing rootfs
>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
>>
>> The couple problem I saw was that the target image file needed to exist,
>> although I think I can patch that then the FS size was much larger than
>> the actual size, I tracked this to the usage of ftw not accounting for
>> symlinks, I have a patch for that which I will send once I finish
>> getting the other issues resolved.
>>
>> Next issue I hit was an assertion failure after getting "not enough free
>> space" message:
>>
>> not enough free space
>> add_file_items failed
>> unable to traverse_directory
>> Making image is aborted.
>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
>>
>> I am kind of stuck on this one, took it as far as I can right now.
>> Would I be better off dropping back to 0.19 or can we move forward
>> fixing this?
>
> Hi Saul,
>
> Update on my end, the problem is the image code expects every file to
> fit inside a single chunk.  It's only creating 8MB chunks, so any file
> over 8MB in size is causing problems.
>
> I'm fixing it up here, I should have a patch for you on Monday.
>
Ah great news!  I want to verify is your git repo for btrfs-progs the 
main upstream?  I see loads of other patches flying around, but not 
applied there.

Thanks again

	Sau!


> Thanks!
>
> -chris
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Issue building a file based rootfs image with mkfs.btrfs

[David Sterba]

On Sat, Sep 28, 2013 at 10:51:54AM -0700, Saul Wold wrote:
> On 09/28/2013 05:29 AM, Chris Mason wrote:
> Ah great news!  I want to verify is your git repo for btrfs-progs the main
> upstream?  I see loads of other patches flying around, but not applied
> there.

The patches land in the integration git repo

http://repo.or.cz/w/btrfs-progs-unstable/devel.git

at branches tagged by date.

Re: Issue building a file based rootfs image with mkfs.btrfs

[Wang Shilong]

> On Sat, Sep 28, 2013 at 10:51:54AM -0700, Saul Wold wrote:
>> On 09/28/2013 05:29 AM, Chris Mason wrote:
>> Ah great news!  I want to verify is your git repo for btrfs-progs the main
>> upstream?  I see loads of other patches flying around, but not applied
>> there.
> 
> The patches land in the integration git repo
> 
> http://repo.or.cz/w/btrfs-progs-unstable/devel.git

David no only did a great job in maintaining btrfs-progs but also helped a lot with us newbies towards btrfs world.

Anyway, i think we do need some necessary btrfs-progs update(for example some regression bug fixes)…

Thanks,
Wang
> 
> at branches tagged by date.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Issue building a file based rootfs image with mkfs.btrfs

[Saul Wold]

On 09/28/2013 05:29 AM, Chris Mason wrote:
> Quoting Saul Wold (2013-09-19 14:19:34)
>> Hi there,
>>
>> I am attempting to build a rootfs image from an existing rootfs
>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
>>
>> The couple problem I saw was that the target image file needed to exist,
>> although I think I can patch that then the FS size was much larger than
>> the actual size, I tracked this to the usage of ftw not accounting for
>> symlinks, I have a patch for that which I will send once I finish
>> getting the other issues resolved.
>>
>> Next issue I hit was an assertion failure after getting "not enough free
>> space" message:
>>
>> not enough free space
>> add_file_items failed
>> unable to traverse_directory
>> Making image is aborted.
>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
>>
>> I am kind of stuck on this one, took it as far as I can right now.
>> Would I be better off dropping back to 0.19 or can we move forward
>> fixing this?
>
> Hi Saul,
>
> Update on my end, the problem is the image code expects every file to
> fit inside a single chunk.  It's only creating 8MB chunks, so any file
> over 8MB in size is causing problems.
>
> I'm fixing it up here, I should have a patch for you on Monday.
>
Any update on this?  Just curious.

Thanks
	Sau!

> Thanks!
>
> -chris
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

RE: Issue building a file based rootfs image with mkfs.btrfs

[Chris Mason]

Hi Saul,

The patch ended up a little bigger than I expected because it is sharing infrastructure with btfs-convert.  Travel added a little more delay, but I'm almost there.

-chris
________________________________________
From: Saul Wold [sgw@linux.intel.com]
Sent: Tuesday, October 01, 2013 4:50 PM
To: Chris Mason
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Issue building a file based rootfs image with mkfs.btrfs

On 09/28/2013 05:29 AM, Chris Mason wrote:
> Quoting Saul Wold (2013-09-19 14:19:34)
>> Hi there,
>>
>> I am attempting to build a rootfs image from an existing rootfs
>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
>>
>> The couple problem I saw was that the target image file needed to exist,
>> although I think I can patch that then the FS size was much larger than
>> the actual size, I tracked this to the usage of ftw not accounting for
>> symlinks, I have a patch for that which I will send once I finish
>> getting the other issues resolved.
>>
>> Next issue I hit was an assertion failure after getting "not enough free
>> space" message:
>>
>> not enough free space
>> add_file_items failed
>> unable to traverse_directory
>> Making image is aborted.
>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
>>
>> I am kind of stuck on this one, took it as far as I can right now.
>> Would I be better off dropping back to 0.19 or can we move forward
>> fixing this?
>
> Hi Saul,
>
> Update on my end, the problem is the image code expects every file to
> fit inside a single chunk.  It's only creating 8MB chunks, so any file
> over 8MB in size is causing problems.
>
> I'm fixing it up here, I should have a patch for you on Monday.
>
Any update on this?  Just curious.

Thanks
        Sau!

> Thanks!
>
> -chris
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Issue building a file based rootfs image with mkfs.btrfs

[Saul Wold]

On 10/01/2013 06:18 PM, Chris Mason wrote:
> Hi Saul,
>
> The patch ended up a little bigger than I expected because it is sharing infrastructure with btfs-convert.  Travel added a little more delay, but I'm almost there.
>

Any news on this patch?

Sau!

> -chris
> ________________________________________
> From: Saul Wold [sgw@linux.intel.com]
> Sent: Tuesday, October 01, 2013 4:50 PM
> To: Chris Mason
> Cc: linux-btrfs@vger.kernel.org
> Subject: Re: Issue building a file based rootfs image with mkfs.btrfs
>
> On 09/28/2013 05:29 AM, Chris Mason wrote:
>> Quoting Saul Wold (2013-09-19 14:19:34)
>>> Hi there,
>>>
>>> I am attempting to build a rootfs image from an existing rootfs
>>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
>>>
>>> The couple problem I saw was that the target image file needed to exist,
>>> although I think I can patch that then the FS size was much larger than
>>> the actual size, I tracked this to the usage of ftw not accounting for
>>> symlinks, I have a patch for that which I will send once I finish
>>> getting the other issues resolved.
>>>
>>> Next issue I hit was an assertion failure after getting "not enough free
>>> space" message:
>>>
>>> not enough free space
>>> add_file_items failed
>>> unable to traverse_directory
>>> Making image is aborted.
>>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
>>>
>>> I am kind of stuck on this one, took it as far as I can right now.
>>> Would I be better off dropping back to 0.19 or can we move forward
>>> fixing this?
>>
>> Hi Saul,
>>
>> Update on my end, the problem is the image code expects every file to
>> fit inside a single chunk.  It's only creating 8MB chunks, so any file
>> over 8MB in size is causing problems.
>>
>> I'm fixing it up here, I should have a patch for you on Monday.
>>
> Any update on this?  Just curious.
>
> Thanks
>          Sau!
>
>> Thanks!
>>
>> -chris
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

Re: Issue building a file based rootfs image with mkfs.btrfs

[Chris Mason]

Hi Saul,

Sorry for the delay, I'll push the patch out this afternoon.

-chris

Quoting Saul Wold (2013-10-14 11:25:28)
> On 10/01/2013 06:18 PM, Chris Mason wrote:
> > Hi Saul,
> >
> > The patch ended up a little bigger than I expected because it is sharing infrastructure with btfs-convert.  Travel added a little more delay, but I'm almost there.
> >
> 
> Any news on this patch?
> 
> Sau!
> 
> > -chris
> > ________________________________________
> > From: Saul Wold [sgw@linux.intel.com]
> > Sent: Tuesday, October 01, 2013 4:50 PM
> > To: Chris Mason
> > Cc: linux-btrfs@vger.kernel.org
> > Subject: Re: Issue building a file based rootfs image with mkfs.btrfs
> >
> > On 09/28/2013 05:29 AM, Chris Mason wrote:
> >> Quoting Saul Wold (2013-09-19 14:19:34)
> >>> Hi there,
> >>>
> >>> I am attempting to build a rootfs image from an existing rootfs
> >>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
> >>>
> >>> The couple problem I saw was that the target image file needed to exist,
> >>> although I think I can patch that then the FS size was much larger than
> >>> the actual size, I tracked this to the usage of ftw not accounting for
> >>> symlinks, I have a patch for that which I will send once I finish
> >>> getting the other issues resolved.
> >>>
> >>> Next issue I hit was an assertion failure after getting "not enough free
> >>> space" message:
> >>>
> >>> not enough free space
> >>> add_file_items failed
> >>> unable to traverse_directory
> >>> Making image is aborted.
> >>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
> >>>
> >>> I am kind of stuck on this one, took it as far as I can right now.
> >>> Would I be better off dropping back to 0.19 or can we move forward
> >>> fixing this?
> >>
> >> Hi Saul,
> >>
> >> Update on my end, the problem is the image code expects every file to
> >> fit inside a single chunk.  It's only creating 8MB chunks, so any file
> >> over 8MB in size is causing problems.
> >>
> >> I'm fixing it up here, I should have a patch for you on Monday.
> >>
> > Any update on this?  Just curious.
> >
> > Thanks
> >          Sau!
> >
> >> Thanks!
> >>
> >> -chris
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>

Re: Issue building a file based rootfs image with mkfs.btrfs

[Chris Mason]

Hi Saul,

Well, a little bigger than I expected turned into a lot bigger than I
expected.  I've pushed the current code to my integration branch, but
the next step is to re-integrate it with Dave's current tree and push
out rc2.

I need to retest convert as well.

-chris

Quoting Chris Mason (2013-10-14 11:26:55)
> Hi Saul,
> 
> Sorry for the delay, I'll push the patch out this afternoon.
> 
> -chris
> 
> Quoting Saul Wold (2013-10-14 11:25:28)
> > On 10/01/2013 06:18 PM, Chris Mason wrote:
> > > Hi Saul,
> > >
> > > The patch ended up a little bigger than I expected because it is sharing infrastructure with btfs-convert.  Travel added a little more delay, but I'm almost there.
> > >
> > 
> > Any news on this patch?
> > 
> > Sau!
> > 
> > > -chris
> > > ________________________________________
> > > From: Saul Wold [sgw@linux.intel.com]
> > > Sent: Tuesday, October 01, 2013 4:50 PM
> > > To: Chris Mason
> > > Cc: linux-btrfs@vger.kernel.org
> > > Subject: Re: Issue building a file based rootfs image with mkfs.btrfs
> > >
> > > On 09/28/2013 05:29 AM, Chris Mason wrote:
> > >> Quoting Saul Wold (2013-09-19 14:19:34)
> > >>> Hi there,
> > >>>
> > >>> I am attempting to build a rootfs image from an existing rootfs
> > >>> directory tree.  I am using the 0.20 @ 194aa4a of Chris's git repo.
> > >>>
> > >>> The couple problem I saw was that the target image file needed to exist,
> > >>> although I think I can patch that then the FS size was much larger than
> > >>> the actual size, I tracked this to the usage of ftw not accounting for
> > >>> symlinks, I have a patch for that which I will send once I finish
> > >>> getting the other issues resolved.
> > >>>
> > >>> Next issue I hit was an assertion failure after getting "not enough free
> > >>> space" message:
> > >>>
> > >>> not enough free space
> > >>> add_file_items failed
> > >>> unable to traverse_directory
> > >>> Making image is aborted.
> > >>> mkfs.btrfs: mkfs.c:1542: main: Assertion `!(ret)' failed.
> > >>>
> > >>> I am kind of stuck on this one, took it as far as I can right now.
> > >>> Would I be better off dropping back to 0.19 or can we move forward
> > >>> fixing this?
> > >>
> > >> Hi Saul,
> > >>
> > >> Update on my end, the problem is the image code expects every file to
> > >> fit inside a single chunk.  It's only creating 8MB chunks, so any file
> > >> over 8MB in size is causing problems.
> > >>
> > >> I'm fixing it up here, I should have a patch for you on Monday.
> > >>
> > > Any update on this?  Just curious.
> > >
> > > Thanks
> > >          Sau!
> > >
> > >> Thanks!
> > >>
> > >> -chris
> > >>
> > >> --
> > >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > >> the body of a message to majordomo@vger.kernel.org
> > >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html