* [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root
@ 2015-09-21 17:49 Elliot Smith
2015-09-22 2:41 ` Brian Avery
2015-09-22 8:03 ` Ed Bartosh
0 siblings, 2 replies; 6+ messages in thread
From: Elliot Smith @ 2015-09-21 17:49 UTC (permalink / raw)
To: toaster
The files-in-image.txt file is produced by bitbake after an
image is created, listing all the files in the image.
However, this list doesn't include the root directory ('/').
buildinfohelper.py then tries to construct the filesystem
tree from this file, assuming that every directory apart from
the root directory (which is special-cased) can be assigned
a parent. But because the root directory isn't listed in
files-in-image.txt, an object for the root directory is never
created.
The direct subdirectories of the root ('./bin', './usr' etc.)
then can't be assigned a parent directory, as the object
representing the root directory doesn't exist. This
results in a Target_File lookup error and causes the
directory listing page to fail.
Fix this by creating a fake entry for the root directory
in the Target_File table, so that the direct subdirectories
of / can be assigned a parent. Note that it doesn't matter
that the root is faked, as its properties are never shown
in the directory structure tree.
[YOCTO #8280]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++--------------
1 file changed, 17 insertions(+), 14 deletions(-)
diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index 6e313fe..b2201f6 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -353,26 +353,29 @@ class ORMWrapper(object):
files = filedata['files']
syms = filedata['syms']
- # we insert directories, ordered by name depth
+ # always create the root directory as a special case;
+ # note that this is never displayed, so the owner, group,
+ # size, permission are irrelevant
+ tf_obj = Target_File.objects.create(target = target_obj,
+ path = '/',
+ size = 0,
+ owner = '',
+ group = '',
+ permission = '',
+ inodetype = Target_File.ITYPE_DIRECTORY)
+ tf_obj.save()
+
+ # insert directories, ordered by name depth
for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))):
(user, group, size) = d[1:4]
permission = d[0][1:]
path = d[4].lstrip(".")
+
+ # we already created the root directory, so ignore any
+ # entry for it
if len(path) == 0:
- # we create the root directory as a special case
- path = "/"
- tf_obj = Target_File.objects.create(
- target = target_obj,
- path = path,
- size = size,
- inodetype = Target_File.ITYPE_DIRECTORY,
- permission = permission,
- owner = user,
- group = group,
- )
- tf_obj.directory = tf_obj
- tf_obj.save()
continue
+
parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1])
if len(parent_path) == 0:
parent_path = "/"
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root
2015-09-21 17:49 [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root Elliot Smith
@ 2015-09-22 2:41 ` Brian Avery
2015-09-22 8:03 ` Ed Bartosh
1 sibling, 0 replies; 6+ messages in thread
From: Brian Avery @ 2015-09-22 2:41 UTC (permalink / raw)
To: Elliot Smith; +Cc: toaster
upstreamed.
ty,
b
On Mon, Sep 21, 2015 at 10:49 AM, Elliot Smith <elliot.smith@intel.com> wrote:
> The files-in-image.txt file is produced by bitbake after an
> image is created, listing all the files in the image.
> However, this list doesn't include the root directory ('/').
>
> buildinfohelper.py then tries to construct the filesystem
> tree from this file, assuming that every directory apart from
> the root directory (which is special-cased) can be assigned
> a parent. But because the root directory isn't listed in
> files-in-image.txt, an object for the root directory is never
> created.
>
> The direct subdirectories of the root ('./bin', './usr' etc.)
> then can't be assigned a parent directory, as the object
> representing the root directory doesn't exist. This
> results in a Target_File lookup error and causes the
> directory listing page to fail.
>
> Fix this by creating a fake entry for the root directory
> in the Target_File table, so that the direct subdirectories
> of / can be assigned a parent. Note that it doesn't matter
> that the root is faked, as its properties are never shown
> in the directory structure tree.
>
> [YOCTO #8280]
>
> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
> ---
> bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++--------------
> 1 file changed, 17 insertions(+), 14 deletions(-)
>
> diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
> index 6e313fe..b2201f6 100644
> --- a/bitbake/lib/bb/ui/buildinfohelper.py
> +++ b/bitbake/lib/bb/ui/buildinfohelper.py
> @@ -353,26 +353,29 @@ class ORMWrapper(object):
> files = filedata['files']
> syms = filedata['syms']
>
> - # we insert directories, ordered by name depth
> + # always create the root directory as a special case;
> + # note that this is never displayed, so the owner, group,
> + # size, permission are irrelevant
> + tf_obj = Target_File.objects.create(target = target_obj,
> + path = '/',
> + size = 0,
> + owner = '',
> + group = '',
> + permission = '',
> + inodetype = Target_File.ITYPE_DIRECTORY)
> + tf_obj.save()
> +
> + # insert directories, ordered by name depth
> for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))):
> (user, group, size) = d[1:4]
> permission = d[0][1:]
> path = d[4].lstrip(".")
> +
> + # we already created the root directory, so ignore any
> + # entry for it
> if len(path) == 0:
> - # we create the root directory as a special case
> - path = "/"
> - tf_obj = Target_File.objects.create(
> - target = target_obj,
> - path = path,
> - size = size,
> - inodetype = Target_File.ITYPE_DIRECTORY,
> - permission = permission,
> - owner = user,
> - group = group,
> - )
> - tf_obj.directory = tf_obj
> - tf_obj.save()
> continue
> +
> parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1])
> if len(parent_path) == 0:
> parent_path = "/"
> --
> Elliot Smith
> Software Engineer
> Intel OTC
>
> ---------------------------------------------------------------------
> Intel Corporation (UK) Limited
> Registered No. 1134945 (England)
> Registered Office: Pipers Way, Swindon SN3 1RJ
> VAT No: 860 2173 47
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root
2015-09-21 17:49 [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root Elliot Smith
2015-09-22 2:41 ` Brian Avery
@ 2015-09-22 8:03 ` Ed Bartosh
2015-09-22 8:37 ` Smith, Elliot
1 sibling, 1 reply; 6+ messages in thread
From: Ed Bartosh @ 2015-09-22 8:03 UTC (permalink / raw)
To: Elliot Smith; +Cc: toaster
On Mon, Sep 21, 2015 at 06:49:22PM +0100, Elliot Smith wrote:
> The files-in-image.txt file is produced by bitbake after an
> image is created, listing all the files in the image.
> However, this list doesn't include the root directory ('/').
>
Did you try to look what caused this change? Bitbake used to include
root dir into this file previously, right?
> buildinfohelper.py then tries to construct the filesystem
> tree from this file, assuming that every directory apart from
> the root directory (which is special-cased) can be assigned
> a parent. But because the root directory isn't listed in
> files-in-image.txt, an object for the root directory is never
> created.
>
> The direct subdirectories of the root ('./bin', './usr' etc.)
> then can't be assigned a parent directory, as the object
> representing the root directory doesn't exist. This
> results in a Target_File lookup error and causes the
> directory listing page to fail.
>
> Fix this by creating a fake entry for the root directory
> in the Target_File table, so that the direct subdirectories
> of / can be assigned a parent. Note that it doesn't matter
> that the root is faked, as its properties are never shown
> in the directory structure tree.
>
This looks like a workaround to me. It should be fixed in bitbake code
if it's possible.
Regards,
Ed
> [YOCTO #8280]
>
> Signed-off-by: Elliot Smith <elliot.smith@intel.com>
> ---
> bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++--------------
> 1 file changed, 17 insertions(+), 14 deletions(-)
>
> diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
> index 6e313fe..b2201f6 100644
> --- a/bitbake/lib/bb/ui/buildinfohelper.py
> +++ b/bitbake/lib/bb/ui/buildinfohelper.py
> @@ -353,26 +353,29 @@ class ORMWrapper(object):
> files = filedata['files']
> syms = filedata['syms']
>
> - # we insert directories, ordered by name depth
> + # always create the root directory as a special case;
> + # note that this is never displayed, so the owner, group,
> + # size, permission are irrelevant
> + tf_obj = Target_File.objects.create(target = target_obj,
> + path = '/',
> + size = 0,
> + owner = '',
> + group = '',
> + permission = '',
> + inodetype = Target_File.ITYPE_DIRECTORY)
> + tf_obj.save()
> +
> + # insert directories, ordered by name depth
> for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))):
> (user, group, size) = d[1:4]
> permission = d[0][1:]
> path = d[4].lstrip(".")
> +
> + # we already created the root directory, so ignore any
> + # entry for it
> if len(path) == 0:
> - # we create the root directory as a special case
> - path = "/"
> - tf_obj = Target_File.objects.create(
> - target = target_obj,
> - path = path,
> - size = size,
> - inodetype = Target_File.ITYPE_DIRECTORY,
> - permission = permission,
> - owner = user,
> - group = group,
> - )
> - tf_obj.directory = tf_obj
> - tf_obj.save()
> continue
> +
> parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1])
> if len(parent_path) == 0:
> parent_path = "/"
> --
> Elliot Smith
> Software Engineer
> Intel OTC
>
> ---------------------------------------------------------------------
> Intel Corporation (UK) Limited
> Registered No. 1134945 (England)
> Registered Office: Pipers Way, Swindon SN3 1RJ
> VAT No: 860 2173 47
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
--
--
Regards,
Ed
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root
2015-09-22 8:03 ` Ed Bartosh
@ 2015-09-22 8:37 ` Smith, Elliot
2015-09-22 16:47 ` Brian Avery
0 siblings, 1 reply; 6+ messages in thread
From: Smith, Elliot @ 2015-09-22 8:37 UTC (permalink / raw)
To: ed.bartosh; +Cc: toaster
[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]
On 22 September 2015 at 09:03, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:
> On Mon, Sep 21, 2015 at 06:49:22PM +0100, Elliot Smith wrote:
> > The files-in-image.txt file is produced by bitbake after an
> > image is created, listing all the files in the image.
> > However, this list doesn't include the root directory ('/').
> >
> Did you try to look what caused this change? Bitbake used to include
> root dir into this file previously, right?
>
I'm not sure if it ever did include this file. I had a brief look at how
buildhistory_analysis.py does its work (git diffs if I read it correctly),
but didn't dig right into it.
> > buildinfohelper.py then tries to construct the filesystem
> > tree from this file, assuming that every directory apart from
> > the root directory (which is special-cased) can be assigned
> > a parent. But because the root directory isn't listed in
> > files-in-image.txt, an object for the root directory is never
> > created.
> >
> > The direct subdirectories of the root ('./bin', './usr' etc.)
> > then can't be assigned a parent directory, as the object
> > representing the root directory doesn't exist. This
> > results in a Target_File lookup error and causes the
> > directory listing page to fail.
> >
> > Fix this by creating a fake entry for the root directory
> > in the Target_File table, so that the direct subdirectories
> > of / can be assigned a parent. Note that it doesn't matter
> > that the root is faked, as its properties are never shown
> > in the directory structure tree.
> >
> This looks like a workaround to me. It should be fixed in bitbake code
> if it's possible.
>
I discussed this with Brian, and the decision was that a work-around was
sufficient for now. (See the options I suggested in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8280#c9.)
I can investigate this some more and/or raise a bug, if we think that's
worthwhile.
Elliot
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
[-- Attachment #2: Type: text/html, Size: 2993 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root
2015-09-22 8:37 ` Smith, Elliot
@ 2015-09-22 16:47 ` Brian Avery
2015-09-23 7:12 ` Smith, Elliot
0 siblings, 1 reply; 6+ messages in thread
From: Brian Avery @ 2015-09-22 16:47 UTC (permalink / raw)
To: Smith, Elliot; +Cc: toaster
yes, please raise the buildhistory bug with bitbake but for now, I'm
ok with this workaround, especially since you did the sensible thing
of checking to see if it exists before creating your entry i.e. this
code will still work even after/if they fix the buildhistory bug.
-b
On Tue, Sep 22, 2015 at 1:37 AM, Smith, Elliot <elliot.smith@intel.com> wrote:
> On 22 September 2015 at 09:03, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
>>
>> On Mon, Sep 21, 2015 at 06:49:22PM +0100, Elliot Smith wrote:
>> > The files-in-image.txt file is produced by bitbake after an
>> > image is created, listing all the files in the image.
>> > However, this list doesn't include the root directory ('/').
>> >
>> Did you try to look what caused this change? Bitbake used to include
>> root dir into this file previously, right?
>
>
> I'm not sure if it ever did include this file. I had a brief look at how
> buildhistory_analysis.py does its work (git diffs if I read it correctly),
> but didn't dig right into it.
>
>>
>> > buildinfohelper.py then tries to construct the filesystem
>> > tree from this file, assuming that every directory apart from
>> > the root directory (which is special-cased) can be assigned
>> > a parent. But because the root directory isn't listed in
>> > files-in-image.txt, an object for the root directory is never
>> > created.
>> >
>> > The direct subdirectories of the root ('./bin', './usr' etc.)
>> > then can't be assigned a parent directory, as the object
>> > representing the root directory doesn't exist. This
>> > results in a Target_File lookup error and causes the
>> > directory listing page to fail.
>> >
>> > Fix this by creating a fake entry for the root directory
>> > in the Target_File table, so that the direct subdirectories
>> > of / can be assigned a parent. Note that it doesn't matter
>> > that the root is faked, as its properties are never shown
>> > in the directory structure tree.
>> >
>> This looks like a workaround to me. It should be fixed in bitbake code
>> if it's possible.
>
>
> I discussed this with Brian, and the decision was that a work-around was
> sufficient for now. (See the options I suggested in
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=8280#c9.)
>
> I can investigate this some more and/or raise a bug, if we think that's
> worthwhile.
>
> Elliot
> --
> Elliot Smith
> Software Engineer
> Intel Open Source Technology Centre
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-09-23 7:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-21 17:49 [review-request][PATCH] toaster: Add fake entry to Target_File for filesystem root Elliot Smith
2015-09-22 2:41 ` Brian Avery
2015-09-22 8:03 ` Ed Bartosh
2015-09-22 8:37 ` Smith, Elliot
2015-09-22 16:47 ` Brian Avery
2015-09-23 7:12 ` Smith, Elliot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.