[yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[Antonin Godard]

`subdir` is the correct attribute to use for extracting the package in
the correct directory.

From the bitbake doc:

  subdir: Places the file (or extracts its contents) into the specified
    subdirectory. This option is useful for unusual tarballs or other
    archives that do not have their files already in a subdirectory within
    the archive.
  subpath: Limits the checkout to a specific subpath of the tree when
    using the Git fetcher is used.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/classes.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index b92f4e4f20ea8f702c90f4e3d29251b2461d07d0..2ac94630d80386cf613222b70f7bd6323b9948f2 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -176,7 +176,7 @@ example use for this class.
    are extracted into the subdirectory expected by the default value of
    :term:`S`::
 
-      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
+      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subdir=${BP}"
 
    See the ":ref:`bitbake-user-manual/bitbake-user-manual-fetching:fetchers`" section in the BitBake User Manual for
    more information on supported BitBake Fetchers.

---
base-commit: 7d5eb0cee5b2b7096969819d7d7ce569a3c92f27
change-id: 20241115-fix-bin-package-eed7633fbecd

Best regards,
-- 
Antonin Godard <antonin.godard@bootlin.com>

Re: [docs] [yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[Quentin Schulz]

Hi Antonin,

On 11/18/24 9:46 AM, Antonin Godard via lists.yoctoproject.org wrote:
> `subdir` is the correct attribute to use for extracting the package in
> the correct directory.
> 
>  From the bitbake doc:
> 
>    subdir: Places the file (or extracts its contents) into the specified
>      subdirectory. This option is useful for unusual tarballs or other
>      archives that do not have their files already in a subdirectory within
>      the archive.
>    subpath: Limits the checkout to a specific subpath of the tree when
>      using the Git fetcher is used.
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>   documentation/ref-manual/classes.rst | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
> index b92f4e4f20ea8f702c90f4e3d29251b2461d07d0..2ac94630d80386cf613222b70f7bd6323b9948f2 100644
> --- a/documentation/ref-manual/classes.rst
> +++ b/documentation/ref-manual/classes.rst
> @@ -176,7 +176,7 @@ example use for this class.
>      are extracted into the subdirectory expected by the default value of
>      :term:`S`::
>   
> -      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
> +      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subdir=${BP}"
>   

This is not enough, subpath is mentioned in the note admonition just 
above a few times, those need to be fixed as well.

Additionally the wording is weird for the bin_package class.

"class is a helper class for recipes that extract the contents of a 
binary package (e.g. an RPM)"
[...]
"For RPMs and other packages that do not contain a subdirectory,"

Maybe we should use another example of a binary package (e.g. a .deb 
package) in the first sentence?

Also I am not entirely sure what this example is supposed to show.

SRC_URI = 
"git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"

This is very confusing to me, is this really a somepackage.rpm **git** 
repo? or was this a mix between fetching via git and unpacking an RPM?

I'm wondering what we were supposed to show with this example. Is it 
about fetching an RPM from a git repo and only extracting a part of that 
RPM? Is that even possible? If so, we probably want a combination of 
subpath and subdir? subdir for the path internal to the repo to the RPM 
and subpath for the path internal to the RPM to extract?

Or, we should provide two examples, one for RPM downloaded from HTTP for 
example, using subpath, and one for a directory from a git repo, using 
subdir?

Cheers,
Quentin

Re: [docs] [yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[Antonin Godard]

Hi Quentin,

On Tue Nov 19, 2024 at 10:38 AM CET, Quentin Schulz via lists.yoctoproject.org wrote:
> Hi Antonin,
>
> On 11/18/24 9:46 AM, Antonin Godard via lists.yoctoproject.org wrote:
>> `subdir` is the correct attribute to use for extracting the package in
>> the correct directory.
>> 
>>  From the bitbake doc:
>> 
>>    subdir: Places the file (or extracts its contents) into the specified
>>      subdirectory. This option is useful for unusual tarballs or other
>>      archives that do not have their files already in a subdirectory within
>>      the archive.
>>    subpath: Limits the checkout to a specific subpath of the tree when
>>      using the Git fetcher is used.
>> 
>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>> ---
>>   documentation/ref-manual/classes.rst | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
>> index b92f4e4f20ea8f702c90f4e3d29251b2461d07d0..2ac94630d80386cf613222b70f7bd6323b9948f2 100644
>> --- a/documentation/ref-manual/classes.rst
>> +++ b/documentation/ref-manual/classes.rst
>> @@ -176,7 +176,7 @@ example use for this class.
>>      are extracted into the subdirectory expected by the default value of
>>      :term:`S`::
>>   
>> -      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>> +      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subdir=${BP}"
>>   
>
> This is not enough, subpath is mentioned in the note admonition just 
> above a few times, those need to be fixed as well.
>
> Additionally the wording is weird for the bin_package class.
>
> "class is a helper class for recipes that extract the contents of a 
> binary package (e.g. an RPM)"
> [...]
> "For RPMs and other packages that do not contain a subdirectory,"
>
> Maybe we should use another example of a binary package (e.g. a .deb 
> package) in the first sentence?
>
> Also I am not entirely sure what this example is supposed to show.
>
> SRC_URI = 
> "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>
> This is very confusing to me, is this really a somepackage.rpm **git** 
> repo? or was this a mix between fetching via git and unpacking an RPM?
>
> I'm wondering what we were supposed to show with this example. Is it 
> about fetching an RPM from a git repo and only extracting a part of that 
> RPM? Is that even possible? If so, we probably want a combination of 
> subpath and subdir? subdir for the path internal to the repo to the RPM 
> and subpath for the path internal to the RPM to extract?

Actually I think I missed the point of the class description at first. It seems
to document how to fetch a subpath of a git repository containing rpm files…
Which makes me wonder: how often do you need to fetch binary from git
repositories? Probably not often.

I would have thought a more common case to be for extracting a local file, or
fetching a file over HTTP.

So I guess the description should be updated to give some more examples, yes. I
would even remove the git binary example, as it's bad practice to store binaries
in repos anyway… What do you think?

> Or, we should provide two examples, one for RPM downloaded from HTTP for 
> example, using subpath, and one for a directory from a git repo, using 
> subdir?

Actually: you'd use subpath for the git repo (to extract only a specific file
from the repo), and subdir for e.g. a tarball, to extract it in the right place.

The current description of the class is really confusing, I'll send a patch to
update it.


Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[Quentin Schulz]

Hi Antonin,

On 11/19/24 4:05 PM, Antonin Godard via lists.yoctoproject.org wrote:
> Hi Quentin,
> 
> On Tue Nov 19, 2024 at 10:38 AM CET, Quentin Schulz via lists.yoctoproject.org wrote:
>> Hi Antonin,
>>
>> On 11/18/24 9:46 AM, Antonin Godard via lists.yoctoproject.org wrote:
>>> `subdir` is the correct attribute to use for extracting the package in
>>> the correct directory.
>>>
>>>   From the bitbake doc:
>>>
>>>     subdir: Places the file (or extracts its contents) into the specified
>>>       subdirectory. This option is useful for unusual tarballs or other
>>>       archives that do not have their files already in a subdirectory within
>>>       the archive.
>>>     subpath: Limits the checkout to a specific subpath of the tree when
>>>       using the Git fetcher is used.
>>>
>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>> ---
>>>    documentation/ref-manual/classes.rst | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
>>> index b92f4e4f20ea8f702c90f4e3d29251b2461d07d0..2ac94630d80386cf613222b70f7bd6323b9948f2 100644
>>> --- a/documentation/ref-manual/classes.rst
>>> +++ b/documentation/ref-manual/classes.rst
>>> @@ -176,7 +176,7 @@ example use for this class.
>>>       are extracted into the subdirectory expected by the default value of
>>>       :term:`S`::
>>>    
>>> -      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>>> +      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subdir=${BP}"
>>>    
>>
>> This is not enough, subpath is mentioned in the note admonition just
>> above a few times, those need to be fixed as well.
>>
>> Additionally the wording is weird for the bin_package class.
>>
>> "class is a helper class for recipes that extract the contents of a
>> binary package (e.g. an RPM)"
>> [...]
>> "For RPMs and other packages that do not contain a subdirectory,"
>>
>> Maybe we should use another example of a binary package (e.g. a .deb
>> package) in the first sentence?
>>
>> Also I am not entirely sure what this example is supposed to show.
>>
>> SRC_URI =
>> "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>>
>> This is very confusing to me, is this really a somepackage.rpm **git**
>> repo? or was this a mix between fetching via git and unpacking an RPM?
>>
>> I'm wondering what we were supposed to show with this example. Is it
>> about fetching an RPM from a git repo and only extracting a part of that
>> RPM? Is that even possible? If so, we probably want a combination of
>> subpath and subdir? subdir for the path internal to the repo to the RPM
>> and subpath for the path internal to the RPM to extract?
> 
> Actually I think I missed the point of the class description at first. It seems
> to document how to fetch a subpath of a git repository containing rpm files…

Would the content of the rpm files be extracted if they are fetched by 
the git fetcher?

> Which makes me wonder: how often do you need to fetch binary from git
> repositories? Probably not often.
> 

Probably not, but not THAT uncommon. For example Rockchip provides the 
libmali binaries mostly via deb packages (they provide "sources" but I 
was unable to compile them into something that works, so....).

> I would have thought a more common case to be for extracting a local file, or
> fetching a file over HTTP.
> 

You can still want to version deb/rpm packages. But I think it makes the 
example needlessly complex, though we could still keep that example for 
this special case (provided it actually works).

> So I guess the description should be updated to give some more examples, yes. I
> would even remove the git binary example, as it's bad practice to store binaries
> in repos anyway… What do you think?
> 

I think it makes more sense to store the binaries in a git repo than in 
the layer itself and properly setting up an HTTP server is not 
accessible to everyone (especially if it's public, GitHub/GitLab is a 
pretty reliable source for example).

>> Or, we should provide two examples, one for RPM downloaded from HTTP for
>> example, using subpath, and one for a directory from a git repo, using
>> subdir?
> 
> Actually: you'd use subpath for the git repo (to extract only a specific file
> from the repo), and subdir for e.g. a tarball, to extract it in the right place.
> 
> The current description of the class is really confusing, I'll send a patch to
> update it.
> 

At least we agree on that part :)

Looking forward to it!

Cheers,
Quentin

Re: [docs] [yocto-docs PATCH] ref-manual: classes: bin_package: fix SRC_URI example

[Antonin Godard]

Hi Quentin,

On Tue Nov 19, 2024 at 4:15 PM CET, Quentin Schulz wrote:
[...]
>>>> -      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>>>> +      SRC_URI = "git://example.com/downloads/somepackage.rpm;branch=main;subdir=${BP}"
>>>>    
>>>
>>> This is not enough, subpath is mentioned in the note admonition just
>>> above a few times, those need to be fixed as well.
>>>
>>> Additionally the wording is weird for the bin_package class.
>>>
>>> "class is a helper class for recipes that extract the contents of a
>>> binary package (e.g. an RPM)"
>>> [...]
>>> "For RPMs and other packages that do not contain a subdirectory,"
>>>
>>> Maybe we should use another example of a binary package (e.g. a .deb
>>> package) in the first sentence?
>>>
>>> Also I am not entirely sure what this example is supposed to show.
>>>
>>> SRC_URI =
>>> "git://example.com/downloads/somepackage.rpm;branch=main;subpath=${BP}"
>>>
>>> This is very confusing to me, is this really a somepackage.rpm **git**
>>> repo? or was this a mix between fetching via git and unpacking an RPM?
>>>
>>> I'm wondering what we were supposed to show with this example. Is it
>>> about fetching an RPM from a git repo and only extracting a part of that
>>> RPM? Is that even possible? If so, we probably want a combination of
>>> subpath and subdir? subdir for the path internal to the repo to the RPM
>>> and subpath for the path internal to the RPM to extract?
>> 
>> Actually I think I missed the point of the class description at first. It seems
>> to document how to fetch a subpath of a git repository containing rpm files…
>
> Would the content of the rpm files be extracted if they are fetched by 
> the git fetcher?

Actually, no! I just tried. You can't just pass a path to a file inside a git
repo to the git fetcher. You can pass the git repo URL, but the content of the
git repo won't be extracted, the git fetcher won't do that by itself.

The bin_package class expects content of the rpm/tarball/else to already be
extracted in S before installing it in D.

So the fetcher should be selected with that in mind.

>> Which makes me wonder: how often do you need to fetch binary from git
>> repositories? Probably not often.
>> 
>
> Probably not, but not THAT uncommon. For example Rockchip provides the 
> libmali binaries mostly via deb packages (they provide "sources" but I 
> was unable to compile them into something that works, so....).

I would expect most git hosting service to provide HTTP link to files, so using
https://...<file>.rpm is probably possible in most cases?


Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com