[Buildroot] buildroot, python{,3}, and pip

[Buildroot] buildroot, python{,3}, and pip

[Patrick Doyle]

Does anybody have experience using pip on a buildroot generated system?

Poking around, I have found patches to add a python-pip module, (for
example http://lists.busybox.net/pipermail/buildroot/2013-July/075194.html)
but I don't see that in my 2018.02 release.

I have a severely memory constrained system and have already found that

# python3 -mensure-pip

...will add a "pip" command to my rootfs once I have booted up... but
sometimes that command fails (apparently depending on the amount of
free RAM I have, as a first pass diagnosis) and "pip install
<some_package>" works sometimes, and fails other times, again
apparently package size/free RAM relatedly.

So, I might have other system issues to deal with, but in the mean
time, I thought I would ask why pip isn't available as an option to
begin with.

Yes... I get the whole philosophy of "buildroot doesn't support
package managers".  Is that the primary reason it isn't supported?  Or
am I setting myself up for failure should I go down the rathole of
adding support myself?

--wpd

[Buildroot] buildroot, python{,3}, and pip

[Matthew Weber]

Patrick,
On Fri, Aug 10, 2018 at 12:11 PM Patrick Doyle <wpdster@gmail.com> wrote:
>
> Does anybody have experience using pip on a buildroot generated system?
>
> Poking around, I have found patches to add a python-pip module, (for
> example http://lists.busybox.net/pipermail/buildroot/2013-July/075194.html)
> but I don't see that in my 2018.02 release.
>

It was actually just merged yesterday to the next branch and will hit
master for the 2018.11 release cycle.  You can definitely pick it into
your local copy for use now.

[Buildroot] buildroot, python{,3}, and pip

[Patrick Doyle]

On Fri, Aug 10, 2018 at 1:18 PM Matthew Weber
<matthew.weber@rockwellcollins.com> wrote:
> Patrick,
> On Fri, Aug 10, 2018 at 12:11 PM Patrick Doyle <wpdster@gmail.com> wrote:
> >
> > Does anybody have experience using pip on a buildroot generated system?
> It was actually just merged yesterday to the next branch
Wow!  That's awesome! (and timely!)

Thanks.

Now I need to decide whether to mirror the 2018.02 branch onto an
internal server, so I can cherry pick that/those commit(s), or figure
out how to turn that into a local package in the short term.  I guess
I should start by reviewing the work you did.

Thanks.  That's fabulous!

--wpd

--wpd

[Buildroot] buildroot, python{,3}, and pip

[Thomas Petazzoni]

Hello Patrick,

On Fri, 10 Aug 2018 13:11:04 -0400, Patrick Doyle wrote:
> Does anybody have experience using pip on a buildroot generated system?
> 
> Poking around, I have found patches to add a python-pip module, (for
> example http://lists.busybox.net/pipermail/buildroot/2013-July/075194.html)
> but I don't see that in my 2018.02 release.
> 
> I have a severely memory constrained system and have already found that
> 
> # python3 -mensure-pip
> 
> ...will add a "pip" command to my rootfs once I have booted up... but
> sometimes that command fails (apparently depending on the amount of
> free RAM I have, as a first pass diagnosis) and "pip install
> <some_package>" works sometimes, and fails other times, again
> apparently package size/free RAM relatedly.
> 
> So, I might have other system issues to deal with, but in the mean
> time, I thought I would ask why pip isn't available as an option to
> begin with.
> 
> Yes... I get the whole philosophy of "buildroot doesn't support
> package managers".  Is that the primary reason it isn't supported?  Or
> am I setting myself up for failure should I go down the rathole of
> adding support myself?

The bigger question is why do you need python-pip in the first place ?
It goes against the Buildroot principle of reproducible builds. Why
don't you create Buildroot packages for the Python modules that you
need ? The utils/scanpypi script makes this relatively easy for most
Python modules.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] buildroot, python{,3}, and pip

[Patrick Doyle]

On Fri, Aug 10, 2018 at 3:53 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Hello Patrick,

> The bigger question is why do you need python-pip in the first place ?
> It goes against the Buildroot principle of reproducible builds. Why
> don't you create Buildroot packages for the Python modules that you
> need ? The utils/scanpypi script makes this relatively easy for most
> Python modules.
>
Yeah, yeah, I know... the answer is that we expect our application
code to change much more often than the system code.  And our
application architect would like to distribute the application via
pip.  I'm still not sure we have the memory (RAM) to do this.  Early
experimentation shows that pip requires a _lot_ of RAM.

We'll see where we end up.

Besides, I would rather humor him and stay in my preferred development
environment, than to have to support a Yocto/OE development
environment just so he can use pip :-)

--wpd