FW: [RFC] blacklisted variables

FW: [RFC] blacklisted variables

[Barros Pena, Belen]

Resending this. Let's see if we can agree on a list of blacklisted
variables for Toaster. We need it for

https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588

Thanks!

Belén


On 22/08/2014 15:25, "Barros Pena, Belen" <belen.barros.pena@intel.com>
wrote:

>In prior discussions it was agreed that, at least for Toaster 1.7, we will
>blacklist any variables that influence how things will be built or where
>artefacts are stored. This means that Toaster will no allow you to set
>such variables (to see how, see page 13 of the design document attached to
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588).
>
>I guess we should put together a list of the blacklisted variables. The
>following is a first go (mainly taken from local.conf and Hob). Please add
>to the list as needed:
>
>BB_NUMBER_THREADS
>PARALLEL_MAKE
>DL_DIR
>SSTATE_DIR
>TMPDIR
>BB_DISKMON_DIRS
>SSTATE_DIR
>SSTATE_MIRRORS
>http_proxy 
>https_proxy
>ftp_proxy
>all_proxy
>CVS_PROXY_HOST
>CVS_PROXY_PORT
>
>
>Thanks!
>
>Belén
>
>
>
>-- 
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster

Re: FW: [RFC] blacklisted variables

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 2872 bytes --]

I have some points:

0. We face a major security risk here, and I can't stress this enough. The
variables can probably be used to make bitbake execute user input, and
exposing this functionality to 3rd parties, e.g. outside customers, can
easily lead to remote code execution, which is a big no-no in cloud-based
offerings.

*Do Not* put hosted-mode code, where people can set ANY variables, into a
production setting, as it can expose your whole infrastructure to Bad Guys
TM. We need a major auditing effort in bitbake itself before putting this
into production, to make sure that the variables we set cannot be exploited.

The filtering that we're doing here is not designed to keep Bad People out,
but prevent inadvertent mistakes that can bring down the service for Well
Intentioned People.

1. We need to make sure that the variable name check is implemented on the
server side too, not just on the frontend side.

2. We need to add some variables here. Digging through the sources, I came
up with this list, by no means exhaustive.

BBPATH
​TOPDIR​

FAKEROOTCMD
FAKEROOTBASEENV
DL_DIR
BB_NUMBER_PARSE_THREADS
ASSUME_PROVIDED
PERSISTENT_DIR
FETCH_CMD*
BBSCHEDULER
BBSCHEDULERS


​


​Cheers,
Alex​


On Mon, Sep 15, 2014 at 3:50 PM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

> Resending this. Let's see if we can agree on a list of blacklisted
> variables for Toaster. We need it for
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588
>
> Thanks!
>
> Belén
>
>
> On 22/08/2014 15:25, "Barros Pena, Belen" <belen.barros.pena@intel.com>
> wrote:
>
> >In prior discussions it was agreed that, at least for Toaster 1.7, we will
> >blacklist any variables that influence how things will be built or where
> >artefacts are stored. This means that Toaster will no allow you to set
> >such variables (to see how, see page 13 of the design document attached to
> >https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588).
> >
> >I guess we should put together a list of the blacklisted variables. The
> >following is a first go (mainly taken from local.conf and Hob). Please add
> >to the list as needed:
> >
> >BB_NUMBER_THREADS
> >PARALLEL_MAKE
> >DL_DIR
> >SSTATE_DIR
> >TMPDIR
> >BB_DISKMON_DIRS
> >SSTATE_DIR
> >SSTATE_MIRRORS
> >http_proxy
> >https_proxy
> >ftp_proxy
> >all_proxy
> >CVS_PROXY_HOST
> >CVS_PROXY_PORT
> >
> >
> >Thanks!
> >
> >Belén
> >
> >
> >
> >--
> >_______________________________________________
> >toaster mailing list
> >toaster@yoctoproject.org
> >https://lists.yoctoproject.org/listinfo/toaster
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>



-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 5222 bytes --]

Re: FW: [RFC] blacklisted variables

[Barros Pena, Belen]

Thanks for replying, Alex. Some comments below:

On 18/09/2014 12:22, "Damian, Alexandru" <alexandru.damian@intel.com>
wrote:

>I have some points:
>
>
>0. We face a major security risk here, and I can't stress this enough.
>The variables can probably be used to make bitbake execute user input,
>and exposing this functionality to 3rd parties, e.g. outside customers,
>can easily lead to remote code execution, which
> is a big no-no in cloud-based offerings.
>
>Do Not put hosted-mode code, where people can set ANY variables, into a
>production setting, as it can expose your whole infrastructure to Bad
>Guys TM. We need a major auditing effort in bitbake itself before putting
>this into production, to make sure
> that the variables we set cannot be exploited.

Who should run this audit?

>
>
>The filtering that we're doing here is not designed to keep Bad People
>out, but prevent inadvertent mistakes that can bring down the service for
>Well Intentioned People.
>
>
>
>1. We need to make sure that the variable name check is implemented on
>the server side too, not just on the frontend side.

Fair point. 

>
>
>2. We need to add some variables here. Digging through the sources, I
>came up with this list, by no means exhaustive.

Is this on top of the list I've sent?

>
>
>
>BBPATH
>
>​TOPDIR​
>
>FAKEROOTCMD
>FAKEROOTBASEENV
>DL_DIR
>BB_NUMBER_PARSE_THREADS
>
>ASSUME_PROVIDED
>
>PERSISTENT_DIR
>
>FETCH_CMD*
>
>BBSCHEDULER
>
>BBSCHEDULERS
>
>
>
>​
>
>
>​Cheers,
>Alex​
>
>
>On Mon, Sep 15, 2014 at 3:50 PM, Barros Pena, Belen
><belen.barros.pena@intel.com> wrote:
>
>Resending this. Let's see if we can agree on a list of blacklisted
>variables for Toaster. We need it for
>
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588
>
>Thanks!
>
>Belén
>
>
>On 22/08/2014 15:25, "Barros Pena, Belen" <belen.barros.pena@intel.com>
>wrote:
>
>>In prior discussions it was agreed that, at least for Toaster 1.7, we
>>will
>>blacklist any variables that influence how things will be built or where
>>artefacts are stored. This means that Toaster will no allow you to set
>>such variables (to see how, see page 13 of the design document attached
>>to
>>https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588).
>>
>>I guess we should put together a list of the blacklisted variables. The
>>following is a first go (mainly taken from local.conf and Hob). Please
>>add
>>to the list as needed:
>>
>>BB_NUMBER_THREADS
>>PARALLEL_MAKE
>>DL_DIR
>>SSTATE_DIR
>>TMPDIR
>>BB_DISKMON_DIRS
>>SSTATE_DIR
>>SSTATE_MIRRORS
>>http_proxy
>>https_proxy
>>ftp_proxy
>>all_proxy
>>CVS_PROXY_HOST
>>CVS_PROXY_PORT
>>
>>
>>Thanks!
>>
>>Belén
>>
>>
>>
>>--
>>_______________________________________________
>>toaster mailing list
>>toaster@yoctoproject.org
>>https://lists.yoctoproject.org/listinfo/toaster
>
>--
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster
>
>
>
>
>
>
>-- 
>Alex Damian
>Yocto Project
>
>SSG / OTC 
>
>
>

Re: FW: [RFC] blacklisted variables

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 4222 bytes --]

On Thu, Sep 18, 2014 at 1:19 PM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

> Thanks for replying, Alex. Some comments below:
>
> On 18/09/2014 12:22, "Damian, Alexandru" <alexandru.damian@intel.com>
> wrote:
>
> >I have some points:
> >
> >
> >0. We face a major security risk here, and I can't stress this enough.
> >The variables can probably be used to make bitbake execute user input,
> >and exposing this functionality to 3rd parties, e.g. outside customers,
> >can easily lead to remote code execution, which
> > is a big no-no in cloud-based offerings.
> >
> >Do Not put hosted-mode code, where people can set ANY variables, into a
> >production setting, as it can expose your whole infrastructure to Bad
> >Guys TM. We need a major auditing effort in bitbake itself before putting
> >this into production, to make sure
> > that the variables we set cannot be exploited.
>
> Who should run this audit?
>
>
​Difficult to say; it's in the interest of those who would run toaster
services open to the public.​

If we would be the ones to host the Toaster service, the effort would have
to be supported by probably the whole Yocto team; we need to look ​into
vulnerabilities that can be triggered by setting variables, e.g. remote
code executions - I guess that Richard and Paul should take a look at how
variables can be exploited in Bitbake, and then we need to audit the core
layers too, since basically each recipe can do whatever it chooses with
variables.


> >
> >
> >The filtering that we're doing here is not designed to keep Bad People
> >out, but prevent inadvertent mistakes that can bring down the service for
> >Well Intentioned People.
> >
> >
> >
> >1. We need to make sure that the variable name check is implemented on
> >the server side too, not just on the frontend side.
>
> Fair point.
>
> >
> >
> >2. We need to add some variables here. Digging through the sources, I
> >came up with this list, by no means exhaustive.
>
> Is this on top of the list I've sent?
>

​Yep, in addition to.​



>
> >
> >
> >
> >BBPATH
> >
> >​TOPDIR​
> >
> >FAKEROOTCMD
> >FAKEROOTBASEENV
> >DL_DIR
> >BB_NUMBER_PARSE_THREADS
> >
> >ASSUME_PROVIDED
> >
> >PERSISTENT_DIR
> >
> >FETCH_CMD*
> >
> >BBSCHEDULER
> >
> >BBSCHEDULERS
> >
> >
> >
> >​
> >
> >
> >​Cheers,
> >Alex​
> >
> >
> >On Mon, Sep 15, 2014 at 3:50 PM, Barros Pena, Belen
> ><belen.barros.pena@intel.com> wrote:
> >
> >Resending this. Let's see if we can agree on a list of blacklisted
> >variables for Toaster. We need it for
> >
> >https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588
> >
> >Thanks!
> >
> >Belén
> >
> >
> >On 22/08/2014 15:25, "Barros Pena, Belen" <belen.barros.pena@intel.com>
> >wrote:
> >
> >>In prior discussions it was agreed that, at least for Toaster 1.7, we
> >>will
> >>blacklist any variables that influence how things will be built or where
> >>artefacts are stored. This means that Toaster will no allow you to set
> >>such variables (to see how, see page 13 of the design document attached
> >>to
> >>https://bugzilla.yoctoproject.org/show_bug.cgi?id=6588).
> >>
> >>I guess we should put together a list of the blacklisted variables. The
> >>following is a first go (mainly taken from local.conf and Hob). Please
> >>add
> >>to the list as needed:
> >>
> >>BB_NUMBER_THREADS
> >>PARALLEL_MAKE
> >>DL_DIR
> >>SSTATE_DIR
> >>TMPDIR
> >>BB_DISKMON_DIRS
> >>SSTATE_DIR
> >>SSTATE_MIRRORS
> >>http_proxy
> >>https_proxy
> >>ftp_proxy
> >>all_proxy
> >>CVS_PROXY_HOST
> >>CVS_PROXY_PORT
> >>
> >>
> >>Thanks!
> >>
> >>Belén
> >>
> >>
> >>
> >>--
> >>_______________________________________________
> >>toaster mailing list
> >>toaster@yoctoproject.org
> >>https://lists.yoctoproject.org/listinfo/toaster
> >
> >--
> >_______________________________________________
> >toaster mailing list
> >toaster@yoctoproject.org
> >https://lists.yoctoproject.org/listinfo/toaster
> >
> >
> >
> >
> >
> >
> >--
> >Alex Damian
> >Yocto Project
> >
> >SSG / OTC
> >
> >
> >
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 6705 bytes --]

Re: FW: [RFC] blacklisted variables

[Richard Purdie]

On Fri, 2014-09-19 at 11:56 +0100, Damian, Alexandru wrote:
> ​Difficult to say; it's in the interest of those who would run toaster
> services open to the public.​
>
> If we would be the ones to host the Toaster service, the effort would
> have to be supported by probably the whole Yocto team; we need to
> look ​into vulnerabilities that can be triggered by setting variables,
> e.g. remote code executions - I guess that Richard and Paul should
> take a look at how variables can be exploited in Bitbake, and then we
> need to audit the core layers too, since basically each recipe can do
> whatever it chooses with variables.

The nature of the way the metadata works is such that if you allow the
user any variable or metadata access, its like giving them a shell
account as the build user. 

I don't see any way to change that and any of these audits are just
going to hide the problem, not solve it since fundamentally, bitbake is
a code execution engine. A simple example is setting a variable to 
${@" ".join(open("/etc/shadow", "r").readlines())}. You could call into
your own function which opens up a socket and starts an ssh server quite
easily.

The structure of any public offering therefore needs to be structured
such that there are accounts and those accounts are controlled in some
way. The actual build slaves need to be monitored and probably
firewalled with the slave user accounts suitably locked down.

Cheers,

Richard

Re: FW: [RFC] blacklisted variables

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

I wholeheartedly agree to Richards' POV below. I already said that what we
do here is trying to make sure that good users don't accidentally knock
down servers, and it isn't enough to keep the bad guys out.

I would propose firing each build in it's own highly restricted VM (or LXC,
or something similarly suitable), but this hard lock-down needs to be
implemented by the infrastructure administrators, outside Toaster, and we
need to clearly express and document this need.


On Fri, Sep 19, 2014 at 12:26 PM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2014-09-19 at 11:56 +0100, Damian, Alexandru wrote:
> > ​Difficult to say; it's in the interest of those who would run toaster
> > services open to the public.​
> >
> > If we would be the ones to host the Toaster service, the effort would
> > have to be supported by probably the whole Yocto team; we need to
> > look ​into vulnerabilities that can be triggered by setting variables,
> > e.g. remote code executions - I guess that Richard and Paul should
> > take a look at how variables can be exploited in Bitbake, and then we
> > need to audit the core layers too, since basically each recipe can do
> > whatever it chooses with variables.
>
> The nature of the way the metadata works is such that if you allow the
> user any variable or metadata access, its like giving them a shell
> account as the build user.
>
> I don't see any way to change that and any of these audits are just
> going to hide the problem, not solve it since fundamentally, bitbake is
> a code execution engine. A simple example is setting a variable to
> ${@" ".join(open("/etc/shadow", "r").readlines())}. You could call into
> your own function which opens up a socket and starts an ssh server quite
> easily.
>
> The structure of any public offering therefore needs to be structured
> such that there are accounts and those accounts are controlled in some
> way. The actual build slaves need to be monitored and probably
> firewalled with the slave user accounts suitably locked down.
>
> Cheers,
>
> Richard
>
>
>
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 2812 bytes --]

Re: FW: [RFC] blacklisted variables

[Barros Pena, Belen]

So, with Alex additions, the list of blacklisted variables is like this:

BB_NUMBER_THREADS
PARALLEL_MAKE
DL_DIR
SSTATE_DIR
TMPDIR
BB_DISKMON_DIRS
SSTATE_DIR
SSTATE_MIRRORS
http_proxy
https_proxy
ftp_proxy
all_proxy
CVS_PROXY_HOST
BBPATH
​TOPDIR​
 
FAKEROOTCMD
FAKEROOTBASEENV
BB_NUMBER_PARSE_THREADS
ASSUME_PROVIDED
PERSISTENT_DIR
FETCH_CMD*
BBSCHEDULER
BBSCHEDULERS


Any others we should add?

Thanks!

Belén

On 22/09/2014 10:38, "Damian, Alexandru" <alexandru.damian@intel.com>
wrote:

>I wholeheartedly agree to Richards' POV below. I already said that what
>we do here is trying to make sure that good users don't accidentally
>knock down servers, and it isn't enough to keep the
> bad guys out.
>
>I would propose firing each build in it's own highly restricted VM (or
>LXC, or something similarly suitable), but this hard lock-down needs to
>be implemented by the infrastructure administrators, outside Toaster, and
>we need to clearly express and document
> this need.
> 
>
>
>
>On Fri, Sep 19, 2014 at 12:26 PM, Richard Purdie
><richard.purdie@linuxfoundation.org> wrote:
>
>On Fri, 2014-09-19 at 11:56 +0100, Damian, Alexandru wrote:
>> ​Difficult to say; it's in the interest of those who would run toaster
>> services open to the public.​
>>
>> If we would be the ones to host the Toaster service, the effort would
>> have to be supported by probably the whole Yocto team; we need to
>> look ​into vulnerabilities that can be triggered by setting variables,
>> e.g. remote code executions - I guess that Richard and Paul should
>> take a look at how variables can be exploited in Bitbake, and then we
>> need to audit the core layers too, since basically each recipe can do
>> whatever it chooses with variables.
>
>The nature of the way the metadata works is such that if you allow the
>user any variable or metadata access, its like giving them a shell
>account as the build user.
>
>I don't see any way to change that and any of these audits are just
>going to hide the problem, not solve it since fundamentally, bitbake is
>a code execution engine. A simple example is setting a variable to
>${@" ".join(open("/etc/shadow", "r").readlines())}. You could call into
>your own function which opens up a socket and starts an ssh server quite
>easily.
>
>The structure of any public offering therefore needs to be structured
>such that there are accounts and those accounts are controlled in some
>way. The actual build slaves need to be monitored and probably
>firewalled with the slave user accounts suitably locked down.
>
>Cheers,
>
>Richard
>
>
>
>
>
>
>
>
>
>
>-- 
>Alex Damian
>Yocto Project
>
>SSG / OTC 
>
>