Re: [OE-core] TMPDIR, bitbake v posix

Re: [OE-core] TMPDIR, bitbake v posix

[Rasmus Villemoes]

On Fri, Jan 16 2026, "Rasmus Villemoes via lists.openembedded.org" <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:

> Hi
>
> When some task uses mktemp(1) or similar functionality (e.g. the
> python tempfile module) to create a temporary file or dir, those end up
> being created in the global /tmp directory. That's obviously not ideal,
> as those might be left behind in case of an unclean shutdown or if the
> script is not properly written, and won't be cleaned by do_clean or
> RM_WORK. Also, /tmp is often a tmpfs instance, while some tasks might
> want to create rather large temporary files.
>
> Most such utilities/interfaces respect the TMPDIR environment variable,
> if set. However, since that name is used by bitbake for a rather central
> concept, one cannot simply 'export TMPDIR', as one doesn't want that
> top-level build dir polluted by random build files, and it still
> wouldn't be removed by do_clean or RM_WORK. Ideally, all tasks would
> have a TMPDIR _environment_ variable set to something below ${WORKDIR},
> maybe even something unique per task (say ${T}/${taskname}.d). But
> there's no simple way to do that that doesn't conflict with bitbake's
> use of the TMPDIR name.
>
> Can we do anything about this?
>
> (1) Do nothing, live with this limitation.
>
> (2) Special-case TMPDIR, so that all tasks are created with a proper
> value of TMPDIR in the environment, not derived from the bitbake
> variable of the same name. Could be settable with some new TASK_TMPDIR
> or do_foo[tmpdir] flag. Somewhat hacky, though.
>
> (3) Bite the bullet and plan for renaming TMPDIR -> BBTMPDIR, so that
> the TMPDIR name becomes available for use with its standard POSIX
> meaning. Obviously a huge thing to do, but hopefully most direct TMPDIR
> references are in core or bitbake and used for deriving other standard
> variables.

I tried doing TMPDIR -> BB_TMPDIR in bitbake and most places in core
(there are some places which use TMPDIR in the posix sense), and it
seems to build a core-image-minimal without problems.

So it's not impossible, and one could prepare for the transition by
eliminating quite a few direct TMPDIR references using already existing
or new derived variables (e.g. BASE_SHAREDWORKDIR =
${TMPDIR}/work-shared), making the number of places that would need
the rename rather small.

I'd be happy to send such preparatory patches, but it would be pointless
if the community doesn't deem this to be a problem worth fixing.

Rasmus

Re: [OE-core] TMPDIR, bitbake v posix

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 3015 bytes --]

On Mon, 2026-03-16 at 10:24 +0100, Rasmus Villemoes via
lists.openembedded.org wrote:
> On Fri, Jan 16 2026, "Rasmus Villemoes via lists.openembedded.org" <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:
> 
> > Hi
> > 
> > When some task uses mktemp(1) or similar functionality (e.g. the
> > python tempfile module) to create a temporary file or dir, those end up
> > being created in the global /tmp directory. That's obviously not ideal,
> > as those might be left behind in case of an unclean shutdown or if the
> > script is not properly written, and won't be cleaned by do_clean or
> > RM_WORK. Also, /tmp is often a tmpfs instance, while some tasks might
> > want to create rather large temporary files.
> > 
> > Most such utilities/interfaces respect the TMPDIR environment variable,
> > if set. However, since that name is used by bitbake for a rather central
> > concept, one cannot simply 'export TMPDIR', as one doesn't want that
> > top-level build dir polluted by random build files, and it still
> > wouldn't be removed by do_clean or RM_WORK. Ideally, all tasks would
> > have a TMPDIR _environment_ variable set to something below ${WORKDIR},
> > maybe even something unique per task (say ${T}/${taskname}.d). But
> > there's no simple way to do that that doesn't conflict with bitbake's
> > use of the TMPDIR name.
> > 
> > Can we do anything about this?
> > 
> > (1) Do nothing, live with this limitation.
> > 
> > (2) Special-case TMPDIR, so that all tasks are created with a proper
> > value of TMPDIR in the environment, not derived from the bitbake
> > variable of the same name. Could be settable with some new TASK_TMPDIR
> > or do_foo[tmpdir] flag. Somewhat hacky, though.
> > 
> > (3) Bite the bullet and plan for renaming TMPDIR -> BBTMPDIR, so that
> > the TMPDIR name becomes available for use with its standard POSIX
> > meaning. Obviously a huge thing to do, but hopefully most direct TMPDIR
> > references are in core or bitbake and used for deriving other standard
> > variables.
> 
> I tried doing TMPDIR -> BB_TMPDIR in bitbake and most places in core
> (there are some places which use TMPDIR in the posix sense), and it
> seems to build a core-image-minimal without problems.
> 
> So it's not impossible, and one could prepare for the transition by
> eliminating quite a few direct TMPDIR references using already existing
> or new derived variables (e.g. BASE_SHAREDWORKDIR =
> ${TMPDIR}/work-shared), making the number of places that would need
> the rename rather small.
> 
> I'd be happy to send such preparatory patches, but it would be pointless
> if the community doesn't deem this to be a problem worth fixing.

Hi Rasmus,

I think it's worth trying to disambiguate between the two uses of TMPDIR
if we can, but it's not a change we can take this close to the next LTS
release. I recommend sending an RFC patch for people to ealuate, after
the 6.0 "wrynose" release.

Thanks,

-- 
Paul Barker



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [OE-core] TMPDIR, bitbake v posix

[Richard Purdie]

On Mon, 2026-03-16 at 10:24 +0100, Rasmus Villemoes wrote:
> On Fri, Jan 16 2026, "Rasmus Villemoes via lists.openembedded.org" <rasmus.villemoes=prevas.dk@lists.openembedded.org> wrote:
> 
> > Hi
> > 
> > When some task uses mktemp(1) or similar functionality (e.g. the
> > python tempfile module) to create a temporary file or dir, those end up
> > being created in the global /tmp directory. That's obviously not ideal,
> > as those might be left behind in case of an unclean shutdown or if the
> > script is not properly written, and won't be cleaned by do_clean or
> > RM_WORK. Also, /tmp is often a tmpfs instance, while some tasks might
> > want to create rather large temporary files.
> > 
> > Most such utilities/interfaces respect the TMPDIR environment variable,
> > if set. However, since that name is used by bitbake for a rather central
> > concept, one cannot simply 'export TMPDIR', as one doesn't want that
> > top-level build dir polluted by random build files, and it still
> > wouldn't be removed by do_clean or RM_WORK. Ideally, all tasks would
> > have a TMPDIR _environment_ variable set to something below ${WORKDIR},
> > maybe even something unique per task (say ${T}/${taskname}.d). But
> > there's no simple way to do that that doesn't conflict with bitbake's
> > use of the TMPDIR name.
> > 
> > Can we do anything about this?
> > 
> > (1) Do nothing, live with this limitation.
> > 
> > (2) Special-case TMPDIR, so that all tasks are created with a proper
> > value of TMPDIR in the environment, not derived from the bitbake
> > variable of the same name. Could be settable with some new TASK_TMPDIR
> > or do_foo[tmpdir] flag. Somewhat hacky, though.
> > 
> > (3) Bite the bullet and plan for renaming TMPDIR -> BBTMPDIR, so that
> > the TMPDIR name becomes available for use with its standard POSIX
> > meaning. Obviously a huge thing to do, but hopefully most direct TMPDIR
> > references are in core or bitbake and used for deriving other standard
> > variables.
> 
> I tried doing TMPDIR -> BB_TMPDIR in bitbake and most places in core
> (there are some places which use TMPDIR in the posix sense), and it
> seems to build a core-image-minimal without problems.
> 
> So it's not impossible, and one could prepare for the transition by
> eliminating quite a few direct TMPDIR references using already existing
> or new derived variables (e.g. BASE_SHAREDWORKDIR =
> ${TMPDIR}/work-shared), making the number of places that would need
> the rename rather small.
> 
> I'd be happy to send such preparatory patches, but it would be pointless
> if the community doesn't deem this to be a problem worth fixing.

I'm not against the idea however right now, we have several other
ongoing issues and the feature freeze for the LTS looming one week
away. I'm struggling to get the intermittent failures in our automated
testing under control and this particular issue doesn't seem as
pressing as others. I feel like I'm drowning with different problems
and adding the fall out from this change isn't particularly appealing
to me.

The change will potentially have huge impact as you mention and it
needs to be handled correctly.

I've thought "tmp" was a bad choice of name so if we do this, we should
take the opportunity to find a better one. The first step would
therefore probably be to get agreement on what the new name would be.

As well as getting the general community buy in on the idea, we'd also
need to agree when we'd try and make the change.

Cheers,

Richard