Backdoor in xz, should we switch compression format for tarballs?

Backdoor in xz, should we switch compression format for tarballs?

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 304 bytes --]

For more info, see
https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/
but, essentially, xz was backdoored and it seems like upstream was directly
responsible for this.

Based on this, should we switch our distribution from bz2+xz to bz2+zstd or
bz2+lzip?

Thanks,

Paolo

[-- Attachment #2: Type: text/html, Size: 564 bytes --]

Re: Backdoor in xz, should we switch compression format for tarballs?

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

Um maybe?

From what I've read so far it doesn't seem the format is compromised but it
certainly seems like a concerted attempt to subvert an upstream. However a
knee-jerk jump to another format might be premature without carefully
considering if other upstreams have been targeted.

I guess zstd is overseen by Facebook but it's still a mostly single
contributor repo. Lzip's history directly ties to the original author of xz
and we haven't heard from them yet.

We should certainly keep an eye on the situation but let's not be too hasty.

On Sat, 30 Mar 2024, 05:00 Paolo Bonzini, <pbonzini@redhat.com> wrote:

> For more info, see
> https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/
> but, essentially, xz was backdoored and it seems like upstream was directly
> responsible for this.
>
> Based on this, should we switch our distribution from bz2+xz to bz2+zstd
> or bz2+lzip?
>
> Thanks,
>
> Paolo
>

[-- Attachment #2: Type: text/html, Size: 1692 bytes --]

Re: Backdoor in xz, should we switch compression format for tarballs?

[Daniel P. Berrangé]

On Fri, Mar 29, 2024 at 06:59:30PM +0100, Paolo Bonzini wrote:
> For more info, see
> https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/
> but, essentially, xz was backdoored and it seems like upstream was directly
> responsible for this.
> 
> Based on this, should we switch our distribution from bz2+xz to bz2+zstd or
> bz2+lzip?

Based on the attack vector of pre-loading git with an exploit, but then
modifying the tarball to activate it, there's a bigger question of whether
users should really trust manually created tarballs at all ? You don't
anything about either the tarball creator, or the state of creators' machine,
even if it is signed. How can you trust that its contents is a faithful
representation of the tagged release from git it claims to be?

This issue could prompt a push towards distros only handling tarballs
directly auto-generated from a git tag, in a reliably reproducible manner.

Obviously you couldn't actually trust the upstream maintainer in this
case, but at least if you're using a reproducible git tarball you can
verify every link in the chain right through each git commit, and don't
have this manual tarball whose contents need to be to picked apart.

TL;DR; I think we should consider our tarball distribution options, but
lets wait for the dust to settle and not rush into decisions.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Backdoor in xz, should we switch compression format for tarballs?

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 1956 bytes --]

Also does qemu link to libarchive? The original analysis wasn't a full
reverse engineer of the payload so we don't know if it only affects sshd.

On Sat, 30 Mar 2024, 07:01 Daniel P. Berrangé, <berrange@redhat.com> wrote:

> On Fri, Mar 29, 2024 at 06:59:30PM +0100, Paolo Bonzini wrote:
> > For more info, see
> >
> https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/
> > but, essentially, xz was backdoored and it seems like upstream was
> directly
> > responsible for this.
> >
> > Based on this, should we switch our distribution from bz2+xz to bz2+zstd
> or
> > bz2+lzip?
>
> Based on the attack vector of pre-loading git with an exploit, but then
> modifying the tarball to activate it, there's a bigger question of whether
> users should really trust manually created tarballs at all ? You don't
> anything about either the tarball creator, or the state of creators'
> machine,
> even if it is signed. How can you trust that its contents is a faithful
> representation of the tagged release from git it claims to be?
>
> This issue could prompt a push towards distros only handling tarballs
> directly auto-generated from a git tag, in a reliably reproducible manner.
>
> Obviously you couldn't actually trust the upstream maintainer in this
> case, but at least if you're using a reproducible git tarball you can
> verify every link in the chain right through each git commit, and don't
> have this manual tarball whose contents need to be to picked apart.
>
> TL;DR; I think we should consider our tarball distribution options, but
> lets wait for the dust to settle and not rush into decisions.
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|
>
>
>

[-- Attachment #2: Type: text/html, Size: 3021 bytes --]

Re: Backdoor in xz, should we switch compression format for tarballs?

[Stefan Hajnoczi]

On Fri, 29 Mar 2024 at 14:00, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> For more info, see https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/ but, essentially, xz was backdoored and it seems like upstream was directly responsible for this.
>
> Based on this, should we switch our distribution from bz2+xz to bz2+zstd or bz2+lzip?

I think it's reasonable to drop xz as a precaution due to the
long-term control the attacker may have had over the code base. I
haven't researched the alternatives though.

I CCed Michael Tokarev because he looked at compression formats for
distributing QEMU recently and may have thoughts on which alternative
is suitable.

For the record, I confirmed that the following QEMU servers do not
have xz-utils 5.6.0 or 5.6.1 packages installed:
- shell1.qemu.org
- node1.qemu.org
- ci1 at OSUOSL
- qemu2.osuosl.org

Stefan

Re: Backdoor in xz, should we switch compression format for tarballs?

[Michael Tokarev]

30.03.2024 13:03, Stefan Hajnoczi :
> On Fri, 29 Mar 2024 at 14:00, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> For more info, see https://lwn.net/ml/oss-security/20240329155126.kjjfduxw2yrlxgzm@awork3.anarazel.de/ but, essentially, xz was backdoored and it seems like upstream was directly responsible for this.
>>
>> Based on this, should we switch our distribution from bz2+xz to bz2+zstd or bz2+lzip?
> 
> I think it's reasonable to drop xz as a precaution due to the
> long-term control the attacker may have had over the code base. I
> haven't researched the alternatives though.

I agree with Daniel here, - lets' not rush into conclusions so far.

Even with this long-term control, so far it does not look like .xz
format itself is somehow bad (but it can be improved for sure), or
it poses a treat.

> I CCed Michael Tokarev because he looked at compression formats for
> distributing QEMU recently and may have thoughts on which alternative
> is suitable.

The only my intention at the time was to avoid keeping things in *two*
forms, - as it looked like there's no reason for that.   My reasons was
that .xz is used for quite some time as default download link on qemu.org
website so it should be safe to assume everyone has .xz support by now
and there's no need to keep .bz2.  Now with this incident in mind, maybe
that wasn't a good idea and some other format should be kept still.

But once again, - I think it's a bit preliminary to make decisions while
the dust still not settled.

/mjt