Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Sergej Schumilo]

Dear all, 
after reporting the following bug to the Ubuntu security team, we were asked to report the bug directly to the kernel developers. 
I have attached the original bug report as well as a link to a zip archive containing all relevant files (including the oops and KASAN report and the causing HFS image).

https://ruhr-uni-bochum.sciebo.de/s/7J7paq4FvyKeMv1/download

——————————————————————

Dear all,
The following null pointer dereference bug was found by a modified version of the kAFL fuzzer (https://github.com/RUB-SysSec/kAFL). I have attached the causing hfs filesystem image, the dmesg report and the source code of a simple mounting tool to reproduce this issue.

A local users who have been granted the privileges necessary to mount filesystems (or a system components which auto mounts filesystems) could trigger a null pointer dereference or a kernel panic (depending on panic_on_oops).

We can verify this issues for Linux 4.15.0-15.16 (Ubuntu 16.04.4 LTS / sources from "pull-lp-source linux"). The desktop version of ubuntu auto-mounts this file system if provided via USB.

Credits: Sergej Schumilo, Cornelius Aschermann (both of Ruhr-Universität Bochum)

Best regards,
Sergej Schumilo

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Matthew Wilcox]

On Wed, Apr 18, 2018 at 06:26:41PM +0200, Sergej Schumilo wrote:
> Dear all,
> The following null pointer dereference bug was found by a modified version of the kAFL fuzzer (https://github.com/RUB-SysSec/kAFL). I have attached the causing hfs filesystem image, the dmesg report and the source code of a simple mounting tool to reproduce this issue.
> 
> A local users who have been granted the privileges necessary to mount filesystems (or a system components which auto mounts filesystems) could trigger a null pointer dereference or a kernel panic (depending on panic_on_oops).

I have to say this isn't going to rank very highly in terms of bugs we're
likely to spend a lot of time on.  Almost nobody uses HFS on Linux,
and it has no maintainer.  I'd suggest that Ubuntu disables it from
their configuration, or if it's important to them, that they contribute
somebody's time to working on it.

You'd probably get a more interesting response if you fuzzed ext4, btrfs
or XFS.  Or FAT or iso9660; things people are likely to have an USB keys.
There may be a tooling problem here where some filesystems should be
whitelisted for automounting.  I just don't think you're going to find
anyone interested in fixing this.

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Eric Biggers]

On Wed, Apr 18, 2018 at 10:30:28AM -0700, Matthew Wilcox wrote:
> On Wed, Apr 18, 2018 at 06:26:41PM +0200, Sergej Schumilo wrote:
> > Dear all,
> > The following null pointer dereference bug was found by a modified version of the kAFL fuzzer (https://github.com/RUB-SysSec/kAFL). I have attached the causing hfs filesystem image, the dmesg report and the source code of a simple mounting tool to reproduce this issue.
> > 
> > A local users who have been granted the privileges necessary to mount filesystems (or a system components which auto mounts filesystems) could trigger a null pointer dereference or a kernel panic (depending on panic_on_oops).
> 
> I have to say this isn't going to rank very highly in terms of bugs we're
> likely to spend a lot of time on.  Almost nobody uses HFS on Linux,
> and it has no maintainer.  I'd suggest that Ubuntu disables it from
> their configuration, or if it's important to them, that they contribute
> somebody's time to working on it.
> 
> You'd probably get a more interesting response if you fuzzed ext4, btrfs
> or XFS.  Or FAT or iso9660; things people are likely to have an USB keys.
> There may be a tooling problem here where some filesystems should be
> whitelisted for automounting.  I just don't think you're going to find
> anyone interested in fixing this.

Also this looks the same as this bug that was already reported by syzbot, with a
C reproducer:

    Thread: https://groups.google.com/forum/#!msg/syzkaller-bugs/9SgQk_6tSZ4/zLhTm4r1AwAJ
    Dashboard link: https://syzkaller.appspot.com/bug?id=c2fd07ae5213991a1b95eadee9205b41ac1aa6e2

In fact there are already 4 open HFS bugs on the syzbot dashboard at
https://syzkaller.appspot.com/.  IMO, time would be much better spent actually
fixing the bugs, rather than redundantly finding them again.

Also Sergej, I know that you want to consider this a "security bug" and report
it to "security" teams, and maybe even file a CVE number.  But the reality is
that NULL pointer dereferences rarely have much security impact, and many
"security" teams seem to be wasting so much time with such bugs that they are
ignoring bugs with actual security impact, like the 34 use-after-free bugs that
are currently open in the syzbot dashbard.  So IMO, going through the full
security circus on NULL pointer dereferences is actually detriminal to security.
(Though, they still need to be fixed of course!)

Thanks,

Eric

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Matthew Wilcox]

On Wed, Apr 18, 2018 at 10:54:21AM -0700, Eric Biggers wrote:
> Also Sergej, I know that you want to consider this a "security bug" and report
> it to "security" teams, and maybe even file a CVE number.  But the reality is
> that NULL pointer dereferences rarely have much security impact, and many
> "security" teams seem to be wasting so much time with such bugs that they are
> ignoring bugs with actual security impact, like the 34 use-after-free bugs that
> are currently open in the syzbot dashbard.  So IMO, going through the full
> security circus on NULL pointer dereferences is actually detriminal to security.
> (Though, they still need to be fixed of course!)

I don't think this really needs to be fixed.  I think the security bug
is that Ubuntu have configured their system in such a way that it will
attempt to automount an HFS filesystem on a USB key.  By going through
FUSE or some other userspace filesystem, the security risk would be
eliminated.

Is it time to start moving unmaintained obsolescent filesystems with
few remaining users into staging?  ... Hey, that sounds like a good
topic for next week!

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Darrick J. Wong]

On Wed, Apr 18, 2018 at 10:30:28AM -0700, Matthew Wilcox wrote:
> On Wed, Apr 18, 2018 at 06:26:41PM +0200, Sergej Schumilo wrote:
> > Dear all,
> > The following null pointer dereference bug was found by a modified version of the kAFL fuzzer (https://github.com/RUB-SysSec/kAFL). I have attached the causing hfs filesystem image, the dmesg report and the source code of a simple mounting tool to reproduce this issue.
> > 
> > A local users who have been granted the privileges necessary to mount filesystems (or a system components which auto mounts filesystems) could trigger a null pointer dereference or a kernel panic (depending on panic_on_oops).
> 
> I have to say this isn't going to rank very highly in terms of bugs we're
> likely to spend a lot of time on.  Almost nobody uses HFS on Linux,
> and it has no maintainer.  I'd suggest that Ubuntu disables it from
> their configuration, or if it's important to them, that they contribute
> somebody's time to working on it.
> 
> You'd probably get a more interesting response if you fuzzed ext4, btrfs
> or XFS.  Or FAT or iso9660; things people are likely to have an USB keys.
> There may be a tooling problem here where some filesystems should be
> whitelisted for automounting.  I just don't think you're going to find
> anyone interested in fixing this.

They already are fuzzing ext4 and xfs and generating a fair amount of
list / bugzilla traffic.  Regrettable that almost nobody sends us
patches to fix the things they find, which at some point is just going
to make the review backlog problems worse as peoples' attention get
diverted to triaging the flood of reports.

Sorry just being a grumpy maintainer.

--D

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Dave Chinner]

On Wed, Apr 18, 2018 at 10:59:06AM -0700, Darrick J. Wong wrote:
> On Wed, Apr 18, 2018 at 10:30:28AM -0700, Matthew Wilcox wrote:
> > On Wed, Apr 18, 2018 at 06:26:41PM +0200, Sergej Schumilo wrote:
> > > Dear all,
> > > The following null pointer dereference bug was found by a modified version of the kAFL fuzzer (https://github.com/RUB-SysSec/kAFL). I have attached the causing hfs filesystem image, the dmesg report and the source code of a simple mounting tool to reproduce this issue.
> > > 
> > > A local users who have been granted the privileges necessary to mount filesystems (or a system components which auto mounts filesystems) could trigger a null pointer dereference or a kernel panic (depending on panic_on_oops).
> > 
> > I have to say this isn't going to rank very highly in terms of bugs we're
> > likely to spend a lot of time on.  Almost nobody uses HFS on Linux,
> > and it has no maintainer.  I'd suggest that Ubuntu disables it from
> > their configuration, or if it's important to them, that they contribute
> > somebody's time to working on it.
> > 
> > You'd probably get a more interesting response if you fuzzed ext4, btrfs
> > or XFS.  Or FAT or iso9660; things people are likely to have an USB keys.
> > There may be a tooling problem here where some filesystems should be
> > whitelisted for automounting.  I just don't think you're going to find
> > anyone interested in fixing this.
> 
> They already are fuzzing ext4 and xfs and generating a fair amount of
> list / bugzilla traffic.  Regrettable that almost nobody sends us
> patches to fix the things they find, which at some point is just going
> to make the review backlog problems worse as peoples' attention get
> diverted to triaging the flood of reports.

I think that the other thing to note here is that the fuzzing being
done is so far from the state of the art that it's probably harmful
to ongoing development rather than improving anything.

i.e. Fuzzing legacy filesystem formats that don't have CRC
protection for the metadata and/or redundant information to reliably
detect corruption doesn't help us improve the resilience of modern
filesystems.  We know we cannot detect random bit corruptions in
these legacy filesystem formats and we've known this for a long
time. In the case of XFS, we fixed this vulnerability to random bit
corruptions years ago and we have defaulted to creating CRC enabled
filesystems for the past few years. Hence there's a large (and ever
growing!) pool of users whose filesystems will detect random bit
corruptions in metadata and just do the right thing.

i.e. random bit fuzzing of the legacy format doesn't help us find
problems that we'll come across in the future, because we know we
our filesystems already reject >99.999% of random structure
corruptions that random bit fuzzers introduce via the CRC checking
mechanisms. Spending time chasing stuff like this down takes away
from doing things like making filesystem be able to do instantenous
online repair of detected corruptions.

So what we really need to know about is the corruption problems that
get through the CRC checking and the more robust verification that
the current on-disk formats fail to catch.  We've already got
CRC-defeating, structure aware fuzzing in fstests for v5 filesystems
[see common/fuzzy and the 75+ tests that make use of that
infrastructure for fuzzing XFS filesystems].

Anyone wanting to fuzz XFS filesystems needs to look as this stuff
and then either build on top of it or make a better tool we can use
for doing this sort of testing inside fstests. i.e. adding more
tests to fstests that improve fuzzing coverage of v5 filesystems is
far more useful to us than throwing broken v4 filesystem images over
the wall at us.

> Sorry just being a grumpy maintainer.

Sorry, just being a grumpy engineer.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Null-Pointer Deference in hfs.ko (Linux 4.15.0-15.16 Ubuntu)

[Darrick J. Wong]

On Thu, Apr 19, 2018 at 11:44:02AM +1000, Dave Chinner wrote:
> On Wed, Apr 18, 2018 at 10:59:06AM -0700, Darrick J. Wong wrote:
> > On Wed, Apr 18, 2018 at 10:30:28AM -0700, Matthew Wilcox wrote:
> > > On Wed, Apr 18, 2018 at 06:26:41PM +0200, Sergej Schumilo wrote:
> > > > Dear all,
> > > > The following null pointer dereference bug was found by a
> > > > modified version of the kAFL fuzzer
> > > > (https://github.com/RUB-SysSec/kAFL). I have attached the
> > > > causing hfs filesystem image, the dmesg report and the source
> > > > code of a simple mounting tool to reproduce this issue.
> > > > 
> > > > A local users who have been granted the privileges necessary to
> > > > mount filesystems (or a system components which auto mounts
> > > > filesystems) could trigger a null pointer dereference or a
> > > > kernel panic (depending on panic_on_oops).
> > > 
> > > I have to say this isn't going to rank very highly in terms of bugs we're
> > > likely to spend a lot of time on.  Almost nobody uses HFS on Linux,
> > > and it has no maintainer.  I'd suggest that Ubuntu disables it from
> > > their configuration, or if it's important to them, that they contribute
> > > somebody's time to working on it.
> > > 
> > > You'd probably get a more interesting response if you fuzzed ext4, btrfs
> > > or XFS.  Or FAT or iso9660; things people are likely to have an USB keys.
> > > There may be a tooling problem here where some filesystems should be
> > > whitelisted for automounting.  I just don't think you're going to find
> > > anyone interested in fixing this.
> > 
> > They already are fuzzing ext4 and xfs and generating a fair amount of
> > list / bugzilla traffic.  Regrettable that almost nobody sends us
> > patches to fix the things they find, which at some point is just going
> > to make the review backlog problems worse as peoples' attention get
> > diverted to triaging the flood of reports.
> 
> I think that the other thing to note here is that the fuzzing being
> done is so far from the state of the art that it's probably harmful
> to ongoing development rather than improving anything.
> 
> i.e. Fuzzing legacy filesystem formats that don't have CRC
> protection for the metadata and/or redundant information to reliably
> detect corruption doesn't help us improve the resilience of modern
> filesystems.  We know we cannot detect random bit corruptions in
> these legacy filesystem formats and we've known this for a long
> time. In the case of XFS, we fixed this vulnerability to random bit
> corruptions years ago and we have defaulted to creating CRC enabled
> filesystems for the past few years. Hence there's a large (and ever
> growing!) pool of users whose filesystems will detect random bit
> corruptions in metadata and just do the right thing.
> 
> i.e. random bit fuzzing of the legacy format doesn't help us find
> problems that we'll come across in the future, because we know we
> our filesystems already reject >99.999% of random structure
> corruptions that random bit fuzzers introduce via the CRC checking
> mechanisms. Spending time chasing stuff like this down takes away
> from doing things like making filesystem be able to do instantenous
> online repair of detected corruptions.

Or at least write new CRCs in anger like I do. >:O

> So what we really need to know about is the corruption problems that
> get through the CRC checking and the more robust verification that
> the current on-disk formats fail to catch.  We've already got
> CRC-defeating, structure aware fuzzing in fstests for v5 filesystems
> [see common/fuzzy and the 75+ tests that make use of that
> infrastructure for fuzzing XFS filesystems].

Seriously.

$ grep fuzzer ./xfstests/tests/xfs/group  | wc -l
112

Granted, the online fsck fuzzers (350-430) spray false positives
everywhere so you'll want to figure out a baseline of what's broken (ok
ok I have one djwong/xfstests.git on kernel.org) and figure out which
ones look interesting.

> Anyone wanting to fuzz XFS filesystems needs to look as this stuff
> and then either build on top of it or make a better tool we can use
> for doing this sort of testing inside fstests. i.e. adding more
> tests to fstests that improve fuzzing coverage of v5 filesystems is
> far more useful to us than throwing broken v4 filesystem images over
> the wall at us.

Or just improving the ones we have already.  That giant patchbomb I sent
to the xfs list yesterday?  That has a /lot/ of patches to fix problems
that the fuzzers have been smashing into on a regular basis, and that's
not including the online fsck code.  That's what I meant when I say that
my employers care enough to let me fuzz, triage, and fix, not just dump
reports on the mailing list.

--D

> 
> > Sorry just being a grumpy maintainer.
> 
> Sorry, just being a grumpy engineer.
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com