Fw: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE

Fw: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE

[Stephen Hemminger]

Begin forwarded message:

Date: Mon, 28 Oct 2019 02:55:44 +0000
From: bugzilla-daemon@bugzilla.kernel.org
To: stephen@networkplumber.org
Subject: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE


https://bugzilla.kernel.org/show_bug.cgi?id=205339

            Bug ID: 205339
           Summary: epoll can fail to report a socket readable after
                    enabling SO_OOBINLINE
           Product: Networking
           Version: 2.5
    Kernel Version: 5.0
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: low
          Priority: P1
         Component: Other
          Assignee: stephen@networkplumber.org
          Reporter: njs@pobox.com
        Regression: No

Created attachment 285671
  --> https://bugzilla.kernel.org/attachment.cgi?id=285671&action=edit  
reproducer

Consider the following sequence of events:

1. OOB data arrives on a socket.
2. The socket is registered with epoll with EPOLLIN
3. The socket has SO_OOBINLINE toggled from False → True

In this case, the socket is now readable, and select() reports that it's
readable, but epoll does *not* report that it's readable.

This is a pretty minor issue, but it seems like an unambiguous bug so I figured
I'd report it.

Weirdly, this doesn't appear to be a general problem with SO_OOBINLINE+epoll.
For example, this very similar sequence works correctly:

1. The socket is registered with epoll with EPOLLIN
2. OOB data arrives on the socket.
3. The socket has SO_OOBINLINE toggled from False → True

After step 2, epoll reports the socket as not readable, and then after step 3
it reports it as readable, as you'd expect.

In the attached reproducer script, "scenario 4" is the buggy one, and "scenario
3" is the very similar non-buggy one. Output on Ubuntu 19.04, kernel
5.0.0-32-generic, x86-64:

-- Scenario 1: no data --
select() says: sock is NOT readable
epoll says: sock is NOT readable
reality: NOT readable

-- Scenario 2: OOB data arrives --
select() says: sock is NOT readable
epoll says: sock is NOT readable
reality: NOT readable

-- Scenario 3: register -> OOB data arrives -> toggle SO_OOBINLINE=True --
select() says: sock is readable
epoll says: sock is readable
reality: read succeeded

-- Scenario 4: OOB data arrives -> register -> toggle SO_OOBINLINE=True --
select() says: sock is readable
epoll says: sock is NOT readable
reality: read succeeded

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: Fw: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE

[Eric Dumazet]

Please Stephen CC the reporter when you forward a bugzilla bug to the list

On 10/28/19 8:11 AM, Stephen Hemminger wrote:
> 
> 
> Begin forwarded message:
> 
> Date: Mon, 28 Oct 2019 02:55:44 +0000
> From: bugzilla-daemon@bugzilla.kernel.org
> To: stephen@networkplumber.org
> Subject: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE
> 
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=205339
> 
>             Bug ID: 205339
>            Summary: epoll can fail to report a socket readable after
>                     enabling SO_OOBINLINE
>            Product: Networking
>            Version: 2.5
>     Kernel Version: 5.0
>           Hardware: All
>                 OS: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: low
>           Priority: P1
>          Component: Other
>           Assignee: stephen@networkplumber.org
>           Reporter: njs@pobox.com
>         Regression: No

> Created attachment 285671
>   --> https://bugzilla.kernel.org/attachment.cgi?id=285671&action=edit  
> reproducer
> 
> Consider the following sequence of events:
> 
> 1. OOB data arrives on a socket.
> 2. The socket is registered with epoll with EPOLLIN
> 3. The socket has SO_OOBINLINE toggled from False → True
> 
> In this case, the socket is now readable, and select() reports that it's
> readable, but epoll does *not* report that it's readable.
> 
> This is a pretty minor issue, but it seems like an unambiguous bug so I figured
> I'd report it.
> 
> Weirdly, this doesn't appear to be a general problem with SO_OOBINLINE+epoll.
> For example, this very similar sequence works correctly:
> 
> 1. The socket is registered with epoll with EPOLLIN
> 2. OOB data arrives on the socket.
> 3. The socket has SO_OOBINLINE toggled from False → True
> 
> After step 2, epoll reports the socket as not readable, and then after step 3
> it reports it as readable, as you'd expect.
> 
> In the attached reproducer script, "scenario 4" is the buggy one, and "scenario
> 3" is the very similar non-buggy one. Output on Ubuntu 19.04, kernel
> 5.0.0-32-generic, x86-64:
> 
> -- Scenario 1: no data --
> select() says: sock is NOT readable
> epoll says: sock is NOT readable
> reality: NOT readable
> 
> -- Scenario 2: OOB data arrives --
> select() says: sock is NOT readable
> epoll says: sock is NOT readable
> reality: NOT readable
> 
> -- Scenario 3: register -> OOB data arrives -> toggle SO_OOBINLINE=True --
> select() says: sock is readable
> epoll says: sock is readable
> reality: read succeeded
> 
> -- Scenario 4: OOB data arrives -> register -> toggle SO_OOBINLINE=True --
> select() says: sock is readable
> epoll says: sock is NOT readable
> reality: read succeeded
> 

I really wonder how much energy we should put in maintaining this archaic thing.

We do not have a single packetdrill test at Google using URG stuff.

Re: Fw: [Bug 205339] New: epoll can fail to report a socket readable after enabling SO_OOBINLINE

[Nathaniel Smith]

On Mon, Oct 28, 2019 at 2:22 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > -- Scenario 4: OOB data arrives -> register -> toggle SO_OOBINLINE=True --
> > select() says: sock is readable
> > epoll says: sock is NOT readable
> > reality: read succeeded
> >
>
> I really wonder how much energy we should put in maintaining this archaic thing.
>
> We do not have a single packetdrill test at Google using URG stuff.

Yeah, URG is pretty useless. I didn't find this because I was trying
to use URG; I found it because I was trying to avoid having to think
about URG :-).

The problem with URG is that it lets untrusted remote peers trivially
trigger weird socket semantics that most userspace developers haven't
tested or thought about at all. Once I started looking around I found
lots of prominent apps that react badly to receiving URG, plus there's
a history of nasty stuff like [1]. SO_OOBINLINE is interesting because
it makes the URG semantics more similar to the regular semantics that
apps are expecting, and empirically it would have mitigated lots of
these bugs. So I started wondering whether we should enable
SO_OOBINLINE unconditionally in the networking library I maintain, as
a general hardening measure, and while writing tests for that I
stumbled on this bug.

This specific bug is pretty unimportant... in practice you'd always
enable SO_OOBINLINE when a socket is created, not after you're already
polling it, so whatever. For the larger question about maintaining the
archaic URG code: having *some* kind of predictable semantics is
important! Though for me, even SO_OOBINLINE is fancier than I really
want; I'd be just as happy if the way to get boring, predictable
semantics was a simple SO_DISABLEOOBENTIRELY. I can also see an
argument for offering a system-level config option to disable URG
handling globally, and encouraging distros to turn it on...

-n

[1] https://sandstorm.io/news/2015-04-08-osx-security-bug

-- 
Nathaniel J. Smith -- https://vorpus.org