Not receiving SCTP_SHUTDOWN_COMP notifications

Not receiving SCTP_SHUTDOWN_COMP notifications

[Julien Pourtet]

Hi,

I am digging up a thread from the lksctp-developers mailing list:
http://sourceforge.net/p/lksctp/mailman/message/9054592/

At the time of the post (March, 2005) it looked like
SCTP_SHUTDOWN_COMP SCTP_ASSOC_CHANGE notifications were not delivered
to the application due to too restrictive a check in
ulpqueue.c:sctp_ulpq_tail_event().

As of today, using Linux Kernel 4.4.0-rc3, I experience the same
behaviour. I am using a tweaked version of
https://github.com/tdimitrov/sctp-sandbox/blob/one-to-many_noitf/ to
reproduce the case.

Note that the server app does see SCTP_SHUTDOWN_COMP notifications.
The issue is on the client app.

My "tweak" for the client app is using a one-to-one socket
(SOCK_STREAM) instead of one-to-many. I also perform a
shutdown(SHUT_RDWR) before close() is called, as well as a recvmsg()
[get_reply() in that code base semantics] to receive the notifications
before closing the file descriptor.

Commenting out the following lines in ulpqueue.c does get the
notifications delivered to the client app:
         /* If the socket is just going to throw this away, do not
          * even try to deliver it.
          */
         if (sock_flag(sk, SOCK_DEAD) || (sk->sk_shutdown & RCV_SHUTDOWN))
                 goto out_free;

However, I guess that's not a viable solution as this may have
undesirable side-effects (I am not familiar with this code base). It
was more of a sanity check for me.

Question is, is this a bug? Should we fix it?

[Note: it's my first time writing to this ML, sorry if I am breaking
any implicit rules or explicit conventions].

Cheers,
Julien

Re: Not receiving SCTP_SHUTDOWN_COMP notifications

[Marcelo Ricardo Leitner]

Hi,

On Mon, Nov 30, 2015 at 03:41:13PM -0800, Julien Pourtet wrote:
> My "tweak" for the client app is using a one-to-one socket
> (SOCK_STREAM) instead of one-to-many. I also perform a
> shutdown(SHUT_RDWR) before close() is called, as well as a recvmsg()
> [get_reply() in that code base semantics] to receive the notifications
> before closing the file descriptor.
> 
> Commenting out the following lines in ulpqueue.c does get the
> notifications delivered to the client app:
>          /* If the socket is just going to throw this away, do not
>           * even try to deliver it.
>           */
>          if (sock_flag(sk, SOCK_DEAD) || (sk->sk_shutdown & RCV_SHUTDOWN))
>                  goto out_free;

That's pretty much the thing.. when you did shutdown(SHUT_RDWR), you set
that RCV_SHUTDOWN flag. You told the sockets layer that you don't want
to receive anything else from this fd.

Please change that SHUT_RDWR to just SHUT_WR and see how it goes.

For one-to-many, there is also this possibility:
https://tools.ietf.org/html/rfc6458#section-3.2
To shut down the association gracefully, the user must call sendmsg()
with no data and with the SCTP_EOF flag set as described in
Section 5.3.4.  The function returns immediately, and completion of
the graceful shutdown is indicated by an SCTP_ASSOC_CHANGE
notification of type SCTP_SHUTDOWN_COMP (see Section 6.1.1).  Note
that this can also be done using the sctp_sendv() call described in
Section 9.12."

in case you're intestered. (This cannot be used with
one-to-one/tcp-style)

> However, I guess that's not a viable solution as this may have
> undesirable side-effects (I am not familiar with this code base). It
> was more of a sanity check for me.

I think that with the above explanation you can see the effects it would
have.

> Question is, is this a bug? Should we fix it?

Nope :)

> [Note: it's my first time writing to this ML, sorry if I am breaking
> any implicit rules or explicit conventions].

It's all good, just sorry for the delay.

  Marcelo

Re: Not receiving SCTP_SHUTDOWN_COMP notifications

[Julien Pourtet]

Hello,

On Thu, Dec 3, 2015 at 4:18 AM, Marcelo Ricardo Leitner
<marcelo.leitner@gmail.com> wrote:
> That's pretty much the thing.. when you did shutdown(SHUT_RDWR), you set
> that RCV_SHUTDOWN flag. You told the sockets layer that you don't want
> to receive anything else from this fd.
>
> Please change that SHUT_RDWR to just SHUT_WR and see how it goes.

Duh! Alright, it works fine with SHUT_WR. I was so focused on the
transport layer that it didn't occur to me that SHUT_RDWR was what was
setting the RCV_SHUTDOWN flag on the socket. Sorry about that.

> For one-to-many, there is also this possibility:
> https://tools.ietf.org/html/rfc6458#section-3.2
> To shut down the association gracefully, the user must call sendmsg()
> with no data and with the SCTP_EOF flag set as described in
> Section 5.3.4.  The function returns immediately, and completion of
> the graceful shutdown is indicated by an SCTP_ASSOC_CHANGE
> notification of type SCTP_SHUTDOWN_COMP (see Section 6.1.1).  Note
> that this can also be done using the sctp_sendv() call described in
> Section 9.12."
>
> in case you're intestered. (This cannot be used with
> one-to-one/tcp-style)

Thanks for the clarification about one-to-many. Definitely helpful.

>> Question is, is this a bug? Should we fix it?
>
> Nope :)

Indeed ;) Thank you very much for your time.

Julien