sunrpc port allocation and IANA reserved list

sunrpc port allocation and IANA reserved list

[Chris Friesen]

By default sunrpc ports are allocated at random in the range 665-1023U.
 However, there are many ports within this range which have been
reserved by the IANA (and others like port 921 which are not formally
reserved but are "well-known").

Given that a userspace application can be stopped and restarted at any
time, and a sunrpc registration can happen at any time, what is the
expected mechanism to prevent the kernel from allocating a port for use
by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with
this, but is there a standard solution?  Should there be?

The current setup seems suboptimal.

Chris

Re: sunrpc port allocation and IANA reserved list

[Ben Hutchings]

On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
> By default sunrpc ports are allocated at random in the range 665-1023U.
>  However, there are many ports within this range which have been
> reserved by the IANA (and others like port 921 which are not formally
> reserved but are "well-known").
> 
> Given that a userspace application can be stopped and restarted at any
> time, and a sunrpc registration can happen at any time, what is the
> expected mechanism to prevent the kernel from allocating a port for use
> by sunrpc that reserved or well-known?
> 
> Apparently Redhat and Debian have distro-specific ways of dealing with
> this, but is there a standard solution?  Should there be?
> 
> The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: sunrpc port allocation and IANA reserved list

[Chris Friesen]

On 11/10/2009 11:53 AM, Ben Hutchings wrote:
> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:

>> Given that a userspace application can be stopped and restarted at any
>> time, and a sunrpc registration can happen at any time, what is the
>> expected mechanism to prevent the kernel from allocating a port for use
>> by sunrpc that reserved or well-known?
>>
>> Apparently Redhat and Debian have distro-specific ways of dealing with
>> this, but is there a standard solution?  Should there be?
>>
>> The current setup seems suboptimal.
> 
> I believe both RH and Debian are using the same implementation:
> <http://cyberelk.net/tim/software/portreserve/>.

That helps with the startup case, but still leaves a possible hole if an
app using a fixed port number is restarted at runtime.  During the
window where nobody is bound to the port, the kernel could randomly
assign it to someone else.

Chris

Re: sunrpc port allocation and IANA reserved list

[Chris Friesen]

On 11/10/2009 12:37 PM, Chris Friesen wrote:
> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
>> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
> 
>>> Given that a userspace application can be stopped and restarted at any
>>> time, and a sunrpc registration can happen at any time, what is the
>>> expected mechanism to prevent the kernel from allocating a port for use
>>> by sunrpc that reserved or well-known?
>>>
>>> Apparently Redhat and Debian have distro-specific ways of dealing with
>>> this, but is there a standard solution?  Should there be?
>>>
>>> The current setup seems suboptimal.
>>
>> I believe both RH and Debian are using the same implementation:
>> <http://cyberelk.net/tim/software/portreserve/>.
> 
> That helps with the startup case, but still leaves a possible hole if an
> app using a fixed port number is restarted at runtime.  During the
> window where nobody is bound to the port, the kernel could randomly
> assign it to someone else.

After some reflection it seems to me that the only way to close this
race condition is to store the list of reserved ports in the kernel and
simply avoid handing out a reserved address unless it is specifically
requested.

Maybe we could keep the config files of the existing portreserve
package, but rather than maintaining the reservation list itself the
portreserve app would simply feed the reservations into the kernel (via
/proc or netlink or something) at startup.

This would also avoid the need to modify the startup scripts of
applications wanting to use a fixed port.  The config file containing
the port number would still be necessary, however.

Chris

Re: sunrpc port allocation and IANA reserved list

[Trond Myklebust]

On Tue, 2009-11-10 at 12:37 -0600, Chris Friesen wrote:
> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
> > On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
> 
> >> Given that a userspace application can be stopped and restarted at any
> >> time, and a sunrpc registration can happen at any time, what is the
> >> expected mechanism to prevent the kernel from allocating a port for use
> >> by sunrpc that reserved or well-known?
> >>
> >> Apparently Redhat and Debian have distro-specific ways of dealing with
> >> this, but is there a standard solution?  Should there be?
> >>
> >> The current setup seems suboptimal.
> > 
> > I believe both RH and Debian are using the same implementation:
> > <http://cyberelk.net/tim/software/portreserve/>.
> 
> That helps with the startup case, but still leaves a possible hole if an
> app using a fixed port number is restarted at runtime.  During the
> window where nobody is bound to the port, the kernel could randomly
> assign it to someone else.

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
range used to a safer one. That's what it is for...

Trond

Re: sunrpc port allocation and IANA reserved list

[Chris Friesen]

On 11/10/2009 02:26 PM, Trond Myklebust wrote:
> On Tue, 2009-11-10 at 12:37 -0600, Chris Friesen wrote:
>> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
>>> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
>>
>>>> Given that a userspace application can be stopped and restarted at any
>>>> time, and a sunrpc registration can happen at any time, what is the
>>>> expected mechanism to prevent the kernel from allocating a port for use
>>>> by sunrpc that reserved or well-known?
>>>>
>>>> Apparently Redhat and Debian have distro-specific ways of dealing with
>>>> this, but is there a standard solution?  Should there be?
>>>>
>>>> The current setup seems suboptimal.
>>>
>>> I believe both RH and Debian are using the same implementation:
>>> <http://cyberelk.net/tim/software/portreserve/>.
>>
>> That helps with the startup case, but still leaves a possible hole if an
>> app using a fixed port number is restarted at runtime.  During the
>> window where nobody is bound to the port, the kernel could randomly
>> assign it to someone else.
> 
> Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
> range used to a safer one. That's what it is for...

What constitutes a "safer range"?  IANA has ports assigned
intermittently all the way through the default RPC range.  The largest
unassigned range is 922-988 (since 921 is used by lwresd).  If someone
needs more than 66 ports, how are they supposed to handle it?

Chris

Re: sunrpc port allocation and IANA reserved list

[Trond Myklebust]

On Tue, 2009-11-10 at 15:06 -0600, Chris Friesen wrote:
> On 11/10/2009 02:26 PM, Trond Myklebust wrote:
> > On Tue, 2009-11-10 at 12:37 -0600, Chris Friesen wrote:
> >> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
> >>> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
> >>
> >>>> Given that a userspace application can be stopped and restarted at any
> >>>> time, and a sunrpc registration can happen at any time, what is the
> >>>> expected mechanism to prevent the kernel from allocating a port for use
> >>>> by sunrpc that reserved or well-known?
> >>>>
> >>>> Apparently Redhat and Debian have distro-specific ways of dealing with
> >>>> this, but is there a standard solution?  Should there be?
> >>>>
> >>>> The current setup seems suboptimal.
> >>>
> >>> I believe both RH and Debian are using the same implementation:
> >>> <http://cyberelk.net/tim/software/portreserve/>.
> >>
> >> That helps with the startup case, but still leaves a possible hole if an
> >> app using a fixed port number is restarted at runtime.  During the
> >> window where nobody is bound to the port, the kernel could randomly
> >> assign it to someone else.
> > 
> > Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
> > range used to a safer one. That's what it is for...
> 
> What constitutes a "safer range"?  IANA has ports assigned
> intermittently all the way through the default RPC range.  The largest
> unassigned range is 922-988 (since 921 is used by lwresd).  If someone
> needs more than 66 ports, how are they supposed to handle it?

Distributions can, and should, set the default to whatever range that
constitutes a broad percentile of their customers, and then leave
instructions on how to change that default for the people who run the
funky setups.

The people who are trying to run absolutely all IANA registered services
on a single Linux machine that is also trying to run as an NFS client
may have a problem, but then again, how many setups do you know who are
trying to do that?

  Trond

Re: sunrpc port allocation and IANA reserved list

[Trond Myklebust]

On Wed, 2009-11-11 at 06:17 +0900, Trond Myklebust wrote:
> The people who are trying to run absolutely all IANA registered services
> on a single Linux machine that is also trying to run as an NFS client
> may have a problem, but then again, how many setups do you know who are
> trying to do that?

BTW: Even these setups can be fixed. One way to extend the available
port ranges is to allocate more than one IP address; either through the
use of multiple NICs, or through address aliasing. You can set up the
services to listen on one interface, while setting up your routing
tables to ensure that any outgoing connections to your NFS servers go
through through a different interface.

You could also use virtualised machines to host the services, etc...

Trond

Re: sunrpc port allocation and IANA reserved list

[Ben Hutchings]

On Tue, 2009-11-10 at 15:06 -0600, Chris Friesen wrote:
> On 11/10/2009 02:26 PM, Trond Myklebust wrote:
> > On Tue, 2009-11-10 at 12:37 -0600, Chris Friesen wrote:
> >> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
> >>> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
> >>
> >>>> Given that a userspace application can be stopped and restarted at any
> >>>> time, and a sunrpc registration can happen at any time, what is the
> >>>> expected mechanism to prevent the kernel from allocating a port for use
> >>>> by sunrpc that reserved or well-known?
> >>>>
> >>>> Apparently Redhat and Debian have distro-specific ways of dealing with
> >>>> this, but is there a standard solution?  Should there be?
> >>>>
> >>>> The current setup seems suboptimal.
> >>>
> >>> I believe both RH and Debian are using the same implementation:
> >>> <http://cyberelk.net/tim/software/portreserve/>.
> >>
> >> That helps with the startup case, but still leaves a possible hole if an
> >> app using a fixed port number is restarted at runtime.  During the
> >> window where nobody is bound to the port, the kernel could randomly
> >> assign it to someone else.
> > 
> > Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
> > range used to a safer one. That's what it is for...

Unless I'm much mistaken, that only affects in-kernel SunRPC users.

> What constitutes a "safer range"?  IANA has ports assigned
> intermittently all the way through the default RPC range.  The largest
> unassigned range is 922-988 (since 921 is used by lwresd).  If someone
> needs more than 66 ports, how are they supposed to handle it?

I'm sure we could afford 128 bytes for a blacklist of privileged ports.
However, the problem is that there is no API for userland to request
'any free privileged port' - it has to just try binding to different
ports until it finds one available.  This means that the kernel can't
tell whether a process is trying to allocate a specifically assigned
port or whether the blacklist should be applied.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: sunrpc port allocation and IANA reserved list

[Chuck Lever]

On Nov 10, 2009, at 4:32 PM, Ben Hutchings wrote:

> On Tue, 2009-11-10 at 15:06 -0600, Chris Friesen wrote:
>> On 11/10/2009 02:26 PM, Trond Myklebust wrote:
>>> On Tue, 2009-11-10 at 12:37 -0600, Chris Friesen wrote:
>>>> On 11/10/2009 11:53 AM, Ben Hutchings wrote:
>>>>> On Tue, 2009-11-10 at 11:43 -0600, Chris Friesen wrote:
>>>>
>>>>>> Given that a userspace application can be stopped and restarted  
>>>>>> at any
>>>>>> time, and a sunrpc registration can happen at any time, what is  
>>>>>> the
>>>>>> expected mechanism to prevent the kernel from allocating a port  
>>>>>> for use
>>>>>> by sunrpc that reserved or well-known?
>>>>>>
>>>>>> Apparently Redhat and Debian have distro-specific ways of  
>>>>>> dealing with
>>>>>> this, but is there a standard solution?  Should there be?
>>>>>>
>>>>>> The current setup seems suboptimal.
>>>>>
>>>>> I believe both RH and Debian are using the same implementation:
>>>>> <http://cyberelk.net/tim/software/portreserve/>.
>>>>
>>>> That helps with the startup case, but still leaves a possible  
>>>> hole if an
>>>> app using a fixed port number is restarted at runtime.  During the
>>>> window where nobody is bound to the port, the kernel could randomly
>>>> assign it to someone else.
>>>
>>> Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict  
>>> the
>>> range used to a safer one. That's what it is for...
>
> Unless I'm much mistaken, that only affects in-kernel SunRPC users.
>
>> What constitutes a "safer range"?  IANA has ports assigned
>> intermittently all the way through the default RPC range.  The  
>> largest
>> unassigned range is 922-988 (since 921 is used by lwresd).  If  
>> someone
>> needs more than 66 ports, how are they supposed to handle it?
>
> I'm sure we could afford 128 bytes for a blacklist of privileged  
> ports.
> However, the problem is that there is no API for userland to request
> 'any free privileged port' - it has to just try binding to different
> ports until it finds one available.

bindresvport(3) and bindresvport_sa(3t) ?

> This means that the kernel can't
> tell whether a process is trying to allocate a specifically assigned
> port or whether the blacklist should be applied.

Such a blacklist would have to be managed by glibc or libtirpc.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: sunrpc port allocation and IANA reserved list

[Ben Hutchings]

On Tue, 2009-11-10 at 16:34 -0500, Chuck Lever wrote:
> On Nov 10, 2009, at 4:32 PM, Ben Hutchings wrote:
> 
> > On Tue, 2009-11-10 at 15:06 -0600, Chris Friesen wrote:
> >> On 11/10/2009 02:26 PM, Trond Myklebust wrote:
[...]
> >>> Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict  
> >>> the
> >>> range used to a safer one. That's what it is for...
> >
> > Unless I'm much mistaken, that only affects in-kernel SunRPC users.
> >
> >> What constitutes a "safer range"?  IANA has ports assigned
> >> intermittently all the way through the default RPC range.  The  
> >> largest
> >> unassigned range is 922-988 (since 921 is used by lwresd).  If  
> >> someone
> >> needs more than 66 ports, how are they supposed to handle it?
> >
> > I'm sure we could afford 128 bytes for a blacklist of privileged  
> > ports.
> > However, the problem is that there is no API for userland to request
> > 'any free privileged port' - it has to just try binding to different
> > ports until it finds one available.
> 
> bindresvport(3) and bindresvport_sa(3t) ?

These are library calls; they are not an API between userland the
kernel.

> > This means that the kernel can't
> > tell whether a process is trying to allocate a specifically assigned
> > port or whether the blacklist should be applied.
> 
> Such a blacklist would have to be managed by glibc or libtirpc.

Right.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: sunrpc port allocation and IANA reserved list

[Chris Friesen]

On 11/10/2009 03:32 PM, Ben Hutchings wrote:

> I'm sure we could afford 128 bytes for a blacklist of privileged ports.
> However, the problem is that there is no API for userland to request
> 'any free privileged port' - it has to just try binding to different
> ports until it finds one available.  This means that the kernel can't
> tell whether a process is trying to allocate a specifically assigned
> port or whether the blacklist should be applied.

That's unfortunate.  Maybe a bindresvport syscall would make sense, but
that's starting to get bloated.  At least for userspace apps netstat
will show who the owner is and it might be possible to clean it up
without rebooting.

For the ports allocated in the kernel via xs_get_random_port() and
xprt_bindresvport() it would be fairly easy to check a blacklist.

Interestingly, the current kernel range starts at 665 because IPMI is
known to use 664.  The current glibc range to be tried in bindresvport()
starts at 600, which seems to be a bug waiting to happen.

Chris

Re: sunrpc port allocation and IANA reserved list

[Trond Myklebust]

On Tue, 2009-11-10 at 21:32 +0000, Ben Hutchings wrote:
> On Tue, 2009-11-10 at 15:06 -0600, Chris Friesen wrote:
> > On 11/10/2009 02:26 PM, Trond Myklebust wrote: 
> > > Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
> > > range used to a safer one. That's what it is for...
> 
> Unless I'm much mistaken, that only affects in-kernel SunRPC users.

There shouldn't be that many non-kernel sunrpc users that require
privileged ports.

The exceptions I can think of are

      * the 'mount' program, if you are using the legacy binary mount
        interface. Use text mounts instead...
      * rpc.statd might also use privileged ports when booting the
        system when it needs to notify servers of the reboot. You can
        set that port number using the '-o' option.

Others? Do any of the NIS services require it?

Trond