Re: Idea for reducing sysfs memory usage

[Greg Kroah-Hartman <gregkh@linuxfoundation.org>]

On Tue, Feb 16, 2016 at 05:44:54PM -0700, David Ahern wrote:
> On 2/16/16 5:37 PM, Edward Cree wrote:
> >On 16/02/16 23:55, Greg Kroah-Hartman wrote:
> >>On Tue, Feb 16, 2016 at 11:46:49PM +0000, Edward Cree wrote:
> >>>Sorry if this has been suggested before, but if so I couldn't find it.
> >>>Short version: could a sysfs dir reference a list of default attributes
> >>>rather than having to instantiate them all?
> >>Shorter version, why do you think it is?  :)
> >>
> >>Have you done some testing of the amount of memory that sysfs entries
> >>consume and found any problems with it?
> >Two reasons:
> >a) in his netdev1.1 talk "Scaling the Number of Network Interfaces on
> >Linux",
> >    David Ahern claimed a memory overhead of (iirc) about 45kB per
> >netdevice,
> >    of which he attributed (again, iirc) about 20kB to sysfs entries. He
> >also
> >    indicated that this was a problem for his use case.  (My apologies to
> >    David if I've misrepresented him.  CCed him so he can correct me.)
> 
> Close enough. :-)
> 
> I analyzed memory allocations for the creation of a dummy netdevice. All
> total 36,450 bytes were requested resulting in 43,944 bytes allocated. Of
> that kobject and sysfs is 14,568. So kobject and sysfs is roughly 1/3 the
> memory overhead of a netdevice and is also a significant overhead in the
> time to create the netdevice.

Time?  Hm, I thought we reduced the time down by a lot a few years ago,
any pointers to where we are spending too much time?  The PPC people did
some work on our list traversal issues for thousands of devices, perhaps
that doesn't scale to 10s of thousands?

As for 1/3 the memory needed, is that acceptable given all that the
interface and functionality that the code gives you?  14k does seem
pretty large to me, how many attributes/files are assigned to each
device?

thanks,

greg k-h