making an in-memory hashing table ["name" -> ino_t] with thousands of entries

making an in-memory hashing table ["name" -> ino_t] with thousands of entries

[Luke Kenneth Casson Leighton]

dear kernel hackers,

i seek advice on how to do an in-kernel lookup table capable of storing
thousands, potentially hundreds of thousands, of entries.

the reason for this is to move the fuse userspace inode lookup
tables into the kernel.

fuse userspace servers have their own in-memory database of
unique inode numbers which represent the file names, and there is a
communication mechanism between userspace and kernelspace that transfers
those inode numbers, amongst other things.

is there any _sane_ way to do this or should i leave the inode lookup
table where it presently is - in userspace?

bearing in mind that for every file accessed via a fuse
filesystem, a cache entry is created, and therefore the number
of entries could potentially run into hundreds of thousands
of entries.

l.

-- 
--
Truth, honesty and respect are rare commodities that all spring from
the same well: Love.  If you love yourself and everyone and everything
around you, funnily and coincidentally enough, life gets a lot better.
--
<a href="http://lkcl.net">      lkcl.net      </a> <br />
<a href="mailto:lkcl@lkcl.net"> lkcl@lkcl.net </a> <br />

Re: making an in-memory hashing table ["name" -> ino_t] with thousands of entries

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 513 bytes --]

hi :)

On Fri, Oct 01, 2004 at 01:02:22PM +0100, Luke Kenneth Casson Leighton wrote:
> bearing in mind that for every file accessed via a fuse
> filesystem, a cache entry is created, and therefore the number
> of entries could potentially run into hundreds of thousands
> of entries.

but these can be cleaned if needed.

why do you have to move all inodes into the kernel when the kernel
already cache those that are really needed?
(perhaps I'm misunderstandig fuses role here)

-- 
Martin Waitz

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: making an in-memory hashing table ["name" -> ino_t] with thousands of entries

[Luke Kenneth Casson Leighton]

On Fri, Oct 01, 2004 at 02:30:19PM +0200, Martin Waitz wrote:
> hi :)

 hello martin: thanks for responding.

> On Fri, Oct 01, 2004 at 01:02:22PM +0100, Luke Kenneth Casson Leighton wrote:
> > bearing in mind that for every file accessed via a fuse
> > filesystem, a cache entry is created, and therefore the number
> > of entries could potentially run into hundreds of thousands
> > of entries.
> 
> but these can be cleaned if needed.
> 
> why do you have to move all inodes into the kernel when the kernel
> already cache those that are really needed?
> (perhaps I'm misunderstandig fuses role here)
 
 fuse is a userspace filesystem.

 it can be absolutely anything.

 however, the author of fuse, in order to simplify the userspace
 interface, manages the userspace-inode <-> userspace-fullpathnames
 in a library.

 for the sake of argument, let's call this the fuse-inode-cache.

 entries in the fuse-inode-cache must be persistent for as long as a
 filename exists on the userspace file system.

 also, if an entry happens not to exist at the time a lookup
 is done, then an entry into the cache is created, associating that full
 path name with a newly created unique inode number (in the userspace
 fuse-inode-cache).

 then, that information (the unique inode number) is communicated _back_
 to the fuse module, along with the stat information (yes, a userspace
 lstat is done as well), and the kernel's inode structure is updated
 from that information.


 anyway: for various reasons, inside the kernel, it must
 be possible to do a lookup from the full path name to the
 inode number.


 smbfs suffers from exactly the same problem: inodes don't exist on the
 SMB protocol.

 hm, i wonder if i should just rip stacks of code from the smbfs kernel
 module?

 i note with interest the existence of a function smb_refresh_inode()
 which gets passed a dentry.

 then i can do a getattr and go from there.

 oh, man, this is hair-raising stuff.

 l.

 

-- 
--
Truth, honesty and respect are rare commodities that all spring from
the same well: Love.  If you love yourself and everyone and everything
around you, funnily and coincidentally enough, life gets a lot better.
--
<a href="http://lkcl.net">      lkcl.net      </a> <br />
<a href="mailto:lkcl@lkcl.net"> lkcl@lkcl.net </a> <br />