metafs

[Nagy Zoltan <kirk@bteam.hu>]

Hello all,

sorry for the large posting - and for my intermediate english.

the past few months i tried to figure out a better way to organize data,
because in extereme conditions automatic/dynamic (re)organiziation would
be very good to have - i recently found this paper, which i found very 
similar
to this: http://bryanmills.net:8086/uploads/metafs/bmills-final.pdf

i imagined a two sided approach to came up with an acceptable solution
 * object tagging:
    it is obvious that people can give very precise information to a given
    object, in this example i would take some papers
        [obj1]    -    filename:    'something1.pdf'
                    type = "paper"
                    filetype = "application/pdf"
                    title = {Title1},
                    author = {Author1},
                    year = {2005},
        [obj2]    -    filename:    'xasd1.XXXpdf'
                    type = "paper"
                    filetype = "pdf"
                    title = {Title2},
                    author = {Author2},
                    year = {2005}
        [obj3]    -    filename:    'qqq.pdf'
                    type = "paper"
                    filetype = "pdf"
                    title = {Title3},
                    author = {Author3},
                    /** year = {2005} no set **/
        [obj4]    -    filename:    'asd.png'
                    type = "image/png"
                    filetype = "png"
                    title = {Title2},
                    author = {Author2},
                    year = {2005}

    all this data is accessible thru extended attributes

 * layout rules
    the user must give some guidance - about in what way he want's to see
    those files in this example i will choose the following rules:
        x_node0    =    _root_
        x_node1    =    {parent:x_node0}
                    {entry:"paper",
                        require:{type = "paper"},
                            unassigned:{collect,"unassigned"}
                                                        }
        x_node2    =    {parent:x_node1}    {entry:"by_year"}
        x_node3    =    {parent:x_node2}    {entry:#year,
                        type:collection,
                            unassigned:{collect,"year-not-set"}
        x_node4    =    {parent:x_node3}    {entry:#author|'_'|#title,
                        unassigned:{cascade}    }
    
    i hope that this understandable

the desired layout would be for those files:
    
    /
    /paper
    /paper/by_year
    /paper/by_year/2005
    /paper/by_year/2005/Author1_Title1.pdf    -->    [obj1]
    /paper/by_year/2005/Author2_Title2.pdf    -->    [obj2]
    /paper/by_year/year-not-set/qqq.pdf    -->    [obj3]
    /unassigned/asd.png                    -->    [obj4]

and if someone adds some new rules
        x_node5    =    {parent:x_node2}    {entry:"by_author",
                        type:collection,
                            unassigned:{collect,"unknown-author"}
        x_node6    =    {parent:x_node5}    {entry:#title|'_'|#year    }
then the following nodes should appear
    /paper/by_author
    /paper/by_author/Author1/Title1_2003    --> [obj1]
    [...]

the biggest power of this, that it is possible for an object to show up
at many places dynamically.
i think that this can be implemented as an 'overlay' (userland)filesystem,
with some lightweight database help

i know that algorithm for this kind of dynamic layout comes at a
very high price - it is/and will be a problem, it will be a big challange
to minimize it's impact on the overall performance - but for now, i think
about a relativly 'slow' changing object set.

what's the biggest problem with this?
how could it be
    - more adaptive to user's needs
    - faster at runtime
    - i welcome tips for implementing it - the right way
    - ..or why should i forget it :)

cheers,
kirk