Re: writing a ceph cliente for MS windows

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Commercial libraries are a pain ...

If we want the more permossive licence offered by callback file system 
we have to buy it for 20.000 usd. Then we will have to provide a backbox 
that we have no control upon and that will kill our product anytime they 
want anf if they decide to stop their commercial activity we will be in 
the same situation that with dokanfs but without having the source code 
of the black box. If i have to spend 20 000 dollars i would prefere 
paying someone to retake dokanfs or to write from scratch a dokanfs 
fuselike software make it all shiny and pumpy fantastic and ready to 
plug to ceph client.

I would prefere if people have to pay something to get access to 
ceph4win that this money goes in ceph main branch pockets... Or as a 
gift you donante to ceph 10 dollars  you get 2 free registration codes 
for ceph4win... or something like that.

If ceph4win as to be comercial then I would prefer delegate the task to 
a company like south river technologies and their great product 
webdrive. I would mininaly get involved in that project and simply buy 
the final product to sell it together with my ceph based product (which 
could be a calxeda ceph box or something like that).

I m open anyway to any proposition. But I doubt that callback filesystem 
offers us a suitable solution in the way I see ceph4win to be spread and 
used... I m maybe wrong. And anything that will be done around ceph4win 
will be public documented etc... And licensed the way that if someone 
want to build a commercial solution on top of it, that would be a 
possibility.

My idea is to giveback somehow to ceph project and at same time forge a 
better knowledge in ceph technologies. Because like many in libre world 
I think the business is in the services around the software more than on 
the software. That the ones writing code should be financed and benefits 
from the one selling and giving support of the software at all levels. I 
m probably too idealistic. And too optimistic after all I m the one 
saying I will do this stuff I have no idea how but well it is 
interesting and fun so lets do it.

Regards,

P.S: using commercial backend libraries appart including their own cost 
will force you to use commercial IDE like MS VisualStudio because their 
library has some kind of drm that only that IDE compiler can use. So 
alot of cost and yet there is nothing done. If I had to open a 
kickstarter project saying we need 60 000 USD to do ceph4win with that 
monney we will buy the right to use and share a commercial copyrighted 
library but abandonned punctually to us in  public domaine and that we 
will eventually produce something out of it. I doubt I will get a dollar.

We still can suggest the idea to Edlos the commercial company that has 
the copyright of Callback FS, Or to buy them their product in a blender 
way (blender was bought with donation before being put opensource and 
public domaine), Or to open source their library. But in commercial 
minds opensourcing = death of their technical advantage and death of 
their marketing strategy. They will have to invent something more to 
retrieve monney from it.

El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com 
<mailto:d.ketor@gmail.com>> escribió:

    Hi Alphe,
              I think you could try Callback Filesystem dev framework. It
    is a commerical dev framework and is maintained by Edlos today.
              I have communicated with Edlos to get a try code for
    development. To dokan, Callback Filesystem has vary document and maybe
    more stabilize.

              Regards.



    On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
    <mailto:asalas@kepler.cl>> wrote:
     > Hello ketor thank you for your interest un ceph4win. Since muy
    first mail I
     > exposed the lacks of dokanfs and that I m far from being a
    specialist un
     > filesystems.
     > I exposed what i like un dokanfs bit I not a fanátic of it. Muy
    goal is to
     > have something working quickly.
     >
     > So I am up to any proposición sure the one with the more docs and
    support
     > will be the best choice. As for right now what I need is
    understand what are
     > the files involved what are the interfaces functions and what are
    the needed
     > library dependencies and if they exist ported to windows with
    cygwin. And
     > all that is retrieved from source code.
     >
     > Regards.
     >
     > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
    <mailto:d.ketor@gmail.com>> escribió:
     >
     >> Hi Alphe,
     >>      We are taking an interest in your work on Ceph Client for
    Windows
     >> with Dokan.As we know, the performance of Dokan is not very
    good, and it's
     >> abandoned 3 years ago.
     >>      I have learned and used OpenDedup(SDFS) for a long time.
      OpenDedup
     >> has a Dokan version. And the author of OpenDedup said
     >>
     >> The Dokan library is quite flakey  and testing should be
    performed before
     >> putting into production
     >>
     >>       So what do you think about this? And if there is another
    solution of
     >> fuse-like filesystem dev framwork on Windows?
     >>
     >>        Best Wish!
     >>
     >>
     >>
     >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
    <asalas@kepler.cl <mailto:asalas@kepler.cl>>
     >> wrote:
     >>>
     >>> Hello I created the github repository for this project
     >>> https://github.com/alphe/Ceph4Win
     >>>
     >>> Regards,
     >>>
     >>> signature
     >>>
     >>> *Alphé Salas*
     >>> Ingeniero T.I
     >>>
     >>> asalas@kepler.cl <mailto:asalas@kepler.cl>
     >>> *<http://www.kepler.cl>*
     >>>
     >>> On 11/05/13 21:00, Sage Weil wrote:
     >>>>
     >>>> Hi Alphe,
     >>>>
     >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
     >>>>>
     >>>>> signature *Hi, Sage !
     >>>>> thank you for you enthousiast reply.
     >>>>> I sure want to make the best use of everything or anything
    previously
     >>>>> done to
     >>>>> tend to
     >>>>> write ceph cliente for windows.
     >>>>>
     >>>>> Apart using libre tools for building the future ceph cliente
    I am open
     >>>>> to
     >>>>> anything.
     >>>>> I would recommand eclipse CDT or Code::BLocks they are based
    on mingwin
     >>>>> open
     >>>>> and easyly enhanceable.**
     >>>>>
     >>>>> more free tools can be found here:
     >>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
     >>>>>
     >>>>>
     >>>>> I will read libcephfs source code and take some notes about the
     >>>>> protocol.
     >>>>
     >>>> I think you don't need to worry about hte protocol at all, since
     >>>> libcephs
     >>>> implements it for you (and will capture any future changes).
     >>>>
     >>>>> I was more going from what I know and trying to track down how
     >>>>> mount.ceph work
     >>>>> with the parameters passed to it.
     >>>>> since it point finally to Kernel/fs/ceph and that I don t really
     >>>>> understand
     >>>>> how that module work and that it probably points to some other
     >>>>> dependencies
     >>>>> Reading libcephfs source code could be a big gain of time.
     >>>>
     >>>> (I would also ignore mount.ceph as everything it does it
    specific to
     >>>> how Linux mounts work.)
     >>>>
     >>>>> basically on the protocol what is  need are:
     >>>>>
     >>>>> 1) open and maintain a connection (socket open, auth, etc )
     >>>>> 2) retreive a map of directories and disk Quota (disk sizing
    Y TB free,
     >>>>> Z TB
     >>>>> total)
     >>>>> 3) procedure to send files / directories in a maner that it
    will allow
     >>>>> our
     >>>>> client to fit ceph transmission protocols
     >>>>> (limit bandwith for stability?, limit connection amount?,
    limit cpu
     >>>>> use?,
     >>>>> Cache for preparing data transfer (a FIFO cache)?)
     >>>>> 4)Procedure to retreive files / directory from ceph cluster
     >>>>> 5) Management copy/move files /Directories, FS stats,
    Connection Stats.
     >>>>> logging.
     >>>>>
     >>>>> My idea to progress is to take those main bulletpoint in ceph
    protocol
     >>>>> based
     >>>>> on general ideas of what ceph file system does and start
    identifying
     >>>>> parts
     >>>>> from libcephfs to match those "needs".
     >>>>
     >>>> Instead, I would look at include/cephfs/libcephfs.h, the
    interface that
     >>>> libcephfs provides, and try to map that to what the fuse layer
    expects.
     >>>> There is both a path-based that I suspsect lends itself well
    to the
     >>>> Windows interface and (very soon now) a handle based API that is
     >>>> targetted
     >>>> at the Unix-style VFS layers.  I'm mostly guessing, though,
    since I've
     >>>> never seen any low-level fs code in windows before.
     >>>>
     >>>> In this case, the analogous code for Linux should be
    client/fuse_ll.cc
     >>>> itself (and not much else), although there will probably be a
    few tricks
     >>>> necessary to map cleanly onto how the windows interfaces work.
     >>>>
     >>>> Does that make sense?
     >>>>
     >>>> Cheers!
     >>>> sage
     >>>>
     >>>>
     >>>>> Any suggestion and contributions are welcome.
     >>>>>
     >>>>>
     >>>>> *
     >>>>> On 11/05/13 11:23, Sage Weil wrote:
     >>>>>>
     >>>>>> Hi Alphe,
     >>>>>>
     >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
     >>>>>>>
     >>>>>>> Good day developers!
     >>>>>>>
     >>>>>>> I would like to propose to the one interested  work with me to
     >>>>>>> develop a
     >>>>>>> ceph
     >>>>>>> cliente for MS windows world, Basing us on dokanFS.
     >>>>>>>
     >>>>>>> My company is a ceph enthousiast that use on a dayly basis
    ceph and
     >>>>>>> that
     >>>>>>> need
     >>>>>>> both transfer speed and big expendable and cheap storage.
     >>>>>>> My company is specialised in data recovery and we want to
    participate
     >>>>>>> to
     >>>>>>> ceph
     >>>>>>> effort by bringing a ceph cliente for windows.
     >>>>>>
     >>>>>> Awesome!
     >>>>>>
     >>>>>>> Our experience shows us that the best gateway is each
    clientes being
     >>>>>>> its
     >>>>>>> own
     >>>>>>> gateway, instead of having a bottle neck server or a cluster of
     >>>>>>> bottle
     >>>>>>> neck
     >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
     >>>>>>>
     >>>>>>> We already did some research in that domain.
     >>>>>>>
     >>>>>>> Dokan FS is an intent  to write an opensource fuse like
    cliente for
     >>>>>>> MS
     >>>>>>> windows.
     >>>>>>>
     >>>>>>> More information on DOKANFS can be triggered here
     >>>>>>> http://dokan-dev.net/en/download/
     >>>>>>>
     >>>>>>> Positive points of using DOKANFS.
     >>>>>>>
     >>>>>>> - its opensourced and well licenced mit licence, gpl
    licence and lgpl
     >>>>>>> licence.
     >>>>>>>
     >>>>>>> Negative point of using DOKAN FS.
     >>>>>>> - unreachable author
     >>>>>>> - Poor documentation . Dev comments in japanese.
     >>>>>>> - Work in progress so it is unstable and needs to be updated,
     >>>>>>> debugged and
     >>>>>>> maintained by a MS Windows file system expert developper.
     >>>>>>
     >>>>>> I am not very familiar with windows storage APIs, but
    somebody told me
     >>>>>> at once point there were several interfaces against which a
    new file
     >>>>>> system could be implemented, everything from a full
    in-kernel driver
     >>>>>> to
     >>>>>> something that is explorer-based.  Are any of those
    suitable?  Using a
     >>>>>> potentially abandoned fuse-like layer makes me a bit nervous.
     >>>>>>
     >>>>>> That said,
     >>>>>>
     >>>>>>>
     >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
     >>>>>>> here are what I learnt.
     >>>>>>> - Ceph-fuse and related source code is around 60 000 lines
    of code.
     >>>>>>> - Ceph protocol isn t documented so it is like trying to
    draw a map
     >>>>>>> of
     >>>>>>> america
     >>>>>>> using only a sextan and a compass.
     >>>>>>>
     >>>>>>> Those led me to those conclusions:
     >>>>>>> - I can t do it alone.
     >>>>>>> - It is easier to draw down the ceph protocol way to work from
     >>>>>>> kernel/fs/ceph
     >>>>>>> sources and mount.ceph
     >>>>>>> - Ceph depending libraries may be unexistant or not up to
    date in
     >>>>>>> their
     >>>>>>> port
     >>>>>>> on MS Windows (cygwin)
     >>>>>>
     >>>>>> I think the most sane path should be to make libcephfs
    sufficiently
     >>>>>> portable to build on windows (or cygwin).  For the bits used
    by the
     >>>>>> client-side coe, I don't think there should be much in the
    way of
     >>>>>> dependencies, and the main challenge would be untangling the
    build for
     >>>>>> the necessary pieces out from the rest of Ceph.
     >>>>>>
     >>>>>> Have you seen the wip-port portability work that is
    currently underway
     >>>>>> by
     >>>>>> Noah and Alan?  That may solve many of the cygwin problems
    you are
     >>>>>> seeing
     >>>>>> today.
     >>>>>>
     >>>>>>> - MS file system specialist are hard do find in the "open
    source
     >>>>>>> libre
     >>>>>>> world"
     >>>>>>> so I will try in the commercial world.
     >>>>>>>
     >>>>>>> The commercial world has some problems too. They need ceph
    protocol
     >>>>>>> draft
     >>>>>>> to
     >>>>>>> implemente it to their own product They will have licencing
     >>>>>>> /commercial
     >>>>>>> politics that infringe lgpl, and hide that most of the work
    is done
     >>>>>>> by
     >>>>>>> people
     >>>>>>> other than them. They will not participate in a financial
    way to ceph
     >>>>>>> enhancement and growth.
     >>>>>>
     >>>>>> I don't think reimplementing the client code is an efficient way
     >>>>>> forward.
     >>>>>> Unless the goal is a pure kernel implementation...but a
    significant
     >>>>>> ongoing investment in development resources would be needed
    for that
     >>>>>> going
     >>>>>> forward.  I suspect that is a challenge for a platform that
    does not
     >>>>>> typically rally that sort of community effort.
     >>>>>>
     >>>>>> The easiest thing is of course just to use CIFS and Samba
    (which works
     >>>>>> today).  A fuse-like approach is probably a reasonably
    middle ground
     >>>>>> (both
     >>>>>> in initial effort and maintainability going forward)...
     >>>>>>
     >>>>>> sage
     >>>>>>
     >>>>>>
     >>>>>
     >>>
     >>> --
     >>> To unsubscribe from this list: send the line "unsubscribe
    ceph-devel" in
     >>> the body of a message to majordomo@vger.kernel.org
    <mailto:majordomo@vger.kernel.org>
     >>> More majordomo info at http://vger.kernel.org/majordomo-info.html
     >>
     >>
     >



--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Ketor D]

Hi Alphe:
      Yes Callback Filesystem is very expensive and can't open source.
It's not a good choice for ceph4win.
      Another way for ceph4win maybe develop a kernel-mode fs like
pnfs. pnfs has a kernel-mode windows client. I think you can read its
src code and maybe migrating from ceph kernel client to windows kernel
fs is easier than from userspace ceph fuse client.And a kernel-mode fs
client has greater performance than userspace fs like ceph-fuse client
and ceph kernel client.

      Regards.



On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels <asalas@kepler.cl> wrote:
>
> Commercial libraries are a pain ...
>
> If we want the more permossive licence offered by callback file system we
> have to buy it for 20.000 usd. Then we will have to provide a backbox that
> we have no control upon and that will kill our product anytime they want anf
> if they decide to stop their commercial activity we will be in the same
> situation that with dokanfs but without having the source code of the black
> box. If i have to spend 20 000 dollars i would prefere paying someone to
> retake dokanfs or to write from scratch a dokanfs fuselike software make it
> all shiny and pumpy fantastic and ready to plug to ceph client.
>
> I would prefere if people have to pay something to get access to ceph4win
> that this money goes in ceph main branch pockets... Or as a gift you donante
> to ceph 10 dollars  you get 2 free registration codes for ceph4win... or
> something like that.
>
> If ceph4win as to be comercial then I would prefer delegate the task to a
> company like south river technologies and their great product webdrive. I
> would mininaly get involved in that project and simply buy the final product
> to sell it together with my ceph based product (which could be a calxeda
> ceph box or something like that).
>
> I m open anyway to any proposition. But I doubt that callback filesystem
> offers us a suitable solution in the way I see ceph4win to be spread and
> used... I m maybe wrong. And anything that will be done around ceph4win will
> be public documented etc... And licensed the way that if someone want to
> build a commercial solution on top of it, that would be a possibility.
>
> My idea is to giveback somehow to ceph project and at same time forge a
> better knowledge in ceph technologies. Because like many in libre world I
> think the business is in the services around the software more than on the
> software. That the ones writing code should be financed and benefits from
> the one selling and giving support of the software at all levels. I m
> probably too idealistic. And too optimistic after all I m the one saying I
> will do this stuff I have no idea how but well it is interesting and fun so
> lets do it.
>
> Regards,
>
> P.S: using commercial backend libraries appart including their own cost will
> force you to use commercial IDE like MS VisualStudio because their library
> has some kind of drm that only that IDE compiler can use. So alot of cost
> and yet there is nothing done. If I had to open a kickstarter project saying
> we need 60 000 USD to do ceph4win with that monney we will buy the right to
> use and share a commercial copyrighted library but abandonned punctually to
> us in  public domaine and that we will eventually produce something out of
> it. I doubt I will get a dollar.
>
> We still can suggest the idea to Edlos the commercial company that has the
> copyright of Callback FS, Or to buy them their product in a blender way
> (blender was bought with donation before being put opensource and public
> domaine), Or to open source their library. But in commercial minds
> opensourcing = death of their technical advantage and death of their
> marketing strategy. They will have to invent something more to retrieve
> monney from it.
>
> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> <mailto:d.ketor@gmail.com>> escribió:
>
>
>    Hi Alphe,
>              I think you could try Callback Filesystem dev framework. It
>    is a commerical dev framework and is maintained by Edlos today.
>              I have communicated with Edlos to get a try code for
>    development. To dokan, Callback Filesystem has vary document and maybe
>    more stabilize.
>
>              Regards.
>
>
>
>    On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>    <mailto:asalas@kepler.cl>> wrote:
>     > Hello ketor thank you for your interest un ceph4win. Since muy
>    first mail I
>     > exposed the lacks of dokanfs and that I m far from being a
>    specialist un
>     > filesystems.
>     > I exposed what i like un dokanfs bit I not a fanátic of it. Muy
>    goal is to
>     > have something working quickly.
>     >
>     > So I am up to any proposición sure the one with the more docs and
>    support
>     > will be the best choice. As for right now what I need is
>    understand what are
>     > the files involved what are the interfaces functions and what are
>    the needed
>     > library dependencies and if they exist ported to windows with
>    cygwin. And
>     > all that is retrieved from source code.
>     >
>     > Regards.
>     >
>     > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>    <mailto:d.ketor@gmail.com>> escribió:
>
>     >
>     >> Hi Alphe,
>     >>      We are taking an interest in your work on Ceph Client for
>    Windows
>     >> with Dokan.As we know, the performance of Dokan is not very
>    good, and it's
>     >> abandoned 3 years ago.
>     >>      I have learned and used OpenDedup(SDFS) for a long time.
>      OpenDedup
>     >> has a Dokan version. And the author of OpenDedup said
>     >>
>     >> The Dokan library is quite flakey  and testing should be
>    performed before
>     >> putting into production
>     >>
>     >>       So what do you think about this? And if there is another
>    solution of
>     >> fuse-like filesystem dev framwork on Windows?
>     >>
>     >>        Best Wish!
>     >>
>     >>
>     >>
>     >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>    <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
>     >> wrote:
>     >>>
>     >>> Hello I created the github repository for this project
>     >>> https://github.com/alphe/Ceph4Win
>     >>>
>     >>> Regards,
>     >>>
>     >>> signature
>     >>>
>     >>> *Alphé Salas*
>     >>> Ingeniero T.I
>     >>>
>     >>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>
>     >>> *<http://www.kepler.cl>*
>     >>>
>     >>> On 11/05/13 21:00, Sage Weil wrote:
>     >>>>
>     >>>> Hi Alphe,
>     >>>>
>     >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>     >>>>>
>     >>>>> signature *Hi, Sage !
>     >>>>> thank you for you enthousiast reply.
>     >>>>> I sure want to make the best use of everything or anything
>    previously
>     >>>>> done to
>     >>>>> tend to
>     >>>>> write ceph cliente for windows.
>     >>>>>
>     >>>>> Apart using libre tools for building the future ceph cliente
>    I am open
>     >>>>> to
>     >>>>> anything.
>     >>>>> I would recommand eclipse CDT or Code::BLocks they are based
>    on mingwin
>     >>>>> open
>     >>>>> and easyly enhanceable.**
>     >>>>>
>     >>>>> more free tools can be found here:
>     >>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
>     >>>>>
>     >>>>>
>     >>>>> I will read libcephfs source code and take some notes about the
>     >>>>> protocol.
>     >>>>
>     >>>> I think you don't need to worry about hte protocol at all, since
>     >>>> libcephs
>     >>>> implements it for you (and will capture any future changes).
>     >>>>
>     >>>>> I was more going from what I know and trying to track down how
>     >>>>> mount.ceph work
>     >>>>> with the parameters passed to it.
>     >>>>> since it point finally to Kernel/fs/ceph and that I don t really
>     >>>>> understand
>     >>>>> how that module work and that it probably points to some other
>     >>>>> dependencies
>     >>>>> Reading libcephfs source code could be a big gain of time.
>     >>>>
>     >>>> (I would also ignore mount.ceph as everything it does it
>    specific to
>     >>>> how Linux mounts work.)
>     >>>>
>     >>>>> basically on the protocol what is  need are:
>     >>>>>
>     >>>>> 1) open and maintain a connection (socket open, auth, etc )
>     >>>>> 2) retreive a map of directories and disk Quota (disk sizing
>    Y TB free,
>     >>>>> Z TB
>     >>>>> total)
>     >>>>> 3) procedure to send files / directories in a maner that it
>    will allow
>     >>>>> our
>     >>>>> client to fit ceph transmission protocols
>     >>>>> (limit bandwith for stability?, limit connection amount?,
>    limit cpu
>     >>>>> use?,
>     >>>>> Cache for preparing data transfer (a FIFO cache)?)
>     >>>>> 4)Procedure to retreive files / directory from ceph cluster
>     >>>>> 5) Management copy/move files /Directories, FS stats,
>    Connection Stats.
>     >>>>> logging.
>     >>>>>
>     >>>>> My idea to progress is to take those main bulletpoint in ceph
>    protocol
>     >>>>> based
>     >>>>> on general ideas of what ceph file system does and start
>    identifying
>     >>>>> parts
>     >>>>> from libcephfs to match those "needs".
>     >>>>
>     >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>    interface that
>     >>>> libcephfs provides, and try to map that to what the fuse layer
>    expects.
>     >>>> There is both a path-based that I suspsect lends itself well
>    to the
>     >>>> Windows interface and (very soon now) a handle based API that is
>     >>>> targetted
>     >>>> at the Unix-style VFS layers.  I'm mostly guessing, though,
>    since I've
>     >>>> never seen any low-level fs code in windows before.
>     >>>>
>     >>>> In this case, the analogous code for Linux should be
>    client/fuse_ll.cc
>     >>>> itself (and not much else), although there will probably be a
>    few tricks
>     >>>> necessary to map cleanly onto how the windows interfaces work.
>     >>>>
>     >>>> Does that make sense?
>     >>>>
>     >>>> Cheers!
>     >>>> sage
>     >>>>
>     >>>>
>     >>>>> Any suggestion and contributions are welcome.
>     >>>>>
>     >>>>>
>     >>>>> *
>     >>>>> On 11/05/13 11:23, Sage Weil wrote:
>     >>>>>>
>     >>>>>> Hi Alphe,
>     >>>>>>
>     >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>     >>>>>>>
>     >>>>>>> Good day developers!
>     >>>>>>>
>     >>>>>>> I would like to propose to the one interested  work with me to
>     >>>>>>> develop a
>     >>>>>>> ceph
>     >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>     >>>>>>>
>     >>>>>>> My company is a ceph enthousiast that use on a dayly basis
>    ceph and
>     >>>>>>> that
>     >>>>>>> need
>     >>>>>>> both transfer speed and big expendable and cheap storage.
>     >>>>>>> My company is specialised in data recovery and we want to
>    participate
>     >>>>>>> to
>     >>>>>>> ceph
>     >>>>>>> effort by bringing a ceph cliente for windows.
>     >>>>>>
>     >>>>>> Awesome!
>     >>>>>>
>     >>>>>>> Our experience shows us that the best gateway is each
>    clientes being
>     >>>>>>> its
>     >>>>>>> own
>     >>>>>>> gateway, instead of having a bottle neck server or a cluster of
>     >>>>>>> bottle
>     >>>>>>> neck
>     >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>     >>>>>>>
>     >>>>>>> We already did some research in that domain.
>     >>>>>>>
>     >>>>>>> Dokan FS is an intent  to write an opensource fuse like
>    cliente for
>     >>>>>>> MS
>     >>>>>>> windows.
>     >>>>>>>
>     >>>>>>> More information on DOKANFS can be triggered here
>     >>>>>>> http://dokan-dev.net/en/download/
>     >>>>>>>
>     >>>>>>> Positive points of using DOKANFS.
>     >>>>>>>
>     >>>>>>> - its opensourced and well licenced mit licence, gpl
>    licence and lgpl
>     >>>>>>> licence.
>     >>>>>>>
>     >>>>>>> Negative point of using DOKAN FS.
>     >>>>>>> - unreachable author
>     >>>>>>> - Poor documentation . Dev comments in japanese.
>     >>>>>>> - Work in progress so it is unstable and needs to be updated,
>     >>>>>>> debugged and
>     >>>>>>> maintained by a MS Windows file system expert developper.
>     >>>>>>
>     >>>>>> I am not very familiar with windows storage APIs, but
>    somebody told me
>     >>>>>> at once point there were several interfaces against which a
>    new file
>     >>>>>> system could be implemented, everything from a full
>    in-kernel driver
>     >>>>>> to
>     >>>>>> something that is explorer-based.  Are any of those
>    suitable?  Using a
>     >>>>>> potentially abandoned fuse-like layer makes me a bit nervous.
>     >>>>>>
>     >>>>>> That said,
>     >>>>>>
>     >>>>>>>
>     >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>     >>>>>>> here are what I learnt.
>     >>>>>>> - Ceph-fuse and related source code is around 60 000 lines
>    of code.
>     >>>>>>> - Ceph protocol isn t documented so it is like trying to
>    draw a map
>     >>>>>>> of
>     >>>>>>> america
>     >>>>>>> using only a sextan and a compass.
>     >>>>>>>
>     >>>>>>> Those led me to those conclusions:
>     >>>>>>> - I can t do it alone.
>     >>>>>>> - It is easier to draw down the ceph protocol way to work from
>     >>>>>>> kernel/fs/ceph
>     >>>>>>> sources and mount.ceph
>     >>>>>>> - Ceph depending libraries may be unexistant or not up to
>    date in
>     >>>>>>> their
>     >>>>>>> port
>     >>>>>>> on MS Windows (cygwin)
>     >>>>>>
>     >>>>>> I think the most sane path should be to make libcephfs
>    sufficiently
>     >>>>>> portable to build on windows (or cygwin).  For the bits used
>    by the
>     >>>>>> client-side coe, I don't think there should be much in the
>    way of
>     >>>>>> dependencies, and the main challenge would be untangling the
>    build for
>     >>>>>> the necessary pieces out from the rest of Ceph.
>     >>>>>>
>     >>>>>> Have you seen the wip-port portability work that is
>    currently underway
>     >>>>>> by
>     >>>>>> Noah and Alan?  That may solve many of the cygwin problems
>    you are
>     >>>>>> seeing
>     >>>>>> today.
>     >>>>>>
>     >>>>>>> - MS file system specialist are hard do find in the "open
>    source
>     >>>>>>> libre
>     >>>>>>> world"
>     >>>>>>> so I will try in the commercial world.
>     >>>>>>>
>     >>>>>>> The commercial world has some problems too. They need ceph
>    protocol
>     >>>>>>> draft
>     >>>>>>> to
>     >>>>>>> implemente it to their own product They will have licencing
>     >>>>>>> /commercial
>     >>>>>>> politics that infringe lgpl, and hide that most of the work
>    is done
>     >>>>>>> by
>     >>>>>>> people
>     >>>>>>> other than them. They will not participate in a financial
>    way to ceph
>     >>>>>>> enhancement and growth.
>     >>>>>>
>     >>>>>> I don't think reimplementing the client code is an efficient way
>     >>>>>> forward.
>     >>>>>> Unless the goal is a pure kernel implementation...but a
>    significant
>     >>>>>> ongoing investment in development resources would be needed
>    for that
>     >>>>>> going
>     >>>>>> forward.  I suspect that is a challenge for a platform that
>    does not
>     >>>>>> typically rally that sort of community effort.
>     >>>>>>
>     >>>>>> The easiest thing is of course just to use CIFS and Samba
>    (which works
>     >>>>>> today).  A fuse-like approach is probably a reasonably
>    middle ground
>     >>>>>> (both
>     >>>>>> in initial effort and maintainability going forward)...
>     >>>>>>
>     >>>>>> sage
>     >>>>>>
>     >>>>>>
>     >>>>>
>     >>>
>     >>> --
>     >>> To unsubscribe from this list: send the line "unsubscribe
>    ceph-devel" in
>     >>> the body of a message to majordomo@vger.kernel.org
>    <mailto:majordomo@vger.kernel.org>
>
>     >>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>     >>
>     >>
>     >
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Matt W. Benjamin]

Hi,

The Window NFS v4.1 client is what we work on, so this may be good for
code sharing.  The license is lgplv2, like Ceph's.

Something important to be aware of is that the client uses rdbss, which
is a (partial) fsd abstraction that simplified implementation
quite a bit, kind of like a mini driver.  However, Microsoft's support
for rdbss has been in limbo for a bit.  For example, to link with
the rdbss symbols you can't use the Windows 8 driver kit--you'll need
to use the one for Windows 7.  (There's a private rdbss2 used internally
by Microsoft's SMB implemenation.  A the moment, 3rd party drivers
can't use that.)

We've been in communication with Microsoft about this issue, and know of
a few other fsds using it, but it could be a good thing for that lobbying
effort to have another user--or it could be a dead end :(.

There are a couple of other choices if you're looking to go this route,
that I'm aware of (and we may need to take them too, if RDBSS has no
way forward), but the required work could be a lot larger.

Matt

----- "Ketor D" <d.ketor@gmail.com> wrote:

> Hi Alphe:
>       Yes Callback Filesystem is very expensive and can't open
> source.
> It's not a good choice for ceph4win.
>       Another way for ceph4win maybe develop a kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think you can read its
> src code and maybe migrating from ceph kernel client to windows
> kernel
> fs is easier than from userspace ceph fuse client.And a kernel-mode
> fs
> client has greater performance than userspace fs like ceph-fuse
> client
> and ceph kernel client.
> 
>       Regards.
> 
> 
> 
> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels <asalas@kepler.cl>
> wrote:
> >
> > Commercial libraries are a pain ...
> >
> > If we want the more permossive licence offered by callback file
> system we
> > have to buy it for 20.000 usd. Then we will have to provide a
> backbox that
> > we have no control upon and that will kill our product anytime they
> want anf
> > if they decide to stop their commercial activity we will be in the
> same
> > situation that with dokanfs but without having the source code of
> the black
> > box. If i have to spend 20 000 dollars i would prefere paying
> someone to
> > retake dokanfs or to write from scratch a dokanfs fuselike software
> make it
> > all shiny and pumpy fantastic and ready to plug to ceph client.
> >
> > I would prefere if people have to pay something to get access to
> ceph4win
> > that this money goes in ceph main branch pockets... Or as a gift you
> donante
> > to ceph 10 dollars  you get 2 free registration codes for
> ceph4win... or
> > something like that.
> >
> > If ceph4win as to be comercial then I would prefer delegate the task
> to a
> > company like south river technologies and their great product
> webdrive. I
> > would mininaly get involved in that project and simply buy the final
> product
> > to sell it together with my ceph based product (which could be a
> calxeda
> > ceph box or something like that).
> >
> > I m open anyway to any proposition. But I doubt that callback
> filesystem
> > offers us a suitable solution in the way I see ceph4win to be spread
> and
> > used... I m maybe wrong. And anything that will be done around
> ceph4win will
> > be public documented etc... And licensed the way that if someone
> want to
> > build a commercial solution on top of it, that would be a
> possibility.
> >
> > My idea is to giveback somehow to ceph project and at same time
> forge a
> > better knowledge in ceph technologies. Because like many in libre
> world I
> > think the business is in the services around the software more than
> on the
> > software. That the ones writing code should be financed and benefits
> from
> > the one selling and giving support of the software at all levels. I
> m
> > probably too idealistic. And too optimistic after all I m the one
> saying I
> > will do this stuff I have no idea how but well it is interesting and
> fun so
> > lets do it.
> >
> > Regards,
> >
> > P.S: using commercial backend libraries appart including their own
> cost will
> > force you to use commercial IDE like MS VisualStudio because their
> library
> > has some kind of drm that only that IDE compiler can use. So alot of
> cost
> > and yet there is nothing done. If I had to open a kickstarter
> project saying
> > we need 60 000 USD to do ceph4win with that monney we will buy the
> right to
> > use and share a commercial copyrighted library but abandonned
> punctually to
> > us in  public domaine and that we will eventually produce something
> out of
> > it. I doubt I will get a dollar.
> >
> > We still can suggest the idea to Edlos the commercial company that
> has the
> > copyright of Callback FS, Or to buy them their product in a blender
> way
> > (blender was bought with donation before being put opensource and
> public
> > domaine), Or to open source their library. But in commercial minds
> > opensourcing = death of their technical advantage and death of
> their
> > marketing strategy. They will have to invent something more to
> retrieve
> > monney from it.
> >
> > El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> > <mailto:d.ketor@gmail.com>> escribió:
> >
> >
> >    Hi Alphe,
> >              I think you could try Callback Filesystem dev
> framework. It
> >    is a commerical dev framework and is maintained by Edlos today.
> >              I have communicated with Edlos to get a try code for
> >    development. To dokan, Callback Filesystem has vary document and
> maybe
> >    more stabilize.
> >
> >              Regards.
> >
> >
> >
> >    On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
> >    <mailto:asalas@kepler.cl>> wrote:
> >     > Hello ketor thank you for your interest un ceph4win. Since
> muy
> >    first mail I
> >     > exposed the lacks of dokanfs and that I m far from being a
> >    specialist un
> >     > filesystems.
> >     > I exposed what i like un dokanfs bit I not a fanátic of it.
> Muy
> >    goal is to
> >     > have something working quickly.
> >     >
> >     > So I am up to any proposición sure the one with the more docs
> and
> >    support
> >     > will be the best choice. As for right now what I need is
> >    understand what are
> >     > the files involved what are the interfaces functions and what
> are
> >    the needed
> >     > library dependencies and if they exist ported to windows with
> >    cygwin. And
> >     > all that is retrieved from source code.
> >     >
> >     > Regards.
> >     >
> >     > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
> >    <mailto:d.ketor@gmail.com>> escribió:
> >
> >     >
> >     >> Hi Alphe,
> >     >>      We are taking an interest in your work on Ceph Client
> for
> >    Windows
> >     >> with Dokan.As we know, the performance of Dokan is not very
> >    good, and it's
> >     >> abandoned 3 years ago.
> >     >>      I have learned and used OpenDedup(SDFS) for a long
> time.
> >      OpenDedup
> >     >> has a Dokan version. And the author of OpenDedup said
> >     >>
> >     >> The Dokan library is quite flakey  and testing should be
> >    performed before
> >     >> putting into production
> >     >>
> >     >>       So what do you think about this? And if there is
> another
> >    solution of
> >     >> fuse-like filesystem dev framwork on Windows?
> >     >>
> >     >>        Best Wish!
> >     >>
> >     >>
> >     >>
> >     >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
> >    <asalas@kepler.cl <mailto:asalas@kepler.cl>>
> >
> >     >> wrote:
> >     >>>
> >     >>> Hello I created the github repository for this project
> >     >>> https://github.com/alphe/Ceph4Win
> >     >>>
> >     >>> Regards,
> >     >>>
> >     >>> signature
> >     >>>
> >     >>> *Alphé Salas*
> >     >>> Ingeniero T.I
> >     >>>
> >     >>> asalas@kepler.cl <mailto:asalas@kepler.cl>
> >
> >     >>> *<http://www.kepler.cl>*
> >     >>>
> >     >>> On 11/05/13 21:00, Sage Weil wrote:
> >     >>>>
> >     >>>> Hi Alphe,
> >     >>>>
> >     >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> >     >>>>>
> >     >>>>> signature *Hi, Sage !
> >     >>>>> thank you for you enthousiast reply.
> >     >>>>> I sure want to make the best use of everything or
> anything
> >    previously
> >     >>>>> done to
> >     >>>>> tend to
> >     >>>>> write ceph cliente for windows.
> >     >>>>>
> >     >>>>> Apart using libre tools for building the future ceph
> cliente
> >    I am open
> >     >>>>> to
> >     >>>>> anything.
> >     >>>>> I would recommand eclipse CDT or Code::BLocks they are
> based
> >    on mingwin
> >     >>>>> open
> >     >>>>> and easyly enhanceable.**
> >     >>>>>
> >     >>>>> more free tools can be found here:
> >     >>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
> >     >>>>>
> >     >>>>>
> >     >>>>> I will read libcephfs source code and take some notes
> about the
> >     >>>>> protocol.
> >     >>>>
> >     >>>> I think you don't need to worry about hte protocol at all,
> since
> >     >>>> libcephs
> >     >>>> implements it for you (and will capture any future
> changes).
> >     >>>>
> >     >>>>> I was more going from what I know and trying to track down
> how
> >     >>>>> mount.ceph work
> >     >>>>> with the parameters passed to it.
> >     >>>>> since it point finally to Kernel/fs/ceph and that I don t
> really
> >     >>>>> understand
> >     >>>>> how that module work and that it probably points to some
> other
> >     >>>>> dependencies
> >     >>>>> Reading libcephfs source code could be a big gain of
> time.
> >     >>>>
> >     >>>> (I would also ignore mount.ceph as everything it does it
> >    specific to
> >     >>>> how Linux mounts work.)
> >     >>>>
> >     >>>>> basically on the protocol what is  need are:
> >     >>>>>
> >     >>>>> 1) open and maintain a connection (socket open, auth, etc
> )
> >     >>>>> 2) retreive a map of directories and disk Quota (disk
> sizing
> >    Y TB free,
> >     >>>>> Z TB
> >     >>>>> total)
> >     >>>>> 3) procedure to send files / directories in a maner that
> it
> >    will allow
> >     >>>>> our
> >     >>>>> client to fit ceph transmission protocols
> >     >>>>> (limit bandwith for stability?, limit connection amount?,
> >    limit cpu
> >     >>>>> use?,
> >     >>>>> Cache for preparing data transfer (a FIFO cache)?)
> >     >>>>> 4)Procedure to retreive files / directory from ceph
> cluster
> >     >>>>> 5) Management copy/move files /Directories, FS stats,
> >    Connection Stats.
> >     >>>>> logging.
> >     >>>>>
> >     >>>>> My idea to progress is to take those main bulletpoint in
> ceph
> >    protocol
> >     >>>>> based
> >     >>>>> on general ideas of what ceph file system does and start
> >    identifying
> >     >>>>> parts
> >     >>>>> from libcephfs to match those "needs".
> >     >>>>
> >     >>>> Instead, I would look at include/cephfs/libcephfs.h, the
> >    interface that
> >     >>>> libcephfs provides, and try to map that to what the fuse
> layer
> >    expects.
> >     >>>> There is both a path-based that I suspsect lends itself
> well
> >    to the
> >     >>>> Windows interface and (very soon now) a handle based API
> that is
> >     >>>> targetted
> >     >>>> at the Unix-style VFS layers.  I'm mostly guessing,
> though,
> >    since I've
> >     >>>> never seen any low-level fs code in windows before.
> >     >>>>
> >     >>>> In this case, the analogous code for Linux should be
> >    client/fuse_ll.cc
> >     >>>> itself (and not much else), although there will probably be
> a
> >    few tricks
> >     >>>> necessary to map cleanly onto how the windows interfaces
> work.
> >     >>>>
> >     >>>> Does that make sense?
> >     >>>>
> >     >>>> Cheers!
> >     >>>> sage
> >     >>>>
> >     >>>>
> >     >>>>> Any suggestion and contributions are welcome.
> >     >>>>>
> >     >>>>>
> >     >>>>> *
> >     >>>>> On 11/05/13 11:23, Sage Weil wrote:
> >     >>>>>>
> >     >>>>>> Hi Alphe,
> >     >>>>>>
> >     >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> >     >>>>>>>
> >     >>>>>>> Good day developers!
> >     >>>>>>>
> >     >>>>>>> I would like to propose to the one interested  work with
> me to
> >     >>>>>>> develop a
> >     >>>>>>> ceph
> >     >>>>>>> cliente for MS windows world, Basing us on dokanFS.
> >     >>>>>>>
> >     >>>>>>> My company is a ceph enthousiast that use on a dayly
> basis
> >    ceph and
> >     >>>>>>> that
> >     >>>>>>> need
> >     >>>>>>> both transfer speed and big expendable and cheap
> storage.
> >     >>>>>>> My company is specialised in data recovery and we want
> to
> >    participate
> >     >>>>>>> to
> >     >>>>>>> ceph
> >     >>>>>>> effort by bringing a ceph cliente for windows.
> >     >>>>>>
> >     >>>>>> Awesome!
> >     >>>>>>
> >     >>>>>>> Our experience shows us that the best gateway is each
> >    clientes being
> >     >>>>>>> its
> >     >>>>>>> own
> >     >>>>>>> gateway, instead of having a bottle neck server or a
> cluster of
> >     >>>>>>> bottle
> >     >>>>>>> neck
> >     >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
> etc..).
> >     >>>>>>>
> >     >>>>>>> We already did some research in that domain.
> >     >>>>>>>
> >     >>>>>>> Dokan FS is an intent  to write an opensource fuse like
> >    cliente for
> >     >>>>>>> MS
> >     >>>>>>> windows.
> >     >>>>>>>
> >     >>>>>>> More information on DOKANFS can be triggered here
> >     >>>>>>> http://dokan-dev.net/en/download/
> >     >>>>>>>
> >     >>>>>>> Positive points of using DOKANFS.
> >     >>>>>>>
> >     >>>>>>> - its opensourced and well licenced mit licence, gpl
> >    licence and lgpl
> >     >>>>>>> licence.
> >     >>>>>>>
> >     >>>>>>> Negative point of using DOKAN FS.
> >     >>>>>>> - unreachable author
> >     >>>>>>> - Poor documentation . Dev comments in japanese.
> >     >>>>>>> - Work in progress so it is unstable and needs to be
> updated,
> >     >>>>>>> debugged and
> >     >>>>>>> maintained by a MS Windows file system expert
> developper.
> >     >>>>>>
> >     >>>>>> I am not very familiar with windows storage APIs, but
> >    somebody told me
> >     >>>>>> at once point there were several interfaces against which
> a
> >    new file
> >     >>>>>> system could be implemented, everything from a full
> >    in-kernel driver
> >     >>>>>> to
> >     >>>>>> something that is explorer-based.  Are any of those
> >    suitable?  Using a
> >     >>>>>> potentially abandoned fuse-like layer makes me a bit
> nervous.
> >     >>>>>>
> >     >>>>>> That said,
> >     >>>>>>
> >     >>>>>>>
> >     >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
> >     >>>>>>> here are what I learnt.
> >     >>>>>>> - Ceph-fuse and related source code is around 60 000
> lines
> >    of code.
> >     >>>>>>> - Ceph protocol isn t documented so it is like trying
> to
> >    draw a map
> >     >>>>>>> of
> >     >>>>>>> america
> >     >>>>>>> using only a sextan and a compass.
> >     >>>>>>>
> >     >>>>>>> Those led me to those conclusions:
> >     >>>>>>> - I can t do it alone.
> >     >>>>>>> - It is easier to draw down the ceph protocol way to
> work from
> >     >>>>>>> kernel/fs/ceph
> >     >>>>>>> sources and mount.ceph
> >     >>>>>>> - Ceph depending libraries may be unexistant or not up
> to
> >    date in
> >     >>>>>>> their
> >     >>>>>>> port
> >     >>>>>>> on MS Windows (cygwin)
> >     >>>>>>
> >     >>>>>> I think the most sane path should be to make libcephfs
> >    sufficiently
> >     >>>>>> portable to build on windows (or cygwin).  For the bits
> used
> >    by the
> >     >>>>>> client-side coe, I don't think there should be much in
> the
> >    way of
> >     >>>>>> dependencies, and the main challenge would be untangling
> the
> >    build for
> >     >>>>>> the necessary pieces out from the rest of Ceph.
> >     >>>>>>
> >     >>>>>> Have you seen the wip-port portability work that is
> >    currently underway
> >     >>>>>> by
> >     >>>>>> Noah and Alan?  That may solve many of the cygwin
> problems
> >    you are
> >     >>>>>> seeing
> >     >>>>>> today.
> >     >>>>>>
> >     >>>>>>> - MS file system specialist are hard do find in the
> "open
> >    source
> >     >>>>>>> libre
> >     >>>>>>> world"
> >     >>>>>>> so I will try in the commercial world.
> >     >>>>>>>
> >     >>>>>>> The commercial world has some problems too. They need
> ceph
> >    protocol
> >     >>>>>>> draft
> >     >>>>>>> to
> >     >>>>>>> implemente it to their own product They will have
> licencing
> >     >>>>>>> /commercial
> >     >>>>>>> politics that infringe lgpl, and hide that most of the
> work
> >    is done
> >     >>>>>>> by
> >     >>>>>>> people
> >     >>>>>>> other than them. They will not participate in a
> financial
> >    way to ceph
> >     >>>>>>> enhancement and growth.
> >     >>>>>>
> >     >>>>>> I don't think reimplementing the client code is an
> efficient way
> >     >>>>>> forward.
> >     >>>>>> Unless the goal is a pure kernel implementation...but a
> >    significant
> >     >>>>>> ongoing investment in development resources would be
> needed
> >    for that
> >     >>>>>> going
> >     >>>>>> forward.  I suspect that is a challenge for a platform
> that
> >    does not
> >     >>>>>> typically rally that sort of community effort.
> >     >>>>>>
> >     >>>>>> The easiest thing is of course just to use CIFS and
> Samba
> >    (which works
> >     >>>>>> today).  A fuse-like approach is probably a reasonably
> >    middle ground
> >     >>>>>> (both
> >     >>>>>> in initial effort and maintainability going forward)...
> >     >>>>>>
> >     >>>>>> sage
> >     >>>>>>
> >     >>>>>>
> >     >>>>>
> >     >>>
> >     >>> --
> >     >>> To unsubscribe from this list: send the line "unsubscribe
> >    ceph-devel" in
> >     >>> the body of a message to majordomo@vger.kernel.org
> >    <mailto:majordomo@vger.kernel.org>
> >
> >     >>> More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> >     >>
> >     >>
> >     >
> >
> >
> >
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello D.Ketor and Matt Benjamin,
You give me alot to think about and this is great!
I merged your previous post to make a single reply that anyone can 
report to easyly

Windows NFS 4.1 is available here:
http://www.citi.umich.edu/projects/nfsv4/windows/readme.html

pnfs is another name for NFS4.X. It is presented as alternative to ceph 
and we get known terminology as MDS and OSD but without the self healing 
part if I understand well my rapid look on the topic. (when I say rapid 
look I mean ... 5 minutes spent in that... which is really small amount 
of time to get an accurate view on something)


starting from mount.ceph ... I know that mount.ceph does little but it 
is a great hint to know what ceph needs and do things.
Basically mount.ceph modprobe the ceph driver in the linux kernel then 
call mount with the line command passed args and the cephfs type as 
argument. Then the kernel does the work I don t understand yet what is 
the start calls that are made to the ceph driver but it seemed to me 
that is was relatively light. (a first impression compared to ceph-fuse.)

I think I will do both isolate source code from ceph-client kernel 
(cephfs module for linux kernel) and the one pointed by Sage starting 
from client/fuse_ll.cc in ceph master branch. The common files betwin 
those 2 extractions will be our core set of mandatory features.

Then we try to compile with cygwin a cephfs client library . Then we 
will try to interface with a modified windows nfs 4.1 client or pnfs or 
any other that will accept to be compiled with gcc for win32...

the fact that windows 8.1 is and windows 2012 are out of reach at the 
moment is not a problem to me.

Our first concern is to understand what is ceph protocol. Then adapt it 
to something that can be used on windows prior windows 8.1. Dokan fs if 
I remember well use too the WDK (windows driver dev-kit ) for it s 
compilation so possibly we will see the same limitations.

We need to multiply our source of information by example regarding 
ceph-client (kernel or fuse, radosgw is on a different layer so I will 
not try anything around it at first.) And we need to multiply our source 
of information by example regarding virtual file system technologies on 
windoes OS.
Alot of work but all of those available source code everyone point at me 
will make our best solution. And in the end we will choose technologies 
knowing what we do and what concequencies they have.

regards,




Regards

signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl


On 11/07/13 11:29, Ketor D wrote:
> Hi Alphe:
>        Yes Callback Filesystem is very expensive and can't open source.
> It's not a good choice for ceph4win.
>        Another way for ceph4win maybe develop a kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think you can read its
> src code and maybe migrating from ceph kernel client to windows kernel
> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
> client has greater performance than userspace fs like ceph-fuse client
> and ceph kernel client.
>
>        Regards.
>
On 11/07/13 11:50, Matt W. Benjamin wrote:
> Hi,
>
> The Window NFS v4.1 client is what we work on, so this may be good for
> code sharing.  The license is lgplv2, like Ceph's.
>
> Something important to be aware of is that the client uses rdbss, which
> is a (partial) fsd abstraction that simplified implementation
> quite a bit, kind of like a mini driver.  However, Microsoft's support
> for rdbss has been in limbo for a bit.  For example, to link with
> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
> to use the one for Windows 7.  (There's a private rdbss2 used internally
> by Microsoft's SMB implemenation.  A the moment, 3rd party drivers
> can't use that.)
>
> We've been in communication with Microsoft about this issue, and know of
> a few other fsds using it, but it could be a good thing for that lobbying
> effort to have another user--or it could be a dead end :(.
>
> There are a couple of other choices if you're looking to go this route,
> that I'm aware of (and we may need to take them too, if RDBSS has no
> way forward), but the required work could be a lot larger.
>
> Matt
>
> ----- "Ketor D"<d.ketor@gmail.com>  wrote:
>
>> Hi Alphe:
>>        Yes Callback Filesystem is very expensive and can't open
>> source.
>> It's not a good choice for ceph4win.
>>        Another way for ceph4win maybe develop a kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>> src code and maybe migrating from ceph kernel client to windows
>> kernel
>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>> fs
>> client has greater performance than userspace fs like ceph-fuse
>> client
>> and ceph kernel client.
>>
>>        Regards.
>>
>>
>>
>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>> wrote:
>>> Commercial libraries are a pain ...
>>>
>>> If we want the more permossive licence offered by callback file
>> system we
>>> have to buy it for 20.000 usd. Then we will have to provide a
>> backbox that
>>> we have no control upon and that will kill our product anytime they
>> want anf
>>> if they decide to stop their commercial activity we will be in the
>> same
>>> situation that with dokanfs but without having the source code of
>> the black
>>> box. If i have to spend 20 000 dollars i would prefere paying
>> someone to
>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>> make it
>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>
>>> I would prefere if people have to pay something to get access to
>> ceph4win
>>> that this money goes in ceph main branch pockets... Or as a gift you
>> donante
>>> to ceph 10 dollars  you get 2 free registration codes for
>> ceph4win... or
>>> something like that.
>>>
>>> If ceph4win as to be comercial then I would prefer delegate the task
>> to a
>>> company like south river technologies and their great product
>> webdrive. I
>>> would mininaly get involved in that project and simply buy the final
>> product
>>> to sell it together with my ceph based product (which could be a
>> calxeda
>>> ceph box or something like that).
>>>
>>> I m open anyway to any proposition. But I doubt that callback
>> filesystem
>>> offers us a suitable solution in the way I see ceph4win to be spread
>> and
>>> used... I m maybe wrong. And anything that will be done around
>> ceph4win will
>>> be public documented etc... And licensed the way that if someone
>> want to
>>> build a commercial solution on top of it, that would be a
>> possibility.
>>> My idea is to giveback somehow to ceph project and at same time
>> forge a
>>> better knowledge in ceph technologies. Because like many in libre
>> world I
>>> think the business is in the services around the software more than
>> on the
>>> software. That the ones writing code should be financed and benefits
>> from
>>> the one selling and giving support of the software at all levels. I
>> m
>>> probably too idealistic. And too optimistic after all I m the one
>> saying I
>>> will do this stuff I have no idea how but well it is interesting and
>> fun so
>>> lets do it.
>>>
>>> Regards,
>>>
>>> P.S: using commercial backend libraries appart including their own
>> cost will
>>> force you to use commercial IDE like MS VisualStudio because their
>> library
>>> has some kind of drm that only that IDE compiler can use. So alot of
>> cost
>>> and yet there is nothing done. If I had to open a kickstarter
>> project saying
>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>> right to
>>> use and share a commercial copyrighted library but abandonned
>> punctually to
>>> us in  public domaine and that we will eventually produce something
>> out of
>>> it. I doubt I will get a dollar.
>>>
>>> We still can suggest the idea to Edlos the commercial company that
>> has the
>>> copyright of Callback FS, Or to buy them their product in a blender
>> way
>>> (blender was bought with donation before being put opensource and
>> public
>>> domaine), Or to open source their library. But in commercial minds
>>> opensourcing = death of their technical advantage and death of
>> their
>>> marketing strategy. They will have to invent something more to
>> retrieve
>>> monney from it.
>>>
>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>> <mailto:d.ketor@gmail.com>> escribió:
>>>
>>>
>>>     Hi Alphe,
>>>               I think you could try Callback Filesystem dev
>> framework. It
>>>     is a commerical dev framework and is maintained by Edlos today.
>>>               I have communicated with Edlos to get a try code for
>>>     development. To dokan, Callback Filesystem has vary document and
>> maybe
>>>     more stabilize.
>>>
>>>               Regards.
>>>
>>>
>>>
>>>     On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>>     <mailto:asalas@kepler.cl>> wrote:
>>>      > Hello ketor thank you for your interest un ceph4win. Since
>> muy
>>>     first mail I
>>>      > exposed the lacks of dokanfs and that I m far from being a
>>>     specialist un
>>>      > filesystems.
>>>      > I exposed what i like un dokanfs bit I not a fanátic of it.
>> Muy
>>>     goal is to
>>>      > have something working quickly.
>>>      >
>>>      > So I am up to any proposición sure the one with the more docs
>> and
>>>     support
>>>      > will be the best choice. As for right now what I need is
>>>     understand what are
>>>      > the files involved what are the interfaces functions and what
>> are
>>>     the needed
>>>      > library dependencies and if they exist ported to windows with
>>>     cygwin. And
>>>      > all that is retrieved from source code.
>>>      >
>>>      > Regards.
>>>      >
>>>      > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>>     <mailto:d.ketor@gmail.com>> escribió:
>>>
>>>      >
>>>      >> Hi Alphe,
>>>      >>      We are taking an interest in your work on Ceph Client
>> for
>>>     Windows
>>>      >> with Dokan.As we know, the performance of Dokan is not very
>>>     good, and it's
>>>      >> abandoned 3 years ago.
>>>      >>      I have learned and used OpenDedup(SDFS) for a long
>> time.
>>>       OpenDedup
>>>      >> has a Dokan version. And the author of OpenDedup said
>>>      >>
>>>      >> The Dokan library is quite flakey  and testing should be
>>>     performed before
>>>      >> putting into production
>>>      >>
>>>      >>       So what do you think about this? And if there is
>> another
>>>     solution of
>>>      >> fuse-like filesystem dev framwork on Windows?
>>>      >>
>>>      >>        Best Wish!
>>>      >>
>>>      >>
>>>      >>
>>>      >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>>     <asalas@kepler.cl  <mailto:asalas@kepler.cl>>
>>>
>>>      >> wrote:
>>>      >>>
>>>      >>> Hello I created the github repository for this project
>>>      >>>https://github.com/alphe/Ceph4Win
>>>      >>>
>>>      >>> Regards,
>>>      >>>
>>>      >>> signature
>>>      >>>
>>>      >>> *Alphé Salas*
>>>      >>> Ingeniero T.I
>>>      >>>
>>>      >>>asalas@kepler.cl  <mailto:asalas@kepler.cl>
>>>
>>>      >>> *<http://www.kepler.cl>*
>>>      >>>
>>>      >>> On 11/05/13 21:00, Sage Weil wrote:
>>>      >>>>
>>>      >>>> Hi Alphe,
>>>      >>>>
>>>      >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>      >>>>>
>>>      >>>>> signature *Hi, Sage !
>>>      >>>>> thank you for you enthousiast reply.
>>>      >>>>> I sure want to make the best use of everything or
>> anything
>>>     previously
>>>      >>>>> done to
>>>      >>>>> tend to
>>>      >>>>> write ceph cliente for windows.
>>>      >>>>>
>>>      >>>>> Apart using libre tools for building the future ceph
>> cliente
>>>     I am open
>>>      >>>>> to
>>>      >>>>> anything.
>>>      >>>>> I would recommand eclipse CDT or Code::BLocks they are
>> based
>>>     on mingwin
>>>      >>>>> open
>>>      >>>>> and easyly enhanceable.**
>>>      >>>>>
>>>      >>>>> more free tools can be found here:
>>>      >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>      >>>>>
>>>      >>>>>
>>>      >>>>> I will read libcephfs source code and take some notes
>> about the
>>>      >>>>> protocol.
>>>      >>>>
>>>      >>>> I think you don't need to worry about hte protocol at all,
>> since
>>>      >>>> libcephs
>>>      >>>> implements it for you (and will capture any future
>> changes).
>>>      >>>>
>>>      >>>>> I was more going from what I know and trying to track down
>> how
>>>      >>>>> mount.ceph work
>>>      >>>>> with the parameters passed to it.
>>>      >>>>> since it point finally to Kernel/fs/ceph and that I don t
>> really
>>>      >>>>> understand
>>>      >>>>> how that module work and that it probably points to some
>> other
>>>      >>>>> dependencies
>>>      >>>>> Reading libcephfs source code could be a big gain of
>> time.
>>>      >>>>
>>>      >>>> (I would also ignore mount.ceph as everything it does it
>>>     specific to
>>>      >>>> how Linux mounts work.)
>>>      >>>>
>>>      >>>>> basically on the protocol what is  need are:
>>>      >>>>>
>>>      >>>>> 1) open and maintain a connection (socket open, auth, etc
>> )
>>>      >>>>> 2) retreive a map of directories and disk Quota (disk
>> sizing
>>>     Y TB free,
>>>      >>>>> Z TB
>>>      >>>>> total)
>>>      >>>>> 3) procedure to send files / directories in a maner that
>> it
>>>     will allow
>>>      >>>>> our
>>>      >>>>> client to fit ceph transmission protocols
>>>      >>>>> (limit bandwith for stability?, limit connection amount?,
>>>     limit cpu
>>>      >>>>> use?,
>>>      >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>      >>>>> 4)Procedure to retreive files / directory from ceph
>> cluster
>>>      >>>>> 5) Management copy/move files /Directories, FS stats,
>>>     Connection Stats.
>>>      >>>>> logging.
>>>      >>>>>
>>>      >>>>> My idea to progress is to take those main bulletpoint in
>> ceph
>>>     protocol
>>>      >>>>> based
>>>      >>>>> on general ideas of what ceph file system does and start
>>>     identifying
>>>      >>>>> parts
>>>      >>>>> from libcephfs to match those "needs".
>>>      >>>>
>>>      >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>>     interface that
>>>      >>>> libcephfs provides, and try to map that to what the fuse
>> layer
>>>     expects.
>>>      >>>> There is both a path-based that I suspsect lends itself
>> well
>>>     to the
>>>      >>>> Windows interface and (very soon now) a handle based API
>> that is
>>>      >>>> targetted
>>>      >>>> at the Unix-style VFS layers.  I'm mostly guessing,
>> though,
>>>     since I've
>>>      >>>> never seen any low-level fs code in windows before.
>>>      >>>>
>>>      >>>> In this case, the analogous code for Linux should be
>>>     client/fuse_ll.cc
>>>      >>>> itself (and not much else), although there will probably be
>> a
>>>     few tricks
>>>      >>>> necessary to map cleanly onto how the windows interfaces
>> work.
>>>      >>>>
>>>      >>>> Does that make sense?
>>>      >>>>
>>>      >>>> Cheers!
>>>      >>>> sage
>>>      >>>>
>>>      >>>>
>>>      >>>>> Any suggestion and contributions are welcome.
>>>      >>>>>
>>>      >>>>>
>>>      >>>>> *
>>>      >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>      >>>>>>
>>>      >>>>>> Hi Alphe,
>>>      >>>>>>
>>>      >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>      >>>>>>>
>>>      >>>>>>> Good day developers!
>>>      >>>>>>>
>>>      >>>>>>> I would like to propose to the one interested  work with
>> me to
>>>      >>>>>>> develop a
>>>      >>>>>>> ceph
>>>      >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>      >>>>>>>
>>>      >>>>>>> My company is a ceph enthousiast that use on a dayly
>> basis
>>>     ceph and
>>>      >>>>>>> that
>>>      >>>>>>> need
>>>      >>>>>>> both transfer speed and big expendable and cheap
>> storage.
>>>      >>>>>>> My company is specialised in data recovery and we want
>> to
>>>     participate
>>>      >>>>>>> to
>>>      >>>>>>> ceph
>>>      >>>>>>> effort by bringing a ceph cliente for windows.
>>>      >>>>>>
>>>      >>>>>> Awesome!
>>>      >>>>>>
>>>      >>>>>>> Our experience shows us that the best gateway is each
>>>     clientes being
>>>      >>>>>>> its
>>>      >>>>>>> own
>>>      >>>>>>> gateway, instead of having a bottle neck server or a
>> cluster of
>>>      >>>>>>> bottle
>>>      >>>>>>> neck
>>>      >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>> etc..).
>>>      >>>>>>>
>>>      >>>>>>> We already did some research in that domain.
>>>      >>>>>>>
>>>      >>>>>>> Dokan FS is an intent  to write an opensource fuse like
>>>     cliente for
>>>      >>>>>>> MS
>>>      >>>>>>> windows.
>>>      >>>>>>>
>>>      >>>>>>> More information on DOKANFS can be triggered here
>>>      >>>>>>>http://dokan-dev.net/en/download/
>>>      >>>>>>>
>>>      >>>>>>> Positive points of using DOKANFS.
>>>      >>>>>>>
>>>      >>>>>>> - its opensourced and well licenced mit licence, gpl
>>>     licence and lgpl
>>>      >>>>>>> licence.
>>>      >>>>>>>
>>>      >>>>>>> Negative point of using DOKAN FS.
>>>      >>>>>>> - unreachable author
>>>      >>>>>>> - Poor documentation . Dev comments in japanese.
>>>      >>>>>>> - Work in progress so it is unstable and needs to be
>> updated,
>>>      >>>>>>> debugged and
>>>      >>>>>>> maintained by a MS Windows file system expert
>> developper.
>>>      >>>>>>
>>>      >>>>>> I am not very familiar with windows storage APIs, but
>>>     somebody told me
>>>      >>>>>> at once point there were several interfaces against which
>> a
>>>     new file
>>>      >>>>>> system could be implemented, everything from a full
>>>     in-kernel driver
>>>      >>>>>> to
>>>      >>>>>> something that is explorer-based.  Are any of those
>>>     suitable?  Using a
>>>      >>>>>> potentially abandoned fuse-like layer makes me a bit
>> nervous.
>>>      >>>>>>
>>>      >>>>>> That said,
>>>      >>>>>>
>>>      >>>>>>>
>>>      >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>      >>>>>>> here are what I learnt.
>>>      >>>>>>> - Ceph-fuse and related source code is around 60 000
>> lines
>>>     of code.
>>>      >>>>>>> - Ceph protocol isn t documented so it is like trying
>> to
>>>     draw a map
>>>      >>>>>>> of
>>>      >>>>>>> america
>>>      >>>>>>> using only a sextan and a compass.
>>>      >>>>>>>
>>>      >>>>>>> Those led me to those conclusions:
>>>      >>>>>>> - I can t do it alone.
>>>      >>>>>>> - It is easier to draw down the ceph protocol way to
>> work from
>>>      >>>>>>> kernel/fs/ceph
>>>      >>>>>>> sources and mount.ceph
>>>      >>>>>>> - Ceph depending libraries may be unexistant or not up
>> to
>>>     date in
>>>      >>>>>>> their
>>>      >>>>>>> port
>>>      >>>>>>> on MS Windows (cygwin)
>>>      >>>>>>
>>>      >>>>>> I think the most sane path should be to make libcephfs
>>>     sufficiently
>>>      >>>>>> portable to build on windows (or cygwin).  For the bits
>> used
>>>     by the
>>>      >>>>>> client-side coe, I don't think there should be much in
>> the
>>>     way of
>>>      >>>>>> dependencies, and the main challenge would be untangling
>> the
>>>     build for
>>>      >>>>>> the necessary pieces out from the rest of Ceph.
>>>      >>>>>>
>>>      >>>>>> Have you seen the wip-port portability work that is
>>>     currently underway
>>>      >>>>>> by
>>>      >>>>>> Noah and Alan?  That may solve many of the cygwin
>> problems
>>>     you are
>>>      >>>>>> seeing
>>>      >>>>>> today.
>>>      >>>>>>
>>>      >>>>>>> - MS file system specialist are hard do find in the
>> "open
>>>     source
>>>      >>>>>>> libre
>>>      >>>>>>> world"
>>>      >>>>>>> so I will try in the commercial world.
>>>      >>>>>>>
>>>      >>>>>>> The commercial world has some problems too. They need
>> ceph
>>>     protocol
>>>      >>>>>>> draft
>>>      >>>>>>> to
>>>      >>>>>>> implemente it to their own product They will have
>> licencing
>>>      >>>>>>> /commercial
>>>      >>>>>>> politics that infringe lgpl, and hide that most of the
>> work
>>>     is done
>>>      >>>>>>> by
>>>      >>>>>>> people
>>>      >>>>>>> other than them. They will not participate in a
>> financial
>>>     way to ceph
>>>      >>>>>>> enhancement and growth.
>>>      >>>>>>
>>>      >>>>>> I don't think reimplementing the client code is an
>> efficient way
>>>      >>>>>> forward.
>>>      >>>>>> Unless the goal is a pure kernel implementation...but a
>>>     significant
>>>      >>>>>> ongoing investment in development resources would be
>> needed
>>>     for that
>>>      >>>>>> going
>>>      >>>>>> forward.  I suspect that is a challenge for a platform
>> that
>>>     does not
>>>      >>>>>> typically rally that sort of community effort.
>>>      >>>>>>
>>>      >>>>>> The easiest thing is of course just to use CIFS and
>> Samba
>>>     (which works
>>>      >>>>>> today).  A fuse-like approach is probably a reasonably
>>>     middle ground
>>>      >>>>>> (both
>>>      >>>>>> in initial effort and maintainability going forward)...
>>>      >>>>>>
>>>      >>>>>> sage
>>>      >>>>>>
>>>      >>>>>>
>>>      >>>>>
>>>      >>>
>>>      >>> --
>>>      >>> To unsubscribe from this list: send the line "unsubscribe
>>>     ceph-devel" in
>>>      >>> the body of a message tomajordomo@vger.kernel.org
>>>     <mailto:majordomo@vger.kernel.org>
>>>
>>>      >>> More majordomo info at
>> http://vger.kernel.org/majordomo-info.html
>>>      >>
>>>      >>
>>>      >
>>>
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>> in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello all I finally finished my first source code extraction that starts 
from ceph/src/client/fuse_ll.c
The result is accurate unlike previous provided results. basically the 
script start from a file extract all the private includes definitions  
#include "something.h" and recursively extract private includes too. the 
best way to know who is related with who.

starting from fuse_ll.cc I optain 390 files retreived and 120 000 lines 
of code !
involved dirs are : in ceph/src
objclass/, common/, msg/, common/, osdc/, include/, client/, mds/, 
global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/

probably not a good way to analyse what amount of work it means since 
most of those directories are the implementation of servers (osd, mon, 
mds) and even if only a tiny bit of them is needed at client level. you 
need two structures from ./osd/OSD.h and  my script by relation will 
take into acount the whole directory...

I ran the script with libcephfs.cc as start point and got almost the 
same results. 131 000 lines of code and 386 files most of the same dirs 
involved.



I think I will spend alot of time doing the manual source code isolation 
and understand way each #include is set in the files I read (what 
purpose they have do they allow to integrate a crucial data type or not.


The other way around will be to read src/libcephfs.cc. It seems shorter 
but without understanding what part is used for each included header I 
can t say anything...



I will keep reading the source code and take notes. I think in the case 
of libcephfs I will gain alot of time.

signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl
*www.kepler.cl <http://www.kepler.cl>*

On 11/07/13 15:02, Alphe Salas Michels wrote:
> Hello D.Ketor and Matt Benjamin,
> You give me alot to think about and this is great!
> I merged your previous post to make a single reply that anyone can 
> report to easyly
>
> Windows NFS 4.1 is available here:
> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>
> pnfs is another name for NFS4.X. It is presented as alternative to 
> ceph and we get known terminology as MDS and OSD but without the self 
> healing part if I understand well my rapid look on the topic. (when I 
> say rapid look I mean ... 5 minutes spent in that... which is really 
> small amount of time to get an accurate view on something)
>
>
> starting from mount.ceph ... I know that mount.ceph does little but it 
> is a great hint to know what ceph needs and do things.
> Basically mount.ceph modprobe the ceph driver in the linux kernel then 
> call mount with the line command passed args and the cephfs type as 
> argument. Then the kernel does the work I don t understand yet what is 
> the start calls that are made to the ceph driver but it seemed to me 
> that is was relatively light. (a first impression compared to ceph-fuse.)
>
> I think I will do both isolate source code from ceph-client kernel 
> (cephfs module for linux kernel) and the one pointed by Sage starting 
> from client/fuse_ll.cc in ceph master branch. The common files betwin 
> those 2 extractions will be our core set of mandatory features.
>
> Then we try to compile with cygwin a cephfs client library . Then we 
> will try to interface with a modified windows nfs 4.1 client or pnfs 
> or any other that will accept to be compiled with gcc for win32...
>
> the fact that windows 8.1 is and windows 2012 are out of reach at the 
> moment is not a problem to me.
>
> Our first concern is to understand what is ceph protocol. Then adapt 
> it to something that can be used on windows prior windows 8.1. Dokan 
> fs if I remember well use too the WDK (windows driver dev-kit ) for it 
> s compilation so possibly we will see the same limitations.
>
> We need to multiply our source of information by example regarding 
> ceph-client (kernel or fuse, radosgw is on a different layer so I will 
> not try anything around it at first.) And we need to multiply our 
> source of information by example regarding virtual file system 
> technologies on windoes OS.
> Alot of work but all of those available source code everyone point at 
> me will make our best solution. And in the end we will choose 
> technologies knowing what we do and what concequencies they have.
>
> regards,
>
>
>
>
> Regards
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl
>
>
> On 11/07/13 11:29, Ketor D wrote:
>> Hi Alphe:
>>        Yes Callback Filesystem is very expensive and can't open source.
>> It's not a good choice for ceph4win.
>>        Another way for ceph4win maybe develop a kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>> src code and maybe migrating from ceph kernel client to windows kernel
>> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
>> client has greater performance than userspace fs like ceph-fuse client
>> and ceph kernel client.
>>
>>        Regards.
>>
> On 11/07/13 11:50, Matt W. Benjamin wrote:
>> Hi,
>>
>> The Window NFS v4.1 client is what we work on, so this may be good for
>> code sharing.  The license is lgplv2, like Ceph's.
>>
>> Something important to be aware of is that the client uses rdbss, which
>> is a (partial) fsd abstraction that simplified implementation
>> quite a bit, kind of like a mini driver.  However, Microsoft's support
>> for rdbss has been in limbo for a bit.  For example, to link with
>> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
>> to use the one for Windows 7.  (There's a private rdbss2 used internally
>> by Microsoft's SMB implemenation.  A the moment, 3rd party drivers
>> can't use that.)
>>
>> We've been in communication with Microsoft about this issue, and know of
>> a few other fsds using it, but it could be a good thing for that 
>> lobbying
>> effort to have another user--or it could be a dead end :(.
>>
>> There are a couple of other choices if you're looking to go this route,
>> that I'm aware of (and we may need to take them too, if RDBSS has no
>> way forward), but the required work could be a lot larger.
>>
>> Matt
>>
>> ----- "Ketor D"<d.ketor@gmail.com>  wrote:
>>
>>> Hi Alphe:
>>>        Yes Callback Filesystem is very expensive and can't open
>>> source.
>>> It's not a good choice for ceph4win.
>>>        Another way for ceph4win maybe develop a kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>> src code and maybe migrating from ceph kernel client to windows
>>> kernel
>>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>>> fs
>>> client has greater performance than userspace fs like ceph-fuse
>>> client
>>> and ceph kernel client.
>>>
>>>        Regards.
>>>
>>>
>>>
>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>>> wrote:
>>>> Commercial libraries are a pain ...
>>>>
>>>> If we want the more permossive licence offered by callback file
>>> system we
>>>> have to buy it for 20.000 usd. Then we will have to provide a
>>> backbox that
>>>> we have no control upon and that will kill our product anytime they
>>> want anf
>>>> if they decide to stop their commercial activity we will be in the
>>> same
>>>> situation that with dokanfs but without having the source code of
>>> the black
>>>> box. If i have to spend 20 000 dollars i would prefere paying
>>> someone to
>>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>>> make it
>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>>
>>>> I would prefere if people have to pay something to get access to
>>> ceph4win
>>>> that this money goes in ceph main branch pockets... Or as a gift you
>>> donante
>>>> to ceph 10 dollars  you get 2 free registration codes for
>>> ceph4win... or
>>>> something like that.
>>>>
>>>> If ceph4win as to be comercial then I would prefer delegate the task
>>> to a
>>>> company like south river technologies and their great product
>>> webdrive. I
>>>> would mininaly get involved in that project and simply buy the final
>>> product
>>>> to sell it together with my ceph based product (which could be a
>>> calxeda
>>>> ceph box or something like that).
>>>>
>>>> I m open anyway to any proposition. But I doubt that callback
>>> filesystem
>>>> offers us a suitable solution in the way I see ceph4win to be spread
>>> and
>>>> used... I m maybe wrong. And anything that will be done around
>>> ceph4win will
>>>> be public documented etc... And licensed the way that if someone
>>> want to
>>>> build a commercial solution on top of it, that would be a
>>> possibility.
>>>> My idea is to giveback somehow to ceph project and at same time
>>> forge a
>>>> better knowledge in ceph technologies. Because like many in libre
>>> world I
>>>> think the business is in the services around the software more than
>>> on the
>>>> software. That the ones writing code should be financed and benefits
>>> from
>>>> the one selling and giving support of the software at all levels. I
>>> m
>>>> probably too idealistic. And too optimistic after all I m the one
>>> saying I
>>>> will do this stuff I have no idea how but well it is interesting and
>>> fun so
>>>> lets do it.
>>>>
>>>> Regards,
>>>>
>>>> P.S: using commercial backend libraries appart including their own
>>> cost will
>>>> force you to use commercial IDE like MS VisualStudio because their
>>> library
>>>> has some kind of drm that only that IDE compiler can use. So alot of
>>> cost
>>>> and yet there is nothing done. If I had to open a kickstarter
>>> project saying
>>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>>> right to
>>>> use and share a commercial copyrighted library but abandonned
>>> punctually to
>>>> us in  public domaine and that we will eventually produce something
>>> out of
>>>> it. I doubt I will get a dollar.
>>>>
>>>> We still can suggest the idea to Edlos the commercial company that
>>> has the
>>>> copyright of Callback FS, Or to buy them their product in a blender
>>> way
>>>> (blender was bought with donation before being put opensource and
>>> public
>>>> domaine), Or to open source their library. But in commercial minds
>>>> opensourcing = death of their technical advantage and death of
>>> their
>>>> marketing strategy. They will have to invent something more to
>>> retrieve
>>>> monney from it.
>>>>
>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>
>>>>
>>>>     Hi Alphe,
>>>>               I think you could try Callback Filesystem dev
>>> framework. It
>>>>     is a commerical dev framework and is maintained by Edlos today.
>>>>               I have communicated with Edlos to get a try code for
>>>>     development. To dokan, Callback Filesystem has vary document and
>>> maybe
>>>>     more stabilize.
>>>>
>>>>               Regards.
>>>>
>>>>
>>>>
>>>>     On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>>>     <mailto:asalas@kepler.cl>> wrote:
>>>>      > Hello ketor thank you for your interest un ceph4win. Since
>>> muy
>>>>     first mail I
>>>>      > exposed the lacks of dokanfs and that I m far from being a
>>>>     specialist un
>>>>      > filesystems.
>>>>      > I exposed what i like un dokanfs bit I not a fanátic of it.
>>> Muy
>>>>     goal is to
>>>>      > have something working quickly.
>>>>      >
>>>>      > So I am up to any proposición sure the one with the more docs
>>> and
>>>>     support
>>>>      > will be the best choice. As for right now what I need is
>>>>     understand what are
>>>>      > the files involved what are the interfaces functions and what
>>> are
>>>>     the needed
>>>>      > library dependencies and if they exist ported to windows with
>>>>     cygwin. And
>>>>      > all that is retrieved from source code.
>>>>      >
>>>>      > Regards.
>>>>      >
>>>>      > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>>>     <mailto:d.ketor@gmail.com>> escribió:
>>>>
>>>>      >
>>>>      >> Hi Alphe,
>>>>      >>      We are taking an interest in your work on Ceph Client
>>> for
>>>>     Windows
>>>>      >> with Dokan.As we know, the performance of Dokan is not very
>>>>     good, and it's
>>>>      >> abandoned 3 years ago.
>>>>      >>      I have learned and used OpenDedup(SDFS) for a long
>>> time.
>>>>       OpenDedup
>>>>      >> has a Dokan version. And the author of OpenDedup said
>>>>      >>
>>>>      >> The Dokan library is quite flakey  and testing should be
>>>>     performed before
>>>>      >> putting into production
>>>>      >>
>>>>      >>       So what do you think about this? And if there is
>>> another
>>>>     solution of
>>>>      >> fuse-like filesystem dev framwork on Windows?
>>>>      >>
>>>>      >>        Best Wish!
>>>>      >>
>>>>      >>
>>>>      >>
>>>>      >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>>>     <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>>
>>>>      >> wrote:
>>>>      >>>
>>>>      >>> Hello I created the github repository for this project
>>>>      >>>https://github.com/alphe/Ceph4Win
>>>>      >>>
>>>>      >>> Regards,
>>>>      >>>
>>>>      >>> signature
>>>>      >>>
>>>>      >>> *Alphé Salas*
>>>>      >>> Ingeniero T.I
>>>>      >>>
>>>>      >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>>
>>>>      >>> *<http://www.kepler.cl>*
>>>>      >>>
>>>>      >>> On 11/05/13 21:00, Sage Weil wrote:
>>>>      >>>>
>>>>      >>>> Hi Alphe,
>>>>      >>>>
>>>>      >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>>      >>>>>
>>>>      >>>>> signature *Hi, Sage !
>>>>      >>>>> thank you for you enthousiast reply.
>>>>      >>>>> I sure want to make the best use of everything or
>>> anything
>>>>     previously
>>>>      >>>>> done to
>>>>      >>>>> tend to
>>>>      >>>>> write ceph cliente for windows.
>>>>      >>>>>
>>>>      >>>>> Apart using libre tools for building the future ceph
>>> cliente
>>>>     I am open
>>>>      >>>>> to
>>>>      >>>>> anything.
>>>>      >>>>> I would recommand eclipse CDT or Code::BLocks they are
>>> based
>>>>     on mingwin
>>>>      >>>>> open
>>>>      >>>>> and easyly enhanceable.**
>>>>      >>>>>
>>>>      >>>>> more free tools can be found here:
>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>>      >>>>>
>>>>      >>>>>
>>>>      >>>>> I will read libcephfs source code and take some notes
>>> about the
>>>>      >>>>> protocol.
>>>>      >>>>
>>>>      >>>> I think you don't need to worry about hte protocol at all,
>>> since
>>>>      >>>> libcephs
>>>>      >>>> implements it for you (and will capture any future
>>> changes).
>>>>      >>>>
>>>>      >>>>> I was more going from what I know and trying to track down
>>> how
>>>>      >>>>> mount.ceph work
>>>>      >>>>> with the parameters passed to it.
>>>>      >>>>> since it point finally to Kernel/fs/ceph and that I don t
>>> really
>>>>      >>>>> understand
>>>>      >>>>> how that module work and that it probably points to some
>>> other
>>>>      >>>>> dependencies
>>>>      >>>>> Reading libcephfs source code could be a big gain of
>>> time.
>>>>      >>>>
>>>>      >>>> (I would also ignore mount.ceph as everything it does it
>>>>     specific to
>>>>      >>>> how Linux mounts work.)
>>>>      >>>>
>>>>      >>>>> basically on the protocol what is need are:
>>>>      >>>>>
>>>>      >>>>> 1) open and maintain a connection (socket open, auth, etc
>>> )
>>>>      >>>>> 2) retreive a map of directories and disk Quota (disk
>>> sizing
>>>>     Y TB free,
>>>>      >>>>> Z TB
>>>>      >>>>> total)
>>>>      >>>>> 3) procedure to send files / directories in a maner that
>>> it
>>>>     will allow
>>>>      >>>>> our
>>>>      >>>>> client to fit ceph transmission protocols
>>>>      >>>>> (limit bandwith for stability?, limit connection amount?,
>>>>     limit cpu
>>>>      >>>>> use?,
>>>>      >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>>      >>>>> 4)Procedure to retreive files / directory from ceph
>>> cluster
>>>>      >>>>> 5) Management copy/move files /Directories, FS stats,
>>>>     Connection Stats.
>>>>      >>>>> logging.
>>>>      >>>>>
>>>>      >>>>> My idea to progress is to take those main bulletpoint in
>>> ceph
>>>>     protocol
>>>>      >>>>> based
>>>>      >>>>> on general ideas of what ceph file system does and start
>>>>     identifying
>>>>      >>>>> parts
>>>>      >>>>> from libcephfs to match those "needs".
>>>>      >>>>
>>>>      >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>>>     interface that
>>>>      >>>> libcephfs provides, and try to map that to what the fuse
>>> layer
>>>>     expects.
>>>>      >>>> There is both a path-based that I suspsect lends itself
>>> well
>>>>     to the
>>>>      >>>> Windows interface and (very soon now) a handle based API
>>> that is
>>>>      >>>> targetted
>>>>      >>>> at the Unix-style VFS layers.  I'm mostly guessing,
>>> though,
>>>>     since I've
>>>>      >>>> never seen any low-level fs code in windows before.
>>>>      >>>>
>>>>      >>>> In this case, the analogous code for Linux should be
>>>>     client/fuse_ll.cc
>>>>      >>>> itself (and not much else), although there will probably be
>>> a
>>>>     few tricks
>>>>      >>>> necessary to map cleanly onto how the windows interfaces
>>> work.
>>>>      >>>>
>>>>      >>>> Does that make sense?
>>>>      >>>>
>>>>      >>>> Cheers!
>>>>      >>>> sage
>>>>      >>>>
>>>>      >>>>
>>>>      >>>>> Any suggestion and contributions are welcome.
>>>>      >>>>>
>>>>      >>>>>
>>>>      >>>>> *
>>>>      >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>>      >>>>>>
>>>>      >>>>>> Hi Alphe,
>>>>      >>>>>>
>>>>      >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>>      >>>>>>>
>>>>      >>>>>>> Good day developers!
>>>>      >>>>>>>
>>>>      >>>>>>> I would like to propose to the one interested  work with
>>> me to
>>>>      >>>>>>> develop a
>>>>      >>>>>>> ceph
>>>>      >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>      >>>>>>>
>>>>      >>>>>>> My company is a ceph enthousiast that use on a dayly
>>> basis
>>>>     ceph and
>>>>      >>>>>>> that
>>>>      >>>>>>> need
>>>>      >>>>>>> both transfer speed and big expendable and cheap
>>> storage.
>>>>      >>>>>>> My company is specialised in data recovery and we want
>>> to
>>>>     participate
>>>>      >>>>>>> to
>>>>      >>>>>>> ceph
>>>>      >>>>>>> effort by bringing a ceph cliente for windows.
>>>>      >>>>>>
>>>>      >>>>>> Awesome!
>>>>      >>>>>>
>>>>      >>>>>>> Our experience shows us that the best gateway is each
>>>>     clientes being
>>>>      >>>>>>> its
>>>>      >>>>>>> own
>>>>      >>>>>>> gateway, instead of having a bottle neck server or a
>>> cluster of
>>>>      >>>>>>> bottle
>>>>      >>>>>>> neck
>>>>      >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>>> etc..).
>>>>      >>>>>>>
>>>>      >>>>>>> We already did some research in that domain.
>>>>      >>>>>>>
>>>>      >>>>>>> Dokan FS is an intent  to write an opensource fuse like
>>>>     cliente for
>>>>      >>>>>>> MS
>>>>      >>>>>>> windows.
>>>>      >>>>>>>
>>>>      >>>>>>> More information on DOKANFS can be triggered here
>>>> >>>>>>>http://dokan-dev.net/en/download/
>>>>      >>>>>>>
>>>>      >>>>>>> Positive points of using DOKANFS.
>>>>      >>>>>>>
>>>>      >>>>>>> - its opensourced and well licenced mit licence, gpl
>>>>     licence and lgpl
>>>>      >>>>>>> licence.
>>>>      >>>>>>>
>>>>      >>>>>>> Negative point of using DOKAN FS.
>>>>      >>>>>>> - unreachable author
>>>>      >>>>>>> - Poor documentation . Dev comments in japanese.
>>>>      >>>>>>> - Work in progress so it is unstable and needs to be
>>> updated,
>>>>      >>>>>>> debugged and
>>>>      >>>>>>> maintained by a MS Windows file system expert
>>> developper.
>>>>      >>>>>>
>>>>      >>>>>> I am not very familiar with windows storage APIs, but
>>>>     somebody told me
>>>>      >>>>>> at once point there were several interfaces against which
>>> a
>>>>     new file
>>>>      >>>>>> system could be implemented, everything from a full
>>>>     in-kernel driver
>>>>      >>>>>> to
>>>>      >>>>>> something that is explorer-based.  Are any of those
>>>>     suitable?  Using a
>>>>      >>>>>> potentially abandoned fuse-like layer makes me a bit
>>> nervous.
>>>>      >>>>>>
>>>>      >>>>>> That said,
>>>>      >>>>>>
>>>>      >>>>>>>
>>>>      >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>>      >>>>>>> here are what I learnt.
>>>>      >>>>>>> - Ceph-fuse and related source code is around 60 000
>>> lines
>>>>     of code.
>>>>      >>>>>>> - Ceph protocol isn t documented so it is like trying
>>> to
>>>>     draw a map
>>>>      >>>>>>> of
>>>>      >>>>>>> america
>>>>      >>>>>>> using only a sextan and a compass.
>>>>      >>>>>>>
>>>>      >>>>>>> Those led me to those conclusions:
>>>>      >>>>>>> - I can t do it alone.
>>>>      >>>>>>> - It is easier to draw down the ceph protocol way to
>>> work from
>>>>      >>>>>>> kernel/fs/ceph
>>>>      >>>>>>> sources and mount.ceph
>>>>      >>>>>>> - Ceph depending libraries may be unexistant or not up
>>> to
>>>>     date in
>>>>      >>>>>>> their
>>>>      >>>>>>> port
>>>>      >>>>>>> on MS Windows (cygwin)
>>>>      >>>>>>
>>>>      >>>>>> I think the most sane path should be to make libcephfs
>>>>     sufficiently
>>>>      >>>>>> portable to build on windows (or cygwin).  For the bits
>>> used
>>>>     by the
>>>>      >>>>>> client-side coe, I don't think there should be much in
>>> the
>>>>     way of
>>>>      >>>>>> dependencies, and the main challenge would be untangling
>>> the
>>>>     build for
>>>>      >>>>>> the necessary pieces out from the rest of Ceph.
>>>>      >>>>>>
>>>>      >>>>>> Have you seen the wip-port portability work that is
>>>>     currently underway
>>>>      >>>>>> by
>>>>      >>>>>> Noah and Alan?  That may solve many of the cygwin
>>> problems
>>>>     you are
>>>>      >>>>>> seeing
>>>>      >>>>>> today.
>>>>      >>>>>>
>>>>      >>>>>>> - MS file system specialist are hard do find in the
>>> "open
>>>>     source
>>>>      >>>>>>> libre
>>>>      >>>>>>> world"
>>>>      >>>>>>> so I will try in the commercial world.
>>>>      >>>>>>>
>>>>      >>>>>>> The commercial world has some problems too. They need
>>> ceph
>>>>     protocol
>>>>      >>>>>>> draft
>>>>      >>>>>>> to
>>>>      >>>>>>> implemente it to their own product They will have
>>> licencing
>>>>      >>>>>>> /commercial
>>>>      >>>>>>> politics that infringe lgpl, and hide that most of the
>>> work
>>>>     is done
>>>>      >>>>>>> by
>>>>      >>>>>>> people
>>>>      >>>>>>> other than them. They will not participate in a
>>> financial
>>>>     way to ceph
>>>>      >>>>>>> enhancement and growth.
>>>>      >>>>>>
>>>>      >>>>>> I don't think reimplementing the client code is an
>>> efficient way
>>>>      >>>>>> forward.
>>>>      >>>>>> Unless the goal is a pure kernel implementation...but a
>>>>     significant
>>>>      >>>>>> ongoing investment in development resources would be
>>> needed
>>>>     for that
>>>>      >>>>>> going
>>>>      >>>>>> forward.  I suspect that is a challenge for a platform
>>> that
>>>>     does not
>>>>      >>>>>> typically rally that sort of community effort.
>>>>      >>>>>>
>>>>      >>>>>> The easiest thing is of course just to use CIFS and
>>> Samba
>>>>     (which works
>>>>      >>>>>> today).  A fuse-like approach is probably a reasonably
>>>>     middle ground
>>>>      >>>>>> (both
>>>>      >>>>>> in initial effort and maintainability going forward)...
>>>>      >>>>>>
>>>>      >>>>>> sage
>>>>      >>>>>>
>>>>      >>>>>>
>>>>      >>>>>
>>>>      >>>
>>>>      >>> --
>>>>      >>> To unsubscribe from this list: send the line "unsubscribe
>>>>     ceph-devel" in
>>>>      >>> the body of a message tomajordomo@vger.kernel.org
>>>>     <mailto:majordomo@vger.kernel.org>
>>>>
>>>>      >>> More majordomo info at
>>> http://vger.kernel.org/majordomo-info.html
>>>>      >>
>>>>      >>
>>>>      >
>>>>
>>>>
>>>>
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>> in
>>> the body of a message tomajordomo@vger.kernel.org
>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Malcolm Haak]

I'm just going to throw these in there.

http://www.acc.umu.se/~bosse/

They are GPLv2 some already use sockets and such from inside the kernel. 
  Heck you might even be able to mod the HTTP one to use rados gateway. 
I don't know as I havent sat down and pulled them apart enough yet.

They might help, but they might be useless. Not sure.

On 08/11/13 06:47, Alphe Salas Michels wrote:
> Hello all I finally finished my first source code extraction that starts
> from ceph/src/client/fuse_ll.c
> The result is accurate unlike previous provided results. basically the
> script start from a file extract all the private includes definitions
> #include "something.h" and recursively extract private includes too. the
> best way to know who is related with who.
>
> starting from fuse_ll.cc I optain 390 files retreived and 120 000 lines
> of code !
> involved dirs are : in ceph/src
> objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
> global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>
> probably not a good way to analyse what amount of work it means since
> most of those directories are the implementation of servers (osd, mon,
> mds) and even if only a tiny bit of them is needed at client level. you
> need two structures from ./osd/OSD.h and  my script by relation will
> take into acount the whole directory...
>
> I ran the script with libcephfs.cc as start point and got almost the
> same results. 131 000 lines of code and 386 files most of the same dirs
> involved.
>
>
>
> I think I will spend alot of time doing the manual source code isolation
> and understand way each #include is set in the files I read (what
> purpose they have do they allow to integrate a crucial data type or not.
>
>
> The other way around will be to read src/libcephfs.cc. It seems shorter
> but without understanding what part is used for each included header I
> can t say anything...
>
>
>
> I will keep reading the source code and take notes. I think in the case
> of libcephfs I will gain alot of time.
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl
> *www.kepler.cl <http://www.kepler.cl>*
>
> On 11/07/13 15:02, Alphe Salas Michels wrote:
>> Hello D.Ketor and Matt Benjamin,
>> You give me alot to think about and this is great!
>> I merged your previous post to make a single reply that anyone can
>> report to easyly
>>
>> Windows NFS 4.1 is available here:
>> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>
>> pnfs is another name for NFS4.X. It is presented as alternative to
>> ceph and we get known terminology as MDS and OSD but without the self
>> healing part if I understand well my rapid look on the topic. (when I
>> say rapid look I mean ... 5 minutes spent in that... which is really
>> small amount of time to get an accurate view on something)
>>
>>
>> starting from mount.ceph ... I know that mount.ceph does little but it
>> is a great hint to know what ceph needs and do things.
>> Basically mount.ceph modprobe the ceph driver in the linux kernel then
>> call mount with the line command passed args and the cephfs type as
>> argument. Then the kernel does the work I don t understand yet what is
>> the start calls that are made to the ceph driver but it seemed to me
>> that is was relatively light. (a first impression compared to ceph-fuse.)
>>
>> I think I will do both isolate source code from ceph-client kernel
>> (cephfs module for linux kernel) and the one pointed by Sage starting
>> from client/fuse_ll.cc in ceph master branch. The common files betwin
>> those 2 extractions will be our core set of mandatory features.
>>
>> Then we try to compile with cygwin a cephfs client library . Then we
>> will try to interface with a modified windows nfs 4.1 client or pnfs
>> or any other that will accept to be compiled with gcc for win32...
>>
>> the fact that windows 8.1 is and windows 2012 are out of reach at the
>> moment is not a problem to me.
>>
>> Our first concern is to understand what is ceph protocol. Then adapt
>> it to something that can be used on windows prior windows 8.1. Dokan
>> fs if I remember well use too the WDK (windows driver dev-kit ) for it
>> s compilation so possibly we will see the same limitations.
>>
>> We need to multiply our source of information by example regarding
>> ceph-client (kernel or fuse, radosgw is on a different layer so I will
>> not try anything around it at first.) And we need to multiply our
>> source of information by example regarding virtual file system
>> technologies on windoes OS.
>> Alot of work but all of those available source code everyone point at
>> me will make our best solution. And in the end we will choose
>> technologies knowing what we do and what concequencies they have.
>>
>> regards,
>>
>>
>>
>>
>> Regards
>>
>> signature
>>
>> *Alphé Salas*
>> Ingeniero T.I
>>
>> asalas@kepler.cl
>>
>>
>> On 11/07/13 11:29, Ketor D wrote:
>>> Hi Alphe:
>>>        Yes Callback Filesystem is very expensive and can't open source.
>>> It's not a good choice for ceph4win.
>>>        Another way for ceph4win maybe develop a kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>> src code and maybe migrating from ceph kernel client to windows kernel
>>> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
>>> client has greater performance than userspace fs like ceph-fuse client
>>> and ceph kernel client.
>>>
>>>        Regards.
>>>
>> On 11/07/13 11:50, Matt W. Benjamin wrote:
>>> Hi,
>>>
>>> The Window NFS v4.1 client is what we work on, so this may be good for
>>> code sharing.  The license is lgplv2, like Ceph's.
>>>
>>> Something important to be aware of is that the client uses rdbss, which
>>> is a (partial) fsd abstraction that simplified implementation
>>> quite a bit, kind of like a mini driver.  However, Microsoft's support
>>> for rdbss has been in limbo for a bit.  For example, to link with
>>> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
>>> to use the one for Windows 7.  (There's a private rdbss2 used internally
>>> by Microsoft's SMB implemenation.  A the moment, 3rd party drivers
>>> can't use that.)
>>>
>>> We've been in communication with Microsoft about this issue, and know of
>>> a few other fsds using it, but it could be a good thing for that
>>> lobbying
>>> effort to have another user--or it could be a dead end :(.
>>>
>>> There are a couple of other choices if you're looking to go this route,
>>> that I'm aware of (and we may need to take them too, if RDBSS has no
>>> way forward), but the required work could be a lot larger.
>>>
>>> Matt
>>>
>>> ----- "Ketor D"<d.ketor@gmail.com>  wrote:
>>>
>>>> Hi Alphe:
>>>>        Yes Callback Filesystem is very expensive and can't open
>>>> source.
>>>> It's not a good choice for ceph4win.
>>>>        Another way for ceph4win maybe develop a kernel-mode fs like
>>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>>> src code and maybe migrating from ceph kernel client to windows
>>>> kernel
>>>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>>>> fs
>>>> client has greater performance than userspace fs like ceph-fuse
>>>> client
>>>> and ceph kernel client.
>>>>
>>>>        Regards.
>>>>
>>>>
>>>>
>>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>>>> wrote:
>>>>> Commercial libraries are a pain ...
>>>>>
>>>>> If we want the more permossive licence offered by callback file
>>>> system we
>>>>> have to buy it for 20.000 usd. Then we will have to provide a
>>>> backbox that
>>>>> we have no control upon and that will kill our product anytime they
>>>> want anf
>>>>> if they decide to stop their commercial activity we will be in the
>>>> same
>>>>> situation that with dokanfs but without having the source code of
>>>> the black
>>>>> box. If i have to spend 20 000 dollars i would prefere paying
>>>> someone to
>>>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>>>> make it
>>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>>>
>>>>> I would prefere if people have to pay something to get access to
>>>> ceph4win
>>>>> that this money goes in ceph main branch pockets... Or as a gift you
>>>> donante
>>>>> to ceph 10 dollars  you get 2 free registration codes for
>>>> ceph4win... or
>>>>> something like that.
>>>>>
>>>>> If ceph4win as to be comercial then I would prefer delegate the task
>>>> to a
>>>>> company like south river technologies and their great product
>>>> webdrive. I
>>>>> would mininaly get involved in that project and simply buy the final
>>>> product
>>>>> to sell it together with my ceph based product (which could be a
>>>> calxeda
>>>>> ceph box or something like that).
>>>>>
>>>>> I m open anyway to any proposition. But I doubt that callback
>>>> filesystem
>>>>> offers us a suitable solution in the way I see ceph4win to be spread
>>>> and
>>>>> used... I m maybe wrong. And anything that will be done around
>>>> ceph4win will
>>>>> be public documented etc... And licensed the way that if someone
>>>> want to
>>>>> build a commercial solution on top of it, that would be a
>>>> possibility.
>>>>> My idea is to giveback somehow to ceph project and at same time
>>>> forge a
>>>>> better knowledge in ceph technologies. Because like many in libre
>>>> world I
>>>>> think the business is in the services around the software more than
>>>> on the
>>>>> software. That the ones writing code should be financed and benefits
>>>> from
>>>>> the one selling and giving support of the software at all levels. I
>>>> m
>>>>> probably too idealistic. And too optimistic after all I m the one
>>>> saying I
>>>>> will do this stuff I have no idea how but well it is interesting and
>>>> fun so
>>>>> lets do it.
>>>>>
>>>>> Regards,
>>>>>
>>>>> P.S: using commercial backend libraries appart including their own
>>>> cost will
>>>>> force you to use commercial IDE like MS VisualStudio because their
>>>> library
>>>>> has some kind of drm that only that IDE compiler can use. So alot of
>>>> cost
>>>>> and yet there is nothing done. If I had to open a kickstarter
>>>> project saying
>>>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>>>> right to
>>>>> use and share a commercial copyrighted library but abandonned
>>>> punctually to
>>>>> us in  public domaine and that we will eventually produce something
>>>> out of
>>>>> it. I doubt I will get a dollar.
>>>>>
>>>>> We still can suggest the idea to Edlos the commercial company that
>>>> has the
>>>>> copyright of Callback FS, Or to buy them their product in a blender
>>>> way
>>>>> (blender was bought with donation before being put opensource and
>>>> public
>>>>> domaine), Or to open source their library. But in commercial minds
>>>>> opensourcing = death of their technical advantage and death of
>>>> their
>>>>> marketing strategy. They will have to invent something more to
>>>> retrieve
>>>>> monney from it.
>>>>>
>>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>>
>>>>>
>>>>>     Hi Alphe,
>>>>>               I think you could try Callback Filesystem dev
>>>> framework. It
>>>>>     is a commerical dev framework and is maintained by Edlos today.
>>>>>               I have communicated with Edlos to get a try code for
>>>>>     development. To dokan, Callback Filesystem has vary document and
>>>> maybe
>>>>>     more stabilize.
>>>>>
>>>>>               Regards.
>>>>>
>>>>>
>>>>>
>>>>>     On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>>>>     <mailto:asalas@kepler.cl>> wrote:
>>>>>      > Hello ketor thank you for your interest un ceph4win. Since
>>>> muy
>>>>>     first mail I
>>>>>      > exposed the lacks of dokanfs and that I m far from being a
>>>>>     specialist un
>>>>>      > filesystems.
>>>>>      > I exposed what i like un dokanfs bit I not a fanátic of it.
>>>> Muy
>>>>>     goal is to
>>>>>      > have something working quickly.
>>>>>      >
>>>>>      > So I am up to any proposición sure the one with the more docs
>>>> and
>>>>>     support
>>>>>      > will be the best choice. As for right now what I need is
>>>>>     understand what are
>>>>>      > the files involved what are the interfaces functions and what
>>>> are
>>>>>     the needed
>>>>>      > library dependencies and if they exist ported to windows with
>>>>>     cygwin. And
>>>>>      > all that is retrieved from source code.
>>>>>      >
>>>>>      > Regards.
>>>>>      >
>>>>>      > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>>>>     <mailto:d.ketor@gmail.com>> escribió:
>>>>>
>>>>>      >
>>>>>      >> Hi Alphe,
>>>>>      >>      We are taking an interest in your work on Ceph Client
>>>> for
>>>>>     Windows
>>>>>      >> with Dokan.As we know, the performance of Dokan is not very
>>>>>     good, and it's
>>>>>      >> abandoned 3 years ago.
>>>>>      >>      I have learned and used OpenDedup(SDFS) for a long
>>>> time.
>>>>>       OpenDedup
>>>>>      >> has a Dokan version. And the author of OpenDedup said
>>>>>      >>
>>>>>      >> The Dokan library is quite flakey  and testing should be
>>>>>     performed before
>>>>>      >> putting into production
>>>>>      >>
>>>>>      >>       So what do you think about this? And if there is
>>>> another
>>>>>     solution of
>>>>>      >> fuse-like filesystem dev framwork on Windows?
>>>>>      >>
>>>>>      >>        Best Wish!
>>>>>      >>
>>>>>      >>
>>>>>      >>
>>>>>      >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>>>>     <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>>>
>>>>>      >> wrote:
>>>>>      >>>
>>>>>      >>> Hello I created the github repository for this project
>>>>>      >>>https://github.com/alphe/Ceph4Win
>>>>>      >>>
>>>>>      >>> Regards,
>>>>>      >>>
>>>>>      >>> signature
>>>>>      >>>
>>>>>      >>> *Alphé Salas*
>>>>>      >>> Ingeniero T.I
>>>>>      >>>
>>>>>      >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>>>
>>>>>      >>> *<http://www.kepler.cl>*
>>>>>      >>>
>>>>>      >>> On 11/05/13 21:00, Sage Weil wrote:
>>>>>      >>>>
>>>>>      >>>> Hi Alphe,
>>>>>      >>>>
>>>>>      >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>>>      >>>>>
>>>>>      >>>>> signature *Hi, Sage !
>>>>>      >>>>> thank you for you enthousiast reply.
>>>>>      >>>>> I sure want to make the best use of everything or
>>>> anything
>>>>>     previously
>>>>>      >>>>> done to
>>>>>      >>>>> tend to
>>>>>      >>>>> write ceph cliente for windows.
>>>>>      >>>>>
>>>>>      >>>>> Apart using libre tools for building the future ceph
>>>> cliente
>>>>>     I am open
>>>>>      >>>>> to
>>>>>      >>>>> anything.
>>>>>      >>>>> I would recommand eclipse CDT or Code::BLocks they are
>>>> based
>>>>>     on mingwin
>>>>>      >>>>> open
>>>>>      >>>>> and easyly enhanceable.**
>>>>>      >>>>>
>>>>>      >>>>> more free tools can be found here:
>>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>>>      >>>>>
>>>>>      >>>>>
>>>>>      >>>>> I will read libcephfs source code and take some notes
>>>> about the
>>>>>      >>>>> protocol.
>>>>>      >>>>
>>>>>      >>>> I think you don't need to worry about hte protocol at all,
>>>> since
>>>>>      >>>> libcephs
>>>>>      >>>> implements it for you (and will capture any future
>>>> changes).
>>>>>      >>>>
>>>>>      >>>>> I was more going from what I know and trying to track down
>>>> how
>>>>>      >>>>> mount.ceph work
>>>>>      >>>>> with the parameters passed to it.
>>>>>      >>>>> since it point finally to Kernel/fs/ceph and that I don t
>>>> really
>>>>>      >>>>> understand
>>>>>      >>>>> how that module work and that it probably points to some
>>>> other
>>>>>      >>>>> dependencies
>>>>>      >>>>> Reading libcephfs source code could be a big gain of
>>>> time.
>>>>>      >>>>
>>>>>      >>>> (I would also ignore mount.ceph as everything it does it
>>>>>     specific to
>>>>>      >>>> how Linux mounts work.)
>>>>>      >>>>
>>>>>      >>>>> basically on the protocol what is need are:
>>>>>      >>>>>
>>>>>      >>>>> 1) open and maintain a connection (socket open, auth, etc
>>>> )
>>>>>      >>>>> 2) retreive a map of directories and disk Quota (disk
>>>> sizing
>>>>>     Y TB free,
>>>>>      >>>>> Z TB
>>>>>      >>>>> total)
>>>>>      >>>>> 3) procedure to send files / directories in a maner that
>>>> it
>>>>>     will allow
>>>>>      >>>>> our
>>>>>      >>>>> client to fit ceph transmission protocols
>>>>>      >>>>> (limit bandwith for stability?, limit connection amount?,
>>>>>     limit cpu
>>>>>      >>>>> use?,
>>>>>      >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>>>      >>>>> 4)Procedure to retreive files / directory from ceph
>>>> cluster
>>>>>      >>>>> 5) Management copy/move files /Directories, FS stats,
>>>>>     Connection Stats.
>>>>>      >>>>> logging.
>>>>>      >>>>>
>>>>>      >>>>> My idea to progress is to take those main bulletpoint in
>>>> ceph
>>>>>     protocol
>>>>>      >>>>> based
>>>>>      >>>>> on general ideas of what ceph file system does and start
>>>>>     identifying
>>>>>      >>>>> parts
>>>>>      >>>>> from libcephfs to match those "needs".
>>>>>      >>>>
>>>>>      >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>>>>     interface that
>>>>>      >>>> libcephfs provides, and try to map that to what the fuse
>>>> layer
>>>>>     expects.
>>>>>      >>>> There is both a path-based that I suspsect lends itself
>>>> well
>>>>>     to the
>>>>>      >>>> Windows interface and (very soon now) a handle based API
>>>> that is
>>>>>      >>>> targetted
>>>>>      >>>> at the Unix-style VFS layers.  I'm mostly guessing,
>>>> though,
>>>>>     since I've
>>>>>      >>>> never seen any low-level fs code in windows before.
>>>>>      >>>>
>>>>>      >>>> In this case, the analogous code for Linux should be
>>>>>     client/fuse_ll.cc
>>>>>      >>>> itself (and not much else), although there will probably be
>>>> a
>>>>>     few tricks
>>>>>      >>>> necessary to map cleanly onto how the windows interfaces
>>>> work.
>>>>>      >>>>
>>>>>      >>>> Does that make sense?
>>>>>      >>>>
>>>>>      >>>> Cheers!
>>>>>      >>>> sage
>>>>>      >>>>
>>>>>      >>>>
>>>>>      >>>>> Any suggestion and contributions are welcome.
>>>>>      >>>>>
>>>>>      >>>>>
>>>>>      >>>>> *
>>>>>      >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>>>      >>>>>>
>>>>>      >>>>>> Hi Alphe,
>>>>>      >>>>>>
>>>>>      >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>>>      >>>>>>>
>>>>>      >>>>>>> Good day developers!
>>>>>      >>>>>>>
>>>>>      >>>>>>> I would like to propose to the one interested  work with
>>>> me to
>>>>>      >>>>>>> develop a
>>>>>      >>>>>>> ceph
>>>>>      >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>>      >>>>>>>
>>>>>      >>>>>>> My company is a ceph enthousiast that use on a dayly
>>>> basis
>>>>>     ceph and
>>>>>      >>>>>>> that
>>>>>      >>>>>>> need
>>>>>      >>>>>>> both transfer speed and big expendable and cheap
>>>> storage.
>>>>>      >>>>>>> My company is specialised in data recovery and we want
>>>> to
>>>>>     participate
>>>>>      >>>>>>> to
>>>>>      >>>>>>> ceph
>>>>>      >>>>>>> effort by bringing a ceph cliente for windows.
>>>>>      >>>>>>
>>>>>      >>>>>> Awesome!
>>>>>      >>>>>>
>>>>>      >>>>>>> Our experience shows us that the best gateway is each
>>>>>     clientes being
>>>>>      >>>>>>> its
>>>>>      >>>>>>> own
>>>>>      >>>>>>> gateway, instead of having a bottle neck server or a
>>>> cluster of
>>>>>      >>>>>>> bottle
>>>>>      >>>>>>> neck
>>>>>      >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>>>> etc..).
>>>>>      >>>>>>>
>>>>>      >>>>>>> We already did some research in that domain.
>>>>>      >>>>>>>
>>>>>      >>>>>>> Dokan FS is an intent  to write an opensource fuse like
>>>>>     cliente for
>>>>>      >>>>>>> MS
>>>>>      >>>>>>> windows.
>>>>>      >>>>>>>
>>>>>      >>>>>>> More information on DOKANFS can be triggered here
>>>>> >>>>>>>http://dokan-dev.net/en/download/
>>>>>      >>>>>>>
>>>>>      >>>>>>> Positive points of using DOKANFS.
>>>>>      >>>>>>>
>>>>>      >>>>>>> - its opensourced and well licenced mit licence, gpl
>>>>>     licence and lgpl
>>>>>      >>>>>>> licence.
>>>>>      >>>>>>>
>>>>>      >>>>>>> Negative point of using DOKAN FS.
>>>>>      >>>>>>> - unreachable author
>>>>>      >>>>>>> - Poor documentation . Dev comments in japanese.
>>>>>      >>>>>>> - Work in progress so it is unstable and needs to be
>>>> updated,
>>>>>      >>>>>>> debugged and
>>>>>      >>>>>>> maintained by a MS Windows file system expert
>>>> developper.
>>>>>      >>>>>>
>>>>>      >>>>>> I am not very familiar with windows storage APIs, but
>>>>>     somebody told me
>>>>>      >>>>>> at once point there were several interfaces against which
>>>> a
>>>>>     new file
>>>>>      >>>>>> system could be implemented, everything from a full
>>>>>     in-kernel driver
>>>>>      >>>>>> to
>>>>>      >>>>>> something that is explorer-based.  Are any of those
>>>>>     suitable?  Using a
>>>>>      >>>>>> potentially abandoned fuse-like layer makes me a bit
>>>> nervous.
>>>>>      >>>>>>
>>>>>      >>>>>> That said,
>>>>>      >>>>>>
>>>>>      >>>>>>>
>>>>>      >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>>>      >>>>>>> here are what I learnt.
>>>>>      >>>>>>> - Ceph-fuse and related source code is around 60 000
>>>> lines
>>>>>     of code.
>>>>>      >>>>>>> - Ceph protocol isn t documented so it is like trying
>>>> to
>>>>>     draw a map
>>>>>      >>>>>>> of
>>>>>      >>>>>>> america
>>>>>      >>>>>>> using only a sextan and a compass.
>>>>>      >>>>>>>
>>>>>      >>>>>>> Those led me to those conclusions:
>>>>>      >>>>>>> - I can t do it alone.
>>>>>      >>>>>>> - It is easier to draw down the ceph protocol way to
>>>> work from
>>>>>      >>>>>>> kernel/fs/ceph
>>>>>      >>>>>>> sources and mount.ceph
>>>>>      >>>>>>> - Ceph depending libraries may be unexistant or not up
>>>> to
>>>>>     date in
>>>>>      >>>>>>> their
>>>>>      >>>>>>> port
>>>>>      >>>>>>> on MS Windows (cygwin)
>>>>>      >>>>>>
>>>>>      >>>>>> I think the most sane path should be to make libcephfs
>>>>>     sufficiently
>>>>>      >>>>>> portable to build on windows (or cygwin).  For the bits
>>>> used
>>>>>     by the
>>>>>      >>>>>> client-side coe, I don't think there should be much in
>>>> the
>>>>>     way of
>>>>>      >>>>>> dependencies, and the main challenge would be untangling
>>>> the
>>>>>     build for
>>>>>      >>>>>> the necessary pieces out from the rest of Ceph.
>>>>>      >>>>>>
>>>>>      >>>>>> Have you seen the wip-port portability work that is
>>>>>     currently underway
>>>>>      >>>>>> by
>>>>>      >>>>>> Noah and Alan?  That may solve many of the cygwin
>>>> problems
>>>>>     you are
>>>>>      >>>>>> seeing
>>>>>      >>>>>> today.
>>>>>      >>>>>>
>>>>>      >>>>>>> - MS file system specialist are hard do find in the
>>>> "open
>>>>>     source
>>>>>      >>>>>>> libre
>>>>>      >>>>>>> world"
>>>>>      >>>>>>> so I will try in the commercial world.
>>>>>      >>>>>>>
>>>>>      >>>>>>> The commercial world has some problems too. They need
>>>> ceph
>>>>>     protocol
>>>>>      >>>>>>> draft
>>>>>      >>>>>>> to
>>>>>      >>>>>>> implemente it to their own product They will have
>>>> licencing
>>>>>      >>>>>>> /commercial
>>>>>      >>>>>>> politics that infringe lgpl, and hide that most of the
>>>> work
>>>>>     is done
>>>>>      >>>>>>> by
>>>>>      >>>>>>> people
>>>>>      >>>>>>> other than them. They will not participate in a
>>>> financial
>>>>>     way to ceph
>>>>>      >>>>>>> enhancement and growth.
>>>>>      >>>>>>
>>>>>      >>>>>> I don't think reimplementing the client code is an
>>>> efficient way
>>>>>      >>>>>> forward.
>>>>>      >>>>>> Unless the goal is a pure kernel implementation...but a
>>>>>     significant
>>>>>      >>>>>> ongoing investment in development resources would be
>>>> needed
>>>>>     for that
>>>>>      >>>>>> going
>>>>>      >>>>>> forward.  I suspect that is a challenge for a platform
>>>> that
>>>>>     does not
>>>>>      >>>>>> typically rally that sort of community effort.
>>>>>      >>>>>>
>>>>>      >>>>>> The easiest thing is of course just to use CIFS and
>>>> Samba
>>>>>     (which works
>>>>>      >>>>>> today).  A fuse-like approach is probably a reasonably
>>>>>     middle ground
>>>>>      >>>>>> (both
>>>>>      >>>>>> in initial effort and maintainability going forward)...
>>>>>      >>>>>>
>>>>>      >>>>>> sage
>>>>>      >>>>>>
>>>>>      >>>>>>
>>>>>      >>>>>
>>>>>      >>>
>>>>>      >>> --
>>>>>      >>> To unsubscribe from this list: send the line "unsubscribe
>>>>>     ceph-devel" in
>>>>>      >>> the body of a message tomajordomo@vger.kernel.org
>>>>>     <mailto:majordomo@vger.kernel.org>
>>>>>
>>>>>      >>> More majordomo info at
>>>> http://vger.kernel.org/majordomo-info.html
>>>>>      >>
>>>>>      >>
>>>>>      >
>>>>>
>>>>>
>>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>>> in
>>>> the body of a message tomajordomo@vger.kernel.org
>>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Matt W. Benjamin]

They certainly could be.  The OpenAFS kdfs driver is particularly complete.
I've built it and worked on it a bit in earlier versions than those shipping
today.

It's a hybrid driver.  Networking is actually in userspace (that's true of the
the NFSv4 client too).  The two differ greatly in architecture, however, and
the AFS driver has complex licensing.  You should be able to use the kernel
components by Kernel Drivers/Secure Endpoints (YFS?), but the legacy AFS code
is not GPL compatible.

(Can't speak for the others.)

Matt

----- "Malcolm Haak" <malcolm@sgi.com> wrote:

> I'm just going to throw these in there.
> 
> http://www.acc.umu.se/~bosse/
> 
> They are GPLv2 some already use sockets and such from inside the
> kernel. 
>   Heck you might even be able to mod the HTTP one to use rados
> gateway. 
> I don't know as I havent sat down and pulled them apart enough yet.
> 
> They might help, but they might be useless. Not sure.
> 
> On 08/11/13 06:47, Alphe Salas Michels wrote:
> > Hello all I finally finished my first source code extraction that
> starts
> > from ceph/src/client/fuse_ll.c
> > The result is accurate unlike previous provided results. basically
> the
> > script start from a file extract all the private includes
> definitions
> > #include "something.h" and recursively extract private includes too.
> the
> > best way to know who is related with who.
> >
> > starting from fuse_ll.cc I optain 390 files retreived and 120 000
> lines
> > of code !
> > involved dirs are : in ceph/src
> > objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
> > global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
> >
> > probably not a good way to analyse what amount of work it means
> since
> > most of those directories are the implementation of servers (osd,
> mon,
> > mds) and even if only a tiny bit of them is needed at client level.
> you
> > need two structures from ./osd/OSD.h and  my script by relation
> will
> > take into acount the whole directory...
> >
> > I ran the script with libcephfs.cc as start point and got almost
> the
> > same results. 131 000 lines of code and 386 files most of the same
> dirs
> > involved.
> >
> >
> >
> > I think I will spend alot of time doing the manual source code
> isolation
> > and understand way each #include is set in the files I read (what
> > purpose they have do they allow to integrate a crucial data type or
> not.
> >
> >
> > The other way around will be to read src/libcephfs.cc. It seems
> shorter
> > but without understanding what part is used for each included header
> I
> > can t say anything...
> >
> >
> >
> > I will keep reading the source code and take notes. I think in the
> case
> > of libcephfs I will gain alot of time.
> >
> > signature
> >
> > *Alphé Salas*
> > Ingeniero T.I
> >
> > asalas@kepler.cl
> > *www.kepler.cl <http://www.kepler.cl>*
> >
> > On 11/07/13 15:02, Alphe Salas Michels wrote:
> >> Hello D.Ketor and Matt Benjamin,
> >> You give me alot to think about and this is great!
> >> I merged your previous post to make a single reply that anyone can
> >> report to easyly
> >>
> >> Windows NFS 4.1 is available here:
> >> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
> >>
> >> pnfs is another name for NFS4.X. It is presented as alternative to
> >> ceph and we get known terminology as MDS and OSD but without the
> self
> >> healing part if I understand well my rapid look on the topic. (when
> I
> >> say rapid look I mean ... 5 minutes spent in that... which is
> really
> >> small amount of time to get an accurate view on something)
> >>
> >>
> >> starting from mount.ceph ... I know that mount.ceph does little but
> it
> >> is a great hint to know what ceph needs and do things.
> >> Basically mount.ceph modprobe the ceph driver in the linux kernel
> then
> >> call mount with the line command passed args and the cephfs type
> as
> >> argument. Then the kernel does the work I don t understand yet what
> is
> >> the start calls that are made to the ceph driver but it seemed to
> me
> >> that is was relatively light. (a first impression compared to
> ceph-fuse.)
> >>
> >> I think I will do both isolate source code from ceph-client kernel
> >> (cephfs module for linux kernel) and the one pointed by Sage
> starting
> >> from client/fuse_ll.cc in ceph master branch. The common files
> betwin
> >> those 2 extractions will be our core set of mandatory features.
> >>
> >> Then we try to compile with cygwin a cephfs client library . Then
> we
> >> will try to interface with a modified windows nfs 4.1 client or
> pnfs
> >> or any other that will accept to be compiled with gcc for win32...
> >>
> >> the fact that windows 8.1 is and windows 2012 are out of reach at
> the
> >> moment is not a problem to me.
> >>
> >> Our first concern is to understand what is ceph protocol. Then
> adapt
> >> it to something that can be used on windows prior windows 8.1.
> Dokan
> >> fs if I remember well use too the WDK (windows driver dev-kit ) for
> it
> >> s compilation so possibly we will see the same limitations.
> >>
> >> We need to multiply our source of information by example regarding
> >> ceph-client (kernel or fuse, radosgw is on a different layer so I
> will
> >> not try anything around it at first.) And we need to multiply our
> >> source of information by example regarding virtual file system
> >> technologies on windoes OS.
> >> Alot of work but all of those available source code everyone point
> at
> >> me will make our best solution. And in the end we will choose
> >> technologies knowing what we do and what concequencies they have.
> >>
> >> regards,
> >>
> >>
> >>
> >>
> >> Regards
> >>
> >> signature
> >>
> >> *Alphé Salas*
> >> Ingeniero T.I
> >>
> >> asalas@kepler.cl
> >>
> >>
> >> On 11/07/13 11:29, Ketor D wrote:
> >>> Hi Alphe:
> >>>        Yes Callback Filesystem is very expensive and can't open
> source.
> >>> It's not a good choice for ceph4win.
> >>>        Another way for ceph4win maybe develop a kernel-mode fs
> like
> >>> pnfs. pnfs has a kernel-mode windows client. I think you can read
> its
> >>> src code and maybe migrating from ceph kernel client to windows
> kernel
> >>> fs is easier than from userspace ceph fuse client.And a
> kernel-mode fs
> >>> client has greater performance than userspace fs like ceph-fuse
> client
> >>> and ceph kernel client.
> >>>
> >>>        Regards.
> >>>
> >> On 11/07/13 11:50, Matt W. Benjamin wrote:
> >>> Hi,
> >>>
> >>> The Window NFS v4.1 client is what we work on, so this may be good
> for
> >>> code sharing.  The license is lgplv2, like Ceph's.
> >>>
> >>> Something important to be aware of is that the client uses rdbss,
> which
> >>> is a (partial) fsd abstraction that simplified implementation
> >>> quite a bit, kind of like a mini driver.  However, Microsoft's
> support
> >>> for rdbss has been in limbo for a bit.  For example, to link with
> >>> the rdbss symbols you can't use the Windows 8 driver kit--you'll
> need
> >>> to use the one for Windows 7.  (There's a private rdbss2 used
> internally
> >>> by Microsoft's SMB implemenation.  A the moment, 3rd party
> drivers
> >>> can't use that.)
> >>>
> >>> We've been in communication with Microsoft about this issue, and
> know of
> >>> a few other fsds using it, but it could be a good thing for that
> >>> lobbying
> >>> effort to have another user--or it could be a dead end :(.
> >>>
> >>> There are a couple of other choices if you're looking to go this
> route,
> >>> that I'm aware of (and we may need to take them too, if RDBSS has
> no
> >>> way forward), but the required work could be a lot larger.
> >>>
> >>> Matt
> >>>
> >>> ----- "Ketor D"<d.ketor@gmail.com>  wrote:
> >>>
> >>>> Hi Alphe:
> >>>>        Yes Callback Filesystem is very expensive and can't open
> >>>> source.
> >>>> It's not a good choice for ceph4win.
> >>>>        Another way for ceph4win maybe develop a kernel-mode fs
> like
> >>>> pnfs. pnfs has a kernel-mode windows client. I think you can read
> its
> >>>> src code and maybe migrating from ceph kernel client to windows
> >>>> kernel
> >>>> fs is easier than from userspace ceph fuse client.And a
> kernel-mode
> >>>> fs
> >>>> client has greater performance than userspace fs like ceph-fuse
> >>>> client
> >>>> and ceph kernel client.
> >>>>
> >>>>        Regards.
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
> Michels<asalas@kepler.cl>
> >>>> wrote:
> >>>>> Commercial libraries are a pain ...
> >>>>>
> >>>>> If we want the more permossive licence offered by callback file
> >>>> system we
> >>>>> have to buy it for 20.000 usd. Then we will have to provide a
> >>>> backbox that
> >>>>> we have no control upon and that will kill our product anytime
> they
> >>>> want anf
> >>>>> if they decide to stop their commercial activity we will be in
> the
> >>>> same
> >>>>> situation that with dokanfs but without having the source code
> of
> >>>> the black
> >>>>> box. If i have to spend 20 000 dollars i would prefere paying
> >>>> someone to
> >>>>> retake dokanfs or to write from scratch a dokanfs fuselike
> software
> >>>> make it
> >>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
> >>>>>
> >>>>> I would prefere if people have to pay something to get access
> to
> >>>> ceph4win
> >>>>> that this money goes in ceph main branch pockets... Or as a gift
> you
> >>>> donante
> >>>>> to ceph 10 dollars  you get 2 free registration codes for
> >>>> ceph4win... or
> >>>>> something like that.
> >>>>>
> >>>>> If ceph4win as to be comercial then I would prefer delegate the
> task
> >>>> to a
> >>>>> company like south river technologies and their great product
> >>>> webdrive. I
> >>>>> would mininaly get involved in that project and simply buy the
> final
> >>>> product
> >>>>> to sell it together with my ceph based product (which could be
> a
> >>>> calxeda
> >>>>> ceph box or something like that).
> >>>>>
> >>>>> I m open anyway to any proposition. But I doubt that callback
> >>>> filesystem
> >>>>> offers us a suitable solution in the way I see ceph4win to be
> spread
> >>>> and
> >>>>> used... I m maybe wrong. And anything that will be done around
> >>>> ceph4win will
> >>>>> be public documented etc... And licensed the way that if
> someone
> >>>> want to
> >>>>> build a commercial solution on top of it, that would be a
> >>>> possibility.
> >>>>> My idea is to giveback somehow to ceph project and at same time
> >>>> forge a
> >>>>> better knowledge in ceph technologies. Because like many in
> libre
> >>>> world I
> >>>>> think the business is in the services around the software more
> than
> >>>> on the
> >>>>> software. That the ones writing code should be financed and
> benefits
> >>>> from
> >>>>> the one selling and giving support of the software at all
> levels. I
> >>>> m
> >>>>> probably too idealistic. And too optimistic after all I m the
> one
> >>>> saying I
> >>>>> will do this stuff I have no idea how but well it is interesting
> and
> >>>> fun so
> >>>>> lets do it.
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> P.S: using commercial backend libraries appart including their
> own
> >>>> cost will
> >>>>> force you to use commercial IDE like MS VisualStudio because
> their
> >>>> library
> >>>>> has some kind of drm that only that IDE compiler can use. So
> alot of
> >>>> cost
> >>>>> and yet there is nothing done. If I had to open a kickstarter
> >>>> project saying
> >>>>> we need 60 000 USD to do ceph4win with that monney we will buy
> the
> >>>> right to
> >>>>> use and share a commercial copyrighted library but abandonned
> >>>> punctually to
> >>>>> us in  public domaine and that we will eventually produce
> something
> >>>> out of
> >>>>> it. I doubt I will get a dollar.
> >>>>>
> >>>>> We still can suggest the idea to Edlos the commercial company
> that
> >>>> has the
> >>>>> copyright of Callback FS, Or to buy them their product in a
> blender
> >>>> way
> >>>>> (blender was bought with donation before being put opensource
> and
> >>>> public
> >>>>> domaine), Or to open source their library. But in commercial
> minds
> >>>>> opensourcing = death of their technical advantage and death of
> >>>> their
> >>>>> marketing strategy. They will have to invent something more to
> >>>> retrieve
> >>>>> monney from it.
> >>>>>
> >>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> >>>>> <mailto:d.ketor@gmail.com>> escribió:
> >>>>>
> >>>>>
> >>>>>     Hi Alphe,
> >>>>>               I think you could try Callback Filesystem dev
> >>>> framework. It
> >>>>>     is a commerical dev framework and is maintained by Edlos
> today.
> >>>>>               I have communicated with Edlos to get a try code
> for
> >>>>>     development. To dokan, Callback Filesystem has vary document
> and
> >>>> maybe
> >>>>>     more stabilize.
> >>>>>
> >>>>>               Regards.
> >>>>>
> >>>>>
> >>>>>
> >>>>>     On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas
> <asalas@kepler.cl
> >>>>>     <mailto:asalas@kepler.cl>> wrote:
> >>>>>      > Hello ketor thank you for your interest un ceph4win.
> Since
> >>>> muy
> >>>>>     first mail I
> >>>>>      > exposed the lacks of dokanfs and that I m far from being
> a
> >>>>>     specialist un
> >>>>>      > filesystems.
> >>>>>      > I exposed what i like un dokanfs bit I not a fanátic of
> it.
> >>>> Muy
> >>>>>     goal is to
> >>>>>      > have something working quickly.
> >>>>>      >
> >>>>>      > So I am up to any proposición sure the one with the more
> docs
> >>>> and
> >>>>>     support
> >>>>>      > will be the best choice. As for right now what I need is
> >>>>>     understand what are
> >>>>>      > the files involved what are the interfaces functions and
> what
> >>>> are
> >>>>>     the needed
> >>>>>      > library dependencies and if they exist ported to windows
> with
> >>>>>     cygwin. And
> >>>>>      > all that is retrieved from source code.
> >>>>>      >
> >>>>>      > Regards.
> >>>>>      >
> >>>>>      > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
> >>>>>     <mailto:d.ketor@gmail.com>> escribió:
> >>>>>
> >>>>>      >
> >>>>>      >> Hi Alphe,
> >>>>>      >>      We are taking an interest in your work on Ceph
> Client
> >>>> for
> >>>>>     Windows
> >>>>>      >> with Dokan.As we know, the performance of Dokan is not
> very
> >>>>>     good, and it's
> >>>>>      >> abandoned 3 years ago.
> >>>>>      >>      I have learned and used OpenDedup(SDFS) for a long
> >>>> time.
> >>>>>       OpenDedup
> >>>>>      >> has a Dokan version. And the author of OpenDedup said
> >>>>>      >>
> >>>>>      >> The Dokan library is quite flakey  and testing should
> be
> >>>>>     performed before
> >>>>>      >> putting into production
> >>>>>      >>
> >>>>>      >>       So what do you think about this? And if there is
> >>>> another
> >>>>>     solution of
> >>>>>      >> fuse-like filesystem dev framwork on Windows?
> >>>>>      >>
> >>>>>      >>        Best Wish!
> >>>>>      >>
> >>>>>      >>
> >>>>>      >>
> >>>>>      >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
> >>>>>     <asalas@kepler.cl <mailto:asalas@kepler.cl>>
> >>>>>
> >>>>>      >> wrote:
> >>>>>      >>>
> >>>>>      >>> Hello I created the github repository for this project
> >>>>>      >>>https://github.com/alphe/Ceph4Win
> >>>>>      >>>
> >>>>>      >>> Regards,
> >>>>>      >>>
> >>>>>      >>> signature
> >>>>>      >>>
> >>>>>      >>> *Alphé Salas*
> >>>>>      >>> Ingeniero T.I
> >>>>>      >>>
> >>>>>      >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
> >>>>>
> >>>>>      >>> *<http://www.kepler.cl>*
> >>>>>      >>>
> >>>>>      >>> On 11/05/13 21:00, Sage Weil wrote:
> >>>>>      >>>>
> >>>>>      >>>> Hi Alphe,
> >>>>>      >>>>
> >>>>>      >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> >>>>>      >>>>>
> >>>>>      >>>>> signature *Hi, Sage !
> >>>>>      >>>>> thank you for you enthousiast reply.
> >>>>>      >>>>> I sure want to make the best use of everything or
> >>>> anything
> >>>>>     previously
> >>>>>      >>>>> done to
> >>>>>      >>>>> tend to
> >>>>>      >>>>> write ceph cliente for windows.
> >>>>>      >>>>>
> >>>>>      >>>>> Apart using libre tools for building the future ceph
> >>>> cliente
> >>>>>     I am open
> >>>>>      >>>>> to
> >>>>>      >>>>> anything.
> >>>>>      >>>>> I would recommand eclipse CDT or Code::BLocks they
> are
> >>>> based
> >>>>>     on mingwin
> >>>>>      >>>>> open
> >>>>>      >>>>> and easyly enhanceable.**
> >>>>>      >>>>>
> >>>>>      >>>>> more free tools can be found here:
> >>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
> >>>>>      >>>>>
> >>>>>      >>>>>
> >>>>>      >>>>> I will read libcephfs source code and take some
> notes
> >>>> about the
> >>>>>      >>>>> protocol.
> >>>>>      >>>>
> >>>>>      >>>> I think you don't need to worry about hte protocol at
> all,
> >>>> since
> >>>>>      >>>> libcephs
> >>>>>      >>>> implements it for you (and will capture any future
> >>>> changes).
> >>>>>      >>>>
> >>>>>      >>>>> I was more going from what I know and trying to track
> down
> >>>> how
> >>>>>      >>>>> mount.ceph work
> >>>>>      >>>>> with the parameters passed to it.
> >>>>>      >>>>> since it point finally to Kernel/fs/ceph and that I
> don t
> >>>> really
> >>>>>      >>>>> understand
> >>>>>      >>>>> how that module work and that it probably points to
> some
> >>>> other
> >>>>>      >>>>> dependencies
> >>>>>      >>>>> Reading libcephfs source code could be a big gain of
> >>>> time.
> >>>>>      >>>>
> >>>>>      >>>> (I would also ignore mount.ceph as everything it does
> it
> >>>>>     specific to
> >>>>>      >>>> how Linux mounts work.)
> >>>>>      >>>>
> >>>>>      >>>>> basically on the protocol what is need are:
> >>>>>      >>>>>
> >>>>>      >>>>> 1) open and maintain a connection (socket open, auth,
> etc
> >>>> )
> >>>>>      >>>>> 2) retreive a map of directories and disk Quota
> (disk
> >>>> sizing
> >>>>>     Y TB free,
> >>>>>      >>>>> Z TB
> >>>>>      >>>>> total)
> >>>>>      >>>>> 3) procedure to send files / directories in a maner
> that
> >>>> it
> >>>>>     will allow
> >>>>>      >>>>> our
> >>>>>      >>>>> client to fit ceph transmission protocols
> >>>>>      >>>>> (limit bandwith for stability?, limit connection
> amount?,
> >>>>>     limit cpu
> >>>>>      >>>>> use?,
> >>>>>      >>>>> Cache for preparing data transfer (a FIFO cache)?)
> >>>>>      >>>>> 4)Procedure to retreive files / directory from ceph
> >>>> cluster
> >>>>>      >>>>> 5) Management copy/move files /Directories, FS
> stats,
> >>>>>     Connection Stats.
> >>>>>      >>>>> logging.
> >>>>>      >>>>>
> >>>>>      >>>>> My idea to progress is to take those main bulletpoint
> in
> >>>> ceph
> >>>>>     protocol
> >>>>>      >>>>> based
> >>>>>      >>>>> on general ideas of what ceph file system does and
> start
> >>>>>     identifying
> >>>>>      >>>>> parts
> >>>>>      >>>>> from libcephfs to match those "needs".
> >>>>>      >>>>
> >>>>>      >>>> Instead, I would look at include/cephfs/libcephfs.h,
> the
> >>>>>     interface that
> >>>>>      >>>> libcephfs provides, and try to map that to what the
> fuse
> >>>> layer
> >>>>>     expects.
> >>>>>      >>>> There is both a path-based that I suspsect lends
> itself
> >>>> well
> >>>>>     to the
> >>>>>      >>>> Windows interface and (very soon now) a handle based
> API
> >>>> that is
> >>>>>      >>>> targetted
> >>>>>      >>>> at the Unix-style VFS layers.  I'm mostly guessing,
> >>>> though,
> >>>>>     since I've
> >>>>>      >>>> never seen any low-level fs code in windows before.
> >>>>>      >>>>
> >>>>>      >>>> In this case, the analogous code for Linux should be
> >>>>>     client/fuse_ll.cc
> >>>>>      >>>> itself (and not much else), although there will
> probably be
> >>>> a
> >>>>>     few tricks
> >>>>>      >>>> necessary to map cleanly onto how the windows
> interfaces
> >>>> work.
> >>>>>      >>>>
> >>>>>      >>>> Does that make sense?
> >>>>>      >>>>
> >>>>>      >>>> Cheers!
> >>>>>      >>>> sage
> >>>>>      >>>>
> >>>>>      >>>>
> >>>>>      >>>>> Any suggestion and contributions are welcome.
> >>>>>      >>>>>
> >>>>>      >>>>>
> >>>>>      >>>>> *
> >>>>>      >>>>> On 11/05/13 11:23, Sage Weil wrote:
> >>>>>      >>>>>>
> >>>>>      >>>>>> Hi Alphe,
> >>>>>      >>>>>>
> >>>>>      >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> Good day developers!
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> I would like to propose to the one interested  work
> with
> >>>> me to
> >>>>>      >>>>>>> develop a
> >>>>>      >>>>>>> ceph
> >>>>>      >>>>>>> cliente for MS windows world, Basing us on
> dokanFS.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> My company is a ceph enthousiast that use on a
> dayly
> >>>> basis
> >>>>>     ceph and
> >>>>>      >>>>>>> that
> >>>>>      >>>>>>> need
> >>>>>      >>>>>>> both transfer speed and big expendable and cheap
> >>>> storage.
> >>>>>      >>>>>>> My company is specialised in data recovery and we
> want
> >>>> to
> >>>>>     participate
> >>>>>      >>>>>>> to
> >>>>>      >>>>>>> ceph
> >>>>>      >>>>>>> effort by bringing a ceph cliente for windows.
> >>>>>      >>>>>>
> >>>>>      >>>>>> Awesome!
> >>>>>      >>>>>>
> >>>>>      >>>>>>> Our experience shows us that the best gateway is
> each
> >>>>>     clientes being
> >>>>>      >>>>>>> its
> >>>>>      >>>>>>> own
> >>>>>      >>>>>>> gateway, instead of having a bottle neck server or
> a
> >>>> cluster of
> >>>>>      >>>>>>> bottle
> >>>>>      >>>>>>> neck
> >>>>>      >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
> >>>> etc..).
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> We already did some research in that domain.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> Dokan FS is an intent  to write an opensource fuse
> like
> >>>>>     cliente for
> >>>>>      >>>>>>> MS
> >>>>>      >>>>>>> windows.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> More information on DOKANFS can be triggered here
> >>>>> >>>>>>>http://dokan-dev.net/en/download/
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> Positive points of using DOKANFS.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> - its opensourced and well licenced mit licence,
> gpl
> >>>>>     licence and lgpl
> >>>>>      >>>>>>> licence.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> Negative point of using DOKAN FS.
> >>>>>      >>>>>>> - unreachable author
> >>>>>      >>>>>>> - Poor documentation . Dev comments in japanese.
> >>>>>      >>>>>>> - Work in progress so it is unstable and needs to
> be
> >>>> updated,
> >>>>>      >>>>>>> debugged and
> >>>>>      >>>>>>> maintained by a MS Windows file system expert
> >>>> developper.
> >>>>>      >>>>>>
> >>>>>      >>>>>> I am not very familiar with windows storage APIs,
> but
> >>>>>     somebody told me
> >>>>>      >>>>>> at once point there were several interfaces against
> which
> >>>> a
> >>>>>     new file
> >>>>>      >>>>>> system could be implemented, everything from a full
> >>>>>     in-kernel driver
> >>>>>      >>>>>> to
> >>>>>      >>>>>> something that is explorer-based.  Are any of those
> >>>>>     suitable?  Using a
> >>>>>      >>>>>> potentially abandoned fuse-like layer makes me a
> bit
> >>>> nervous.
> >>>>>      >>>>>>
> >>>>>      >>>>>> That said,
> >>>>>      >>>>>>
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> I try past year to do a merge from ceph-fuse to
> dokanfs
> >>>>>      >>>>>>> here are what I learnt.
> >>>>>      >>>>>>> - Ceph-fuse and related source code is around 60
> 000
> >>>> lines
> >>>>>     of code.
> >>>>>      >>>>>>> - Ceph protocol isn t documented so it is like
> trying
> >>>> to
> >>>>>     draw a map
> >>>>>      >>>>>>> of
> >>>>>      >>>>>>> america
> >>>>>      >>>>>>> using only a sextan and a compass.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> Those led me to those conclusions:
> >>>>>      >>>>>>> - I can t do it alone.
> >>>>>      >>>>>>> - It is easier to draw down the ceph protocol way
> to
> >>>> work from
> >>>>>      >>>>>>> kernel/fs/ceph
> >>>>>      >>>>>>> sources and mount.ceph
> >>>>>      >>>>>>> - Ceph depending libraries may be unexistant or not
> up
> >>>> to
> >>>>>     date in
> >>>>>      >>>>>>> their
> >>>>>      >>>>>>> port
> >>>>>      >>>>>>> on MS Windows (cygwin)
> >>>>>      >>>>>>
> >>>>>      >>>>>> I think the most sane path should be to make
> libcephfs
> >>>>>     sufficiently
> >>>>>      >>>>>> portable to build on windows (or cygwin).  For the
> bits
> >>>> used
> >>>>>     by the
> >>>>>      >>>>>> client-side coe, I don't think there should be much
> in
> >>>> the
> >>>>>     way of
> >>>>>      >>>>>> dependencies, and the main challenge would be
> untangling
> >>>> the
> >>>>>     build for
> >>>>>      >>>>>> the necessary pieces out from the rest of Ceph.
> >>>>>      >>>>>>
> >>>>>      >>>>>> Have you seen the wip-port portability work that is
> >>>>>     currently underway
> >>>>>      >>>>>> by
> >>>>>      >>>>>> Noah and Alan?  That may solve many of the cygwin
> >>>> problems
> >>>>>     you are
> >>>>>      >>>>>> seeing
> >>>>>      >>>>>> today.
> >>>>>      >>>>>>
> >>>>>      >>>>>>> - MS file system specialist are hard do find in
> the
> >>>> "open
> >>>>>     source
> >>>>>      >>>>>>> libre
> >>>>>      >>>>>>> world"
> >>>>>      >>>>>>> so I will try in the commercial world.
> >>>>>      >>>>>>>
> >>>>>      >>>>>>> The commercial world has some problems too. They
> need
> >>>> ceph
> >>>>>     protocol
> >>>>>      >>>>>>> draft
> >>>>>      >>>>>>> to
> >>>>>      >>>>>>> implemente it to their own product They will have
> >>>> licencing
> >>>>>      >>>>>>> /commercial
> >>>>>      >>>>>>> politics that infringe lgpl, and hide that most of
> the
> >>>> work
> >>>>>     is done
> >>>>>      >>>>>>> by
> >>>>>      >>>>>>> people
> >>>>>      >>>>>>> other than them. They will not participate in a
> >>>> financial
> >>>>>     way to ceph
> >>>>>      >>>>>>> enhancement and growth.
> >>>>>      >>>>>>
> >>>>>      >>>>>> I don't think reimplementing the client code is an
> >>>> efficient way
> >>>>>      >>>>>> forward.
> >>>>>      >>>>>> Unless the goal is a pure kernel
> implementation...but a
> >>>>>     significant
> >>>>>      >>>>>> ongoing investment in development resources would
> be
> >>>> needed
> >>>>>     for that
> >>>>>      >>>>>> going
> >>>>>      >>>>>> forward.  I suspect that is a challenge for a
> platform
> >>>> that
> >>>>>     does not
> >>>>>      >>>>>> typically rally that sort of community effort.
> >>>>>      >>>>>>
> >>>>>      >>>>>> The easiest thing is of course just to use CIFS and
> >>>> Samba
> >>>>>     (which works
> >>>>>      >>>>>> today).  A fuse-like approach is probably a
> reasonably
> >>>>>     middle ground
> >>>>>      >>>>>> (both
> >>>>>      >>>>>> in initial effort and maintainability going
> forward)...
> >>>>>      >>>>>>
> >>>>>      >>>>>> sage
> >>>>>      >>>>>>
> >>>>>      >>>>>>
> >>>>>      >>>>>
> >>>>>      >>>
> >>>>>      >>> --
> >>>>>      >>> To unsubscribe from this list: send the line
> "unsubscribe
> >>>>>     ceph-devel" in
> >>>>>      >>> the body of a message tomajordomo@vger.kernel.org
> >>>>>     <mailto:majordomo@vger.kernel.org>
> >>>>>
> >>>>>      >>> More majordomo info at
> >>>> http://vger.kernel.org/majordomo-info.html
> >>>>>      >>
> >>>>>      >>
> >>>>>      >
> >>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> To unsubscribe from this list: send the line "unsubscribe
> ceph-devel"
> >>>> in
> >>>> the body of a message tomajordomo@vger.kernel.org
> >>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
> >>
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello malcom and matt thank you for apporting some more information 
source. OpenAFS is sure interesting httpfs too.

I hope it will help us on deciding the best path to follow in our 
interface with window.
Actually I still trying to isolate the needed client code in the 
shortest way possible.

Regards.

Alphe Salas

El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com 
<mailto:malcolm@sgi.com>> escribió:

    I'm just going to throw these in there.

    http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>

    They are GPLv2 some already use sockets and such from inside the
    kernel.  Heck you might even be able to mod the HTTP one to use
    rados gateway. I don't know as I havent sat down and pulled them
    apart enough yet.

    They might help, but they might be useless. Not sure.

    On 08/11/13 06:47, Alphe Salas Michels wrote:

        Hello all I finally finished my first source code extraction
        that starts
        from ceph/src/client/fuse_ll.c
        The result is accurate unlike previous provided results.
        basically the
        script start from a file extract all the private includes
        definitions
        #include "something.h" and recursively extract private includes
        too. the
        best way to know who is related with who.

        starting from fuse_ll.cc I optain 390 files retreived and 120
        000 lines
        of code !
        involved dirs are : in ceph/src
        objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
        global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/

        probably not a good way to analyse what amount of work it means
        since
        most of those directories are the implementation of servers
        (osd, mon,
        mds) and even if only a tiny bit of them is needed at client
        level. you
        need two structures from ./osd/OSD.h and  my script by relation will
        take into acount the whole directory...

        I ran the script with libcephfs.cc as start point and got almost the
        same results. 131 000 lines of code and 386 files most of the
        same dirs
        involved.



        I think I will spend alot of time doing the manual source code
        isolation
        and understand way each #include is set in the files I read (what
        purpose they have do they allow to integrate a crucial data type
        or not.


        The other way around will be to read src/libcephfs.cc. It seems
        shorter
        but without understanding what part is used for each included
        header I
        can t say anything...



        I will keep reading the source code and take notes. I think in
        the case
        of libcephfs I will gain alot of time.

        signature

        *Alphé Salas*
        Ingeniero T.I

        asalas@kepler.cl <mailto:asalas@kepler.cl>
        *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*

        On 11/07/13 15:02, Alphe Salas Michels wrote:

            Hello D.Ketor and Matt Benjamin,
            You give me alot to think about and this is great!
            I merged your previous post to make a single reply that
            anyone can
            report to easyly

            Windows NFS 4.1 is available here:
            http://www.citi.umich.edu/projects/nfsv4/windows/readme.html

            pnfs is another name for NFS4.X. It is presented as
            alternative to
            ceph and we get known terminology as MDS and OSD but without
            the self
            healing part if I understand well my rapid look on the
            topic. (when I
            say rapid look I mean ... 5 minutes spent in that... which
            is really
            small amount of time to get an accurate view on something)


            starting from mount.ceph ... I know that mount.ceph does
            little but it
            is a great hint to know what ceph needs and do things.
            Basically mount.ceph modprobe the ceph driver in the linux
            kernel then
            call mount with the line command passed args and the cephfs
            type as
            argument. Then the kernel does the work I don t understand
            yet what is
            the start calls that are made to the ceph driver but it
            seemed to me
            that is was relatively light. (a first impression compared
            to ceph-fuse.)

            I think I will do both isolate source code from ceph-client
            kernel
            (cephfs module for linux kernel) and the one pointed by Sage
            starting
            from client/fuse_ll.cc in ceph master branch. The common
            files betwin
            those 2 extractions will be our core set of mandatory features.

            Then we try to compile with cygwin a cephfs client library .
            Then we
            will try to interface with a modified windows nfs 4.1 client
            or pnfs
            or any other that will accept to be compiled with gcc for
            win32...

            the fact that windows 8.1 is and windows 2012 are out of
            reach at the
            moment is not a problem to me.

            Our first concern is to understand what is ceph protocol.
            Then adapt
            it to something that can be used on windows prior windows
            8.1. Dokan
            fs if I remember well use too the WDK (windows driver
            dev-kit ) for it
            s compilation so possibly we will see the same limitations.

            We need to multiply our source of information by example
            regarding
            ceph-client (kernel or fuse, radosgw is on a different layer
            so I will
            not try anything around it at first.) And we need to
            multiply our
            source of information by example regarding virtual file system
            technologies on windoes OS.
            Alot of work but all of those available source code everyone
            point at
            me will make our best solution. And in the end we will choose
            technologies knowing what we do and what concequencies they
            have.

            regards,




            Regards

            signature

            *Alphé Salas*
            Ingeniero T.I

            asalas@kepler.cl <mailto:asalas@kepler.cl>


            On 11/07/13 11:29, Ketor D wrote:

                Hi Alphe:
                        Yes Callback Filesystem is very expensive and
                can't open source.
                It's not a good choice for ceph4win.
                        Another way for ceph4win maybe develop a
                kernel-mode fs like
                pnfs. pnfs has a kernel-mode windows client. I think you
                can read its
                src code and maybe migrating from ceph kernel client to
                windows kernel
                fs is easier than from userspace ceph fuse client.And a
                kernel-mode fs
                client has greater performance than userspace fs like
                ceph-fuse client
                and ceph kernel client.

                        Regards.

            On 11/07/13 11:50, Matt W. Benjamin wrote:

                Hi,

                The Window NFS v4.1 client is what we work on, so this
                may be good for
                code sharing.  The license is lgplv2, like Ceph's.

                Something important to be aware of is that the client
                uses rdbss, which
                is a (partial) fsd abstraction that simplified
                implementation
                quite a bit, kind of like a mini driver.  However,
                Microsoft's support
                for rdbss has been in limbo for a bit.  For example, to
                link with
                the rdbss symbols you can't use the Windows 8 driver
                kit--you'll need
                to use the one for Windows 7.  (There's a private rdbss2
                used internally
                by Microsoft's SMB implemenation.  A the moment, 3rd
                party drivers
                can't use that.)

                We've been in communication with Microsoft about this
                issue, and know of
                a few other fsds using it, but it could be a good thing
                for that
                lobbying
                effort to have another user--or it could be a dead end :(.

                There are a couple of other choices if you're looking to
                go this route,
                that I'm aware of (and we may need to take them too, if
                RDBSS has no
                way forward), but the required work could be a lot larger.

                Matt

                ----- "Ketor D"<d.ketor@gmail.com
                <mailto:d.ketor@gmail.com>>  wrote:

                    Hi Alphe:
                            Yes Callback Filesystem is very expensive
                    and can't open
                    source.
                    It's not a good choice for ceph4win.
                            Another way for ceph4win maybe develop a
                    kernel-mode fs like
                    pnfs. pnfs has a kernel-mode windows client. I think
                    you can read its
                    src code and maybe migrating from ceph kernel client
                    to windows
                    kernel
                    fs is easier than from userspace ceph fuse
                    client.And a kernel-mode
                    fs
                    client has greater performance than userspace fs
                    like ceph-fuse
                    client
                    and ceph kernel client.

                            Regards.



                    On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
                    Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
                    wrote:

                        Commercial libraries are a pain ...

                        If we want the more permossive licence offered
                        by callback file

                    system we

                        have to buy it for 20.000 usd. Then we will have
                        to provide a

                    backbox that

                        we have no control upon and that will kill our
                        product anytime they

                    want anf

                        if they decide to stop their commercial activity
                        we will be in the

                    same

                        situation that with dokanfs but without having
                        the source code of

                    the black

                        box. If i have to spend 20 000 dollars i would
                        prefere paying

                    someone to

                        retake dokanfs or to write from scratch a
                        dokanfs fuselike software

                    make it

                        all shiny and pumpy fantastic and ready to plug
                        to ceph client.

                        I would prefere if people have to pay something
                        to get access to

                    ceph4win

                        that this money goes in ceph main branch
                        pockets... Or as a gift you

                    donante

                        to ceph 10 dollars  you get 2 free registration
                        codes for

                    ceph4win... or

                        something like that.

                        If ceph4win as to be comercial then I would
                        prefer delegate the task

                    to a

                        company like south river technologies and their
                        great product

                    webdrive. I

                        would mininaly get involved in that project and
                        simply buy the final

                    product

                        to sell it together with my ceph based product
                        (which could be a

                    calxeda

                        ceph box or something like that).

                        I m open anyway to any proposition. But I doubt
                        that callback

                    filesystem

                        offers us a suitable solution in the way I see
                        ceph4win to be spread

                    and

                        used... I m maybe wrong. And anything that will
                        be done around

                    ceph4win will

                        be public documented etc... And licensed the way
                        that if someone

                    want to

                        build a commercial solution on top of it, that
                        would be a

                    possibility.

                        My idea is to giveback somehow to ceph project
                        and at same time

                    forge a

                        better knowledge in ceph technologies. Because
                        like many in libre

                    world I

                        think the business is in the services around the
                        software more than

                    on the

                        software. That the ones writing code should be
                        financed and benefits

                    from

                        the one selling and giving support of the
                        software at all levels. I

                    m

                        probably too idealistic. And too optimistic
                        after all I m the one

                    saying I

                        will do this stuff I have no idea how but well
                        it is interesting and

                    fun so

                        lets do it.

                        Regards,

                        P.S: using commercial backend libraries appart
                        including their own

                    cost will

                        force you to use commercial IDE like MS
                        VisualStudio because their

                    library

                        has some kind of drm that only that IDE compiler
                        can use. So alot of

                    cost

                        and yet there is nothing done. If I had to open
                        a kickstarter

                    project saying

                        we need 60 000 USD to do ceph4win with that
                        monney we will buy the

                    right to

                        use and share a commercial copyrighted library
                        but abandonned

                    punctually to

                        us in  public domaine and that we will
                        eventually produce something

                    out of

                        it. I doubt I will get a dollar.

                        We still can suggest the idea to Edlos the
                        commercial company that

                    has the

                        copyright of Callback FS, Or to buy them their
                        product in a blender

                    way

                        (blender was bought with donation before being
                        put opensource and

                    public

                        domaine), Or to open source their library. But
                        in commercial minds
                        opensourcing = death of their technical
                        advantage and death of

                    their

                        marketing strategy. They will have to invent
                        something more to

                    retrieve

                        monney from it.

                        El nov 6, 2013 11:22 p.m., "Ketor D"
                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
                        <mailto:d.ketor@gmail.com
                        <mailto:d.ketor@gmail.com>>> escribió:


                             Hi Alphe,
                                       I think you could try Callback
                        Filesystem dev

                    framework. It

                             is a commerical dev framework and is
                        maintained by Edlos today.
                                       I have communicated with Edlos to
                        get a try code for
                             development. To dokan, Callback Filesystem
                        has vary document and

                    maybe

                             more stabilize.

                                       Regards.



                             On Thu, Nov 7, 2013 at 10:00 AM, Alphe
                        Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
                             <mailto:asalas@kepler.cl
                        <mailto:asalas@kepler.cl>>> wrote:
                              > Hello ketor thank you for your interest
                        un ceph4win. Since

                    muy

                             first mail I
                              > exposed the lacks of dokanfs and that I
                        m far from being a
                             specialist un
                              > filesystems.
                              > I exposed what i like un dokanfs bit I
                        not a fanátic of it.

                    Muy

                             goal is to
                              > have something working quickly.
                              >
                              > So I am up to any proposición sure the
                        one with the more docs

                    and

                             support
                              > will be the best choice. As for right
                        now what I need is
                             understand what are
                              > the files involved what are the
                        interfaces functions and what

                    are

                             the needed
                              > library dependencies and if they exist
                        ported to windows with
                             cygwin. And
                              > all that is retrieved from source code.
                              >
                              > Regards.
                              >
                              > El nov 6, 2013 10:34 p.m., "Ketor D"
                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
                             <mailto:d.ketor@gmail.com
                        <mailto:d.ketor@gmail.com>>> escribió:

                              >
                              >> Hi Alphe,
                              >>      We are taking an interest in your
                        work on Ceph Client

                    for

                             Windows
                              >> with Dokan.As we know, the performance
                        of Dokan is not very
                             good, and it's
                              >> abandoned 3 years ago.
                              >>      I have learned and used
                        OpenDedup(SDFS) for a long

                    time.

                               OpenDedup
                              >> has a Dokan version. And the author of
                        OpenDedup said
                              >>
                              >> The Dokan library is quite flakey  and
                        testing should be
                             performed before
                              >> putting into production
                              >>
                              >>       So what do you think about this?
                        And if there is

                    another

                             solution of
                              >> fuse-like filesystem dev framwork on
                        Windows?
                              >>
                              >>        Best Wish!
                              >>
                              >>
                              >>
                              >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
                        Salas Michels
                             <asalas@kepler.cl <mailto:asalas@kepler.cl>
                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>

                              >> wrote:
                              >>>
                              >>> Hello I created the github repository
                        for this project
                              >>>https://github.com/alphe/Ceph4Win
                              >>>
                              >>> Regards,
                              >>>
                              >>> signature
                              >>>
                              >>> *Alphé Salas*
                              >>> Ingeniero T.I
                              >>>
                              >>>asalas@kepler.cl
                        <mailto:asalas@kepler.cl>
                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>

                              >>> *<http://www.kepler.cl>*
                              >>>
                              >>> On 11/05/13 21:00, Sage Weil wrote:
                              >>>>
                              >>>> Hi Alphe,
                              >>>>
                              >>>> On Tue, 5 Nov 2013, Alphe Salas
                        Michels wrote:
                              >>>>>
                              >>>>> signature *Hi, Sage !
                              >>>>> thank you for you enthousiast reply.
                              >>>>> I sure want to make the best use of
                        everything or

                    anything

                             previously
                              >>>>> done to
                              >>>>> tend to
                              >>>>> write ceph cliente for windows.
                              >>>>>
                              >>>>> Apart using libre tools for building
                        the future ceph

                    cliente

                             I am open
                              >>>>> to
                              >>>>> anything.
                              >>>>> I would recommand eclipse CDT or
                        Code::BLocks they are

                    based

                             on mingwin
                              >>>>> open
                              >>>>> and easyly enhanceable.**
                              >>>>>
                              >>>>> more free tools can be found here:
                         >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
                              >>>>>
                              >>>>>
                              >>>>> I will read libcephfs source code
                        and take some notes

                    about the

                              >>>>> protocol.
                              >>>>
                              >>>> I think you don't need to worry about
                        hte protocol at all,

                    since

                              >>>> libcephs
                              >>>> implements it for you (and will
                        capture any future

                    changes).

                              >>>>
                              >>>>> I was more going from what I know
                        and trying to track down

                    how

                              >>>>> mount.ceph work
                              >>>>> with the parameters passed to it.
                              >>>>> since it point finally to
                        Kernel/fs/ceph and that I don t

                    really

                              >>>>> understand
                              >>>>> how that module work and that it
                        probably points to some

                    other

                              >>>>> dependencies
                              >>>>> Reading libcephfs source code could
                        be a big gain of

                    time.

                              >>>>
                              >>>> (I would also ignore mount.ceph as
                        everything it does it
                             specific to
                              >>>> how Linux mounts work.)
                              >>>>
                              >>>>> basically on the protocol what is
                        need are:
                              >>>>>
                              >>>>> 1) open and maintain a connection
                        (socket open, auth, etc

                    )

                              >>>>> 2) retreive a map of directories and
                        disk Quota (disk

                    sizing

                             Y TB free,
                              >>>>> Z TB
                              >>>>> total)
                              >>>>> 3) procedure to send files /
                        directories in a maner that

                    it

                             will allow
                              >>>>> our
                              >>>>> client to fit ceph transmission
                        protocols
                              >>>>> (limit bandwith for stability?,
                        limit connection amount?,
                             limit cpu
                              >>>>> use?,
                              >>>>> Cache for preparing data transfer (a
                        FIFO cache)?)
                              >>>>> 4)Procedure to retreive files /
                        directory from ceph

                    cluster

                              >>>>> 5) Management copy/move files
                        /Directories, FS stats,
                             Connection Stats.
                              >>>>> logging.
                              >>>>>
                              >>>>> My idea to progress is to take those
                        main bulletpoint in

                    ceph

                             protocol
                              >>>>> based
                              >>>>> on general ideas of what ceph file
                        system does and start
                             identifying
                              >>>>> parts
                              >>>>> from libcephfs to match those "needs".
                              >>>>
                              >>>> Instead, I would look at
                        include/cephfs/libcephfs.h, the
                             interface that
                              >>>> libcephfs provides, and try to map
                        that to what the fuse

                    layer

                             expects.
                              >>>> There is both a path-based that I
                        suspsect lends itself

                    well

                             to the
                              >>>> Windows interface and (very soon now)
                        a handle based API

                    that is

                              >>>> targetted
                              >>>> at the Unix-style VFS layers.  I'm
                        mostly guessing,

                    though,

                             since I've
                              >>>> never seen any low-level fs code in
                        windows before.
                              >>>>
                              >>>> In this case, the analogous code for
                        Linux should be
                             client/fuse_ll.cc
                              >>>> itself (and not much else), although
                        there will probably be

                    a

                             few tricks
                              >>>> necessary to map cleanly onto how the
                        windows interfaces

                    work.

                              >>>>
                              >>>> Does that make sense?
                              >>>>
                              >>>> Cheers!
                              >>>> sage
                              >>>>
                              >>>>
                              >>>>> Any suggestion and contributions are
                        welcome.
                              >>>>>
                              >>>>>
                              >>>>> *
                              >>>>> On 11/05/13 11:23, Sage Weil wrote:
                              >>>>>>
                              >>>>>> Hi Alphe,
                              >>>>>>
                              >>>>>> On Mon, 4 Nov 2013, Alphe Salas
                        Michels wrote:
                              >>>>>>>
                              >>>>>>> Good day developers!
                              >>>>>>>
                              >>>>>>> I would like to propose to the one
                        interested  work with

                    me to

                              >>>>>>> develop a
                              >>>>>>> ceph
                              >>>>>>> cliente for MS windows world,
                        Basing us on dokanFS.
                              >>>>>>>
                              >>>>>>> My company is a ceph enthousiast
                        that use on a dayly

                    basis

                             ceph and
                              >>>>>>> that
                              >>>>>>> need
                              >>>>>>> both transfer speed and big
                        expendable and cheap

                    storage.

                              >>>>>>> My company is specialised in data
                        recovery and we want

                    to

                             participate
                              >>>>>>> to
                              >>>>>>> ceph
                              >>>>>>> effort by bringing a ceph cliente
                        for windows.
                              >>>>>>
                              >>>>>> Awesome!
                              >>>>>>
                              >>>>>>> Our experience shows us that the
                        best gateway is each
                             clientes being
                              >>>>>>> its
                              >>>>>>> own
                              >>>>>>> gateway, instead of having a
                        bottle neck server or a

                    cluster of

                              >>>>>>> bottle
                              >>>>>>> neck
                              >>>>>>> servers as gateway (FTP, samba,
                        SFTP,webdav, s3,

                    etc..).

                              >>>>>>>
                              >>>>>>> We already did some research in
                        that domain.
                              >>>>>>>
                              >>>>>>> Dokan FS is an intent  to write an
                        opensource fuse like
                             cliente for
                              >>>>>>> MS
                              >>>>>>> windows.
                              >>>>>>>
                              >>>>>>> More information on DOKANFS can be
                        triggered here
                         >>>>>>>http://dokan-dev.net/en/download/
                              >>>>>>>
                              >>>>>>> Positive points of using DOKANFS.
                              >>>>>>>
                              >>>>>>> - its opensourced and well
                        licenced mit licence, gpl
                             licence and lgpl
                              >>>>>>> licence.
                              >>>>>>>
                              >>>>>>> Negative point of using DOKAN FS.
                              >>>>>>> - unreachable author
                              >>>>>>> - Poor documentation . Dev
                        comments in japanese.
                              >>>>>>> - Work in progress so it is
                        unstable and needs to be

                    updated,

                              >>>>>>> debugged and
                              >>>>>>> maintained by a MS Windows file
                        system expert

                    developper.

                              >>>>>>
                              >>>>>> I am not very familiar with windows
                        storage APIs, but
                             somebody told me
                              >>>>>> at once point there were several
                        interfaces against which

                    a

                             new file
                              >>>>>> system could be implemented,
                        everything from a full
                             in-kernel driver
                              >>>>>> to
                              >>>>>> something that is explorer-based.
                          Are any of those
                             suitable?  Using a
                              >>>>>> potentially abandoned fuse-like
                        layer makes me a bit

                    nervous.

                              >>>>>>
                              >>>>>> That said,
                              >>>>>>
                              >>>>>>>
                              >>>>>>> I try past year to do a merge from
                        ceph-fuse to dokanfs
                              >>>>>>> here are what I learnt.
                              >>>>>>> - Ceph-fuse and related source
                        code is around 60 000

                    lines

                             of code.
                              >>>>>>> - Ceph protocol isn t documented
                        so it is like trying

                    to

                             draw a map
                              >>>>>>> of
                              >>>>>>> america
                              >>>>>>> using only a sextan and a compass.
                              >>>>>>>
                              >>>>>>> Those led me to those conclusions:
                              >>>>>>> - I can t do it alone.
                              >>>>>>> - It is easier to draw down the
                        ceph protocol way to

                    work from

                              >>>>>>> kernel/fs/ceph
                              >>>>>>> sources and mount.ceph
                              >>>>>>> - Ceph depending libraries may be
                        unexistant or not up

                    to

                             date in
                              >>>>>>> their
                              >>>>>>> port
                              >>>>>>> on MS Windows (cygwin)
                              >>>>>>
                              >>>>>> I think the most sane path should
                        be to make libcephfs
                             sufficiently
                              >>>>>> portable to build on windows (or
                        cygwin).  For the bits

                    used

                             by the
                              >>>>>> client-side coe, I don't think
                        there should be much in

                    the

                             way of
                              >>>>>> dependencies, and the main
                        challenge would be untangling

                    the

                             build for
                              >>>>>> the necessary pieces out from the
                        rest of Ceph.
                              >>>>>>
                              >>>>>> Have you seen the wip-port
                        portability work that is
                             currently underway
                              >>>>>> by
                              >>>>>> Noah and Alan?  That may solve many
                        of the cygwin

                    problems

                             you are
                              >>>>>> seeing
                              >>>>>> today.
                              >>>>>>
                              >>>>>>> - MS file system specialist are
                        hard do find in the

                    "open

                             source
                              >>>>>>> libre
                              >>>>>>> world"
                              >>>>>>> so I will try in the commercial world.
                              >>>>>>>
                              >>>>>>> The commercial world has some
                        problems too. They need

                    ceph

                             protocol
                              >>>>>>> draft
                              >>>>>>> to
                              >>>>>>> implemente it to their own product
                        They will have

                    licencing

                              >>>>>>> /commercial
                              >>>>>>> politics that infringe lgpl, and
                        hide that most of the

                    work

                             is done
                              >>>>>>> by
                              >>>>>>> people
                              >>>>>>> other than them. They will not
                        participate in a

                    financial

                             way to ceph
                              >>>>>>> enhancement and growth.
                              >>>>>>
                              >>>>>> I don't think reimplementing the
                        client code is an

                    efficient way

                              >>>>>> forward.
                              >>>>>> Unless the goal is a pure kernel
                        implementation...but a
                             significant
                              >>>>>> ongoing investment in development
                        resources would be

                    needed

                             for that
                              >>>>>> going
                              >>>>>> forward.  I suspect that is a
                        challenge for a platform

                    that

                             does not
                              >>>>>> typically rally that sort of
                        community effort.
                              >>>>>>
                              >>>>>> The easiest thing is of course just
                        to use CIFS and

                    Samba

                             (which works
                              >>>>>> today).  A fuse-like approach is
                        probably a reasonably
                             middle ground
                              >>>>>> (both
                              >>>>>> in initial effort and
                        maintainability going forward)...
                              >>>>>>
                              >>>>>> sage
                              >>>>>>
                              >>>>>>
                              >>>>>
                              >>>
                              >>> --
                              >>> To unsubscribe from this list: send
                        the line "unsubscribe
                             ceph-devel" in
                              >>> the body of a message
                        tomajordomo@vger.kernel.org
                        <mailto:tomajordomo@vger.kernel.org>
                             <mailto:majordomo@vger.kernel.org
                        <mailto:majordomo@vger.kernel.org>>

                              >>> More majordomo info at

                    http://vger.kernel.org/majordomo-info.html

                              >>
                              >>
                              >



                    --
                    To unsubscribe from this list: send the line
                    "unsubscribe ceph-devel"
                    in
                    the body of a message tomajordomo@vger.kernel.org
                    <mailto:tomajordomo@vger.kernel.org>
                    More majordomo info
                    athttp://vger.kernel.org/majordomo-info.html
                    <http://vger.kernel.org/majordomo-info.html>



        --
        To unsubscribe from this list: send the line "unsubscribe
        ceph-devel" in
        the body of a message to majordomo@vger.kernel.org
        <mailto:majordomo@vger.kernel.org>
        More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Ketor D]

Is here any progress on Cephfs Windows Client ?

2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
> Hello malcom and matt thank you for apporting some more information source.
> OpenAFS is sure interesting httpfs too.
>
> I hope it will help us on deciding the best path to follow in our interface
> with window.
> Actually I still trying to isolate the needed client code in the shortest
> way possible.
>
> Regards.
>
> Alphe Salas
>
> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
> <mailto:malcolm@sgi.com>> escribió:
>
>    I'm just going to throw these in there.
>
>    http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>
>
>    They are GPLv2 some already use sockets and such from inside the
>    kernel.  Heck you might even be able to mod the HTTP one to use
>    rados gateway. I don't know as I havent sat down and pulled them
>    apart enough yet.
>
>    They might help, but they might be useless. Not sure.
>
>    On 08/11/13 06:47, Alphe Salas Michels wrote:
>
>        Hello all I finally finished my first source code extraction
>        that starts
>        from ceph/src/client/fuse_ll.c
>        The result is accurate unlike previous provided results.
>        basically the
>        script start from a file extract all the private includes
>        definitions
>        #include "something.h" and recursively extract private includes
>        too. the
>        best way to know who is related with who.
>
>        starting from fuse_ll.cc I optain 390 files retreived and 120
>        000 lines
>        of code !
>        involved dirs are : in ceph/src
>        objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>        global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>
>        probably not a good way to analyse what amount of work it means
>        since
>        most of those directories are the implementation of servers
>        (osd, mon,
>        mds) and even if only a tiny bit of them is needed at client
>        level. you
>        need two structures from ./osd/OSD.h and  my script by relation will
>        take into acount the whole directory...
>
>        I ran the script with libcephfs.cc as start point and got almost the
>        same results. 131 000 lines of code and 386 files most of the
>        same dirs
>        involved.
>
>
>
>        I think I will spend alot of time doing the manual source code
>        isolation
>        and understand way each #include is set in the files I read (what
>        purpose they have do they allow to integrate a crucial data type
>        or not.
>
>
>        The other way around will be to read src/libcephfs.cc. It seems
>        shorter
>        but without understanding what part is used for each included
>        header I
>        can t say anything...
>
>
>
>        I will keep reading the source code and take notes. I think in
>        the case
>        of libcephfs I will gain alot of time.
>
>        signature
>
>        *Alphé Salas*
>        Ingeniero T.I
>
>        asalas@kepler.cl <mailto:asalas@kepler.cl>
>        *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>
>
>        On 11/07/13 15:02, Alphe Salas Michels wrote:
>
>            Hello D.Ketor and Matt Benjamin,
>            You give me alot to think about and this is great!
>            I merged your previous post to make a single reply that
>            anyone can
>            report to easyly
>
>            Windows NFS 4.1 is available here:
>            http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>
>            pnfs is another name for NFS4.X. It is presented as
>            alternative to
>            ceph and we get known terminology as MDS and OSD but without
>            the self
>            healing part if I understand well my rapid look on the
>            topic. (when I
>            say rapid look I mean ... 5 minutes spent in that... which
>            is really
>            small amount of time to get an accurate view on something)
>
>
>            starting from mount.ceph ... I know that mount.ceph does
>            little but it
>            is a great hint to know what ceph needs and do things.
>            Basically mount.ceph modprobe the ceph driver in the linux
>            kernel then
>            call mount with the line command passed args and the cephfs
>            type as
>            argument. Then the kernel does the work I don t understand
>            yet what is
>            the start calls that are made to the ceph driver but it
>            seemed to me
>            that is was relatively light. (a first impression compared
>            to ceph-fuse.)
>
>            I think I will do both isolate source code from ceph-client
>            kernel
>            (cephfs module for linux kernel) and the one pointed by Sage
>            starting
>            from client/fuse_ll.cc in ceph master branch. The common
>            files betwin
>            those 2 extractions will be our core set of mandatory features.
>
>            Then we try to compile with cygwin a cephfs client library .
>            Then we
>            will try to interface with a modified windows nfs 4.1 client
>            or pnfs
>            or any other that will accept to be compiled with gcc for
>            win32...
>
>            the fact that windows 8.1 is and windows 2012 are out of
>            reach at the
>            moment is not a problem to me.
>
>            Our first concern is to understand what is ceph protocol.
>            Then adapt
>            it to something that can be used on windows prior windows
>            8.1. Dokan
>            fs if I remember well use too the WDK (windows driver
>            dev-kit ) for it
>            s compilation so possibly we will see the same limitations.
>
>            We need to multiply our source of information by example
>            regarding
>            ceph-client (kernel or fuse, radosgw is on a different layer
>            so I will
>            not try anything around it at first.) And we need to
>            multiply our
>            source of information by example regarding virtual file system
>            technologies on windoes OS.
>            Alot of work but all of those available source code everyone
>            point at
>            me will make our best solution. And in the end we will choose
>            technologies knowing what we do and what concequencies they
>            have.
>
>            regards,
>
>
>
>
>            Regards
>
>            signature
>
>            *Alphé Salas*
>            Ingeniero T.I
>
>            asalas@kepler.cl <mailto:asalas@kepler.cl>
>
>
>
>            On 11/07/13 11:29, Ketor D wrote:
>
>                Hi Alphe:
>                        Yes Callback Filesystem is very expensive and
>                can't open source.
>                It's not a good choice for ceph4win.
>                        Another way for ceph4win maybe develop a
>                kernel-mode fs like
>                pnfs. pnfs has a kernel-mode windows client. I think you
>                can read its
>                src code and maybe migrating from ceph kernel client to
>                windows kernel
>                fs is easier than from userspace ceph fuse client.And a
>                kernel-mode fs
>                client has greater performance than userspace fs like
>                ceph-fuse client
>                and ceph kernel client.
>
>                        Regards.
>
>            On 11/07/13 11:50, Matt W. Benjamin wrote:
>
>                Hi,
>
>                The Window NFS v4.1 client is what we work on, so this
>                may be good for
>                code sharing.  The license is lgplv2, like Ceph's.
>
>                Something important to be aware of is that the client
>                uses rdbss, which
>                is a (partial) fsd abstraction that simplified
>                implementation
>                quite a bit, kind of like a mini driver.  However,
>                Microsoft's support
>                for rdbss has been in limbo for a bit.  For example, to
>                link with
>                the rdbss symbols you can't use the Windows 8 driver
>                kit--you'll need
>                to use the one for Windows 7.  (There's a private rdbss2
>                used internally
>                by Microsoft's SMB implemenation.  A the moment, 3rd
>                party drivers
>                can't use that.)
>
>                We've been in communication with Microsoft about this
>                issue, and know of
>                a few other fsds using it, but it could be a good thing
>                for that
>                lobbying
>                effort to have another user--or it could be a dead end :(.
>
>                There are a couple of other choices if you're looking to
>                go this route,
>                that I'm aware of (and we may need to take them too, if
>                RDBSS has no
>                way forward), but the required work could be a lot larger.
>
>                Matt
>
>                ----- "Ketor D"<d.ketor@gmail.com
>                <mailto:d.ketor@gmail.com>>  wrote:
>
>                    Hi Alphe:
>                            Yes Callback Filesystem is very expensive
>                    and can't open
>                    source.
>                    It's not a good choice for ceph4win.
>                            Another way for ceph4win maybe develop a
>                    kernel-mode fs like
>                    pnfs. pnfs has a kernel-mode windows client. I think
>                    you can read its
>                    src code and maybe migrating from ceph kernel client
>                    to windows
>                    kernel
>                    fs is easier than from userspace ceph fuse
>                    client.And a kernel-mode
>                    fs
>                    client has greater performance than userspace fs
>                    like ceph-fuse
>                    client
>                    and ceph kernel client.
>
>                            Regards.
>
>
>
>                    On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>                    Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
>                    wrote:
>
>                        Commercial libraries are a pain ...
>
>                        If we want the more permossive licence offered
>                        by callback file
>
>                    system we
>
>                        have to buy it for 20.000 usd. Then we will have
>                        to provide a
>
>                    backbox that
>
>                        we have no control upon and that will kill our
>                        product anytime they
>
>                    want anf
>
>                        if they decide to stop their commercial activity
>                        we will be in the
>
>                    same
>
>                        situation that with dokanfs but without having
>                        the source code of
>
>                    the black
>
>                        box. If i have to spend 20 000 dollars i would
>                        prefere paying
>
>                    someone to
>
>                        retake dokanfs or to write from scratch a
>                        dokanfs fuselike software
>
>                    make it
>
>                        all shiny and pumpy fantastic and ready to plug
>                        to ceph client.
>
>                        I would prefere if people have to pay something
>                        to get access to
>
>                    ceph4win
>
>                        that this money goes in ceph main branch
>                        pockets... Or as a gift you
>
>                    donante
>
>                        to ceph 10 dollars  you get 2 free registration
>                        codes for
>
>                    ceph4win... or
>
>                        something like that.
>
>                        If ceph4win as to be comercial then I would
>                        prefer delegate the task
>
>                    to a
>
>                        company like south river technologies and their
>                        great product
>
>                    webdrive. I
>
>                        would mininaly get involved in that project and
>                        simply buy the final
>
>                    product
>
>                        to sell it together with my ceph based product
>                        (which could be a
>
>                    calxeda
>
>                        ceph box or something like that).
>
>                        I m open anyway to any proposition. But I doubt
>                        that callback
>
>                    filesystem
>
>                        offers us a suitable solution in the way I see
>                        ceph4win to be spread
>
>                    and
>
>                        used... I m maybe wrong. And anything that will
>                        be done around
>
>                    ceph4win will
>
>                        be public documented etc... And licensed the way
>                        that if someone
>
>                    want to
>
>                        build a commercial solution on top of it, that
>                        would be a
>
>                    possibility.
>
>                        My idea is to giveback somehow to ceph project
>                        and at same time
>
>                    forge a
>
>                        better knowledge in ceph technologies. Because
>                        like many in libre
>
>                    world I
>
>                        think the business is in the services around the
>                        software more than
>
>                    on the
>
>                        software. That the ones writing code should be
>                        financed and benefits
>
>                    from
>
>                        the one selling and giving support of the
>                        software at all levels. I
>
>                    m
>
>                        probably too idealistic. And too optimistic
>                        after all I m the one
>
>                    saying I
>
>                        will do this stuff I have no idea how but well
>                        it is interesting and
>
>                    fun so
>
>                        lets do it.
>
>                        Regards,
>
>                        P.S: using commercial backend libraries appart
>                        including their own
>
>                    cost will
>
>                        force you to use commercial IDE like MS
>                        VisualStudio because their
>
>                    library
>
>                        has some kind of drm that only that IDE compiler
>                        can use. So alot of
>
>                    cost
>
>                        and yet there is nothing done. If I had to open
>                        a kickstarter
>
>                    project saying
>
>                        we need 60 000 USD to do ceph4win with that
>                        monney we will buy the
>
>                    right to
>
>                        use and share a commercial copyrighted library
>                        but abandonned
>
>                    punctually to
>
>                        us in  public domaine and that we will
>                        eventually produce something
>
>                    out of
>
>                        it. I doubt I will get a dollar.
>
>                        We still can suggest the idea to Edlos the
>                        commercial company that
>
>                    has the
>
>                        copyright of Callback FS, Or to buy them their
>                        product in a blender
>
>                    way
>
>                        (blender was bought with donation before being
>                        put opensource and
>
>                    public
>
>                        domaine), Or to open source their library. But
>                        in commercial minds
>                        opensourcing = death of their technical
>                        advantage and death of
>
>                    their
>
>                        marketing strategy. They will have to invent
>                        something more to
>
>                    retrieve
>
>                        monney from it.
>
>                        El nov 6, 2013 11:22 p.m., "Ketor D"
>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>                        <mailto:d.ketor@gmail.com
>                        <mailto:d.ketor@gmail.com>>> escribió:
>
>
>                             Hi Alphe,
>                                       I think you could try Callback
>                        Filesystem dev
>
>                    framework. It
>
>                             is a commerical dev framework and is
>                        maintained by Edlos today.
>                                       I have communicated with Edlos to
>                        get a try code for
>                             development. To dokan, Callback Filesystem
>                        has vary document and
>
>                    maybe
>
>                             more stabilize.
>
>                                       Regards.
>
>
>
>                             On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>                        Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
>                             <mailto:asalas@kepler.cl
>
>                        <mailto:asalas@kepler.cl>>> wrote:
>                              > Hello ketor thank you for your interest
>                        un ceph4win. Since
>
>                    muy
>
>                             first mail I
>                              > exposed the lacks of dokanfs and that I
>                        m far from being a
>                             specialist un
>                              > filesystems.
>                              > I exposed what i like un dokanfs bit I
>                        not a fanátic of it.
>
>                    Muy
>
>                             goal is to
>                              > have something working quickly.
>                              >
>                              > So I am up to any proposición sure the
>                        one with the more docs
>
>                    and
>
>                             support
>                              > will be the best choice. As for right
>                        now what I need is
>                             understand what are
>                              > the files involved what are the
>                        interfaces functions and what
>
>                    are
>
>                             the needed
>                              > library dependencies and if they exist
>                        ported to windows with
>                             cygwin. And
>                              > all that is retrieved from source code.
>                              >
>                              > Regards.
>                              >
>                              > El nov 6, 2013 10:34 p.m., "Ketor D"
>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>                             <mailto:d.ketor@gmail.com
>
>                        <mailto:d.ketor@gmail.com>>> escribió:
>
>                              >
>                              >> Hi Alphe,
>                              >>      We are taking an interest in your
>                        work on Ceph Client
>
>                    for
>
>                             Windows
>                              >> with Dokan.As we know, the performance
>                        of Dokan is not very
>                             good, and it's
>                              >> abandoned 3 years ago.
>                              >>      I have learned and used
>                        OpenDedup(SDFS) for a long
>
>                    time.
>
>                               OpenDedup
>                              >> has a Dokan version. And the author of
>                        OpenDedup said
>                              >>
>                              >> The Dokan library is quite flakey  and
>                        testing should be
>                             performed before
>                              >> putting into production
>                              >>
>                              >>       So what do you think about this?
>                        And if there is
>
>                    another
>
>                             solution of
>                              >> fuse-like filesystem dev framwork on
>                        Windows?
>                              >>
>                              >>        Best Wish!
>                              >>
>                              >>
>                              >>
>                              >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>                        Salas Michels
>                             <asalas@kepler.cl <mailto:asalas@kepler.cl>
>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>
>                              >> wrote:
>                              >>>
>                              >>> Hello I created the github repository
>                        for this project
>                              >>>https://github.com/alphe/Ceph4Win
>                              >>>
>                              >>> Regards,
>                              >>>
>                              >>> signature
>                              >>>
>                              >>> *Alphé Salas*
>                              >>> Ingeniero T.I
>                              >>>
>                              >>>asalas@kepler.cl
>                        <mailto:asalas@kepler.cl>
>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
>
>                              >>> *<http://www.kepler.cl>*
>                              >>>
>                              >>> On 11/05/13 21:00, Sage Weil wrote:
>                              >>>>
>                              >>>> Hi Alphe,
>                              >>>>
>                              >>>> On Tue, 5 Nov 2013, Alphe Salas
>                        Michels wrote:
>                              >>>>>
>                              >>>>> signature *Hi, Sage !
>                              >>>>> thank you for you enthousiast reply.
>                              >>>>> I sure want to make the best use of
>                        everything or
>
>                    anything
>
>                             previously
>                              >>>>> done to
>                              >>>>> tend to
>                              >>>>> write ceph cliente for windows.
>                              >>>>>
>                              >>>>> Apart using libre tools for building
>                        the future ceph
>
>                    cliente
>
>                             I am open
>                              >>>>> to
>                              >>>>> anything.
>                              >>>>> I would recommand eclipse CDT or
>                        Code::BLocks they are
>
>                    based
>
>                             on mingwin
>                              >>>>> open
>                              >>>>> and easyly enhanceable.**
>                              >>>>>
>                              >>>>> more free tools can be found here:
>
>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>                              >>>>>
>                              >>>>>
>                              >>>>> I will read libcephfs source code
>                        and take some notes
>
>                    about the
>
>                              >>>>> protocol.
>                              >>>>
>                              >>>> I think you don't need to worry about
>                        hte protocol at all,
>
>                    since
>
>                              >>>> libcephs
>                              >>>> implements it for you (and will
>                        capture any future
>
>                    changes).
>
>                              >>>>
>                              >>>>> I was more going from what I know
>                        and trying to track down
>
>                    how
>
>                              >>>>> mount.ceph work
>                              >>>>> with the parameters passed to it.
>                              >>>>> since it point finally to
>                        Kernel/fs/ceph and that I don t
>
>                    really
>
>                              >>>>> understand
>                              >>>>> how that module work and that it
>                        probably points to some
>
>                    other
>
>                              >>>>> dependencies
>                              >>>>> Reading libcephfs source code could
>                        be a big gain of
>
>                    time.
>
>                              >>>>
>                              >>>> (I would also ignore mount.ceph as
>                        everything it does it
>                             specific to
>                              >>>> how Linux mounts work.)
>                              >>>>
>                              >>>>> basically on the protocol what is
>                        need are:
>                              >>>>>
>                              >>>>> 1) open and maintain a connection
>                        (socket open, auth, etc
>
>                    )
>
>                              >>>>> 2) retreive a map of directories and
>                        disk Quota (disk
>
>                    sizing
>
>                             Y TB free,
>                              >>>>> Z TB
>                              >>>>> total)
>                              >>>>> 3) procedure to send files /
>                        directories in a maner that
>
>                    it
>
>                             will allow
>                              >>>>> our
>                              >>>>> client to fit ceph transmission
>                        protocols
>                              >>>>> (limit bandwith for stability?,
>                        limit connection amount?,
>                             limit cpu
>                              >>>>> use?,
>                              >>>>> Cache for preparing data transfer (a
>                        FIFO cache)?)
>                              >>>>> 4)Procedure to retreive files /
>                        directory from ceph
>
>                    cluster
>
>                              >>>>> 5) Management copy/move files
>                        /Directories, FS stats,
>                             Connection Stats.
>                              >>>>> logging.
>                              >>>>>
>                              >>>>> My idea to progress is to take those
>                        main bulletpoint in
>
>                    ceph
>
>                             protocol
>                              >>>>> based
>                              >>>>> on general ideas of what ceph file
>                        system does and start
>                             identifying
>                              >>>>> parts
>                              >>>>> from libcephfs to match those "needs".
>                              >>>>
>                              >>>> Instead, I would look at
>                        include/cephfs/libcephfs.h, the
>                             interface that
>                              >>>> libcephfs provides, and try to map
>                        that to what the fuse
>
>                    layer
>
>                             expects.
>                              >>>> There is both a path-based that I
>                        suspsect lends itself
>
>                    well
>
>                             to the
>                              >>>> Windows interface and (very soon now)
>                        a handle based API
>
>                    that is
>
>                              >>>> targetted
>                              >>>> at the Unix-style VFS layers.  I'm
>                        mostly guessing,
>
>                    though,
>
>                             since I've
>                              >>>> never seen any low-level fs code in
>                        windows before.
>                              >>>>
>                              >>>> In this case, the analogous code for
>                        Linux should be
>                             client/fuse_ll.cc
>                              >>>> itself (and not much else), although
>                        there will probably be
>
>                    a
>
>                             few tricks
>                              >>>> necessary to map cleanly onto how the
>                        windows interfaces
>
>                    work.
>
>                              >>>>
>                              >>>> Does that make sense?
>                              >>>>
>                              >>>> Cheers!
>                              >>>> sage
>                              >>>>
>                              >>>>
>                              >>>>> Any suggestion and contributions are
>                        welcome.
>                              >>>>>
>                              >>>>>
>                              >>>>> *
>                              >>>>> On 11/05/13 11:23, Sage Weil wrote:
>                              >>>>>>
>                              >>>>>> Hi Alphe,
>                              >>>>>>
>                              >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>                        Michels wrote:
>                              >>>>>>>
>                              >>>>>>> Good day developers!
>                              >>>>>>>
>                              >>>>>>> I would like to propose to the one
>                        interested  work with
>
>                    me to
>
>                              >>>>>>> develop a
>                              >>>>>>> ceph
>                              >>>>>>> cliente for MS windows world,
>                        Basing us on dokanFS.
>                              >>>>>>>
>                              >>>>>>> My company is a ceph enthousiast
>                        that use on a dayly
>
>                    basis
>
>                             ceph and
>                              >>>>>>> that
>                              >>>>>>> need
>                              >>>>>>> both transfer speed and big
>                        expendable and cheap
>
>                    storage.
>
>                              >>>>>>> My company is specialised in data
>                        recovery and we want
>
>                    to
>
>                             participate
>                              >>>>>>> to
>                              >>>>>>> ceph
>                              >>>>>>> effort by bringing a ceph cliente
>                        for windows.
>                              >>>>>>
>                              >>>>>> Awesome!
>                              >>>>>>
>                              >>>>>>> Our experience shows us that the
>                        best gateway is each
>                             clientes being
>                              >>>>>>> its
>                              >>>>>>> own
>                              >>>>>>> gateway, instead of having a
>                        bottle neck server or a
>
>                    cluster of
>
>                              >>>>>>> bottle
>                              >>>>>>> neck
>                              >>>>>>> servers as gateway (FTP, samba,
>                        SFTP,webdav, s3,
>
>                    etc..).
>
>                              >>>>>>>
>                              >>>>>>> We already did some research in
>                        that domain.
>                              >>>>>>>
>                              >>>>>>> Dokan FS is an intent  to write an
>                        opensource fuse like
>                             cliente for
>                              >>>>>>> MS
>                              >>>>>>> windows.
>                              >>>>>>>
>                              >>>>>>> More information on DOKANFS can be
>                        triggered here
>                         >>>>>>>http://dokan-dev.net/en/download/
>                              >>>>>>>
>                              >>>>>>> Positive points of using DOKANFS.
>                              >>>>>>>
>                              >>>>>>> - its opensourced and well
>                        licenced mit licence, gpl
>                             licence and lgpl
>                              >>>>>>> licence.
>                              >>>>>>>
>                              >>>>>>> Negative point of using DOKAN FS.
>                              >>>>>>> - unreachable author
>                              >>>>>>> - Poor documentation . Dev
>                        comments in japanese.
>                              >>>>>>> - Work in progress so it is
>                        unstable and needs to be
>
>                    updated,
>
>                              >>>>>>> debugged and
>                              >>>>>>> maintained by a MS Windows file
>                        system expert
>
>                    developper.
>
>                              >>>>>>
>                              >>>>>> I am not very familiar with windows
>                        storage APIs, but
>                             somebody told me
>                              >>>>>> at once point there were several
>                        interfaces against which
>
>                    a
>
>                             new file
>                              >>>>>> system could be implemented,
>                        everything from a full
>                             in-kernel driver
>                              >>>>>> to
>                              >>>>>> something that is explorer-based.
>                          Are any of those
>                             suitable?  Using a
>                              >>>>>> potentially abandoned fuse-like
>                        layer makes me a bit
>
>                    nervous.
>
>                              >>>>>>
>                              >>>>>> That said,
>                              >>>>>>
>                              >>>>>>>
>                              >>>>>>> I try past year to do a merge from
>                        ceph-fuse to dokanfs
>                              >>>>>>> here are what I learnt.
>                              >>>>>>> - Ceph-fuse and related source
>                        code is around 60 000
>
>                    lines
>
>                             of code.
>                              >>>>>>> - Ceph protocol isn t documented
>                        so it is like trying
>
>                    to
>
>                             draw a map
>                              >>>>>>> of
>                              >>>>>>> america
>                              >>>>>>> using only a sextan and a compass.
>                              >>>>>>>
>                              >>>>>>> Those led me to those conclusions:
>                              >>>>>>> - I can t do it alone.
>                              >>>>>>> - It is easier to draw down the
>                        ceph protocol way to
>
>                    work from
>
>                              >>>>>>> kernel/fs/ceph
>                              >>>>>>> sources and mount.ceph
>                              >>>>>>> - Ceph depending libraries may be
>                        unexistant or not up
>
>                    to
>
>                             date in
>                              >>>>>>> their
>                              >>>>>>> port
>                              >>>>>>> on MS Windows (cygwin)
>                              >>>>>>
>                              >>>>>> I think the most sane path should
>                        be to make libcephfs
>                             sufficiently
>                              >>>>>> portable to build on windows (or
>                        cygwin).  For the bits
>
>                    used
>
>                             by the
>                              >>>>>> client-side coe, I don't think
>                        there should be much in
>
>                    the
>
>                             way of
>                              >>>>>> dependencies, and the main
>                        challenge would be untangling
>
>                    the
>
>                             build for
>                              >>>>>> the necessary pieces out from the
>                        rest of Ceph.
>                              >>>>>>
>                              >>>>>> Have you seen the wip-port
>                        portability work that is
>                             currently underway
>                              >>>>>> by
>                              >>>>>> Noah and Alan?  That may solve many
>                        of the cygwin
>
>                    problems
>
>                             you are
>                              >>>>>> seeing
>                              >>>>>> today.
>                              >>>>>>
>                              >>>>>>> - MS file system specialist are
>                        hard do find in the
>
>                    "open
>
>                             source
>                              >>>>>>> libre
>                              >>>>>>> world"
>                              >>>>>>> so I will try in the commercial world.
>                              >>>>>>>
>                              >>>>>>> The commercial world has some
>                        problems too. They need
>
>                    ceph
>
>                             protocol
>                              >>>>>>> draft
>                              >>>>>>> to
>                              >>>>>>> implemente it to their own product
>                        They will have
>
>                    licencing
>
>                              >>>>>>> /commercial
>                              >>>>>>> politics that infringe lgpl, and
>                        hide that most of the
>
>                    work
>
>                             is done
>                              >>>>>>> by
>                              >>>>>>> people
>                              >>>>>>> other than them. They will not
>                        participate in a
>
>                    financial
>
>                             way to ceph
>                              >>>>>>> enhancement and growth.
>                              >>>>>>
>                              >>>>>> I don't think reimplementing the
>                        client code is an
>
>                    efficient way
>
>                              >>>>>> forward.
>                              >>>>>> Unless the goal is a pure kernel
>                        implementation...but a
>                             significant
>                              >>>>>> ongoing investment in development
>                        resources would be
>
>                    needed
>
>                             for that
>                              >>>>>> going
>                              >>>>>> forward.  I suspect that is a
>                        challenge for a platform
>
>                    that
>
>                             does not
>                              >>>>>> typically rally that sort of
>                        community effort.
>                              >>>>>>
>                              >>>>>> The easiest thing is of course just
>                        to use CIFS and
>
>                    Samba
>
>                             (which works
>                              >>>>>> today).  A fuse-like approach is
>                        probably a reasonably
>                             middle ground
>                              >>>>>> (both
>                              >>>>>> in initial effort and
>                        maintainability going forward)...
>                              >>>>>>
>                              >>>>>> sage
>                              >>>>>>
>                              >>>>>>
>                              >>>>>
>                              >>>
>                              >>> --
>                              >>> To unsubscribe from this list: send
>                        the line "unsubscribe
>                             ceph-devel" in
>                              >>> the body of a message
>                        tomajordomo@vger.kernel.org
>                        <mailto:tomajordomo@vger.kernel.org>
>                             <mailto:majordomo@vger.kernel.org
>                        <mailto:majordomo@vger.kernel.org>>
>
>                              >>> More majordomo info at
>
>                    http://vger.kernel.org/majordomo-info.html
>
>                              >>
>                              >>
>                              >
>
>
>
>                    --
>                    To unsubscribe from this list: send the line
>                    "unsubscribe ceph-devel"
>                    in
>                    the body of a message tomajordomo@vger.kernel.org
>                    <mailto:tomajordomo@vger.kernel.org>
>                    More majordomo info
>                    athttp://vger.kernel.org/majordomo-info.html
>                    <http://vger.kernel.org/majordomo-info.html>
>
>
>
>        --
>        To unsubscribe from this list: send the line "unsubscribe
>        ceph-devel" in
>        the body of a message to majordomo@vger.kernel.org
>        <mailto:majordomo@vger.kernel.org>
>        More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Dong Yuan]

Why don't you push your impl? Maybe I can have a BP first.

On 27 December 2014 at 01:10, Ketor D <d.ketor@gmail.com> wrote:
> Is here any progress on Cephfs Windows Client ?
>
> 2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
>> Hello malcom and matt thank you for apporting some more information source.
>> OpenAFS is sure interesting httpfs too.
>>
>> I hope it will help us on deciding the best path to follow in our interface
>> with window.
>> Actually I still trying to isolate the needed client code in the shortest
>> way possible.
>>
>> Regards.
>>
>> Alphe Salas
>>
>> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
>> <mailto:malcolm@sgi.com>> escribió:
>>
>>    I'm just going to throw these in there.
>>
>>    http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>>
>>
>>    They are GPLv2 some already use sockets and such from inside the
>>    kernel.  Heck you might even be able to mod the HTTP one to use
>>    rados gateway. I don't know as I havent sat down and pulled them
>>    apart enough yet.
>>
>>    They might help, but they might be useless. Not sure.
>>
>>    On 08/11/13 06:47, Alphe Salas Michels wrote:
>>
>>        Hello all I finally finished my first source code extraction
>>        that starts
>>        from ceph/src/client/fuse_ll.c
>>        The result is accurate unlike previous provided results.
>>        basically the
>>        script start from a file extract all the private includes
>>        definitions
>>        #include "something.h" and recursively extract private includes
>>        too. the
>>        best way to know who is related with who.
>>
>>        starting from fuse_ll.cc I optain 390 files retreived and 120
>>        000 lines
>>        of code !
>>        involved dirs are : in ceph/src
>>        objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>>        global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>>
>>        probably not a good way to analyse what amount of work it means
>>        since
>>        most of those directories are the implementation of servers
>>        (osd, mon,
>>        mds) and even if only a tiny bit of them is needed at client
>>        level. you
>>        need two structures from ./osd/OSD.h and  my script by relation will
>>        take into acount the whole directory...
>>
>>        I ran the script with libcephfs.cc as start point and got almost the
>>        same results. 131 000 lines of code and 386 files most of the
>>        same dirs
>>        involved.
>>
>>
>>
>>        I think I will spend alot of time doing the manual source code
>>        isolation
>>        and understand way each #include is set in the files I read (what
>>        purpose they have do they allow to integrate a crucial data type
>>        or not.
>>
>>
>>        The other way around will be to read src/libcephfs.cc. It seems
>>        shorter
>>        but without understanding what part is used for each included
>>        header I
>>        can t say anything...
>>
>>
>>
>>        I will keep reading the source code and take notes. I think in
>>        the case
>>        of libcephfs I will gain alot of time.
>>
>>        signature
>>
>>        *Alphé Salas*
>>        Ingeniero T.I
>>
>>        asalas@kepler.cl <mailto:asalas@kepler.cl>
>>        *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>>
>>
>>        On 11/07/13 15:02, Alphe Salas Michels wrote:
>>
>>            Hello D.Ketor and Matt Benjamin,
>>            You give me alot to think about and this is great!
>>            I merged your previous post to make a single reply that
>>            anyone can
>>            report to easyly
>>
>>            Windows NFS 4.1 is available here:
>>            http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>
>>            pnfs is another name for NFS4.X. It is presented as
>>            alternative to
>>            ceph and we get known terminology as MDS and OSD but without
>>            the self
>>            healing part if I understand well my rapid look on the
>>            topic. (when I
>>            say rapid look I mean ... 5 minutes spent in that... which
>>            is really
>>            small amount of time to get an accurate view on something)
>>
>>
>>            starting from mount.ceph ... I know that mount.ceph does
>>            little but it
>>            is a great hint to know what ceph needs and do things.
>>            Basically mount.ceph modprobe the ceph driver in the linux
>>            kernel then
>>            call mount with the line command passed args and the cephfs
>>            type as
>>            argument. Then the kernel does the work I don t understand
>>            yet what is
>>            the start calls that are made to the ceph driver but it
>>            seemed to me
>>            that is was relatively light. (a first impression compared
>>            to ceph-fuse.)
>>
>>            I think I will do both isolate source code from ceph-client
>>            kernel
>>            (cephfs module for linux kernel) and the one pointed by Sage
>>            starting
>>            from client/fuse_ll.cc in ceph master branch. The common
>>            files betwin
>>            those 2 extractions will be our core set of mandatory features.
>>
>>            Then we try to compile with cygwin a cephfs client library .
>>            Then we
>>            will try to interface with a modified windows nfs 4.1 client
>>            or pnfs
>>            or any other that will accept to be compiled with gcc for
>>            win32...
>>
>>            the fact that windows 8.1 is and windows 2012 are out of
>>            reach at the
>>            moment is not a problem to me.
>>
>>            Our first concern is to understand what is ceph protocol.
>>            Then adapt
>>            it to something that can be used on windows prior windows
>>            8.1. Dokan
>>            fs if I remember well use too the WDK (windows driver
>>            dev-kit ) for it
>>            s compilation so possibly we will see the same limitations.
>>
>>            We need to multiply our source of information by example
>>            regarding
>>            ceph-client (kernel or fuse, radosgw is on a different layer
>>            so I will
>>            not try anything around it at first.) And we need to
>>            multiply our
>>            source of information by example regarding virtual file system
>>            technologies on windoes OS.
>>            Alot of work but all of those available source code everyone
>>            point at
>>            me will make our best solution. And in the end we will choose
>>            technologies knowing what we do and what concequencies they
>>            have.
>>
>>            regards,
>>
>>
>>
>>
>>            Regards
>>
>>            signature
>>
>>            *Alphé Salas*
>>            Ingeniero T.I
>>
>>            asalas@kepler.cl <mailto:asalas@kepler.cl>
>>
>>
>>
>>            On 11/07/13 11:29, Ketor D wrote:
>>
>>                Hi Alphe:
>>                        Yes Callback Filesystem is very expensive and
>>                can't open source.
>>                It's not a good choice for ceph4win.
>>                        Another way for ceph4win maybe develop a
>>                kernel-mode fs like
>>                pnfs. pnfs has a kernel-mode windows client. I think you
>>                can read its
>>                src code and maybe migrating from ceph kernel client to
>>                windows kernel
>>                fs is easier than from userspace ceph fuse client.And a
>>                kernel-mode fs
>>                client has greater performance than userspace fs like
>>                ceph-fuse client
>>                and ceph kernel client.
>>
>>                        Regards.
>>
>>            On 11/07/13 11:50, Matt W. Benjamin wrote:
>>
>>                Hi,
>>
>>                The Window NFS v4.1 client is what we work on, so this
>>                may be good for
>>                code sharing.  The license is lgplv2, like Ceph's.
>>
>>                Something important to be aware of is that the client
>>                uses rdbss, which
>>                is a (partial) fsd abstraction that simplified
>>                implementation
>>                quite a bit, kind of like a mini driver.  However,
>>                Microsoft's support
>>                for rdbss has been in limbo for a bit.  For example, to
>>                link with
>>                the rdbss symbols you can't use the Windows 8 driver
>>                kit--you'll need
>>                to use the one for Windows 7.  (There's a private rdbss2
>>                used internally
>>                by Microsoft's SMB implemenation.  A the moment, 3rd
>>                party drivers
>>                can't use that.)
>>
>>                We've been in communication with Microsoft about this
>>                issue, and know of
>>                a few other fsds using it, but it could be a good thing
>>                for that
>>                lobbying
>>                effort to have another user--or it could be a dead end :(.
>>
>>                There are a couple of other choices if you're looking to
>>                go this route,
>>                that I'm aware of (and we may need to take them too, if
>>                RDBSS has no
>>                way forward), but the required work could be a lot larger.
>>
>>                Matt
>>
>>                ----- "Ketor D"<d.ketor@gmail.com
>>                <mailto:d.ketor@gmail.com>>  wrote:
>>
>>                    Hi Alphe:
>>                            Yes Callback Filesystem is very expensive
>>                    and can't open
>>                    source.
>>                    It's not a good choice for ceph4win.
>>                            Another way for ceph4win maybe develop a
>>                    kernel-mode fs like
>>                    pnfs. pnfs has a kernel-mode windows client. I think
>>                    you can read its
>>                    src code and maybe migrating from ceph kernel client
>>                    to windows
>>                    kernel
>>                    fs is easier than from userspace ceph fuse
>>                    client.And a kernel-mode
>>                    fs
>>                    client has greater performance than userspace fs
>>                    like ceph-fuse
>>                    client
>>                    and ceph kernel client.
>>
>>                            Regards.
>>
>>
>>
>>                    On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>>                    Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>
>>                    wrote:
>>
>>                        Commercial libraries are a pain ...
>>
>>                        If we want the more permossive licence offered
>>                        by callback file
>>
>>                    system we
>>
>>                        have to buy it for 20.000 usd. Then we will have
>>                        to provide a
>>
>>                    backbox that
>>
>>                        we have no control upon and that will kill our
>>                        product anytime they
>>
>>                    want anf
>>
>>                        if they decide to stop their commercial activity
>>                        we will be in the
>>
>>                    same
>>
>>                        situation that with dokanfs but without having
>>                        the source code of
>>
>>                    the black
>>
>>                        box. If i have to spend 20 000 dollars i would
>>                        prefere paying
>>
>>                    someone to
>>
>>                        retake dokanfs or to write from scratch a
>>                        dokanfs fuselike software
>>
>>                    make it
>>
>>                        all shiny and pumpy fantastic and ready to plug
>>                        to ceph client.
>>
>>                        I would prefere if people have to pay something
>>                        to get access to
>>
>>                    ceph4win
>>
>>                        that this money goes in ceph main branch
>>                        pockets... Or as a gift you
>>
>>                    donante
>>
>>                        to ceph 10 dollars  you get 2 free registration
>>                        codes for
>>
>>                    ceph4win... or
>>
>>                        something like that.
>>
>>                        If ceph4win as to be comercial then I would
>>                        prefer delegate the task
>>
>>                    to a
>>
>>                        company like south river technologies and their
>>                        great product
>>
>>                    webdrive. I
>>
>>                        would mininaly get involved in that project and
>>                        simply buy the final
>>
>>                    product
>>
>>                        to sell it together with my ceph based product
>>                        (which could be a
>>
>>                    calxeda
>>
>>                        ceph box or something like that).
>>
>>                        I m open anyway to any proposition. But I doubt
>>                        that callback
>>
>>                    filesystem
>>
>>                        offers us a suitable solution in the way I see
>>                        ceph4win to be spread
>>
>>                    and
>>
>>                        used... I m maybe wrong. And anything that will
>>                        be done around
>>
>>                    ceph4win will
>>
>>                        be public documented etc... And licensed the way
>>                        that if someone
>>
>>                    want to
>>
>>                        build a commercial solution on top of it, that
>>                        would be a
>>
>>                    possibility.
>>
>>                        My idea is to giveback somehow to ceph project
>>                        and at same time
>>
>>                    forge a
>>
>>                        better knowledge in ceph technologies. Because
>>                        like many in libre
>>
>>                    world I
>>
>>                        think the business is in the services around the
>>                        software more than
>>
>>                    on the
>>
>>                        software. That the ones writing code should be
>>                        financed and benefits
>>
>>                    from
>>
>>                        the one selling and giving support of the
>>                        software at all levels. I
>>
>>                    m
>>
>>                        probably too idealistic. And too optimistic
>>                        after all I m the one
>>
>>                    saying I
>>
>>                        will do this stuff I have no idea how but well
>>                        it is interesting and
>>
>>                    fun so
>>
>>                        lets do it.
>>
>>                        Regards,
>>
>>                        P.S: using commercial backend libraries appart
>>                        including their own
>>
>>                    cost will
>>
>>                        force you to use commercial IDE like MS
>>                        VisualStudio because their
>>
>>                    library
>>
>>                        has some kind of drm that only that IDE compiler
>>                        can use. So alot of
>>
>>                    cost
>>
>>                        and yet there is nothing done. If I had to open
>>                        a kickstarter
>>
>>                    project saying
>>
>>                        we need 60 000 USD to do ceph4win with that
>>                        monney we will buy the
>>
>>                    right to
>>
>>                        use and share a commercial copyrighted library
>>                        but abandonned
>>
>>                    punctually to
>>
>>                        us in  public domaine and that we will
>>                        eventually produce something
>>
>>                    out of
>>
>>                        it. I doubt I will get a dollar.
>>
>>                        We still can suggest the idea to Edlos the
>>                        commercial company that
>>
>>                    has the
>>
>>                        copyright of Callback FS, Or to buy them their
>>                        product in a blender
>>
>>                    way
>>
>>                        (blender was bought with donation before being
>>                        put opensource and
>>
>>                    public
>>
>>                        domaine), Or to open source their library. But
>>                        in commercial minds
>>                        opensourcing = death of their technical
>>                        advantage and death of
>>
>>                    their
>>
>>                        marketing strategy. They will have to invent
>>                        something more to
>>
>>                    retrieve
>>
>>                        monney from it.
>>
>>                        El nov 6, 2013 11:22 p.m., "Ketor D"
>>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>                        <mailto:d.ketor@gmail.com
>>                        <mailto:d.ketor@gmail.com>>> escribió:
>>
>>
>>                             Hi Alphe,
>>                                       I think you could try Callback
>>                        Filesystem dev
>>
>>                    framework. It
>>
>>                             is a commerical dev framework and is
>>                        maintained by Edlos today.
>>                                       I have communicated with Edlos to
>>                        get a try code for
>>                             development. To dokan, Callback Filesystem
>>                        has vary document and
>>
>>                    maybe
>>
>>                             more stabilize.
>>
>>                                       Regards.
>>
>>
>>
>>                             On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>>                        Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>                             <mailto:asalas@kepler.cl
>>
>>                        <mailto:asalas@kepler.cl>>> wrote:
>>                              > Hello ketor thank you for your interest
>>                        un ceph4win. Since
>>
>>                    muy
>>
>>                             first mail I
>>                              > exposed the lacks of dokanfs and that I
>>                        m far from being a
>>                             specialist un
>>                              > filesystems.
>>                              > I exposed what i like un dokanfs bit I
>>                        not a fanátic of it.
>>
>>                    Muy
>>
>>                             goal is to
>>                              > have something working quickly.
>>                              >
>>                              > So I am up to any proposición sure the
>>                        one with the more docs
>>
>>                    and
>>
>>                             support
>>                              > will be the best choice. As for right
>>                        now what I need is
>>                             understand what are
>>                              > the files involved what are the
>>                        interfaces functions and what
>>
>>                    are
>>
>>                             the needed
>>                              > library dependencies and if they exist
>>                        ported to windows with
>>                             cygwin. And
>>                              > all that is retrieved from source code.
>>                              >
>>                              > Regards.
>>                              >
>>                              > El nov 6, 2013 10:34 p.m., "Ketor D"
>>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>                             <mailto:d.ketor@gmail.com
>>
>>                        <mailto:d.ketor@gmail.com>>> escribió:
>>
>>                              >
>>                              >> Hi Alphe,
>>                              >>      We are taking an interest in your
>>                        work on Ceph Client
>>
>>                    for
>>
>>                             Windows
>>                              >> with Dokan.As we know, the performance
>>                        of Dokan is not very
>>                             good, and it's
>>                              >> abandoned 3 years ago.
>>                              >>      I have learned and used
>>                        OpenDedup(SDFS) for a long
>>
>>                    time.
>>
>>                               OpenDedup
>>                              >> has a Dokan version. And the author of
>>                        OpenDedup said
>>                              >>
>>                              >> The Dokan library is quite flakey  and
>>                        testing should be
>>                             performed before
>>                              >> putting into production
>>                              >>
>>                              >>       So what do you think about this?
>>                        And if there is
>>
>>                    another
>>
>>                             solution of
>>                              >> fuse-like filesystem dev framwork on
>>                        Windows?
>>                              >>
>>                              >>        Best Wish!
>>                              >>
>>                              >>
>>                              >>
>>                              >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>>                        Salas Michels
>>                             <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>>
>>                              >> wrote:
>>                              >>>
>>                              >>> Hello I created the github repository
>>                        for this project
>>                              >>>https://github.com/alphe/Ceph4Win
>>                              >>>
>>                              >>> Regards,
>>                              >>>
>>                              >>> signature
>>                              >>>
>>                              >>> *Alphé Salas*
>>                              >>> Ingeniero T.I
>>                              >>>
>>                              >>>asalas@kepler.cl
>>                        <mailto:asalas@kepler.cl>
>>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>
>>
>>                              >>> *<http://www.kepler.cl>*
>>                              >>>
>>                              >>> On 11/05/13 21:00, Sage Weil wrote:
>>                              >>>>
>>                              >>>> Hi Alphe,
>>                              >>>>
>>                              >>>> On Tue, 5 Nov 2013, Alphe Salas
>>                        Michels wrote:
>>                              >>>>>
>>                              >>>>> signature *Hi, Sage !
>>                              >>>>> thank you for you enthousiast reply.
>>                              >>>>> I sure want to make the best use of
>>                        everything or
>>
>>                    anything
>>
>>                             previously
>>                              >>>>> done to
>>                              >>>>> tend to
>>                              >>>>> write ceph cliente for windows.
>>                              >>>>>
>>                              >>>>> Apart using libre tools for building
>>                        the future ceph
>>
>>                    cliente
>>
>>                             I am open
>>                              >>>>> to
>>                              >>>>> anything.
>>                              >>>>> I would recommand eclipse CDT or
>>                        Code::BLocks they are
>>
>>                    based
>>
>>                             on mingwin
>>                              >>>>> open
>>                              >>>>> and easyly enhanceable.**
>>                              >>>>>
>>                              >>>>> more free tools can be found here:
>>
>>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>                              >>>>>
>>                              >>>>>
>>                              >>>>> I will read libcephfs source code
>>                        and take some notes
>>
>>                    about the
>>
>>                              >>>>> protocol.
>>                              >>>>
>>                              >>>> I think you don't need to worry about
>>                        hte protocol at all,
>>
>>                    since
>>
>>                              >>>> libcephs
>>                              >>>> implements it for you (and will
>>                        capture any future
>>
>>                    changes).
>>
>>                              >>>>
>>                              >>>>> I was more going from what I know
>>                        and trying to track down
>>
>>                    how
>>
>>                              >>>>> mount.ceph work
>>                              >>>>> with the parameters passed to it.
>>                              >>>>> since it point finally to
>>                        Kernel/fs/ceph and that I don t
>>
>>                    really
>>
>>                              >>>>> understand
>>                              >>>>> how that module work and that it
>>                        probably points to some
>>
>>                    other
>>
>>                              >>>>> dependencies
>>                              >>>>> Reading libcephfs source code could
>>                        be a big gain of
>>
>>                    time.
>>
>>                              >>>>
>>                              >>>> (I would also ignore mount.ceph as
>>                        everything it does it
>>                             specific to
>>                              >>>> how Linux mounts work.)
>>                              >>>>
>>                              >>>>> basically on the protocol what is
>>                        need are:
>>                              >>>>>
>>                              >>>>> 1) open and maintain a connection
>>                        (socket open, auth, etc
>>
>>                    )
>>
>>                              >>>>> 2) retreive a map of directories and
>>                        disk Quota (disk
>>
>>                    sizing
>>
>>                             Y TB free,
>>                              >>>>> Z TB
>>                              >>>>> total)
>>                              >>>>> 3) procedure to send files /
>>                        directories in a maner that
>>
>>                    it
>>
>>                             will allow
>>                              >>>>> our
>>                              >>>>> client to fit ceph transmission
>>                        protocols
>>                              >>>>> (limit bandwith for stability?,
>>                        limit connection amount?,
>>                             limit cpu
>>                              >>>>> use?,
>>                              >>>>> Cache for preparing data transfer (a
>>                        FIFO cache)?)
>>                              >>>>> 4)Procedure to retreive files /
>>                        directory from ceph
>>
>>                    cluster
>>
>>                              >>>>> 5) Management copy/move files
>>                        /Directories, FS stats,
>>                             Connection Stats.
>>                              >>>>> logging.
>>                              >>>>>
>>                              >>>>> My idea to progress is to take those
>>                        main bulletpoint in
>>
>>                    ceph
>>
>>                             protocol
>>                              >>>>> based
>>                              >>>>> on general ideas of what ceph file
>>                        system does and start
>>                             identifying
>>                              >>>>> parts
>>                              >>>>> from libcephfs to match those "needs".
>>                              >>>>
>>                              >>>> Instead, I would look at
>>                        include/cephfs/libcephfs.h, the
>>                             interface that
>>                              >>>> libcephfs provides, and try to map
>>                        that to what the fuse
>>
>>                    layer
>>
>>                             expects.
>>                              >>>> There is both a path-based that I
>>                        suspsect lends itself
>>
>>                    well
>>
>>                             to the
>>                              >>>> Windows interface and (very soon now)
>>                        a handle based API
>>
>>                    that is
>>
>>                              >>>> targetted
>>                              >>>> at the Unix-style VFS layers.  I'm
>>                        mostly guessing,
>>
>>                    though,
>>
>>                             since I've
>>                              >>>> never seen any low-level fs code in
>>                        windows before.
>>                              >>>>
>>                              >>>> In this case, the analogous code for
>>                        Linux should be
>>                             client/fuse_ll.cc
>>                              >>>> itself (and not much else), although
>>                        there will probably be
>>
>>                    a
>>
>>                             few tricks
>>                              >>>> necessary to map cleanly onto how the
>>                        windows interfaces
>>
>>                    work.
>>
>>                              >>>>
>>                              >>>> Does that make sense?
>>                              >>>>
>>                              >>>> Cheers!
>>                              >>>> sage
>>                              >>>>
>>                              >>>>
>>                              >>>>> Any suggestion and contributions are
>>                        welcome.
>>                              >>>>>
>>                              >>>>>
>>                              >>>>> *
>>                              >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>                              >>>>>>
>>                              >>>>>> Hi Alphe,
>>                              >>>>>>
>>                              >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>>                        Michels wrote:
>>                              >>>>>>>
>>                              >>>>>>> Good day developers!
>>                              >>>>>>>
>>                              >>>>>>> I would like to propose to the one
>>                        interested  work with
>>
>>                    me to
>>
>>                              >>>>>>> develop a
>>                              >>>>>>> ceph
>>                              >>>>>>> cliente for MS windows world,
>>                        Basing us on dokanFS.
>>                              >>>>>>>
>>                              >>>>>>> My company is a ceph enthousiast
>>                        that use on a dayly
>>
>>                    basis
>>
>>                             ceph and
>>                              >>>>>>> that
>>                              >>>>>>> need
>>                              >>>>>>> both transfer speed and big
>>                        expendable and cheap
>>
>>                    storage.
>>
>>                              >>>>>>> My company is specialised in data
>>                        recovery and we want
>>
>>                    to
>>
>>                             participate
>>                              >>>>>>> to
>>                              >>>>>>> ceph
>>                              >>>>>>> effort by bringing a ceph cliente
>>                        for windows.
>>                              >>>>>>
>>                              >>>>>> Awesome!
>>                              >>>>>>
>>                              >>>>>>> Our experience shows us that the
>>                        best gateway is each
>>                             clientes being
>>                              >>>>>>> its
>>                              >>>>>>> own
>>                              >>>>>>> gateway, instead of having a
>>                        bottle neck server or a
>>
>>                    cluster of
>>
>>                              >>>>>>> bottle
>>                              >>>>>>> neck
>>                              >>>>>>> servers as gateway (FTP, samba,
>>                        SFTP,webdav, s3,
>>
>>                    etc..).
>>
>>                              >>>>>>>
>>                              >>>>>>> We already did some research in
>>                        that domain.
>>                              >>>>>>>
>>                              >>>>>>> Dokan FS is an intent  to write an
>>                        opensource fuse like
>>                             cliente for
>>                              >>>>>>> MS
>>                              >>>>>>> windows.
>>                              >>>>>>>
>>                              >>>>>>> More information on DOKANFS can be
>>                        triggered here
>>                         >>>>>>>http://dokan-dev.net/en/download/
>>                              >>>>>>>
>>                              >>>>>>> Positive points of using DOKANFS.
>>                              >>>>>>>
>>                              >>>>>>> - its opensourced and well
>>                        licenced mit licence, gpl
>>                             licence and lgpl
>>                              >>>>>>> licence.
>>                              >>>>>>>
>>                              >>>>>>> Negative point of using DOKAN FS.
>>                              >>>>>>> - unreachable author
>>                              >>>>>>> - Poor documentation . Dev
>>                        comments in japanese.
>>                              >>>>>>> - Work in progress so it is
>>                        unstable and needs to be
>>
>>                    updated,
>>
>>                              >>>>>>> debugged and
>>                              >>>>>>> maintained by a MS Windows file
>>                        system expert
>>
>>                    developper.
>>
>>                              >>>>>>
>>                              >>>>>> I am not very familiar with windows
>>                        storage APIs, but
>>                             somebody told me
>>                              >>>>>> at once point there were several
>>                        interfaces against which
>>
>>                    a
>>
>>                             new file
>>                              >>>>>> system could be implemented,
>>                        everything from a full
>>                             in-kernel driver
>>                              >>>>>> to
>>                              >>>>>> something that is explorer-based.
>>                          Are any of those
>>                             suitable?  Using a
>>                              >>>>>> potentially abandoned fuse-like
>>                        layer makes me a bit
>>
>>                    nervous.
>>
>>                              >>>>>>
>>                              >>>>>> That said,
>>                              >>>>>>
>>                              >>>>>>>
>>                              >>>>>>> I try past year to do a merge from
>>                        ceph-fuse to dokanfs
>>                              >>>>>>> here are what I learnt.
>>                              >>>>>>> - Ceph-fuse and related source
>>                        code is around 60 000
>>
>>                    lines
>>
>>                             of code.
>>                              >>>>>>> - Ceph protocol isn t documented
>>                        so it is like trying
>>
>>                    to
>>
>>                             draw a map
>>                              >>>>>>> of
>>                              >>>>>>> america
>>                              >>>>>>> using only a sextan and a compass.
>>                              >>>>>>>
>>                              >>>>>>> Those led me to those conclusions:
>>                              >>>>>>> - I can t do it alone.
>>                              >>>>>>> - It is easier to draw down the
>>                        ceph protocol way to
>>
>>                    work from
>>
>>                              >>>>>>> kernel/fs/ceph
>>                              >>>>>>> sources and mount.ceph
>>                              >>>>>>> - Ceph depending libraries may be
>>                        unexistant or not up
>>
>>                    to
>>
>>                             date in
>>                              >>>>>>> their
>>                              >>>>>>> port
>>                              >>>>>>> on MS Windows (cygwin)
>>                              >>>>>>
>>                              >>>>>> I think the most sane path should
>>                        be to make libcephfs
>>                             sufficiently
>>                              >>>>>> portable to build on windows (or
>>                        cygwin).  For the bits
>>
>>                    used
>>
>>                             by the
>>                              >>>>>> client-side coe, I don't think
>>                        there should be much in
>>
>>                    the
>>
>>                             way of
>>                              >>>>>> dependencies, and the main
>>                        challenge would be untangling
>>
>>                    the
>>
>>                             build for
>>                              >>>>>> the necessary pieces out from the
>>                        rest of Ceph.
>>                              >>>>>>
>>                              >>>>>> Have you seen the wip-port
>>                        portability work that is
>>                             currently underway
>>                              >>>>>> by
>>                              >>>>>> Noah and Alan?  That may solve many
>>                        of the cygwin
>>
>>                    problems
>>
>>                             you are
>>                              >>>>>> seeing
>>                              >>>>>> today.
>>                              >>>>>>
>>                              >>>>>>> - MS file system specialist are
>>                        hard do find in the
>>
>>                    "open
>>
>>                             source
>>                              >>>>>>> libre
>>                              >>>>>>> world"
>>                              >>>>>>> so I will try in the commercial world.
>>                              >>>>>>>
>>                              >>>>>>> The commercial world has some
>>                        problems too. They need
>>
>>                    ceph
>>
>>                             protocol
>>                              >>>>>>> draft
>>                              >>>>>>> to
>>                              >>>>>>> implemente it to their own product
>>                        They will have
>>
>>                    licencing
>>
>>                              >>>>>>> /commercial
>>                              >>>>>>> politics that infringe lgpl, and
>>                        hide that most of the
>>
>>                    work
>>
>>                             is done
>>                              >>>>>>> by
>>                              >>>>>>> people
>>                              >>>>>>> other than them. They will not
>>                        participate in a
>>
>>                    financial
>>
>>                             way to ceph
>>                              >>>>>>> enhancement and growth.
>>                              >>>>>>
>>                              >>>>>> I don't think reimplementing the
>>                        client code is an
>>
>>                    efficient way
>>
>>                              >>>>>> forward.
>>                              >>>>>> Unless the goal is a pure kernel
>>                        implementation...but a
>>                             significant
>>                              >>>>>> ongoing investment in development
>>                        resources would be
>>
>>                    needed
>>
>>                             for that
>>                              >>>>>> going
>>                              >>>>>> forward.  I suspect that is a
>>                        challenge for a platform
>>
>>                    that
>>
>>                             does not
>>                              >>>>>> typically rally that sort of
>>                        community effort.
>>                              >>>>>>
>>                              >>>>>> The easiest thing is of course just
>>                        to use CIFS and
>>
>>                    Samba
>>
>>                             (which works
>>                              >>>>>> today).  A fuse-like approach is
>>                        probably a reasonably
>>                             middle ground
>>                              >>>>>> (both
>>                              >>>>>> in initial effort and
>>                        maintainability going forward)...
>>                              >>>>>>
>>                              >>>>>> sage
>>                              >>>>>>
>>                              >>>>>>
>>                              >>>>>
>>                              >>>
>>                              >>> --
>>                              >>> To unsubscribe from this list: send
>>                        the line "unsubscribe
>>                             ceph-devel" in
>>                              >>> the body of a message
>>                        tomajordomo@vger.kernel.org
>>                        <mailto:tomajordomo@vger.kernel.org>
>>                             <mailto:majordomo@vger.kernel.org
>>                        <mailto:majordomo@vger.kernel.org>>
>>
>>                              >>> More majordomo info at
>>
>>                    http://vger.kernel.org/majordomo-info.html
>>
>>                              >>
>>                              >>
>>                              >
>>
>>
>>
>>                    --
>>                    To unsubscribe from this list: send the line
>>                    "unsubscribe ceph-devel"
>>                    in
>>                    the body of a message tomajordomo@vger.kernel.org
>>                    <mailto:tomajordomo@vger.kernel.org>
>>                    More majordomo info
>>                    athttp://vger.kernel.org/majordomo-info.html
>>                    <http://vger.kernel.org/majordomo-info.html>
>>
>>
>>
>>        --
>>        To unsubscribe from this list: send the line "unsubscribe
>>        ceph-devel" in
>>        the body of a message to majordomo@vger.kernel.org
>>        <mailto:majordomo@vger.kernel.org>
>>        More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Dong Yuan
Email:yuandong1222@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Dong Yuan]

Sorry, I mean YOU can have a BP first.

On 27 December 2014 at 13:14, Dong Yuan <yuandong1222@gmail.com> wrote:
> Why don't you push your impl? Maybe I can have a BP first.
>
> On 27 December 2014 at 01:10, Ketor D <d.ketor@gmail.com> wrote:
>> Is here any progress on Cephfs Windows Client ?
>>
>> 2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
>>> Hello malcom and matt thank you for apporting some more information source.
>>> OpenAFS is sure interesting httpfs too.
>>>
>>> I hope it will help us on deciding the best path to follow in our interface
>>> with window.
>>> Actually I still trying to isolate the needed client code in the shortest
>>> way possible.
>>>
>>> Regards.
>>>
>>> Alphe Salas
>>>
>>> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
>>> <mailto:malcolm@sgi.com>> escribió:
>>>
>>>    I'm just going to throw these in there.
>>>
>>>    http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>>>
>>>
>>>    They are GPLv2 some already use sockets and such from inside the
>>>    kernel.  Heck you might even be able to mod the HTTP one to use
>>>    rados gateway. I don't know as I havent sat down and pulled them
>>>    apart enough yet.
>>>
>>>    They might help, but they might be useless. Not sure.
>>>
>>>    On 08/11/13 06:47, Alphe Salas Michels wrote:
>>>
>>>        Hello all I finally finished my first source code extraction
>>>        that starts
>>>        from ceph/src/client/fuse_ll.c
>>>        The result is accurate unlike previous provided results.
>>>        basically the
>>>        script start from a file extract all the private includes
>>>        definitions
>>>        #include "something.h" and recursively extract private includes
>>>        too. the
>>>        best way to know who is related with who.
>>>
>>>        starting from fuse_ll.cc I optain 390 files retreived and 120
>>>        000 lines
>>>        of code !
>>>        involved dirs are : in ceph/src
>>>        objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>>>        global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>>>
>>>        probably not a good way to analyse what amount of work it means
>>>        since
>>>        most of those directories are the implementation of servers
>>>        (osd, mon,
>>>        mds) and even if only a tiny bit of them is needed at client
>>>        level. you
>>>        need two structures from ./osd/OSD.h and  my script by relation will
>>>        take into acount the whole directory...
>>>
>>>        I ran the script with libcephfs.cc as start point and got almost the
>>>        same results. 131 000 lines of code and 386 files most of the
>>>        same dirs
>>>        involved.
>>>
>>>
>>>
>>>        I think I will spend alot of time doing the manual source code
>>>        isolation
>>>        and understand way each #include is set in the files I read (what
>>>        purpose they have do they allow to integrate a crucial data type
>>>        or not.
>>>
>>>
>>>        The other way around will be to read src/libcephfs.cc. It seems
>>>        shorter
>>>        but without understanding what part is used for each included
>>>        header I
>>>        can t say anything...
>>>
>>>
>>>
>>>        I will keep reading the source code and take notes. I think in
>>>        the case
>>>        of libcephfs I will gain alot of time.
>>>
>>>        signature
>>>
>>>        *Alphé Salas*
>>>        Ingeniero T.I
>>>
>>>        asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>        *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>>>
>>>
>>>        On 11/07/13 15:02, Alphe Salas Michels wrote:
>>>
>>>            Hello D.Ketor and Matt Benjamin,
>>>            You give me alot to think about and this is great!
>>>            I merged your previous post to make a single reply that
>>>            anyone can
>>>            report to easyly
>>>
>>>            Windows NFS 4.1 is available here:
>>>            http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>>
>>>            pnfs is another name for NFS4.X. It is presented as
>>>            alternative to
>>>            ceph and we get known terminology as MDS and OSD but without
>>>            the self
>>>            healing part if I understand well my rapid look on the
>>>            topic. (when I
>>>            say rapid look I mean ... 5 minutes spent in that... which
>>>            is really
>>>            small amount of time to get an accurate view on something)
>>>
>>>
>>>            starting from mount.ceph ... I know that mount.ceph does
>>>            little but it
>>>            is a great hint to know what ceph needs and do things.
>>>            Basically mount.ceph modprobe the ceph driver in the linux
>>>            kernel then
>>>            call mount with the line command passed args and the cephfs
>>>            type as
>>>            argument. Then the kernel does the work I don t understand
>>>            yet what is
>>>            the start calls that are made to the ceph driver but it
>>>            seemed to me
>>>            that is was relatively light. (a first impression compared
>>>            to ceph-fuse.)
>>>
>>>            I think I will do both isolate source code from ceph-client
>>>            kernel
>>>            (cephfs module for linux kernel) and the one pointed by Sage
>>>            starting
>>>            from client/fuse_ll.cc in ceph master branch. The common
>>>            files betwin
>>>            those 2 extractions will be our core set of mandatory features.
>>>
>>>            Then we try to compile with cygwin a cephfs client library .
>>>            Then we
>>>            will try to interface with a modified windows nfs 4.1 client
>>>            or pnfs
>>>            or any other that will accept to be compiled with gcc for
>>>            win32...
>>>
>>>            the fact that windows 8.1 is and windows 2012 are out of
>>>            reach at the
>>>            moment is not a problem to me.
>>>
>>>            Our first concern is to understand what is ceph protocol.
>>>            Then adapt
>>>            it to something that can be used on windows prior windows
>>>            8.1. Dokan
>>>            fs if I remember well use too the WDK (windows driver
>>>            dev-kit ) for it
>>>            s compilation so possibly we will see the same limitations.
>>>
>>>            We need to multiply our source of information by example
>>>            regarding
>>>            ceph-client (kernel or fuse, radosgw is on a different layer
>>>            so I will
>>>            not try anything around it at first.) And we need to
>>>            multiply our
>>>            source of information by example regarding virtual file system
>>>            technologies on windoes OS.
>>>            Alot of work but all of those available source code everyone
>>>            point at
>>>            me will make our best solution. And in the end we will choose
>>>            technologies knowing what we do and what concequencies they
>>>            have.
>>>
>>>            regards,
>>>
>>>
>>>
>>>
>>>            Regards
>>>
>>>            signature
>>>
>>>            *Alphé Salas*
>>>            Ingeniero T.I
>>>
>>>            asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>
>>>
>>>
>>>            On 11/07/13 11:29, Ketor D wrote:
>>>
>>>                Hi Alphe:
>>>                        Yes Callback Filesystem is very expensive and
>>>                can't open source.
>>>                It's not a good choice for ceph4win.
>>>                        Another way for ceph4win maybe develop a
>>>                kernel-mode fs like
>>>                pnfs. pnfs has a kernel-mode windows client. I think you
>>>                can read its
>>>                src code and maybe migrating from ceph kernel client to
>>>                windows kernel
>>>                fs is easier than from userspace ceph fuse client.And a
>>>                kernel-mode fs
>>>                client has greater performance than userspace fs like
>>>                ceph-fuse client
>>>                and ceph kernel client.
>>>
>>>                        Regards.
>>>
>>>            On 11/07/13 11:50, Matt W. Benjamin wrote:
>>>
>>>                Hi,
>>>
>>>                The Window NFS v4.1 client is what we work on, so this
>>>                may be good for
>>>                code sharing.  The license is lgplv2, like Ceph's.
>>>
>>>                Something important to be aware of is that the client
>>>                uses rdbss, which
>>>                is a (partial) fsd abstraction that simplified
>>>                implementation
>>>                quite a bit, kind of like a mini driver.  However,
>>>                Microsoft's support
>>>                for rdbss has been in limbo for a bit.  For example, to
>>>                link with
>>>                the rdbss symbols you can't use the Windows 8 driver
>>>                kit--you'll need
>>>                to use the one for Windows 7.  (There's a private rdbss2
>>>                used internally
>>>                by Microsoft's SMB implemenation.  A the moment, 3rd
>>>                party drivers
>>>                can't use that.)
>>>
>>>                We've been in communication with Microsoft about this
>>>                issue, and know of
>>>                a few other fsds using it, but it could be a good thing
>>>                for that
>>>                lobbying
>>>                effort to have another user--or it could be a dead end :(.
>>>
>>>                There are a couple of other choices if you're looking to
>>>                go this route,
>>>                that I'm aware of (and we may need to take them too, if
>>>                RDBSS has no
>>>                way forward), but the required work could be a lot larger.
>>>
>>>                Matt
>>>
>>>                ----- "Ketor D"<d.ketor@gmail.com
>>>                <mailto:d.ketor@gmail.com>>  wrote:
>>>
>>>                    Hi Alphe:
>>>                            Yes Callback Filesystem is very expensive
>>>                    and can't open
>>>                    source.
>>>                    It's not a good choice for ceph4win.
>>>                            Another way for ceph4win maybe develop a
>>>                    kernel-mode fs like
>>>                    pnfs. pnfs has a kernel-mode windows client. I think
>>>                    you can read its
>>>                    src code and maybe migrating from ceph kernel client
>>>                    to windows
>>>                    kernel
>>>                    fs is easier than from userspace ceph fuse
>>>                    client.And a kernel-mode
>>>                    fs
>>>                    client has greater performance than userspace fs
>>>                    like ceph-fuse
>>>                    client
>>>                    and ceph kernel client.
>>>
>>>                            Regards.
>>>
>>>
>>>
>>>                    On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>>>                    Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>
>>>                    wrote:
>>>
>>>                        Commercial libraries are a pain ...
>>>
>>>                        If we want the more permossive licence offered
>>>                        by callback file
>>>
>>>                    system we
>>>
>>>                        have to buy it for 20.000 usd. Then we will have
>>>                        to provide a
>>>
>>>                    backbox that
>>>
>>>                        we have no control upon and that will kill our
>>>                        product anytime they
>>>
>>>                    want anf
>>>
>>>                        if they decide to stop their commercial activity
>>>                        we will be in the
>>>
>>>                    same
>>>
>>>                        situation that with dokanfs but without having
>>>                        the source code of
>>>
>>>                    the black
>>>
>>>                        box. If i have to spend 20 000 dollars i would
>>>                        prefere paying
>>>
>>>                    someone to
>>>
>>>                        retake dokanfs or to write from scratch a
>>>                        dokanfs fuselike software
>>>
>>>                    make it
>>>
>>>                        all shiny and pumpy fantastic and ready to plug
>>>                        to ceph client.
>>>
>>>                        I would prefere if people have to pay something
>>>                        to get access to
>>>
>>>                    ceph4win
>>>
>>>                        that this money goes in ceph main branch
>>>                        pockets... Or as a gift you
>>>
>>>                    donante
>>>
>>>                        to ceph 10 dollars  you get 2 free registration
>>>                        codes for
>>>
>>>                    ceph4win... or
>>>
>>>                        something like that.
>>>
>>>                        If ceph4win as to be comercial then I would
>>>                        prefer delegate the task
>>>
>>>                    to a
>>>
>>>                        company like south river technologies and their
>>>                        great product
>>>
>>>                    webdrive. I
>>>
>>>                        would mininaly get involved in that project and
>>>                        simply buy the final
>>>
>>>                    product
>>>
>>>                        to sell it together with my ceph based product
>>>                        (which could be a
>>>
>>>                    calxeda
>>>
>>>                        ceph box or something like that).
>>>
>>>                        I m open anyway to any proposition. But I doubt
>>>                        that callback
>>>
>>>                    filesystem
>>>
>>>                        offers us a suitable solution in the way I see
>>>                        ceph4win to be spread
>>>
>>>                    and
>>>
>>>                        used... I m maybe wrong. And anything that will
>>>                        be done around
>>>
>>>                    ceph4win will
>>>
>>>                        be public documented etc... And licensed the way
>>>                        that if someone
>>>
>>>                    want to
>>>
>>>                        build a commercial solution on top of it, that
>>>                        would be a
>>>
>>>                    possibility.
>>>
>>>                        My idea is to giveback somehow to ceph project
>>>                        and at same time
>>>
>>>                    forge a
>>>
>>>                        better knowledge in ceph technologies. Because
>>>                        like many in libre
>>>
>>>                    world I
>>>
>>>                        think the business is in the services around the
>>>                        software more than
>>>
>>>                    on the
>>>
>>>                        software. That the ones writing code should be
>>>                        financed and benefits
>>>
>>>                    from
>>>
>>>                        the one selling and giving support of the
>>>                        software at all levels. I
>>>
>>>                    m
>>>
>>>                        probably too idealistic. And too optimistic
>>>                        after all I m the one
>>>
>>>                    saying I
>>>
>>>                        will do this stuff I have no idea how but well
>>>                        it is interesting and
>>>
>>>                    fun so
>>>
>>>                        lets do it.
>>>
>>>                        Regards,
>>>
>>>                        P.S: using commercial backend libraries appart
>>>                        including their own
>>>
>>>                    cost will
>>>
>>>                        force you to use commercial IDE like MS
>>>                        VisualStudio because their
>>>
>>>                    library
>>>
>>>                        has some kind of drm that only that IDE compiler
>>>                        can use. So alot of
>>>
>>>                    cost
>>>
>>>                        and yet there is nothing done. If I had to open
>>>                        a kickstarter
>>>
>>>                    project saying
>>>
>>>                        we need 60 000 USD to do ceph4win with that
>>>                        monney we will buy the
>>>
>>>                    right to
>>>
>>>                        use and share a commercial copyrighted library
>>>                        but abandonned
>>>
>>>                    punctually to
>>>
>>>                        us in  public domaine and that we will
>>>                        eventually produce something
>>>
>>>                    out of
>>>
>>>                        it. I doubt I will get a dollar.
>>>
>>>                        We still can suggest the idea to Edlos the
>>>                        commercial company that
>>>
>>>                    has the
>>>
>>>                        copyright of Callback FS, Or to buy them their
>>>                        product in a blender
>>>
>>>                    way
>>>
>>>                        (blender was bought with donation before being
>>>                        put opensource and
>>>
>>>                    public
>>>
>>>                        domaine), Or to open source their library. But
>>>                        in commercial minds
>>>                        opensourcing = death of their technical
>>>                        advantage and death of
>>>
>>>                    their
>>>
>>>                        marketing strategy. They will have to invent
>>>                        something more to
>>>
>>>                    retrieve
>>>
>>>                        monney from it.
>>>
>>>                        El nov 6, 2013 11:22 p.m., "Ketor D"
>>>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>>                        <mailto:d.ketor@gmail.com
>>>                        <mailto:d.ketor@gmail.com>>> escribió:
>>>
>>>
>>>                             Hi Alphe,
>>>                                       I think you could try Callback
>>>                        Filesystem dev
>>>
>>>                    framework. It
>>>
>>>                             is a commerical dev framework and is
>>>                        maintained by Edlos today.
>>>                                       I have communicated with Edlos to
>>>                        get a try code for
>>>                             development. To dokan, Callback Filesystem
>>>                        has vary document and
>>>
>>>                    maybe
>>>
>>>                             more stabilize.
>>>
>>>                                       Regards.
>>>
>>>
>>>
>>>                             On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>>>                        Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>                             <mailto:asalas@kepler.cl
>>>
>>>                        <mailto:asalas@kepler.cl>>> wrote:
>>>                              > Hello ketor thank you for your interest
>>>                        un ceph4win. Since
>>>
>>>                    muy
>>>
>>>                             first mail I
>>>                              > exposed the lacks of dokanfs and that I
>>>                        m far from being a
>>>                             specialist un
>>>                              > filesystems.
>>>                              > I exposed what i like un dokanfs bit I
>>>                        not a fanátic of it.
>>>
>>>                    Muy
>>>
>>>                             goal is to
>>>                              > have something working quickly.
>>>                              >
>>>                              > So I am up to any proposición sure the
>>>                        one with the more docs
>>>
>>>                    and
>>>
>>>                             support
>>>                              > will be the best choice. As for right
>>>                        now what I need is
>>>                             understand what are
>>>                              > the files involved what are the
>>>                        interfaces functions and what
>>>
>>>                    are
>>>
>>>                             the needed
>>>                              > library dependencies and if they exist
>>>                        ported to windows with
>>>                             cygwin. And
>>>                              > all that is retrieved from source code.
>>>                              >
>>>                              > Regards.
>>>                              >
>>>                              > El nov 6, 2013 10:34 p.m., "Ketor D"
>>>                        <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>>                             <mailto:d.ketor@gmail.com
>>>
>>>                        <mailto:d.ketor@gmail.com>>> escribió:
>>>
>>>                              >
>>>                              >> Hi Alphe,
>>>                              >>      We are taking an interest in your
>>>                        work on Ceph Client
>>>
>>>                    for
>>>
>>>                             Windows
>>>                              >> with Dokan.As we know, the performance
>>>                        of Dokan is not very
>>>                             good, and it's
>>>                              >> abandoned 3 years ago.
>>>                              >>      I have learned and used
>>>                        OpenDedup(SDFS) for a long
>>>
>>>                    time.
>>>
>>>                               OpenDedup
>>>                              >> has a Dokan version. And the author of
>>>                        OpenDedup said
>>>                              >>
>>>                              >> The Dokan library is quite flakey  and
>>>                        testing should be
>>>                             performed before
>>>                              >> putting into production
>>>                              >>
>>>                              >>       So what do you think about this?
>>>                        And if there is
>>>
>>>                    another
>>>
>>>                             solution of
>>>                              >> fuse-like filesystem dev framwork on
>>>                        Windows?
>>>                              >>
>>>                              >>        Best Wish!
>>>                              >>
>>>                              >>
>>>                              >>
>>>                              >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>>>                        Salas Michels
>>>                             <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>>>
>>>                              >> wrote:
>>>                              >>>
>>>                              >>> Hello I created the github repository
>>>                        for this project
>>>                              >>>https://github.com/alphe/Ceph4Win
>>>                              >>>
>>>                              >>> Regards,
>>>                              >>>
>>>                              >>> signature
>>>                              >>>
>>>                              >>> *Alphé Salas*
>>>                              >>> Ingeniero T.I
>>>                              >>>
>>>                              >>>asalas@kepler.cl
>>>                        <mailto:asalas@kepler.cl>
>>>                        <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>
>>>
>>>                              >>> *<http://www.kepler.cl>*
>>>                              >>>
>>>                              >>> On 11/05/13 21:00, Sage Weil wrote:
>>>                              >>>>
>>>                              >>>> Hi Alphe,
>>>                              >>>>
>>>                              >>>> On Tue, 5 Nov 2013, Alphe Salas
>>>                        Michels wrote:
>>>                              >>>>>
>>>                              >>>>> signature *Hi, Sage !
>>>                              >>>>> thank you for you enthousiast reply.
>>>                              >>>>> I sure want to make the best use of
>>>                        everything or
>>>
>>>                    anything
>>>
>>>                             previously
>>>                              >>>>> done to
>>>                              >>>>> tend to
>>>                              >>>>> write ceph cliente for windows.
>>>                              >>>>>
>>>                              >>>>> Apart using libre tools for building
>>>                        the future ceph
>>>
>>>                    cliente
>>>
>>>                             I am open
>>>                              >>>>> to
>>>                              >>>>> anything.
>>>                              >>>>> I would recommand eclipse CDT or
>>>                        Code::BLocks they are
>>>
>>>                    based
>>>
>>>                             on mingwin
>>>                              >>>>> open
>>>                              >>>>> and easyly enhanceable.**
>>>                              >>>>>
>>>                              >>>>> more free tools can be found here:
>>>
>>>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>                              >>>>>
>>>                              >>>>>
>>>                              >>>>> I will read libcephfs source code
>>>                        and take some notes
>>>
>>>                    about the
>>>
>>>                              >>>>> protocol.
>>>                              >>>>
>>>                              >>>> I think you don't need to worry about
>>>                        hte protocol at all,
>>>
>>>                    since
>>>
>>>                              >>>> libcephs
>>>                              >>>> implements it for you (and will
>>>                        capture any future
>>>
>>>                    changes).
>>>
>>>                              >>>>
>>>                              >>>>> I was more going from what I know
>>>                        and trying to track down
>>>
>>>                    how
>>>
>>>                              >>>>> mount.ceph work
>>>                              >>>>> with the parameters passed to it.
>>>                              >>>>> since it point finally to
>>>                        Kernel/fs/ceph and that I don t
>>>
>>>                    really
>>>
>>>                              >>>>> understand
>>>                              >>>>> how that module work and that it
>>>                        probably points to some
>>>
>>>                    other
>>>
>>>                              >>>>> dependencies
>>>                              >>>>> Reading libcephfs source code could
>>>                        be a big gain of
>>>
>>>                    time.
>>>
>>>                              >>>>
>>>                              >>>> (I would also ignore mount.ceph as
>>>                        everything it does it
>>>                             specific to
>>>                              >>>> how Linux mounts work.)
>>>                              >>>>
>>>                              >>>>> basically on the protocol what is
>>>                        need are:
>>>                              >>>>>
>>>                              >>>>> 1) open and maintain a connection
>>>                        (socket open, auth, etc
>>>
>>>                    )
>>>
>>>                              >>>>> 2) retreive a map of directories and
>>>                        disk Quota (disk
>>>
>>>                    sizing
>>>
>>>                             Y TB free,
>>>                              >>>>> Z TB
>>>                              >>>>> total)
>>>                              >>>>> 3) procedure to send files /
>>>                        directories in a maner that
>>>
>>>                    it
>>>
>>>                             will allow
>>>                              >>>>> our
>>>                              >>>>> client to fit ceph transmission
>>>                        protocols
>>>                              >>>>> (limit bandwith for stability?,
>>>                        limit connection amount?,
>>>                             limit cpu
>>>                              >>>>> use?,
>>>                              >>>>> Cache for preparing data transfer (a
>>>                        FIFO cache)?)
>>>                              >>>>> 4)Procedure to retreive files /
>>>                        directory from ceph
>>>
>>>                    cluster
>>>
>>>                              >>>>> 5) Management copy/move files
>>>                        /Directories, FS stats,
>>>                             Connection Stats.
>>>                              >>>>> logging.
>>>                              >>>>>
>>>                              >>>>> My idea to progress is to take those
>>>                        main bulletpoint in
>>>
>>>                    ceph
>>>
>>>                             protocol
>>>                              >>>>> based
>>>                              >>>>> on general ideas of what ceph file
>>>                        system does and start
>>>                             identifying
>>>                              >>>>> parts
>>>                              >>>>> from libcephfs to match those "needs".
>>>                              >>>>
>>>                              >>>> Instead, I would look at
>>>                        include/cephfs/libcephfs.h, the
>>>                             interface that
>>>                              >>>> libcephfs provides, and try to map
>>>                        that to what the fuse
>>>
>>>                    layer
>>>
>>>                             expects.
>>>                              >>>> There is both a path-based that I
>>>                        suspsect lends itself
>>>
>>>                    well
>>>
>>>                             to the
>>>                              >>>> Windows interface and (very soon now)
>>>                        a handle based API
>>>
>>>                    that is
>>>
>>>                              >>>> targetted
>>>                              >>>> at the Unix-style VFS layers.  I'm
>>>                        mostly guessing,
>>>
>>>                    though,
>>>
>>>                             since I've
>>>                              >>>> never seen any low-level fs code in
>>>                        windows before.
>>>                              >>>>
>>>                              >>>> In this case, the analogous code for
>>>                        Linux should be
>>>                             client/fuse_ll.cc
>>>                              >>>> itself (and not much else), although
>>>                        there will probably be
>>>
>>>                    a
>>>
>>>                             few tricks
>>>                              >>>> necessary to map cleanly onto how the
>>>                        windows interfaces
>>>
>>>                    work.
>>>
>>>                              >>>>
>>>                              >>>> Does that make sense?
>>>                              >>>>
>>>                              >>>> Cheers!
>>>                              >>>> sage
>>>                              >>>>
>>>                              >>>>
>>>                              >>>>> Any suggestion and contributions are
>>>                        welcome.
>>>                              >>>>>
>>>                              >>>>>
>>>                              >>>>> *
>>>                              >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>                              >>>>>>
>>>                              >>>>>> Hi Alphe,
>>>                              >>>>>>
>>>                              >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>>>                        Michels wrote:
>>>                              >>>>>>>
>>>                              >>>>>>> Good day developers!
>>>                              >>>>>>>
>>>                              >>>>>>> I would like to propose to the one
>>>                        interested  work with
>>>
>>>                    me to
>>>
>>>                              >>>>>>> develop a
>>>                              >>>>>>> ceph
>>>                              >>>>>>> cliente for MS windows world,
>>>                        Basing us on dokanFS.
>>>                              >>>>>>>
>>>                              >>>>>>> My company is a ceph enthousiast
>>>                        that use on a dayly
>>>
>>>                    basis
>>>
>>>                             ceph and
>>>                              >>>>>>> that
>>>                              >>>>>>> need
>>>                              >>>>>>> both transfer speed and big
>>>                        expendable and cheap
>>>
>>>                    storage.
>>>
>>>                              >>>>>>> My company is specialised in data
>>>                        recovery and we want
>>>
>>>                    to
>>>
>>>                             participate
>>>                              >>>>>>> to
>>>                              >>>>>>> ceph
>>>                              >>>>>>> effort by bringing a ceph cliente
>>>                        for windows.
>>>                              >>>>>>
>>>                              >>>>>> Awesome!
>>>                              >>>>>>
>>>                              >>>>>>> Our experience shows us that the
>>>                        best gateway is each
>>>                             clientes being
>>>                              >>>>>>> its
>>>                              >>>>>>> own
>>>                              >>>>>>> gateway, instead of having a
>>>                        bottle neck server or a
>>>
>>>                    cluster of
>>>
>>>                              >>>>>>> bottle
>>>                              >>>>>>> neck
>>>                              >>>>>>> servers as gateway (FTP, samba,
>>>                        SFTP,webdav, s3,
>>>
>>>                    etc..).
>>>
>>>                              >>>>>>>
>>>                              >>>>>>> We already did some research in
>>>                        that domain.
>>>                              >>>>>>>
>>>                              >>>>>>> Dokan FS is an intent  to write an
>>>                        opensource fuse like
>>>                             cliente for
>>>                              >>>>>>> MS
>>>                              >>>>>>> windows.
>>>                              >>>>>>>
>>>                              >>>>>>> More information on DOKANFS can be
>>>                        triggered here
>>>                         >>>>>>>http://dokan-dev.net/en/download/
>>>                              >>>>>>>
>>>                              >>>>>>> Positive points of using DOKANFS.
>>>                              >>>>>>>
>>>                              >>>>>>> - its opensourced and well
>>>                        licenced mit licence, gpl
>>>                             licence and lgpl
>>>                              >>>>>>> licence.
>>>                              >>>>>>>
>>>                              >>>>>>> Negative point of using DOKAN FS.
>>>                              >>>>>>> - unreachable author
>>>                              >>>>>>> - Poor documentation . Dev
>>>                        comments in japanese.
>>>                              >>>>>>> - Work in progress so it is
>>>                        unstable and needs to be
>>>
>>>                    updated,
>>>
>>>                              >>>>>>> debugged and
>>>                              >>>>>>> maintained by a MS Windows file
>>>                        system expert
>>>
>>>                    developper.
>>>
>>>                              >>>>>>
>>>                              >>>>>> I am not very familiar with windows
>>>                        storage APIs, but
>>>                             somebody told me
>>>                              >>>>>> at once point there were several
>>>                        interfaces against which
>>>
>>>                    a
>>>
>>>                             new file
>>>                              >>>>>> system could be implemented,
>>>                        everything from a full
>>>                             in-kernel driver
>>>                              >>>>>> to
>>>                              >>>>>> something that is explorer-based.
>>>                          Are any of those
>>>                             suitable?  Using a
>>>                              >>>>>> potentially abandoned fuse-like
>>>                        layer makes me a bit
>>>
>>>                    nervous.
>>>
>>>                              >>>>>>
>>>                              >>>>>> That said,
>>>                              >>>>>>
>>>                              >>>>>>>
>>>                              >>>>>>> I try past year to do a merge from
>>>                        ceph-fuse to dokanfs
>>>                              >>>>>>> here are what I learnt.
>>>                              >>>>>>> - Ceph-fuse and related source
>>>                        code is around 60 000
>>>
>>>                    lines
>>>
>>>                             of code.
>>>                              >>>>>>> - Ceph protocol isn t documented
>>>                        so it is like trying
>>>
>>>                    to
>>>
>>>                             draw a map
>>>                              >>>>>>> of
>>>                              >>>>>>> america
>>>                              >>>>>>> using only a sextan and a compass.
>>>                              >>>>>>>
>>>                              >>>>>>> Those led me to those conclusions:
>>>                              >>>>>>> - I can t do it alone.
>>>                              >>>>>>> - It is easier to draw down the
>>>                        ceph protocol way to
>>>
>>>                    work from
>>>
>>>                              >>>>>>> kernel/fs/ceph
>>>                              >>>>>>> sources and mount.ceph
>>>                              >>>>>>> - Ceph depending libraries may be
>>>                        unexistant or not up
>>>
>>>                    to
>>>
>>>                             date in
>>>                              >>>>>>> their
>>>                              >>>>>>> port
>>>                              >>>>>>> on MS Windows (cygwin)
>>>                              >>>>>>
>>>                              >>>>>> I think the most sane path should
>>>                        be to make libcephfs
>>>                             sufficiently
>>>                              >>>>>> portable to build on windows (or
>>>                        cygwin).  For the bits
>>>
>>>                    used
>>>
>>>                             by the
>>>                              >>>>>> client-side coe, I don't think
>>>                        there should be much in
>>>
>>>                    the
>>>
>>>                             way of
>>>                              >>>>>> dependencies, and the main
>>>                        challenge would be untangling
>>>
>>>                    the
>>>
>>>                             build for
>>>                              >>>>>> the necessary pieces out from the
>>>                        rest of Ceph.
>>>                              >>>>>>
>>>                              >>>>>> Have you seen the wip-port
>>>                        portability work that is
>>>                             currently underway
>>>                              >>>>>> by
>>>                              >>>>>> Noah and Alan?  That may solve many
>>>                        of the cygwin
>>>
>>>                    problems
>>>
>>>                             you are
>>>                              >>>>>> seeing
>>>                              >>>>>> today.
>>>                              >>>>>>
>>>                              >>>>>>> - MS file system specialist are
>>>                        hard do find in the
>>>
>>>                    "open
>>>
>>>                             source
>>>                              >>>>>>> libre
>>>                              >>>>>>> world"
>>>                              >>>>>>> so I will try in the commercial world.
>>>                              >>>>>>>
>>>                              >>>>>>> The commercial world has some
>>>                        problems too. They need
>>>
>>>                    ceph
>>>
>>>                             protocol
>>>                              >>>>>>> draft
>>>                              >>>>>>> to
>>>                              >>>>>>> implemente it to their own product
>>>                        They will have
>>>
>>>                    licencing
>>>
>>>                              >>>>>>> /commercial
>>>                              >>>>>>> politics that infringe lgpl, and
>>>                        hide that most of the
>>>
>>>                    work
>>>
>>>                             is done
>>>                              >>>>>>> by
>>>                              >>>>>>> people
>>>                              >>>>>>> other than them. They will not
>>>                        participate in a
>>>
>>>                    financial
>>>
>>>                             way to ceph
>>>                              >>>>>>> enhancement and growth.
>>>                              >>>>>>
>>>                              >>>>>> I don't think reimplementing the
>>>                        client code is an
>>>
>>>                    efficient way
>>>
>>>                              >>>>>> forward.
>>>                              >>>>>> Unless the goal is a pure kernel
>>>                        implementation...but a
>>>                             significant
>>>                              >>>>>> ongoing investment in development
>>>                        resources would be
>>>
>>>                    needed
>>>
>>>                             for that
>>>                              >>>>>> going
>>>                              >>>>>> forward.  I suspect that is a
>>>                        challenge for a platform
>>>
>>>                    that
>>>
>>>                             does not
>>>                              >>>>>> typically rally that sort of
>>>                        community effort.
>>>                              >>>>>>
>>>                              >>>>>> The easiest thing is of course just
>>>                        to use CIFS and
>>>
>>>                    Samba
>>>
>>>                             (which works
>>>                              >>>>>> today).  A fuse-like approach is
>>>                        probably a reasonably
>>>                             middle ground
>>>                              >>>>>> (both
>>>                              >>>>>> in initial effort and
>>>                        maintainability going forward)...
>>>                              >>>>>>
>>>                              >>>>>> sage
>>>                              >>>>>>
>>>                              >>>>>>
>>>                              >>>>>
>>>                              >>>
>>>                              >>> --
>>>                              >>> To unsubscribe from this list: send
>>>                        the line "unsubscribe
>>>                             ceph-devel" in
>>>                              >>> the body of a message
>>>                        tomajordomo@vger.kernel.org
>>>                        <mailto:tomajordomo@vger.kernel.org>
>>>                             <mailto:majordomo@vger.kernel.org
>>>                        <mailto:majordomo@vger.kernel.org>>
>>>
>>>                              >>> More majordomo info at
>>>
>>>                    http://vger.kernel.org/majordomo-info.html
>>>
>>>                              >>
>>>                              >>
>>>                              >
>>>
>>>
>>>
>>>                    --
>>>                    To unsubscribe from this list: send the line
>>>                    "unsubscribe ceph-devel"
>>>                    in
>>>                    the body of a message tomajordomo@vger.kernel.org
>>>                    <mailto:tomajordomo@vger.kernel.org>
>>>                    More majordomo info
>>>                    athttp://vger.kernel.org/majordomo-info.html
>>>                    <http://vger.kernel.org/majordomo-info.html>
>>>
>>>
>>>
>>>        --
>>>        To unsubscribe from this list: send the line "unsubscribe
>>>        ceph-devel" in
>>>        the body of a message to majordomo@vger.kernel.org
>>>        <mailto:majordomo@vger.kernel.org>
>>>        More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Dong Yuan
> Email:yuandong1222@gmail.com



-- 
Dong Yuan
Email:yuandong1222@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

writing a ceph cliente for MS windows

[Alphe Salas Michels]

Good day developers!

I would like to propose to the one interested  work with me to develop a 
ceph cliente for MS windows world, Basing us on dokanFS.

My company is a ceph enthousiast that use on a dayly basis ceph and that 
need both transfer speed and big expendable and cheap storage.
My company is specialised in data recovery and we want to participate to 
ceph effort by bringing a ceph cliente for windows.
Our experience shows us that the best gateway is each clientes being its 
own gateway, instead of having a bottle neck server or a cluster of 
bottle neck servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).

We already did some research in that domain.

Dokan FS is an intent  to write an opensource fuse like cliente for MS 
windows.

More information on DOKANFS can be triggered here
http://dokan-dev.net/en/download/

Positive points of using DOKANFS.

- its opensourced and well licenced mit licence, gpl licence and lgpl 
licence.

Negative point of using DOKAN FS.
- unreachable author
- Poor documentation . Dev comments in japanese.
- Work in progress so it is unstable and needs to be updated, debugged 
and maintained by a MS Windows file system expert developper.

I try past year to do a merge from ceph-fuse to dokanfs
here are what I learnt.
- Ceph-fuse and related source code is around 60 000 lines of code.
- Ceph protocol isn t documented so it is like trying to draw a map of 
america using only a sextan and a compass.

Those led me to those conclusions:
- I can t do it alone.
- It is easier to draw down the ceph protocol way to work from 
kernel/fs/ceph sources and mount.ceph
- Ceph depending libraries may be unexistant or not up to date in their 
port on MS Windows (cygwin)
- MS file system specialist are hard do find in the "open source libre 
world" so I will try in the commercial world.

The commercial world has some problems too. They need ceph protocol 
draft to implemente it to their own product They will have licencing 
/commercial politics that infringe lgpl, and hide that most of the work 
is done by people other than them. They will not participate in a 
financial way to ceph enhancement and growth.

As for the modalities of the work if we come to work togather on that 
topic, it up to you. We can / should use the most possible libre tools 
for that work (eclipse, mingwin/cygwin etc...) Sharpening the dev tools 
in libre can already be a big problem. But my way to see this is the 
more libre are our tools on MS Windows the most people can easyly join us.


Best Regards,


Alphe Salas

asalas@kepler.cl
I.T ingeneer.
*<http://www.kepler.cl>*

Re: writing a ceph cliente for MS windows

[Sage Weil]

Hi Alphe,

On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> Good day developers!
> 
> I would like to propose to the one interested  work with me to develop a ceph
> cliente for MS windows world, Basing us on dokanFS.
> 
> My company is a ceph enthousiast that use on a dayly basis ceph and that need
> both transfer speed and big expendable and cheap storage.
> My company is specialised in data recovery and we want to participate to ceph
> effort by bringing a ceph cliente for windows.

Awesome!

> Our experience shows us that the best gateway is each clientes being its own
> gateway, instead of having a bottle neck server or a cluster of bottle neck
> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
> 
> We already did some research in that domain.
> 
> Dokan FS is an intent  to write an opensource fuse like cliente for MS
> windows.
> 
> More information on DOKANFS can be triggered here
> http://dokan-dev.net/en/download/
> 
> Positive points of using DOKANFS.
> 
> - its opensourced and well licenced mit licence, gpl licence and lgpl licence.
> 
> Negative point of using DOKAN FS.
> - unreachable author
> - Poor documentation . Dev comments in japanese.
> - Work in progress so it is unstable and needs to be updated, debugged and
> maintained by a MS Windows file system expert developper.

I am not very familiar with windows storage APIs, but somebody told me 
at once point there were several interfaces against which a new file 
system could be implemented, everything from a full in-kernel driver to 
something that is explorer-based.  Are any of those suitable?  Using a 
potentially abandoned fuse-like layer makes me a bit nervous.

That said,
 
> I try past year to do a merge from ceph-fuse to dokanfs
> here are what I learnt.
> - Ceph-fuse and related source code is around 60 000 lines of code.
> - Ceph protocol isn t documented so it is like trying to draw a map of america
> using only a sextan and a compass.
> 
> Those led me to those conclusions:
> - I can t do it alone.
> - It is easier to draw down the ceph protocol way to work from kernel/fs/ceph
> sources and mount.ceph
> - Ceph depending libraries may be unexistant or not up to date in their port
> on MS Windows (cygwin)

I think the most sane path should be to make libcephfs sufficiently 
portable to build on windows (or cygwin).  For the bits used by the 
client-side coe, I don't think there should be much in the way of 
dependencies, and the main challenge would be untangling the build for 
the necessary pieces out from the rest of Ceph.

Have you seen the wip-port portability work that is currently underway by 
Noah and Alan?  That may solve many of the cygwin problems you are seeing 
today.

> - MS file system specialist are hard do find in the "open source libre world"
> so I will try in the commercial world.
> 
> The commercial world has some problems too. They need ceph protocol draft to
> implemente it to their own product They will have licencing /commercial
> politics that infringe lgpl, and hide that most of the work is done by people
> other than them. They will not participate in a financial way to ceph
> enhancement and growth.

I don't think reimplementing the client code is an efficient way forward.  
Unless the goal is a pure kernel implementation...but a significant 
ongoing investment in development resources would be needed for that going 
forward.  I suspect that is a challenge for a platform that does not 
typically rally that sort of community effort.

The easiest thing is of course just to use CIFS and Samba (which works 
today).  A fuse-like approach is probably a reasonably middle ground (both 
in initial effort and maintainability going forward)...

sage

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

signature *Hi, Sage !
thank you for you enthousiast reply.
I sure want to make the best use of everything or anything previously 
done to tend to
write ceph cliente for windows.

Apart using libre tools for building the future ceph cliente I am open 
to anything.
I would recommand eclipse CDT or Code::BLocks they are based on mingwin 
open and easyly enhanceable.**

more free tools can be found here:
http://www.freebyte.com/programming/cpp/#cppcompilers


I will read libcephfs source code and take some notes about the protocol.
I was more going from what I know and trying to track down how 
mount.ceph work with the parameters passed to it.
since it point finally to Kernel/fs/ceph and that I don t really 
understand how that module work and that it probably points to some 
other dependencies Reading libcephfs source code could be a big gain of 
time.

basically on the protocol what is  need are:

1) open and maintain a connection (socket open, auth, etc )
2) retreive a map of directories and disk Quota (disk sizing Y TB free,  
Z TB total)
3) procedure to send files / directories in a maner that it will allow 
our client to fit ceph transmission protocols
(limit bandwith for stability?, limit connection amount?, limit cpu 
use?, Cache for preparing data transfer (a FIFO cache)?)
4)Procedure to retreive files / directory from ceph cluster
5) Management copy/move files /Directories, FS stats, Connection Stats. 
logging.

My idea to progress is to take those main bulletpoint in ceph protocol 
based on general ideas of what ceph file system does and start 
identifying parts from libcephfs to match those "needs".

Any suggestion and contributions are welcome.


*
On 11/05/13 11:23, Sage Weil wrote:
> Hi Alphe,
>
> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>> Good day developers!
>>
>> I would like to propose to the one interested  work with me to develop a ceph
>> cliente for MS windows world, Basing us on dokanFS.
>>
>> My company is a ceph enthousiast that use on a dayly basis ceph and that need
>> both transfer speed and big expendable and cheap storage.
>> My company is specialised in data recovery and we want to participate to ceph
>> effort by bringing a ceph cliente for windows.
> Awesome!
>
>> Our experience shows us that the best gateway is each clientes being its own
>> gateway, instead of having a bottle neck server or a cluster of bottle neck
>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>
>> We already did some research in that domain.
>>
>> Dokan FS is an intent  to write an opensource fuse like cliente for MS
>> windows.
>>
>> More information on DOKANFS can be triggered here
>> http://dokan-dev.net/en/download/
>>
>> Positive points of using DOKANFS.
>>
>> - its opensourced and well licenced mit licence, gpl licence and lgpl licence.
>>
>> Negative point of using DOKAN FS.
>> - unreachable author
>> - Poor documentation . Dev comments in japanese.
>> - Work in progress so it is unstable and needs to be updated, debugged and
>> maintained by a MS Windows file system expert developper.
> I am not very familiar with windows storage APIs, but somebody told me
> at once point there were several interfaces against which a new file
> system could be implemented, everything from a full in-kernel driver to
> something that is explorer-based.  Are any of those suitable?  Using a
> potentially abandoned fuse-like layer makes me a bit nervous.
>
> That said,
>   
>> I try past year to do a merge from ceph-fuse to dokanfs
>> here are what I learnt.
>> - Ceph-fuse and related source code is around 60 000 lines of code.
>> - Ceph protocol isn t documented so it is like trying to draw a map of america
>> using only a sextan and a compass.
>>
>> Those led me to those conclusions:
>> - I can t do it alone.
>> - It is easier to draw down the ceph protocol way to work from kernel/fs/ceph
>> sources and mount.ceph
>> - Ceph depending libraries may be unexistant or not up to date in their port
>> on MS Windows (cygwin)
> I think the most sane path should be to make libcephfs sufficiently
> portable to build on windows (or cygwin).  For the bits used by the
> client-side coe, I don't think there should be much in the way of
> dependencies, and the main challenge would be untangling the build for
> the necessary pieces out from the rest of Ceph.
>
> Have you seen the wip-port portability work that is currently underway by
> Noah and Alan?  That may solve many of the cygwin problems you are seeing
> today.
>
>> - MS file system specialist are hard do find in the "open source libre world"
>> so I will try in the commercial world.
>>
>> The commercial world has some problems too. They need ceph protocol draft to
>> implemente it to their own product They will have licencing /commercial
>> politics that infringe lgpl, and hide that most of the work is done by people
>> other than them. They will not participate in a financial way to ceph
>> enhancement and growth.
> I don't think reimplementing the client code is an efficient way forward.
> Unless the goal is a pure kernel implementation...but a significant
> ongoing investment in development resources would be needed for that going
> forward.  I suspect that is a challenge for a platform that does not
> typically rally that sort of community effort.
>
> The easiest thing is of course just to use CIFS and Samba (which works
> today).  A fuse-like approach is probably a reasonably middle ground (both
> in initial effort and maintainability going forward)...
>
> sage
>
>

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello sage,
I followed your lead and went a bit further in my source code reading 
and I notice serveral problems:
first most of the id use in ceph osd_clients, mds_clients, mon_client 
use the data type u64 which will be a problem
as most of the windows in use are windows XP or Windows server 2003 in 
32 bits. As they are used for id tag for
things like snapshot pages (if I understand well) that can be a problem 
for a port no?

What I read so far was
mount.ceph files > kernel/fs/ceph files which contains the mds_client 
files and the kernel ioctl interface ... >  include / linux / ceph  
files which integrates libcephfs.h. and all the needed files

there is three clients, there is auth.h a messenger, a msgpool, buffer, 
rados that depends on msgr that include in every aspect of the messages 
formation  one or more __le64 message.

I m far from understanding the whole code and the interactions betwin 
all the files and which we can skip and which we can keep.

how can we translate those data type into 32bit in order for ceph 
cluster to understand them and ceph-windows-client to transmit them?

Regards

signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl




On 11/05/13 14:40, Alphe Salas Michels wrote:
>
> signature *Hi, Sage !
> thank you for you enthousiast reply.
> I sure want to make the best use of everything or anything previously 
> done to tend to
> write ceph cliente for windows.
>
> Apart using libre tools for building the future ceph cliente I am open 
> to anything.
> I would recommand eclipse CDT or Code::BLocks they are based on 
> mingwin open and easyly enhanceable.**
>
> more free tools can be found here:
> http://www.freebyte.com/programming/cpp/#cppcompilers
>
>
> I will read libcephfs source code and take some notes about the protocol.
> I was more going from what I know and trying to track down how 
> mount.ceph work with the parameters passed to it.
> since it point finally to Kernel/fs/ceph and that I don t really 
> understand how that module work and that it probably points to some 
> other dependencies Reading libcephfs source code could be a big gain 
> of time.
>
> basically on the protocol what is  need are:
>
> 1) open and maintain a connection (socket open, auth, etc )
> 2) retreive a map of directories and disk Quota (disk sizing Y TB 
> free,  Z TB total)
> 3) procedure to send files / directories in a maner that it will allow 
> our client to fit ceph transmission protocols
> (limit bandwith for stability?, limit connection amount?, limit cpu 
> use?, Cache for preparing data transfer (a FIFO cache)?)
> 4)Procedure to retreive files / directory from ceph cluster
> 5) Management copy/move files /Directories, FS stats, Connection 
> Stats. logging.
>
> My idea to progress is to take those main bulletpoint in ceph protocol 
> based on general ideas of what ceph file system does and start 
> identifying parts from libcephfs to match those "needs".
>
> Any suggestion and contributions are welcome.
>
>
> *
> On 11/05/13 11:23, Sage Weil wrote:
>> Hi Alphe,
>>
>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>> Good day developers!
>>>
>>> I would like to propose to the one interested  work with me to 
>>> develop a ceph
>>> cliente for MS windows world, Basing us on dokanFS.
>>>
>>> My company is a ceph enthousiast that use on a dayly basis ceph and 
>>> that need
>>> both transfer speed and big expendable and cheap storage.
>>> My company is specialised in data recovery and we want to 
>>> participate to ceph
>>> effort by bringing a ceph cliente for windows.
>> Awesome!
>>
>>> Our experience shows us that the best gateway is each clientes being 
>>> its own
>>> gateway, instead of having a bottle neck server or a cluster of 
>>> bottle neck
>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>
>>> We already did some research in that domain.
>>>
>>> Dokan FS is an intent  to write an opensource fuse like cliente for MS
>>> windows.
>>>
>>> More information on DOKANFS can be triggered here
>>> http://dokan-dev.net/en/download/
>>>
>>> Positive points of using DOKANFS.
>>>
>>> - its opensourced and well licenced mit licence, gpl licence and 
>>> lgpl licence.
>>>
>>> Negative point of using DOKAN FS.
>>> - unreachable author
>>> - Poor documentation . Dev comments in japanese.
>>> - Work in progress so it is unstable and needs to be updated, 
>>> debugged and
>>> maintained by a MS Windows file system expert developper.
>> I am not very familiar with windows storage APIs, but somebody told me
>> at once point there were several interfaces against which a new file
>> system could be implemented, everything from a full in-kernel driver to
>> something that is explorer-based.  Are any of those suitable? Using a
>> potentially abandoned fuse-like layer makes me a bit nervous.
>>
>> That said,
>>> I try past year to do a merge from ceph-fuse to dokanfs
>>> here are what I learnt.
>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>> - Ceph protocol isn t documented so it is like trying to draw a map 
>>> of america
>>> using only a sextan and a compass.
>>>
>>> Those led me to those conclusions:
>>> - I can t do it alone.
>>> - It is easier to draw down the ceph protocol way to work from 
>>> kernel/fs/ceph
>>> sources and mount.ceph
>>> - Ceph depending libraries may be unexistant or not up to date in 
>>> their port
>>> on MS Windows (cygwin)
>> I think the most sane path should be to make libcephfs sufficiently
>> portable to build on windows (or cygwin).  For the bits used by the
>> client-side coe, I don't think there should be much in the way of
>> dependencies, and the main challenge would be untangling the build for
>> the necessary pieces out from the rest of Ceph.
>>
>> Have you seen the wip-port portability work that is currently 
>> underway by
>> Noah and Alan?  That may solve many of the cygwin problems you are 
>> seeing
>> today.
>>
>>> - MS file system specialist are hard do find in the "open source 
>>> libre world"
>>> so I will try in the commercial world.
>>>
>>> The commercial world has some problems too. They need ceph protocol 
>>> draft to
>>> implemente it to their own product They will have licencing /commercial
>>> politics that infringe lgpl, and hide that most of the work is done 
>>> by people
>>> other than them. They will not participate in a financial way to ceph
>>> enhancement and growth.
>> I don't think reimplementing the client code is an efficient way 
>> forward.
>> Unless the goal is a pure kernel implementation...but a significant
>> ongoing investment in development resources would be needed for that 
>> going
>> forward.  I suspect that is a challenge for a platform that does not
>> typically rally that sort of community effort.
>>
>> The easiest thing is of course just to use CIFS and Samba (which works
>> today).  A fuse-like approach is probably a reasonably middle ground 
>> (both
>> in initial effort and maintainability going forward)...
>>
>> sage
>>
>>
>
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Yehuda Sadeh]

On Tue, Nov 5, 2013 at 1:49 PM, Alphe Salas Michels <asalas@kepler.cl> wrote:
> Hello sage,
> I followed your lead and went a bit further in my source code reading and I
> notice serveral problems:
> first most of the id use in ceph osd_clients, mds_clients, mon_client use
> the data type u64 which will be a problem
> as most of the windows in use are windows XP or Windows server 2003 in 32
> bits. As they are used for id tag for
> things like snapshot pages (if I understand well) that can be a problem for
> a port no?
>
> What I read so far was
> mount.ceph files > kernel/fs/ceph files which contains the mds_client files
> and the kernel ioctl interface ... >  include / linux / ceph  files which
> integrates libcephfs.h. and all the needed files
>
> there is three clients, there is auth.h a messenger, a msgpool, buffer,
> rados that depends on msgr that include in every aspect of the messages
> formation  one or more __le64 message.
>
> I m far from understanding the whole code and the interactions betwin all
> the files and which we can skip and which we can keep.
>
> how can we translate those data type into 32bit in order for ceph cluster to
> understand them and ceph-windows-client to transmit them?


It is possible to define 64 bit variables on 32 bit architectures. The
compiler handles it for you.

Yehuda

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello sage and Yehuda,
yes nwat told me that u64 was an alias for unsigned long which is 
default C compiler data type.
I wasn't cautious.

I read some other things like the cygwin mount.cc /mount.h files ... 
they include cifs and nfs filetype things like samba too. We could 
integrate a new data type in it with our ceph-win32-client.

Obviously everyone would prefere something with a nice point and click 
window to setup their ceph-win32-client. But as a starter... maybe we 
could understand better the way windows handle file systems starting 
from cygwin/mount files.

Regards,
signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl*<http://www.kepler.cl>*

On 11/05/13 18:59, Yehuda Sadeh wrote:
> On Tue, Nov 5, 2013 at 1:49 PM, Alphe Salas Michels<asalas@kepler.cl>  wrote:
>> Hello sage,
>> I followed your lead and went a bit further in my source code reading and I
>> notice serveral problems:
>> first most of the id use in ceph osd_clients, mds_clients, mon_client use
>> the data type u64 which will be a problem
>> as most of the windows in use are windows XP or Windows server 2003 in 32
>> bits. As they are used for id tag for
>> things like snapshot pages (if I understand well) that can be a problem for
>> a port no?
>>
>> What I read so far was
>> mount.ceph files > kernel/fs/ceph files which contains the mds_client files
>> and the kernel ioctl interface ... >  include / linux / ceph  files which
>> integrates libcephfs.h. and all the needed files
>>
>> there is three clients, there is auth.h a messenger, a msgpool, buffer,
>> rados that depends on msgr that include in every aspect of the messages
>> formation  one or more __le64 message.
>>
>> I m far from understanding the whole code and the interactions betwin all
>> the files and which we can skip and which we can keep.
>>
>> how can we translate those data type into 32bit in order for ceph cluster to
>> understand them and ceph-windows-client to transmit them?
> It is possible to define 64 bit variables on 32 bit architectures. The
> compiler handles it for you.
>
> Yehuda
>
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Sage Weil]

Hi Alphe,

On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> 
> signature *Hi, Sage !
> thank you for you enthousiast reply.
> I sure want to make the best use of everything or anything previously done to
> tend to
> write ceph cliente for windows.
> 
> Apart using libre tools for building the future ceph cliente I am open to
> anything.
> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
> and easyly enhanceable.**
> 
> more free tools can be found here:
> http://www.freebyte.com/programming/cpp/#cppcompilers
> 
> 
> I will read libcephfs source code and take some notes about the protocol.

I think you don't need to worry about hte protocol at all, since libcephs 
implements it for you (and will capture any future changes).

> I was more going from what I know and trying to track down how mount.ceph work
> with the parameters passed to it.
> since it point finally to Kernel/fs/ceph and that I don t really understand
> how that module work and that it probably points to some other dependencies
> Reading libcephfs source code could be a big gain of time.

(I would also ignore mount.ceph as everything it does it specific to 
how Linux mounts work.)

> basically on the protocol what is  need are:
> 
> 1) open and maintain a connection (socket open, auth, etc )
> 2) retreive a map of directories and disk Quota (disk sizing Y TB free,  Z TB
> total)
> 3) procedure to send files / directories in a maner that it will allow our
> client to fit ceph transmission protocols
> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
> Cache for preparing data transfer (a FIFO cache)?)
> 4)Procedure to retreive files / directory from ceph cluster
> 5) Management copy/move files /Directories, FS stats, Connection Stats.
> logging.
> 
> My idea to progress is to take those main bulletpoint in ceph protocol based
> on general ideas of what ceph file system does and start identifying parts
> from libcephfs to match those "needs".

Instead, I would look at include/cephfs/libcephfs.h, the interface that 
libcephfs provides, and try to map that to what the fuse layer expects.  
There is both a path-based that I suspsect lends itself well to the 
Windows interface and (very soon now) a handle based API that is targetted 
at the Unix-style VFS layers.  I'm mostly guessing, though, since I've 
never seen any low-level fs code in windows before.

In this case, the analogous code for Linux should be client/fuse_ll.cc 
itself (and not much else), although there will probably be a few tricks 
necessary to map cleanly onto how the windows interfaces work.

Does that make sense?

Cheers!
sage


> 
> Any suggestion and contributions are welcome.
> 
> 
> *
> On 11/05/13 11:23, Sage Weil wrote:
> > Hi Alphe,
> > 
> > On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> > > Good day developers!
> > > 
> > > I would like to propose to the one interested  work with me to develop a
> > > ceph
> > > cliente for MS windows world, Basing us on dokanFS.
> > > 
> > > My company is a ceph enthousiast that use on a dayly basis ceph and that
> > > need
> > > both transfer speed and big expendable and cheap storage.
> > > My company is specialised in data recovery and we want to participate to
> > > ceph
> > > effort by bringing a ceph cliente for windows.
> > Awesome!
> > 
> > > Our experience shows us that the best gateway is each clientes being its
> > > own
> > > gateway, instead of having a bottle neck server or a cluster of bottle
> > > neck
> > > servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
> > > 
> > > We already did some research in that domain.
> > > 
> > > Dokan FS is an intent  to write an opensource fuse like cliente for MS
> > > windows.
> > > 
> > > More information on DOKANFS can be triggered here
> > > http://dokan-dev.net/en/download/
> > > 
> > > Positive points of using DOKANFS.
> > > 
> > > - its opensourced and well licenced mit licence, gpl licence and lgpl
> > > licence.
> > > 
> > > Negative point of using DOKAN FS.
> > > - unreachable author
> > > - Poor documentation . Dev comments in japanese.
> > > - Work in progress so it is unstable and needs to be updated, debugged and
> > > maintained by a MS Windows file system expert developper.
> > I am not very familiar with windows storage APIs, but somebody told me
> > at once point there were several interfaces against which a new file
> > system could be implemented, everything from a full in-kernel driver to
> > something that is explorer-based.  Are any of those suitable?  Using a
> > potentially abandoned fuse-like layer makes me a bit nervous.
> > 
> > That said,
> >   
> > > I try past year to do a merge from ceph-fuse to dokanfs
> > > here are what I learnt.
> > > - Ceph-fuse and related source code is around 60 000 lines of code.
> > > - Ceph protocol isn t documented so it is like trying to draw a map of
> > > america
> > > using only a sextan and a compass.
> > > 
> > > Those led me to those conclusions:
> > > - I can t do it alone.
> > > - It is easier to draw down the ceph protocol way to work from
> > > kernel/fs/ceph
> > > sources and mount.ceph
> > > - Ceph depending libraries may be unexistant or not up to date in their
> > > port
> > > on MS Windows (cygwin)
> > I think the most sane path should be to make libcephfs sufficiently
> > portable to build on windows (or cygwin).  For the bits used by the
> > client-side coe, I don't think there should be much in the way of
> > dependencies, and the main challenge would be untangling the build for
> > the necessary pieces out from the rest of Ceph.
> > 
> > Have you seen the wip-port portability work that is currently underway by
> > Noah and Alan?  That may solve many of the cygwin problems you are seeing
> > today.
> > 
> > > - MS file system specialist are hard do find in the "open source libre
> > > world"
> > > so I will try in the commercial world.
> > > 
> > > The commercial world has some problems too. They need ceph protocol draft
> > > to
> > > implemente it to their own product They will have licencing /commercial
> > > politics that infringe lgpl, and hide that most of the work is done by
> > > people
> > > other than them. They will not participate in a financial way to ceph
> > > enhancement and growth.
> > I don't think reimplementing the client code is an efficient way forward.
> > Unless the goal is a pure kernel implementation...but a significant
> > ongoing investment in development resources would be needed for that going
> > forward.  I suspect that is a challenge for a platform that does not
> > typically rally that sort of community effort.
> > 
> > The easiest thing is of course just to use CIFS and Samba (which works
> > today).  A fuse-like approach is probably a reasonably middle ground (both
> > in initial effort and maintainability going forward)...
> > 
> > sage
> > 
> > 
> 
> 

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hi Sage,
When I initially planned past year to  do a client for ceph on windows 
what I inspected was ceph_fuse.cc
from there I tracked all the related files using a script that read the 
includes precompiler instructions and
retreive the related files from the ceph source code git master branch. 
In the end the work seemed to me gigantic.

Now that we have an open discussion on that topic and that you giveme 
the libcephfs.h and fuseII.cc hints I will start
by isolate the source code related. I will modify my previous isolate 
script to do so. I will create a github branch under ceph if that is ok 
with you, to put the documentation, the isolated files and the dev tools 
related.


Regards,

--- For notes my script working from content and related content 
ceph_fuse.cc produced this:

find . -name '*' | xargs wc -l
wc: .: Is a directory
       0 .
wc: ./common: Is a directory
       0 ./common
      29 ./common/compiler_extensions.h
      39 ./common/simple_spin.h
      31 ./common/config_obs.h
     481 ./common/admin_socket.cc
      33 ./common/pipe.h
     119 ./common/LogEntry.h
     135 ./common/DoutStreambuf.h
     111 ./common/Cond.h
      42 ./common/code_environment.h
     112 ./common/Formatter.h
     760 ./common/config.cc
      52 ./common/escape.h
     644 ./common/DoutStreambuf.cc
     202 ./common/LogClient.cc
     573 ./common/ConfUtils.cc
     199 ./common/escape.c
      84 ./common/Timer.h
      25 ./common/hex.h
     395 ./common/Formatter.cc
      52 ./common/safe_io.h
      82 ./common/HeartbeatMap.h
      74 ./common/code_environment.cc
      17 ./common/errno.cc
      42 ./common/signal.h
     144 ./common/Mutex.h
      96 ./common/admin_socket.h
      83 ./common/common_init.h
      34 ./common/BackTrace.h
      29 ./common/version.h
      74 ./common/snap_types.h
     238 ./common/ceph_context.cc
      84 ./common/Finisher.cc
      74 ./common/ceph_argparse.h
     187 ./common/utf8.c
      39 ./common/Clock.cc
      50 ./common/Thread.h
     148 ./common/HeartbeatMap.cc
      51 ./common/utf8.h
     160 ./common/Thread.cc
       9 ./common/errno.h
      62 ./common/BackTrace.cc
      39 ./common/hex.cc
     120 ./common/safe_io.c
      27 ./common/Clock.h
     161 ./common/DecayCounter.h
     189 ./common/Timer.cc
     119 ./common/Throttle.h
     126 ./common/strtol.cc
     320 ./common/perf_counters.cc
     119 ./common/LogClient.h
      33 ./common/debug.h
      81 ./common/signal.cc
      44 ./common/pipe.c
     210 ./common/config.h
      24 ./common/static_assert.h
     186 ./common/entity_name.cc
     124 ./common/armor.c
      24 ./common/likely.h
      43 ./common/simple_spin.cc
      73 ./common/ceph_crypto.cc
      79 ./common/entity_name.h
      41 ./common/version.cc
     106 ./common/dout.h
      28 ./common/strtol.h
      86 ./common/ConfUtils.h
      86 ./common/Finisher.h
     108 ./common/LogEntry.cc
     179 ./common/perf_counters.h
      16 ./common/armor.h
      93 ./common/snap_types.cc
     409 ./common/ceph_argparse.cc
     110 ./common/common_init.cc
     114 ./common/ceph_context.h
     160 ./common/ceph_crypto.h
wc: ./msg: Is a directory
       0 ./msg
     623 ./msg/SimpleMessenger.h
     673 ./msg/Message.cc
     263 ./msg/Messenger.h
     478 ./msg/Message.h
     153 ./msg/msg_types.cc
      64 ./msg/Dispatcher.h
    2824 ./msg/SimpleMessenger.cc
     425 ./msg/msg_types.h
wc: ./messages: Is a directory
       0 ./messages
      81 ./messages/MOSDPGCreate.h
      67 ./messages/MRoute.h
      72 ./messages/MForward.h
      83 ./messages/MMonElection.h
      44 ./messages/MPGStatsAck.h
      67 ./messages/MClientSession.h
      65 ./messages/MOSDBoot.h
      90 ./messages/MClientReconnect.h
      69 ./messages/MOSDFailure.h
      78 ./messages/MOSDPing.h
      47 ./messages/MLogAck.h
     206 ./messages/MClientRequest.h
      51 ./messages/MClientCapRelease.h
      57 ./messages/MMonObserve.h
      50 ./messages/MMonSubscribeAck.h
      57 ./messages/MOSDPGMissing.h
      66 ./messages/MClientSnap.h
      55 ./messages/MCommandReply.h
     106 ./messages/MMonProbe.h
      64 ./messages/MOSDPGQuery.h
      85 ./messages/MOSDPGNotify.h
      95 ./messages/MMonSubscribe.h
     223 ./messages/MOSDSubOp.h
      55 ./messages/MGetPoolStatsReply.h
      62 ./messages/MMonObserveNotify.h
      64 ./messages/PaxosServiceMessage.h
      59 ./messages/MMonJoin.h
     101 ./messages/MPoolOp.h
      58 ./messages/MLog.h
      57 ./messages/MAuth.h
      44 ./messages/MStatfsReply.h
      54 ./messages/MMonCommandAck.h
      35 ./messages/MMonGetMap.h
      78 ./messages/MOSDPGLog.h
      52 ./messages/MStatfs.h
      54 ./messages/MOSDPGTrim.h
     172 ./messages/MClientCaps.h
     130 ./messages/MMonPaxos.h
     143 ./messages/MOSDMap.h
     222 ./messages/MOSDOpReply.h
      62 ./messages/MOSDPGRemove.h
      81 ./messages/MOSDPGScan.h
      39 ./messages/MGenericMessage.h
     146 ./messages/MOSDSubOpReply.h
      54 ./messages/MOSDPGInfo.h
      52 ./messages/MOSDPGTemp.h
      70 ./messages/MOSDRepScrub.h
     388 ./messages/MOSDOp.h
      63 ./messages/MClientRequestForward.h
      64 ./messages/MMonGetVersionReply.h
      66 ./messages/MOSDScrub.h
      59 ./messages/MCommand.h
     266 ./messages/MClientReply.h
      58 ./messages/MMonGetVersion.h
      61 ./messages/MPGStats.h
      49 ./messages/MOSDAlive.h
      51 ./messages/MRemoveSnaps.h
      83 ./messages/MClientLease.h
      35 ./messages/MPing.h
      83 ./messages/MOSDPGBackfill.h
      60 ./messages/MMonCommand.h
      80 ./messages/MPoolOpReply.h
      57 ./messages/MGetPoolStats.h
      96 ./messages/MMDSMap.h
      43 ./messages/MMonMap.h
      65 ./messages/MAuthReply.h
      12 ./ceph_ver
     186 ./acconfig
wc: ./osdc: Is a directory
       0 ./osdc
     283 ./osdc/Filer.h
    2023 ./osdc/Objecter.cc
    1774 ./osdc/ObjectCacher.cc
    1479 ./osdc/Objecter.h
     571 ./osdc/rados_bencher.h
     388 ./osdc/Filer.cc
     694 ./osdc/ObjectCacher.h
wc: ./include: Is a directory
       0 ./include
     600 ./include/frag.h
      28 ./include/addr_parsing.h
      43 ./include/ceph_features.h
      18 ./include/page.h
      22 ./include/compat.h
     192 ./include/CompatSet.h
     335 ./include/lru.h
     253 ./include/utime.h
     113 ./include/assert.h
      47 ./include/blobhash.h
      83 ./include/ceph_fs.cc
     786 ./include/encoding.h
     547 ./include/interval_set.h
     312 ./include/Context.h
     499 ./include/buffer.h
     750 ./include/ceph_fs.h
     183 ./include/msgr.h
     124 ./include/atomic.h
      29 ./include/err.h
     225 ./include/filepath.h
      38 ./include/intarith.h
      14 ./include/str_list.h
      13 ./include/color.h
      28 ./include/inttypes.h
     428 ./include/types.h
     150 ./include/addr_parsing.c
     105 ./include/cmp.h
     411 ./include/rados.h
     192 ./include/object.h
     165 ./include/xlist.h
wc: ./client: Is a directory
       0 ./client
     627 ./client/fuse_ll.cc
     661 ./client/Client.h
      15 ./client/fuse_ll.h
    6923 ./client/Client.cc
     177 ./ceph_fuse
wc: ./none: Is a directory
       0 ./none
wc: ./mds: Is a directory
       0 ./mds
     253 ./mds/MDSMap.cc
      74 ./mds/inode_backtrace.h
    1581 ./mds/mdstypes.h
     622 ./mds/MDSMap.h
wc: ./global: Is a directory
       0 ./global
      28 ./global/pidfile.h
      98 ./global/pidfile.cc
      45 ./global/signal_handler.h
     338 ./global/signal_handler.cc
     226 ./global/global_init.cc
      23 ./global/global_context.cc
      64 ./global/global_init.h
      33 ./global/global_context.h
wc: ./json_spirit: Is a directory
       0 ./json_spirit
       8 ./json_spirit/json_spirit_value.cpp
     585 ./json_spirit/json_spirit_value.h
wc: ./cephx: Is a directory
       0 ./cephx
wc: ./os: Is a directory
       0 ./os
      79 ./os/hobject.cc
     639 ./os/ObjectStore.cc
     129 ./os/hobject.h
     738 ./os/ObjectStore.h
    1378 ./rados
wc: ./crush: Is a directory
       0 ./crush
     518 ./crush/CrushWrapper.cc
      47 ./crush/CrushWrapper.i
     427 ./crush/CrushWrapper.h
wc: ./mon: Is a directory
       0 ./mon
     406 ./mon/MonCaps.cc
     106 ./mon/MonCaps.h
      40 ./mon/mon_types.h
     158 ./mon/Session.h
     810 ./mon/MonClient.cc
     267 ./mon/MonClient.h
     130 ./mon/MonMap.cc
     204 ./mon/MonMap.h
wc: ./osd: Is a directory
       0 ./osd
     694 ./osd/OSDMap.h
    1755 ./osd/osd_types.h
    1310 ./osd/OSDMap.cc
    2464 ./osd/osd_types.cc
wc: ./auth: Is a directory
       0 ./auth
      33 ./auth/AuthServiceHandler.cc
     118 ./auth/Crypto.h
      37 ./auth/AuthSupported.h
      44 ./auth/AuthServiceHandler.h
      85 ./auth/KeyRing.h
     437 ./auth/Crypto.cc
      54 ./auth/RotatingKeyRing.h
wc: ./auth/none: Is a directory
       0 ./auth/none
      52 ./auth/none/AuthNoneClientHandler.h
      31 ./auth/none/AuthNoneAuthorizeHandler.h
      41 ./auth/none/AuthNoneServiceHandler.h
      32 ./auth/none/AuthNoneProtocol.h
      39 ./auth/none/AuthNoneAuthorizeHandler.cc
      77 ./auth/RotatingKeyRing.cc
     259 ./auth/KeyRing.cc
      39 ./auth/AuthClientHandler.cc
      89 ./auth/AuthClientHandler.h
wc: ./auth/cephx: Is a directory
       0 ./auth/cephx
      69 ./auth/cephx/CephxClientHandler.h
     505 ./auth/cephx/CephxProtocol.cc
     209 ./auth/cephx/CephxClientHandler.cc
     418 ./auth/cephx/CephxKeyServer.cc
      33 ./auth/cephx/CephxAuthorizeHandler.cc
     203 ./auth/cephx/CephxServiceHandler.cc
     487 ./auth/cephx/CephxProtocol.h
      31 ./auth/cephx/CephxAuthorizeHandler.h
     298 ./auth/cephx/CephxKeyServer.h
      37 ./auth/cephx/CephxServiceHandler.h
     246 ./auth/Auth.h
      51 ./auth/AuthSupported.cc
   63501 total

So ceph_fuse.cc and related files /dependencies as a ceph related only 
layer involves 63501 lines of source code.
note that fuseII.cc is part of that extraction but libcephfs no.

signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl

On 11/05/13 21:00, Sage Weil wrote:
> Hi Alphe,
>
> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>> signature *Hi, Sage !
>> thank you for you enthousiast reply.
>> I sure want to make the best use of everything or anything previously done to
>> tend to
>> write ceph cliente for windows.
>>
>> Apart using libre tools for building the future ceph cliente I am open to
>> anything.
>> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
>> and easyly enhanceable.**
>>
>> more free tools can be found here:
>> http://www.freebyte.com/programming/cpp/#cppcompilers
>>
>>
>> I will read libcephfs source code and take some notes about the protocol.
> I think you don't need to worry about hte protocol at all, since libcephs
> implements it for you (and will capture any future changes).
>
>> I was more going from what I know and trying to track down how mount.ceph work
>> with the parameters passed to it.
>> since it point finally to Kernel/fs/ceph and that I don t really understand
>> how that module work and that it probably points to some other dependencies
>> Reading libcephfs source code could be a big gain of time.
> (I would also ignore mount.ceph as everything it does it specific to
> how Linux mounts work.)
>
>> basically on the protocol what is  need are:
>>
>> 1) open and maintain a connection (socket open, auth, etc )
>> 2) retreive a map of directories and disk Quota (disk sizing Y TB free,  Z TB
>> total)
>> 3) procedure to send files / directories in a maner that it will allow our
>> client to fit ceph transmission protocols
>> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
>> Cache for preparing data transfer (a FIFO cache)?)
>> 4)Procedure to retreive files / directory from ceph cluster
>> 5) Management copy/move files /Directories, FS stats, Connection Stats.
>> logging.
>>
>> My idea to progress is to take those main bulletpoint in ceph protocol based
>> on general ideas of what ceph file system does and start identifying parts
>> from libcephfs to match those "needs".
> Instead, I would look at include/cephfs/libcephfs.h, the interface that
> libcephfs provides, and try to map that to what the fuse layer expects.
> There is both a path-based that I suspsect lends itself well to the
> Windows interface and (very soon now) a handle based API that is targetted
> at the Unix-style VFS layers.  I'm mostly guessing, though, since I've
> never seen any low-level fs code in windows before.
>
> In this case, the analogous code for Linux should be client/fuse_ll.cc
> itself (and not much else), although there will probably be a few tricks
> necessary to map cleanly onto how the windows interfaces work.
>
> Does that make sense?
>
> Cheers!
> sage
>
>
>> Any suggestion and contributions are welcome.
>>
>>
>> *
>> On 11/05/13 11:23, Sage Weil wrote:
>>> Hi Alphe,
>>>
>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>> Good day developers!
>>>>
>>>> I would like to propose to the one interested  work with me to develop a
>>>> ceph
>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>
>>>> My company is a ceph enthousiast that use on a dayly basis ceph and that
>>>> need
>>>> both transfer speed and big expendable and cheap storage.
>>>> My company is specialised in data recovery and we want to participate to
>>>> ceph
>>>> effort by bringing a ceph cliente for windows.
>>> Awesome!
>>>
>>>> Our experience shows us that the best gateway is each clientes being its
>>>> own
>>>> gateway, instead of having a bottle neck server or a cluster of bottle
>>>> neck
>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>>
>>>> We already did some research in that domain.
>>>>
>>>> Dokan FS is an intent  to write an opensource fuse like cliente for MS
>>>> windows.
>>>>
>>>> More information on DOKANFS can be triggered here
>>>> http://dokan-dev.net/en/download/
>>>>
>>>> Positive points of using DOKANFS.
>>>>
>>>> - its opensourced and well licenced mit licence, gpl licence and lgpl
>>>> licence.
>>>>
>>>> Negative point of using DOKAN FS.
>>>> - unreachable author
>>>> - Poor documentation . Dev comments in japanese.
>>>> - Work in progress so it is unstable and needs to be updated, debugged and
>>>> maintained by a MS Windows file system expert developper.
>>> I am not very familiar with windows storage APIs, but somebody told me
>>> at once point there were several interfaces against which a new file
>>> system could be implemented, everything from a full in-kernel driver to
>>> something that is explorer-based.  Are any of those suitable?  Using a
>>> potentially abandoned fuse-like layer makes me a bit nervous.
>>>
>>> That said,
>>>    
>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>> here are what I learnt.
>>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>>> - Ceph protocol isn t documented so it is like trying to draw a map of
>>>> america
>>>> using only a sextan and a compass.
>>>>
>>>> Those led me to those conclusions:
>>>> - I can t do it alone.
>>>> - It is easier to draw down the ceph protocol way to work from
>>>> kernel/fs/ceph
>>>> sources and mount.ceph
>>>> - Ceph depending libraries may be unexistant or not up to date in their
>>>> port
>>>> on MS Windows (cygwin)
>>> I think the most sane path should be to make libcephfs sufficiently
>>> portable to build on windows (or cygwin).  For the bits used by the
>>> client-side coe, I don't think there should be much in the way of
>>> dependencies, and the main challenge would be untangling the build for
>>> the necessary pieces out from the rest of Ceph.
>>>
>>> Have you seen the wip-port portability work that is currently underway by
>>> Noah and Alan?  That may solve many of the cygwin problems you are seeing
>>> today.
>>>
>>>> - MS file system specialist are hard do find in the "open source libre
>>>> world"
>>>> so I will try in the commercial world.
>>>>
>>>> The commercial world has some problems too. They need ceph protocol draft
>>>> to
>>>> implemente it to their own product They will have licencing /commercial
>>>> politics that infringe lgpl, and hide that most of the work is done by
>>>> people
>>>> other than them. They will not participate in a financial way to ceph
>>>> enhancement and growth.
>>> I don't think reimplementing the client code is an efficient way forward.
>>> Unless the goal is a pure kernel implementation...but a significant
>>> ongoing investment in development resources would be needed for that going
>>> forward.  I suspect that is a challenge for a platform that does not
>>> typically rally that sort of community effort.
>>>
>>> The easiest thing is of course just to use CIFS and Samba (which works
>>> today).  A fuse-like approach is probably a reasonably middle ground (both
>>> in initial effort and maintainability going forward)...
>>>
>>> sage
>>>
>>>
>>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: writing a ceph cliente for MS windows

[Alphe Salas Michels]

Hello I created the github repository for this project 
https://github.com/alphe/Ceph4Win

Regards,
signature

*Alphé Salas*
Ingeniero T.I

asalas@kepler.cl
*<http://www.kepler.cl>*
On 11/05/13 21:00, Sage Weil wrote:
> Hi Alphe,
>
> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>> signature *Hi, Sage !
>> thank you for you enthousiast reply.
>> I sure want to make the best use of everything or anything previously done to
>> tend to
>> write ceph cliente for windows.
>>
>> Apart using libre tools for building the future ceph cliente I am open to
>> anything.
>> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
>> and easyly enhanceable.**
>>
>> more free tools can be found here:
>> http://www.freebyte.com/programming/cpp/#cppcompilers
>>
>>
>> I will read libcephfs source code and take some notes about the protocol.
> I think you don't need to worry about hte protocol at all, since libcephs
> implements it for you (and will capture any future changes).
>
>> I was more going from what I know and trying to track down how mount.ceph work
>> with the parameters passed to it.
>> since it point finally to Kernel/fs/ceph and that I don t really understand
>> how that module work and that it probably points to some other dependencies
>> Reading libcephfs source code could be a big gain of time.
> (I would also ignore mount.ceph as everything it does it specific to
> how Linux mounts work.)
>
>> basically on the protocol what is  need are:
>>
>> 1) open and maintain a connection (socket open, auth, etc )
>> 2) retreive a map of directories and disk Quota (disk sizing Y TB free,  Z TB
>> total)
>> 3) procedure to send files / directories in a maner that it will allow our
>> client to fit ceph transmission protocols
>> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
>> Cache for preparing data transfer (a FIFO cache)?)
>> 4)Procedure to retreive files / directory from ceph cluster
>> 5) Management copy/move files /Directories, FS stats, Connection Stats.
>> logging.
>>
>> My idea to progress is to take those main bulletpoint in ceph protocol based
>> on general ideas of what ceph file system does and start identifying parts
>> from libcephfs to match those "needs".
> Instead, I would look at include/cephfs/libcephfs.h, the interface that
> libcephfs provides, and try to map that to what the fuse layer expects.
> There is both a path-based that I suspsect lends itself well to the
> Windows interface and (very soon now) a handle based API that is targetted
> at the Unix-style VFS layers.  I'm mostly guessing, though, since I've
> never seen any low-level fs code in windows before.
>
> In this case, the analogous code for Linux should be client/fuse_ll.cc
> itself (and not much else), although there will probably be a few tricks
> necessary to map cleanly onto how the windows interfaces work.
>
> Does that make sense?
>
> Cheers!
> sage
>
>
>> Any suggestion and contributions are welcome.
>>
>>
>> *
>> On 11/05/13 11:23, Sage Weil wrote:
>>> Hi Alphe,
>>>
>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>> Good day developers!
>>>>
>>>> I would like to propose to the one interested  work with me to develop a
>>>> ceph
>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>
>>>> My company is a ceph enthousiast that use on a dayly basis ceph and that
>>>> need
>>>> both transfer speed and big expendable and cheap storage.
>>>> My company is specialised in data recovery and we want to participate to
>>>> ceph
>>>> effort by bringing a ceph cliente for windows.
>>> Awesome!
>>>
>>>> Our experience shows us that the best gateway is each clientes being its
>>>> own
>>>> gateway, instead of having a bottle neck server or a cluster of bottle
>>>> neck
>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>>
>>>> We already did some research in that domain.
>>>>
>>>> Dokan FS is an intent  to write an opensource fuse like cliente for MS
>>>> windows.
>>>>
>>>> More information on DOKANFS can be triggered here
>>>> http://dokan-dev.net/en/download/
>>>>
>>>> Positive points of using DOKANFS.
>>>>
>>>> - its opensourced and well licenced mit licence, gpl licence and lgpl
>>>> licence.
>>>>
>>>> Negative point of using DOKAN FS.
>>>> - unreachable author
>>>> - Poor documentation . Dev comments in japanese.
>>>> - Work in progress so it is unstable and needs to be updated, debugged and
>>>> maintained by a MS Windows file system expert developper.
>>> I am not very familiar with windows storage APIs, but somebody told me
>>> at once point there were several interfaces against which a new file
>>> system could be implemented, everything from a full in-kernel driver to
>>> something that is explorer-based.  Are any of those suitable?  Using a
>>> potentially abandoned fuse-like layer makes me a bit nervous.
>>>
>>> That said,
>>>    
>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>> here are what I learnt.
>>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>>> - Ceph protocol isn t documented so it is like trying to draw a map of
>>>> america
>>>> using only a sextan and a compass.
>>>>
>>>> Those led me to those conclusions:
>>>> - I can t do it alone.
>>>> - It is easier to draw down the ceph protocol way to work from
>>>> kernel/fs/ceph
>>>> sources and mount.ceph
>>>> - Ceph depending libraries may be unexistant or not up to date in their
>>>> port
>>>> on MS Windows (cygwin)
>>> I think the most sane path should be to make libcephfs sufficiently
>>> portable to build on windows (or cygwin).  For the bits used by the
>>> client-side coe, I don't think there should be much in the way of
>>> dependencies, and the main challenge would be untangling the build for
>>> the necessary pieces out from the rest of Ceph.
>>>
>>> Have you seen the wip-port portability work that is currently underway by
>>> Noah and Alan?  That may solve many of the cygwin problems you are seeing
>>> today.
>>>
>>>> - MS file system specialist are hard do find in the "open source libre
>>>> world"
>>>> so I will try in the commercial world.
>>>>
>>>> The commercial world has some problems too. They need ceph protocol draft
>>>> to
>>>> implemente it to their own product They will have licencing /commercial
>>>> politics that infringe lgpl, and hide that most of the work is done by
>>>> people
>>>> other than them. They will not participate in a financial way to ceph
>>>> enhancement and growth.
>>> I don't think reimplementing the client code is an efficient way forward.
>>> Unless the goal is a pure kernel implementation...but a significant
>>> ongoing investment in development resources would be needed for that going
>>> forward.  I suspect that is a challenge for a platform that does not
>>> typically rally that sort of community effort.
>>>
>>> The easiest thing is of course just to use CIFS and Samba (which works
>>> today).  A fuse-like approach is probably a reasonably middle ground (both
>>> in initial effort and maintainability going forward)...
>>>
>>> sage
>>>
>>>
>>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: writing a ceph cliente for MS windows

[James Harper]

> Good day developers!
> 
> I would like to propose to the one interested  work with me to develop a
> ceph cliente for MS windows world, Basing us on dokanFS.
> 

I've looked at porting the rbd client to windows a little while back. That would require a kernel driver and all the rbd stuff is C++ (windows kernel is strictly C) so it was looking like a lot of work. Porting the rbd kernel implementation from Linux would have been easier.

An alternative would be to write a fuse-like kernel driver that talks to a userspace implementation of librbd. That would have been slower though, and have some restrictions like no swap space and no booting (although booting would have been a hard problem to solve even with a kernel-only implementation...)

Good luck!

James