Re: [PATCH v2] Samba RADOS service registration

[Jeff Layton <jlayton@samba.org>]

On Thu, 2018-02-15 at 19:52 +0100, David Disseldorp wrote:
> Hi Jeff,
> 
> On Thu, 15 Feb 2018 13:09:45 -0500, Jeff Layton wrote:
> 
> > IIUC, if the program (samba in this case) dies, and then is restarted
> > (maybe host reboots?), it'll get a new instance_guid, right? Or am I
> > misunderstanding what rados_get_instance_id will return here?
> 
> Yes, that's right. On smbd restart it'll reconnect and get a new
> instance ID. The hostname is carried in the metadata, which makes it a
> little easier to identify what's going on from the Ceph side.
> 
> AFAICT the service record (unfortunately) remains present after
> rados_shutdown() until a timeout on the manager is tripped, so
> it's quite possible to have to registrations for the same host for a
> short amount of time.
> 
> Cheers, David

Got it, thanks. I guess the fact that it changes it not a problem here?
I'm not that well versed in what ceph-mgr does, tbqh...

One thing you may want to be careful about: I found some thread safety
problems a few months ago in how CephContext objects are handled in the
ceph code. While I patched them up as best I could, a lot of older
shipping versions still have those bugs. Even with those fixes, I think
there may still be lurking problems when we have multiple ceph/rados
clients within the same process image.

The upshot here is that if you have the rados client here and (e.g.) a
vfs_ceph client for exporting cephfs, you can hit some races
particularly on setup and teardown of those clients that can cause
crashes.

libcephfs has a ceph_create_with_context image and librados has
something similar. You might consider having a truly global cct object,
and then have the vfs_ceph create a client handle with the same object.

-- 
Jeff Layton <jlayton@samba.org>