* rbd lock list command failure [not found] <638484117.21434546.1441815449448.JavaMail.zimbra@redhat.com> @ 2015-09-09 16:26 ` Huamin Chen 2015-09-14 22:35 ` Josh Durgin 0 siblings, 1 reply; 4+ messages in thread From: Huamin Chen @ 2015-09-09 16:26 UTC (permalink / raw) To: ceph-devel Hi Running "rbd lock list" inside a Docker container yields mixed results. Sometimes I can get the right results but most times I just get errors. A good run is like this: [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== 2015-09-09 16:11:16.812858 7f3337ff5840 -1 did not load config file, using default settings. There is 1 exclusive lock on this image. Locker ID Address client.4333 kubelet_lock_magic_host 10.16.154.78:0/1000001 Same commands but with error: [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== 2015-09-09 16:11:30.430345 7f4159766840 -1 did not load config file, using default settings. 2015-09-09 16:11:30.464193 7f4159766840 -1 librbd::ImageCtx: error finding header: (108) Cannot send after transport endpoint shutdown rbd: error opening image foo: (108) Cannot send after transport endpoint shutdown rbd version: [root@host server]# docker run -ti ceph/base ceph -v ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) ceph server version: # ceph -v ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047) ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: rbd lock list command failure 2015-09-09 16:26 ` rbd lock list command failure Huamin Chen @ 2015-09-14 22:35 ` Josh Durgin 2015-09-15 13:10 ` Huamin Chen 0 siblings, 1 reply; 4+ messages in thread From: Josh Durgin @ 2015-09-14 22:35 UTC (permalink / raw) To: Huamin Chen, ceph-devel On 09/09/2015 09:26 AM, Huamin Chen wrote: > Hi > > Running "rbd lock list" inside a Docker container yields mixed results. Sometimes I can get the right results but most times I just get errors. > > A good run is like this: > [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== > 2015-09-09 16:11:16.812858 7f3337ff5840 -1 did not load config file, using default settings. > There is 1 exclusive lock on this image. > Locker ID Address > client.4333 kubelet_lock_magic_host 10.16.154.78:0/1000001 > > Same commands but with error: > [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== > 2015-09-09 16:11:30.430345 7f4159766840 -1 did not load config file, using default settings. > 2015-09-09 16:11:30.464193 7f4159766840 -1 librbd::ImageCtx: error finding header: (108) Cannot send after transport endpoint shutdown > rbd: error opening image foo: (108) Cannot send after transport endpoint shutdown Are you using blacklisting? That is the error that a blacklisted client gets from rados. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: rbd lock list command failure 2015-09-14 22:35 ` Josh Durgin @ 2015-09-15 13:10 ` Huamin Chen 2015-09-15 15:59 ` Josh Durgin 0 siblings, 1 reply; 4+ messages in thread From: Huamin Chen @ 2015-09-15 13:10 UTC (permalink / raw) To: Josh Durgin; +Cc: ceph-devel Hey Josh, Indeed, from "ceph -w", I found my client got blacklisted the moment the "rbd lock" command was executed. I think this might be caused by my configuration that both Ceph OSD and rbd command are running inside the containers on the same host. Thanks Huamin ----- Original Message ----- From: "Josh Durgin" <josh.durgin@inktank.com> To: "Huamin Chen" <hchen@redhat.com>, ceph-devel@vger.kernel.org Sent: Monday, September 14, 2015 6:35:49 PM Subject: Re: rbd lock list command failure On 09/09/2015 09:26 AM, Huamin Chen wrote: > Hi > > Running "rbd lock list" inside a Docker container yields mixed results. Sometimes I can get the right results but most times I just get errors. > > A good run is like this: > [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== > 2015-09-09 16:11:16.812858 7f3337ff5840 -1 did not load config file, using default settings. > There is 1 exclusive lock on this image. > Locker ID Address > client.4333 kubelet_lock_magic_host 10.16.154.78:0/1000001 > > Same commands but with error: > [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== > 2015-09-09 16:11:30.430345 7f4159766840 -1 did not load config file, using default settings. > 2015-09-09 16:11:30.464193 7f4159766840 -1 librbd::ImageCtx: error finding header: (108) Cannot send after transport endpoint shutdown > rbd: error opening image foo: (108) Cannot send after transport endpoint shutdown Are you using blacklisting? That is the error that a blacklisted client gets from rados. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: rbd lock list command failure 2015-09-15 13:10 ` Huamin Chen @ 2015-09-15 15:59 ` Josh Durgin 0 siblings, 0 replies; 4+ messages in thread From: Josh Durgin @ 2015-09-15 15:59 UTC (permalink / raw) To: Huamin Chen; +Cc: ceph-devel On 09/15/2015 06:10 AM, Huamin Chen wrote: > Hey Josh, > > Indeed, from "ceph -w", I found my client got blacklisted the moment the "rbd lock" command was executed. > > I think this might be caused by my configuration that both Ceph OSD and rbd command are running inside the containers on the same host. Yeah, that's probably due to this bug: http://tracker.ceph.com/issues/13032 A previously blacklisted client might end up with the same client address as a future process due to running in containers, and librados using the pid as part of that address. Josh > > ----- Original Message ----- > From: "Josh Durgin" <josh.durgin@inktank.com> > To: "Huamin Chen" <hchen@redhat.com>, ceph-devel@vger.kernel.org > Sent: Monday, September 14, 2015 6:35:49 PM > Subject: Re: rbd lock list command failure > > On 09/09/2015 09:26 AM, Huamin Chen wrote: >> Hi >> >> Running "rbd lock list" inside a Docker container yields mixed results. Sometimes I can get the right results but most times I just get errors. >> >> A good run is like this: >> [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== >> 2015-09-09 16:11:16.812858 7f3337ff5840 -1 did not load config file, using default settings. >> There is 1 exclusive lock on this image. >> Locker ID Address >> client.4333 kubelet_lock_magic_host 10.16.154.78:0/1000001 >> >> Same commands but with error: >> [root@host server]# docker run --privileged --net=host -v /dev:/dev -v /sys:/sys ceph/base rbd lock list foo --pool kube --id kube -m host:6789 --key=AQAMgXhVwBCeDhAA9nlPaFyfUSatGD4drFWDvQ== >> 2015-09-09 16:11:30.430345 7f4159766840 -1 did not load config file, using default settings. >> 2015-09-09 16:11:30.464193 7f4159766840 -1 librbd::ImageCtx: error finding header: (108) Cannot send after transport endpoint shutdown >> rbd: error opening image foo: (108) Cannot send after transport endpoint shutdown > > Are you using blacklisting? That is the error that a blacklisted client > gets from rados. > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-09-15 15:59 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <638484117.21434546.1441815449448.JavaMail.zimbra@redhat.com>
2015-09-09 16:26 ` rbd lock list command failure Huamin Chen
2015-09-14 22:35 ` Josh Durgin
2015-09-15 13:10 ` Huamin Chen
2015-09-15 15:59 ` Josh Durgin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.