Mounting

Mounting

[Zenon Panoussis]

I'm facing yet another one of "those problems", more likely caused by
lack of knowledge and/or documentation than by anything else.

# mount.ceph 192.168.178.100:6789:/ /vol
mount error 22 = Invalid argument

# dmesg -c
libceph: no secret set (for auth_x protocol)
libceph: error -22 on auth protocol 2 init

It looks very much like http://marc.info/?l=ceph-devel&m=130072087704874&w=2 .
Yes, I have enabled cephx, but I have no clue how to tell the client what key
to use. I tried

# cauthtool -l /etc/ceph/admin.keyring
# mount.ceph 192.168.178.100:6789:/ /vol -v -o name=<auid>,secret=<key>

but all I got was

ceph: Unknown mount option name
ceph: Unknown mount option secret
mount error 1 = Operation not permitted

Yet, 'man mount.ceph' says

name	string, used with authx, Default: guest
secret	string, used with authx

Commenting out "auth supported = cephx" gets rid of the problem and
lets the store be mounted, but it's a workaround, not a solution.

Any ideas? This is ceph-0.26 with vanilla kernel 2.6.38.2.

Z

Re: Mounting

[Tommi Virtanen]

On Fri, Apr 15, 2011 at 01:44:39AM +0200, Zenon Panoussis wrote:
> I'm facing yet another one of "those problems", more likely caused by
> lack of knowledge and/or documentation than by anything else.
> 
> # mount.ceph 192.168.178.100:6789:/ /vol
> mount error 22 = Invalid argument
> 
> # dmesg -c
> libceph: no secret set (for auth_x protocol)
> libceph: error -22 on auth protocol 2 init
> 
> It looks very much like http://marc.info/?l=ceph-devel&m=130072087704874&w=2 .
> Yes, I have enabled cephx, but I have no clue how to tell the client what key
> to use. I tried

Yup, those mean your server demands authentication but your client
didn't provide it.

> # cauthtool -l /etc/ceph/admin.keyring
> # mount.ceph 192.168.178.100:6789:/ /vol -v -o name=<auid>,secret=<key>
> 
> but all I got was
> 
> ceph: Unknown mount option name
> ceph: Unknown mount option secret
> mount error 1 = Operation not permitted

> 
> Yet, 'man mount.ceph' says
> 
> name	string, used with authx, Default: guest
> secret	string, used with authx
> 
> Commenting out "auth supported = cephx" gets rid of the problem and
> lets the store be mounted, but it's a workaround, not a solution.
> 
> Any ideas? This is ceph-0.26 with vanilla kernel 2.6.38.2.

The name and secret error messages seem to be just you running a
version of Ceph that does not include commit
9233889a7be8f672729a71e48681a1e779dabd1c; that'll be in 0.27.  Ignore
those two lines for now.

It looks like you're passing in the key the right way, but it's hard
to tell as you've obscured the actual key. The error you are seeing
is exactly what you would get if the key was invalid, or didn't have
the capabilities needed for mounting Ceph.

Perhaps you could share the ceph.conf, keyring and exact mount command
you are using, and once you get it going regenerate the keys to get
back your security.

-- 
:(){ :|:&};:

Re: Mounting

[Zenon Panoussis]

On 04/15/2011 02:00 AM, Tommi Virtanen wrote:

> It looks like you're passing in the key the right way, but it's hard
> to tell as you've obscured the actual key. The error you are seeing
> is exactly what you would get if the key was invalid, or didn't have
> the capabilities needed for mounting Ceph.

> Perhaps you could share the ceph.conf, keyring and exact mount command
> you are using, and once you get it going regenerate the keys to get
> back your security.

I tried them all:

# cauthtool -l /etc/ceph/admin.keyring
[client.admin]
	key = AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
	auid = 18446744073709551615
# mount.ceph 192.168.178.100:6789:/ /vol -v -o name=18446744073709551615,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
parsing options: name=18446744073709551615,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
ceph: Unknown mount option name
ceph: Unknown mount option secret
mount error 1 = Operation not permitted

# cauthtool -l /etc/ceph/<tab>
admin.keyring       ceph.conf           keyring.mds.node01  keyring.mds.node02  keyring.osd.0       keyring.osd.1       sample.ceph.conf

# cauthtool -l /etc/ceph/keyring.mds.node01
[mds.node01]
	key = AQASSKdNkI4MKhAAkBzdcxP+GBaiceeYXTCaGQ==
	auid = 18446744073709551615

# mount.ceph 192.168.178.100:6789:/ /vol -v -o name=18446744073709551615,secret=AQASSKdNkI4MKhAAkBzdcxP+GBaiceeYXTCaGQ==
parsing options: name=18446744073709551615,secret=AQASSKdNkI4MKhAAkBzdcxP+GBaiceeYXTCaGQ==
ceph: Unknown mount option name
ceph: Unknown mount option secret
mount error 1 = Operation not permitted

# cauthtool -l /etc/ceph/keyring.osd.0
[osd.0]
	key = AQARSKdNYDP3EBAASb7bPEtf0v7XUued0Yzorg==
	auid = 18446744073709551615

# mount.ceph 192.168.178.100:6789:/ /vol -v -o name=18446744073709551615,secret=AQARSKdNYDP3EBAASb7bPEtf0v7XUued0Yzorg==
parsing options: name=18446744073709551615,secret=AQARSKdNYDP3EBAASb7bPEtf0v7XUued0Yzorg==
ceph: Unknown mount option name
ceph: Unknown mount option secret
mount error 1 = Operation not permitted

# cat /etc/ceph/ceph.conf
[global]
	auth supported = cephx
	keyring = /etc/ceph/admin.keyring
        max open files = 131072

; monitors
;  You need at least one.  You need at least three if you want to
;  tolerate any node failures.  Always create an odd number.
[mon]
	mon data = /var/ceph/data/mon$id

	; logging, for debugging monitor crashes, in order of
	; their likelihood of being helpful :)
	;debug ms = 1
	;debug mon = 20
	;debug paxos = 20
	;debug auth = 20

[mon0]
	host = node01
	mon addr = 192.168.178.100:6789

;[mon1]
;	host = node02
;	mon addr = 192.168.178.101:6789

;[mon2]
;	host = node03
;	mon addr = 192.168.0.102:6789


; mds
;  You need at least one.  Define two to get a standby.
[mds]
	; where the mds keeps it's secret encryption keys
	keyring = /etc/ceph/keyring.$name

	; mds logging to debug issues.
	;debug ms = 1
	;debug mds = 20

[mds.node01]
	host = node01

[mds.node02]
	host = node02

[osd]
	osd data = /mnt/osd
	keyring = /etc/ceph/keyring.$name
	osd journal = /var/ceph/data/osd$id/journal
	osd journal size = 1000 ; journal size, in megabytes

	; osd logging to debug osd issues, in order of likelihood of being
	; helpful
	debug ms = 1
	debug osd = 20
	debug filestore = 20
	debug journal = 20

[osd0]
	host = node01
;	btrfs devs = /dev/mapper/sda6

[osd1]
	host = node02
;	btrfs devs = /dev/mapper/md4

;[osd2]
;	host = node03
;	btrfs devs = /dev/mapper/sda6


Z

Re: Mounting

[Sage Weil]

> I tried them all:
> 
> # cauthtool -l /etc/ceph/admin.keyring
> [client.admin]
> 	key = AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
> 	auid = 18446744073709551615
> # mount.ceph 192.168.178.100:6789:/ /vol -v -o name=18446744073709551615,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==

The client 'name' to use for this key is 'admin'.

sage

Re: Mounting

[Zenon Panoussis]

On 04/15/2011 05:40 AM, Sage Weil wrote:

>> I tried them all:

>> # cauthtool -l /etc/ceph/admin.keyring
>> [client.admin]
>> 	key = AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
>> 	auid = 18446744073709551615
>> # mount.ceph 192.168.178.100:6789:/ /vol -v -o name=18446744073709551615,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==

> The client 'name' to use for this key is 'admin'.

Oops. Thanks!

Z (feeling stupid)