* [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums
2013-08-13 21:47 [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections andros
@ 2013-08-13 21:47 ` andros
2013-08-22 18:50 ` Myklebust, Trond
2013-08-13 21:47 ` [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients andros
` (2 subsequent siblings)
3 siblings, 1 reply; 11+ messages in thread
From: andros @ 2013-08-13 21:47 UTC (permalink / raw)
To: trond.myklebust; +Cc: linux-nfs, Andy Adamson
From: Andy Adamson <andros@netapp.com>
SPKM and LIPKEY are not supported.
Signed-off-by: Andy Adamson <andros@netapp.com>
---
fs/nfs/super.c | 34 ----------------------------------
include/linux/sunrpc/msg_prot.h | 6 ------
2 files changed, 40 deletions(-)
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index d26bd63..f4d3768 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -209,8 +209,6 @@ static const match_table_t nfs_xprt_protocol_tokens = {
enum {
Opt_sec_none, Opt_sec_sys,
Opt_sec_krb5, Opt_sec_krb5i, Opt_sec_krb5p,
- Opt_sec_lkey, Opt_sec_lkeyi, Opt_sec_lkeyp,
- Opt_sec_spkm, Opt_sec_spkmi, Opt_sec_spkmp,
Opt_sec_err
};
@@ -224,14 +222,6 @@ static const match_table_t nfs_secflavor_tokens = {
{ Opt_sec_krb5i, "krb5i" },
{ Opt_sec_krb5p, "krb5p" },
- { Opt_sec_lkey, "lkey" },
- { Opt_sec_lkeyi, "lkeyi" },
- { Opt_sec_lkeyp, "lkeyp" },
-
- { Opt_sec_spkm, "spkm3" },
- { Opt_sec_spkmi, "spkm3i" },
- { Opt_sec_spkmp, "spkm3p" },
-
{ Opt_sec_err, NULL }
};
@@ -502,12 +492,6 @@ static const char *nfs_pseudoflavour_to_name(rpc_authflavor_t flavour)
{ RPC_AUTH_GSS_KRB5, "krb5" },
{ RPC_AUTH_GSS_KRB5I, "krb5i" },
{ RPC_AUTH_GSS_KRB5P, "krb5p" },
- { RPC_AUTH_GSS_LKEY, "lkey" },
- { RPC_AUTH_GSS_LKEYI, "lkeyi" },
- { RPC_AUTH_GSS_LKEYP, "lkeyp" },
- { RPC_AUTH_GSS_SPKM, "spkm" },
- { RPC_AUTH_GSS_SPKMI, "spkmi" },
- { RPC_AUTH_GSS_SPKMP, "spkmp" },
{ UINT_MAX, "unknown" }
};
int i;
@@ -1044,24 +1028,6 @@ static int nfs_parse_security_flavors(char *value,
case Opt_sec_krb5p:
mnt->auth_flavors[0] = RPC_AUTH_GSS_KRB5P;
break;
- case Opt_sec_lkey:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_LKEY;
- break;
- case Opt_sec_lkeyi:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_LKEYI;
- break;
- case Opt_sec_lkeyp:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_LKEYP;
- break;
- case Opt_sec_spkm:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_SPKM;
- break;
- case Opt_sec_spkmi:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_SPKMI;
- break;
- case Opt_sec_spkmp:
- mnt->auth_flavors[0] = RPC_AUTH_GSS_SPKMP;
- break;
default:
return 0;
}
diff --git a/include/linux/sunrpc/msg_prot.h b/include/linux/sunrpc/msg_prot.h
index aadc6a0..1666da0 100644
--- a/include/linux/sunrpc/msg_prot.h
+++ b/include/linux/sunrpc/msg_prot.h
@@ -29,12 +29,6 @@ enum rpc_auth_flavors {
RPC_AUTH_GSS_KRB5 = 390003,
RPC_AUTH_GSS_KRB5I = 390004,
RPC_AUTH_GSS_KRB5P = 390005,
- RPC_AUTH_GSS_LKEY = 390006,
- RPC_AUTH_GSS_LKEYI = 390007,
- RPC_AUTH_GSS_LKEYP = 390008,
- RPC_AUTH_GSS_SPKM = 390009,
- RPC_AUTH_GSS_SPKMI = 390010,
- RPC_AUTH_GSS_SPKMP = 390011,
};
/* Maximum size (in bytes) of an rpc credential or verifier */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums
2013-08-13 21:47 ` [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums andros
@ 2013-08-22 18:50 ` Myklebust, Trond
2013-08-22 21:22 ` Adamson, Andy
0 siblings, 1 reply; 11+ messages in thread
From: Myklebust, Trond @ 2013-08-22 18:50 UTC (permalink / raw)
To: Adamson, Andy; +Cc: linux-nfs@vger.kernel.org
T24gVHVlLCAyMDEzLTA4LTEzIGF0IDE3OjQ3IC0wNDAwLCBhbmRyb3NAbmV0YXBwLmNvbSB3cm90
ZToNCj4gRnJvbTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gDQo+IFNQS00g
YW5kIExJUEtFWSBhcmUgbm90IHN1cHBvcnRlZC4NCg0KV2UgZG9uJ3Qgc2hpcCB3aXRoIFNQS00g
YW5kIExJUEtFWSwgYnV0IGFmYWlrIG5vdGhpbmcgcHJldmVudHMgc29tZW9uZQ0KZnJvbSBhZGRp
bmcgc3VwcG9ydCBhZ2Fpbi4NCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50
IG1haW50YWluZXINCg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5l
dGFwcC5jb20NCg==
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums
2013-08-22 18:50 ` Myklebust, Trond
@ 2013-08-22 21:22 ` Adamson, Andy
0 siblings, 0 replies; 11+ messages in thread
From: Adamson, Andy @ 2013-08-22 21:22 UTC (permalink / raw)
To: Myklebust, Trond; +Cc: Adamson, Andy, linux-nfs@vger.kernel.org
On Aug 22, 2013, at 2:50 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
wrote:
> On Tue, 2013-08-13 at 17:47 -0400, andros@netapp.com wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> SPKM and LIPKEY are not supported.
>
> We don't ship with SPKM and LIPKEY, but afaik nothing prevents someone
> from adding support again.
I think that the lack of an internet draft will prevent someone from adding support again. draft-adamson-rfc2847-bis-01.txt was the last attempt to fix SPKM3 - and was rejected by the Security AD after much debate. It died in Aug 2006. So, there is no standard to code to.
So I see no need to keep these un-used declarations around.
-->Andy
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients
2013-08-13 21:47 [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections andros
2013-08-13 21:47 ` [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums andros
@ 2013-08-13 21:47 ` andros
2013-08-22 18:49 ` Myklebust, Trond
2013-09-03 16:04 ` Myklebust, Trond
2013-08-13 21:47 ` [PATCH Version 5 3/3] NFSv4.1 Use the shared nfs_client rpc_clnts for pNFS data server connections andros
2013-08-23 14:01 ` [PATCH Version 5 0/3] Use the MDS nfs_server authflavor " Simo Sorce
3 siblings, 2 replies; 11+ messages in thread
From: andros @ 2013-08-13 21:47 UTC (permalink / raw)
To: trond.myklebust; +Cc: linux-nfs, Andy Adamson
From: Andy Adamson <andros@netapp.com>
The current use of __rpc_clone_client results in multiple rpc clients with
the same GSS rpc_auth type, each with it's own gss_context cache.
This can result in multiple gss_contexts for <user, pseudoflavor, server> tuple.
To avoid multiple gss_contexts share per server/pseudoflavor clients and the resultant gss_context cache.
We also add the RPC_AUTH_UNIX client for the pNFS Data Server case used in
a subsequent patch, even though the RPC_AUTH_UNIX auth cache is already shared.
Signed-off-by: Andy Adamson <andros@netapp.com>
---
fs/nfs/client.c | 80 ++++++++++++++++++++++++++++++++++++++---
fs/nfs/internal.h | 4 +++
fs/nfs/nfs4namespace.c | 5 +--
fs/nfs/nfs4proc.c | 9 +++--
fs/nfs/nfs4state.c | 4 +--
include/linux/nfs_fs_sb.h | 1 +
include/linux/sunrpc/msg_prot.h | 3 ++
7 files changed, 94 insertions(+), 12 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 2dceee4..1b4967f 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -152,7 +152,7 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
{
struct nfs_client *clp;
struct rpc_cred *cred;
- int err = -ENOMEM;
+ int err = -ENOMEM, i;
if ((clp = kzalloc(sizeof(*clp), GFP_KERNEL)) == NULL)
goto error_0;
@@ -178,6 +178,10 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
INIT_LIST_HEAD(&clp->cl_superblocks);
clp->cl_rpcclient = ERR_PTR(-EINVAL);
+ /* The three Kerberos pseudoflavors plus RPC_AUTH_UNIX */
+ for (i = 0; i < RPC_AUTH_MAX_PSEUDO_FLAVOR + 1; i++)
+ clp->cl_rpc_clnt[i] = ERR_PTR(-EINVAL);
+
clp->cl_proto = cl_init->proto;
clp->cl_net = get_net(cl_init->net);
@@ -244,7 +248,7 @@ void nfs_free_client(struct nfs_client *clp)
/* -EIO all pending I/O */
if (!IS_ERR(clp->cl_rpcclient))
- rpc_shutdown_client(clp->cl_rpcclient);
+ nfs_shutdown_rpc_client(clp, clp->cl_rpcclient);
if (clp->cl_machine_cred != NULL)
put_rpccred(clp->cl_machine_cred);
@@ -568,6 +572,68 @@ void nfs_init_timeout_values(struct rpc_timeout *to, int proto,
}
EXPORT_SYMBOL_GPL(nfs_init_timeout_values);
+/**
+ * Translate an authflavor into an nfs_client cl_rpc_clnt array index.
+ * Note cl_gss_array is of RPC_AUTH_MAX_PSEUDO_FLAVOR + 1 size.
+ */
+static inline
+int nfs_flavor_to_index(rpc_authflavor_t flavor)
+{
+ switch (flavor) {
+ case RPC_AUTH_UNIX:
+ return 0;
+ case RPC_AUTH_GSS_KRB5:
+ return 1;
+ case RPC_AUTH_GSS_KRB5I:
+ return 2;
+ case RPC_AUTH_GSS_KRB5P:
+ return 3;
+ default:
+ return -EINVAL;
+ }
+}
+
+struct rpc_clnt *
+nfs_get_auth_rpc_client(struct nfs_client *clp, rpc_authflavor_t flavor)
+{
+ struct rpc_clnt *clnt = clp->cl_rpcclient;
+ int idx;
+
+ dprintk("--> %s cl_rpcclient %p flavor %d\n", __func__, clnt, flavor);
+
+ idx = nfs_flavor_to_index(flavor);
+ if (idx < 0)
+ return ERR_PTR(idx);
+
+ if (IS_ERR(clp->cl_rpc_clnt[idx]))
+ clp->cl_rpc_clnt[idx] = rpc_clone_client_set_auth(clnt, flavor);
+ else
+ atomic_inc(&clp->cl_rpc_clnt[idx]->cl_count);
+
+ return clp->cl_rpc_clnt[idx];
+}
+EXPORT_SYMBOL_GPL(nfs_get_auth_rpc_client);
+
+void
+nfs_shutdown_rpc_client(struct nfs_client *clp, struct rpc_clnt *clnt)
+{
+ int idx;
+
+ dprintk("--> %s clnt %p flavor %d rpc clnt cl_count (%d)\n", __func__,
+ clnt, clnt->cl_auth->au_flavor, atomic_read(&clnt->cl_count));
+
+ /* rpc client creation checked the validity of the flavor */
+ idx = nfs_flavor_to_index(clnt->cl_auth->au_flavor);
+
+ /* Dec the refcount unless it is 1 in which case call shutdown */
+ if (atomic_add_unless(&clp->cl_rpc_clnt[idx]->cl_count, -1, 1) == 0) {
+ rpc_shutdown_client(clnt);
+ clp->cl_rpc_clnt[idx] = ERR_PTR(-EINVAL);
+ return;
+ }
+}
+EXPORT_SYMBOL_GPL(nfs_shutdown_rpc_client);
+
/*
* Create an RPC client handle
*/
@@ -587,6 +653,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
.version = clp->rpc_ops->version,
.authflavor = flavor,
};
+ int idx;
if (test_bit(NFS_CS_DISCRTRY, &clp->cl_flags))
args.flags |= RPC_CLNT_CREATE_DISCRTRY;
@@ -605,7 +672,11 @@ int nfs_create_rpc_client(struct nfs_client *clp,
return PTR_ERR(clnt);
}
+ /* rpcauth_create has verified the validity of the authflavor. */
+ idx = nfs_flavor_to_index(flavor);
+ clp->cl_rpc_clnt[idx] = clnt;
clp->cl_rpcclient = clnt;
+
return 0;
}
EXPORT_SYMBOL_GPL(nfs_create_rpc_client);
@@ -668,8 +739,7 @@ int nfs_init_server_rpcclient(struct nfs_server *server,
{
struct nfs_client *clp = server->nfs_client;
- server->client = rpc_clone_client_set_auth(clp->cl_rpcclient,
- pseudoflavour);
+ server->client = nfs_get_auth_rpc_client(clp, pseudoflavour);
if (IS_ERR(server->client)) {
dprintk("%s: couldn't create rpc_client!\n", __func__);
return PTR_ERR(server->client);
@@ -1018,7 +1088,7 @@ void nfs_free_server(struct nfs_server *server)
if (!IS_ERR(server->client_acl))
rpc_shutdown_client(server->client_acl);
if (!IS_ERR(server->client))
- rpc_shutdown_client(server->client);
+ nfs_shutdown_rpc_client(server->nfs_client, server->client);
nfs_put_client(server->nfs_client);
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 9b694f1..4aa20ba 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -185,6 +185,10 @@ extern struct nfs_client *nfs4_set_ds_client(struct nfs_client* mds_clp,
int ds_addrlen, int ds_proto,
unsigned int ds_timeo,
unsigned int ds_retrans);
+extern struct rpc_clnt *nfs_get_auth_rpc_client(struct nfs_client *clp,
+ rpc_authflavor_t flavor);
+extern void nfs_shutdown_rpc_client(struct nfs_client *clp,
+ struct rpc_clnt *clnt);
#ifdef CONFIG_PROC_FS
extern int __init nfs_fs_proc_init(void);
extern void nfs_fs_proc_exit(void);
diff --git a/fs/nfs/nfs4namespace.c b/fs/nfs/nfs4namespace.c
index cdb0b41..b274427 100644
--- a/fs/nfs/nfs4namespace.c
+++ b/fs/nfs/nfs4namespace.c
@@ -205,7 +205,7 @@ struct rpc_clnt *nfs4_create_sec_client(struct rpc_clnt *clnt, struct inode *ino
if ((int)flavor < 0)
return ERR_PTR((int)flavor);
- return rpc_clone_client_set_auth(clnt, flavor);
+ return nfs_get_auth_rpc_client(NFS_SERVER(inode)->nfs_client, flavor);
}
static struct vfsmount *try_location(struct nfs_clone_mount *mountdata,
@@ -370,6 +370,7 @@ struct vfsmount *nfs4_submount(struct nfs_server *server, struct dentry *dentry,
struct nfs_fh *fh, struct nfs_fattr *fattr)
{
struct dentry *parent = dget_parent(dentry);
+ struct nfs_client *clp = NFS_SERVER(parent->d_inode)->nfs_client;
struct rpc_clnt *client;
struct vfsmount *mnt;
@@ -384,6 +385,6 @@ struct vfsmount *nfs4_submount(struct nfs_server *server, struct dentry *dentry,
else
mnt = nfs_do_submount(dentry, fh, fattr, client->cl_auth->au_flavor);
- rpc_shutdown_client(client);
+ nfs_shutdown_rpc_client(clp, client);
return mnt;
}
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index f672c34..9160fcb 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3047,7 +3047,7 @@ out:
if (err == 0)
*clnt = client;
else if (client != *clnt)
- rpc_shutdown_client(client);
+ nfs_shutdown_rpc_client(NFS_SERVER(dir)->nfs_client, client);
return err;
}
@@ -3061,7 +3061,7 @@ static int nfs4_proc_lookup(struct inode *dir, struct qstr *name,
status = nfs4_proc_lookup_common(&client, dir, name, fhandle, fattr, label);
if (client != NFS_CLIENT(dir)) {
- rpc_shutdown_client(client);
+ nfs_shutdown_rpc_client(NFS_SERVER(dir)->nfs_client, client);
nfs_fixup_secinfo_attributes(fattr);
}
return status;
@@ -3072,12 +3072,15 @@ nfs4_proc_lookup_mountpoint(struct inode *dir, struct qstr *name,
struct nfs_fh *fhandle, struct nfs_fattr *fattr)
{
struct rpc_clnt *client = NFS_CLIENT(dir);
+ rpc_authflavor_t flavor = client->cl_auth->au_flavor;
+ struct nfs_client *clp = NFS_SERVER(dir)->nfs_client;
int status;
status = nfs4_proc_lookup_common(&client, dir, name, fhandle, fattr, NULL);
if (status < 0)
return ERR_PTR(status);
- return (client == NFS_CLIENT(dir)) ? rpc_clone_client(client) : client;
+ return (client == NFS_CLIENT(dir)) ?
+ nfs_get_auth_rpc_client(clp, flavor) : client;
}
static int _nfs4_proc_access(struct inode *inode, struct nfs_access_entry *entry)
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 6818964..f7815ce 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1876,7 +1876,7 @@ again:
break;
case -NFS4ERR_CLID_INUSE:
case -NFS4ERR_WRONGSEC:
- clnt = rpc_clone_client_set_auth(clnt, RPC_AUTH_UNIX);
+ clnt = nfs_get_auth_rpc_client(clp, RPC_AUTH_UNIX);
if (IS_ERR(clnt)) {
status = PTR_ERR(clnt);
break;
@@ -1886,7 +1886,7 @@ again:
* clp->cl_rpcclient
*/
clnt = xchg(&clp->cl_rpcclient, clnt);
- rpc_shutdown_client(clnt);
+ nfs_shutdown_rpc_client(clp, clnt);
clnt = clp->cl_rpcclient;
goto again;
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index d221243..e112cf2 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -56,6 +56,7 @@ struct nfs_client {
struct rpc_cred *cl_machine_cred;
#if IS_ENABLED(CONFIG_NFS_V4)
+ struct rpc_clnt * cl_rpc_clnt[RPC_AUTH_MAX_PSEUDO_FLAVOR + 1];
u64 cl_clientid; /* constant */
nfs4_verifier cl_confirm; /* Clientid verifier */
unsigned long cl_state;
diff --git a/include/linux/sunrpc/msg_prot.h b/include/linux/sunrpc/msg_prot.h
index 1666da0..53c9a13 100644
--- a/include/linux/sunrpc/msg_prot.h
+++ b/include/linux/sunrpc/msg_prot.h
@@ -17,6 +17,9 @@
/* spec defines authentication flavor as an unsigned 32 bit integer */
typedef u32 rpc_authflavor_t;
+/* The number of supported pseudoflavors */
+#define RPC_AUTH_MAX_PSEUDO_FLAVOR 3
+
enum rpc_auth_flavors {
RPC_AUTH_NULL = 0,
RPC_AUTH_UNIX = 1,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients
2013-08-13 21:47 ` [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients andros
@ 2013-08-22 18:49 ` Myklebust, Trond
2013-08-22 21:24 ` Adamson, Andy
2013-09-03 16:04 ` Myklebust, Trond
1 sibling, 1 reply; 11+ messages in thread
From: Myklebust, Trond @ 2013-08-22 18:49 UTC (permalink / raw)
To: Adamson, Andy; +Cc: linux-nfs@vger.kernel.org
T24gVHVlLCAyMDEzLTA4LTEzIGF0IDE3OjQ3IC0wNDAwLCBhbmRyb3NAbmV0YXBwLmNvbSB3cm90
ZToNCj4gRnJvbTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gDQo+IFRoZSBj
dXJyZW50IHVzZSBvZiBfX3JwY19jbG9uZV9jbGllbnQgcmVzdWx0cyBpbiBtdWx0aXBsZSBycGMg
Y2xpZW50cyB3aXRoDQo+IHRoZSBzYW1lIEdTUyBycGNfYXV0aCB0eXBlLCBlYWNoIHdpdGggaXQn
cyBvd24gZ3NzX2NvbnRleHQgY2FjaGUuDQo+IFRoaXMgY2FuIHJlc3VsdCBpbiBtdWx0aXBsZSBn
c3NfY29udGV4dHMgZm9yIDx1c2VyLCBwc2V1ZG9mbGF2b3IsIHNlcnZlcj4gdHVwbGUuDQo+IA0K
PiBUbyBhdm9pZCBtdWx0aXBsZSBnc3NfY29udGV4dHMgc2hhcmUgcGVyIHNlcnZlci9wc2V1ZG9m
bGF2b3IgY2xpZW50cyBhbmQgdGhlIHJlc3VsdGFudCBnc3NfY29udGV4dCBjYWNoZS4NCj4gDQo+
IFdlIGFsc28gYWRkIHRoZSBSUENfQVVUSF9VTklYIGNsaWVudCBmb3IgdGhlIHBORlMgRGF0YSBT
ZXJ2ZXIgY2FzZSB1c2VkIGluDQo+IGEgc3Vic2VxdWVudCBwYXRjaCwgZXZlbiB0aG91Z2ggdGhl
IFJQQ19BVVRIX1VOSVggYXV0aCBjYWNoZSBpcyBhbHJlYWR5IHNoYXJlZC4NCj4gDQo+IFNpZ25l
ZC1vZmYtYnk6IEFuZHkgQWRhbXNvbiA8YW5kcm9zQG5ldGFwcC5jb20+DQo+IC0tLQ0KPiAgZnMv
bmZzL2NsaWVudC5jICAgICAgICAgICAgICAgICB8IDgwICsrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrLS0tDQo+ICBmcy9uZnMvaW50ZXJuYWwuaCAgICAgICAgICAgICAgIHwg
IDQgKysrDQo+ICBmcy9uZnMvbmZzNG5hbWVzcGFjZS5jICAgICAgICAgIHwgIDUgKy0tDQo+ICBm
cy9uZnMvbmZzNHByb2MuYyAgICAgICAgICAgICAgIHwgIDkgKysrLS0NCj4gIGZzL25mcy9uZnM0
c3RhdGUuYyAgICAgICAgICAgICAgfCAgNCArLS0NCj4gIGluY2x1ZGUvbGludXgvbmZzX2ZzX3Ni
LmggICAgICAgfCAgMSArDQo+ICBpbmNsdWRlL2xpbnV4L3N1bnJwYy9tc2dfcHJvdC5oIHwgIDMg
KysNCj4gIDcgZmlsZXMgY2hhbmdlZCwgOTQgaW5zZXJ0aW9ucygrKSwgMTIgZGVsZXRpb25zKC0p
DQo+IA0KPiBkaWZmIC0tZ2l0IGEvZnMvbmZzL2NsaWVudC5jIGIvZnMvbmZzL2NsaWVudC5jDQo+
IGluZGV4IDJkY2VlZTQuLjFiNDk2N2YgMTAwNjQ0DQo+IC0tLSBhL2ZzL25mcy9jbGllbnQuYw0K
PiArKysgYi9mcy9uZnMvY2xpZW50LmMNCj4gQEAgLTE1Miw3ICsxNTIsNyBAQCBzdHJ1Y3QgbmZz
X2NsaWVudCAqbmZzX2FsbG9jX2NsaWVudChjb25zdCBzdHJ1Y3QgbmZzX2NsaWVudF9pbml0ZGF0
YSAqY2xfaW5pdCkNCj4gIHsNCj4gIAlzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwOw0KPiAgCXN0cnVj
dCBycGNfY3JlZCAqY3JlZDsNCj4gLQlpbnQgZXJyID0gLUVOT01FTTsNCj4gKwlpbnQgZXJyID0g
LUVOT01FTSwgaTsNCj4gIA0KPiAgCWlmICgoY2xwID0ga3phbGxvYyhzaXplb2YoKmNscCksIEdG
UF9LRVJORUwpKSA9PSBOVUxMKQ0KPiAgCQlnb3RvIGVycm9yXzA7DQo+IEBAIC0xNzgsNiArMTc4
LDEwIEBAIHN0cnVjdCBuZnNfY2xpZW50ICpuZnNfYWxsb2NfY2xpZW50KGNvbnN0IHN0cnVjdCBu
ZnNfY2xpZW50X2luaXRkYXRhICpjbF9pbml0KQ0KPiAgCUlOSVRfTElTVF9IRUFEKCZjbHAtPmNs
X3N1cGVyYmxvY2tzKTsNCj4gIAljbHAtPmNsX3JwY2NsaWVudCA9IEVSUl9QVFIoLUVJTlZBTCk7
DQo+ICANCj4gKwkvKiBUaGUgdGhyZWUgS2VyYmVyb3MgcHNldWRvZmxhdm9ycyBwbHVzIFJQQ19B
VVRIX1VOSVggKi8NCj4gKwlmb3IgKGkgPSAwOyBpIDwgUlBDX0FVVEhfTUFYX1BTRVVET19GTEFW
T1IgKyAxOyBpKyspDQo+ICsJCWNscC0+Y2xfcnBjX2NsbnRbaV0gPSBFUlJfUFRSKC1FSU5WQUwp
Ow0KPiArDQo+ICAJY2xwLT5jbF9wcm90byA9IGNsX2luaXQtPnByb3RvOw0KPiAgCWNscC0+Y2xf
bmV0ID0gZ2V0X25ldChjbF9pbml0LT5uZXQpOw0KPiAgDQo+IEBAIC0yNDQsNyArMjQ4LDcgQEAg
dm9pZCBuZnNfZnJlZV9jbGllbnQoc3RydWN0IG5mc19jbGllbnQgKmNscCkNCj4gIA0KPiAgCS8q
IC1FSU8gYWxsIHBlbmRpbmcgSS9PICovDQo+ICAJaWYgKCFJU19FUlIoY2xwLT5jbF9ycGNjbGll
bnQpKQ0KPiAtCQlycGNfc2h1dGRvd25fY2xpZW50KGNscC0+Y2xfcnBjY2xpZW50KTsNCj4gKwkJ
bmZzX3NodXRkb3duX3JwY19jbGllbnQoY2xwLCBjbHAtPmNsX3JwY2NsaWVudCk7DQo+ICANCj4g
IAlpZiAoY2xwLT5jbF9tYWNoaW5lX2NyZWQgIT0gTlVMTCkNCj4gIAkJcHV0X3JwY2NyZWQoY2xw
LT5jbF9tYWNoaW5lX2NyZWQpOw0KPiBAQCAtNTY4LDYgKzU3Miw2OCBAQCB2b2lkIG5mc19pbml0
X3RpbWVvdXRfdmFsdWVzKHN0cnVjdCBycGNfdGltZW91dCAqdG8sIGludCBwcm90bywNCj4gIH0N
Cj4gIEVYUE9SVF9TWU1CT0xfR1BMKG5mc19pbml0X3RpbWVvdXRfdmFsdWVzKTsNCj4gIA0KPiAr
LyoqDQo+ICsgKiBUcmFuc2xhdGUgYW4gYXV0aGZsYXZvciBpbnRvIGFuIG5mc19jbGllbnQgY2xf
cnBjX2NsbnQgYXJyYXkgaW5kZXguDQo+ICsgKiBOb3RlIGNsX2dzc19hcnJheSBpcyBvZiBSUENf
QVVUSF9NQVhfUFNFVURPX0ZMQVZPUiArIDEgc2l6ZS4NCj4gKyAqLw0KPiArc3RhdGljIGlubGlu
ZQ0KPiAraW50IG5mc19mbGF2b3JfdG9faW5kZXgocnBjX2F1dGhmbGF2b3JfdCBmbGF2b3IpDQo+
ICt7DQo+ICsJc3dpdGNoIChmbGF2b3IpIHsNCj4gKwljYXNlIFJQQ19BVVRIX1VOSVg6DQo+ICsJ
CXJldHVybiAwOw0KPiArCWNhc2UgUlBDX0FVVEhfR1NTX0tSQjU6DQo+ICsJCXJldHVybiAxOw0K
PiArCWNhc2UgUlBDX0FVVEhfR1NTX0tSQjVJOg0KPiArCQlyZXR1cm4gMjsNCj4gKwljYXNlIFJQ
Q19BVVRIX0dTU19LUkI1UDoNCj4gKwkJcmV0dXJuIDM7DQo+ICsJZGVmYXVsdDoNCj4gKwkJcmV0
dXJuIC1FSU5WQUw7DQo+ICsJfQ0KPiArfQ0KPiArDQo+ICtzdHJ1Y3QgcnBjX2NsbnQgKg0KPiAr
bmZzX2dldF9hdXRoX3JwY19jbGllbnQoc3RydWN0IG5mc19jbGllbnQgKmNscCwgcnBjX2F1dGhm
bGF2b3JfdCBmbGF2b3IpDQo+ICt7DQo+ICsJc3RydWN0IHJwY19jbG50ICpjbG50ID0gY2xwLT5j
bF9ycGNjbGllbnQ7DQo+ICsJaW50IGlkeDsNCj4gKw0KPiArCWRwcmludGsoIi0tPiAlcyBjbF9y
cGNjbGllbnQgJXAgZmxhdm9yICVkXG4iLCBfX2Z1bmNfXywgY2xudCwgZmxhdm9yKTsNCj4gKw0K
PiArCWlkeCA9IG5mc19mbGF2b3JfdG9faW5kZXgoZmxhdm9yKTsNCj4gKwlpZiAoaWR4IDwgMCkN
Cj4gKwkJcmV0dXJuIEVSUl9QVFIoaWR4KTsNCj4gKw0KPiArCWlmIChJU19FUlIoY2xwLT5jbF9y
cGNfY2xudFtpZHhdKSkNCj4gKwkJY2xwLT5jbF9ycGNfY2xudFtpZHhdID0gcnBjX2Nsb25lX2Ns
aWVudF9zZXRfYXV0aChjbG50LCBmbGF2b3IpOw0KPiArCWVsc2UNCj4gKwkJYXRvbWljX2luYygm
Y2xwLT5jbF9ycGNfY2xudFtpZHhdLT5jbF9jb3VudCk7DQo+ICsNCj4gKwlyZXR1cm4gY2xwLT5j
bF9ycGNfY2xudFtpZHhdOw0KPiArfQ0KPiArRVhQT1JUX1NZTUJPTF9HUEwobmZzX2dldF9hdXRo
X3JwY19jbGllbnQpOw0KPiArDQo+ICt2b2lkDQo+ICtuZnNfc2h1dGRvd25fcnBjX2NsaWVudChz
dHJ1Y3QgbmZzX2NsaWVudCAqY2xwLCBzdHJ1Y3QgcnBjX2NsbnQgKmNsbnQpDQo+ICt7DQo+ICsJ
aW50IGlkeDsNCj4gKw0KPiArCWRwcmludGsoIi0tPiAlcyBjbG50ICVwIGZsYXZvciAlZCBycGMg
Y2xudCBjbF9jb3VudCAoJWQpXG4iLCBfX2Z1bmNfXywNCj4gKwkJY2xudCwgY2xudC0+Y2xfYXV0
aC0+YXVfZmxhdm9yLCBhdG9taWNfcmVhZCgmY2xudC0+Y2xfY291bnQpKTsNCj4gKw0KPiArCS8q
IHJwYyBjbGllbnQgY3JlYXRpb24gY2hlY2tlZCB0aGUgdmFsaWRpdHkgb2YgdGhlIGZsYXZvciAq
Lw0KPiArCWlkeCA9IG5mc19mbGF2b3JfdG9faW5kZXgoY2xudC0+Y2xfYXV0aC0+YXVfZmxhdm9y
KTsNCj4gKw0KPiArCS8qIERlYyB0aGUgcmVmY291bnQgdW5sZXNzIGl0IGlzIDEgaW4gd2hpY2gg
Y2FzZSBjYWxsIHNodXRkb3duICovDQo+ICsJaWYgKGF0b21pY19hZGRfdW5sZXNzKCZjbHAtPmNs
X3JwY19jbG50W2lkeF0tPmNsX2NvdW50LCAtMSwgMSkgPT0gMCkgew0KPiArCQlycGNfc2h1dGRv
d25fY2xpZW50KGNsbnQpOw0KDQpycGNfc2h1dGRvd25fY2xpZW50IGNhbiBzbGVlcCBmb3IgYSBs
b29vb25nIHRpbWUuIFdoYXQgcHJldmVudHMNCm5mc19nZXRfYXV0aF9ycGNfY2xpZW50KCkgZnJv
bSB0cnlpbmcgdG8gdXNlIHRoZSBycGMgY2xpZW50Pw0KDQo+ICsJCWNscC0+Y2xfcnBjX2NsbnRb
aWR4XSA9IEVSUl9QVFIoLUVJTlZBTCk7DQo+ICsJCXJldHVybjsNCj4gKwl9DQo+ICt9DQo+ICtF
WFBPUlRfU1lNQk9MX0dQTChuZnNfc2h1dGRvd25fcnBjX2NsaWVudCk7DQo+ICsNCj4gIC8qDQo+
ICAgKiBDcmVhdGUgYW4gUlBDIGNsaWVudCBoYW5kbGUNCj4gICAqLw0KPiBAQCAtNTg3LDYgKzY1
Myw3IEBAIGludCBuZnNfY3JlYXRlX3JwY19jbGllbnQoc3RydWN0IG5mc19jbGllbnQgKmNscCwN
Cj4gIAkJLnZlcnNpb24JPSBjbHAtPnJwY19vcHMtPnZlcnNpb24sDQo+ICAJCS5hdXRoZmxhdm9y
CT0gZmxhdm9yLA0KPiAgCX07DQo+ICsJaW50IGlkeDsNCj4gIA0KPiAgCWlmICh0ZXN0X2JpdChO
RlNfQ1NfRElTQ1JUUlksICZjbHAtPmNsX2ZsYWdzKSkNCj4gIAkJYXJncy5mbGFncyB8PSBSUENf
Q0xOVF9DUkVBVEVfRElTQ1JUUlk7DQo+IEBAIC02MDUsNyArNjcyLDExIEBAIGludCBuZnNfY3Jl
YXRlX3JwY19jbGllbnQoc3RydWN0IG5mc19jbGllbnQgKmNscCwNCj4gIAkJcmV0dXJuIFBUUl9F
UlIoY2xudCk7DQo+ICAJfQ0KPiAgDQo+ICsJLyogcnBjYXV0aF9jcmVhdGUgaGFzIHZlcmlmaWVk
IHRoZSB2YWxpZGl0eSBvZiB0aGUgYXV0aGZsYXZvci4gKi8NCj4gKwlpZHggPSBuZnNfZmxhdm9y
X3RvX2luZGV4KGZsYXZvcik7DQo+ICsJY2xwLT5jbF9ycGNfY2xudFtpZHhdID0gY2xudDsNCj4g
IAljbHAtPmNsX3JwY2NsaWVudCA9IGNsbnQ7DQo+ICsNCj4gIAlyZXR1cm4gMDsNCj4gIH0NCj4g
IEVYUE9SVF9TWU1CT0xfR1BMKG5mc19jcmVhdGVfcnBjX2NsaWVudCk7DQo+IEBAIC02NjgsOCAr
NzM5LDcgQEAgaW50IG5mc19pbml0X3NlcnZlcl9ycGNjbGllbnQoc3RydWN0IG5mc19zZXJ2ZXIg
KnNlcnZlciwNCj4gIHsNCj4gIAlzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwID0gc2VydmVyLT5uZnNf
Y2xpZW50Ow0KPiAgDQo+IC0Jc2VydmVyLT5jbGllbnQgPSBycGNfY2xvbmVfY2xpZW50X3NldF9h
dXRoKGNscC0+Y2xfcnBjY2xpZW50LA0KPiAtCQkJCQkJCXBzZXVkb2ZsYXZvdXIpOw0KPiArCXNl
cnZlci0+Y2xpZW50ID0gbmZzX2dldF9hdXRoX3JwY19jbGllbnQoY2xwLCBwc2V1ZG9mbGF2b3Vy
KTsNCj4gIAlpZiAoSVNfRVJSKHNlcnZlci0+Y2xpZW50KSkgew0KPiAgCQlkcHJpbnRrKCIlczog
Y291bGRuJ3QgY3JlYXRlIHJwY19jbGllbnQhXG4iLCBfX2Z1bmNfXyk7DQo+ICAJCXJldHVybiBQ
VFJfRVJSKHNlcnZlci0+Y2xpZW50KTsNCj4gQEAgLTEwMTgsNyArMTA4OCw3IEBAIHZvaWQgbmZz
X2ZyZWVfc2VydmVyKHN0cnVjdCBuZnNfc2VydmVyICpzZXJ2ZXIpDQo+ICAJaWYgKCFJU19FUlIo
c2VydmVyLT5jbGllbnRfYWNsKSkNCj4gIAkJcnBjX3NodXRkb3duX2NsaWVudChzZXJ2ZXItPmNs
aWVudF9hY2wpOw0KPiAgCWlmICghSVNfRVJSKHNlcnZlci0+Y2xpZW50KSkNCj4gLQkJcnBjX3No
dXRkb3duX2NsaWVudChzZXJ2ZXItPmNsaWVudCk7DQo+ICsJCW5mc19zaHV0ZG93bl9ycGNfY2xp
ZW50KHNlcnZlci0+bmZzX2NsaWVudCwgc2VydmVyLT5jbGllbnQpOw0KPiAgDQo+ICAJbmZzX3B1
dF9jbGllbnQoc2VydmVyLT5uZnNfY2xpZW50KTsNCj4gIA0KPiBkaWZmIC0tZ2l0IGEvZnMvbmZz
L2ludGVybmFsLmggYi9mcy9uZnMvaW50ZXJuYWwuaA0KPiBpbmRleCA5YjY5NGYxLi40YWEyMGJh
IDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvaW50ZXJuYWwuaA0KPiArKysgYi9mcy9uZnMvaW50ZXJu
YWwuaA0KPiBAQCAtMTg1LDYgKzE4NSwxMCBAQCBleHRlcm4gc3RydWN0IG5mc19jbGllbnQgKm5m
czRfc2V0X2RzX2NsaWVudChzdHJ1Y3QgbmZzX2NsaWVudCogbWRzX2NscCwNCj4gIAkJCQkJICAg
ICBpbnQgZHNfYWRkcmxlbiwgaW50IGRzX3Byb3RvLA0KPiAgCQkJCQkgICAgIHVuc2lnbmVkIGlu
dCBkc190aW1lbywNCj4gIAkJCQkJICAgICB1bnNpZ25lZCBpbnQgZHNfcmV0cmFucyk7DQo+ICtl
eHRlcm4gc3RydWN0IHJwY19jbG50ICpuZnNfZ2V0X2F1dGhfcnBjX2NsaWVudChzdHJ1Y3QgbmZz
X2NsaWVudCAqY2xwLA0KPiArCQkJCQlycGNfYXV0aGZsYXZvcl90IGZsYXZvcik7DQo+ICtleHRl
cm4gdm9pZCBuZnNfc2h1dGRvd25fcnBjX2NsaWVudChzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwLA0K
PiArCQkJCQlzdHJ1Y3QgcnBjX2NsbnQgKmNsbnQpOw0KPiAgI2lmZGVmIENPTkZJR19QUk9DX0ZT
DQo+ICBleHRlcm4gaW50IF9faW5pdCBuZnNfZnNfcHJvY19pbml0KHZvaWQpOw0KPiAgZXh0ZXJu
IHZvaWQgbmZzX2ZzX3Byb2NfZXhpdCh2b2lkKTsNCj4gZGlmZiAtLWdpdCBhL2ZzL25mcy9uZnM0
bmFtZXNwYWNlLmMgYi9mcy9uZnMvbmZzNG5hbWVzcGFjZS5jDQo+IGluZGV4IGNkYjBiNDEuLmIy
NzQ0MjcgMTAwNjQ0DQo+IC0tLSBhL2ZzL25mcy9uZnM0bmFtZXNwYWNlLmMNCj4gKysrIGIvZnMv
bmZzL25mczRuYW1lc3BhY2UuYw0KPiBAQCAtMjA1LDcgKzIwNSw3IEBAIHN0cnVjdCBycGNfY2xu
dCAqbmZzNF9jcmVhdGVfc2VjX2NsaWVudChzdHJ1Y3QgcnBjX2NsbnQgKmNsbnQsIHN0cnVjdCBp
bm9kZSAqaW5vDQo+ICAJaWYgKChpbnQpZmxhdm9yIDwgMCkNCj4gIAkJcmV0dXJuIEVSUl9QVFIo
KGludClmbGF2b3IpOw0KPiAgDQo+IC0JcmV0dXJuIHJwY19jbG9uZV9jbGllbnRfc2V0X2F1dGgo
Y2xudCwgZmxhdm9yKTsNCj4gKwlyZXR1cm4gbmZzX2dldF9hdXRoX3JwY19jbGllbnQoTkZTX1NF
UlZFUihpbm9kZSktPm5mc19jbGllbnQsIGZsYXZvcik7DQo+ICB9DQo+ICANCj4gIHN0YXRpYyBz
dHJ1Y3QgdmZzbW91bnQgKnRyeV9sb2NhdGlvbihzdHJ1Y3QgbmZzX2Nsb25lX21vdW50ICptb3Vu
dGRhdGEsDQo+IEBAIC0zNzAsNiArMzcwLDcgQEAgc3RydWN0IHZmc21vdW50ICpuZnM0X3N1Ym1v
dW50KHN0cnVjdCBuZnNfc2VydmVyICpzZXJ2ZXIsIHN0cnVjdCBkZW50cnkgKmRlbnRyeSwNCj4g
IAkJCSAgICAgICBzdHJ1Y3QgbmZzX2ZoICpmaCwgc3RydWN0IG5mc19mYXR0ciAqZmF0dHIpDQo+
ICB7DQo+ICAJc3RydWN0IGRlbnRyeSAqcGFyZW50ID0gZGdldF9wYXJlbnQoZGVudHJ5KTsNCj4g
KwlzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwID0gTkZTX1NFUlZFUihwYXJlbnQtPmRfaW5vZGUpLT5u
ZnNfY2xpZW50Ow0KPiAgCXN0cnVjdCBycGNfY2xudCAqY2xpZW50Ow0KPiAgCXN0cnVjdCB2ZnNt
b3VudCAqbW50Ow0KPiAgDQo+IEBAIC0zODQsNiArMzg1LDYgQEAgc3RydWN0IHZmc21vdW50ICpu
ZnM0X3N1Ym1vdW50KHN0cnVjdCBuZnNfc2VydmVyICpzZXJ2ZXIsIHN0cnVjdCBkZW50cnkgKmRl
bnRyeSwNCj4gIAllbHNlDQo+ICAJCW1udCA9IG5mc19kb19zdWJtb3VudChkZW50cnksIGZoLCBm
YXR0ciwgY2xpZW50LT5jbF9hdXRoLT5hdV9mbGF2b3IpOw0KPiAgDQo+IC0JcnBjX3NodXRkb3du
X2NsaWVudChjbGllbnQpOw0KPiArCW5mc19zaHV0ZG93bl9ycGNfY2xpZW50KGNscCwgY2xpZW50
KTsNCj4gIAlyZXR1cm4gbW50Ow0KPiAgfQ0KPiBkaWZmIC0tZ2l0IGEvZnMvbmZzL25mczRwcm9j
LmMgYi9mcy9uZnMvbmZzNHByb2MuYw0KPiBpbmRleCBmNjcyYzM0Li45MTYwZmNiIDEwMDY0NA0K
PiAtLS0gYS9mcy9uZnMvbmZzNHByb2MuYw0KPiArKysgYi9mcy9uZnMvbmZzNHByb2MuYw0KPiBA
QCAtMzA0Nyw3ICszMDQ3LDcgQEAgb3V0Og0KPiAgCWlmIChlcnIgPT0gMCkNCj4gIAkJKmNsbnQg
PSBjbGllbnQ7DQo+ICAJZWxzZSBpZiAoY2xpZW50ICE9ICpjbG50KQ0KPiAtCQlycGNfc2h1dGRv
d25fY2xpZW50KGNsaWVudCk7DQo+ICsJCW5mc19zaHV0ZG93bl9ycGNfY2xpZW50KE5GU19TRVJW
RVIoZGlyKS0+bmZzX2NsaWVudCwgY2xpZW50KTsNCj4gIA0KPiAgCXJldHVybiBlcnI7DQo+ICB9
DQo+IEBAIC0zMDYxLDcgKzMwNjEsNyBAQCBzdGF0aWMgaW50IG5mczRfcHJvY19sb29rdXAoc3Ry
dWN0IGlub2RlICpkaXIsIHN0cnVjdCBxc3RyICpuYW1lLA0KPiAgDQo+ICAJc3RhdHVzID0gbmZz
NF9wcm9jX2xvb2t1cF9jb21tb24oJmNsaWVudCwgZGlyLCBuYW1lLCBmaGFuZGxlLCBmYXR0ciwg
bGFiZWwpOw0KPiAgCWlmIChjbGllbnQgIT0gTkZTX0NMSUVOVChkaXIpKSB7DQo+IC0JCXJwY19z
aHV0ZG93bl9jbGllbnQoY2xpZW50KTsNCj4gKwkJbmZzX3NodXRkb3duX3JwY19jbGllbnQoTkZT
X1NFUlZFUihkaXIpLT5uZnNfY2xpZW50LCBjbGllbnQpOw0KPiAgCQluZnNfZml4dXBfc2VjaW5m
b19hdHRyaWJ1dGVzKGZhdHRyKTsNCj4gIAl9DQo+ICAJcmV0dXJuIHN0YXR1czsNCj4gQEAgLTMw
NzIsMTIgKzMwNzIsMTUgQEAgbmZzNF9wcm9jX2xvb2t1cF9tb3VudHBvaW50KHN0cnVjdCBpbm9k
ZSAqZGlyLCBzdHJ1Y3QgcXN0ciAqbmFtZSwNCj4gIAkJCSAgICBzdHJ1Y3QgbmZzX2ZoICpmaGFu
ZGxlLCBzdHJ1Y3QgbmZzX2ZhdHRyICpmYXR0cikNCj4gIHsNCj4gIAlzdHJ1Y3QgcnBjX2NsbnQg
KmNsaWVudCA9IE5GU19DTElFTlQoZGlyKTsNCj4gKwlycGNfYXV0aGZsYXZvcl90IGZsYXZvciA9
IGNsaWVudC0+Y2xfYXV0aC0+YXVfZmxhdm9yOw0KPiArCXN0cnVjdCBuZnNfY2xpZW50ICpjbHAg
PSBORlNfU0VSVkVSKGRpciktPm5mc19jbGllbnQ7DQo+ICAJaW50IHN0YXR1czsNCj4gIA0KPiAg
CXN0YXR1cyA9IG5mczRfcHJvY19sb29rdXBfY29tbW9uKCZjbGllbnQsIGRpciwgbmFtZSwgZmhh
bmRsZSwgZmF0dHIsIE5VTEwpOw0KPiAgCWlmIChzdGF0dXMgPCAwKQ0KPiAgCQlyZXR1cm4gRVJS
X1BUUihzdGF0dXMpOw0KPiAtCXJldHVybiAoY2xpZW50ID09IE5GU19DTElFTlQoZGlyKSkgPyBy
cGNfY2xvbmVfY2xpZW50KGNsaWVudCkgOiBjbGllbnQ7DQo+ICsJcmV0dXJuIChjbGllbnQgPT0g
TkZTX0NMSUVOVChkaXIpKSA/DQo+ICsJCQkJbmZzX2dldF9hdXRoX3JwY19jbGllbnQoY2xwLCBm
bGF2b3IpIDogY2xpZW50Ow0KPiAgfQ0KPiAgDQo+ICBzdGF0aWMgaW50IF9uZnM0X3Byb2NfYWNj
ZXNzKHN0cnVjdCBpbm9kZSAqaW5vZGUsIHN0cnVjdCBuZnNfYWNjZXNzX2VudHJ5ICplbnRyeSkN
Cj4gZGlmZiAtLWdpdCBhL2ZzL25mcy9uZnM0c3RhdGUuYyBiL2ZzL25mcy9uZnM0c3RhdGUuYw0K
PiBpbmRleCA2ODE4OTY0Li5mNzgxNWNlIDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvbmZzNHN0YXRl
LmMNCj4gKysrIGIvZnMvbmZzL25mczRzdGF0ZS5jDQo+IEBAIC0xODc2LDcgKzE4NzYsNyBAQCBh
Z2FpbjoNCj4gIAkJCWJyZWFrOw0KPiAgCWNhc2UgLU5GUzRFUlJfQ0xJRF9JTlVTRToNCj4gIAlj
YXNlIC1ORlM0RVJSX1dST05HU0VDOg0KPiAtCQljbG50ID0gcnBjX2Nsb25lX2NsaWVudF9zZXRf
YXV0aChjbG50LCBSUENfQVVUSF9VTklYKTsNCj4gKwkJY2xudCA9IG5mc19nZXRfYXV0aF9ycGNf
Y2xpZW50KGNscCwgUlBDX0FVVEhfVU5JWCk7DQo+ICAJCWlmIChJU19FUlIoY2xudCkpIHsNCj4g
IAkJCXN0YXR1cyA9IFBUUl9FUlIoY2xudCk7DQo+ICAJCQlicmVhazsNCj4gQEAgLTE4ODYsNyAr
MTg4Niw3IEBAIGFnYWluOg0KPiAgCQkgKiBjbHAtPmNsX3JwY2NsaWVudA0KPiAgCQkgKi8NCj4g
IAkJY2xudCA9IHhjaGcoJmNscC0+Y2xfcnBjY2xpZW50LCBjbG50KTsNCj4gLQkJcnBjX3NodXRk
b3duX2NsaWVudChjbG50KTsNCj4gKwkJbmZzX3NodXRkb3duX3JwY19jbGllbnQoY2xwLCBjbG50
KTsNCj4gIAkJY2xudCA9IGNscC0+Y2xfcnBjY2xpZW50Ow0KPiAgCQlnb3RvIGFnYWluOw0KPiAg
DQo+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L25mc19mc19zYi5oIGIvaW5jbHVkZS9saW51
eC9uZnNfZnNfc2IuaA0KPiBpbmRleCBkMjIxMjQzLi5lMTEyY2YyIDEwMDY0NA0KPiAtLS0gYS9p
bmNsdWRlL2xpbnV4L25mc19mc19zYi5oDQo+ICsrKyBiL2luY2x1ZGUvbGludXgvbmZzX2ZzX3Ni
LmgNCj4gQEAgLTU2LDYgKzU2LDcgQEAgc3RydWN0IG5mc19jbGllbnQgew0KPiAgCXN0cnVjdCBy
cGNfY3JlZAkJKmNsX21hY2hpbmVfY3JlZDsNCj4gIA0KPiAgI2lmIElTX0VOQUJMRUQoQ09ORklH
X05GU19WNCkNCj4gKwlzdHJ1Y3QgcnBjX2NsbnQgKgljbF9ycGNfY2xudFtSUENfQVVUSF9NQVhf
UFNFVURPX0ZMQVZPUiArIDFdOw0KDQpDYW4gd2UgbWFrZSB0aGlzIGR5bmFtaWNhbGx5IHNpemVk
PyBXZSBjdXJyZW50bHkgb25seSBzaGlwIHRoZSBORlMNCmNsaWVudCB3aXRoIDQgc3VwcG9ydGVk
IGF1dGggZmxhdm91cnMsIGJ1dCB0aGUgUlBDIGxheWVyIGl0c2VsZiBhbHJlYWR5DQpzdXBwb3J0
cyBhcmJpdHJhcnkgZmxhdm91cnMsIGFuZCBzbyBkb2VzIHRoZSBORlN2NCBzZWN1cml0eSBuZWdv
dGlhdGlvbi4NCg0KQWxzbywgd2h5IGRvIHdlIG9ubHkgbmVlZCB0aGlzIGZvciBORlN2ND8gRG9l
c24ndCBORlN2MyB3aXRoIHNlY3VyaXR5DQpuZWdvdGlhdGlvbiBoYXZlIHRoZSBzYW1lIHByb2Js
ZW0/DQoNCj4gIAl1NjQJCQljbF9jbGllbnRpZDsJLyogY29uc3RhbnQgKi8NCj4gIAluZnM0X3Zl
cmlmaWVyCQljbF9jb25maXJtOwkvKiBDbGllbnRpZCB2ZXJpZmllciAqLw0KPiAgCXVuc2lnbmVk
IGxvbmcJCWNsX3N0YXRlOw0KPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9zdW5ycGMvbXNn
X3Byb3QuaCBiL2luY2x1ZGUvbGludXgvc3VucnBjL21zZ19wcm90LmgNCj4gaW5kZXggMTY2NmRh
MC4uNTNjOWExMyAxMDA2NDQNCj4gLS0tIGEvaW5jbHVkZS9saW51eC9zdW5ycGMvbXNnX3Byb3Qu
aA0KPiArKysgYi9pbmNsdWRlL2xpbnV4L3N1bnJwYy9tc2dfcHJvdC5oDQo+IEBAIC0xNyw2ICsx
Nyw5IEBADQo+ICAvKiBzcGVjIGRlZmluZXMgYXV0aGVudGljYXRpb24gZmxhdm9yIGFzIGFuIHVu
c2lnbmVkIDMyIGJpdCBpbnRlZ2VyICovDQo+ICB0eXBlZGVmIHUzMglycGNfYXV0aGZsYXZvcl90
Ow0KPiAgDQo+ICsvKiBUaGUgbnVtYmVyIG9mIHN1cHBvcnRlZCBwc2V1ZG9mbGF2b3JzICovDQo+
ICsjZGVmaW5lIFJQQ19BVVRIX01BWF9QU0VVRE9fRkxBVk9SCTMNCj4gKw0KPiAgZW51bSBycGNf
YXV0aF9mbGF2b3JzIHsNCj4gIAlSUENfQVVUSF9OVUxMICA9IDAsDQo+ICAJUlBDX0FVVEhfVU5J
WCAgPSAxLA0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFp
bmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29t
DQo=
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients
2013-08-22 18:49 ` Myklebust, Trond
@ 2013-08-22 21:24 ` Adamson, Andy
0 siblings, 0 replies; 11+ messages in thread
From: Adamson, Andy @ 2013-08-22 21:24 UTC (permalink / raw)
To: Myklebust, Trond; +Cc: Adamson, Andy, linux-nfs@vger.kernel.org
On Aug 22, 2013, at 2:49 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
wrote:
> On Tue, 2013-08-13 at 17:47 -0400, andros@netapp.com wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> The current use of __rpc_clone_client results in multiple rpc clients with
>> the same GSS rpc_auth type, each with it's own gss_context cache.
>> This can result in multiple gss_contexts for <user, pseudoflavor, server> tuple.
>>
>> To avoid multiple gss_contexts share per server/pseudoflavor clients and the resultant gss_context cache.
>>
>> We also add the RPC_AUTH_UNIX client for the pNFS Data Server case used in
>> a subsequent patch, even though the RPC_AUTH_UNIX auth cache is already shared.
>>
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> ---
>> fs/nfs/client.c | 80 ++++++++++++++++++++++++++++++++++++++---
>> fs/nfs/internal.h | 4 +++
>> fs/nfs/nfs4namespace.c | 5 +--
>> fs/nfs/nfs4proc.c | 9 +++--
>> fs/nfs/nfs4state.c | 4 +--
>> include/linux/nfs_fs_sb.h | 1 +
>> include/linux/sunrpc/msg_prot.h | 3 ++
>> 7 files changed, 94 insertions(+), 12 deletions(-)
>>
>> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
>> index 2dceee4..1b4967f 100644
>> --- a/fs/nfs/client.c
>> +++ b/fs/nfs/client.c
>> @@ -152,7 +152,7 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
>> {
>> struct nfs_client *clp;
>> struct rpc_cred *cred;
>> - int err = -ENOMEM;
>> + int err = -ENOMEM, i;
>>
>> if ((clp = kzalloc(sizeof(*clp), GFP_KERNEL)) == NULL)
>> goto error_0;
>> @@ -178,6 +178,10 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
>> INIT_LIST_HEAD(&clp->cl_superblocks);
>> clp->cl_rpcclient = ERR_PTR(-EINVAL);
>>
>> + /* The three Kerberos pseudoflavors plus RPC_AUTH_UNIX */
>> + for (i = 0; i < RPC_AUTH_MAX_PSEUDO_FLAVOR + 1; i++)
>> + clp->cl_rpc_clnt[i] = ERR_PTR(-EINVAL);
>> +
>> clp->cl_proto = cl_init->proto;
>> clp->cl_net = get_net(cl_init->net);
>>
>> @@ -244,7 +248,7 @@ void nfs_free_client(struct nfs_client *clp)
>>
>> /* -EIO all pending I/O */
>> if (!IS_ERR(clp->cl_rpcclient))
>> - rpc_shutdown_client(clp->cl_rpcclient);
>> + nfs_shutdown_rpc_client(clp, clp->cl_rpcclient);
>>
>> if (clp->cl_machine_cred != NULL)
>> put_rpccred(clp->cl_machine_cred);
>> @@ -568,6 +572,68 @@ void nfs_init_timeout_values(struct rpc_timeout *to, int proto,
>> }
>> EXPORT_SYMBOL_GPL(nfs_init_timeout_values);
>>
>> +/**
>> + * Translate an authflavor into an nfs_client cl_rpc_clnt array index.
>> + * Note cl_gss_array is of RPC_AUTH_MAX_PSEUDO_FLAVOR + 1 size.
>> + */
>> +static inline
>> +int nfs_flavor_to_index(rpc_authflavor_t flavor)
>> +{
>> + switch (flavor) {
>> + case RPC_AUTH_UNIX:
>> + return 0;
>> + case RPC_AUTH_GSS_KRB5:
>> + return 1;
>> + case RPC_AUTH_GSS_KRB5I:
>> + return 2;
>> + case RPC_AUTH_GSS_KRB5P:
>> + return 3;
>> + default:
>> + return -EINVAL;
>> + }
>> +}
>> +
>> +struct rpc_clnt *
>> +nfs_get_auth_rpc_client(struct nfs_client *clp, rpc_authflavor_t flavor)
>> +{
>> + struct rpc_clnt *clnt = clp->cl_rpcclient;
>> + int idx;
>> +
>> + dprintk("--> %s cl_rpcclient %p flavor %d\n", __func__, clnt, flavor);
>> +
>> + idx = nfs_flavor_to_index(flavor);
>> + if (idx < 0)
>> + return ERR_PTR(idx);
>> +
>> + if (IS_ERR(clp->cl_rpc_clnt[idx]))
>> + clp->cl_rpc_clnt[idx] = rpc_clone_client_set_auth(clnt, flavor);
>> + else
>> + atomic_inc(&clp->cl_rpc_clnt[idx]->cl_count);
>> +
>> + return clp->cl_rpc_clnt[idx];
>> +}
>> +EXPORT_SYMBOL_GPL(nfs_get_auth_rpc_client);
>> +
>> +void
>> +nfs_shutdown_rpc_client(struct nfs_client *clp, struct rpc_clnt *clnt)
>> +{
>> + int idx;
>> +
>> + dprintk("--> %s clnt %p flavor %d rpc clnt cl_count (%d)\n", __func__,
>> + clnt, clnt->cl_auth->au_flavor, atomic_read(&clnt->cl_count));
>> +
>> + /* rpc client creation checked the validity of the flavor */
>> + idx = nfs_flavor_to_index(clnt->cl_auth->au_flavor);
>> +
>> + /* Dec the refcount unless it is 1 in which case call shutdown */
>> + if (atomic_add_unless(&clp->cl_rpc_clnt[idx]->cl_count, -1, 1) == 0) {
>> + rpc_shutdown_client(clnt);
>
> rpc_shutdown_client can sleep for a loooong time. What prevents
> nfs_get_auth_rpc_client() from trying to use the rpc client?
Good point.
>
>> + clp->cl_rpc_clnt[idx] = ERR_PTR(-EINVAL);
>> + return;
>> + }
>> +}
>> +EXPORT_SYMBOL_GPL(nfs_shutdown_rpc_client);
>> +
>> /*
>> * Create an RPC client handle
>> */
>> @@ -587,6 +653,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
>> .version = clp->rpc_ops->version,
>> .authflavor = flavor,
>> };
>> + int idx;
>>
>> if (test_bit(NFS_CS_DISCRTRY, &clp->cl_flags))
>> args.flags |= RPC_CLNT_CREATE_DISCRTRY;
>> @@ -605,7 +672,11 @@ int nfs_create_rpc_client(struct nfs_client *clp,
>> return PTR_ERR(clnt);
>> }
>>
>> + /* rpcauth_create has verified the validity of the authflavor. */
>> + idx = nfs_flavor_to_index(flavor);
>> + clp->cl_rpc_clnt[idx] = clnt;
>> clp->cl_rpcclient = clnt;
>> +
>> return 0;
>> }
>> EXPORT_SYMBOL_GPL(nfs_create_rpc_client);
>> @@ -668,8 +739,7 @@ int nfs_init_server_rpcclient(struct nfs_server *server,
>> {
>> struct nfs_client *clp = server->nfs_client;
>>
>> - server->client = rpc_clone_client_set_auth(clp->cl_rpcclient,
>> - pseudoflavour);
>> + server->client = nfs_get_auth_rpc_client(clp, pseudoflavour);
>> if (IS_ERR(server->client)) {
>> dprintk("%s: couldn't create rpc_client!\n", __func__);
>> return PTR_ERR(server->client);
>> @@ -1018,7 +1088,7 @@ void nfs_free_server(struct nfs_server *server)
>> if (!IS_ERR(server->client_acl))
>> rpc_shutdown_client(server->client_acl);
>> if (!IS_ERR(server->client))
>> - rpc_shutdown_client(server->client);
>> + nfs_shutdown_rpc_client(server->nfs_client, server->client);
>>
>> nfs_put_client(server->nfs_client);
>>
>> diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
>> index 9b694f1..4aa20ba 100644
>> --- a/fs/nfs/internal.h
>> +++ b/fs/nfs/internal.h
>> @@ -185,6 +185,10 @@ extern struct nfs_client *nfs4_set_ds_client(struct nfs_client* mds_clp,
>> int ds_addrlen, int ds_proto,
>> unsigned int ds_timeo,
>> unsigned int ds_retrans);
>> +extern struct rpc_clnt *nfs_get_auth_rpc_client(struct nfs_client *clp,
>> + rpc_authflavor_t flavor);
>> +extern void nfs_shutdown_rpc_client(struct nfs_client *clp,
>> + struct rpc_clnt *clnt);
>> #ifdef CONFIG_PROC_FS
>> extern int __init nfs_fs_proc_init(void);
>> extern void nfs_fs_proc_exit(void);
>> diff --git a/fs/nfs/nfs4namespace.c b/fs/nfs/nfs4namespace.c
>> index cdb0b41..b274427 100644
>> --- a/fs/nfs/nfs4namespace.c
>> +++ b/fs/nfs/nfs4namespace.c
>> @@ -205,7 +205,7 @@ struct rpc_clnt *nfs4_create_sec_client(struct rpc_clnt *clnt, struct inode *ino
>> if ((int)flavor < 0)
>> return ERR_PTR((int)flavor);
>>
>> - return rpc_clone_client_set_auth(clnt, flavor);
>> + return nfs_get_auth_rpc_client(NFS_SERVER(inode)->nfs_client, flavor);
>> }
>>
>> static struct vfsmount *try_location(struct nfs_clone_mount *mountdata,
>> @@ -370,6 +370,7 @@ struct vfsmount *nfs4_submount(struct nfs_server *server, struct dentry *dentry,
>> struct nfs_fh *fh, struct nfs_fattr *fattr)
>> {
>> struct dentry *parent = dget_parent(dentry);
>> + struct nfs_client *clp = NFS_SERVER(parent->d_inode)->nfs_client;
>> struct rpc_clnt *client;
>> struct vfsmount *mnt;
>>
>> @@ -384,6 +385,6 @@ struct vfsmount *nfs4_submount(struct nfs_server *server, struct dentry *dentry,
>> else
>> mnt = nfs_do_submount(dentry, fh, fattr, client->cl_auth->au_flavor);
>>
>> - rpc_shutdown_client(client);
>> + nfs_shutdown_rpc_client(clp, client);
>> return mnt;
>> }
>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>> index f672c34..9160fcb 100644
>> --- a/fs/nfs/nfs4proc.c
>> +++ b/fs/nfs/nfs4proc.c
>> @@ -3047,7 +3047,7 @@ out:
>> if (err == 0)
>> *clnt = client;
>> else if (client != *clnt)
>> - rpc_shutdown_client(client);
>> + nfs_shutdown_rpc_client(NFS_SERVER(dir)->nfs_client, client);
>>
>> return err;
>> }
>> @@ -3061,7 +3061,7 @@ static int nfs4_proc_lookup(struct inode *dir, struct qstr *name,
>>
>> status = nfs4_proc_lookup_common(&client, dir, name, fhandle, fattr, label);
>> if (client != NFS_CLIENT(dir)) {
>> - rpc_shutdown_client(client);
>> + nfs_shutdown_rpc_client(NFS_SERVER(dir)->nfs_client, client);
>> nfs_fixup_secinfo_attributes(fattr);
>> }
>> return status;
>> @@ -3072,12 +3072,15 @@ nfs4_proc_lookup_mountpoint(struct inode *dir, struct qstr *name,
>> struct nfs_fh *fhandle, struct nfs_fattr *fattr)
>> {
>> struct rpc_clnt *client = NFS_CLIENT(dir);
>> + rpc_authflavor_t flavor = client->cl_auth->au_flavor;
>> + struct nfs_client *clp = NFS_SERVER(dir)->nfs_client;
>> int status;
>>
>> status = nfs4_proc_lookup_common(&client, dir, name, fhandle, fattr, NULL);
>> if (status < 0)
>> return ERR_PTR(status);
>> - return (client == NFS_CLIENT(dir)) ? rpc_clone_client(client) : client;
>> + return (client == NFS_CLIENT(dir)) ?
>> + nfs_get_auth_rpc_client(clp, flavor) : client;
>> }
>>
>> static int _nfs4_proc_access(struct inode *inode, struct nfs_access_entry *entry)
>> diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
>> index 6818964..f7815ce 100644
>> --- a/fs/nfs/nfs4state.c
>> +++ b/fs/nfs/nfs4state.c
>> @@ -1876,7 +1876,7 @@ again:
>> break;
>> case -NFS4ERR_CLID_INUSE:
>> case -NFS4ERR_WRONGSEC:
>> - clnt = rpc_clone_client_set_auth(clnt, RPC_AUTH_UNIX);
>> + clnt = nfs_get_auth_rpc_client(clp, RPC_AUTH_UNIX);
>> if (IS_ERR(clnt)) {
>> status = PTR_ERR(clnt);
>> break;
>> @@ -1886,7 +1886,7 @@ again:
>> * clp->cl_rpcclient
>> */
>> clnt = xchg(&clp->cl_rpcclient, clnt);
>> - rpc_shutdown_client(clnt);
>> + nfs_shutdown_rpc_client(clp, clnt);
>> clnt = clp->cl_rpcclient;
>> goto again;
>>
>> diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
>> index d221243..e112cf2 100644
>> --- a/include/linux/nfs_fs_sb.h
>> +++ b/include/linux/nfs_fs_sb.h
>> @@ -56,6 +56,7 @@ struct nfs_client {
>> struct rpc_cred *cl_machine_cred;
>>
>> #if IS_ENABLED(CONFIG_NFS_V4)
>> + struct rpc_clnt * cl_rpc_clnt[RPC_AUTH_MAX_PSEUDO_FLAVOR + 1];
>
> Can we make this dynamically sized? We currently only ship the NFS
> client with 4 supported auth flavours, but the RPC layer itself already
> supports arbitrary flavours, and so does the NFSv4 security negotiation.
Yes - I can make it dynamic..
>
> Also, why do we only need this for NFSv4? Doesn't NFSv3 with security
> negotiation have the same problem?
A better approach might be to move the gss_auth caches into rpc_xprt for sharing.
-->Andy
>
>> u64 cl_clientid; /* constant */
>> nfs4_verifier cl_confirm; /* Clientid verifier */
>> unsigned long cl_state;
>> diff --git a/include/linux/sunrpc/msg_prot.h b/include/linux/sunrpc/msg_prot.h
>> index 1666da0..53c9a13 100644
>> --- a/include/linux/sunrpc/msg_prot.h
>> +++ b/include/linux/sunrpc/msg_prot.h
>> @@ -17,6 +17,9 @@
>> /* spec defines authentication flavor as an unsigned 32 bit integer */
>> typedef u32 rpc_authflavor_t;
>>
>> +/* The number of supported pseudoflavors */
>> +#define RPC_AUTH_MAX_PSEUDO_FLAVOR 3
>> +
>> enum rpc_auth_flavors {
>> RPC_AUTH_NULL = 0,
>> RPC_AUTH_UNIX = 1,
>
> --
> Trond Myklebust
> Linux NFS client maintainer
>
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients
2013-08-13 21:47 ` [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients andros
2013-08-22 18:49 ` Myklebust, Trond
@ 2013-09-03 16:04 ` Myklebust, Trond
1 sibling, 0 replies; 11+ messages in thread
From: Myklebust, Trond @ 2013-09-03 16:04 UTC (permalink / raw)
To: Adamson, Andy; +Cc: linux-nfs@vger.kernel.org
SGkgQW5keSwNCg0KT24gVHVlLCAyMDEzLTA4LTEzIGF0IDE3OjQ3IC0wNDAwLCBhbmRyb3NAbmV0
YXBwLmNvbSB3cm90ZToNCj4gRnJvbTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4N
Cj4gDQo+IFRoZSBjdXJyZW50IHVzZSBvZiBfX3JwY19jbG9uZV9jbGllbnQgcmVzdWx0cyBpbiBt
dWx0aXBsZSBycGMgY2xpZW50cyB3aXRoDQo+IHRoZSBzYW1lIEdTUyBycGNfYXV0aCB0eXBlLCBl
YWNoIHdpdGggaXQncyBvd24gZ3NzX2NvbnRleHQgY2FjaGUuDQo+IFRoaXMgY2FuIHJlc3VsdCBp
biBtdWx0aXBsZSBnc3NfY29udGV4dHMgZm9yIDx1c2VyLCBwc2V1ZG9mbGF2b3IsIHNlcnZlcj4g
dHVwbGUuDQo+IA0KPiBUbyBhdm9pZCBtdWx0aXBsZSBnc3NfY29udGV4dHMgc2hhcmUgcGVyIHNl
cnZlci9wc2V1ZG9mbGF2b3IgY2xpZW50cyBhbmQgdGhlIHJlc3VsdGFudCBnc3NfY29udGV4dCBj
YWNoZS4NCg0KVGhpcyBpc3N1ZSBoYXMgbm93IGJlZW4gc29sdmVkIGF0IHRoZSBycGNfYXV0aCBs
ZXZlbC4gUGxlYXNlIG5vdGUgdGhhdA0Kd2UgX2hhZF8gdG8gc29sdmUgaXQgdGhlcmUgc2luY2Ug
dGhlIHVwY2FsbCBwaXBlcyBmb3Iga3JiNSwga3JiNWkgYW5kDQprcmI1cCBuZWVkIHRvIGJlIHNo
YXJlZCB0b28uDQoNCj4gV2UgYWxzbyBhZGQgdGhlIFJQQ19BVVRIX1VOSVggY2xpZW50IGZvciB0
aGUgcE5GUyBEYXRhIFNlcnZlciBjYXNlIHVzZWQgaW4NCj4gYSBzdWJzZXF1ZW50IHBhdGNoLCBl
dmVuIHRob3VnaCB0aGUgUlBDX0FVVEhfVU5JWCBhdXRoIGNhY2hlIGlzIGFscmVhZHkgc2hhcmVk
Lg0KDQpDYW4geW91IHBsZWFzZSBsb29rIGludG8gd2hldGhlciBvciBub3Qgd2Ugc3RpbGwgd2Fu
dCB0byBkbyB0aGlzPw0KDQpDaGVlcnMsDQogIFRyb25kDQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0
DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RA
bmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH Version 5 3/3] NFSv4.1 Use the shared nfs_client rpc_clnts for pNFS data server connections
2013-08-13 21:47 [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections andros
2013-08-13 21:47 ` [PATCH Version 5 1/3] NFSv4 Remove unused LIPKEY and SPKM enums andros
2013-08-13 21:47 ` [PATCH Version 5 2/3] NFS Share RPC_AUTH_GSS rpc clients andros
@ 2013-08-13 21:47 ` andros
2013-08-23 14:01 ` [PATCH Version 5 0/3] Use the MDS nfs_server authflavor " Simo Sorce
3 siblings, 0 replies; 11+ messages in thread
From: andros @ 2013-08-13 21:47 UTC (permalink / raw)
To: trond.myklebust; +Cc: linux-nfs, Andy Adamson
From: Andy Adamson <andros@netapp.com>
pNFS data servers are not mounted in the normal sense as there is no associated
nfs_server structure.
Commit 4edaa308 "NFS: Use "krb5i" to establish NFSv4 state whenever possible"
uses the nfs_client cl_rpcclient for all state management operations, and
will use krb5i or auth_sys with no regard to the mount command authflavor
choice. For normal mounted servers, the nfs_server client authflavor is used
for all non-state management operations
Data servers use the same authflavor as the MDS mount for non-state management
operations. Note that the MDS has performed any sub-mounts and created an
nfs_server rpc client. Use the array of struct rpc_clnt in struct
nfs_client, one for each possible auth flavor for data server RPC connections.
Signed-off-by: Andy Adamson <andros@netapp.com>
---
fs/nfs/nfs4filelayout.c | 69 ++++++++++++++++++++++++++++++++++++++--------
fs/nfs/nfs4filelayout.h | 3 ++
fs/nfs/nfs4filelayoutdev.c | 4 ++-
3 files changed, 64 insertions(+), 12 deletions(-)
diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
index 17ed87e..f8faad7 100644
--- a/fs/nfs/nfs4filelayout.c
+++ b/fs/nfs/nfs4filelayout.c
@@ -83,6 +83,35 @@ filelayout_get_dserver_offset(struct pnfs_layout_segment *lseg, loff_t offset)
BUG();
}
+/* Use the au_flavor of the MDS nfs_server RPC client to find or clone the
+ * correct data server RPC client.
+ *
+ * Note that the MDS has already performed any sub-mounts and negotiated
+ * a security flavor.
+ */
+static struct rpc_clnt *
+filelayout_rpc_clnt(struct inode *inode, struct nfs_client *clp)
+{
+ rpc_authflavor_t flav = NFS_SERVER(inode)->client->cl_auth->au_flavor;
+
+ return nfs_get_auth_rpc_client(clp, flav);
+}
+
+void
+filelayout_shutdown_rpc_clnt(struct nfs_client *clp)
+{
+ int i;
+
+ /* The three Kerberos pseudoflavors plus RPC_AUTH_UNIX */
+ for (i = 0; i < RPC_AUTH_MAX_PSEUDO_FLAVOR + 1; i++) {
+ if (!IS_ERR(clp->cl_rpc_clnt[i])) {
+ dprintk("%s shutdown data server client %p index %d\n",
+ __func__, clp->cl_rpc_clnt[i], i);
+ nfs_shutdown_rpc_client(clp, clp->cl_rpc_clnt[i]);
+ }
+ }
+}
+
static void filelayout_reset_write(struct nfs_write_data *data)
{
struct nfs_pgio_header *hdr = data->header;
@@ -524,6 +553,7 @@ filelayout_read_pagelist(struct nfs_read_data *data)
struct nfs_pgio_header *hdr = data->header;
struct pnfs_layout_segment *lseg = hdr->lseg;
struct nfs4_pnfs_ds *ds;
+ struct rpc_clnt *ds_clnt;
loff_t offset = data->args.offset;
u32 j, idx;
struct nfs_fh *fh;
@@ -538,6 +568,11 @@ filelayout_read_pagelist(struct nfs_read_data *data)
ds = nfs4_fl_prepare_ds(lseg, idx);
if (!ds)
return PNFS_NOT_ATTEMPTED;
+
+ ds_clnt = filelayout_rpc_clnt(hdr->inode, ds->ds_clp);
+ if (IS_ERR(ds_clnt))
+ return PNFS_NOT_ATTEMPTED;
+
dprintk("%s USE DS: %s cl_count %d\n", __func__,
ds->ds_remotestr, atomic_read(&ds->ds_clp->cl_count));
@@ -552,8 +587,8 @@ filelayout_read_pagelist(struct nfs_read_data *data)
data->mds_offset = offset;
/* Perform an asynchronous read to ds */
- nfs_initiate_read(ds->ds_clp->cl_rpcclient, data,
- &filelayout_read_call_ops, RPC_TASK_SOFTCONN);
+ nfs_initiate_read(ds_clnt, data, &filelayout_read_call_ops,
+ RPC_TASK_SOFTCONN);
return PNFS_ATTEMPTED;
}
@@ -564,6 +599,7 @@ filelayout_write_pagelist(struct nfs_write_data *data, int sync)
struct nfs_pgio_header *hdr = data->header;
struct pnfs_layout_segment *lseg = hdr->lseg;
struct nfs4_pnfs_ds *ds;
+ struct rpc_clnt *ds_clnt;
loff_t offset = data->args.offset;
u32 j, idx;
struct nfs_fh *fh;
@@ -574,6 +610,11 @@ filelayout_write_pagelist(struct nfs_write_data *data, int sync)
ds = nfs4_fl_prepare_ds(lseg, idx);
if (!ds)
return PNFS_NOT_ATTEMPTED;
+
+ ds_clnt = filelayout_rpc_clnt(hdr->inode, ds->ds_clp);
+ if (IS_ERR(ds_clnt))
+ return PNFS_NOT_ATTEMPTED;
+
dprintk("%s ino %lu sync %d req %Zu@%llu DS: %s cl_count %d\n",
__func__, hdr->inode->i_ino, sync, (size_t) data->args.count,
offset, ds->ds_remotestr, atomic_read(&ds->ds_clp->cl_count));
@@ -591,9 +632,8 @@ filelayout_write_pagelist(struct nfs_write_data *data, int sync)
data->args.offset = filelayout_get_dserver_offset(lseg, offset);
/* Perform an asynchronous write */
- nfs_initiate_write(ds->ds_clp->cl_rpcclient, data,
- &filelayout_write_call_ops, sync,
- RPC_TASK_SOFTCONN);
+ nfs_initiate_write(ds_clnt, data, &filelayout_write_call_ops, sync,
+ RPC_TASK_SOFTCONN);
return PNFS_ATTEMPTED;
}
@@ -1101,16 +1141,19 @@ static int filelayout_initiate_commit(struct nfs_commit_data *data, int how)
{
struct pnfs_layout_segment *lseg = data->lseg;
struct nfs4_pnfs_ds *ds;
+ struct rpc_clnt *ds_clnt;
u32 idx;
struct nfs_fh *fh;
idx = calc_ds_index_from_commit(lseg, data->ds_commit_index);
ds = nfs4_fl_prepare_ds(lseg, idx);
- if (!ds) {
- prepare_to_resend_writes(data);
- filelayout_commit_release(data);
- return -EAGAIN;
- }
+ if (!ds)
+ goto out_err;
+
+ ds_clnt = filelayout_rpc_clnt(data->inode, ds->ds_clp);
+ if (IS_ERR(ds_clnt))
+ goto out_err;
+
dprintk("%s ino %lu, how %d cl_count %d\n", __func__,
data->inode->i_ino, how, atomic_read(&ds->ds_clp->cl_count));
data->commit_done_cb = filelayout_commit_done_cb;
@@ -1119,9 +1162,13 @@ static int filelayout_initiate_commit(struct nfs_commit_data *data, int how)
fh = select_ds_fh_from_commit(lseg, data->ds_commit_index);
if (fh)
data->args.fh = fh;
- return nfs_initiate_commit(ds->ds_clp->cl_rpcclient, data,
+ return nfs_initiate_commit(ds_clnt, data,
&filelayout_commit_call_ops, how,
RPC_TASK_SOFTCONN);
+out_err:
+ prepare_to_resend_writes(data);
+ filelayout_commit_release(data);
+ return -EAGAIN;
}
static int
diff --git a/fs/nfs/nfs4filelayout.h b/fs/nfs/nfs4filelayout.h
index cebd20e..64a2528 100644
--- a/fs/nfs/nfs4filelayout.h
+++ b/fs/nfs/nfs4filelayout.h
@@ -139,6 +139,9 @@ filelayout_test_devid_invalid(struct nfs4_deviceid_node *node)
extern bool
filelayout_test_devid_unavailable(struct nfs4_deviceid_node *node);
+extern void
+filelayout_shutdown_rpc_clnt(struct nfs_client *clp);
+
extern struct nfs_fh *
nfs4_fl_select_ds_fh(struct pnfs_layout_segment *lseg, u32 j);
diff --git a/fs/nfs/nfs4filelayoutdev.c b/fs/nfs/nfs4filelayoutdev.c
index 95604f6..cc10fd3 100644
--- a/fs/nfs/nfs4filelayoutdev.c
+++ b/fs/nfs/nfs4filelayoutdev.c
@@ -162,7 +162,7 @@ nfs4_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
int status = 0;
dprintk("--> %s DS %s au_flavor %d\n", __func__, ds->ds_remotestr,
- mds_srv->nfs_client->cl_rpcclient->cl_auth->au_flavor);
+ mds_srv->client->cl_auth->au_flavor);
list_for_each_entry(da, &ds->ds_addrs, da_node) {
dprintk("%s: DS %s: trying address %s\n",
@@ -203,6 +203,8 @@ destroy_ds(struct nfs4_pnfs_ds *ds)
ifdebug(FACILITY)
print_ds(ds);
+ filelayout_shutdown_rpc_clnt(ds->ds_clp);
+
if (ds->ds_clp)
nfs_put_client(ds->ds_clp);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections
2013-08-13 21:47 [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections andros
` (2 preceding siblings ...)
2013-08-13 21:47 ` [PATCH Version 5 3/3] NFSv4.1 Use the shared nfs_client rpc_clnts for pNFS data server connections andros
@ 2013-08-23 14:01 ` Simo Sorce
2013-08-23 15:24 ` Adamson, Andy
3 siblings, 1 reply; 11+ messages in thread
From: Simo Sorce @ 2013-08-23 14:01 UTC (permalink / raw)
To: andros; +Cc: trond.myklebust, linux-nfs
On Tue, 2013-08-13 at 17:47 -0400, andros@netapp.com wrote:
> From: Andy Adamson <andros@netapp.com>
>
> Differences from Version 4
> - added NFSv4 Remove unused LIPKEY and SPKM enums
> - shared GSS context caches when possible via shared rpc_clnt structs
If you share the gss context don't you risk mixing up sequence numbers
and sending out of sequence packets ?
Is that posible ? Is that going to be a problem with any
implementation ?
I do not see GSS_C_SEQUENCE_FLAG used in nfs-utils, nor do I see any
check for sequence in the unwrap call in the kernel so probably not a
problem (at least for linux-to-linux), but just thought about asking to
make sure anyway.
Simo.
> - DS conections use the shared rpc_clnts
>
> Andy Adamson (3):
> NFSv4 Remove unused LIPKEY and SPKM enums
> NFS Share RPC_AUTH_GSS rpc clients
> NFSv4.1 Use the shared nfs_client rpc_clnts for pNFS data server
> connections
>
> fs/nfs/client.c | 80 ++++++++++++++++++++++++++++++++++++++---
> fs/nfs/internal.h | 4 +++
> fs/nfs/nfs4filelayout.c | 69 +++++++++++++++++++++++++++++------
> fs/nfs/nfs4filelayout.h | 3 ++
> fs/nfs/nfs4filelayoutdev.c | 4 ++-
> fs/nfs/nfs4namespace.c | 5 +--
> fs/nfs/nfs4proc.c | 9 +++--
> fs/nfs/nfs4state.c | 4 +--
> fs/nfs/super.c | 34 ------------------
> include/linux/nfs_fs_sb.h | 1 +
> include/linux/sunrpc/msg_prot.h | 9 ++---
> 11 files changed, 158 insertions(+), 64 deletions(-)
>
--
Simo Sorce * Red Hat, Inc * New York
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH Version 5 0/3] Use the MDS nfs_server authflavor for pNFS data server connections
2013-08-23 14:01 ` [PATCH Version 5 0/3] Use the MDS nfs_server authflavor " Simo Sorce
@ 2013-08-23 15:24 ` Adamson, Andy
0 siblings, 0 replies; 11+ messages in thread
From: Adamson, Andy @ 2013-08-23 15:24 UTC (permalink / raw)
To: Simo Sorce; +Cc: Myklebust, Trond, <linux-nfs@vger.kernel.org>
On Aug 23, 2013, at 10:01 AM, Simo Sorce <simo@redhat.com> wrote:
> On Tue, 2013-08-13 at 17:47 -0400, andros@netapp.com wrote:
>> From: Andy Adamson <andros@netapp.com>
>>
>> Differences from Version 4
>> - added NFSv4 Remove unused LIPKEY and SPKM enums
>> - shared GSS context caches when possible via shared rpc_clnt structs
>
> If you share the gss context don't you risk mixing up sequence numbers
> and sending out of sequence packets ?
No, for a couple of reasons
1) RPCSEC_GSS requires that the GSS-API level sequencing is turned off - e.g. the sequence_req_flag is set to false.
rfc2203:
When GSS_Init_sec_context() is called, the parameters
replay_det_req_flag and sequence_req_flag must be turned off. The
reasons for this are:
So the RPCSEC_GSS layer does the sequencing, not the GSS layer.
2) I'm not really sharing GSS contexts - just trying to avoid creating multiple GSS contexts per <principal, target> tuple.
Really what I should be doing is creating a single GSS RPC auth cache for each supported RPC_AUTH_GSS pseudoflavor in the rpc_xprt struct, and have all rpc_clnt's that use the rpc_xprt use the GSS RPC auth.
-->Andy
> Is that posible ? Is that going to be a problem with any
> implementation ?
>
> I do not see GSS_C_SEQUENCE_FLAG used in nfs-utils, nor do I see any
> check for sequence in the unwrap call in the kernel so probably not a
> problem (at least for linux-to-linux), but just thought about asking to
> make sure anyway.
>
> Simo.
>
>> - DS conections use the shared rpc_clnts
>>
>> Andy Adamson (3):
>> NFSv4 Remove unused LIPKEY and SPKM enums
>> NFS Share RPC_AUTH_GSS rpc clients
>> NFSv4.1 Use the shared nfs_client rpc_clnts for pNFS data server
>> connections
>>
>> fs/nfs/client.c | 80 ++++++++++++++++++++++++++++++++++++++---
>> fs/nfs/internal.h | 4 +++
>> fs/nfs/nfs4filelayout.c | 69 +++++++++++++++++++++++++++++------
>> fs/nfs/nfs4filelayout.h | 3 ++
>> fs/nfs/nfs4filelayoutdev.c | 4 ++-
>> fs/nfs/nfs4namespace.c | 5 +--
>> fs/nfs/nfs4proc.c | 9 +++--
>> fs/nfs/nfs4state.c | 4 +--
>> fs/nfs/super.c | 34 ------------------
>> include/linux/nfs_fs_sb.h | 1 +
>> include/linux/sunrpc/msg_prot.h | 9 ++---
>> 11 files changed, 158 insertions(+), 64 deletions(-)
>>
>
>
> --
> Simo Sorce * Red Hat, Inc * New York
>
^ permalink raw reply [flat|nested] 11+ messages in thread