* [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP).
@ 2024-04-25 16:59 Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 1/6] arp: Move ATF_COM setting in arp_req_set() Kuniyuki Iwashima
` (5 more replies)
0 siblings, 6 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 16:59 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
arp_ioctl() holds rtnl_lock() regardless of cmd (SIOCDARP, SIOCSARP,
and SIOCGARP) to get net_device by __dev_get_by_name().
In the SIOCGARP path, arp_req_get() calls neigh_lookup(), which looks
up a neighbour entry under RCU.
This series cleans up ioctl() code a bit and extends the RCU section not
to take rtnl_lock() and instead use dev_get_by_name_rcu() for SIOCGARP.
Changes:
v2:
Patch 5: s/!IS_ERR/IS_ERR/ in arp_req_delete().
v1: https://lore.kernel.org/netdev/20240422194755.4221-1-kuniyu@amazon.com/
Kuniyuki Iwashima (6):
arp: Move ATF_COM setting in arp_req_set().
arp: Validate netmask earlier for SIOCDARP and SIOCSARP in
arp_ioctl().
arp: Factorise ip_route_output() call in arp_req_set() and
arp_req_delete().
arp: Remove a nest in arp_req_get().
arp: Get dev after calling arp_req_(delete|set|get)().
arp: Convert ioctl(SIOCGARP) to RCU.
net/ipv4/arp.c | 203 ++++++++++++++++++++++++++++++-------------------
1 file changed, 123 insertions(+), 80 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 1/6] arp: Move ATF_COM setting in arp_req_set().
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
@ 2024-04-25 16:59 ` Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 2/6] arp: Validate netmask earlier for SIOCDARP and SIOCSARP in arp_ioctl() Kuniyuki Iwashima
` (4 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 16:59 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
In arp_req_set(), if ATF_PERM is set in arpreq.arp_flags,
ATF_COM is set automatically.
The flag will be used later for neigh_update() only when
a neighbour entry is found.
Let's set ATF_COM just before calling neigh_update().
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index ab82ca104496..3093374165fa 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1054,8 +1054,7 @@ static int arp_req_set(struct net *net, struct arpreq *r,
return arp_req_set_public(net, r, dev);
ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
- if (r->arp_flags & ATF_PERM)
- r->arp_flags |= ATF_COM;
+
if (!dev) {
struct rtable *rt = ip_route_output(net, ip, 0, 0, 0,
RT_SCOPE_LINK);
@@ -1092,8 +1091,12 @@ static int arp_req_set(struct net *net, struct arpreq *r,
err = PTR_ERR(neigh);
if (!IS_ERR(neigh)) {
unsigned int state = NUD_STALE;
- if (r->arp_flags & ATF_PERM)
+
+ if (r->arp_flags & ATF_PERM) {
+ r->arp_flags |= ATF_COM;
state = NUD_PERMANENT;
+ }
+
err = neigh_update(neigh, (r->arp_flags & ATF_COM) ?
r->arp_ha.sa_data : NULL, state,
NEIGH_UPDATE_F_OVERRIDE |
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 2/6] arp: Validate netmask earlier for SIOCDARP and SIOCSARP in arp_ioctl().
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 1/6] arp: Move ATF_COM setting in arp_req_set() Kuniyuki Iwashima
@ 2024-04-25 16:59 ` Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 3/6] arp: Factorise ip_route_output() call in arp_req_set() and arp_req_delete() Kuniyuki Iwashima
` (3 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 16:59 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
When ioctl(SIOCDARP/SIOCSARP) is issued with ATF_PUBL, r.arp_netmask
must be 0.0.0.0 or 255.255.255.255.
Currently, the netmask is validated in arp_req_delete_public() or
arp_req_set_public() under rtnl_lock().
We have ATF_NETMASK test in arp_ioctl() before holding rtnl_lock(),
so let's move the netmask validation there.
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 3093374165fa..b20a5771d069 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1023,11 +1023,8 @@ static int arp_req_set_proxy(struct net *net, struct net_device *dev, int on)
static int arp_req_set_public(struct net *net, struct arpreq *r,
struct net_device *dev)
{
- __be32 ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
__be32 mask = ((struct sockaddr_in *)&r->arp_netmask)->sin_addr.s_addr;
- if (mask && mask != htonl(0xFFFFFFFF))
- return -EINVAL;
if (!dev && (r->arp_flags & ATF_COM)) {
dev = dev_getbyhwaddr_rcu(net, r->arp_ha.sa_family,
r->arp_ha.sa_data);
@@ -1035,6 +1032,8 @@ static int arp_req_set_public(struct net *net, struct arpreq *r,
return -ENODEV;
}
if (mask) {
+ __be32 ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
+
if (!pneigh_lookup(&arp_tbl, net, &ip, dev, 1))
return -ENOBUFS;
return 0;
@@ -1171,14 +1170,13 @@ int arp_invalidate(struct net_device *dev, __be32 ip, bool force)
static int arp_req_delete_public(struct net *net, struct arpreq *r,
struct net_device *dev)
{
- __be32 ip = ((struct sockaddr_in *) &r->arp_pa)->sin_addr.s_addr;
__be32 mask = ((struct sockaddr_in *)&r->arp_netmask)->sin_addr.s_addr;
- if (mask == htonl(0xFFFFFFFF))
- return pneigh_delete(&arp_tbl, net, &ip, dev);
+ if (mask) {
+ __be32 ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
- if (mask)
- return -EINVAL;
+ return pneigh_delete(&arp_tbl, net, &ip, dev);
+ }
return arp_req_set_proxy(net, dev, 0);
}
@@ -1211,9 +1209,10 @@ static int arp_req_delete(struct net *net, struct arpreq *r,
int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
{
- int err;
- struct arpreq r;
struct net_device *dev = NULL;
+ struct arpreq r;
+ __be32 *netmask;
+ int err;
switch (cmd) {
case SIOCDARP:
@@ -1236,9 +1235,13 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
if (!(r.arp_flags & ATF_PUBL) &&
(r.arp_flags & (ATF_NETMASK | ATF_DONTPUB)))
return -EINVAL;
+
+ netmask = &((struct sockaddr_in *)&r.arp_netmask)->sin_addr.s_addr;
if (!(r.arp_flags & ATF_NETMASK))
- ((struct sockaddr_in *)&r.arp_netmask)->sin_addr.s_addr =
- htonl(0xFFFFFFFFUL);
+ *netmask = htonl(0xFFFFFFFFUL);
+ else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
+ return -EINVAL;
+
rtnl_lock();
if (r.arp_dev[0]) {
err = -ENODEV;
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 3/6] arp: Factorise ip_route_output() call in arp_req_set() and arp_req_delete().
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 1/6] arp: Move ATF_COM setting in arp_req_set() Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 2/6] arp: Validate netmask earlier for SIOCDARP and SIOCSARP in arp_ioctl() Kuniyuki Iwashima
@ 2024-04-25 16:59 ` Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 4/6] arp: Remove a nest in arp_req_get() Kuniyuki Iwashima
` (2 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 16:59 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
When ioctl(SIOCDARP/SIOCSARP) is issued for non-proxy entry (no ATF_COM)
without arpreq.arp_dev[] set, arp_req_set() and arp_req_delete() looks up
dev based on IPv4 address by ip_route_output().
Let's factorise the same code as arp_req_dev().
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 50 ++++++++++++++++++++++++++++++--------------------
1 file changed, 30 insertions(+), 20 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index b20a5771d069..ac3e15799c2f 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1003,6 +1003,27 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
* User level interface (ioctl)
*/
+static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
+{
+ struct net_device *dev;
+ struct rtable *rt;
+ __be32 ip;
+
+ ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
+
+ rt = ip_route_output(net, ip, 0, 0, 0, RT_SCOPE_LINK);
+ if (IS_ERR(rt))
+ return ERR_CAST(rt);
+
+ dev = rt->dst.dev;
+ ip_rt_put(rt);
+
+ if (!dev)
+ return ERR_PTR(-EINVAL);
+
+ return dev;
+}
+
/*
* Set (create) an ARP cache entry.
*/
@@ -1045,25 +1066,17 @@ static int arp_req_set_public(struct net *net, struct arpreq *r,
static int arp_req_set(struct net *net, struct arpreq *r,
struct net_device *dev)
{
- __be32 ip;
struct neighbour *neigh;
+ __be32 ip;
int err;
if (r->arp_flags & ATF_PUBL)
return arp_req_set_public(net, r, dev);
- ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
-
if (!dev) {
- struct rtable *rt = ip_route_output(net, ip, 0, 0, 0,
- RT_SCOPE_LINK);
-
- if (IS_ERR(rt))
- return PTR_ERR(rt);
- dev = rt->dst.dev;
- ip_rt_put(rt);
- if (!dev)
- return -EINVAL;
+ dev = arp_req_dev(net, r);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
}
switch (dev->type) {
#if IS_ENABLED(CONFIG_FDDI)
@@ -1086,6 +1099,8 @@ static int arp_req_set(struct net *net, struct arpreq *r,
break;
}
+ ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
+
neigh = __neigh_lookup_errno(&arp_tbl, &ip, dev);
err = PTR_ERR(neigh);
if (!IS_ERR(neigh)) {
@@ -1191,14 +1206,9 @@ static int arp_req_delete(struct net *net, struct arpreq *r,
ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
if (!dev) {
- struct rtable *rt = ip_route_output(net, ip, 0, 0, 0,
- RT_SCOPE_LINK);
- if (IS_ERR(rt))
- return PTR_ERR(rt);
- dev = rt->dst.dev;
- ip_rt_put(rt);
- if (!dev)
- return -EINVAL;
+ dev = arp_req_dev(net, r);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
}
return arp_invalidate(dev, ip, true);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 4/6] arp: Remove a nest in arp_req_get().
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
` (2 preceding siblings ...)
2024-04-25 16:59 ` [PATCH v2 net-next 3/6] arp: Factorise ip_route_output() call in arp_req_set() and arp_req_delete() Kuniyuki Iwashima
@ 2024-04-25 17:00 ` Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 5/6] arp: Get dev after calling arp_req_(delete|set|get)() Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU Kuniyuki Iwashima
5 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 17:00 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
This is a prep patch to make the following changes tidy.
No functional change intended.
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 31 ++++++++++++++++++-------------
1 file changed, 18 insertions(+), 13 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index ac3e15799c2f..60f633b24ec8 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1138,23 +1138,28 @@ static int arp_req_get(struct arpreq *r, struct net_device *dev)
{
__be32 ip = ((struct sockaddr_in *) &r->arp_pa)->sin_addr.s_addr;
struct neighbour *neigh;
- int err = -ENXIO;
neigh = neigh_lookup(&arp_tbl, &ip, dev);
- if (neigh) {
- if (!(READ_ONCE(neigh->nud_state) & NUD_NOARP)) {
- read_lock_bh(&neigh->lock);
- memcpy(r->arp_ha.sa_data, neigh->ha,
- min(dev->addr_len, sizeof(r->arp_ha.sa_data_min)));
- r->arp_flags = arp_state_to_flags(neigh);
- read_unlock_bh(&neigh->lock);
- r->arp_ha.sa_family = dev->type;
- strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
- err = 0;
- }
+ if (!neigh)
+ return -ENXIO;
+
+ if (READ_ONCE(neigh->nud_state) & NUD_NOARP) {
neigh_release(neigh);
+ return -ENXIO;
}
- return err;
+
+ read_lock_bh(&neigh->lock);
+ memcpy(r->arp_ha.sa_data, neigh->ha,
+ min(dev->addr_len, sizeof(r->arp_ha.sa_data_min)));
+ r->arp_flags = arp_state_to_flags(neigh);
+ read_unlock_bh(&neigh->lock);
+
+ neigh_release(neigh);
+
+ r->arp_ha.sa_family = dev->type;
+ strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
+
+ return 0;
}
int arp_invalidate(struct net_device *dev, __be32 ip, bool force)
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 5/6] arp: Get dev after calling arp_req_(delete|set|get)().
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
` (3 preceding siblings ...)
2024-04-25 17:00 ` [PATCH v2 net-next 4/6] arp: Remove a nest in arp_req_get() Kuniyuki Iwashima
@ 2024-04-25 17:00 ` Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU Kuniyuki Iwashima
5 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 17:00 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
arp_ioctl() holds rtnl_lock() first regardless of cmd (SIOCDARP,
SIOCSARP, and SIOCGARP) to get net_device by __dev_get_by_name().
In the SIOCGARP path, arp_req_get() calls neigh_lookup(), which
looks up a neighbour entry under RCU.
We will extend the RCU section not to take rtnl_lock() and instead
use dev_get_by_name_rcu() for SIOCGARP.
As a preparation, let's move __dev_get_by_name() into another
function and call it from arp_req_delete(), arp_req_set(), and
arp_req_get().
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 86 +++++++++++++++++++++++++++++---------------------
1 file changed, 50 insertions(+), 36 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 60f633b24ec8..5034920be85a 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1003,12 +1003,36 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
* User level interface (ioctl)
*/
+static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
+{
+ struct net_device *dev;
+
+ dev = __dev_get_by_name(net, r->arp_dev);
+ if (!dev)
+ return ERR_PTR(-ENODEV);
+
+ /* Mmmm... It is wrong... ARPHRD_NETROM == 0 */
+ if (!r->arp_ha.sa_family)
+ r->arp_ha.sa_family = dev->type;
+
+ if ((r->arp_flags & ATF_COM) && r->arp_ha.sa_family != dev->type)
+ return ERR_PTR(-EINVAL);
+
+ return dev;
+}
+
static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
{
struct net_device *dev;
struct rtable *rt;
__be32 ip;
+ if (r->arp_dev[0])
+ return arp_req_dev_by_name(net, r);
+
+ if (r->arp_flags & ATF_PUBL)
+ return NULL;
+
ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
rt = ip_route_output(net, ip, 0, 0, 0, RT_SCOPE_LINK);
@@ -1063,21 +1087,20 @@ static int arp_req_set_public(struct net *net, struct arpreq *r,
return arp_req_set_proxy(net, dev, 1);
}
-static int arp_req_set(struct net *net, struct arpreq *r,
- struct net_device *dev)
+static int arp_req_set(struct net *net, struct arpreq *r)
{
struct neighbour *neigh;
+ struct net_device *dev;
__be32 ip;
int err;
+ dev = arp_req_dev(net, r);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
if (r->arp_flags & ATF_PUBL)
return arp_req_set_public(net, r, dev);
- if (!dev) {
- dev = arp_req_dev(net, r);
- if (IS_ERR(dev))
- return PTR_ERR(dev);
- }
switch (dev->type) {
#if IS_ENABLED(CONFIG_FDDI)
case ARPHRD_FDDI:
@@ -1134,10 +1157,18 @@ static unsigned int arp_state_to_flags(struct neighbour *neigh)
* Get an ARP cache entry.
*/
-static int arp_req_get(struct arpreq *r, struct net_device *dev)
+static int arp_req_get(struct net *net, struct arpreq *r)
{
__be32 ip = ((struct sockaddr_in *) &r->arp_pa)->sin_addr.s_addr;
struct neighbour *neigh;
+ struct net_device *dev;
+
+ if (!r->arp_dev[0])
+ return -ENODEV;
+
+ dev = arp_req_dev_by_name(net, r);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
neigh = neigh_lookup(&arp_tbl, &ip, dev);
if (!neigh)
@@ -1201,20 +1232,20 @@ static int arp_req_delete_public(struct net *net, struct arpreq *r,
return arp_req_set_proxy(net, dev, 0);
}
-static int arp_req_delete(struct net *net, struct arpreq *r,
- struct net_device *dev)
+static int arp_req_delete(struct net *net, struct arpreq *r)
{
+ struct net_device *dev;
__be32 ip;
+ dev = arp_req_dev(net, r);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
if (r->arp_flags & ATF_PUBL)
return arp_req_delete_public(net, r, dev);
ip = ((struct sockaddr_in *)&r->arp_pa)->sin_addr.s_addr;
- if (!dev) {
- dev = arp_req_dev(net, r);
- if (IS_ERR(dev))
- return PTR_ERR(dev);
- }
+
return arp_invalidate(dev, ip, true);
}
@@ -1224,7 +1255,6 @@ static int arp_req_delete(struct net *net, struct arpreq *r,
int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
{
- struct net_device *dev = NULL;
struct arpreq r;
__be32 *netmask;
int err;
@@ -1258,35 +1288,19 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
return -EINVAL;
rtnl_lock();
- if (r.arp_dev[0]) {
- err = -ENODEV;
- dev = __dev_get_by_name(net, r.arp_dev);
- if (!dev)
- goto out;
-
- /* Mmmm... It is wrong... ARPHRD_NETROM==0 */
- if (!r.arp_ha.sa_family)
- r.arp_ha.sa_family = dev->type;
- err = -EINVAL;
- if ((r.arp_flags & ATF_COM) && r.arp_ha.sa_family != dev->type)
- goto out;
- } else if (cmd == SIOCGARP) {
- err = -ENODEV;
- goto out;
- }
switch (cmd) {
case SIOCDARP:
- err = arp_req_delete(net, &r, dev);
+ err = arp_req_delete(net, &r);
break;
case SIOCSARP:
- err = arp_req_set(net, &r, dev);
+ err = arp_req_set(net, &r);
break;
case SIOCGARP:
- err = arp_req_get(&r, dev);
+ err = arp_req_get(net, &r);
break;
}
-out:
+
rtnl_unlock();
if (cmd == SIOCGARP && !err && copy_to_user(arg, &r, sizeof(r)))
err = -EFAULT;
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU.
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
` (4 preceding siblings ...)
2024-04-25 17:00 ` [PATCH v2 net-next 5/6] arp: Get dev after calling arp_req_(delete|set|get)() Kuniyuki Iwashima
@ 2024-04-25 17:00 ` Kuniyuki Iwashima
2024-04-25 17:12 ` Eric Dumazet
5 siblings, 1 reply; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 17:00 UTC (permalink / raw)
To: David S. Miller, David Ahern, Eric Dumazet, Jakub Kicinski,
Paolo Abeni
Cc: Kuniyuki Iwashima, Kuniyuki Iwashima, netdev
ioctl(SIOCGARP) holds rtnl_lock() for __dev_get_by_name() and
later calls neigh_lookup(), which calls rcu_read_lock().
Let's replace __dev_get_by_name() with dev_get_by_name_rcu() to
avoid locking rtnl_lock().
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
net/ipv4/arp.c | 26 +++++++++++++++++---------
1 file changed, 17 insertions(+), 9 deletions(-)
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 5034920be85a..9430b64558cd 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1003,11 +1003,15 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
* User level interface (ioctl)
*/
-static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
+static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r,
+ bool getarp)
{
struct net_device *dev;
- dev = __dev_get_by_name(net, r->arp_dev);
+ if (getarp)
+ dev = dev_get_by_name_rcu(net, r->arp_dev);
+ else
+ dev = __dev_get_by_name(net, r->arp_dev);
if (!dev)
return ERR_PTR(-ENODEV);
@@ -1028,7 +1032,7 @@ static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
__be32 ip;
if (r->arp_dev[0])
- return arp_req_dev_by_name(net, r);
+ return arp_req_dev_by_name(net, r, false);
if (r->arp_flags & ATF_PUBL)
return NULL;
@@ -1166,7 +1170,7 @@ static int arp_req_get(struct net *net, struct arpreq *r)
if (!r->arp_dev[0])
return -ENODEV;
- dev = arp_req_dev_by_name(net, r);
+ dev = arp_req_dev_by_name(net, r, true);
if (IS_ERR(dev))
return PTR_ERR(dev);
@@ -1287,23 +1291,27 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
return -EINVAL;
- rtnl_lock();
-
switch (cmd) {
case SIOCDARP:
+ rtnl_lock();
err = arp_req_delete(net, &r);
+ rtnl_unlock();
break;
case SIOCSARP:
+ rtnl_lock();
err = arp_req_set(net, &r);
+ rtnl_unlock();
break;
case SIOCGARP:
+ rcu_read_lock();
err = arp_req_get(net, &r);
+ rcu_read_unlock();
+
+ if (!err && copy_to_user(arg, &r, sizeof(r)))
+ err = -EFAULT;
break;
}
- rtnl_unlock();
- if (cmd == SIOCGARP && !err && copy_to_user(arg, &r, sizeof(r)))
- err = -EFAULT;
return err;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU.
2024-04-25 17:00 ` [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU Kuniyuki Iwashima
@ 2024-04-25 17:12 ` Eric Dumazet
2024-04-25 17:51 ` Kuniyuki Iwashima
0 siblings, 1 reply; 11+ messages in thread
From: Eric Dumazet @ 2024-04-25 17:12 UTC (permalink / raw)
To: Kuniyuki Iwashima
Cc: David S. Miller, David Ahern, Jakub Kicinski, Paolo Abeni,
Kuniyuki Iwashima, netdev
On Thu, Apr 25, 2024 at 7:02 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
>
> ioctl(SIOCGARP) holds rtnl_lock() for __dev_get_by_name() and
> later calls neigh_lookup(), which calls rcu_read_lock().
>
> Let's replace __dev_get_by_name() with dev_get_by_name_rcu() to
> avoid locking rtnl_lock().
>
> Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> ---
> net/ipv4/arp.c | 26 +++++++++++++++++---------
> 1 file changed, 17 insertions(+), 9 deletions(-)
>
> diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
> index 5034920be85a..9430b64558cd 100644
> --- a/net/ipv4/arp.c
> +++ b/net/ipv4/arp.c
> @@ -1003,11 +1003,15 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
> * User level interface (ioctl)
> */
>
> -static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
> +static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r,
> + bool getarp)
> {
> struct net_device *dev;
>
> - dev = __dev_get_by_name(net, r->arp_dev);
> + if (getarp)
> + dev = dev_get_by_name_rcu(net, r->arp_dev);
> + else
> + dev = __dev_get_by_name(net, r->arp_dev);
> if (!dev)
> return ERR_PTR(-ENODEV);
>
> @@ -1028,7 +1032,7 @@ static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
> __be32 ip;
>
> if (r->arp_dev[0])
> - return arp_req_dev_by_name(net, r);
> + return arp_req_dev_by_name(net, r, false);
>
> if (r->arp_flags & ATF_PUBL)
> return NULL;
> @@ -1166,7 +1170,7 @@ static int arp_req_get(struct net *net, struct arpreq *r)
> if (!r->arp_dev[0])
> return -ENODEV;
>
> - dev = arp_req_dev_by_name(net, r);
> + dev = arp_req_dev_by_name(net, r, true);
> if (IS_ERR(dev))
> return PTR_ERR(dev);
>
> @@ -1287,23 +1291,27 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
> else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
> return -EINVAL;
>
> - rtnl_lock();
> -
> switch (cmd) {
> case SIOCDARP:
> + rtnl_lock();
> err = arp_req_delete(net, &r);
> + rtnl_unlock();
> break;
> case SIOCSARP:
> + rtnl_lock();
> err = arp_req_set(net, &r);
> + rtnl_unlock();
> break;
> case SIOCGARP:
> + rcu_read_lock();
> err = arp_req_get(net, &r);
> + rcu_read_unlock();
Note that arp_req_get() uses :
strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
This currently depends on RTNL or devnet_rename_sem
Perhaps we should add a helper and use a seqlock to safely copy
dev->name into a temporary variable.
netdev_get_name() can not be called from rcu_read_lock() at this
moment unfortunately.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU.
2024-04-25 17:12 ` Eric Dumazet
@ 2024-04-25 17:51 ` Kuniyuki Iwashima
2024-04-25 20:35 ` Eric Dumazet
0 siblings, 1 reply; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 17:51 UTC (permalink / raw)
To: edumazet; +Cc: davem, dsahern, kuba, kuni1840, kuniyu, netdev, pabeni
From: Eric Dumazet <edumazet@google.com>
Date: Thu, 25 Apr 2024 19:12:56 +0200
> On Thu, Apr 25, 2024 at 7:02 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> >
> > ioctl(SIOCGARP) holds rtnl_lock() for __dev_get_by_name() and
> > later calls neigh_lookup(), which calls rcu_read_lock().
> >
> > Let's replace __dev_get_by_name() with dev_get_by_name_rcu() to
> > avoid locking rtnl_lock().
> >
> > Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> > ---
> > net/ipv4/arp.c | 26 +++++++++++++++++---------
> > 1 file changed, 17 insertions(+), 9 deletions(-)
> >
> > diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
> > index 5034920be85a..9430b64558cd 100644
> > --- a/net/ipv4/arp.c
> > +++ b/net/ipv4/arp.c
> > @@ -1003,11 +1003,15 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
> > * User level interface (ioctl)
> > */
> >
> > -static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
> > +static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r,
> > + bool getarp)
> > {
> > struct net_device *dev;
> >
> > - dev = __dev_get_by_name(net, r->arp_dev);
> > + if (getarp)
> > + dev = dev_get_by_name_rcu(net, r->arp_dev);
> > + else
> > + dev = __dev_get_by_name(net, r->arp_dev);
> > if (!dev)
> > return ERR_PTR(-ENODEV);
> >
> > @@ -1028,7 +1032,7 @@ static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
> > __be32 ip;
> >
> > if (r->arp_dev[0])
> > - return arp_req_dev_by_name(net, r);
> > + return arp_req_dev_by_name(net, r, false);
> >
> > if (r->arp_flags & ATF_PUBL)
> > return NULL;
> > @@ -1166,7 +1170,7 @@ static int arp_req_get(struct net *net, struct arpreq *r)
> > if (!r->arp_dev[0])
> > return -ENODEV;
> >
> > - dev = arp_req_dev_by_name(net, r);
> > + dev = arp_req_dev_by_name(net, r, true);
> > if (IS_ERR(dev))
> > return PTR_ERR(dev);
> >
> > @@ -1287,23 +1291,27 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
> > else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
> > return -EINVAL;
> >
> > - rtnl_lock();
> > -
> > switch (cmd) {
> > case SIOCDARP:
> > + rtnl_lock();
> > err = arp_req_delete(net, &r);
> > + rtnl_unlock();
> > break;
> > case SIOCSARP:
> > + rtnl_lock();
> > err = arp_req_set(net, &r);
> > + rtnl_unlock();
> > break;
> > case SIOCGARP:
> > + rcu_read_lock();
> > err = arp_req_get(net, &r);
> > + rcu_read_unlock();
>
>
> Note that arp_req_get() uses :
>
> strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
>
> This currently depends on RTNL or devnet_rename_sem
Ah, I missed this point, thanks for catching!
>
> Perhaps we should add a helper and use a seqlock to safely copy
> dev->name into a temporary variable.
So it's preferable to add seqlock around memcpy() in dev_change_name()
and use a helper in arp_req_get() rather than adding devnet_rename_sem
locking around memcpy() in arp_req_get() ?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU.
2024-04-25 17:51 ` Kuniyuki Iwashima
@ 2024-04-25 20:35 ` Eric Dumazet
2024-04-25 20:47 ` Kuniyuki Iwashima
0 siblings, 1 reply; 11+ messages in thread
From: Eric Dumazet @ 2024-04-25 20:35 UTC (permalink / raw)
To: Kuniyuki Iwashima; +Cc: davem, dsahern, kuba, kuni1840, netdev, pabeni
On Thu, Apr 25, 2024 at 7:52 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
> Date: Thu, 25 Apr 2024 19:12:56 +0200
> > On Thu, Apr 25, 2024 at 7:02 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> > >
> > > ioctl(SIOCGARP) holds rtnl_lock() for __dev_get_by_name() and
> > > later calls neigh_lookup(), which calls rcu_read_lock().
> > >
> > > Let's replace __dev_get_by_name() with dev_get_by_name_rcu() to
> > > avoid locking rtnl_lock().
> > >
> > > Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> > > ---
> > > net/ipv4/arp.c | 26 +++++++++++++++++---------
> > > 1 file changed, 17 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
> > > index 5034920be85a..9430b64558cd 100644
> > > --- a/net/ipv4/arp.c
> > > +++ b/net/ipv4/arp.c
> > > @@ -1003,11 +1003,15 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
> > > * User level interface (ioctl)
> > > */
> > >
> > > -static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
> > > +static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r,
> > > + bool getarp)
> > > {
> > > struct net_device *dev;
> > >
> > > - dev = __dev_get_by_name(net, r->arp_dev);
> > > + if (getarp)
> > > + dev = dev_get_by_name_rcu(net, r->arp_dev);
> > > + else
> > > + dev = __dev_get_by_name(net, r->arp_dev);
> > > if (!dev)
> > > return ERR_PTR(-ENODEV);
> > >
> > > @@ -1028,7 +1032,7 @@ static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
> > > __be32 ip;
> > >
> > > if (r->arp_dev[0])
> > > - return arp_req_dev_by_name(net, r);
> > > + return arp_req_dev_by_name(net, r, false);
> > >
> > > if (r->arp_flags & ATF_PUBL)
> > > return NULL;
> > > @@ -1166,7 +1170,7 @@ static int arp_req_get(struct net *net, struct arpreq *r)
> > > if (!r->arp_dev[0])
> > > return -ENODEV;
> > >
> > > - dev = arp_req_dev_by_name(net, r);
> > > + dev = arp_req_dev_by_name(net, r, true);
> > > if (IS_ERR(dev))
> > > return PTR_ERR(dev);
> > >
> > > @@ -1287,23 +1291,27 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
> > > else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
> > > return -EINVAL;
> > >
> > > - rtnl_lock();
> > > -
> > > switch (cmd) {
> > > case SIOCDARP:
> > > + rtnl_lock();
> > > err = arp_req_delete(net, &r);
> > > + rtnl_unlock();
> > > break;
> > > case SIOCSARP:
> > > + rtnl_lock();
> > > err = arp_req_set(net, &r);
> > > + rtnl_unlock();
> > > break;
> > > case SIOCGARP:
> > > + rcu_read_lock();
> > > err = arp_req_get(net, &r);
> > > + rcu_read_unlock();
> >
> >
> > Note that arp_req_get() uses :
> >
> > strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
> >
> > This currently depends on RTNL or devnet_rename_sem
>
> Ah, I missed this point, thanks for catching!
>
>
> >
> > Perhaps we should add a helper and use a seqlock to safely copy
> > dev->name into a temporary variable.
>
> So it's preferable to add seqlock around memcpy() in dev_change_name()
> and use a helper in arp_req_get() rather than adding devnet_rename_sem
> locking around memcpy() in arp_req_get() ?
Under rcu_read_lock(), we can not sleep.
devnet_rename_sem is a semaphore... down_read() might sleep.
So if you plan using current netdev_get_name(), you must call it
outside of rcu_read_lock() section.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU.
2024-04-25 20:35 ` Eric Dumazet
@ 2024-04-25 20:47 ` Kuniyuki Iwashima
0 siblings, 0 replies; 11+ messages in thread
From: Kuniyuki Iwashima @ 2024-04-25 20:47 UTC (permalink / raw)
To: edumazet; +Cc: davem, dsahern, kuba, kuni1840, kuniyu, netdev, pabeni
From: Eric Dumazet <edumazet@google.com>
Date: Thu, 25 Apr 2024 22:35:38 +0200
> On Thu, Apr 25, 2024 at 7:52 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> >
> > From: Eric Dumazet <edumazet@google.com>
> > Date: Thu, 25 Apr 2024 19:12:56 +0200
> > > On Thu, Apr 25, 2024 at 7:02 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> > > >
> > > > ioctl(SIOCGARP) holds rtnl_lock() for __dev_get_by_name() and
> > > > later calls neigh_lookup(), which calls rcu_read_lock().
> > > >
> > > > Let's replace __dev_get_by_name() with dev_get_by_name_rcu() to
> > > > avoid locking rtnl_lock().
> > > >
> > > > Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> > > > ---
> > > > net/ipv4/arp.c | 26 +++++++++++++++++---------
> > > > 1 file changed, 17 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
> > > > index 5034920be85a..9430b64558cd 100644
> > > > --- a/net/ipv4/arp.c
> > > > +++ b/net/ipv4/arp.c
> > > > @@ -1003,11 +1003,15 @@ static int arp_rcv(struct sk_buff *skb, struct net_device *dev,
> > > > * User level interface (ioctl)
> > > > */
> > > >
> > > > -static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r)
> > > > +static struct net_device *arp_req_dev_by_name(struct net *net, struct arpreq *r,
> > > > + bool getarp)
> > > > {
> > > > struct net_device *dev;
> > > >
> > > > - dev = __dev_get_by_name(net, r->arp_dev);
> > > > + if (getarp)
> > > > + dev = dev_get_by_name_rcu(net, r->arp_dev);
> > > > + else
> > > > + dev = __dev_get_by_name(net, r->arp_dev);
> > > > if (!dev)
> > > > return ERR_PTR(-ENODEV);
> > > >
> > > > @@ -1028,7 +1032,7 @@ static struct net_device *arp_req_dev(struct net *net, struct arpreq *r)
> > > > __be32 ip;
> > > >
> > > > if (r->arp_dev[0])
> > > > - return arp_req_dev_by_name(net, r);
> > > > + return arp_req_dev_by_name(net, r, false);
> > > >
> > > > if (r->arp_flags & ATF_PUBL)
> > > > return NULL;
> > > > @@ -1166,7 +1170,7 @@ static int arp_req_get(struct net *net, struct arpreq *r)
> > > > if (!r->arp_dev[0])
> > > > return -ENODEV;
> > > >
> > > > - dev = arp_req_dev_by_name(net, r);
> > > > + dev = arp_req_dev_by_name(net, r, true);
> > > > if (IS_ERR(dev))
> > > > return PTR_ERR(dev);
> > > >
> > > > @@ -1287,23 +1291,27 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
> > > > else if (*netmask && *netmask != htonl(0xFFFFFFFFUL))
> > > > return -EINVAL;
> > > >
> > > > - rtnl_lock();
> > > > -
> > > > switch (cmd) {
> > > > case SIOCDARP:
> > > > + rtnl_lock();
> > > > err = arp_req_delete(net, &r);
> > > > + rtnl_unlock();
> > > > break;
> > > > case SIOCSARP:
> > > > + rtnl_lock();
> > > > err = arp_req_set(net, &r);
> > > > + rtnl_unlock();
> > > > break;
> > > > case SIOCGARP:
> > > > + rcu_read_lock();
> > > > err = arp_req_get(net, &r);
> > > > + rcu_read_unlock();
> > >
> > >
> > > Note that arp_req_get() uses :
> > >
> > > strscpy(r->arp_dev, dev->name, sizeof(r->arp_dev));
> > >
> > > This currently depends on RTNL or devnet_rename_sem
> >
> > Ah, I missed this point, thanks for catching!
> >
> >
> > >
> > > Perhaps we should add a helper and use a seqlock to safely copy
> > > dev->name into a temporary variable.
> >
> > So it's preferable to add seqlock around memcpy() in dev_change_name()
> > and use a helper in arp_req_get() rather than adding devnet_rename_sem
> > locking around memcpy() in arp_req_get() ?
>
> Under rcu_read_lock(), we can not sleep.
>
> devnet_rename_sem is a semaphore... down_read() might sleep.
yes... -ENOCOFFEE :)
>
> So if you plan using current netdev_get_name(), you must call it
> outside of rcu_read_lock() section.
>
Will add seqlock helper in v3.
Thanks!
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-04-25 20:47 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-25 16:59 [PATCH v2 net-next 0/6] arp: Random clean up and RCU conversion for ioctl(SIOCGARP) Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 1/6] arp: Move ATF_COM setting in arp_req_set() Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 2/6] arp: Validate netmask earlier for SIOCDARP and SIOCSARP in arp_ioctl() Kuniyuki Iwashima
2024-04-25 16:59 ` [PATCH v2 net-next 3/6] arp: Factorise ip_route_output() call in arp_req_set() and arp_req_delete() Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 4/6] arp: Remove a nest in arp_req_get() Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 5/6] arp: Get dev after calling arp_req_(delete|set|get)() Kuniyuki Iwashima
2024-04-25 17:00 ` [PATCH v2 net-next 6/6] arp: Convert ioctl(SIOCGARP) to RCU Kuniyuki Iwashima
2024-04-25 17:12 ` Eric Dumazet
2024-04-25 17:51 ` Kuniyuki Iwashima
2024-04-25 20:35 ` Eric Dumazet
2024-04-25 20:47 ` Kuniyuki Iwashima
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).