From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Wong Subject: Re: epoll: possible bug from wakeup_source activation Date: Sun, 10 Mar 2013 01:11:36 +0000 Message-ID: <20130310011136.GA4815@dcvr.yhbt.net> References: <20130307112639.GA25130@dcvr.yhbt.net> <20130308013027.GA31830@dcvr.yhbt.net> <20130308204944.GA27379@dcvr.yhbt.net> <20130309071037.GA13360@dcvr.yhbt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: NeilBrown , "Rafael J. Wysocki" , linux-kernel@vger.kernel.org, Davide Libenzi , Alexander Viro , linux-fsdevel@vger.kernel.org, Andrew Morton To: Arve =?utf-8?B?SGrDuG5uZXbDpWc=?= Return-path: Content-Disposition: inline In-Reply-To: <20130309071037.GA13360@dcvr.yhbt.net> Sender: linux-kernel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org Eric Wong wrote: > Arve Hj=C3=B8nnev=C3=A5g wrote: > > On Fri, Mar 8, 2013 at 12:49 PM, Eric Wong = wrote: > > > What happens if ep_modify calls ep_destroy_wakeup_source > > > while __pm_stay_awake is running on the same epi->ws? > >=20 > > Yes, that looks like a problem. I think calling > > ep_destroy_wakeup_source with ep->lock held should fix that. It is = not > > clear how useful changing EPOLLWAKEUP in ep_modify is, so > > alternatively we could remove that feature and instead only allow i= t > > to be set in ep_insert. >=20 > ep->lock would work, but ep->lock is already a source of heavy > contention in my multithreaded+epoll webservers. >=20 > Perhaps RCU can be used? I've no experience with RCU, but I've been > meaning to get acquainted with RCU. The following is lightly tested, at least I haven't gotten lockdep to complain. --------------------------------- 8< ---------------------------- =46rom 2bcd549893aa204bd858cc1500a70f20b28e47c1 Mon Sep 17 00:00:00 200= 1 =46rom: Eric Wong Date: Sun, 10 Mar 2013 01:06:50 +0000 Subject: [PATCH] epoll: RCU protect wakeup_source in epitem This prevents wakeup_source destruction when a user hits the item with EPOLL_CTL_MOD while ep_poll_callback is running. Signed-off-by: Eric Wong --- fs/eventpoll.c | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 9fec183..e008d54 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -158,7 +158,7 @@ struct epitem { struct list_head fllink; =20 /* wakeup_source used when EPOLLWAKEUP is set */ - struct wakeup_source *ws; + struct wakeup_source __rcu *ws; =20 /* The structure that describe the interested events and the source f= d */ struct epoll_event event; @@ -536,6 +536,17 @@ static void ep_unregister_pollwait(struct eventpol= l *ep, struct epitem *epi) } } =20 +static inline void ep_pm_stay_awake(struct epitem *epi) +{ + struct wakeup_source *ws; + + rcu_read_lock(); + ws =3D rcu_dereference(epi->ws); + if (ws) + __pm_stay_awake(ws); + rcu_read_unlock(); +} + /** * ep_scan_ready_list - Scans the ready list in a way that makes possi= ble for * the scan code, to call f_op->poll(). Also allo= ws for @@ -984,7 +995,7 @@ static int ep_poll_callback(wait_queue_t *wait, uns= igned mode, int sync, void *k /* If this file is already in the ready list we exit soon */ if (!ep_is_linked(&epi->rdllink)) { list_add_tail(&epi->rdllink, &ep->rdllist); - __pm_stay_awake(epi->ws); + ep_pm_stay_awake(epi); } =20 /* @@ -1146,6 +1157,7 @@ static int reverse_path_check(void) static int ep_create_wakeup_source(struct epitem *epi) { const char *name; + struct wakeup_source *ws; =20 if (!epi->ep->ws) { epi->ep->ws =3D wakeup_source_register("eventpoll"); @@ -1154,17 +1166,22 @@ static int ep_create_wakeup_source(struct epite= m *epi) } =20 name =3D epi->ffd.file->f_path.dentry->d_name.name; - epi->ws =3D wakeup_source_register(name); - if (!epi->ws) + ws =3D wakeup_source_register(name); + + if (!ws) return -ENOMEM; + rcu_assign_pointer(epi->ws, ws); =20 return 0; } =20 static void ep_destroy_wakeup_source(struct epitem *epi) { - wakeup_source_unregister(epi->ws); - epi->ws =3D NULL; + struct wakeup_source *ws =3D epi->ws; + + rcu_assign_pointer(epi->ws, NULL); + synchronize_rcu(); /* wait for ep_pm_stay_awake to finish */ + wakeup_source_unregister(ws); } =20 /* @@ -1199,7 +1216,7 @@ static int ep_insert(struct eventpoll *ep, struct= epoll_event *event, if (error) goto error_create_wakeup_source; } else { - epi->ws =3D NULL; + RCU_INIT_POINTER(epi->ws, NULL); } =20 /* Initialize the poll table using the queue callback */ --=20 Eric Wong