* [PATCH v2] n_gsm: race between ld close and gsmtty open
@ 2013-10-17 7:08 channing
0 siblings, 0 replies; 6+ messages in thread
From: channing @ 2013-10-17 7:08 UTC (permalink / raw)
To: gregkh, jslaby; +Cc: linux-kernel
ttyA has ld associated to n_gsm, when ttyA is closing, it triggers
to release gsmttyB's ld data dlci[B], then race would happen if gsmttyB
is opening in parallel.
Here are race cases we found recently in test:
CASE #1
====================================================================
releasing dlci[B] race with gsmtty_install(gsmttyB), then panic
in gsmtty_open(gsmttyB), as below:
tty_release(ttyA) tty_open(gsmttyB)
| |
----- gsmtty_install(gsmttyB)
| |
----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
tty_ldisc_release(ttyA) -----
| |
gsm_dlci_release(dlci[B]) -----
| |
gsm_dlci_free(dlci[B]) -----
| |
----- gsmtty_open(gsmttyB)
gsmtty_open()
{
struct gsm_dlci *dlci = tty->driver_data; => here it uses dlci[B]
...
}
In gsmtty_open(gsmttyA), it uses dlci[B] which was release, so hit a panic.
=====================================================================
CASE #2
=====================================================================
releasing dlci[0] race with gsmtty_install(gsmttyB), then panic
in gsmtty_open(), as below:
tty_release(ttyA) tty_open(gsmttyB)
| |
----- gsmtty_install(gsmttyB)
| |
----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
| |
----- gsmtty_open(gsmttyB) fail
| |
----- tty_release(gsmttyB)
| |
----- gsmtty_close(gsmttyB)
| |
----- gsmtty_detach_dlci(dlci[B])
| |
----- dlci_put(dlci[B])
| |
tty_ldisc_release(ttyA) -----
| |
gsm_dlci_release(dlci[0]) -----
| |
gsm_dlci_free(dlci[0]) -----
| |
----- dlci_put(dlci[0])
In gsmtty_detach_dlci(dlci[B]), it tries to use dlci[0] which was released,
then hit panic.
=====================================================================
IMHO, n_gsm tty operations would refer released ldisc, as long as
gsm_dlci_release() has chance to release ldisc data when some gsmtty operations
are not completed..
This patch is try to avoid it by:
1) in n_gsm driver, use a global gsm spin lock to avoid gsm_dlci_release() run in
parallel with gsmtty_install();
2) Increase dlci's ref count in gsmtty_install() instead of in gsmtty_open(), the
purpose is to prevent gsm_dlci_release() releasing dlci after gsmtty_install()
allocats dlci but before gsmtty_open increases dlci's ref count;
3) Decrease dlci's ref count in gsmtty_remove(), which is a tty framework api, and
this is the opposite process of step 2).
Signed-off-by: Chao Bi <chao.bi@intel.com>
---
drivers/tty/n_gsm.c | 37 +++++++++++++++++++++++++++----------
1 files changed, 27 insertions(+), 10 deletions(-)
diff --git a/drivers/tty/n_gsm.c b/drivers/tty/n_gsm.c
index c0f76da..069bfd6 100644
--- a/drivers/tty/n_gsm.c
+++ b/drivers/tty/n_gsm.c
@@ -2054,9 +2054,11 @@ void gsm_cleanup_mux(struct gsm_mux *gsm)
dlci->state == DLCI_CLOSED);
}
/* Free up any link layer users */
+ spin_lock(&gsm->lock);
for (i = 0; i < NUM_DLCI; i++)
if (gsm->dlci[i])
gsm_dlci_release(gsm->dlci[i]);
+ spin_unlock(&gsm->lock);
/* Now wipe the queues */
list_for_each_entry_safe(txq, ntxq, &gsm->tx_list, list)
kfree(txq);
@@ -2909,23 +2911,33 @@ static int gsmtty_install(struct tty_driver *driver, struct tty_struct *tty)
This is ok from a locking
perspective as we don't have to worry about this
if DLCI0 is lost */
- if (gsm->dlci[0] && gsm->dlci[0]->state != DLCI_OPEN)
+ spin_lock(&gsm->lock);
+ if (gsm->dlci[0] && gsm->dlci[0]->state != DLCI_OPEN) {
+ spin_unlock(&gsm->lock);
return -EL2NSYNC;
+ }
dlci = gsm->dlci[line];
if (dlci == NULL) {
alloc = true;
dlci = gsm_dlci_alloc(gsm, line);
}
- if (dlci == NULL)
+ if (dlci == NULL) {
+ spin_unlock(&gsm->lock);
return -ENOMEM;
+ }
ret = tty_port_install(&dlci->port, driver, tty);
if (ret) {
if (alloc)
dlci_put(dlci);
+ spin_unlock(&gsm->lock);
return ret;
}
+ dlci_get(dlci);
+ dlci_get(gsm->dlci[0]);
+ mux_get(gsm);
tty->driver_data = dlci;
+ spin_unlock(&gsm->lock);
return 0;
}
@@ -2936,9 +2948,6 @@ static int gsmtty_open(struct tty_struct *tty, struct file *filp)
struct tty_port *port = &dlci->port;
port->count++;
- dlci_get(dlci);
- dlci_get(dlci->gsm->dlci[0]);
- mux_get(dlci->gsm);
tty_port_tty_set(port, tty);
dlci->modem_rx = 0;
@@ -2965,7 +2974,7 @@ static void gsmtty_close(struct tty_struct *tty, struct file *filp)
mutex_unlock(&dlci->mutex);
gsm = dlci->gsm;
if (tty_port_close_start(&dlci->port, tty, filp) == 0)
- goto out;
+ return;
gsm_dlci_begin_close(dlci);
if (test_bit(ASYNCB_INITIALIZED, &dlci->port.flags)) {
if (C_HUPCL(tty))
@@ -2973,10 +2982,7 @@ static void gsmtty_close(struct tty_struct *tty, struct file *filp)
}
tty_port_close_end(&dlci->port, tty);
tty_port_tty_set(&dlci->port, NULL);
-out:
- dlci_put(dlci);
- dlci_put(gsm->dlci[0]);
- mux_put(gsm);
+ return;
}
static void gsmtty_hangup(struct tty_struct *tty)
@@ -3153,6 +3159,16 @@ static int gsmtty_break_ctl(struct tty_struct *tty, int state)
return gsmtty_modem_update(dlci, encode);
}
+static void gsmtty_remove(struct tty_driver *driver, struct tty_struct *tty)
+{
+ struct gsm_dlci *dlci = tty->driver_data;
+ struct gsm_mux *gsm = dlci->gsm;
+
+ dlci_put(dlci);
+ dlci_put(gsm->dlci[0]);
+ mux_put(gsm);
+ driver->ttys[tty->index] = NULL;
+}
/* Virtual ttys for the demux */
static const struct tty_operations gsmtty_ops = {
@@ -3172,6 +3188,7 @@ static const struct tty_operations gsmtty_ops = {
.tiocmget = gsmtty_tiocmget,
.tiocmset = gsmtty_tiocmset,
.break_ctl = gsmtty_break_ctl,
+ .remove = gsmtty_remove,
};
--
1.7.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH V2] n_gsm: race between ld close and gsmtty open
2013-11-26 3:14 [PATCH V2] n_gsm: race between ld close and gsmtty open channing
@ 2013-11-26 2:54 ` Greg KH
2013-11-26 3:35 ` channing
0 siblings, 1 reply; 6+ messages in thread
From: Greg KH @ 2013-11-26 2:54 UTC (permalink / raw)
To: channing; +Cc: linux-kernel, fengguang.wu
On Tue, Nov 26, 2013 at 11:14:05AM +0800, channing wrote:
>
> ttyA has ld associated to n_gsm, when ttyA is closing, it triggers
> to release gsmttyB's ld data dlci[B], then race would happen if gsmttyB
> is opening in parallel.
>
> Here are race cases we found recently in test:
>
> CASE #1
> ====================================================================
> releasing dlci[B] race with gsmtty_install(gsmttyB), then panic
> in gsmtty_open(gsmttyB), as below:
>
> tty_release(ttyA) tty_open(gsmttyB)
> | |
> ----- gsmtty_install(gsmttyB)
> | |
> ----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
> tty_ldisc_release(ttyA) -----
> | |
> gsm_dlci_release(dlci[B]) -----
> | |
> gsm_dlci_free(dlci[B]) -----
> | |
> ----- gsmtty_open(gsmttyB)
>
> gsmtty_open()
> {
> struct gsm_dlci *dlci = tty->driver_data; => here it uses dlci[B]
> ...
> }
>
> In gsmtty_open(gsmttyA), it uses dlci[B] which was release, so hit a panic.
> =====================================================================
>
> CASE #2
> =====================================================================
> releasing dlci[0] race with gsmtty_install(gsmttyB), then panic
> in gsmtty_open(), as below:
>
> tty_release(ttyA) tty_open(gsmttyB)
> | |
> ----- gsmtty_install(gsmttyB)
> | |
> ----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
> | |
> ----- gsmtty_open(gsmttyB) fail
> | |
> ----- tty_release(gsmttyB)
> | |
> ----- gsmtty_close(gsmttyB)
> | |
> ----- gsmtty_detach_dlci(dlci[B])
> | |
> ----- dlci_put(dlci[B])
> | |
> tty_ldisc_release(ttyA) -----
> | |
> gsm_dlci_release(dlci[0]) -----
> | |
> gsm_dlci_free(dlci[0]) -----
> | |
> ----- dlci_put(dlci[0])
>
> In gsmtty_detach_dlci(dlci[B]), it tries to use dlci[0] which was released,
> then hit panic.
> =====================================================================
>
> IMHO, n_gsm tty operations would refer released ldisc, as long as
> gsm_dlci_release() has chance to release ldisc data when some gsmtty operations
> are ongoing..
>
> This patch is try to avoid it by:
>
> 1) in n_gsm driver, use a global gsm mutex lock to avoid gsm_dlci_release() run in
> parallel with gsmtty_install();
>
> 2) Increase dlci's ref count in gsmtty_install() instead of in gsmtty_open(), the
> purpose is to prevent gsm_dlci_release() releasing dlci after gsmtty_install()
> allocats dlci but before gsmtty_open increases dlci's ref count;
>
> 3) Decrease dlci's ref count in gsmtty_remove(), a tty framework API, this is the
> opposite process of step 2).
>
> Signed-off-by: Chao Bi <chao.bi@intel.com>
> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
I have not signed off on this additional patch.
What is different from the previous version? That information needs to
be somewhere, otherwise I'm just going to guess and say this is the same
as your last one, which was incorrect.
Also, please fix your "From:" line in your email client to match your
Signed-off-by: line, or else add the proper "From:" line to your patch,
as the Documentation/SubmittingPatches file says.
Care to try again?
greg k-h
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH V2] n_gsm: race between ld close and gsmtty open
@ 2013-11-26 3:14 channing
2013-11-26 2:54 ` Greg KH
0 siblings, 1 reply; 6+ messages in thread
From: channing @ 2013-11-26 3:14 UTC (permalink / raw)
To: gregkh; +Cc: linux-kernel, fengguang.wu
ttyA has ld associated to n_gsm, when ttyA is closing, it triggers
to release gsmttyB's ld data dlci[B], then race would happen if gsmttyB
is opening in parallel.
Here are race cases we found recently in test:
CASE #1
====================================================================
releasing dlci[B] race with gsmtty_install(gsmttyB), then panic
in gsmtty_open(gsmttyB), as below:
tty_release(ttyA) tty_open(gsmttyB)
| |
----- gsmtty_install(gsmttyB)
| |
----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
tty_ldisc_release(ttyA) -----
| |
gsm_dlci_release(dlci[B]) -----
| |
gsm_dlci_free(dlci[B]) -----
| |
----- gsmtty_open(gsmttyB)
gsmtty_open()
{
struct gsm_dlci *dlci = tty->driver_data; => here it uses dlci[B]
...
}
In gsmtty_open(gsmttyA), it uses dlci[B] which was release, so hit a panic.
=====================================================================
CASE #2
=====================================================================
releasing dlci[0] race with gsmtty_install(gsmttyB), then panic
in gsmtty_open(), as below:
tty_release(ttyA) tty_open(gsmttyB)
| |
----- gsmtty_install(gsmttyB)
| |
----- gsm_dlci_alloc(gsmttyB) => alloc dlci[B]
| |
----- gsmtty_open(gsmttyB) fail
| |
----- tty_release(gsmttyB)
| |
----- gsmtty_close(gsmttyB)
| |
----- gsmtty_detach_dlci(dlci[B])
| |
----- dlci_put(dlci[B])
| |
tty_ldisc_release(ttyA) -----
| |
gsm_dlci_release(dlci[0]) -----
| |
gsm_dlci_free(dlci[0]) -----
| |
----- dlci_put(dlci[0])
In gsmtty_detach_dlci(dlci[B]), it tries to use dlci[0] which was released,
then hit panic.
=====================================================================
IMHO, n_gsm tty operations would refer released ldisc, as long as
gsm_dlci_release() has chance to release ldisc data when some gsmtty operations
are ongoing..
This patch is try to avoid it by:
1) in n_gsm driver, use a global gsm mutex lock to avoid gsm_dlci_release() run in
parallel with gsmtty_install();
2) Increase dlci's ref count in gsmtty_install() instead of in gsmtty_open(), the
purpose is to prevent gsm_dlci_release() releasing dlci after gsmtty_install()
allocats dlci but before gsmtty_open increases dlci's ref count;
3) Decrease dlci's ref count in gsmtty_remove(), a tty framework API, this is the
opposite process of step 2).
Signed-off-by: Chao Bi <chao.bi@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/tty/n_gsm.c | 39 +++++++++++++++++++++++++++++----------
1 file changed, 29 insertions(+), 10 deletions(-)
diff --git a/drivers/tty/n_gsm.c b/drivers/tty/n_gsm.c
index c0f76da..d514396 100644
--- a/drivers/tty/n_gsm.c
+++ b/drivers/tty/n_gsm.c
@@ -194,6 +194,7 @@ struct gsm_control {
struct gsm_mux {
struct tty_struct *tty; /* The tty our ldisc is bound to */
spinlock_t lock;
+ struct mutex mutex;
unsigned int num;
struct kref ref;
@@ -2054,9 +2055,11 @@ void gsm_cleanup_mux(struct gsm_mux *gsm)
dlci->state == DLCI_CLOSED);
}
/* Free up any link layer users */
+ mutex_lock(&gsm->mutex);
for (i = 0; i < NUM_DLCI; i++)
if (gsm->dlci[i])
gsm_dlci_release(gsm->dlci[i]);
+ mutex_unlock(&gsm->mutex);
/* Now wipe the queues */
list_for_each_entry_safe(txq, ntxq, &gsm->tx_list, list)
kfree(txq);
@@ -2170,6 +2173,7 @@ struct gsm_mux *gsm_alloc_mux(void)
return NULL;
}
spin_lock_init(&gsm->lock);
+ mutex_init(&gsm->mutex);
kref_init(&gsm->ref);
INIT_LIST_HEAD(&gsm->tx_list);
@@ -2909,23 +2913,33 @@ static int gsmtty_install(struct tty_driver *driver, struct tty_struct *tty)
This is ok from a locking
perspective as we don't have to worry about this
if DLCI0 is lost */
- if (gsm->dlci[0] && gsm->dlci[0]->state != DLCI_OPEN)
+ mutex_lock(&gsm->mutex);
+ if (gsm->dlci[0] && gsm->dlci[0]->state != DLCI_OPEN) {
+ mutex_unlock(&gsm->mutex);
return -EL2NSYNC;
+ }
dlci = gsm->dlci[line];
if (dlci == NULL) {
alloc = true;
dlci = gsm_dlci_alloc(gsm, line);
}
- if (dlci == NULL)
+ if (dlci == NULL) {
+ mutex_unlock(&gsm->mutex);
return -ENOMEM;
+ }
ret = tty_port_install(&dlci->port, driver, tty);
if (ret) {
if (alloc)
dlci_put(dlci);
+ mutex_unlock(&gsm->mutex);
return ret;
}
+ dlci_get(dlci);
+ dlci_get(gsm->dlci[0]);
+ mux_get(gsm);
tty->driver_data = dlci;
+ mutex_unlock(&gsm->mutex);
return 0;
}
@@ -2936,9 +2950,6 @@ static int gsmtty_open(struct tty_struct *tty, struct file *filp)
struct tty_port *port = &dlci->port;
port->count++;
- dlci_get(dlci);
- dlci_get(dlci->gsm->dlci[0]);
- mux_get(dlci->gsm);
tty_port_tty_set(port, tty);
dlci->modem_rx = 0;
@@ -2965,7 +2976,7 @@ static void gsmtty_close(struct tty_struct *tty, struct file *filp)
mutex_unlock(&dlci->mutex);
gsm = dlci->gsm;
if (tty_port_close_start(&dlci->port, tty, filp) == 0)
- goto out;
+ return;
gsm_dlci_begin_close(dlci);
if (test_bit(ASYNCB_INITIALIZED, &dlci->port.flags)) {
if (C_HUPCL(tty))
@@ -2973,10 +2984,7 @@ static void gsmtty_close(struct tty_struct *tty, struct file *filp)
}
tty_port_close_end(&dlci->port, tty);
tty_port_tty_set(&dlci->port, NULL);
-out:
- dlci_put(dlci);
- dlci_put(gsm->dlci[0]);
- mux_put(gsm);
+ return;
}
static void gsmtty_hangup(struct tty_struct *tty)
@@ -3153,6 +3161,16 @@ static int gsmtty_break_ctl(struct tty_struct *tty, int state)
return gsmtty_modem_update(dlci, encode);
}
+static void gsmtty_remove(struct tty_driver *driver, struct tty_struct *tty)
+{
+ struct gsm_dlci *dlci = tty->driver_data;
+ struct gsm_mux *gsm = dlci->gsm;
+
+ dlci_put(dlci);
+ dlci_put(gsm->dlci[0]);
+ mux_put(gsm);
+ driver->ttys[tty->index] = NULL;
+}
/* Virtual ttys for the demux */
static const struct tty_operations gsmtty_ops = {
@@ -3172,6 +3190,7 @@ static const struct tty_operations gsmtty_ops = {
.tiocmget = gsmtty_tiocmget,
.tiocmset = gsmtty_tiocmset,
.break_ctl = gsmtty_break_ctl,
+ .remove = gsmtty_remove,
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH V2] n_gsm: race between ld close and gsmtty open
2013-11-26 3:35 ` channing
@ 2013-11-26 3:16 ` Greg KH
2013-11-26 3:53 ` channing
0 siblings, 1 reply; 6+ messages in thread
From: Greg KH @ 2013-11-26 3:16 UTC (permalink / raw)
To: channing; +Cc: linux-kernel, fengguang.wu
On Tue, Nov 26, 2013 at 11:35:14AM +0800, channing wrote:
> On Mon, 2013-11-25 at 18:54 -0800, Greg KH wrote:
> > On Tue, Nov 26, 2013 at 11:14:05AM +0800, channing wrote:
>
> > > This patch is try to avoid it by:
> > >
> > > 1) in n_gsm driver, use a global gsm mutex lock to avoid gsm_dlci_release() run in
> > > parallel with gsmtty_install();
> The commit is updated here than formal patch set: we use mutex lock in
> patch V2, while use spin lock in patch V1.
>
> > >
> > > 2) Increase dlci's ref count in gsmtty_install() instead of in gsmtty_open(), the
> > > purpose is to prevent gsm_dlci_release() releasing dlci after gsmtty_install()
> > > allocats dlci but before gsmtty_open increases dlci's ref count;
> > >
> > > 3) Decrease dlci's ref count in gsmtty_remove(), a tty framework API, this is the
> > > opposite process of step 2).
> > >
> > > Signed-off-by: Chao Bi <chao.bi@intel.com>
> > > Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > I have not signed off on this additional patch.
> >
> > What is different from the previous version? That information needs to
> > be somewhere, otherwise I'm just going to guess and say this is the same
> > as your last one, which was incorrect.
> The difference with previous one is to use a mutex instead of spin lock
> to avoid race, purpose is to avoid sleep in atomic context. I've also
> updated commit a little as above.
Then be explicit as to what has changed somewhere. We deal with
thousands of patches a week, we can not know that you changed one
sentance in a patch description of a few hundred lines long to know you
made a change to the patch itself as well...
thanks,
greg k-h
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] n_gsm: race between ld close and gsmtty open
2013-11-26 2:54 ` Greg KH
@ 2013-11-26 3:35 ` channing
2013-11-26 3:16 ` Greg KH
0 siblings, 1 reply; 6+ messages in thread
From: channing @ 2013-11-26 3:35 UTC (permalink / raw)
To: Greg KH; +Cc: linux-kernel, fengguang.wu
On Mon, 2013-11-25 at 18:54 -0800, Greg KH wrote:
> On Tue, Nov 26, 2013 at 11:14:05AM +0800, channing wrote:
> > This patch is try to avoid it by:
> >
> > 1) in n_gsm driver, use a global gsm mutex lock to avoid gsm_dlci_release() run in
> > parallel with gsmtty_install();
The commit is updated here than formal patch set: we use mutex lock in
patch V2, while use spin lock in patch V1.
> >
> > 2) Increase dlci's ref count in gsmtty_install() instead of in gsmtty_open(), the
> > purpose is to prevent gsm_dlci_release() releasing dlci after gsmtty_install()
> > allocats dlci but before gsmtty_open increases dlci's ref count;
> >
> > 3) Decrease dlci's ref count in gsmtty_remove(), a tty framework API, this is the
> > opposite process of step 2).
> >
> > Signed-off-by: Chao Bi <chao.bi@intel.com>
> > Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>
> I have not signed off on this additional patch.
>
> What is different from the previous version? That information needs to
> be somewhere, otherwise I'm just going to guess and say this is the same
> as your last one, which was incorrect.
The difference with previous one is to use a mutex instead of spin lock
to avoid race, purpose is to avoid sleep in atomic context. I've also
updated commit a little as above.
>
> Also, please fix your "From:" line in your email client to match your
> Signed-off-by: line, or else add the proper "From:" line to your patch,
> as the Documentation/SubmittingPatches file says.
>
> Care to try again?
Yes, I'll correct it. thanks.
>
> greg k-h
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH V2] n_gsm: race between ld close and gsmtty open
2013-11-26 3:16 ` Greg KH
@ 2013-11-26 3:53 ` channing
0 siblings, 0 replies; 6+ messages in thread
From: channing @ 2013-11-26 3:53 UTC (permalink / raw)
To: Greg KH; +Cc: linux-kernel, fengguang.wu
On Mon, 2013-11-25 at 19:16 -0800, Greg KH wrote:
> > >
> > > What is different from the previous version? That information needs to
> > > be somewhere, otherwise I'm just going to guess and say this is the same
> > > as your last one, which was incorrect.
> > The difference with previous one is to use a mutex instead of spin lock
> > to avoid race, purpose is to avoid sleep in atomic context. I've also
> > updated commit a little as above.
>
> Then be explicit as to what has changed somewhere. We deal with
> thousands of patches a week, we can not know that you changed one
> sentance in a patch description of a few hundred lines long to know you
> made a change to the patch itself as well...
OK, in next patch set, I'll highlight difference with previous patch
set.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-11-26 3:24 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-11-26 3:14 [PATCH V2] n_gsm: race between ld close and gsmtty open channing
2013-11-26 2:54 ` Greg KH
2013-11-26 3:35 ` channing
2013-11-26 3:16 ` Greg KH
2013-11-26 3:53 ` channing
-- strict thread matches above, loose matches on Subject: below --
2013-10-17 7:08 [PATCH v2] " channing
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox