[PATCH v4 1/7] char: xillybus: Improve control of execution flow with mutexes

[Eli Billauer <eli.billauer@gmail.com>]

This commit addresses two issues by using mutexes:

(1) Add a mutex to protect the fifo_buf_order global variable. The
    purpose of this variable is avoid repeated failed calls to
    __get_free_pages() for allocating FIFO memory, when the chunk
    size was too big. However, if two drivers are initialized at the
    same time, fifo_init() may run in parallel, and fifo_buf_order
    may be reduced too much. This is a far-fetched scenario, now
    completely prevented by fifo_buf_order_mutex.

(2) setup_channels() acquires process_in_mutex to prevent
    process_bulk_in() from accessing the xillyusb_dev struct. With
    correctly working hardware, process_bulk_in() is never called while
    setup_channels() runs, because the device has no reason to send data
    in that phase. The mutex ensures that process_bulk_in() does not
    touch the members that setup_channels() alters.

    There is no similar protection for data flow in the other direction,
    because during the setup process, the only outbound data is the
    BULK endpoint used for commands, and it remains untouched after its
    initial setup.

Assisted-by: Deepseek:v4-pro Kimi:K2.6 ChatGPT:GPT-5.5 Claude:Sonnet-4.6
Assisted-by: Sashiko-0.2.5:gemini-3.1-pro-preview
Signed-off-by: Eli Billauer <eli.billauer@gmail.com>
---

Notes:
    Changelog:
    =========
    
    Changes v3->v4:
     -- Use plain mutex_lock() in fifo_init() rather than guard() in order
        to avoid mixing guard() with goto, following Sashiko's remark + add
        attribution to Sashiko for this.
    
    Changes v2->v3:
     -- Add Assisted-by tag to description
    
    No change on v1->v2.

 drivers/char/xillybus/xillyusb.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/char/xillybus/xillyusb.c b/drivers/char/xillybus/xillyusb.c
index 34e7ad3bcab3..560c3568fc96 100644
--- a/drivers/char/xillybus/xillyusb.c
+++ b/drivers/char/xillybus/xillyusb.c
@@ -50,6 +50,7 @@ MODULE_LICENSE("GPL v2");
 static const char xillyname[] = "xillyusb";
 
 static unsigned int fifo_buf_order;
+static DEFINE_MUTEX(fifo_buf_order_mutex);
 static struct workqueue_struct *wakeup_wq;
 
 #define USB_VENDOR_ID_XILINX		0x03fd
@@ -375,6 +376,8 @@ static int fifo_init(struct xillyfifo *fifo,
 
 	unsigned int log2_fifo_buf_size;
 
+	mutex_lock(&fifo_buf_order_mutex);
+
 retry:
 	log2_fifo_buf_size = fifo_buf_order + PAGE_SHIFT;
 
@@ -395,8 +398,10 @@ static int fifo_init(struct xillyfifo *fifo,
 
 	fifo->mem = kmalloc_array(fifo->bufnum, sizeof(void *), GFP_KERNEL);
 
-	if (!fifo->mem)
+	if (!fifo->mem) {
+		mutex_unlock(&fifo_buf_order_mutex);
 		return -ENOMEM;
+	}
 
 	for (i = 0; i < fifo->bufnum; i++) {
 		fifo->mem[i] = (void *)
@@ -413,6 +418,8 @@ static int fifo_init(struct xillyfifo *fifo,
 	fifo->writebuf = 0;
 	spin_lock_init(&fifo->lock);
 	init_waitqueue_head(&fifo->waitq);
+
+	mutex_unlock(&fifo_buf_order_mutex);
 	return 0;
 
 memfail:
@@ -426,6 +433,7 @@ static int fifo_init(struct xillyfifo *fifo,
 		fifo_buf_order--;
 		goto retry;
 	} else {
+		mutex_unlock(&fifo_buf_order_mutex);
 		return -ENOMEM;
 	}
 }
@@ -1943,6 +1951,9 @@ static int setup_channels(struct xillyusb_dev *xdev,
 	struct xillyusb_channel *chan, *new_channels;
 	int i;
 
+	/* Don't let process_bulk_in() run while we change the channels */
+	guard(mutex)(&xdev->process_in_mutex);
+
 	chan = kzalloc_objs(*chan, num_channels);
 	if (!chan)
 		return -ENOMEM;
-- 
2.34.1