Re: floppy.c soft lockup

[Mark Hounschell <markh@compro.net>]

Oleg Nesterov wrote:
> On 05/31, Mark Hounschell wrote:
>> Oleg Nesterov wrote:
>>> On 05/31, Mark Hounschell wrote:
>>>> Basically the main RT-process (which is a CPU bound process on processor-2) signals a
>>>> thread to do some I/O. That RT-thread (running on the other processor) does a simple 
>>> If the main RT-process monopolizes processor-2, flush_workqueue() (or cancel_work_sync())
>>> can hang of course, we can do nothing.
>>>
>>>> ioctl(Q->DevSpec1, FDSETPRM, &medprm)
>>>>
>>>> and there is no return from the call. That thread is hung.
>>> What happens if you kill the main RT-process?
>>>
>> When I kill the main process all its threads also go away. Including the floppy thread.
>> Nothing notable happens with this kernel.
> 
> Aha, I missed the word "thread", this is the single process.
> 
> Still, this means that flush_workqueue() completes when other sub-threads go away,
> otherwise the thread doing ioctl() couldn't exit.
> 
> Thank you very much.
> 
> So, the main question is: is it possible that one of RT processes/threads pins itself
> to some CPU and eats 100% cpu power?
> 

The main process is pinned to a processor(2) with all _non-kernel_  processes/threads forced over to processor 1.
Any already affinitized processes or kernel threads are left as is. Only user land stuff is moved. The main process
is for sure _not_ relinquishing it's processor(2) intentionally. All the I/O threads, floppy included, are running
on the other processor(1). During this failure only 1 or 2 of the I/O threads are actually doing anything.
I assume that what ever is going on in the kernel/floppy driver on behalf of the floppy thread is being done on processor 1? 
Processor 1 has lots of CPU time available. Processor 2 is running balls to the wall.
 
>>                                            On previous (2.6.18) I would get a dump
>> from the floppy driver in the syslog when I killed the process.
> 
> Could you send me this output? just in case...
> 

Today, 2.6.18 is doing the same as 2.6.22-rc3. I hate it when that happens. Maybe it was
on my box at home. I'll verify when I get there. Nothing from here now though.

>>> --- OLD/drivers/block/floppy.c~	2007-04-03 13:04:58.000000000 +0400
>>> +++ OLD/drivers/block/floppy.c	2007-05-31 20:50:18.000000000 +0400
>>> @@ -862,6 +862,8 @@ static void set_fdc(int drive)
>>>  		FDCS->reset = 1;
>>>  }
>>>  
>>> +static DECLARE_WORK(floppy_work, NULL);
>>> +
>>>  /* locks the driver */
>>>  static int _lock_fdc(int drive, int interruptible, int line)
>>>  {
>>> @@ -893,7 +895,7 @@ static int _lock_fdc(int drive, int inte
>>>  		set_current_state(TASK_RUNNING);
>>>  		remove_wait_queue(&fdc_wait, &wait);
>>>  
>>> -		flush_scheduled_work();
>>> +		cancel_work_sync(&floppy_work);
>>>  	}
>>>  	command_status = FD_COMMAND_NONE;
>>>  
>>> @@ -992,8 +994,6 @@ static void empty(void)
>>>  {
>>>  }
>>>  
>>> -static DECLARE_WORK(floppy_work, NULL);
>>> -
>>>  static void schedule_bh(void (*handler) (void))
>>>  {
>>>  	PREPARE_WORK(&floppy_work, (work_func_t)handler);
>>>
>> The patch does make it work.
> 
> I do not understand floppy.c, absolutely, so I am not sure this patch is correct.
> 
> Even if correct, this patch doesn't solve this problem (if we really understand
> what's going on). cancel_work_sync() may still hang if floppy_work->func() runs
> on the starved CPU. This is unlikely, but possible.
> 
> Thanks!
> 
> Oleg.
>