Re: floppy.c soft lockup

[Mark Hounschell <dmarkh@cfl.rr.com>]

Mark Hounschell wrote:
> Oleg Nesterov wrote:
>> On 05/31, Mark Hounschell wrote:
>>> Oleg Nesterov wrote:
>>>> On 05/31, Mark Hounschell wrote:
>>>>> Basically the main RT-process (which is a CPU bound process on processor-2) signals a
>>>>> thread to do some I/O. That RT-thread (running on the other processor) does a simple 
>>>> If the main RT-process monopolizes processor-2, flush_workqueue() (or cancel_work_sync())
>>>> can hang of course, we can do nothing.
>>>>
>>>>> ioctl(Q->DevSpec1, FDSETPRM, &medprm)
>>>>>
>>>>> and there is no return from the call. That thread is hung.
>>>> What happens if you kill the main RT-process?
>>>>
>>> When I kill the main process all its threads also go away. Including the floppy thread.
>>> Nothing notable happens with this kernel.
>> Aha, I missed the word "thread", this is the single process.
>>
>> Still, this means that flush_workqueue() completes when other sub-threads go away,
>> otherwise the thread doing ioctl() couldn't exit.
>>
>> Thank you very much.
>>
>> So, the main question is: is it possible that one of RT processes/threads pins itself
>> to some CPU and eats 100% cpu power?
>>
> 
> The main process is pinned to a processor(2) with all _non-kernel_  processes/threads forced over to processor 1.
> Any already affinitized processes or kernel threads are left as is. Only user land stuff is moved. The main process
> is for sure _not_ relinquishing it's processor(2) intentionally. All the I/O threads, floppy included, are running
> on the other processor(1). During this failure only 1 or 2 of the I/O threads are actually doing anything.
> I assume that what ever is going on in the kernel/floppy driver on behalf of the floppy thread is being done on processor 1? 
> Processor 1 has lots of CPU time available. Processor 2 is running balls to the wall.
>  
>>>                                            On previous (2.6.18) I would get a dump
>>> from the floppy driver in the syslog when I killed the process.
>> Could you send me this output? just in case...
>>
> 
> Today, 2.6.18 is doing the same as 2.6.22-rc3. I hate it when that happens. Maybe it was
> on my box at home. I'll verify when I get there. Nothing from here now though.
> 

Those syslog dumps must have been a result of something I was doing
while trying to pinpoint my problem. I do not get these now. Sorry.

>>>> --- OLD/drivers/block/floppy.c~	2007-04-03 13:04:58.000000000 +0400
>>>> +++ OLD/drivers/block/floppy.c	2007-05-31 20:50:18.000000000 +0400
>>>> @@ -862,6 +862,8 @@ static void set_fdc(int drive)
>>>>  		FDCS->reset = 1;
>>>>  }
>>>>  
>>>> +static DECLARE_WORK(floppy_work, NULL);
>>>> +
>>>>  /* locks the driver */
>>>>  static int _lock_fdc(int drive, int interruptible, int line)
>>>>  {
>>>> @@ -893,7 +895,7 @@ static int _lock_fdc(int drive, int inte
>>>>  		set_current_state(TASK_RUNNING);
>>>>  		remove_wait_queue(&fdc_wait, &wait);
>>>>  
>>>> -		flush_scheduled_work();
>>>> +		cancel_work_sync(&floppy_work);
>>>>  	}
>>>>  	command_status = FD_COMMAND_NONE;
>>>>  
>>>> @@ -992,8 +994,6 @@ static void empty(void)
>>>>  {
>>>>  }
>>>>  
>>>> -static DECLARE_WORK(floppy_work, NULL);
>>>> -
>>>>  static void schedule_bh(void (*handler) (void))
>>>>  {
>>>>  	PREPARE_WORK(&floppy_work, (work_func_t)handler);
>>>>
>>> The patch does make it work.
>> I do not understand floppy.c, absolutely, so I am not sure this patch is correct.
>>
>> Even if correct, this patch doesn't solve this problem (if we really understand
>> what's going on). cancel_work_sync() may still hang if floppy_work->func() runs
>> on the starved CPU. This is unlikely, but possible.
>>
>> Thanks!
>>
>> Oleg.
>>

Thanks and Regards
Mark