Block device driver: how to terminate the block device if media disappears?

Block device driver: how to terminate the block device if media disappears?

[John Beard]

Hi,

I have block driver for a hot-pluggable device PCIe storage device on
kernel version 2.6.43 (which I know isn't a mainline kernel version, but
it's what I am required to build against). The driver itself is
relatively simple and implements the following block operations:

	static struct block_device_operations mydev_ops = {
        	.owner = THIS_MODULE,
        	.open = mydev_open,
        	.release = mydev_release,
        	.getgeo = mydev_getgeo,
        	.ioctl = mydev_ioctl
	};

and the following PCI driver structure:

	static struct pci_driver mydev_driver = {
		.name = DRIVER_NAME,
        	.probe = mydev_pci_init_one,
        	.remove = mydev_pci_remove_one,
        	.id_table = mydev_pci_tbl,
	};

As well as a request_fn with the following signature:

	static void mydev_submit_req(struct request_queue *q);

Whenever I get IO requests, there is the expected pattern of "open, IO,
release", and everything works OK.

However, if the device is physically removed during IO, I never seem to
get a "release", just "open, IO, hang". I believe (but I don't know),
that this is preventing del_gendisk() from completing, thus hanging the
cleanup of the driver, which is triggered by mydev_pci_remove_one() upon
the removal of the device.

I am ending all requests on the queue once an eject happens, but it
still doesn't seem to cause a release.

What is the right way to terminate requests and delete the gendisk in
the case of physically vanished PCI devices (or even devices in general)?

Thanks in advance for any pointers to a solution and apologies to the
IRC folks for annoying them with the same question!

John Beard

Block device driver: how to terminate the block device if media disappears?

[Ezequiel Garcia]

Hi John,

On Thu, Dec 13, 2012 at 12:31 PM, John Beard <johnb@codexdigital.com> wrote:
> Hi,
>
> I have block driver for a hot-pluggable device PCIe storage device on
> kernel version 2.6.43 (which I know isn't a mainline kernel version, but
> it's what I am required to build against). The driver itself is
> relatively simple and implements the following block operations:
>
>         static struct block_device_operations mydev_ops = {
>                 .owner = THIS_MODULE,
>                 .open = mydev_open,
>                 .release = mydev_release,
>                 .getgeo = mydev_getgeo,
>                 .ioctl = mydev_ioctl
>         };
>
> and the following PCI driver structure:
>
>         static struct pci_driver mydev_driver = {
>                 .name = DRIVER_NAME,
>                 .probe = mydev_pci_init_one,
>                 .remove = mydev_pci_remove_one,
>                 .id_table = mydev_pci_tbl,
>         };
>
> As well as a request_fn with the following signature:
>
>         static void mydev_submit_req(struct request_queue *q);
>
> Whenever I get IO requests, there is the expected pattern of "open, IO,
> release", and everything works OK.
>
> However, if the device is physically removed during IO, I never seem to
> get a "release", just "open, IO, hang". I believe (but I don't know),
> that this is preventing del_gendisk() from completing, thus hanging the
> cleanup of the driver, which is triggered by mydev_pci_remove_one() upon
> the removal of the device.
>
> I am ending all requests on the queue once an eject happens, but it
> still doesn't seem to cause a release.
>
> What is the right way to terminate requests and delete the gendisk in
> the case of physically vanished PCI devices (or even devices in general)?
>

There are several block driver examples in drivers/block.
Or you might want to take a loot at mtdblock.c, or perhaps
this simple ubiblock implementation:

http://lwn.net/Articles/525957/

If you send a PATCH with your driver perhaps
someone can take a look at it and tell you what's going on.

    Ezequiel

Block device driver: how to terminate the block device if media disappears?

[John Beard]

On 21/12/12 18:23, Ezequiel Garcia wrote:

> On Thu, Dec 13, 2012 at 12:31 PM, John Beard <johnb@codexdigital.com> wrote:
>>
>> What is the right way to terminate requests and delete the gendisk in
>> the case of physically vanished PCI devices (or even devices in general)?
>>
> 
> There are several block driver examples in drivers/block.
> Or you might want to take a loot at mtdblock.c, or perhaps
> this simple ubiblock implementation:
> 
> http://lwn.net/Articles/525957/

Thanks, Ezequiel - sorry for the delay, I haven't had a chance to touch
my development computer over the break. The workqueue-based approach
from ubiblock.c seems to have solved the problem and the device can now
shut down happily once the request queue is emptied in the submit_req
call following device removal.

Thanks again for your help!

John Beard

Block device driver: how to terminate the block device if media disappears?

[Ezequiel Garcia]

On Mon, Jan 7, 2013 at 7:49 AM, John Beard <johnb@codexdigital.com> wrote:
> On 21/12/12 18:23, Ezequiel Garcia wrote:
>
>> On Thu, Dec 13, 2012 at 12:31 PM, John Beard <johnb@codexdigital.com> wrote:
>>>
>>> What is the right way to terminate requests and delete the gendisk in
>>> the case of physically vanished PCI devices (or even devices in general)?
>>>
>>
>> There are several block driver examples in drivers/block.
>> Or you might want to take a loot at mtdblock.c, or perhaps
>> this simple ubiblock implementation:
>>
>> http://lwn.net/Articles/525957/
>
> Thanks, Ezequiel - sorry for the delay, I haven't had a chance to touch
> my development computer over the break. The workqueue-based approach
> from ubiblock.c seems to have solved the problem and the device can now
> shut down happily once the request queue is emptied in the submit_req
> call following device removal.
>

Great! It's nice to hear the hint helped you.

FYI, a workqueue should always be preferred over a separate kernel thread,
unless there is a good reason for the kernel thread.

-- 
    Ezequiel

Block device driver: how to terminate the block device if media disappears?

[anish kumar]

On Sat, 2013-01-12 at 14:39 -0300, Ezequiel Garcia wrote:
> On Mon, Jan 7, 2013 at 7:49 AM, John Beard <johnb@codexdigital.com> wrote:
> > On 21/12/12 18:23, Ezequiel Garcia wrote:
> >
> >> On Thu, Dec 13, 2012 at 12:31 PM, John Beard <johnb@codexdigital.com> wrote:
> >>>
> >>> What is the right way to terminate requests and delete the gendisk in
> >>> the case of physically vanished PCI devices (or even devices in general)?
> >>>
> >>
> >> There are several block driver examples in drivers/block.
> >> Or you might want to take a loot at mtdblock.c, or perhaps
> >> this simple ubiblock implementation:
> >>
> >> http://lwn.net/Articles/525957/
> >
> > Thanks, Ezequiel - sorry for the delay, I haven't had a chance to touch
> > my development computer over the break. The workqueue-based approach
> > from ubiblock.c seems to have solved the problem and the device can now
> > shut down happily once the request queue is emptied in the submit_req
> > call following device removal.
> >
> 
> Great! It's nice to hear the hint helped you.
> 
> FYI, a workqueue should always be preferred over a separate kernel thread,
> unless there is a good reason for the kernel thread.
I was led to believe that the workqueue implementation is based on
kernel thread.
I wonder in which cases kernel thread should be preferred over
workqueue?
>