How to check when "raw" format driver uses a "regular" file?

How to check when "raw" format driver uses a "regular" file?

[Philippe Mathieu-Daudé]

Hi,

Is it possible to restrict a block driver to a particular set of
options? In my case I'd like to restrict the raw driver to regular files.

I noticed the NFS driver does it locally in nfs_client_open(),
and FUSE has is_regular_file() -- which is POSIX specific however.

When a backend is a SCSI drive, the block layer provide the blk_is_sg()
method to test it.
1/ Should I provide a similar blk_is_regular_file()?

2/ There is no oslib function to check for regular file,
should I add one too?

Thanks,

Phil.

Re: How to check when "raw" format driver uses a "regular" file?

[Kevin Wolf]

Am 25.01.2021 um 16:05 hat Philippe Mathieu-Daudé geschrieben:
> Is it possible to restrict a block driver to a particular set of
> options? In my case I'd like to restrict the raw driver to regular files.
> 
> I noticed the NFS driver does it locally in nfs_client_open(),
> and FUSE has is_regular_file() -- which is POSIX specific however.
> 
> When a backend is a SCSI drive, the block layer provide the blk_is_sg()
> method to test it.
> 1/ Should I provide a similar blk_is_regular_file()?
> 
> 2/ There is no oslib function to check for regular file,
> should I add one too?

I find this question confusing because on one hand you're talking about
block driver implementations like NFS, but on the other hand about SCSI
devices, which are users, not implementations of block drivers.

At which level is the code where you think you need to make this
distinction?

The other problem is that "is this a regular file?" is probably not what
you're really interested in. The content of an image can be spread
across several files (for example, consider backing files) or not use a
local file descriptor at all (network protocol drivers), and block layer
functions should ideally make sense for all drivers unless something can
only possibly make sense for a single driver (blk_is_sg might be a case
of this).

I assume that you are interested in some specific property that regular
files happen to provide. If at all possible, we shouldn't check for a
specific backend type, but for capabilities or properties of a given
block node.

So what are you really trying to do here?

Kevin

Re: How to check when "raw" format driver uses a "regular" file?

[Philippe Mathieu-Daudé]

On 1/25/21 4:43 PM, Kevin Wolf wrote:
> Am 25.01.2021 um 16:05 hat Philippe Mathieu-Daudé geschrieben:
>> Is it possible to restrict a block driver to a particular set of
>> options? In my case I'd like to restrict the raw driver to regular files.
>>
>> I noticed the NFS driver does it locally in nfs_client_open(),
>> and FUSE has is_regular_file() -- which is POSIX specific however.
>>
>> When a backend is a SCSI drive, the block layer provide the blk_is_sg()
>> method to test it.
>> 1/ Should I provide a similar blk_is_regular_file()?
>>
>> 2/ There is no oslib function to check for regular file,
>> should I add one too?
> 
> I find this question confusing because on one hand you're talking about
> block driver implementations like NFS, but on the other hand about SCSI
> devices, which are users, not implementations of block drivers.

Sorry for the confusion and thanks for the quick answer :)

My question is confused because the problem is not clear to me...

> At which level is the code where you think you need to make this
> distinction?

(see below)

> The other problem is that "is this a regular file?" is probably not what
> you're really interested in. The content of an image can be spread
> across several files (for example, consider backing files) or not use a
> local file descriptor at all (network protocol drivers), and block layer
> functions should ideally make sense for all drivers unless something can
> only possibly make sense for a single driver (blk_is_sg might be a case
> of this).
> 
> I assume that you are interested in some specific property that regular
> files happen to provide. If at all possible, we shouldn't check for a
> specific backend type, but for capabilities or properties of a given
> block node.
> 
> So what are you really trying to do here?

Well, maybe this is more an emulation problem rather than a block
one. But system emulation consumes block layer :)

The problem is when emulating devices such NOR Flash (parallel mapping
or SD cards) we expect the block driver being a plain file (either "raw"
format or another) but a "regular" file. When an user passes something
else like a block device, odd things happen.

Well, I guess I self-restricted my question to device emulation. So
in the cases mentioned I would like to add a check in sd_realize()/
pflash_cfi0?_realize() for regular file when a block drive is provided.

Description of problematic user case:

* -pflash /dev/sda
* -sd /dev/mmcblk0

User runs emulation on top of hardware (maybe like passthru?), and
expect underlying block to be in correct state out of QEMU.

Thanks,

Phil.

Re: How to check when "raw" format driver uses a "regular" file?

[Peter Maydell]

On Mon, 25 Jan 2021 at 16:03, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> The problem is when emulating devices such NOR Flash (parallel mapping
> or SD cards) we expect the block driver being a plain file (either "raw"
> format or another) but a "regular" file. When an user passes something
> else like a block device, odd things happen.
>
> Well, I guess I self-restricted my question to device emulation. So
> in the cases mentioned I would like to add a check in sd_realize()/
> pflash_cfi0?_realize() for regular file when a block drive is provided.
>
> Description of problematic user case:
>
> * -pflash /dev/sda
> * -sd /dev/mmcblk0
>
> User runs emulation on top of hardware (maybe like passthru?), and
> expect underlying block to be in correct state out of QEMU.

It would certainly be nice if these cases worked (AIUI they do for
IDE/SCSI disk emulation, right?). So what goes wrong in the SD card
emulation case?

thanks
-- PMM

Re: How to check when "raw" format driver uses a "regular" file?

[Kevin Wolf]

Am 25.01.2021 um 17:01 hat Philippe Mathieu-Daudé geschrieben:
> On 1/25/21 4:43 PM, Kevin Wolf wrote:
> > Am 25.01.2021 um 16:05 hat Philippe Mathieu-Daudé geschrieben:
> >> Is it possible to restrict a block driver to a particular set of
> >> options? In my case I'd like to restrict the raw driver to regular files.
> >>
> >> I noticed the NFS driver does it locally in nfs_client_open(),
> >> and FUSE has is_regular_file() -- which is POSIX specific however.
> >>
> >> When a backend is a SCSI drive, the block layer provide the blk_is_sg()
> >> method to test it.
> >> 1/ Should I provide a similar blk_is_regular_file()?
> >>
> >> 2/ There is no oslib function to check for regular file,
> >> should I add one too?
> > 
> > I find this question confusing because on one hand you're talking about
> > block driver implementations like NFS, but on the other hand about SCSI
> > devices, which are users, not implementations of block drivers.
> 
> Sorry for the confusion and thanks for the quick answer :)
> 
> My question is confused because the problem is not clear to me...
> 
> > At which level is the code where you think you need to make this
> > distinction?
> 
> (see below)
> 
> > The other problem is that "is this a regular file?" is probably not what
> > you're really interested in. The content of an image can be spread
> > across several files (for example, consider backing files) or not use a
> > local file descriptor at all (network protocol drivers), and block layer
> > functions should ideally make sense for all drivers unless something can
> > only possibly make sense for a single driver (blk_is_sg might be a case
> > of this).
> > 
> > I assume that you are interested in some specific property that regular
> > files happen to provide. If at all possible, we shouldn't check for a
> > specific backend type, but for capabilities or properties of a given
> > block node.
> > 
> > So what are you really trying to do here?
> 
> Well, maybe this is more an emulation problem rather than a block
> one. But system emulation consumes block layer :)
> 
> The problem is when emulating devices such NOR Flash (parallel mapping
> or SD cards) we expect the block driver being a plain file (either "raw"
> format or another) but a "regular" file. When an user passes something
> else like a block device, odd things happen.

What kind of odd things?

In theory, a block device shouldn't look much different from a regular
file for almost all block layer functions. The major difference is that
it has a fixed size, but device emulation isn't supposed to change the
size of an image file anyway. I could imagine some differences related
to block status. What else?

So far no other device emulation makes the distinction, so there are a
few options: Either they would need it, but we failed to realise this.
Or there is a block layer bug that happens to only be exposed with your
flash code. Or your flash code is doing something wrong. Or you really
have a unique requirement (which we should then make explicitly known to
block layer people) and there is a limitation in the handling of block
devices that we weren't aware of and should document at least.

"odd things happen" doesn't really give much information about which of
these cases we actually have here.

> Well, I guess I self-restricted my question to device emulation. So
> in the cases mentioned I would like to add a check in sd_realize()/
> pflash_cfi0?_realize() for regular file when a block drive is provided.
> 
> Description of problematic user case:
> 
> * -pflash /dev/sda
> * -sd /dev/mmcblk0
> 
> User runs emulation on top of hardware (maybe like passthru?), and
> expect underlying block to be in correct state out of QEMU.

Indeed, I would expect the same. Why doesn't this work, and is it
fundamentally impossible to make it work?

Kevin