Re: [PATCH] qemu-img: add seek and -n option to dd command

[Max Reitz <mreitz@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2640 bytes --]

On 05.02.21 10:16, Max Reitz wrote:
> On 05.02.21 09:47, Peter Lieven wrote:
>> Am 05.02.21 um 09:18 schrieb Max Reitz:
>>> On 04.02.21 21:09, Peter Lieven wrote:
>>>> Am 02.02.21 um 16:51 schrieb Eric Blake:
>>>>> On 1/28/21 8:07 AM, Peter Lieven wrote:
>>>>>> Signed-off-by: Peter Lieven <pl@kamp.de>
>>>>> Your commit message says 'what', but not 'why'.  Generally, the 
>>>>> one-line
>>>>> 'what' works well as the subject line, but you want the commit body to
>>>>> give an argument why your patch should be applied, rather than blank.
>>>>>
>>>>> Here's the last time we tried to improve qemu-img dd:
>>>>> https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg02618.html
>>>>
>>>>
>>>> I was not aware of that story. My use case is that I want to be
>>>>
>>>> able to "patch" an image that Qemu is able to handle by overwriting
>>>>
>>>> certain sectors. And I especially do not want to "mount" that image
>>>>
>>>> via qemu-nbd because I might not trust it. I totally want to avoid 
>>>> that the host
>>>>
>>>> system tries to analyse that image in terms of scanning the 
>>>> bootsector, partprobe,
>>>>
>>>> lvm etc. pp.
>>>
>>> qemu will have FUSE exporting as of 6.0 (didn’t quite make it into 
>>> 5.2), so you can do something like this:
>>>
>>> $ qemu-storage-daemon \
>>>      --blockdev node-name=export,driver=qcow2,\
>>> file.driver=file,file.filename=image.qcow2 \
>>>      --export fuse,id=fuse,node-name=export,mountpoint=image.qcow2
>>>
>>> This exports the image on image.qcow2 (i.e., on itself) and so by 
>>> accessing the image file you then get raw access to its contents (so 
>>> you can use system tools like dd).
>>>
>>> Doesn’t require root rights, and shouldn’t make the kernel scan 
>>> anything, because it’s exported as just a regular file.
>>
>>
>> Okay, but that is still more housekeeping than just invoking a single 
>> command.
> 
> Yes, but I personally see this as much better than copying all of dd’s 
> functionality into qemu-img.
> 
> My personal complaint is only that it’s a pain in the ass to invoke QSD 
> this way.  It would be nice to have a script that does the same via
> 
> $ qemu-blk-fuse-export image.qcow2
> 
> Would probably be trivial to write, but well, first we gotta do it, and 
> have justification to keep it as part of qemu...
> 
> And if that’s still too much housekeeping, we could even write a qemu-dd 
> script that scans all file arguments for non-raw images, launches a QSD 
> instance to present them as raw, and then invokes dd.

Since today’s Day of Learning at Red Hat, I decided to have some fun 
writing a qemu-dd.py script.

Max

[-- Attachment #2: qemu-dd.py --]
[-- Type: text/x-python, Size: 2981 bytes --]

#!/usr/bin/python

import json
import os
import subprocess
import sys
from typing import Optional

images = {}

for arg in sys.argv[1:]:
    if arg.startswith('if=') or arg.startswith('of='):
        filename = arg[3:]

        # Ignore non-existing files, the user probably wants to make
        # dd create them (as raw images)
        if not os.path.exists(filename):
            continue

        qemu_img = subprocess.Popen(('qemu-img', 'info', filename),
                                    stdout=subprocess.PIPE,
                                    universal_newlines=True)
        output: str = qemu_img.communicate()[0]
        if qemu_img.returncode != 0:
            sys.exit(os.EX_NOINPUT)

        fmt_line = next(line
                        for line in output.split('\n')
                        if line.startswith('file format: '))
        fmt = fmt_line.split(': ', 1)[1]

        if fmt != 'raw':
            images[filename] = fmt

qsd: Optional[subprocess.Popen] = None

if images:
    args = ['qemu-storage-daemon',
            '--chardev', 'stdio,id=monitor',
            '--monitor', 'monitor']

    for i, (image, fmt) in enumerate(images.items()):
        args += ['--blockdev',
                 f'{fmt},node-name=export{i},' +
                 f'file.driver=file,file.filename={image}',
                 '--export',
                 f'fuse,id=fuse{i},node-name=export{i},' +
                 f'mountpoint={image},writable=on']

    qsd = subprocess.Popen(args,
                           stdin=subprocess.PIPE,
                           stdout=subprocess.PIPE,
                           universal_newlines=True)

    assert qsd.stdin is not None
    assert qsd.stdout is not None

    # Do some QMP communication so we know for sure that the exports
    # are up

    qmp_line = json.loads(qsd.stdout.readline())
    if 'QMP' in qmp_line:
        qsd.terminate()
        qsd.wait()
        sys.stderr.write('The QEMU storage daemon did not provide ' +
                         'a QMP monitor\n')
        sys.exit(os.EX_SOFTWARE)

    qsd.stdin.write('{ "execute": "qmp_capabilities" }')
    qsd.stdin.flush()

    # qmp_capabilities response
    if json.loads(qsd.stdout.readline()) != {'return': {}}:
        qsd.terminate()
        qsd.wait()
        sys.stderr.write('Communication error with the QEMU storage daemon\n')
        sys.exit(os.EX_SOFTWARE)

    while True:
        qsd.stdin.write('{ "execute": "query-block-exports" }')
        qsd.stdin.flush()
        exports = json.loads(qsd.stdout.readline())['return']
        if exports is None or len(exports) > len(images):
            qsd.terminate()
            qsd.wait()
            sys.stderr.write('Communication error with the ' +
                             'QEMU storage daemon\n')
            sys.exit(os.EX_SOFTWARE)

        if len(exports) == len(images):
            break

subprocess.Popen(['dd'] + sys.argv[1:]).wait()

if qsd is not None:
    qsd.terminate()
    qsd.wait()