Failing iotest 206

Failing iotest 206

[Thomas Huth]

  Hi,

iotest 206 fails for me with:

$ ./check -qcow2 206
QEMU          -- ".../tests/qemu-iotests/../../qemu-system-x86_64" 
-nodefaults -display none -accel qtest
QEMU_IMG      -- ".../tests/qemu-iotests/../../qemu-img"
QEMU_IO       -- ".../tests/qemu-iotests/../../qemu-io" --cache writeback 
--aio threads -f qcow2
QEMU_NBD      -- ".../tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2
IMGPROTO      -- file
PLATFORM      -- Linux/x86_64 thuth.remote.csb 4.18.0-305.3.1.el8_4.x86_64
TEST_DIR      -- .../tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmpx4hiqpkd
SOCKET_SCM_HELPER -- .../tests/qemu-iotests/socket_scm_helper

206   fail       [10:00:50] [10:00:54]   3.4s   (last: 6.2s)  output 
mismatch (see 206.out.bad)
--- 206.out
+++ 206.out.bad
@@ -99,55 +99,19 @@

  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": 
{"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode": 
"ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg": 
"plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file": 
{"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
  {"return": {}}
+Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
  {"return": {}}

  image: TEST_IMG
  file format: IMGFMT
  virtual size: 32 MiB (33554432 bytes)
-encrypted: yes
  cluster_size: 65536
  Format specific information:
      compat: 1.1
      compression type: zlib
      lazy refcounts: false
      refcount bits: 16
-    encrypt:
-        ivgen alg: plain64
-        hash alg: sha1
-        cipher alg: twofish-128
-        uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
-        format: luks
-        cipher mode: ctr
-        slots:
-            [0]:
-                active: true
-                iters: XXX
-                key offset: 4096
-                stripes: 4000
-            [1]:
-                active: false
-                key offset: 69632
-            [2]:
-                active: false
-                key offset: 135168
-            [3]:
-                active: false
-                key offset: 200704
-            [4]:
-                active: false
-                key offset: 266240
-            [5]:
-                active: false
-                key offset: 331776
-            [6]:
-                active: false
-                key offset: 397312
-            [7]:
-                active: false
-                key offset: 462848
-        payload offset: 528384
-        master key iters: XXX
      corrupt: false
      extended l2: false

Looks like it is missing a check for the availability of the corresponding 
crypto stuff? Does anybody got a clue how to fix this?

  Thomas

Re: Failing iotest 206

[Eric Blake]

On Mon, Jul 19, 2021 at 10:06:01AM +0200, Thomas Huth wrote:
>  Hi,
> 
> iotest 206 fails for me with:
> 

> --- 206.out
> +++ 206.out.bad
> @@ -99,55 +99,19 @@
> 
>  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options":
> {"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode":
> "ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg":
> "plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file":
> {"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
>  {"return": {}}
> +Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
>  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
>  {"return": {}}

> 
> Looks like it is missing a check for the availability of the corresponding
> crypto stuff? Does anybody got a clue how to fix this?

What system is this on? Which crypto library versions are installed?
I suspect this is related to Dan's effort to speed up crypto by
favoring gnutls over nettle, where the switch in favored libraries
failed to account for whether twofish-128 is supported?

https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg03886.html

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: Failing iotest 206

[Thomas Huth]

On 20/07/2021 03.12, Eric Blake wrote:
> On Mon, Jul 19, 2021 at 10:06:01AM +0200, Thomas Huth wrote:
>>   Hi,
>>
>> iotest 206 fails for me with:
>>
> 
>> --- 206.out
>> +++ 206.out.bad
>> @@ -99,55 +99,19 @@
>>
>>   {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options":
>> {"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode":
>> "ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg":
>> "plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file":
>> {"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
>>   {"return": {}}
>> +Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
>>   {"execute": "job-dismiss", "arguments": {"id": "job0"}}
>>   {"return": {}}
> 
>>
>> Looks like it is missing a check for the availability of the corresponding
>> crypto stuff? Does anybody got a clue how to fix this?
> 
> What system is this on?

RHEL 8.4

> Which crypto library versions are installed?

gnutls-3.6.14-8.el8_3.x86_64
nettle-3.4.1-4.el8_3.x86_64

> I suspect this is related to Dan's effort to speed up crypto by
> favoring gnutls over nettle, where the switch in favored libraries
> failed to account for whether twofish-128 is supported?
> 
> https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg03886.html

You're right, I've bisected the problem now, and the commit that introduced 
the issue is this one here:

commit 8bd0931f63008b1d50c8df75a611323a93c052bf
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:   Fri Jul 2 17:38:33 2021 +0100

     crypto: prefer gnutls as the crypto backend if new enough

     If we have gnutls >= 3.6.13, then it has enough functionality
     and performance that we can use it as the preferred crypto
     backend.

     Reviewed-by: Eric Blake <eblake@redhat.com>
     Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

  Thomas

Re: Failing iotest 206

[Daniel P. Berrangé]

On Mon, Jul 19, 2021 at 08:12:58PM -0500, Eric Blake wrote:
> On Mon, Jul 19, 2021 at 10:06:01AM +0200, Thomas Huth wrote:
> >  Hi,
> > 
> > iotest 206 fails for me with:
> > 
> 
> > --- 206.out
> > +++ 206.out.bad
> > @@ -99,55 +99,19 @@
> > 
> >  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options":
> > {"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode":
> > "ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg":
> > "plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file":
> > {"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
> >  {"return": {}}
> > +Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
> >  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
> >  {"return": {}}
> 
> > 
> > Looks like it is missing a check for the availability of the corresponding
> > crypto stuff? Does anybody got a clue how to fix this?
> 
> What system is this on? Which crypto library versions are installed?
> I suspect this is related to Dan's effort to speed up crypto by
> favoring gnutls over nettle, where the switch in favored libraries
> failed to account for whether twofish-128 is supported?
> 
> https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg03886.html

Yes, the gnutls provider doesn't support twofish. This doesn't matter
in real world usage because no one is seriously going to ask for twofish
instead of AES for luks encryption.

I guess that test suite was simply trying to ask for some non-default
values though.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Failing iotest 206

[Kevin Wolf]

Am 20.07.2021 um 10:32 hat Daniel P. Berrangé geschrieben:
> On Mon, Jul 19, 2021 at 08:12:58PM -0500, Eric Blake wrote:
> > On Mon, Jul 19, 2021 at 10:06:01AM +0200, Thomas Huth wrote:
> > >  Hi,
> > > 
> > > iotest 206 fails for me with:
> > > 
> > 
> > > --- 206.out
> > > +++ 206.out.bad
> > > @@ -99,55 +99,19 @@
> > > 
> > >  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options":
> > > {"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode":
> > > "ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg":
> > > "plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file":
> > > {"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
> > >  {"return": {}}
> > > +Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
> > >  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
> > >  {"return": {}}
> > 
> > > 
> > > Looks like it is missing a check for the availability of the corresponding
> > > crypto stuff? Does anybody got a clue how to fix this?
> > 
> > What system is this on? Which crypto library versions are installed?
> > I suspect this is related to Dan's effort to speed up crypto by
> > favoring gnutls over nettle, where the switch in favored libraries
> > failed to account for whether twofish-128 is supported?
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg03886.html
> 
> Yes, the gnutls provider doesn't support twofish. This doesn't matter
> in real world usage because no one is seriously going to ask for twofish
> instead of AES for luks encryption.
> 
> I guess that test suite was simply trying to ask for some non-default
> values though.

Do we already have a patch somewhere that makes it use a different
value? Or if not, which value would be most likely to work everywhere?

Kevin

Re: Failing iotest 206

[Daniel P. Berrangé]

On Tue, Aug 03, 2021 at 07:17:47PM +0200, Kevin Wolf wrote:
> Am 20.07.2021 um 10:32 hat Daniel P. Berrangé geschrieben:
> > On Mon, Jul 19, 2021 at 08:12:58PM -0500, Eric Blake wrote:
> > > On Mon, Jul 19, 2021 at 10:06:01AM +0200, Thomas Huth wrote:
> > > >  Hi,
> > > > 
> > > > iotest 206 fails for me with:
> > > > 
> > > 
> > > > --- 206.out
> > > > +++ 206.out.bad
> > > > @@ -99,55 +99,19 @@
> > > > 
> > > >  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options":
> > > > {"driver": "qcow2", "encrypt": {"cipher-alg": "twofish-128", "cipher-mode":
> > > > "ctr", "format": "luks", "hash-alg": "sha1", "iter-time": 10, "ivgen-alg":
> > > > "plain64", "ivgen-hash-alg": "md5", "key-secret": "keysec0"}, "file":
> > > > {"driver": "file", "filename": "TEST_DIR/PID-t.qcow2"}, "size": 33554432}}}
> > > >  {"return": {}}
> > > > +Job failed: Unsupported cipher algorithm twofish-128 with ctr mode
> > > >  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
> > > >  {"return": {}}
> > > 
> > > > 
> > > > Looks like it is missing a check for the availability of the corresponding
> > > > crypto stuff? Does anybody got a clue how to fix this?
> > > 
> > > What system is this on? Which crypto library versions are installed?
> > > I suspect this is related to Dan's effort to speed up crypto by
> > > favoring gnutls over nettle, where the switch in favored libraries
> > > failed to account for whether twofish-128 is supported?
> > > 
> > > https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg03886.html
> > 
> > Yes, the gnutls provider doesn't support twofish. This doesn't matter
> > in real world usage because no one is seriously going to ask for twofish
> > instead of AES for luks encryption.
> > 
> > I guess that test suite was simply trying to ask for some non-default
> > values though.
> 
> Do we already have a patch somewhere that makes it use a different
> value? Or if not, which value would be most likely to work everywhere?

Ultimately there is only one cipher alg that is guaranteed 'aes',
which can be used in two keysizes 128/256, and two modes cbc/xts.

Sine aes-128 with xts is the default, if you want to exercise
a non-default codepath for LUKS support, i'd suggest aes-256
with cbc mode, and essiv IV generator.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|