Re: [Qemu-devel] encrypt in threads

["Daniel P. Berrangé" <berrange@redhat.com>]

On Fri, Nov 30, 2018 at 10:04:57AM +0000, Vladimir Sementsov-Ogievskiy wrote:
> 30.11.2018 12:48, Daniel P. Berrangé wrote:
> > On Thu, Nov 29, 2018 at 06:28:44PM +0000, Vladimir Sementsov-Ogievskiy wrote:
> >>
> >> On 27.11.2018 16:08, Daniel P. Berrangé wrote:
> >>> On Thu, Nov 22, 2018 at 01:01:20PM +0000, Vladimir Sementsov-Ogievskiy wrote:
> >>>> 21.11.2018 20:30, Vladimir Sementsov-Ogievskiy wrote:
> >>>>> Hi Daniel!
> >>>>>
> >>>>> After moving compression to threads in Qcow2 it's an obvious next step to
> >>>>> "threadyfy" encryption in Qcow2 too.
> >>>>>
> >>>>> But it turned out to be not as simple as I assumed. If I call qcrypto_block_encrypt
> >>>>> in parallel threads with the same first argument (block), it just produce wrong
> >>>>> things (pattern verification fails in iotests)..
> >>>>>
> >>>>> So, can you advise the way to parallelize encryption/decryption?
> >>>>>
> >>>> Hmm, just creating QCryptoBlock per each thread helped. Is it correct thing to do?
> >>> That's rather a heavy weight approach and would cause pain when we want
> >>> to support future options such as keyslot updates, and in the future,
> >>> LUKSv2 with master key changes.
> >>>
> >>> Probably what's needed is change to QCryptoBlock struct so that there
> >>> can be multiple QCryptoCipher instances allocated - one per thread.
> >>>
> >>> We might also need to introduce some locking around usage of the
> >>> QCryptoIVGen object, since that has a QCryptoCipher handle too.
> >>
> >>
> >> Can we also create QCryptoIVGen per thread, as QCryptoCipher? Or it
> >> should be one? If one, why my implementation with QCryptoBlock per
> >> thread works (at least it passes iotests, hmm)
> > 
> > The only IV generator that uses ciphers is the "essiv" one. Even that
> > one uses the cipher in ECB mode, so there is no initialization vector
> > required for its internal cipher. This means there's no critical
> > shared state that would be overwritten by threads. Thus using a
> > separate QCryptoCipher per thread for the essiv IV gen is overkill.
> > None the less I think we should protect the IV generator calls with
> > a mutex just to be safe. I don't think the mutex would have a notable
> > impact on performance of the iv generator.
> 
> But if we use mutex, it means that we can't generate two ivs in parallel,
> and then, it becomes better to have separate iv-gen per thread, instead
> of only one, protected by mutex. (or I don't follow)

I'm questioning whether parallelism of the iv-gen is actually needed
though. It has very short execution time, only encrypting 64 bits of
data, per 512 byte sector with essiv.

If you have show performance numbers that prove a per-thread iv generator
is needed for ESSIV I'd accept that, otherwise I'd just go for a mutex.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|