* Re: BUG: libkcapi tests trigger sleep-in-atomic bug in VMX code (ppc64) [not found] ` <CAAUqJDt_y04ts7Aq0U0fhXYRus2ckkEohBTkga0pjqfoQaBOCA@mail.gmail.com> @ 2018-08-21 15:03 ` Christophe LEROY 2018-08-21 15:12 ` Marcelo Henrique Cerri [not found] ` <20180821151623.26061-1-omosnace@redhat.com> 2018-08-22 6:26 ` [PATCH v2] " Ondrej Mosnacek 2 siblings, 1 reply; 11+ messages in thread From: Christophe LEROY @ 2018-08-21 15:03 UTC (permalink / raw) To: Ondrej Mosnáček, Stephan Mueller Cc: marcelo.cerri, linux-crypto, Herbert Xu, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, linuxppc-dev@lists.ozlabs.org Le 21/08/2018 à 16:38, Ondrej Mosnáček a écrit : > ut 21. 8. 2018 o 16:18 Stephan Mueller <smueller@chronox.de> napísal(a): >> Am Dienstag, 21. August 2018, 14:48:11 CEST schrieb Ondrej Mosnáček: >> >> Hi Ondrej, Marcelo, >> >> (+Marcelo) >> >>> Looking at crypto/algif_skcipher.c, I can see that skcipher_recvmsg() >>> holds the socket lock the whole time and yet passes >>> CRYPTO_TFM_REQ_MAY_SLEEP to the cipher implementation. Isn't that >>> wrong? >> >> I think you are referring to lock_sock(sk)? >> >> If so, this should not be the culprit: the socket lock is in essence a mutex- >> like operation with its own wait queue that it allowed to sleep. In >> lock_sock_nested that is called by lock_sock it even has the call of >> might_sleep which indicates that the caller may be put to sleep. >> >> Looking into the code (without too much debugging) I see in the function >> p8_aes_cbc_encrypt that is part of the stack trace the call to >> preempt_disable() which starts an atomic context. The preempt_enable() is >> invoked after the walk operation. >> >> The preempt_disable increases the preempt_count. That counter is used by >> in_atomic() to check whether we are in atomic context. >> >> The issue is that blkcipher_walk_done may call crypto_yield() which then >> invokes cond_resched if the implementation is allowed to sleep. > > Indeed, you're right, the issue is actually in the vmx_crypto code. I > remember having looked at the 'ctr(aes)' implementation in there a few > days ago (I think I was trying to debug this very issue, but for some > reason I only looked at ctr(aes)...) and I didn't find any bug, so > that's why I jumped to suspecting the algif_skcipher code... I should > have double-checked :) > > It turns out the 'cbc(aes)' (and actually also 'xts(aes)') > implementation is coded a bit differently and they both *do* contain > the sleep-in-atomic bug. I will try to fix them according to the > correct CTR implementation and send a patch. CC: linuxppc-dev@lists.ozlabs.org <linuxppc-dev@lists.ozlabs.org> > > Thanks, > Ondrej > >> @Marcelo: shouldn't be the sleep flag be cleared when entering the >> preempt_disable section? >> >> Ciao >> Stephan >> >> ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: BUG: libkcapi tests trigger sleep-in-atomic bug in VMX code (ppc64) 2018-08-21 15:03 ` BUG: libkcapi tests trigger sleep-in-atomic bug in VMX code (ppc64) Christophe LEROY @ 2018-08-21 15:12 ` Marcelo Henrique Cerri 0 siblings, 0 replies; 11+ messages in thread From: Marcelo Henrique Cerri @ 2018-08-21 15:12 UTC (permalink / raw) To: Christophe LEROY Cc: Ondrej Mosnáček, Stephan Mueller, linux-crypto, Herbert Xu, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, linuxppc-dev@lists.ozlabs.org, Paulo Flabiano Smorigo [-- Attachment #1: Type: text/plain, Size: 2770 bytes --] CC: Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com> Yes, I do believe that CTR is doing it right. Preemption only needs to be disabled during the aes_p8_cbc_encrypt() call, to avoid trashing the VSX registers during the AES operation. -- Regards, Marcelo On Tue, Aug 21, 2018 at 05:03:50PM +0200, Christophe LEROY wrote: > > > Le 21/08/2018 à 16:38, Ondrej Mosnáček a écrit : > > ut 21. 8. 2018 o 16:18 Stephan Mueller <smueller@chronox.de> napísal(a): > > > Am Dienstag, 21. August 2018, 14:48:11 CEST schrieb Ondrej Mosnáček: > > > > > > Hi Ondrej, Marcelo, > > > > > > (+Marcelo) > > > > > > > Looking at crypto/algif_skcipher.c, I can see that skcipher_recvmsg() > > > > holds the socket lock the whole time and yet passes > > > > CRYPTO_TFM_REQ_MAY_SLEEP to the cipher implementation. Isn't that > > > > wrong? > > > > > > I think you are referring to lock_sock(sk)? > > > > > > If so, this should not be the culprit: the socket lock is in essence a mutex- > > > like operation with its own wait queue that it allowed to sleep. In > > > lock_sock_nested that is called by lock_sock it even has the call of > > > might_sleep which indicates that the caller may be put to sleep. > > > > > > Looking into the code (without too much debugging) I see in the function > > > p8_aes_cbc_encrypt that is part of the stack trace the call to > > > preempt_disable() which starts an atomic context. The preempt_enable() is > > > invoked after the walk operation. > > > > > > The preempt_disable increases the preempt_count. That counter is used by > > > in_atomic() to check whether we are in atomic context. > > > > > > The issue is that blkcipher_walk_done may call crypto_yield() which then > > > invokes cond_resched if the implementation is allowed to sleep. > > > > Indeed, you're right, the issue is actually in the vmx_crypto code. I > > remember having looked at the 'ctr(aes)' implementation in there a few > > days ago (I think I was trying to debug this very issue, but for some > > reason I only looked at ctr(aes)...) and I didn't find any bug, so > > that's why I jumped to suspecting the algif_skcipher code... I should > > have double-checked :) > > > > It turns out the 'cbc(aes)' (and actually also 'xts(aes)') > > implementation is coded a bit differently and they both *do* contain > > the sleep-in-atomic bug. I will try to fix them according to the > > correct CTR implementation and send a patch. > > CC: linuxppc-dev@lists.ozlabs.org <linuxppc-dev@lists.ozlabs.org> > > > > > Thanks, > > Ondrej > > > > > @Marcelo: shouldn't be the sleep flag be cleared when entering the > > > preempt_disable section? > > > > > > Ciao > > > Stephan > > > > > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <20180821151623.26061-1-omosnace@redhat.com>]
* Re: [PATCH] crypto: vmx - Fix sleep-in-atomic bugs [not found] ` <20180821151623.26061-1-omosnace@redhat.com> @ 2018-08-21 15:24 ` Ondrej Mosnáček 2018-08-21 16:41 ` Marcelo Henrique Cerri 0 siblings, 1 reply; 11+ messages in thread From: Ondrej Mosnáček @ 2018-08-21 15:24 UTC (permalink / raw) To: omosnace Cc: Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, marcelo.cerri, leo.barbosa, stable, Paulo Flabiano Smorigo, linuxppc-dev CC: Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com>, linuxppc-dev@lists.ozlabs.org (Sorry, sent this before reading new e-mails in the thread...) ut 21. 8. 2018 o 17:18 Ondrej Mosnacek <omosnace@redhat.com> nap=C3=ADsal(a= ): > > This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > implementations. The problem is that the blkcipher_* functions should > not be called in atomic context. > > The bugs can be reproduced via the AF_ALG interface by trying to > encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the > VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > trigger BUG in crypto_yield(): > > [ 891.863680] BUG: sleeping function called from invalid context at incl= ude/crypto/algapi.h:424 > [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: kcap= i-enc > [ 891.864739] 1 lock held by kcapi-enc/12347: > [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skcipher= _recvmsg+0x50/0x530 > [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0.rc0= .git3.1.fc30.ppc64le #1 > [ 891.865251] Call Trace: > [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/0x16= 4 (unreliable) > [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0x2f8= /0x310 > [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_done+= 0x374/0x4a0 > [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encrypt+0= x1c8/0x260 [vmx_crypto] > [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt_blk= cipher+0x60/0x80 > [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg+0x4= 24/0x530 > [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x74/0x= a0 > [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0xf4/= 0x2f0 > [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x68/0= xe0 > [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c/0x7= 0 > > Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX module") > Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > Cc: stable@vger.kernel.org > Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> > --- > This patch should fix the issue, but I didn't test it. (I'll see if I > can find some time tomorrow to try and recompile the kernel on a PPC > machine... in the meantime please review :) > > drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- > drivers/crypto/vmx/aes_xts.c | 19 ++++++++++++------- > 2 files changed, 26 insertions(+), 23 deletions(-) > > diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c > index 5285ece4f33a..b71895871be3 100644 > --- a/drivers/crypto/vmx/aes_cbc.c > +++ b/drivers/crypto/vmx/aes_cbc.c > @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc= *desc, > ret =3D crypto_skcipher_encrypt(req); > skcipher_request_zero(req); > } else { > - preempt_disable(); > - pagefault_disable(); > - enable_kernel_vsx(); > - > blkcipher_walk_init(&walk, dst, src, nbytes); > ret =3D blkcipher_walk_virt(desc, &walk); > while ((nbytes =3D walk.nbytes)) { > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > aes_p8_cbc_encrypt(walk.src.virt.addr, > walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, > &ctx->enc_key, walk.iv, 1); > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > nbytes &=3D AES_BLOCK_SIZE - 1; > ret =3D blkcipher_walk_done(desc, &walk, nbytes); > } > - > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > } > > return ret; > @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc= *desc, > ret =3D crypto_skcipher_decrypt(req); > skcipher_request_zero(req); > } else { > - preempt_disable(); > - pagefault_disable(); > - enable_kernel_vsx(); > - > blkcipher_walk_init(&walk, dst, src, nbytes); > ret =3D blkcipher_walk_virt(desc, &walk); > while ((nbytes =3D walk.nbytes)) { > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > aes_p8_cbc_encrypt(walk.src.virt.addr, > walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, > &ctx->dec_key, walk.iv, 0); > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > nbytes &=3D AES_BLOCK_SIZE - 1; > ret =3D blkcipher_walk_done(desc, &walk, nbytes); > } > - > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > } > > return ret; > diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xts.c > index 8bd9aff0f55f..016ef52390c9 100644 > --- a/drivers/crypto/vmx/aes_xts.c > +++ b/drivers/crypto/vmx/aes_xts.c > @@ -116,13 +116,14 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *= desc, > ret =3D enc? crypto_skcipher_encrypt(req) : crypto_skciph= er_decrypt(req); > skcipher_request_zero(req); > } else { > + blkcipher_walk_init(&walk, dst, src, nbytes); > + > + ret =3D blkcipher_walk_virt(desc, &walk); > + > preempt_disable(); > pagefault_disable(); > enable_kernel_vsx(); > > - blkcipher_walk_init(&walk, dst, src, nbytes); > - > - ret =3D blkcipher_walk_virt(desc, &walk); > iv =3D walk.iv; > memset(tweak, 0, AES_BLOCK_SIZE); > aes_p8_encrypt(iv, tweak, &ctx->tweak_key); > @@ -135,13 +136,17 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *= desc, > aes_p8_xts_decrypt(walk.src.virt.addr, wa= lk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, = &ctx->dec_key, NULL, tweak); > > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > nbytes &=3D AES_BLOCK_SIZE - 1; > ret =3D blkcipher_walk_done(desc, &walk, nbytes); > - } > > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > + } > } > return ret; > } > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-21 15:24 ` [PATCH] crypto: vmx - Fix sleep-in-atomic bugs Ondrej Mosnáček @ 2018-08-21 16:41 ` Marcelo Henrique Cerri 2018-08-22 6:05 ` Ondrej Mosnáček 0 siblings, 1 reply; 11+ messages in thread From: Marcelo Henrique Cerri @ 2018-08-21 16:41 UTC (permalink / raw) To: Ondrej Mosnáček Cc: omosnace, Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, leo.barbosa, stable, Paulo Flabiano Smorigo, linuxppc-dev [-- Attachment #1: Type: text/plain, Size: 7970 bytes --] On Tue, Aug 21, 2018 at 05:24:45PM +0200, Ondrej Mosnáček wrote: > CC: Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com>, > linuxppc-dev@lists.ozlabs.org > > (Sorry, sent this before reading new e-mails in the thread...) > > ut 21. 8. 2018 o 17:18 Ondrej Mosnacek <omosnace@redhat.com> napísal(a): > > > > This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > > implementations. The problem is that the blkcipher_* functions should > > not be called in atomic context. > > > > The bugs can be reproduced via the AF_ALG interface by trying to > > encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the > > VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > > trigger BUG in crypto_yield(): > > > > [ 891.863680] BUG: sleeping function called from invalid context at include/crypto/algapi.h:424 > > [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: kcapi-enc > > [ 891.864739] 1 lock held by kcapi-enc/12347: > > [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skcipher_recvmsg+0x50/0x530 > > [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 > > [ 891.865251] Call Trace: > > [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/0x164 (unreliable) > > [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0x2f8/0x310 > > [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_done+0x374/0x4a0 > > [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] > > [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt_blkcipher+0x60/0x80 > > [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg+0x424/0x530 > > [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x74/0xa0 > > [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0xf4/0x2f0 > > [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x68/0xe0 > > [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c/0x70 > > > > Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX module") > > Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > > Cc: stable@vger.kernel.org > > Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> > > --- > > This patch should fix the issue, but I didn't test it. (I'll see if I > > can find some time tomorrow to try and recompile the kernel on a PPC > > machine... in the meantime please review :) > > > > drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- > > drivers/crypto/vmx/aes_xts.c | 19 ++++++++++++------- > > 2 files changed, 26 insertions(+), 23 deletions(-) > > > > diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c > > index 5285ece4f33a..b71895871be3 100644 > > --- a/drivers/crypto/vmx/aes_cbc.c > > +++ b/drivers/crypto/vmx/aes_cbc.c > > @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc *desc, > > ret = crypto_skcipher_encrypt(req); > > skcipher_request_zero(req); > > } else { > > - preempt_disable(); > > - pagefault_disable(); > > - enable_kernel_vsx(); > > - > > blkcipher_walk_init(&walk, dst, src, nbytes); > > ret = blkcipher_walk_virt(desc, &walk); > > while ((nbytes = walk.nbytes)) { > > + preempt_disable(); > > + pagefault_disable(); > > + enable_kernel_vsx(); > > aes_p8_cbc_encrypt(walk.src.virt.addr, > > walk.dst.virt.addr, > > nbytes & AES_BLOCK_MASK, > > &ctx->enc_key, walk.iv, 1); > > + disable_kernel_vsx(); > > + pagefault_enable(); > > + preempt_enable(); > > + > > nbytes &= AES_BLOCK_SIZE - 1; > > ret = blkcipher_walk_done(desc, &walk, nbytes); > > } > > - > > - disable_kernel_vsx(); > > - pagefault_enable(); > > - preempt_enable(); > > } > > > > return ret; > > @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc *desc, > > ret = crypto_skcipher_decrypt(req); > > skcipher_request_zero(req); > > } else { > > - preempt_disable(); > > - pagefault_disable(); > > - enable_kernel_vsx(); > > - > > blkcipher_walk_init(&walk, dst, src, nbytes); > > ret = blkcipher_walk_virt(desc, &walk); > > while ((nbytes = walk.nbytes)) { > > + preempt_disable(); > > + pagefault_disable(); > > + enable_kernel_vsx(); > > aes_p8_cbc_encrypt(walk.src.virt.addr, > > walk.dst.virt.addr, > > nbytes & AES_BLOCK_MASK, > > &ctx->dec_key, walk.iv, 0); > > + disable_kernel_vsx(); > > + pagefault_enable(); > > + preempt_enable(); > > + > > nbytes &= AES_BLOCK_SIZE - 1; > > ret = blkcipher_walk_done(desc, &walk, nbytes); > > } > > - > > - disable_kernel_vsx(); > > - pagefault_enable(); > > - preempt_enable(); > > } > > > > return ret; > > diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xts.c > > index 8bd9aff0f55f..016ef52390c9 100644 > > --- a/drivers/crypto/vmx/aes_xts.c > > +++ b/drivers/crypto/vmx/aes_xts.c > > @@ -116,13 +116,14 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *desc, > > ret = enc? crypto_skcipher_encrypt(req) : crypto_skcipher_decrypt(req); > > skcipher_request_zero(req); > > } else { > > + blkcipher_walk_init(&walk, dst, src, nbytes); > > + > > + ret = blkcipher_walk_virt(desc, &walk); > > + > > preempt_disable(); > > pagefault_disable(); > > enable_kernel_vsx(); > > > > - blkcipher_walk_init(&walk, dst, src, nbytes); > > - > > - ret = blkcipher_walk_virt(desc, &walk); > > iv = walk.iv; > > memset(tweak, 0, AES_BLOCK_SIZE); > > aes_p8_encrypt(iv, tweak, &ctx->tweak_key); > > @@ -135,13 +136,17 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *desc, > > aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, > > nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); > > > > + disable_kernel_vsx(); > > + pagefault_enable(); > > + preempt_enable(); > > + > > nbytes &= AES_BLOCK_SIZE - 1; > > ret = blkcipher_walk_done(desc, &walk, nbytes); > > - } > > > > - disable_kernel_vsx(); > > - pagefault_enable(); > > - preempt_enable(); > > + preempt_disable(); > > + pagefault_disable(); > > + enable_kernel_vsx(); > > + } That doesn't seem right. It would leave preemption disabled when leaving the function. > > } > > return ret; > > } > > -- > > 2.17.1 > > -- Regards, Marcelo [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-21 16:41 ` Marcelo Henrique Cerri @ 2018-08-22 6:05 ` Ondrej Mosnáček 0 siblings, 0 replies; 11+ messages in thread From: Ondrej Mosnáček @ 2018-08-22 6:05 UTC (permalink / raw) To: marcelo.cerri Cc: omosnace, Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, leo.barbosa, stable, Paulo Flabiano Smorigo, linuxppc-dev ut 21. 8. 2018 o 18:41 Marcelo Henrique Cerri <marcelo.cerri@canonical.com> nap=C3=ADsal(a): > On Tue, Aug 21, 2018 at 05:24:45PM +0200, Ondrej Mosn=C3=A1=C4=8Dek wrote= : > > CC: Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com>, > > linuxppc-dev@lists.ozlabs.org > > > > (Sorry, sent this before reading new e-mails in the thread...) > > > > ut 21. 8. 2018 o 17:18 Ondrej Mosnacek <omosnace@redhat.com> nap=C3=ADs= al(a): > > > > > > This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > > > implementations. The problem is that the blkcipher_* functions should > > > not be called in atomic context. > > > > > > The bugs can be reproduced via the AF_ALG interface by trying to > > > encrypt/decrypt sufficiently large buffers (at least 64 KiB) using th= e > > > VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > > > trigger BUG in crypto_yield(): > > > > > > [ 891.863680] BUG: sleeping function called from invalid context at = include/crypto/algapi.h:424 > > > [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: = kcapi-enc > > > [ 891.864739] 1 lock held by kcapi-enc/12347: > > > [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skci= pher_recvmsg+0x50/0x530 > > > [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0= .rc0.git3.1.fc30.ppc64le #1 > > > [ 891.865251] Call Trace: > > > [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/= 0x164 (unreliable) > > > [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0= x2f8/0x310 > > > [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_d= one+0x374/0x4a0 > > > [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encry= pt+0x1c8/0x260 [vmx_crypto] > > > [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt= _blkcipher+0x60/0x80 > > > [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg= +0x424/0x530 > > > [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x7= 4/0xa0 > > > [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0= xf4/0x2f0 > > > [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x= 68/0xe0 > > > [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c= /0x70 > > > > > > Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX modul= e") > > > Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > > > Cc: stable@vger.kernel.org > > > Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> > > > --- > > > This patch should fix the issue, but I didn't test it. (I'll see if I > > > can find some time tomorrow to try and recompile the kernel on a PPC > > > machine... in the meantime please review :) > > > > > > drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- > > > drivers/crypto/vmx/aes_xts.c | 19 ++++++++++++------- > > > 2 files changed, 26 insertions(+), 23 deletions(-) > > > > > > diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cb= c.c > > > index 5285ece4f33a..b71895871be3 100644 > > > --- a/drivers/crypto/vmx/aes_cbc.c > > > +++ b/drivers/crypto/vmx/aes_cbc.c > > > @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct blkcipher_= desc *desc, > > > ret =3D crypto_skcipher_encrypt(req); > > > skcipher_request_zero(req); > > > } else { > > > - preempt_disable(); > > > - pagefault_disable(); > > > - enable_kernel_vsx(); > > > - > > > blkcipher_walk_init(&walk, dst, src, nbytes); > > > ret =3D blkcipher_walk_virt(desc, &walk); > > > while ((nbytes =3D walk.nbytes)) { > > > + preempt_disable(); > > > + pagefault_disable(); > > > + enable_kernel_vsx(); > > > aes_p8_cbc_encrypt(walk.src.virt.addr, > > > walk.dst.virt.addr, > > > nbytes & AES_BLOCK_MASK, > > > &ctx->enc_key, walk.iv, 1)= ; > > > + disable_kernel_vsx(); > > > + pagefault_enable(); > > > + preempt_enable(); > > > + > > > nbytes &=3D AES_BLOCK_SIZE - 1; > > > ret =3D blkcipher_walk_done(desc, &walk, nbyt= es); > > > } > > > - > > > - disable_kernel_vsx(); > > > - pagefault_enable(); > > > - preempt_enable(); > > > } > > > > > > return ret; > > > @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct blkcipher_= desc *desc, > > > ret =3D crypto_skcipher_decrypt(req); > > > skcipher_request_zero(req); > > > } else { > > > - preempt_disable(); > > > - pagefault_disable(); > > > - enable_kernel_vsx(); > > > - > > > blkcipher_walk_init(&walk, dst, src, nbytes); > > > ret =3D blkcipher_walk_virt(desc, &walk); > > > while ((nbytes =3D walk.nbytes)) { > > > + preempt_disable(); > > > + pagefault_disable(); > > > + enable_kernel_vsx(); > > > aes_p8_cbc_encrypt(walk.src.virt.addr, > > > walk.dst.virt.addr, > > > nbytes & AES_BLOCK_MASK, > > > &ctx->dec_key, walk.iv, 0)= ; > > > + disable_kernel_vsx(); > > > + pagefault_enable(); > > > + preempt_enable(); > > > + > > > nbytes &=3D AES_BLOCK_SIZE - 1; > > > ret =3D blkcipher_walk_done(desc, &walk, nbyt= es); > > > } > > > - > > > - disable_kernel_vsx(); > > > - pagefault_enable(); > > > - preempt_enable(); > > > } > > > > > > return ret; > > > diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xt= s.c > > > index 8bd9aff0f55f..016ef52390c9 100644 > > > --- a/drivers/crypto/vmx/aes_xts.c > > > +++ b/drivers/crypto/vmx/aes_xts.c > > > @@ -116,13 +116,14 @@ static int p8_aes_xts_crypt(struct blkcipher_de= sc *desc, > > > ret =3D enc? crypto_skcipher_encrypt(req) : crypto_sk= cipher_decrypt(req); > > > skcipher_request_zero(req); > > > } else { > > > + blkcipher_walk_init(&walk, dst, src, nbytes); > > > + > > > + ret =3D blkcipher_walk_virt(desc, &walk); > > > + > > > preempt_disable(); > > > pagefault_disable(); > > > enable_kernel_vsx(); > > > > > > - blkcipher_walk_init(&walk, dst, src, nbytes); > > > - > > > - ret =3D blkcipher_walk_virt(desc, &walk); > > > iv =3D walk.iv; > > > memset(tweak, 0, AES_BLOCK_SIZE); > > > aes_p8_encrypt(iv, tweak, &ctx->tweak_key); > > > @@ -135,13 +136,17 @@ static int p8_aes_xts_crypt(struct blkcipher_de= sc *desc, > > > aes_p8_xts_decrypt(walk.src.virt.addr= , walk.dst.virt.addr, > > > nbytes & AES_BLOCK_MA= SK, &ctx->dec_key, NULL, tweak); > > > > > > + disable_kernel_vsx(); > > > + pagefault_enable(); > > > + preempt_enable(); > > > + > > > nbytes &=3D AES_BLOCK_SIZE - 1; > > > ret =3D blkcipher_walk_done(desc, &walk, nbyt= es); > > > - } > > > > > > - disable_kernel_vsx(); > > > - pagefault_enable(); > > > - preempt_enable(); > > > + preempt_disable(); > > > + pagefault_disable(); > > > + enable_kernel_vsx(); > > > + } > > That doesn't seem right. It would leave preemption disabled when > leaving the function. Oh, right! Sorry, I will send a fixed v2 soon... > > > } > > > return ret; > > > } > > > -- > > > 2.17.1 > > > > > -- > Regards, > Marcelo ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs [not found] ` <CAAUqJDt_y04ts7Aq0U0fhXYRus2ckkEohBTkga0pjqfoQaBOCA@mail.gmail.com> 2018-08-21 15:03 ` BUG: libkcapi tests trigger sleep-in-atomic bug in VMX code (ppc64) Christophe LEROY [not found] ` <20180821151623.26061-1-omosnace@redhat.com> @ 2018-08-22 6:26 ` Ondrej Mosnacek 2018-08-23 0:04 ` Marcelo Henrique Cerri 2018-08-25 13:28 ` Herbert Xu 2 siblings, 2 replies; 11+ messages in thread From: Ondrej Mosnacek @ 2018-08-22 6:26 UTC (permalink / raw) To: Herbert Xu Cc: linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Marcelo Henrique Cerri, Leonidas S . Barbosa, Paulo Flabiano Smorigo, linuxppc-dev, Ondrej Mosnacek, stable This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX implementations. The problem is that the blkcipher_* functions should not be called in atomic context. The bugs can be reproduced via the AF_ALG interface by trying to encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then trigger BUG in crypto_yield(): [ 891.863680] BUG: sleeping function called from invalid context at include/crypto/algapi.h:424 [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: kcapi-enc [ 891.864739] 1 lock held by kcapi-enc/12347: [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skcipher_recvmsg+0x50/0x530 [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 [ 891.865251] Call Trace: [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/0x164 (unreliable) [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0x2f8/0x310 [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_done+0x374/0x4a0 [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt_blkcipher+0x60/0x80 [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg+0x424/0x530 [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x74/0xa0 [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0xf4/0x2f0 [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x68/0xe0 [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c/0x70 Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX module") Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") Cc: stable@vger.kernel.org Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> --- Still untested, please test and review if possible. Changes in v2: - fix leaving preemtption, etc. disabled when leaving the function (I switched to the more obvious and less efficient variant for the sake of clarity.) drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- drivers/crypto/vmx/aes_xts.c | 21 ++++++++++++++------- 2 files changed, 28 insertions(+), 23 deletions(-) diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c index 5285ece4f33a..b71895871be3 100644 --- a/drivers/crypto/vmx/aes_cbc.c +++ b/drivers/crypto/vmx/aes_cbc.c @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc *desc, ret = crypto_skcipher_encrypt(req); skcipher_request_zero(req); } else { - preempt_disable(); - pagefault_disable(); - enable_kernel_vsx(); - blkcipher_walk_init(&walk, dst, src, nbytes); ret = blkcipher_walk_virt(desc, &walk); while ((nbytes = walk.nbytes)) { + preempt_disable(); + pagefault_disable(); + enable_kernel_vsx(); aes_p8_cbc_encrypt(walk.src.virt.addr, walk.dst.virt.addr, nbytes & AES_BLOCK_MASK, &ctx->enc_key, walk.iv, 1); + disable_kernel_vsx(); + pagefault_enable(); + preempt_enable(); + nbytes &= AES_BLOCK_SIZE - 1; ret = blkcipher_walk_done(desc, &walk, nbytes); } - - disable_kernel_vsx(); - pagefault_enable(); - preempt_enable(); } return ret; @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc *desc, ret = crypto_skcipher_decrypt(req); skcipher_request_zero(req); } else { - preempt_disable(); - pagefault_disable(); - enable_kernel_vsx(); - blkcipher_walk_init(&walk, dst, src, nbytes); ret = blkcipher_walk_virt(desc, &walk); while ((nbytes = walk.nbytes)) { + preempt_disable(); + pagefault_disable(); + enable_kernel_vsx(); aes_p8_cbc_encrypt(walk.src.virt.addr, walk.dst.virt.addr, nbytes & AES_BLOCK_MASK, &ctx->dec_key, walk.iv, 0); + disable_kernel_vsx(); + pagefault_enable(); + preempt_enable(); + nbytes &= AES_BLOCK_SIZE - 1; ret = blkcipher_walk_done(desc, &walk, nbytes); } - - disable_kernel_vsx(); - pagefault_enable(); - preempt_enable(); } return ret; diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xts.c index 8bd9aff0f55f..e9954a7d4694 100644 --- a/drivers/crypto/vmx/aes_xts.c +++ b/drivers/crypto/vmx/aes_xts.c @@ -116,32 +116,39 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *desc, ret = enc? crypto_skcipher_encrypt(req) : crypto_skcipher_decrypt(req); skcipher_request_zero(req); } else { + blkcipher_walk_init(&walk, dst, src, nbytes); + + ret = blkcipher_walk_virt(desc, &walk); + preempt_disable(); pagefault_disable(); enable_kernel_vsx(); - blkcipher_walk_init(&walk, dst, src, nbytes); - - ret = blkcipher_walk_virt(desc, &walk); iv = walk.iv; memset(tweak, 0, AES_BLOCK_SIZE); aes_p8_encrypt(iv, tweak, &ctx->tweak_key); + disable_kernel_vsx(); + pagefault_enable(); + preempt_enable(); + while ((nbytes = walk.nbytes)) { + preempt_disable(); + pagefault_disable(); + enable_kernel_vsx(); if (enc) aes_p8_xts_encrypt(walk.src.virt.addr, walk.dst.virt.addr, nbytes & AES_BLOCK_MASK, &ctx->enc_key, NULL, tweak); else aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); + disable_kernel_vsx(); + pagefault_enable(); + preempt_enable(); nbytes &= AES_BLOCK_SIZE - 1; ret = blkcipher_walk_done(desc, &walk, nbytes); } - - disable_kernel_vsx(); - pagefault_enable(); - preempt_enable(); } return ret; } -- 2.17.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-22 6:26 ` [PATCH v2] " Ondrej Mosnacek @ 2018-08-23 0:04 ` Marcelo Henrique Cerri 2018-08-23 20:34 ` Paulo Flabiano Smorigo 2018-08-25 13:28 ` Herbert Xu 1 sibling, 1 reply; 11+ messages in thread From: Marcelo Henrique Cerri @ 2018-08-23 0:04 UTC (permalink / raw) To: Ondrej Mosnacek Cc: Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Leonidas S . Barbosa, Paulo Flabiano Smorigo, linuxppc-dev, stable [-- Attachment #1: Type: text/plain, Size: 6255 bytes --] That looks good to me. Maybe Paulo can help testing it. -- Regards, Marcelo On Wed, Aug 22, 2018 at 08:26:31AM +0200, Ondrej Mosnacek wrote: > This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > implementations. The problem is that the blkcipher_* functions should > not be called in atomic context. > > The bugs can be reproduced via the AF_ALG interface by trying to > encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the > VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > trigger BUG in crypto_yield(): > > [ 891.863680] BUG: sleeping function called from invalid context at include/crypto/algapi.h:424 > [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: kcapi-enc > [ 891.864739] 1 lock held by kcapi-enc/12347: > [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skcipher_recvmsg+0x50/0x530 > [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 > [ 891.865251] Call Trace: > [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/0x164 (unreliable) > [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0x2f8/0x310 > [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_done+0x374/0x4a0 > [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] > [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt_blkcipher+0x60/0x80 > [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg+0x424/0x530 > [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x74/0xa0 > [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0xf4/0x2f0 > [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x68/0xe0 > [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c/0x70 > > Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX module") > Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > Cc: stable@vger.kernel.org > Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> > --- > Still untested, please test and review if possible. > > Changes in v2: > - fix leaving preemtption, etc. disabled when leaving the function > (I switched to the more obvious and less efficient variant for the > sake of clarity.) > > drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- > drivers/crypto/vmx/aes_xts.c | 21 ++++++++++++++------- > 2 files changed, 28 insertions(+), 23 deletions(-) > > diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c > index 5285ece4f33a..b71895871be3 100644 > --- a/drivers/crypto/vmx/aes_cbc.c > +++ b/drivers/crypto/vmx/aes_cbc.c > @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct blkcipher_desc *desc, > ret = crypto_skcipher_encrypt(req); > skcipher_request_zero(req); > } else { > - preempt_disable(); > - pagefault_disable(); > - enable_kernel_vsx(); > - > blkcipher_walk_init(&walk, dst, src, nbytes); > ret = blkcipher_walk_virt(desc, &walk); > while ((nbytes = walk.nbytes)) { > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > aes_p8_cbc_encrypt(walk.src.virt.addr, > walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, > &ctx->enc_key, walk.iv, 1); > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > nbytes &= AES_BLOCK_SIZE - 1; > ret = blkcipher_walk_done(desc, &walk, nbytes); > } > - > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > } > > return ret; > @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct blkcipher_desc *desc, > ret = crypto_skcipher_decrypt(req); > skcipher_request_zero(req); > } else { > - preempt_disable(); > - pagefault_disable(); > - enable_kernel_vsx(); > - > blkcipher_walk_init(&walk, dst, src, nbytes); > ret = blkcipher_walk_virt(desc, &walk); > while ((nbytes = walk.nbytes)) { > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > aes_p8_cbc_encrypt(walk.src.virt.addr, > walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, > &ctx->dec_key, walk.iv, 0); > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > nbytes &= AES_BLOCK_SIZE - 1; > ret = blkcipher_walk_done(desc, &walk, nbytes); > } > - > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > } > > return ret; > diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xts.c > index 8bd9aff0f55f..e9954a7d4694 100644 > --- a/drivers/crypto/vmx/aes_xts.c > +++ b/drivers/crypto/vmx/aes_xts.c > @@ -116,32 +116,39 @@ static int p8_aes_xts_crypt(struct blkcipher_desc *desc, > ret = enc? crypto_skcipher_encrypt(req) : crypto_skcipher_decrypt(req); > skcipher_request_zero(req); > } else { > + blkcipher_walk_init(&walk, dst, src, nbytes); > + > + ret = blkcipher_walk_virt(desc, &walk); > + > preempt_disable(); > pagefault_disable(); > enable_kernel_vsx(); > > - blkcipher_walk_init(&walk, dst, src, nbytes); > - > - ret = blkcipher_walk_virt(desc, &walk); > iv = walk.iv; > memset(tweak, 0, AES_BLOCK_SIZE); > aes_p8_encrypt(iv, tweak, &ctx->tweak_key); > > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > + > while ((nbytes = walk.nbytes)) { > + preempt_disable(); > + pagefault_disable(); > + enable_kernel_vsx(); > if (enc) > aes_p8_xts_encrypt(walk.src.virt.addr, walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, &ctx->enc_key, NULL, tweak); > else > aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, > nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); > + disable_kernel_vsx(); > + pagefault_enable(); > + preempt_enable(); > > nbytes &= AES_BLOCK_SIZE - 1; > ret = blkcipher_walk_done(desc, &walk, nbytes); > } > - > - disable_kernel_vsx(); > - pagefault_enable(); > - preempt_enable(); > } > return ret; > } > -- > 2.17.1 > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-23 0:04 ` Marcelo Henrique Cerri @ 2018-08-23 20:34 ` Paulo Flabiano Smorigo 2018-08-24 19:47 ` Paulo Flabiano Smorigo 0 siblings, 1 reply; 11+ messages in thread From: Paulo Flabiano Smorigo @ 2018-08-23 20:34 UTC (permalink / raw) To: Marcelo Henrique Cerri Cc: Ondrej Mosnacek, Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Leonidas S . Barbosa, linuxppc-dev, stable On 2018-08-22 21:04, Marcelo Henrique Cerri wrote: > That looks good to me. Maybe Paulo can help testing it. > > -- > Regards, > Marcelo > > On Wed, Aug 22, 2018 at 08:26:31AM +0200, Ondrej Mosnacek wrote: >> This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX >> implementations. The problem is that the blkcipher_* functions should >> not be called in atomic context. >> >> The bugs can be reproduced via the AF_ALG interface by trying to >> encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the >> VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then >> trigger BUG in crypto_yield(): >> >> [ 891.863680] BUG: sleeping function called from invalid context at >> include/crypto/algapi.h:424 >> [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: >> kcapi-enc >> [ 891.864739] 1 lock held by kcapi-enc/12347: >> [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: >> skcipher_recvmsg+0x50/0x530 >> [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted >> 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 >> [ 891.865251] Call Trace: >> [ 891.865340] [c0000003387578c0] [c000000000d67ea4] >> dump_stack+0xe8/0x164 (unreliable) >> [ 891.865511] [c000000338757910] [c000000000172a58] >> ___might_sleep+0x2f8/0x310 >> [ 891.865679] [c000000338757990] [c0000000006bff74] >> blkcipher_walk_done+0x374/0x4a0 >> [ 891.865825] [c0000003387579e0] [d000000007e73e70] >> p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] >> [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] >> skcipher_encrypt_blkcipher+0x60/0x80 >> [ 891.866128] [c000000338757b10] [c0000000006ec504] >> skcipher_recvmsg+0x424/0x530 >> [ 891.866283] [c000000338757bd0] [c000000000b00654] >> sock_recvmsg+0x74/0xa0 >> [ 891.866403] [c000000338757c10] [c000000000b00f64] >> ___sys_recvmsg+0xf4/0x2f0 >> [ 891.866515] [c000000338757d90] [c000000000b02bb8] >> __sys_recvmsg+0x68/0xe0 >> [ 891.866631] [c000000338757e30] [c00000000000bbe4] >> system_call+0x5c/0x70 >> >> Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX >> module") >> Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") >> Cc: stable@vger.kernel.org >> Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> >> --- >> Still untested, please test and review if possible. >> >> Changes in v2: >> - fix leaving preemtption, etc. disabled when leaving the function >> (I switched to the more obvious and less efficient variant for the >> sake of clarity.) >> >> drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- >> drivers/crypto/vmx/aes_xts.c | 21 ++++++++++++++------- >> 2 files changed, 28 insertions(+), 23 deletions(-) >> >> diff --git a/drivers/crypto/vmx/aes_cbc.c >> b/drivers/crypto/vmx/aes_cbc.c >> index 5285ece4f33a..b71895871be3 100644 >> --- a/drivers/crypto/vmx/aes_cbc.c >> +++ b/drivers/crypto/vmx/aes_cbc.c >> @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct >> blkcipher_desc *desc, >> ret = crypto_skcipher_encrypt(req); >> skcipher_request_zero(req); >> } else { >> - preempt_disable(); >> - pagefault_disable(); >> - enable_kernel_vsx(); >> - >> blkcipher_walk_init(&walk, dst, src, nbytes); >> ret = blkcipher_walk_virt(desc, &walk); >> while ((nbytes = walk.nbytes)) { >> + preempt_disable(); >> + pagefault_disable(); >> + enable_kernel_vsx(); >> aes_p8_cbc_encrypt(walk.src.virt.addr, >> walk.dst.virt.addr, >> nbytes & AES_BLOCK_MASK, >> &ctx->enc_key, walk.iv, 1); >> + disable_kernel_vsx(); >> + pagefault_enable(); >> + preempt_enable(); >> + >> nbytes &= AES_BLOCK_SIZE - 1; >> ret = blkcipher_walk_done(desc, &walk, nbytes); >> } >> - >> - disable_kernel_vsx(); >> - pagefault_enable(); >> - preempt_enable(); >> } >> >> return ret; >> @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct >> blkcipher_desc *desc, >> ret = crypto_skcipher_decrypt(req); >> skcipher_request_zero(req); >> } else { >> - preempt_disable(); >> - pagefault_disable(); >> - enable_kernel_vsx(); >> - >> blkcipher_walk_init(&walk, dst, src, nbytes); >> ret = blkcipher_walk_virt(desc, &walk); >> while ((nbytes = walk.nbytes)) { >> + preempt_disable(); >> + pagefault_disable(); >> + enable_kernel_vsx(); >> aes_p8_cbc_encrypt(walk.src.virt.addr, >> walk.dst.virt.addr, >> nbytes & AES_BLOCK_MASK, >> &ctx->dec_key, walk.iv, 0); >> + disable_kernel_vsx(); >> + pagefault_enable(); >> + preempt_enable(); >> + >> nbytes &= AES_BLOCK_SIZE - 1; >> ret = blkcipher_walk_done(desc, &walk, nbytes); >> } >> - >> - disable_kernel_vsx(); >> - pagefault_enable(); >> - preempt_enable(); >> } >> >> return ret; >> diff --git a/drivers/crypto/vmx/aes_xts.c >> b/drivers/crypto/vmx/aes_xts.c >> index 8bd9aff0f55f..e9954a7d4694 100644 >> --- a/drivers/crypto/vmx/aes_xts.c >> +++ b/drivers/crypto/vmx/aes_xts.c >> @@ -116,32 +116,39 @@ static int p8_aes_xts_crypt(struct >> blkcipher_desc *desc, >> ret = enc? crypto_skcipher_encrypt(req) : >> crypto_skcipher_decrypt(req); >> skcipher_request_zero(req); >> } else { >> + blkcipher_walk_init(&walk, dst, src, nbytes); >> + >> + ret = blkcipher_walk_virt(desc, &walk); >> + >> preempt_disable(); >> pagefault_disable(); >> enable_kernel_vsx(); >> >> - blkcipher_walk_init(&walk, dst, src, nbytes); >> - >> - ret = blkcipher_walk_virt(desc, &walk); >> iv = walk.iv; >> memset(tweak, 0, AES_BLOCK_SIZE); >> aes_p8_encrypt(iv, tweak, &ctx->tweak_key); >> >> + disable_kernel_vsx(); >> + pagefault_enable(); >> + preempt_enable(); >> + >> while ((nbytes = walk.nbytes)) { >> + preempt_disable(); >> + pagefault_disable(); >> + enable_kernel_vsx(); >> if (enc) >> aes_p8_xts_encrypt(walk.src.virt.addr, walk.dst.virt.addr, >> nbytes & AES_BLOCK_MASK, &ctx->enc_key, NULL, tweak); >> else >> aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, >> nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); >> + disable_kernel_vsx(); >> + pagefault_enable(); >> + preempt_enable(); >> >> nbytes &= AES_BLOCK_SIZE - 1; >> ret = blkcipher_walk_done(desc, &walk, nbytes); >> } >> - >> - disable_kernel_vsx(); >> - pagefault_enable(); >> - preempt_enable(); >> } >> return ret; >> } >> -- >> 2.17.1 >> Sure thing. I'll test it later today. Thanks Ondrej for the fix. -- Paulo Flabiano Smorigo IBM Linux Technology Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-23 20:34 ` Paulo Flabiano Smorigo @ 2018-08-24 19:47 ` Paulo Flabiano Smorigo 2018-08-24 20:01 ` Ondrej Mosnacek 0 siblings, 1 reply; 11+ messages in thread From: Paulo Flabiano Smorigo @ 2018-08-24 19:47 UTC (permalink / raw) To: Marcelo Henrique Cerri Cc: Ondrej Mosnacek, Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Leonidas S . Barbosa, linuxppc-dev, stable On 2018-08-23 17:34, Paulo Flabiano Smorigo wrote: > On 2018-08-22 21:04, Marcelo Henrique Cerri wrote: >> That looks good to me. Maybe Paulo can help testing it. >> >> -- >> Regards, >> Marcelo >> >> On Wed, Aug 22, 2018 at 08:26:31AM +0200, Ondrej Mosnacek wrote: >>> This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX >>> implementations. The problem is that the blkcipher_* functions should >>> not be called in atomic context. >>> >>> The bugs can be reproduced via the AF_ALG interface by trying to >>> encrypt/decrypt sufficiently large buffers (at least 64 KiB) using >>> the >>> VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then >>> trigger BUG in crypto_yield(): >>> >>> [ 891.863680] BUG: sleeping function called from invalid context at >>> include/crypto/algapi.h:424 >>> [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: >>> kcapi-enc >>> [ 891.864739] 1 lock held by kcapi-enc/12347: >>> [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: >>> skcipher_recvmsg+0x50/0x530 >>> [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted >>> 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 >>> [ 891.865251] Call Trace: >>> [ 891.865340] [c0000003387578c0] [c000000000d67ea4] >>> dump_stack+0xe8/0x164 (unreliable) >>> [ 891.865511] [c000000338757910] [c000000000172a58] >>> ___might_sleep+0x2f8/0x310 >>> [ 891.865679] [c000000338757990] [c0000000006bff74] >>> blkcipher_walk_done+0x374/0x4a0 >>> [ 891.865825] [c0000003387579e0] [d000000007e73e70] >>> p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] >>> [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] >>> skcipher_encrypt_blkcipher+0x60/0x80 >>> [ 891.866128] [c000000338757b10] [c0000000006ec504] >>> skcipher_recvmsg+0x424/0x530 >>> [ 891.866283] [c000000338757bd0] [c000000000b00654] >>> sock_recvmsg+0x74/0xa0 >>> [ 891.866403] [c000000338757c10] [c000000000b00f64] >>> ___sys_recvmsg+0xf4/0x2f0 >>> [ 891.866515] [c000000338757d90] [c000000000b02bb8] >>> __sys_recvmsg+0x68/0xe0 >>> [ 891.866631] [c000000338757e30] [c00000000000bbe4] >>> system_call+0x5c/0x70 >>> >>> Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX >>> module") >>> Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") >>> Cc: stable@vger.kernel.org >>> Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> >>> --- >>> Still untested, please test and review if possible. >>> >>> Changes in v2: >>> - fix leaving preemtption, etc. disabled when leaving the function >>> (I switched to the more obvious and less efficient variant for the >>> sake of clarity.) >>> >>> drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- >>> drivers/crypto/vmx/aes_xts.c | 21 ++++++++++++++------- >>> 2 files changed, 28 insertions(+), 23 deletions(-) >>> >>> diff --git a/drivers/crypto/vmx/aes_cbc.c >>> b/drivers/crypto/vmx/aes_cbc.c >>> index 5285ece4f33a..b71895871be3 100644 >>> --- a/drivers/crypto/vmx/aes_cbc.c >>> +++ b/drivers/crypto/vmx/aes_cbc.c >>> @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct >>> blkcipher_desc *desc, >>> ret = crypto_skcipher_encrypt(req); >>> skcipher_request_zero(req); >>> } else { >>> - preempt_disable(); >>> - pagefault_disable(); >>> - enable_kernel_vsx(); >>> - >>> blkcipher_walk_init(&walk, dst, src, nbytes); >>> ret = blkcipher_walk_virt(desc, &walk); >>> while ((nbytes = walk.nbytes)) { >>> + preempt_disable(); >>> + pagefault_disable(); >>> + enable_kernel_vsx(); >>> aes_p8_cbc_encrypt(walk.src.virt.addr, >>> walk.dst.virt.addr, >>> nbytes & AES_BLOCK_MASK, >>> &ctx->enc_key, walk.iv, 1); >>> + disable_kernel_vsx(); >>> + pagefault_enable(); >>> + preempt_enable(); >>> + >>> nbytes &= AES_BLOCK_SIZE - 1; >>> ret = blkcipher_walk_done(desc, &walk, nbytes); >>> } >>> - >>> - disable_kernel_vsx(); >>> - pagefault_enable(); >>> - preempt_enable(); >>> } >>> >>> return ret; >>> @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct >>> blkcipher_desc *desc, >>> ret = crypto_skcipher_decrypt(req); >>> skcipher_request_zero(req); >>> } else { >>> - preempt_disable(); >>> - pagefault_disable(); >>> - enable_kernel_vsx(); >>> - >>> blkcipher_walk_init(&walk, dst, src, nbytes); >>> ret = blkcipher_walk_virt(desc, &walk); >>> while ((nbytes = walk.nbytes)) { >>> + preempt_disable(); >>> + pagefault_disable(); >>> + enable_kernel_vsx(); >>> aes_p8_cbc_encrypt(walk.src.virt.addr, >>> walk.dst.virt.addr, >>> nbytes & AES_BLOCK_MASK, >>> &ctx->dec_key, walk.iv, 0); >>> + disable_kernel_vsx(); >>> + pagefault_enable(); >>> + preempt_enable(); >>> + >>> nbytes &= AES_BLOCK_SIZE - 1; >>> ret = blkcipher_walk_done(desc, &walk, nbytes); >>> } >>> - >>> - disable_kernel_vsx(); >>> - pagefault_enable(); >>> - preempt_enable(); >>> } >>> >>> return ret; >>> diff --git a/drivers/crypto/vmx/aes_xts.c >>> b/drivers/crypto/vmx/aes_xts.c >>> index 8bd9aff0f55f..e9954a7d4694 100644 >>> --- a/drivers/crypto/vmx/aes_xts.c >>> +++ b/drivers/crypto/vmx/aes_xts.c >>> @@ -116,32 +116,39 @@ static int p8_aes_xts_crypt(struct >>> blkcipher_desc *desc, >>> ret = enc? crypto_skcipher_encrypt(req) : >>> crypto_skcipher_decrypt(req); >>> skcipher_request_zero(req); >>> } else { >>> + blkcipher_walk_init(&walk, dst, src, nbytes); >>> + >>> + ret = blkcipher_walk_virt(desc, &walk); >>> + >>> preempt_disable(); >>> pagefault_disable(); >>> enable_kernel_vsx(); >>> >>> - blkcipher_walk_init(&walk, dst, src, nbytes); >>> - >>> - ret = blkcipher_walk_virt(desc, &walk); >>> iv = walk.iv; >>> memset(tweak, 0, AES_BLOCK_SIZE); >>> aes_p8_encrypt(iv, tweak, &ctx->tweak_key); >>> >>> + disable_kernel_vsx(); >>> + pagefault_enable(); >>> + preempt_enable(); >>> + >>> while ((nbytes = walk.nbytes)) { >>> + preempt_disable(); >>> + pagefault_disable(); >>> + enable_kernel_vsx(); >>> if (enc) >>> aes_p8_xts_encrypt(walk.src.virt.addr, walk.dst.virt.addr, >>> nbytes & AES_BLOCK_MASK, &ctx->enc_key, NULL, tweak); >>> else >>> aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, >>> nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); >>> + disable_kernel_vsx(); >>> + pagefault_enable(); >>> + preempt_enable(); >>> >>> nbytes &= AES_BLOCK_SIZE - 1; >>> ret = blkcipher_walk_done(desc, &walk, nbytes); >>> } >>> - >>> - disable_kernel_vsx(); >>> - pagefault_enable(); >>> - preempt_enable(); >>> } >>> return ret; >>> } >>> -- >>> 2.17.1 >>> > > Sure thing. I'll test it later today. Thanks Ondrej for the fix. Looks good to me. I made some tests here and no problem was found. -- Paulo Flabiano Smorigo IBM Linux Technology Center ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-24 19:47 ` Paulo Flabiano Smorigo @ 2018-08-24 20:01 ` Ondrej Mosnacek 0 siblings, 0 replies; 11+ messages in thread From: Ondrej Mosnacek @ 2018-08-24 20:01 UTC (permalink / raw) To: Paulo Flabiano Smorigo Cc: Marcelo Henrique Cerri, Herbert Xu, linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Leonidas S . Barbosa, linuxppc-dev, stable On Fri, Aug 24, 2018 at 9:39 PM Paulo Flabiano Smorigo <pfsmorigo@linux.vnet.ibm.com> wrote: > On 2018-08-23 17:34, Paulo Flabiano Smorigo wrote: > > On 2018-08-22 21:04, Marcelo Henrique Cerri wrote: > >> That looks good to me. Maybe Paulo can help testing it. > >> > >> -- > >> Regards, > >> Marcelo > >> > >> On Wed, Aug 22, 2018 at 08:26:31AM +0200, Ondrej Mosnacek wrote: > >>> This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > >>> implementations. The problem is that the blkcipher_* functions should > >>> not be called in atomic context. > >>> > >>> The bugs can be reproduced via the AF_ALG interface by trying to > >>> encrypt/decrypt sufficiently large buffers (at least 64 KiB) using > >>> the > >>> VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > >>> trigger BUG in crypto_yield(): > >>> > >>> [ 891.863680] BUG: sleeping function called from invalid context at > >>> include/crypto/algapi.h:424 > >>> [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: > >>> kcapi-enc > >>> [ 891.864739] 1 lock held by kcapi-enc/12347: > >>> [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: > >>> skcipher_recvmsg+0x50/0x530 > >>> [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted > >>> 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 > >>> [ 891.865251] Call Trace: > >>> [ 891.865340] [c0000003387578c0] [c000000000d67ea4] > >>> dump_stack+0xe8/0x164 (unreliable) > >>> [ 891.865511] [c000000338757910] [c000000000172a58] > >>> ___might_sleep+0x2f8/0x310 > >>> [ 891.865679] [c000000338757990] [c0000000006bff74] > >>> blkcipher_walk_done+0x374/0x4a0 > >>> [ 891.865825] [c0000003387579e0] [d000000007e73e70] > >>> p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] > >>> [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] > >>> skcipher_encrypt_blkcipher+0x60/0x80 > >>> [ 891.866128] [c000000338757b10] [c0000000006ec504] > >>> skcipher_recvmsg+0x424/0x530 > >>> [ 891.866283] [c000000338757bd0] [c000000000b00654] > >>> sock_recvmsg+0x74/0xa0 > >>> [ 891.866403] [c000000338757c10] [c000000000b00f64] > >>> ___sys_recvmsg+0xf4/0x2f0 > >>> [ 891.866515] [c000000338757d90] [c000000000b02bb8] > >>> __sys_recvmsg+0x68/0xe0 > >>> [ 891.866631] [c000000338757e30] [c00000000000bbe4] > >>> system_call+0x5c/0x70 > >>> > >>> Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX > >>> module") > >>> Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > >>> Cc: stable@vger.kernel.org > >>> Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> > >>> --- > >>> Still untested, please test and review if possible. > >>> > >>> Changes in v2: > >>> - fix leaving preemtption, etc. disabled when leaving the function > >>> (I switched to the more obvious and less efficient variant for the > >>> sake of clarity.) > >>> > >>> drivers/crypto/vmx/aes_cbc.c | 30 ++++++++++++++---------------- > >>> drivers/crypto/vmx/aes_xts.c | 21 ++++++++++++++------- > >>> 2 files changed, 28 insertions(+), 23 deletions(-) > >>> > >>> diff --git a/drivers/crypto/vmx/aes_cbc.c > >>> b/drivers/crypto/vmx/aes_cbc.c > >>> index 5285ece4f33a..b71895871be3 100644 > >>> --- a/drivers/crypto/vmx/aes_cbc.c > >>> +++ b/drivers/crypto/vmx/aes_cbc.c > >>> @@ -107,24 +107,23 @@ static int p8_aes_cbc_encrypt(struct > >>> blkcipher_desc *desc, > >>> ret = crypto_skcipher_encrypt(req); > >>> skcipher_request_zero(req); > >>> } else { > >>> - preempt_disable(); > >>> - pagefault_disable(); > >>> - enable_kernel_vsx(); > >>> - > >>> blkcipher_walk_init(&walk, dst, src, nbytes); > >>> ret = blkcipher_walk_virt(desc, &walk); > >>> while ((nbytes = walk.nbytes)) { > >>> + preempt_disable(); > >>> + pagefault_disable(); > >>> + enable_kernel_vsx(); > >>> aes_p8_cbc_encrypt(walk.src.virt.addr, > >>> walk.dst.virt.addr, > >>> nbytes & AES_BLOCK_MASK, > >>> &ctx->enc_key, walk.iv, 1); > >>> + disable_kernel_vsx(); > >>> + pagefault_enable(); > >>> + preempt_enable(); > >>> + > >>> nbytes &= AES_BLOCK_SIZE - 1; > >>> ret = blkcipher_walk_done(desc, &walk, nbytes); > >>> } > >>> - > >>> - disable_kernel_vsx(); > >>> - pagefault_enable(); > >>> - preempt_enable(); > >>> } > >>> > >>> return ret; > >>> @@ -147,24 +146,23 @@ static int p8_aes_cbc_decrypt(struct > >>> blkcipher_desc *desc, > >>> ret = crypto_skcipher_decrypt(req); > >>> skcipher_request_zero(req); > >>> } else { > >>> - preempt_disable(); > >>> - pagefault_disable(); > >>> - enable_kernel_vsx(); > >>> - > >>> blkcipher_walk_init(&walk, dst, src, nbytes); > >>> ret = blkcipher_walk_virt(desc, &walk); > >>> while ((nbytes = walk.nbytes)) { > >>> + preempt_disable(); > >>> + pagefault_disable(); > >>> + enable_kernel_vsx(); > >>> aes_p8_cbc_encrypt(walk.src.virt.addr, > >>> walk.dst.virt.addr, > >>> nbytes & AES_BLOCK_MASK, > >>> &ctx->dec_key, walk.iv, 0); > >>> + disable_kernel_vsx(); > >>> + pagefault_enable(); > >>> + preempt_enable(); > >>> + > >>> nbytes &= AES_BLOCK_SIZE - 1; > >>> ret = blkcipher_walk_done(desc, &walk, nbytes); > >>> } > >>> - > >>> - disable_kernel_vsx(); > >>> - pagefault_enable(); > >>> - preempt_enable(); > >>> } > >>> > >>> return ret; > >>> diff --git a/drivers/crypto/vmx/aes_xts.c > >>> b/drivers/crypto/vmx/aes_xts.c > >>> index 8bd9aff0f55f..e9954a7d4694 100644 > >>> --- a/drivers/crypto/vmx/aes_xts.c > >>> +++ b/drivers/crypto/vmx/aes_xts.c > >>> @@ -116,32 +116,39 @@ static int p8_aes_xts_crypt(struct > >>> blkcipher_desc *desc, > >>> ret = enc? crypto_skcipher_encrypt(req) : > >>> crypto_skcipher_decrypt(req); > >>> skcipher_request_zero(req); > >>> } else { > >>> + blkcipher_walk_init(&walk, dst, src, nbytes); > >>> + > >>> + ret = blkcipher_walk_virt(desc, &walk); > >>> + > >>> preempt_disable(); > >>> pagefault_disable(); > >>> enable_kernel_vsx(); > >>> > >>> - blkcipher_walk_init(&walk, dst, src, nbytes); > >>> - > >>> - ret = blkcipher_walk_virt(desc, &walk); > >>> iv = walk.iv; > >>> memset(tweak, 0, AES_BLOCK_SIZE); > >>> aes_p8_encrypt(iv, tweak, &ctx->tweak_key); > >>> > >>> + disable_kernel_vsx(); > >>> + pagefault_enable(); > >>> + preempt_enable(); > >>> + > >>> while ((nbytes = walk.nbytes)) { > >>> + preempt_disable(); > >>> + pagefault_disable(); > >>> + enable_kernel_vsx(); > >>> if (enc) > >>> aes_p8_xts_encrypt(walk.src.virt.addr, walk.dst.virt.addr, > >>> nbytes & AES_BLOCK_MASK, &ctx->enc_key, NULL, tweak); > >>> else > >>> aes_p8_xts_decrypt(walk.src.virt.addr, walk.dst.virt.addr, > >>> nbytes & AES_BLOCK_MASK, &ctx->dec_key, NULL, tweak); > >>> + disable_kernel_vsx(); > >>> + pagefault_enable(); > >>> + preempt_enable(); > >>> > >>> nbytes &= AES_BLOCK_SIZE - 1; > >>> ret = blkcipher_walk_done(desc, &walk, nbytes); > >>> } > >>> - > >>> - disable_kernel_vsx(); > >>> - pagefault_enable(); > >>> - preempt_enable(); > >>> } > >>> return ret; > >>> } > >>> -- > >>> 2.17.1 > >>> > > > > Sure thing. I'll test it later today. Thanks Ondrej for the fix. > > Looks good to me. I made some tests here and no problem was found. Great, thanks! -- Ondrej Mosnacek <omosnace at redhat dot com> Associate Software Engineer, Security Technologies Red Hat, Inc. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2] crypto: vmx - Fix sleep-in-atomic bugs 2018-08-22 6:26 ` [PATCH v2] " Ondrej Mosnacek 2018-08-23 0:04 ` Marcelo Henrique Cerri @ 2018-08-25 13:28 ` Herbert Xu 1 sibling, 0 replies; 11+ messages in thread From: Herbert Xu @ 2018-08-25 13:28 UTC (permalink / raw) To: Ondrej Mosnacek Cc: linux-crypto, Stephan Mueller, Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, Marcelo Henrique Cerri, Leonidas S . Barbosa, Paulo Flabiano Smorigo, linuxppc-dev, stable On Wed, Aug 22, 2018 at 08:26:31AM +0200, Ondrej Mosnacek wrote: > This patch fixes sleep-in-atomic bugs in AES-CBC and AES-XTS VMX > implementations. The problem is that the blkcipher_* functions should > not be called in atomic context. > > The bugs can be reproduced via the AF_ALG interface by trying to > encrypt/decrypt sufficiently large buffers (at least 64 KiB) using the > VMX implementations of 'cbc(aes)' or 'xts(aes)'. Such operations then > trigger BUG in crypto_yield(): > > [ 891.863680] BUG: sleeping function called from invalid context at include/crypto/algapi.h:424 > [ 891.864622] in_atomic(): 1, irqs_disabled(): 0, pid: 12347, name: kcapi-enc > [ 891.864739] 1 lock held by kcapi-enc/12347: > [ 891.864811] #0: 00000000f5d42c46 (sk_lock-AF_ALG){+.+.}, at: skcipher_recvmsg+0x50/0x530 > [ 891.865076] CPU: 5 PID: 12347 Comm: kcapi-enc Not tainted 4.19.0-0.rc0.git3.1.fc30.ppc64le #1 > [ 891.865251] Call Trace: > [ 891.865340] [c0000003387578c0] [c000000000d67ea4] dump_stack+0xe8/0x164 (unreliable) > [ 891.865511] [c000000338757910] [c000000000172a58] ___might_sleep+0x2f8/0x310 > [ 891.865679] [c000000338757990] [c0000000006bff74] blkcipher_walk_done+0x374/0x4a0 > [ 891.865825] [c0000003387579e0] [d000000007e73e70] p8_aes_cbc_encrypt+0x1c8/0x260 [vmx_crypto] > [ 891.865993] [c000000338757ad0] [c0000000006c0ee0] skcipher_encrypt_blkcipher+0x60/0x80 > [ 891.866128] [c000000338757b10] [c0000000006ec504] skcipher_recvmsg+0x424/0x530 > [ 891.866283] [c000000338757bd0] [c000000000b00654] sock_recvmsg+0x74/0xa0 > [ 891.866403] [c000000338757c10] [c000000000b00f64] ___sys_recvmsg+0xf4/0x2f0 > [ 891.866515] [c000000338757d90] [c000000000b02bb8] __sys_recvmsg+0x68/0xe0 > [ 891.866631] [c000000338757e30] [c00000000000bbe4] system_call+0x5c/0x70 > > Fixes: 8c755ace357c ("crypto: vmx - Adding CBC routines for VMX module") > Fixes: c07f5d3da643 ("crypto: vmx - Adding support for XTS") > Cc: stable@vger.kernel.org > Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com> Patch applied. Thanks. -- Email: Herbert Xu <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-08-25 13:29 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAAUqJDvErj0Y4=GBtXgvBRON7MEic7_-jBZNvjacwcOd0TF_2g@mail.gmail.com>
[not found] ` <3627129.cWTIy1uDMC@tauon.chronox.de>
[not found] ` <CAAUqJDt_y04ts7Aq0U0fhXYRus2ckkEohBTkga0pjqfoQaBOCA@mail.gmail.com>
2018-08-21 15:03 ` BUG: libkcapi tests trigger sleep-in-atomic bug in VMX code (ppc64) Christophe LEROY
2018-08-21 15:12 ` Marcelo Henrique Cerri
[not found] ` <20180821151623.26061-1-omosnace@redhat.com>
2018-08-21 15:24 ` [PATCH] crypto: vmx - Fix sleep-in-atomic bugs Ondrej Mosnáček
2018-08-21 16:41 ` Marcelo Henrique Cerri
2018-08-22 6:05 ` Ondrej Mosnáček
2018-08-22 6:26 ` [PATCH v2] " Ondrej Mosnacek
2018-08-23 0:04 ` Marcelo Henrique Cerri
2018-08-23 20:34 ` Paulo Flabiano Smorigo
2018-08-24 19:47 ` Paulo Flabiano Smorigo
2018-08-24 20:01 ` Ondrej Mosnacek
2018-08-25 13:28 ` Herbert Xu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).