* Verification failures with mixed read/write case @ 2010-02-18 1:20 Ricardo M. Correia 2010-02-18 1:35 ` Ricardo M. Correia 2010-02-23 9:28 ` Jens Axboe 0 siblings, 2 replies; 6+ messages in thread From: Ricardo M. Correia @ 2010-02-18 1:20 UTC (permalink / raw) To: fio (Please CC me in the reply, I'm not subscribed) Hi, I've been observing verification failures with the following configuration file: ----------- [global] ioengine=libaio iodepth=30 rw=randrw bs=512-128k direct=1 rwmixcycle=40 verify=sha256 verify_async=8 [/dev/sda2] -------------------- This happens on both RHEL5.4 and Ubuntu 9.10. I was observing that with fio-1.36, but I just tried cloning fio from the git repo and I'm experiencing the same problem. Any ideas of what might be wrong? I found this email of a few months ago: http://www.spinics.net/lists/fio/msg00205.html And I just verified that this patch is in my just-cloned git tree. Also, an unrelated issue is that fio complains about 'rwmixcycle' being deprecated but the man page doesn't specify what parameter I should be using instead. Thanks in advance, Ricardo ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Verification failures with mixed read/write case 2010-02-18 1:20 Verification failures with mixed read/write case Ricardo M. Correia @ 2010-02-18 1:35 ` Ricardo M. Correia 2010-02-23 9:28 ` Jens Axboe 1 sibling, 0 replies; 6+ messages in thread From: Ricardo M. Correia @ 2010-02-18 1:35 UTC (permalink / raw) To: fio Ok, this appears to be a bug in the SHA256 function after all: sha256: verify failed at 0/1310720K/23461K /s] [0/179 iops] [eta 00m:46s] 4234ee764aba733fd973ca3f52ab654e9b2611287eecd75043b45806e725ea6d3817f1342a03bb6835129936feb6d4738e7ca76195c3106055d45d233867777957496c386cbdfd4bcbaab173d391f168df5d3e04f4821f67af2dc17da4a5ff1f56b2e85abdac00677b61470775117408c0b34116b505e46992f21f29d0302132 4234ee764aba733fd973ca3f52ab654e9b2611287eecd75043b45806e725ea6d3817f1342a03bb6835129936feb6d4738e7ca76195c3106055d45d2338677779e41d69b71069690d300255b680aa84b786c769b7ed9380b7000000000000000000000000f0fd7db7020055b6000285b7c59380b7b4f77db7f4df85b780c968b7 sha256: verify failed at 131072/131072 23d88d7345fae206536dc33adfb9be1a4040c654ee68e86121a3380a08884175e1be76634d8fdf02b7680f30f39183707a0e6a589c9fd928ec06c176b2ee29485722d229075c0c1965ae097e00b05a39a4de530104a8621fada832696571b4711f139a3f80cb95431ee6e7590a500927cd701e116c85843219f7a32e79bbaa7e 23d88d7345fae206536dc33adfb9be1a4040c654ee68e86121a3380a08884175e1be76634d8fdf02b7680f30f39183707a0e6a589c9fd928ec06c176b2ee294800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 It works fine with SHA512. Cheers, Ricardo On Qui, 2010-02-18 at 01:20 +0000, Ricardo M. Correia wrote: > (Please CC me in the reply, I'm not subscribed) > > Hi, > > I've been observing verification failures with the following > configuration file: > > ----------- > [global] > ioengine=libaio > iodepth=30 > rw=randrw > bs=512-128k > direct=1 > rwmixcycle=40 > verify=sha256 > verify_async=8 > > [/dev/sda2] > -------------------- > > This happens on both RHEL5.4 and Ubuntu 9.10. > > I was observing that with fio-1.36, but I just tried cloning fio from > the git repo and I'm experiencing the same problem. > > Any ideas of what might be wrong? > > I found this email of a few months ago: > http://www.spinics.net/lists/fio/msg00205.html > > And I just verified that this patch is in my just-cloned git tree. > > Also, an unrelated issue is that fio complains about 'rwmixcycle' being > deprecated but the man page doesn't specify what parameter I should be > using instead. > > Thanks in advance, > Ricardo > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Verification failures with mixed read/write case 2010-02-18 1:20 Verification failures with mixed read/write case Ricardo M. Correia 2010-02-18 1:35 ` Ricardo M. Correia @ 2010-02-23 9:28 ` Jens Axboe 2010-02-23 9:36 ` Jens Axboe 1 sibling, 1 reply; 6+ messages in thread From: Jens Axboe @ 2010-02-23 9:28 UTC (permalink / raw) To: Ricardo M. Correia; +Cc: fio On Thu, Feb 18 2010, Ricardo M. Correia wrote: > (Please CC me in the reply, I'm not subscribed) > > Hi, > > I've been observing verification failures with the following > configuration file: > > ----------- > [global] > ioengine=libaio > iodepth=30 > rw=randrw > bs=512-128k > direct=1 > rwmixcycle=40 > verify=sha256 > verify_async=8 > > [/dev/sda2] > -------------------- > > This happens on both RHEL5.4 and Ubuntu 9.10. > > I was observing that with fio-1.36, but I just tried cloning fio from > the git repo and I'm experiencing the same problem. > > Any ideas of what might be wrong? > > I found this email of a few months ago: > http://www.spinics.net/lists/fio/msg00205.html > > And I just verified that this patch is in my just-cloned git tree. Mixed read-write workloads generally don't work well with verify. The reason being that if you do verify && read, then fio will assume that you want to verify previously written data. That should not happen for a generated read as part of a mixed workload, I'll check what happens here. > Also, an unrelated issue is that fio complains about 'rwmixcycle' being > deprecated but the man page doesn't specify what parameter I should be > using instead. The time based approach didn't work well for buffered writes, so it was switched to an issue based count instead only. So you use rwmixread/rwmixwrite to set the percentages. -- Jens Axboe ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Verification failures with mixed read/write case 2010-02-23 9:28 ` Jens Axboe @ 2010-02-23 9:36 ` Jens Axboe 2010-02-23 13:48 ` Ricardo M. Correia 0 siblings, 1 reply; 6+ messages in thread From: Jens Axboe @ 2010-02-23 9:36 UTC (permalink / raw) To: Ricardo M. Correia; +Cc: fio On Tue, Feb 23 2010, Jens Axboe wrote: > On Thu, Feb 18 2010, Ricardo M. Correia wrote: > > (Please CC me in the reply, I'm not subscribed) > > > > Hi, > > > > I've been observing verification failures with the following > > configuration file: > > > > ----------- > > [global] > > ioengine=libaio > > iodepth=30 > > rw=randrw > > bs=512-128k > > direct=1 > > rwmixcycle=40 > > verify=sha256 > > verify_async=8 > > > > [/dev/sda2] > > -------------------- > > > > This happens on both RHEL5.4 and Ubuntu 9.10. > > > > I was observing that with fio-1.36, but I just tried cloning fio from > > the git repo and I'm experiencing the same problem. > > > > Any ideas of what might be wrong? > > > > I found this email of a few months ago: > > http://www.spinics.net/lists/fio/msg00205.html > > > > And I just verified that this patch is in my just-cloned git tree. > > Mixed read-write workloads generally don't work well with verify. The > reason being that if you do verify && read, then fio will assume that > you want to verify previously written data. That should not happen for a > generated read as part of a mixed workload, I'll check what happens > here. OK, so this seems to be just a bug in sha256, it's working as expected here. Can you try the below patch? diff --git a/verify.c b/verify.c index 59f1ddf..c894b60 100644 --- a/verify.c +++ b/verify.c @@ -212,7 +212,7 @@ static int verify_io_u_sha256(struct verify_header *hdr, struct io_u *io_u, { void *p = io_u_verify_off(hdr, io_u, header_num); struct vhdr_sha256 *vh = hdr_priv(hdr); - uint8_t sha256[128]; + uint8_t sha256[64]; struct sha256_ctx sha256_ctx = { .buf = sha256, }; diff --git a/verify.h b/verify.h index c12bc7d..be98c54 100644 --- a/verify.h +++ b/verify.h @@ -37,7 +37,7 @@ struct vhdr_sha512 { uint8_t sha512[128]; }; struct vhdr_sha256 { - uint8_t sha256[128]; + uint8_t sha256[64]; }; struct vhdr_sha1 { uint32_t sha1[5]; -- Jens Axboe ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: Verification failures with mixed read/write case 2010-02-23 9:36 ` Jens Axboe @ 2010-02-23 13:48 ` Ricardo M. Correia 2010-02-23 13:59 ` Jens Axboe 0 siblings, 1 reply; 6+ messages in thread From: Ricardo M. Correia @ 2010-02-23 13:48 UTC (permalink / raw) To: Jens Axboe; +Cc: fio Yup, that seems to have worked. Thanks :) On Ter, 2010-02-23 at 10:36 +0100, Jens Axboe wrote: > OK, so this seems to be just a bug in sha256, it's working as expected > here. Can you try the below patch? > > diff --git a/verify.c b/verify.c > index 59f1ddf..c894b60 100644 > --- a/verify.c > +++ b/verify.c > @@ -212,7 +212,7 @@ static int verify_io_u_sha256(struct verify_header *hdr, struct io_u *io_u, > { > void *p = io_u_verify_off(hdr, io_u, header_num); > struct vhdr_sha256 *vh = hdr_priv(hdr); > - uint8_t sha256[128]; > + uint8_t sha256[64]; > struct sha256_ctx sha256_ctx = { > .buf = sha256, > }; > diff --git a/verify.h b/verify.h > index c12bc7d..be98c54 100644 > --- a/verify.h > +++ b/verify.h > @@ -37,7 +37,7 @@ struct vhdr_sha512 { > uint8_t sha512[128]; > }; > struct vhdr_sha256 { > - uint8_t sha256[128]; > + uint8_t sha256[64]; > }; > struct vhdr_sha1 { > uint32_t sha1[5]; > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Verification failures with mixed read/write case 2010-02-23 13:48 ` Ricardo M. Correia @ 2010-02-23 13:59 ` Jens Axboe 0 siblings, 0 replies; 6+ messages in thread From: Jens Axboe @ 2010-02-23 13:59 UTC (permalink / raw) To: Ricardo M. Correia; +Cc: fio On Tue, Feb 23 2010, Ricardo M. Correia wrote: > Yup, that seems to have worked. > > Thanks :) No problem, apparently sha256 verifies aren't that widely used :-) > > On Ter, 2010-02-23 at 10:36 +0100, Jens Axboe wrote: > > OK, so this seems to be just a bug in sha256, it's working as expected > > here. Can you try the below patch? > > > > diff --git a/verify.c b/verify.c > > index 59f1ddf..c894b60 100644 > > --- a/verify.c > > +++ b/verify.c > > @@ -212,7 +212,7 @@ static int verify_io_u_sha256(struct verify_header *hdr, struct io_u *io_u, > > { > > void *p = io_u_verify_off(hdr, io_u, header_num); > > struct vhdr_sha256 *vh = hdr_priv(hdr); > > - uint8_t sha256[128]; > > + uint8_t sha256[64]; > > struct sha256_ctx sha256_ctx = { > > .buf = sha256, > > }; > > diff --git a/verify.h b/verify.h > > index c12bc7d..be98c54 100644 > > --- a/verify.h > > +++ b/verify.h > > @@ -37,7 +37,7 @@ struct vhdr_sha512 { > > uint8_t sha512[128]; > > }; > > struct vhdr_sha256 { > > - uint8_t sha256[128]; > > + uint8_t sha256[64]; > > }; > > struct vhdr_sha1 { > > uint32_t sha1[5]; > > > > -- Jens Axboe ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-02-23 13:59 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-02-18 1:20 Verification failures with mixed read/write case Ricardo M. Correia 2010-02-18 1:35 ` Ricardo M. Correia 2010-02-23 9:28 ` Jens Axboe 2010-02-23 9:36 ` Jens Axboe 2010-02-23 13:48 ` Ricardo M. Correia 2010-02-23 13:59 ` Jens Axboe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox