* CRC algorithm selection
@ 2015-05-06 18:36 Somnath Roy
2015-05-06 18:43 ` Mark Nelson
2015-05-07 5:04 ` Alexandre DERUMIER
0 siblings, 2 replies; 4+ messages in thread
From: Somnath Roy @ 2015-05-06 18:36 UTC (permalink / raw)
To: ceph-devel
Sage,
Based on the discussion of today's performance meeting, I ran the crc32 unittest and here is the result.
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (16 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 1193.86 MB/sec
best choice 0xffffffff = 1193.56 MB/sec
sctp = 1190.6 MB/sec
intel baseline = 261.623 MB/sec
[ OK ] Crc32c.Performance (7064 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (7080 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (7080 ms total)
[ PASSED ] 7 tests.
So, it seems 'sctp' is giving better performance and thus it is recommending this.
So, it seems it is not picking up *_intel_fast method and going back to make file I saw 'configure' was skipping yasm check since in my setup yasm was not installed.
I manually installed 'yasm' and rebuild everything. Now, here is the result..It is picking up *_intel_fast method and it is ~6 times faster !!!
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (4 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 6275.01 MB/sec
best choice 0xffffffff = 6314.59 MB/sec
sctp = 1187.1 MB/sec
intel baseline = 261.652 MB/sec
[ OK ] Crc32c.Performance (5709 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (5713 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (5713 ms total)
[ PASSED ] 7 tests.
Thanks for the pointer and I will enable my journal tests with crc enabled.
BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
Thanks & Regards
Somnath
________________________________
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: CRC algorithm selection
2015-05-06 18:36 CRC algorithm selection Somnath Roy
@ 2015-05-06 18:43 ` Mark Nelson
2015-05-07 5:04 ` Alexandre DERUMIER
1 sibling, 0 replies; 4+ messages in thread
From: Mark Nelson @ 2015-05-06 18:43 UTC (permalink / raw)
To: Somnath Roy, ceph-devel
On 05/06/2015 01:36 PM, Somnath Roy wrote:
> Sage,
> Based on the discussion of today's performance meeting, I ran the crc32 unittest and here is the result.
>
> root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
> Running main() from gmock_main.cc
> [==========] Running 7 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 7 tests from Crc32c
> [ RUN ] Crc32c.Small
> [ OK ] Crc32c.Small (0 ms)
> [ RUN ] Crc32c.PartialWord
> [ OK ] Crc32c.PartialWord (0 ms)
> [ RUN ] Crc32c.Big
> [ OK ] Crc32c.Big (16 ms)
> [ RUN ] Crc32c.Performance
> populating large buffer
> calculating crc
> best choice = 1193.86 MB/sec
> best choice 0xffffffff = 1193.56 MB/sec
> sctp = 1190.6 MB/sec
> intel baseline = 261.623 MB/sec
> [ OK ] Crc32c.Performance (7064 ms)
> [ RUN ] Crc32c.Range
> [ OK ] Crc32c.Range (0 ms)
> [ RUN ] Crc32c.RangeZero
> [ OK ] Crc32c.RangeZero (0 ms)
> [ RUN ] Crc32c.RangeNull
> [ OK ] Crc32c.RangeNull (0 ms)
> [----------] 7 tests from Crc32c (7080 ms total)
>
> [----------] Global test environment tear-down
> [==========] 7 tests from 1 test case ran. (7080 ms total)
> [ PASSED ] 7 tests.
>
>
> So, it seems 'sctp' is giving better performance and thus it is recommending this.
>
> So, it seems it is not picking up *_intel_fast method and going back to make file I saw 'configure' was skipping yasm check since in my setup yasm was not installed.
> I manually installed 'yasm' and rebuild everything. Now, here is the result..It is picking up *_intel_fast method and it is ~6 times faster !!!
>
>
> root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
> Running main() from gmock_main.cc
> [==========] Running 7 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 7 tests from Crc32c
> [ RUN ] Crc32c.Small
> [ OK ] Crc32c.Small (0 ms)
> [ RUN ] Crc32c.PartialWord
> [ OK ] Crc32c.PartialWord (0 ms)
> [ RUN ] Crc32c.Big
> [ OK ] Crc32c.Big (4 ms)
> [ RUN ] Crc32c.Performance
> populating large buffer
> calculating crc
> best choice = 6275.01 MB/sec
> best choice 0xffffffff = 6314.59 MB/sec
> sctp = 1187.1 MB/sec
> intel baseline = 261.652 MB/sec
> [ OK ] Crc32c.Performance (5709 ms)
> [ RUN ] Crc32c.Range
> [ OK ] Crc32c.Range (0 ms)
> [ RUN ] Crc32c.RangeZero
> [ OK ] Crc32c.RangeZero (0 ms)
> [ RUN ] Crc32c.RangeNull
> [ OK ] Crc32c.RangeNull (0 ms)
> [----------] 7 tests from Crc32c (5713 ms total)
>
> [----------] Global test environment tear-down
> [==========] 7 tests from 1 test case ran. (5713 ms total)
> [ PASSED ] 7 tests.
>
>
> Thanks for the pointer and I will enable my journal tests with crc enabled.
>
> BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
Excellent debugging Somnath!
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: CRC algorithm selection
2015-05-06 18:36 CRC algorithm selection Somnath Roy
2015-05-06 18:43 ` Mark Nelson
@ 2015-05-07 5:04 ` Alexandre DERUMIER
2015-05-07 5:46 ` Alexandre DERUMIER
1 sibling, 1 reply; 4+ messages in thread
From: Alexandre DERUMIER @ 2015-05-07 5:04 UTC (permalink / raw)
To: Somnath Roy; +Cc: ceph-devel
Hi Sommath, nice debug !
>>BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
I have found a redhat bugzilla about ceph and yasm.
https://bugzilla.redhat.com/show_bug.cgi?id=1190882
Seem to be blocked by SELinux.
----- Mail original -----
De: "Somnath Roy" <Somnath.Roy@sandisk.com>
À: "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Mercredi 6 Mai 2015 20:36:35
Objet: CRC algorithm selection
Sage,
Based on the discussion of today's performance meeting, I ran the crc32 unittest and here is the result.
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (16 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 1193.86 MB/sec
best choice 0xffffffff = 1193.56 MB/sec
sctp = 1190.6 MB/sec
intel baseline = 261.623 MB/sec
[ OK ] Crc32c.Performance (7064 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (7080 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (7080 ms total)
[ PASSED ] 7 tests.
So, it seems 'sctp' is giving better performance and thus it is recommending this.
So, it seems it is not picking up *_intel_fast method and going back to make file I saw 'configure' was skipping yasm check since in my setup yasm was not installed.
I manually installed 'yasm' and rebuild everything. Now, here is the result..It is picking up *_intel_fast method and it is ~6 times faster !!!
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (4 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 6275.01 MB/sec
best choice 0xffffffff = 6314.59 MB/sec
sctp = 1187.1 MB/sec
intel baseline = 261.652 MB/sec
[ OK ] Crc32c.Performance (5709 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (5713 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (5713 ms total)
[ PASSED ] 7 tests.
Thanks for the pointer and I will enable my journal tests with crc enabled.
BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
Thanks & Regards
Somnath
________________________________
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: CRC algorithm selection
2015-05-07 5:04 ` Alexandre DERUMIER
@ 2015-05-07 5:46 ` Alexandre DERUMIER
0 siblings, 0 replies; 4+ messages in thread
From: Alexandre DERUMIER @ 2015-05-07 5:46 UTC (permalink / raw)
To: Somnath Roy; +Cc: ceph-devel
>>I have found a redhat bugzilla about ceph and yasm.
>>https://bugzilla.redhat.com/show_bug.cgi?id=1190882
>>Seem to be blocked by SELinux.
speak too fast, it's fixed.
I see that yasm is require in all build packages requirements,
src/test/debian-jessie/debian/control: yasm [amd64],
src/test/fedora/ceph.spec.in:BuildRequires: yasm
src/test/ubuntu-12.04/debian/control: yasm [amd64],
src/test/ubuntu-14.04/debian/control: yasm [amd64],
src/test/centos-6/ceph.spec.in:BuildRequires: yasm
src/test/centos-7/ceph.spec.in:BuildRequires: yasm
and also a commit sage about incompatibility with x32
commit 49a08eb46ef277225fff0d9a0920c37427175f54
Author: Thorsten Glaser <tg@mirbsd.de>
Date: Mon Sep 8 12:49:50 2014 -0700
debian: only B-R yasm on amd64
Make yasm dependency amd64 only, it isn?t used elsewhere
but breaks x32 (which is mis-detected as amd64)
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9ab46dc5b49219aa6194861c393c938f23001c52)
----- Mail original -----
De: "aderumier" <aderumier@odiso.com>
À: "Somnath Roy" <Somnath.Roy@sandisk.com>
Cc: "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Jeudi 7 Mai 2015 07:04:58
Objet: Re: CRC algorithm selection
Hi Sommath, nice debug !
>>BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
I have found a redhat bugzilla about ceph and yasm.
https://bugzilla.redhat.com/show_bug.cgi?id=1190882
Seem to be blocked by SELinux.
----- Mail original -----
De: "Somnath Roy" <Somnath.Roy@sandisk.com>
À: "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Mercredi 6 Mai 2015 20:36:35
Objet: CRC algorithm selection
Sage,
Based on the discussion of today's performance meeting, I ran the crc32 unittest and here is the result.
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (16 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 1193.86 MB/sec
best choice 0xffffffff = 1193.56 MB/sec
sctp = 1190.6 MB/sec
intel baseline = 261.623 MB/sec
[ OK ] Crc32c.Performance (7064 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (7080 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (7080 ms total)
[ PASSED ] 7 tests.
So, it seems 'sctp' is giving better performance and thus it is recommending this.
So, it seems it is not picking up *_intel_fast method and going back to make file I saw 'configure' was skipping yasm check since in my setup yasm was not installed.
I manually installed 'yasm' and rebuild everything. Now, here is the result..It is picking up *_intel_fast method and it is ~6 times faster !!!
root@emsserver2:/home/ceph-main/src# ./unittest_crc32c
Running main() from gmock_main.cc
[==========] Running 7 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 7 tests from Crc32c
[ RUN ] Crc32c.Small
[ OK ] Crc32c.Small (0 ms)
[ RUN ] Crc32c.PartialWord
[ OK ] Crc32c.PartialWord (0 ms)
[ RUN ] Crc32c.Big
[ OK ] Crc32c.Big (4 ms)
[ RUN ] Crc32c.Performance
populating large buffer
calculating crc
best choice = 6275.01 MB/sec
best choice 0xffffffff = 6314.59 MB/sec
sctp = 1187.1 MB/sec
intel baseline = 261.652 MB/sec
[ OK ] Crc32c.Performance (5709 ms)
[ RUN ] Crc32c.Range
[ OK ] Crc32c.Range (0 ms)
[ RUN ] Crc32c.RangeZero
[ OK ] Crc32c.RangeZero (0 ms)
[ RUN ] Crc32c.RangeNull
[ OK ] Crc32c.RangeNull (0 ms)
[----------] 7 tests from Crc32c (5713 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 1 test case ran. (5713 ms total)
[ PASSED ] 7 tests.
Thanks for the pointer and I will enable my journal tests with crc enabled.
BTW, I think we need to error out configure script (or at least document ) if build server is not having 'yasm' installed.
Thanks & Regards
Somnath
________________________________
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-05-07 5:46 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-05-06 18:36 CRC algorithm selection Somnath Roy
2015-05-06 18:43 ` Mark Nelson
2015-05-07 5:04 ` Alexandre DERUMIER
2015-05-07 5:46 ` Alexandre DERUMIER
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.