Re: JFFS3 & performance

Artem B. Bityuckiy
2005-01-07T14:23:42Z

[-- Attachment #1: Type: TEXT/PLAIN, Size: 12489 bytes --]

Hello,

here is the test.

I added ARM support and use timers in case of ARM. I use the
system_timer->offset() call which returns the content of the timer
registers casted to nanoseconds. On small problem - the system_timer
structure is not exported, so I failed to create module in case of ARM.
But it is still possible to compile it with kernel and the test will be
performed during linux load.

The new test version is attached. Please, review it. If it is possible,
try on linux board you have. MIPS, PPC, ARM IMHO especially interesting.
If there are some offers to add smth to test, you are welcome :-)

I've run the test on ARM and x86.
My results show that Jorn's adler32r is the winer (adler32 is the next).

Strange results with data walks - two backward walks are fastest. This
doesn't fit what we discussed about passing data forward then backward...
May be 128K array is too small?

P.S. Test is attached.

-----------------------------------------------------------------------
ARM (OMAP):

#cat /proc/cpuinfo
Processor : ARM926EJ-Sid(wb) rev 3 (v5l)
BogoMIPS : 107.72
Features : swp half thumb fastmult edsp java
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant : 0x0
CPU part : 0x926
CPU revision : 3
Cache type : VIPT write-back
Cache clean : cp15 c7 ops
Cache lockdown : format C
Cache format : Harvard
I size : 32768
I assoc : 4
I line length : 32
I sets : 256
D size : 16384
D assoc : 4
D line length : 32
D sets : 128

[crctest] (ARM timers) Data pass both forward, 32 bytes: delta 10, ts1
86284, ts2 86294
[crctest] (ARM timers) Data pass both backward, 32 bytes: delta 8, ts1
165359, ts2 165367
[crctest] (ARM timers) Data pass backward/forward, 32 bytes: delta 9, ts1
244402, ts2 244411
[crctest] (ARM timers) Data pass forward/backward, 32 bytes: delta 9, ts1
323449, ts2 323458
[crctest] (ARM timers) Data pass both forward, 4096 bytes: delta 635, ts1
402496, ts2 403131
[crctest] (ARM timers) Data pass both backward, 4096 bytes: delta 524, ts1
482171, ts2 482695
[crctest] (ARM timers) Data pass backward/forward, 4096 bytes: delta 582,
ts1 561735, ts2 562317
[crctest] (ARM timers) Data pass forward/backward, 4096 bytes: delta 577,
ts1 641359, ts2 641936
[crctest] (ARM timers) Data pass both forward, 32768 bytes: delta 5219,
ts1 720979, ts2 726198
[crctest] (ARM timers) Data pass both backward, 32768 bytes: delta 4361,
ts1 805222, ts2 809583
[crctest] (ARM timers) Data pass backward/forward, 32768 bytes: delta
4695, ts1 888609, ts2 893304
[crctest] (ARM timers) Data pass forward/backward, 32768 bytes: delta
4682, ts1 972332, ts2 977014
[crctest] (ARM timers) Data pass both forward, 65536 bytes: delta 10487,
ts1 1056042, ts2 1066529
[crctest] (ARM timers) Data pass both backward, 65536 bytes: delta 8769,
ts1 1145579, ts2 1154348
[crctest] (ARM timers) Data pass backward/forward, 65536 bytes: delta
9534, ts1 1233398, ts2 1242932
[crctest] (ARM timers) Data pass forward/backward, 65536 bytes: delta
9522, ts1 1321982, ts2 1331504
[crctest] (ARM timers) Data pass both forward, 131072 bytes: delta 20970,
ts1 1410557, ts2 1431527
[crctest] (ARM timers) Data pass both backward, 131072 bytes: delta 17533,
ts1 1510579, ts2 1528112
[crctest] (ARM timers) Data pass backward/forward, 131072 bytes: delta
19157, ts1 1607164, ts2 1626321
[crctest] (ARM timers) Data pass forward/backward, 131072 bytes: delta
19145, ts1 1705375, ts2 1724520
[crctest] (ARM timers) adler32, 32 bytes: delta 10, ts1 1724634, ts2
1724644
[crctest] (ARM timers) adler32, 4096 bytes: delta 662, ts1 4917, ts2 5579
[crctest] (ARM timers) adler32, 32768 bytes: delta 6529, ts1 6299, ts2
12828
[crctest] (ARM timers) adler32, 65536 bytes: delta 13031, ts1 4287, ts2
17318
[crctest] (ARM timers) adler32, 131072 bytes: delta 26044, ts1 10038, ts2
36082
[crctest] (ARM timers) adler32r, 32 bytes: delta 10, ts1 6205, ts2 6215
[crctest] (ARM timers) adler32r, 4096 bytes: delta 639, ts1 6358, ts2 6997
[crctest] (ARM timers) adler32r, 32768 bytes: delta 6480, ts1 7711, ts2
14191
[crctest] (ARM timers) adler32r, 65536 bytes: delta 12944, ts1 5599, ts2
18543
[crctest] (ARM timers) adler32r, 131072 bytes: delta 25873, ts1 11246, ts2
37119
[crctest] (ARM timers) engel32, 32 bytes: delta 11, ts1 7241, ts2 7252
[crctest] (ARM timers) engel32, 4096 bytes: delta 911, ts1 7422, ts2 8333
[crctest] (ARM timers) engel32, 32768 bytes: delta 8553, ts1 9255, ts2
17808
[crctest] (ARM timers) engel32, 65536 bytes: delta 17086, ts1 9629, ts2
26715
[crctest] (ARM timers) engel32, 131072 bytes: delta 34152, ts1 10361, ts2
44513
[crctest] (ARM timers) engel32r, 32 bytes: delta 12, ts1 4644, ts2 4656
[crctest] (ARM timers) engel32r, 4096 bytes: delta 953, ts1 4815, ts2 5768
[crctest] (ARM timers) engel32r, 32768 bytes: delta 7589, ts1 6592, ts2
14181
[crctest] (ARM timers) engel32r, 65536 bytes: delta 15175, ts1 5792, ts2
20967
[crctest] (ARM timers) engel32r, 131072 bytes: delta 30345, ts1 4079, ts2
34424
[crctest] (ARM timers) 16-bit CRC CCITT, 32 bytes: delta 24, ts1 4515, ts2
4539
[crctest] (ARM timers) 16-bit CRC CCITT, 4096 bytes: delta 2817, ts1 4903,
ts2 7720
[crctest] (ARM timers) 16-bit CRC CCITT, 32768 bytes: delta 23752, ts1
10166, ts2 33918
[crctest] (ARM timers) 16-bit CRC CCITT, 65536 bytes: delta 47409, ts1
8775, ts2 56184
[crctest] (ARM timers) 16-bit CRC CCITT, 131072 bytes: delta 94831, ts1
15781, ts2 110612
[crctest] (ARM timers) CRC32, 32 bytes: delta 14, ts1 740, ts2 754
[crctest] (ARM timers) CRC32, 4096 bytes: delta 1402, ts1 971, ts2 2373
[crctest] (ARM timers) CRC32, 32768 bytes: delta 12502, ts1 3689, ts2
16191
[crctest] (ARM timers) CRC32, 65536 bytes: delta 24847, ts1 8787, ts2
33634
[crctest] (ARM timers) CRC32, 131072 bytes: delta 49724, ts1 8714, ts2
58438
[crctest] (ARM timers) CRC32c, 32 bytes: delta 26, ts1 8563, ts2 8589
[crctest] (ARM timers) CRC32c, 4096 bytes: delta 2857, ts1 8953, ts2 11810
[crctest] (ARM timers) CRC32c, 32768 bytes: delta 24094, ts1 4298, ts2
28392
[crctest] (ARM timers) CRC32c, 65536 bytes: delta 48066, ts1 13308, ts2
61374
[crctest] (ARM timers) CRC32c, 131072 bytes: delta 96113, ts1 11096, ts2
107209

-----------------------------------------------------------------------
i686

#cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 6
model name : Celeron (Mendocino)
stepping : 5
cpu MHz : 534.805
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 mtrr pge mca cmov pat
pse36 mmx fxsr
bogomips : 1052.67

[crctst] (get_cycles) Data pass both forward, 32 bytes: delta 731, ts1
5554559977084, ts2 5554559977815
[crctst] (get_cycles) Data pass both backward, 32 bytes: delta 755, ts1
5554571204036, ts2 5554571204791
[crctst] (get_cycles) Data pass backward/forward, 32 bytes: delta 762, ts1
5554582493116, ts2 5554582493878
[crctst] (get_cycles) Data pass forward/backward, 32 bytes: delta 575, ts1
5554593729476, ts2 5554593730051
[crctst] (get_cycles) Data pass both forward, 4096 bytes: delta 70053, ts1
5554604977980, ts2 5554605048033
[crctst] (get_cycles) Data pass both backward, 4096 bytes: delta 70096,
ts1 5554616415540, ts2 5554616485636
[crctst] (get_cycles) Data pass backward/forward, 4096 bytes: delta 70125,
ts1 5554627702308, ts2 5554627772433
[crctst] (get_cycles) Data pass forward/backward, 4096 bytes: delta 70126,
ts1 5554639032212, ts2 5554639102338
[crctst] (get_cycles) Data pass both forward, 32768 bytes: delta 561878,
ts1 5554650362692, ts2 5554650924570
[crctst] (get_cycles) Data pass both backward, 32768 bytes: delta 555688,
ts1 5554662199516, ts2 5554662755204
[crctst] (get_cycles) Data pass backward/forward, 32768 bytes: delta
552954, ts1 5554673970492, ts2 5554674523446
[crctst] (get_cycles) Data pass forward/backward, 32768 bytes: delta
446164, ts1 5554685910812, ts2 5554686356976
[crctst] (get_cycles) Data pass both forward, 65536 bytes: delta 1091671,
ts1 5554697756260, ts2 5554698847931
[crctst] (get_cycles) Data pass both backward, 65536 bytes: delta 1114579,
ts1 5554710185588, ts2 5554711300167
[crctst] (get_cycles) Data pass backward/forward, 65536 bytes: delta
1111689, ts1 5554722654316, ts2 5554723766005
[crctst] (get_cycles) Data pass forward/backward, 65536 bytes: delta
1118957, ts1 5554735058076, ts2 5554736177033
[crctst] (get_cycles) Data pass both forward, 131072 bytes: delta 2131183,
ts1 5554747495004, ts2 5554749626187
[crctst] (get_cycles) Data pass both backward, 131072 bytes: delta
2081712, ts1 5554760991724, ts2 5554763073436
[crctst] (get_cycles) Data pass backward/forward, 131072 bytes: delta
2211375, ts1 5554774343292, ts2 5554776554667
[crctst] (get_cycles) Data pass forward/backward, 131072 bytes: delta
2237850, ts1 5554787816988, ts2 5554790054838
[crctst] (get_cycles) adler32, 32 bytes: delta 208, ts1 5554790070690, ts2
5554790070898
[crctst] (get_cycles) adler32, 4096 bytes: delta 6464, ts1 5554797404641,
ts2 5554797411105
[crctst] (get_cycles) adler32, 32768 bytes: delta 56771, ts1
5554797649035, ts2 5554797705806
[crctst] (get_cycles) adler32, 65536 bytes: delta 113785, ts1
5554798361404, ts2 5554798475189
[crctst] (get_cycles) adler32, 131072 bytes: delta 233520, ts1
5554799374350, ts2 5554799607870
[crctst] (get_cycles) adler32r, 32 bytes: delta 216, ts1 5554800107526,
ts2 5554800107742
[crctst] (get_cycles) adler32r, 4096 bytes: delta 5113, ts1 5554800150365,
ts2 5554800155478
[crctst] (get_cycles) adler32r, 32768 bytes: delta 53081, ts1
5554800379348, ts2 5554800432429
[crctst] (get_cycles) adler32r, 65536 bytes: delta 106798, ts1
5554801010586, ts2 5554801117384
[crctst] (get_cycles) adler32r, 131072 bytes: delta 218664, ts1
5554802198998, ts2 5554802417662
[crctst] (get_cycles) engel32, 32 bytes: delta 144, ts1 5554802654360, ts2
5554802654504
[crctst] (get_cycles) engel32, 4096 bytes: delta 7693, ts1 5554802696176,
ts2 5554802703869
[crctst] (get_cycles) engel32, 32768 bytes: delta 65239, ts1
5554803294336, ts2 5554803359575
[crctst] (get_cycles) engel32, 65536 bytes: delta 130514, ts1
5554803958262, ts2 5554804088776
[crctst] (get_cycles) engel32, 131072 bytes: delta 266126, ts1
5554805379160, ts2 5554805645286
[crctst] (get_cycles) engel32r, 32 bytes: delta 150, ts1 5554806129841,
ts2 5554806129991
[crctst] (get_cycles) engel32r, 4096 bytes: delta 7246, ts1 5554806153189,
ts2 5554806160435
[crctst] (get_cycles) engel32r, 32768 bytes: delta 56813, ts1
5554806224973, ts2 5554806281786
[crctst] (get_cycles) engel32r, 65536 bytes: delta 113411, ts1
5554806402638, ts2 5554806516049
[crctst] (get_cycles) engel32r, 131072 bytes: delta 226662, ts1
5554807652080, ts2 5554807878742
[crctst] (get_cycles) 16-bit CRC CCITT, 32 bytes: delta 316, ts1
5554807901011, ts2 5554807901327
[crctst] (get_cycles) 16-bit CRC CCITT, 4096 bytes: delta 29361, ts1
5554807970344, ts2 5554807999705
[crctst] (get_cycles) 16-bit CRC CCITT, 32768 bytes: delta 235446, ts1
5554808413329, ts2 5554808648775
[crctst] (get_cycles) 16-bit CRC CCITT, 65536 bytes: delta 473151, ts1
5554809580918, ts2 5554810054069
[crctst] (get_cycles) 16-bit CRC CCITT, 131072 bytes: delta 940992, ts1
5554812013686, ts2 5554812954678
[crctst] (get_cycles) CRC32, 32 bytes: delta 242, ts1 5554813156313, ts2
5554813156555
[crctst] (get_cycles) CRC32, 4096 bytes: delta 22075, ts1 5554813404036,
ts2 5554813426111
[crctst] (get_cycles) CRC32, 32768 bytes: delta 176204, ts1 5554813816461,
ts2 5554813992665
[crctst] (get_cycles) CRC32, 65536 bytes: delta 352496, ts1 5554814878041,
ts2 5554815230537
[crctst] (get_cycles) CRC32, 131072 bytes: delta 697640, ts1
5554817199646, ts2 5554817897286
[crctst] (get_cycles) CRC32c, 32 bytes: delta 268, ts1 5554818293652, ts2
5554818293920
[crctst] (get_cycles) CRC32c, 4096 bytes: delta 26211, ts1 5554818362264,
ts2 5554818388475
[crctst] (get_cycles) CRC32c, 32768 bytes: delta 213319, ts1
5554818787865, ts2 5554819001184
[crctst] (get_cycles) CRC32c, 65536 bytes: delta 426616, ts1
5554820096814, ts2 5554820523430
[crctst] (get_cycles) CRC32c, 131072 bytes: delta 859504, ts1
5554822563046, ts2 5554823422550

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

[-- Attachment #2: Type: TEXT/PLAIN, Size: 4079 bytes --]

/*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pagemap.h>
#include <linux/spinlock.h>
#include <linux/crc32.h>
#include <linux/crc32c.h>
#include <linux/crc-ccitt.h>
#include <asm/timex.h>

/* The number of memory cunks to test */
#define MEM_CHUNKS 3

/* The test iterations number */
#define ITERATIONS 1

#define TEST_PREFIX "[crctst] "

/*
* Most architectures have some kind of hight resolution time-stamp
* counter and define the get_cycles() macro to access it.
*/
#define TIMESTAMP get_cycles()

/*
* The sizes of memory chunks for which CRCs should be tested.
*/
static int
memsizes[MEM_CHUNKS] = {32, PAGE_SIZE, 64*1024};

/*
* We perform actual testing in the module initialization function.
*/
static int __init
init_crctest(void)
{
register int i, j;
char *mem[MEM_CHUNKS];
unsigned long flags;
spinlock_t lock = SPIN_LOCK_UNLOCKED;
int ret = 0;
cycles_t ts1, ts2;

memcmp(&mem[0], '\0', MEM_CHUNKS * sizeof(char *));

/* Allocate memory */
for (i = 0; i < MEM_CHUNKS; i++) {
if ((mem[i] = kmalloc(memsizes[i], GFP_KERNEL)) == NULL) {
printk(KERN_ERR TEST_PREFIX "can't allocate %d bytes\n",
memsizes[i]);
ret = -ENOMEM;
goto exit;
}
}

/*
* We do not want to be preempted during the test as well as do
* not want interrupts affect our results. Both of these are
* prevented by spin_lock_irqsave().
*/
spin_lock_irqsave(&lock, flags);

/* Test 16 bit CRC CCITT */
for (i = 0; i < MEM_CHUNKS; i++) {
u16 crc;

/* Do one fake pass to exclude CPU cache influence */
crc = crc_ccitt(0xFFFF, mem[i], memsizes[i]);

ts1 = TIMESTAMP;
for (j = 0; j < ITERATIONS; j++) {
crc = crc_ccitt(0xFFFF, mem[i], memsizes[i]);
}
ts2 = TIMESTAMP;

printk(KERN_NOTICE TEST_PREFIX "16-bit CRC CCITT %d bytes: "
"ts1 %llu, ts2 %llu, delta %llu\n", memsizes[i],
(unsigned long long)ts1, (unsigned long long)ts2,
(unsigned long long)(ts2 - ts1));
}

/* Test crc32 */
for (i = 0; i < MEM_CHUNKS; i++) {
u32 crc;

crc = crc32(0xFFFF, mem[i], memsizes[i]);
ts1 = TIMESTAMP;
for (j = 0; j < ITERATIONS; j++) {
crc = crc32(0xFFFF, mem[i], memsizes[i]);
}
ts2 = TIMESTAMP;

printk(KERN_NOTICE TEST_PREFIX "crc32 %d bytes: "
"ts1 %llu, ts2 %llu, delta %llu\n", memsizes[i],
(unsigned long long)ts1, (unsigned long long)ts2,
(unsigned long long)(ts2 - ts1));
}

/* Test crc32c */
for (i = 0; i < MEM_CHUNKS; i++) {
u32 crc;

crc = crc32c(0xFFFF, mem[i], memsizes[i]);
ts1 = TIMESTAMP;
for (j = 0; j < ITERATIONS; j++) {
crc = crc32c(0xFFFF, mem[i], memsizes[i]);
}
ts2 = TIMESTAMP;

printk(KERN_NOTICE TEST_PREFIX "crc32c %d bytes: "
"ts1 %llu, ts2 %llu, delta %llu\n", memsizes[i],
(unsigned long long)ts1, (unsigned long long)ts2,
(unsigned long long)(ts2 - ts1));
}
spin_unlock_irqrestore(&lock, flags);

exit:
for (i = 0; i < MEM_CHUNKS && mem[i] != NULL; i++)
kfree(mem[i]);

return ret;
}

module_init(init_crctest);

static void __exit
cleanup_crctest(void)
{
return;
}

module_exit(cleanup_crctest);

MODULE_LICENSE ("GPL");
MODULE_AUTHOR ("Artem B. Bityuckiy");
MODULE_DESCRIPTION ("The CRC test");