From: "Stephen Hemminger <shemminger@vyatta.com>, Al Viro" <viro@zeniv.linux.org.uk>
To: Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Cc: Octavian Purdila <opurdila@ixiacom.com>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH] dcache: better name hash function
Date: Mon, 26 Oct 2009 15:36:56 -0700 [thread overview]
Message-ID: <20091026153656.25be4369@nehalam> (raw)
In-Reply-To: <20091025214357.666350d2@nehalam>
Some experiments by Octavian with large numbers of network devices identified
that name_hash does not evenly distribute values causing performance
penalties. The name hashing function is used by dcache et. all
so let's just choose a better one.
Additional standalone tests for 10,000,000 consecutive names
using lots of different algorithms shows fnv as the winner.
It is faster and has almost ideal dispersion.
string10 is slightly faster, but only works for names like ppp0, ppp1,...
Algorithm Time Ratio Max StdDev
string10 0.238201 1.00 2444 0.02
fnv32 0.240595 1.00 2576 1.05
fnv64 0.241224 1.00 2556 0.69
SuperFastHash 0.272872 1.00 2871 2.15
string_hash17 0.295160 1.00 2484 0.40
jhash_string 0.300925 1.00 2606 1.00
crc 1.606741 1.00 2474 0.29
md5_string 2.424771 1.00 2644 0.99
djb2 0.275424 1.15 3821 19.04
string_hash31 0.264806 1.21 4097 22.78
sdbm 0.371136 2.87 13016 67.54
elf 0.371279 3.59 9990 79.50
pjw 0.401172 3.59 9990 79.50
full_name_hash 0.285851 13.09 35174 171.81
kr_hash 0.245068 124.84 468448 549.89
fletcher 0.267664 124.84 468448 549.89
adler32 0.640668 124.84 468448 549.89
xor 0.220545 213.82 583189 720.85
lastchar 0.194604 409.57 1000000 998.78
Time is seconds.
Ratio is how many probes required to lookup all values versus
an ideal hash.
Max is longest chain
Reported-by: Octavian Purdila <opurdila@ixiacom.com>
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/include/linux/dcache.h 2009-10-26 14:58:45.220347300 -0700
+++ b/include/linux/dcache.h 2009-10-26 15:12:15.004160122 -0700
@@ -45,15 +45,28 @@ struct dentry_stat_t {
};
extern struct dentry_stat_t dentry_stat;
-/* Name hashing routines. Initial hash value */
-/* Hash courtesy of the R5 hash in reiserfs modulo sign bits */
-#define init_name_hash() 0
+/*
+ * Fowler / Noll / Vo (FNV) Hash
+ * see: http://www.isthe.com/chongo/tech/comp/fnv/
+ */
+#ifdef CONFIG_64BIT
+#define FNV_PRIME 1099511628211ull
+#define FNV1_INIT 14695981039346656037ull
+#else
+#define FNV_PRIME 16777619u
+#define FNV1_INIT 2166136261u
+#endif
+
+#define init_name_hash() FNV1_INIT
-/* partial hash update function. Assume roughly 4 bits per character */
+/* partial hash update function. */
static inline unsigned long
-partial_name_hash(unsigned long c, unsigned long prevhash)
+partial_name_hash(unsigned char c, unsigned long prevhash)
{
- return (prevhash + (c << 4) + (c >> 4)) * 11;
+ prevhash ^= c;
+ prevhash *= FNV_PRIME;
+
+ return prevhash;
}
/*
next prev parent reply other threads:[~2009-10-26 22:36 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-10-25 19:58 [PATCH next-next-2.6] netdev: better dev_name_hash Octavian Purdila
2009-10-25 20:17 ` Hagen Paul Pfeifer
2009-10-25 21:24 ` Eric Dumazet
2009-10-25 21:55 ` Octavian Purdila
2009-10-25 22:41 ` Hagen Paul Pfeifer
2009-10-25 22:45 ` Octavian Purdila
2009-10-26 5:28 ` Eric Dumazet
2009-10-26 13:07 ` Krishna Kumar2
2009-10-26 14:31 ` Octavian Purdila
2009-10-26 14:55 ` Eric Dumazet
2009-10-26 15:52 ` Octavian Purdila
2009-10-26 16:55 ` Stephen Hemminger
2009-10-26 17:45 ` Stephen Hemminger
2009-10-27 1:24 ` David Miller
2009-10-27 1:40 ` Eric Dumazet
2009-10-26 6:30 ` Stephen Hemminger
2009-10-26 7:48 ` Eric Dumazet
2009-10-26 4:43 ` Stephen Hemminger
2009-10-26 22:36 ` Stephen Hemminger <shemminger@vyatta.com>, Al Viro [this message]
2009-10-27 2:45 ` [PATCH] dcache: better name hash function Eric Dumazet
2009-10-27 3:53 ` Stephen Hemminger
2009-10-27 16:38 ` Rick Jones
[not found] <9986527.24561256620662709.JavaMail.root@tahiti.vyatta.com>
2009-10-27 5:19 ` Stephen Hemminger
2009-10-27 5:24 ` David Miller
2009-10-27 6:07 ` Eric Dumazet
2009-10-27 6:50 ` Eric Dumazet
2009-10-27 7:29 ` Eric Dumazet
2009-10-27 17:07 ` Stephen Hemminger
2009-10-27 17:32 ` Linus Torvalds
2009-10-27 23:08 ` Stephen Hemminger
2009-10-27 23:41 ` Linus Torvalds
2009-10-28 0:10 ` Stephen Hemminger
2009-10-28 0:58 ` Linus Torvalds
2009-10-28 1:56 ` Stephen Hemminger
[not found] ` <4AE72B91.7040700@gmail.com>
2009-10-27 17:35 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20091026153656.25be4369@nehalam \
--to=viro@zeniv.linux.org.uk \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=opurdila@ixiacom.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).