git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* On the many files problem
@ 2007-12-29 18:22 Yannick Gingras
  2007-12-29 19:12 ` Linus Torvalds
  2007-12-29 19:27 ` Junio C Hamano
  0 siblings, 2 replies; 6+ messages in thread
From: Yannick Gingras @ 2007-12-29 18:22 UTC (permalink / raw)
  To: git


Greetings Git hackers,

No doubt, you guys must have discussed this problem before but I will
pretend that I can't find the relevant threads in the archive because
Marc's search is kind of crude.

I'm coding an application that will potentially store quite a bunch of
files in the same directory so I wondered how I should do it.  I tried
a few different files systems and I tried path hashing, that is,
storing the file that hashes to d3b07384d113 in d/d3/d3b07384d113.  As
far as I can tell, that's what Git does.  It turned out to be slower
than anything except ext3 without dir_index.  You can see my results
and the benchmarking code that I used here:

  http://ygingras.net/b/2007/12/too-many-files:-reiser-fs-vs-hashed-paths

Quick like that, I would be tempted to say that hashing paths always
makes things slower but the Git development team includes people with
really intimate knowledge of several file system implementations so
I'm tempted to say that you guys know something that I don't.

Can you describe how you hash the paths and what trick is done to
ensure fast creating and access to the subdirectories?  Is path
hashing generally faster or are you trying to avoid problems for
people using git on baroque file systems?

Best regards, 

-- 
Yannick Gingras

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2007-12-31 23:31 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-12-29 18:22 On the many files problem Yannick Gingras
2007-12-29 19:12 ` Linus Torvalds
2007-12-31 10:13   ` Yannick Gingras
2007-12-31 20:45     ` Linus Torvalds
2007-12-31 23:31     ` Martin Langhoff
2007-12-29 19:27 ` Junio C Hamano

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).