* Great jffs2 speedup @ 2005-09-28 9:36 hinko.kocevar 2005-09-28 10:00 ` Jörn Engel 2005-09-29 8:21 ` Ferenc Havasi 0 siblings, 2 replies; 37+ messages in thread From: hinko.kocevar @ 2005-09-28 9:36 UTC (permalink / raw) To: Linux MTD Hi, We started using new CVS code that provides EBS and fragtree method. Below are some measurements of mount times for old and new setup. NAND partitions size tested were: flash4 - 4Mb flash5 - 12Mb flash6 - 16Mb OLD setup uses ~April 2005 MTD code, 2.6.11 kernel, no EBS and produces following: Mounting flash4 clean partition [1] real 0m 0.83s user 0m 0.01s sys 0m 0.83s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition [2] real 0m 8.37s user 0m 0.01s sys 0m 8.35s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.70s user 0m 0.02s sys 0m 2.38s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 26.42s user 0m 0.01s sys 0m 26.40s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.29s user 0m 0.02s sys 0m 3.16s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 35.81s user 0m 0.01s sys 0m 35.77s -- -- -- -- -- -- -- -- -- -- On new setup - using yesterdays MTD CVS and 2.6.12 kernel with EBS enabled: Mounting flash4 clean partition [1] real 0m 3.75s user 0m 0.02s sys 0m 1.17s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition [2] real 0m 0.75s user 0m 0.02s sys 0m 0.73s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.55s user 0m 0.02s sys 0m 3.47s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 2.12s user 0m 0.03s sys 0m 2.08s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 4.82s user 0m 0.01s sys 0m 4.63s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 2.82s user 0m 0.01s sys 0m 2.80s -- -- -- -- -- -- -- -- -- -- Compared to old setup this is GREAAAAAT improvment (if my mesurements are correct:). Great job! This holds true /only/ if partition is erased beforehand and then mounted, filled with files, umounted and mounted again to mesure mount time for dirty partition. Mounting jffs2 partition filled on old setup shows no performance improvement. We found sumtool utility in tools dir and if my understanding is correct it is used to convert jffs2 image with no EBS to jffs2 image with EBS. Is this the only way to upgrade 'old' jffs2 partitions already in place (on the nand flash) to use EBS? Thing is, we have several system up and running in the wild and would like to upgrade jffs2 partitions as-painless-as-possible, preferably without complete reflash of the systems. I've just ran sumtool on old raw 4Mb mtd char device (containing jffs2 fs) and it produced 4Mb image on the output. I figure, if I copy this image back to flash and test it with EBS supported kernel it should recognize it as new jffs2 image and mount it super-fast. I still need to erase the nand partition and gather enough ram for temporary EBS image on the system and so this could be a problem in our setup. regards, hinko [1] Mounting clean partition means that we erased the partition with flash_eraseall and then mounted it. mount time was measured with 'time'. [2] After cleaning up the partition, then creating, appending, erasing several small files on it, we umounted the partition and mounted it back. mount time was measured eith 'time'. All partitions contained 150 - 700 few kbytes sizes files and were ~80 % full when dirty. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-28 9:36 Great jffs2 speedup hinko.kocevar @ 2005-09-28 10:00 ` Jörn Engel 2005-09-28 15:26 ` hinko.kocevar 2005-09-30 12:26 ` hinko.kocevar 2005-09-29 8:21 ` Ferenc Havasi 1 sibling, 2 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-28 10:00 UTC (permalink / raw) To: hinko.kocevar@cetrtapot.si; +Cc: Linux MTD On Wed, 28 September 2005 11:36:13 +0200, hinko.kocevar@cetrtapot.si wrote: > > On new setup - using yesterdays MTD CVS and 2.6.12 kernel with EBS enabled: > Mounting flash4 clean partition [1] > real 0m 3.75s > user 0m 0.02s > sys 0m 1.17s > -- -- -- -- -- -- -- -- -- -- > Mounting flash4 dirty partition [2] > real 0m 0.75s > user 0m 0.02s > sys 0m 0.73s I assume above numbers are for a dirty partition and below for a clean one? Jörn -- Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. -- Perlis's Programming Proverb #58, SIGPLAN Notices, Sept. 1982 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-28 10:00 ` Jörn Engel @ 2005-09-28 15:26 ` hinko.kocevar 2005-09-29 7:53 ` Jörn Engel 2005-09-30 12:26 ` hinko.kocevar 1 sibling, 1 reply; 37+ messages in thread From: hinko.kocevar @ 2005-09-28 15:26 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD Jörn Engel wrote: > On Wed, 28 September 2005 11:36:13 +0200, hinko.kocevar@cetrtapot.si wrote: > >>On new setup - using yesterdays MTD CVS and 2.6.12 kernel with EBS enabled: >>Mounting flash4 clean partition [1] >>real 0m 3.75s >>user 0m 0.02s >>sys 0m 1.17s >>-- -- -- -- -- -- -- -- -- -- >>Mounting flash4 dirty partition [2] >>real 0m 0.75s >>user 0m 0.02s >>sys 0m 0.73s > > > I assume above numbers are for a dirty partition and below for a clean > one? No. I've used the same script - It seemed little weird, but ... I'll do more tests in days to come, hopefully I'll be able solve this too. regrds, hinko ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-28 15:26 ` hinko.kocevar @ 2005-09-29 7:53 ` Jörn Engel 0 siblings, 0 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-29 7:53 UTC (permalink / raw) To: hinko.kocevar@cetrtapot.si; +Cc: Linux MTD On Wed, 28 September 2005 17:26:05 +0200, hinko.kocevar@cetrtapot.si wrote: > Jörn Engel wrote: > >On Wed, 28 September 2005 11:36:13 +0200, hinko.kocevar@cetrtapot.si wrote: > > > >>On new setup - using yesterdays MTD CVS and 2.6.12 kernel with EBS > >>enabled: > >>Mounting flash4 clean partition [1] > >>real 0m 3.75s > >>user 0m 0.02s > >>sys 0m 1.17s > >>-- -- -- -- -- -- -- -- -- -- > >>Mounting flash4 dirty partition [2] > >>real 0m 0.75s > >>user 0m 0.02s > >>sys 0m 0.73s > > > > > >I assume above numbers are for a dirty partition and below for a clean > >one? > > No. > > I've used the same script - It seemed little weird, but ... > I'll do more tests in days to come, hopefully I'll be able solve this too. Then your results don't make sense to me. Either you made a stupid mistake when measuring this or you have found something quite interesting. Remember, most inventions don't start with the word "Heureka!" but rather with "This is odd!". Can anyone verify/falsify? Jörn -- To announce that there must be no criticism of the President, or that we are to stand by the President, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public. -- Theodore Roosevelt, Kansas City Star, 1918 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-28 10:00 ` Jörn Engel 2005-09-28 15:26 ` hinko.kocevar @ 2005-09-30 12:26 ` hinko.kocevar 1 sibling, 0 replies; 37+ messages in thread From: hinko.kocevar @ 2005-09-30 12:26 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 765 bytes --] Jörn Engel wrote: > On Wed, 28 September 2005 11:36:13 +0200, hinko.kocevar@cetrtapot.si wrote: > >>On new setup - using yesterdays MTD CVS and 2.6.12 kernel with EBS enabled: >>Mounting flash4 clean partition [1] >>real 0m 3.75s >>user 0m 0.02s >>sys 0m 1.17s >>-- -- -- -- -- -- -- -- -- -- >>Mounting flash4 dirty partition [2] >>real 0m 0.75s >>user 0m 0.02s >>sys 0m 0.73s > Here are more tests from our system. Setup is the same as before: system1: 2.6.11, April MTD, no EBS system2: 2.6.12, Latest CVS, EBS I haven't looked at the system2 clean/dirty mount times much closer, but It appears that dirty partition is mounted faster evrytime - hence several iterarions of the test. I also added my testing script... regards, hinko [-- Attachment #2: test_flash_2.6.11-CP_V3_08242005 --] [-- Type: text/plain, Size: 7710 bytes --] FLASH test v1.0 START @ Thu Sep 29 18:21:59 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 2.20s user 0m 0.03s sys 0m 0.55s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.28s user 0m 0.01s sys 0m 5.24s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.70s user 0m 0.02s sys 0m 1.62s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.40s user 0m 0.01s sys 0m 17.34s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.24s user 0m 0.02s sys 0m 2.17s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.29s user 0m 0.03s sys 0m 23.15s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 18:53:14 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 18:53:15 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.02s sys 0m 0.56s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.49s user 0m 0.02s sys 0m 5.44s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.75s user 0m 0.02s sys 0m 1.63s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.52s user 0m 0.01s sys 0m 17.45s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.20s user 0m 0.03s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.13s user 0m 0.01s sys 0m 23.02s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 19:24:11 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 19:24:11 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.59s user 0m 0.01s sys 0m 0.57s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.37s user 0m 0.03s sys 0m 5.32s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.73s user 0m 0.01s sys 0m 1.63s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.47s user 0m 0.02s sys 0m 17.37s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.24s user 0m 0.03s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.25s user 0m 0.03s sys 0m 23.13s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 19:54:47 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 19:54:48 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.01s sys 0m 0.57s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.52s user 0m 0.01s sys 0m 5.49s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.77s user 0m 0.01s sys 0m 1.63s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.29s user 0m 0.02s sys 0m 17.20s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.13s user 0m 0.02s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.14s user 0m 0.01s sys 0m 23.03s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 20:25:24 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 20:25:25 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.03s sys 0m 0.54s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.19s user 0m 0.03s sys 0m 5.15s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.69s user 0m 0.04s sys 0m 1.61s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.42s user 0m 0.01s sys 0m 17.34s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.98s user 0m 0.01s sys 0m 2.16s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.13s user 0m 0.01s sys 0m 23.04s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 20:55:58 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 20:55:58 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.02s sys 0m 0.56s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.41s user 0m 0.03s sys 0m 5.36s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.71s user 0m 0.02s sys 0m 1.63s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.43s user 0m 0.01s sys 0m 17.36s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.22s user 0m 0.02s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.17s user 0m 0.02s sys 0m 23.05s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 21:26:39 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 21:26:39 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.01s sys 0m 0.57s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.12s user 0m 0.03s sys 0m 5.08s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.71s user 0m 0.01s sys 0m 1.63s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.33s user 0m 0.02s sys 0m 17.22s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.21s user 0m 0.02s sys 0m 2.17s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.18s user 0m 0.03s sys 0m 23.08s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 21:57:22 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 21:57:23 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.02s sys 0m 0.56s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.26s user 0m 0.02s sys 0m 5.21s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.26s user 0m 0.02s sys 0m 1.62s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.35s user 0m 0.01s sys 0m 17.26s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.23s user 0m 0.02s sys 0m 2.17s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.21s user 0m 0.03s sys 0m 23.07s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 22:28:18 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 22:28:19 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.58s user 0m 0.01s sys 0m 0.57s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.25s user 0m 0.01s sys 0m 5.22s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.74s user 0m 0.01s sys 0m 1.66s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.21s user 0m 0.02s sys 0m 17.13s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.24s user 0m 0.01s sys 0m 2.18s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.13s user 0m 0.01s sys 0m 23.00s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 22:59:27 UTC 2005 FLASH test v1.0 START @ Thu Sep 29 22:59:27 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 2.20s user 0m 0.02s sys 0m 0.56s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 5.44s user 0m 0.03s sys 0m 5.38s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 1.76s user 0m 0.02s sys 0m 1.65s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 17.41s user 0m 0.03s sys 0m 17.28s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.19s user 0m 0.01s sys 0m 2.17s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 23.15s user 0m 0.01s sys 0m 23.04s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Thu Sep 29 23:30:24 UTC 2005 [-- Attachment #3: test_flash_2.6.12 --] [-- Type: text/plain, Size: 7670 bytes --] FLASH test v1.0 START @ Fri Sep 30 01:23:39 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.75s user 0m 0.02s sys 0m 0.73s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.49s user 0m 0.02s sys 0m 0.48s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.23s user 0m 0.02s sys 0m 2.16s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.44s user 0m 0.01s sys 0m 1.42s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.25s user 0m 0.02s sys 0m 2.85s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.90s user 0m 0.03s sys 0m 1.86s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 01:53:43 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 01:53:43 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.77s user 0m 0.03s sys 0m 0.74s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.48s user 0m 0.02s sys 0m 0.47s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.22s user 0m 0.01s sys 0m 2.16s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.50s user 0m 0.02s sys 0m 1.47s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.89s user 0m 0.02s sys 0m 2.86s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.85s user 0m 0.01s sys 0m 1.85s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 02:24:17 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 02:24:18 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 2.15s user 0m 0.02s sys 0m 0.73s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.49s user 0m 0.02s sys 0m 0.47s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.18s user 0m 0.02s sys 0m 2.13s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.45s user 0m 0.02s sys 0m 1.42s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.18s user 0m 0.02s sys 0m 2.86s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.85s user 0m 0.02s sys 0m 1.82s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 02:55:11 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 02:55:11 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 2.15s user 0m 0.02s sys 0m 0.73s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.50s user 0m 0.01s sys 0m 0.49s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.32s user 0m 0.02s sys 0m 2.14s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.52s user 0m 0.01s sys 0m 1.50s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.92s user 0m 0.01s sys 0m 2.84s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 2.02s user 0m 0.02s sys 0m 2.00s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 03:25:28 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 03:25:29 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.75s user 0m 0.01s sys 0m 0.74s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.51s user 0m 0.02s sys 0m 0.49s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.21s user 0m 0.01s sys 0m 2.14s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.43s user 0m 0.01s sys 0m 1.42s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.22s user 0m 0.03s sys 0m 2.84s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.87s user 0m 0.02s sys 0m 1.83s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 03:55:30 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 03:55:31 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.76s user 0m 0.02s sys 0m 0.74s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.48s user 0m 0.02s sys 0m 0.46s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.33s user 0m 0.02s sys 0m 2.14s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.44s user 0m 0.02s sys 0m 1.41s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 2.92s user 0m 0.01s sys 0m 2.85s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.88s user 0m 0.03s sys 0m 1.84s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 04:27:05 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 04:27:06 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.76s user 0m 0.01s sys 0m 0.74s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.49s user 0m 0.02s sys 0m 0.48s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.29s user 0m 0.04s sys 0m 2.12s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.43s user 0m 0.01s sys 0m 1.42s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.27s user 0m 0.02s sys 0m 2.87s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.89s user 0m 0.01s sys 0m 1.88s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 04:57:26 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 04:57:26 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.75s user 0m 0.03s sys 0m 0.72s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.49s user 0m 0.01s sys 0m 0.48s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 3.20s user 0m 0.01s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.44s user 0m 0.02s sys 0m 1.40s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.22s user 0m 0.03s sys 0m 2.84s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.91s user 0m 0.02s sys 0m 1.87s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 05:27:40 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 05:27:40 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.76s user 0m 0.02s sys 0m 0.73s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.49s user 0m 0.02s sys 0m 0.47s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.21s user 0m 0.02s sys 0m 2.14s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.42s user 0m 0.02s sys 0m 1.39s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.16s user 0m 0.02s sys 0m 2.85s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.88s user 0m 0.02s sys 0m 1.84s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 05:58:35 UTC 2005 FLASH test v1.0 START @ Fri Sep 30 05:58:36 UTC 2005 Nakljucno dodajanje datotek, brisanje datotek Mounting flash4 clean partition real 0m 0.75s user 0m 0.02s sys 0m 0.74s -- -- -- -- -- -- -- -- -- -- Mounting flash4 dirty partition real 0m 0.52s user 0m 0.01s sys 0m 0.51s -- -- -- -- -- -- -- -- -- -- Mounting flash5 clean partition real 0m 2.19s user 0m 0.01s sys 0m 2.15s -- -- -- -- -- -- -- -- -- -- Mounting flash5 dirty partition real 0m 1.45s user 0m 0.02s sys 0m 1.41s -- -- -- -- -- -- -- -- -- -- Mounting flash6 clean partition real 0m 3.25s user 0m 0.02s sys 0m 2.87s -- -- -- -- -- -- -- -- -- -- Mounting flash6 dirty partition real 0m 1.95s user 0m 0.02s sys 0m 1.92s -- -- -- -- -- -- -- -- -- -- FLASH test END @ Fri Sep 30 06:29:16 UTC 2005 [-- Attachment #4: test_flash.sh --] [-- Type: application/x-shellscript, Size: 3249 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-28 9:36 Great jffs2 speedup hinko.kocevar 2005-09-28 10:00 ` Jörn Engel @ 2005-09-29 8:21 ` Ferenc Havasi 2005-09-29 9:34 ` Jörn Engel 2005-09-29 10:15 ` hinko.kocevar 1 sibling, 2 replies; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 8:21 UTC (permalink / raw) To: hinko.kocevar@cetrtapot.si; +Cc: Linux MTD Hi Hinko, hinko.kocevar@cetrtapot.si wrote: > Compared to old setup this is GREAAAAAT improvment (if my mesurements > are correct:). Great job! Thanks. :) > We found sumtool utility in tools dir and if my understanding is > correct it is used to convert jffs2 image with no EBS to jffs2 image > with EBS. > > Is this the only way to upgrade 'old' jffs2 partitions already in > place (on the nand flash) to use EBS? Thing is, we have several system > up and running in the wild and would like to upgrade jffs2 partitions > as-painless-as-possible, preferably without complete reflash of the > systems. At this moment: yes, that is the only way. If more people need it we can implement a special extension for EBS. (but it takes time) This extension would make possible if user set a special mount option the system will convert all the non-summarizied erase blocks to summarized. It would take a lot of time (and need some percent free space), but only at that mount. Or you may try our other technique: CS (Centralized Summary). That technique does not requie user space tool, and cause very fast mount after any clean umount. (if the umount was not clean the speed will be same as before). It is not the part of the official JFFS2 now, we would like to little bit rewrite before bringing on it. But it works already - use the variant which stores the reference in the first erase block. You can download it from our webpage ( http://www.inf.u-szeged.hu/jffs2/ ). Bye, Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 8:21 ` Ferenc Havasi @ 2005-09-29 9:34 ` Jörn Engel 2005-09-29 9:44 ` Artem B. Bityutskiy 2005-09-29 9:52 ` Ferenc Havasi 2005-09-29 10:15 ` hinko.kocevar 1 sibling, 2 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-29 9:34 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si On Thu, 29 September 2005 10:21:22 +0200, Ferenc Havasi wrote: > > > > Is this the only way to upgrade 'old' jffs2 partitions already in > > place (on the nand flash) to use EBS? Thing is, we have several system > > up and running in the wild and would like to upgrade jffs2 partitions > > as-painless-as-possible, preferably without complete reflash of the > > systems. > > At this moment: yes, that is the only way. If more people need it we can > implement a special extension for EBS. (but it takes time) This > extension would make possible if user set a special mount option the > system will convert all the non-summarizied erase blocks to summarized. > It would take a lot of time (and need some percent free space), but only > at that mount. Just fyi - such a patch should not get merged into cvs. Long-term, the demand for it will approximate zero, so there is no good justification for extra code. > Or you may try our other technique: CS (Centralized Summary). That > technique does not requie user space tool, and cause very fast mount > after any clean umount. (if the umount was not clean the speed will be > same as before). It is not the part of the official JFFS2 now, we would > like to little bit rewrite before bringing on it. But it works already - > use the variant which stores the reference in the first erase block. You > can download it from our webpage ( http://www.inf.u-szeged.hu/jffs2/ ). How is the CS information found during mount? Special location? Jörn -- "Security vulnerabilities are here to stay." -- Scott Culp, Manager of the Microsoft Security Response Center, 2001 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:34 ` Jörn Engel @ 2005-09-29 9:44 ` Artem B. Bityutskiy 2005-09-29 9:52 ` Ferenc Havasi 1 sibling, 0 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-29 9:44 UTC (permalink / raw) To: Ferenc Havasi; +Cc: hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 2005-09-29 at 11:34 +0200, Jörn Engel wrote: > On Thu, 29 September 2005 10:21:22 +0200, Ferenc Havasi wrote: > > At this moment: yes, that is the only way. If more people need it we can > > implement a special extension for EBS. (but it takes time) This > > extension would make possible if user set a special mount option the > > system will convert all the non-summarizied erase blocks to summarized. > > It would take a lot of time (and need some percent free space), but only > > at that mount. > > Just fyi - such a patch should not get merged into cvs. Long-term, > the demand for it will approximate zero, so there is no good > justification for extra code. IMO, an external tool would be a good compromise. > > > Or you may try our other technique: CS (Centralized Summary). That > > technique does not requie user space tool, and cause very fast mount > > after any clean umount. (if the umount was not clean the speed will be > > same as before). It is not the part of the official JFFS2 now, we would > > like to little bit rewrite before bringing on it. But it works already - > > use the variant which stores the reference in the first erase block. You > > can download it from our webpage ( http://www.inf.u-szeged.hu/jffs2/ ). > > How is the CS information found during mount? Special location? You may try the approach that is described in the JFFS3 design. That would be cute if one implement it and check how it works :-)) -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:34 ` Jörn Engel 2005-09-29 9:44 ` Artem B. Bityutskiy @ 2005-09-29 9:52 ` Ferenc Havasi 2005-09-29 9:55 ` Jörn Engel 2005-09-29 9:59 ` Artem B. Bityutskiy 1 sibling, 2 replies; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 9:52 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si Jörn Engel wrote: >>At this moment: yes, that is the only way. If more people need it we can >>implement a special extension for EBS. (but it takes time) This >>extension would make possible if user set a special mount option the >>system will convert all the non-summarizied erase blocks to summarized. >>It would take a lot of time (and need some percent free space), but only >>at that mount. >> >> > >Just fyi - such a patch should not get merged into cvs. Long-term, >the demand for it will approximate zero, so there is no good >justification for extra code. > > Yes, I agree. >>Or you may try our other technique: CS (Centralized Summary). That >>technique does not requie user space tool, and cause very fast mount >>after any clean umount. (if the umount was not clean the speed will be >>same as before). It is not the part of the official JFFS2 now, we would >>like to little bit rewrite before bringing on it. But it works already - >>use the variant which stores the reference in the first erase block. You >>can download it from our webpage ( http://www.inf.u-szeged.hu/jffs2/ ). >> >> > >How is the CS information found during mount? Special location? > > There is two way (can be selected via kernel config): The first way stores a reference in the first (usable) erase block. (If it is not free, it moves the data to somewhere else, and reserve). It stores only a reference to it at umount, and a "mount-log" at mount (just to mark that it is invalid in case of unclean umount). It means one page write at mount, one page write at umount. If the erase block consist 16 pages than it is necesarry to erase the first erase block after every 8 mounts. So the typical timelife of it is about 8*100.000 mounting, I think it is big enough. The other way is to specify the cenratlized summary information with mount option (not to use the first erase block to store this pointer). In this case some other system have to store it (typically on an other file system), and the new location after umount can be read using sysfs. Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:52 ` Ferenc Havasi @ 2005-09-29 9:55 ` Jörn Engel 2005-09-29 9:59 ` Artem B. Bityutskiy 1 sibling, 0 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-29 9:55 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si On Thu, 29 September 2005 11:52:57 +0200, Ferenc Havasi wrote: > > > >How is the CS information found during mount? Special location? > > There is two way (can be selected via kernel config): > > The first way stores a reference in the first (usable) erase block. (If > it is not free, it moves the data to somewhere else, and reserve). It > stores only a reference to it at umount, and a "mount-log" at mount > (just to mark that it is invalid in case of unclean umount). It means > one page write at mount, one page write at umount. If the erase block > consist 16 pages than it is necesarry to erase the first erase block > after every 8 mounts. So the typical timelife of it is about 8*100.000 > mounting, I think it is big enough. > > The other way is to specify the cenratlized summary information with > mount option (not to use the first erase block to store this pointer). > In this case some other system have to store it (typically on an other > file system), and the new location after umount can be read using sysfs. Sounds sane. 100k clean unmounts should be fine for most people. I know a couple of testers that would immediatly create a simple while true; do mount mtd0 /mnt -t jffs2; umount /mnt; done testcase just to prove you wrong, but apart from that... Jörn -- To recognize individual spam features you have to try to get into the mind of the spammer, and frankly I want to spend as little time inside the minds of spammers as possible. -- Paul Graham ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:52 ` Ferenc Havasi 2005-09-29 9:55 ` Jörn Engel @ 2005-09-29 9:59 ` Artem B. Bityutskiy 2005-09-29 10:12 ` Jörn Engel ` (2 more replies) 1 sibling, 3 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-29 9:59 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si Ferenc Havasi wrote: > The first way stores a reference in the first (usable) erase block. (If > it is not free, it moves the data to somewhere else, and reserve). It > stores only a reference to it at umount, and a "mount-log" at mount > (just to mark that it is invalid in case of unclean umount). It means > one page write at mount, one page write at umount. If the erase block > consist 16 pages than it is necesarry to erase the first erase block > after every 8 mounts. So the typical timelife of it is about 8*100.000 > mounting, I think it is big enough. This spoils the overall wear-leveling, which is no good. But should work. > The other way is to specify the cenratlized summary information with > mount option (not to use the first erase block to store this pointer). > In this case some other system have to store it (typically on an other > file system), and the new location after umount can be read using sysfs. This also spoils wear-levelling... And I don't like CS in general as it is not tolerant to unclean reboots... Albeit, if this does not matter to a specific system - this will work. Nevertheless, I don't believe CS will make JFFS2 usable on 256MB flashes, due to memory consumprion issues. Nor it will not (IMO) make it scalable, which is sad :-( -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:59 ` Artem B. Bityutskiy @ 2005-09-29 10:12 ` Jörn Engel 2005-09-29 10:21 ` Artem B. Bityutskiy 2005-09-29 10:23 ` Ferenc Havasi 2005-09-29 16:10 ` Peter Grayson 2 siblings, 1 reply; 37+ messages in thread From: Jörn Engel @ 2005-09-29 10:12 UTC (permalink / raw) To: Artem B. Bityutskiy; +Cc: hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 29 September 2005 13:59:53 +0400, Artem B. Bityutskiy wrote: > Ferenc Havasi wrote: > >The first way stores a reference in the first (usable) erase block. (If > >it is not free, it moves the data to somewhere else, and reserve). It > >stores only a reference to it at umount, and a "mount-log" at mount > >(just to mark that it is invalid in case of unclean umount). It means > >one page write at mount, one page write at umount. If the erase block > >consist 16 pages than it is necesarry to erase the first erase block > >after every 8 mounts. So the typical timelife of it is about 8*100.000 > >mounting, I think it is big enough. > This spoils the overall wear-leveling, which is no good. But should work. Dummy argument. Wear levelling is just a means to avoid the real problem - early wear-out of specific blocks. Spending one erase block for cs is pretty sane, as long as this block doesn't wear-out well before the rest. > >The other way is to specify the cenratlized summary information with > >mount option (not to use the first erase block to store this pointer). > >In this case some other system have to store it (typically on an other > >file system), and the new location after umount can be read using sysfs. > This also spoils wear-levelling... How? > And I don't like CS in general as it is not tolerant to unclean > reboots... Albeit, if this does not matter to a specific system - this > will work. > > Nevertheless, I don't believe CS will make JFFS2 usable on 256MB > flashes, due to memory consumprion issues. Nor it will not (IMO) make it > scalable, which is sad :-( CS definitely doesn't solve all problems and it trades the mount time for additional writes - plus the special first erase block. Still, it may have its place. Jörn -- When in doubt, use brute force. -- Ken Thompson ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:12 ` Jörn Engel @ 2005-09-29 10:21 ` Artem B. Bityutskiy 2005-09-29 10:26 ` Jörn Engel 0 siblings, 1 reply; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-29 10:21 UTC (permalink / raw) To: Jörn Engel; +Cc: hinko.kocevar@cetrtapot.si, Linux MTD Jörn Engel wrote: > On Thu, 29 September 2005 13:59:53 +0400, Artem B. Bityutskiy wrote: >>>The first way stores a reference in the first (usable) erase block. (If >>>it is not free, it moves the data to somewhere else, and reserve). It >>>stores only a reference to it at umount, and a "mount-log" at mount >>>(just to mark that it is invalid in case of unclean umount). It means >>>one page write at mount, one page write at umount. If the erase block >>>consist 16 pages than it is necesarry to erase the first erase block >>>after every 8 mounts. So the typical timelife of it is about 8*100.000 >>>mounting, I think it is big enough. >> >>This spoils the overall wear-leveling, which is no good. But should work. > > > Dummy argument. Wear levelling is just a means to avoid the real > problem - early wear-out of specific blocks. Spending one erase block > for cs is pretty sane, as long as this block doesn't wear-out well > before the rest. "As long as" is the essential part. If it is true - nice. In general this can't be true. And, anticipating your remark, I agree that it still may have its place. :-) >>>The other way is to specify the cenratlized summary information with >>>mount option (not to use the first erase block to store this pointer). >>>In this case some other system have to store it (typically on an other >>>file system), and the new location after umount can be read using sysfs. >> >>This also spoils wear-levelling... > > > How? The same way as above unless you evenly rotete the CS eraseblock. And if ("as long as" == TRUE) then all is ok. > CS definitely doesn't solve all problems and it trades the mount time > for additional writes - plus the special first erase block. Still, it > may have its place. Thanks, I have no doubts. :-) -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:21 ` Artem B. Bityutskiy @ 2005-09-29 10:26 ` Jörn Engel 2005-09-29 11:34 ` Ferenc Havasi 0 siblings, 1 reply; 37+ messages in thread From: Jörn Engel @ 2005-09-29 10:26 UTC (permalink / raw) To: Artem B. Bityutskiy; +Cc: hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 29 September 2005 14:21:21 +0400, Artem B. Bityutskiy wrote: > Jörn Engel wrote: > > > >Dummy argument. Wear levelling is just a means to avoid the real > >problem - early wear-out of specific blocks. Spending one erase block > >for cs is pretty sane, as long as this block doesn't wear-out well > >before the rest. > "As long as" is the essential part. If it is true - nice. In general > this can't be true. And, anticipating your remark, I agree that it still > may have its place. :-) Agreed. In general, this patch opens an option for a DoS attack. Jörn -- Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. -- Doug MacIlroy ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:26 ` Jörn Engel @ 2005-09-29 11:34 ` Ferenc Havasi 2005-09-29 11:35 ` Jörn Engel 0 siblings, 1 reply; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 11:34 UTC (permalink / raw) To: Jörn Engel Cc: Artem B. Bityutskiy, hinko.kocevar@cetrtapot.si, Linux MTD Jörn Engel wrote: >On Thu, 29 September 2005 14:21:21 +0400, Artem B. Bityutskiy wrote: > > >>Jörn Engel wrote: >> >> >>>Dummy argument. Wear levelling is just a means to avoid the real >>>problem - early wear-out of specific blocks. Spending one erase block >>>for cs is pretty sane, as long as this block doesn't wear-out well >>>before the rest. >>> >>> >>"As long as" is the essential part. If it is true - nice. In general >>this can't be true. And, anticipating your remark, I agree that it still >>may have its place. :-) >> >> > >Agreed. In general, this patch opens an option for a DoS attack. > > Maybe I don't understand something. How makes CS possible DoS attacks? Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 11:34 ` Ferenc Havasi @ 2005-09-29 11:35 ` Jörn Engel 2005-09-29 11:45 ` Ferenc Havasi 0 siblings, 1 reply; 37+ messages in thread From: Jörn Engel @ 2005-09-29 11:35 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Artem B. Bityutskiy, hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 29 September 2005 13:34:44 +0200, Ferenc Havasi wrote: > > > Maybe I don't understand something. How makes CS possible DoS attacks? while true; do mount mtd0 /mnt -t jffs2; umount /mnt; done Run this for about a day, and the first block will be exhausted. I'm not saying this is easy for a malicious attacker, but certainly possible. Jörn -- Public Domain - Free as in Beer General Public - Free as in Speech BSD License - Free as in Enterprise Shared Source - Free as in "Work will make you..." ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 11:35 ` Jörn Engel @ 2005-09-29 11:45 ` Ferenc Havasi 2005-09-29 11:52 ` Jörn Engel 0 siblings, 1 reply; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 11:45 UTC (permalink / raw) To: Jörn Engel Cc: Artem B. Bityutskiy, hinko.kocevar@cetrtapot.si, Linux MTD Jörn Engel wrote: >On Thu, 29 September 2005 13:34:44 +0200, Ferenc Havasi wrote: > > >>Maybe I don't understand something. How makes CS possible DoS attacks? >> >> > >while true; do mount mtd0 /mnt -t jffs2; umount /mnt; done > >Run this for about a day, and the first block will be exhausted. I'm >not saying this is easy for a malicious attacker, but certainly >possible. > > Yes, it is true. But if an attacker already has the permission to do it, I think he has many simpler and faster way to crash the system. :) Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 11:45 ` Ferenc Havasi @ 2005-09-29 11:52 ` Jörn Engel 2005-09-29 12:39 ` Josh Boyer 0 siblings, 1 reply; 37+ messages in thread From: Jörn Engel @ 2005-09-29 11:52 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Artem B. Bityutskiy, hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 29 September 2005 13:45:14 +0200, Ferenc Havasi wrote: > > > Yes, it is true. But if an attacker already has the permission to do it, > I think he has many simpler and faster way to crash the system. :) Doesn't matter. If there is a good way to avoid this, we should do it. Problem is: Maybe there isn't. Jörn -- Ninety percent of everything is crap. -- Sturgeon's Law ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 11:52 ` Jörn Engel @ 2005-09-29 12:39 ` Josh Boyer 0 siblings, 0 replies; 37+ messages in thread From: Josh Boyer @ 2005-09-29 12:39 UTC (permalink / raw) To: Jörn Engel Cc: Artem B. Bityutskiy, Linux MTD, hinko.kocevar@cetrtapot.si On Thu, 2005-09-29 at 13:52 +0200, Jörn Engel wrote: > On Thu, 29 September 2005 13:45:14 +0200, Ferenc Havasi wrote: > > > > > Yes, it is true. But if an attacker already has the permission to do it, > > I think he has many simpler and faster way to crash the system. :) > > Doesn't matter. If there is a good way to avoid this, we should do > it. > > Problem is: Maybe there isn't. Depending on what your definition of DoS is, JFFS2 _without_ EBS or CS could be considered that already. If mounts take extremely long times on any decent sized flash, that could be perceived as a DoS. Think along the lines of "new ogg vorbis player with 4 GiB NAND flash using JFFS2 as the filesystem". You think people are going to want to wait for anything over 2 seconds for their music to be available? I know, I know... far fetched perhaps. But not any more than your mount/unmount loop :). josh ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:59 ` Artem B. Bityutskiy 2005-09-29 10:12 ` Jörn Engel @ 2005-09-29 10:23 ` Ferenc Havasi 2005-09-29 10:29 ` Jörn Engel 2005-09-29 10:45 ` Artem B. Bityutskiy 2005-09-29 16:10 ` Peter Grayson 2 siblings, 2 replies; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 10:23 UTC (permalink / raw) To: Artem B. Bityutskiy; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si Artem B. Bityutskiy wrote: > Ferenc Havasi wrote: > >> The first way stores a reference in the first (usable) erase block. (If >> it is not free, it moves the data to somewhere else, and reserve). It >> stores only a reference to it at umount, and a "mount-log" at mount >> (just to mark that it is invalid in case of unclean umount). It means >> one page write at mount, one page write at umount. If the erase block >> consist 16 pages than it is necesarry to erase the first erase block >> after every 8 mounts. So the typical timelife of it is about 8*100.000 >> mounting, I think it is big enough. > > This spoils the overall wear-leveling, which is no good. But should work. It concept of goodness is subjective. IMHO if someting has such a design what works well and fit to the concept of the real thing (the flash) that is good. >> The other way is to specify the cenratlized summary information with >> mount option (not to use the first erase block to store this pointer). >> In this case some other system have to store it (typically on an other >> file system), and the new location after umount can be read using sysfs. > > This also spoils wear-levelling... It think it is the method which is really does not spoil the wear-leveling. The selection of the storing place uses exactly the same function what JFFS2 uses to store any other data. > And I don't like CS in general as it is not tolerant to unclean > reboots... Albeit, if this does not matter to a specific system - this > will work. Yes, CS was designed for clean umounts. And it can be combinated with EBS, too. > Nevertheless, I don't believe CS will make JFFS2 usable on 256MB > flashes, due to memory consumprion issues. Nor it will not (IMO) make > it scalable, which is sad :-( Yes, it does not solve the memory problems of JFFS2. It is yet another mount time speed up. Bye, Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:23 ` Ferenc Havasi @ 2005-09-29 10:29 ` Jörn Engel 2005-09-29 10:45 ` Artem B. Bityutskiy 1 sibling, 0 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-29 10:29 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Artem B. Bityutskiy, hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 29 September 2005 12:23:08 +0200, Ferenc Havasi wrote: > Artem B. Bityutskiy wrote: > > > > This also spoils wear-levelling... > > It think it is the method which is really does not spoil the > wear-leveling. The selection of the storing place uses exactly the same > function what JFFS2 uses to store any other data. Fair enough. If not on its own, this should work in combination with EBH. Another problem introduced by this patch, btw, is an increased umount time. If you have to erase some blocks first, that could be substantial. Jörn -- The strong give up and move away, while the weak give up and stay. -- unknown ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:23 ` Ferenc Havasi 2005-09-29 10:29 ` Jörn Engel @ 2005-09-29 10:45 ` Artem B. Bityutskiy 2005-09-29 11:29 ` Ferenc Havasi 1 sibling, 1 reply; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-29 10:45 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si Ferenc Havasi wrote: > It think it is the method which is really does not spoil the > wear-leveling. The selection of the storing place uses exactly the same > function what JFFS2 uses to store any other data. Then I just don't undesstand how it works. A user selects a bolck X where CS ought to be kept. JFFS2 crereates CS in X. Then, on each mount user specifies X in mount options. And if there are too frequent mounts, A becomes worn-out earlier. To avoid this, user should specify 2 options - where to find CS and where to store it next time? And evenly rotate the CS eraseblock? Or you mean JFFS2 is itself picking the needed EB? How to find it on next mount then? -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:45 ` Artem B. Bityutskiy @ 2005-09-29 11:29 ` Ferenc Havasi 2005-09-29 11:32 ` Artem B. Bityutskiy 0 siblings, 1 reply; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 11:29 UTC (permalink / raw) To: Artem B. Bityutskiy; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si Artem B. Bityutskiy wrote: > Ferenc Havasi wrote: > >> It think it is the method which is really does not spoil the >> wear-leveling. The selection of the storing place uses exactly the same >> function what JFFS2 uses to store any other data. > > > Then I just don't undesstand how it works. A user selects a bolck X > where CS ought to be kept. JFFS2 crereates CS in X. Then, on each > mount user specifies X in mount options. And if there are too frequent > mounts, A becomes worn-out earlier. To avoid this, user should specify > 2 options - where to find CS and where to store it next time? And > evenly rotate the CS eraseblock? > > Or you mean JFFS2 is itself picking the needed EB? How to find it on > next mount then? Using this (second way) when the user umounts the fs, CS stores the centralized summary with the same function as JFFS2 stores nodes (using nextblock, etc) The starting location of it can be read from sysfs after umount. The user has to store it somewhere else, and specify it as a mount option at next mount. So nothing is stored in fixed place, so I think it does not spoil wear-leveling. This (second) way can be usefull if there is a small root partition with EBS and big home/usr/... with EBS+CS, and the location of the CS is stored on the root fs. It was /a requisition of our insturial partner. Ferenc / ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 11:29 ` Ferenc Havasi @ 2005-09-29 11:32 ` Artem B. Bityutskiy 0 siblings, 0 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-29 11:32 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD, hinko.kocevar@cetrtapot.si > Using this (second way) when the user umounts the fs, CS stores the > centralized summary with the same function as JFFS2 stores nodes (using > nextblock, etc) The starting location of it can be read from sysfs after > umount. The user has to store it somewhere else, and specify it as a > mount option at next mount. So nothing is stored in fixed place, so I > think it does not spoil wear-leveling. > Well, are you effectively talking about the situation where there is an additional persistent storage available, and the CS position is kept there. Fair enough, providing it is available. But it is often not :-( -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 9:59 ` Artem B. Bityutskiy 2005-09-29 10:12 ` Jörn Engel 2005-09-29 10:23 ` Ferenc Havasi @ 2005-09-29 16:10 ` Peter Grayson 2005-09-29 17:45 ` Sergei Sharonov 2 siblings, 1 reply; 37+ messages in thread From: Peter Grayson @ 2005-09-29 16:10 UTC (permalink / raw) To: Artem B. Bityutskiy; +Cc: hinko.kocevar@cetrtapot.si, Linux MTD On Thu, 2005-09-29 at 13:59 +0400, Artem B. Bityutskiy wrote: > Nevertheless, I don't believe CS will make JFFS2 usable on 256MB > flashes, due to memory consumprion issues. Nor it will not (IMO) make it > scalable, which is sad :-( CS does, in fact, go a long way toward making 256MB flash parts usable. The devices I am building use 256MB, 512MB, and 1024MB nand flash parts. We have been using Ferenc's CS patch for some time now and we are able to get sub 1 second mount times on 512+ MB filesystems. A big part of this performance is that I have a nand flash controller that interfaces with the nand part on my board. But nonetheless, jffs2 can be used successfully on very large nand parts. Ferenc's EBS and CS work are essential, IMHO, for getting there. Pete ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 16:10 ` Peter Grayson @ 2005-09-29 17:45 ` Sergei Sharonov 2005-09-30 4:19 ` Peter Grayson 0 siblings, 1 reply; 37+ messages in thread From: Sergei Sharonov @ 2005-09-29 17:45 UTC (permalink / raw) To: linux-mtd Hi, > CS does, in fact, go a long way toward making 256MB flash parts usable. Plese define usable. Yes, it does make flash available for read operations fast but the first time you try to write to it you will be blocked for, I'd say, anywhere from 5 minutes to a few hours depending on how the files were written and assuming full 256 MByte filesystem. > The devices I am building use 256MB, 512MB, and 1024MB nand flash parts. > We have been using Ferenc's CS patch for some time now and we are able > to get sub 1 second mount times on 512+ MB filesystems. Could you please provide time to first write for 256, 512 and 1024 MByte full filesystems? I presume MB stands for MByte, not Mbit ;-) > A big part of > this performance is that I have a nand flash controller that interfaces > with the nand part on my board. Good point.. Can you estimate speedup due to the hw controller? > But nonetheless, jffs2 can be used > successfully on very large nand parts. Ferenc's EBS and CS work are > essential, IMHO, for getting there. Most embedded systems cannot rely on proper shutdown. So, IMHO CS is of limited use in this respect. Sergei Sharonov ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 17:45 ` Sergei Sharonov @ 2005-09-30 4:19 ` Peter Grayson 2005-09-30 8:58 ` Artem B. Bityutskiy 2005-09-30 21:15 ` Sergei Sharonov 0 siblings, 2 replies; 37+ messages in thread From: Peter Grayson @ 2005-09-30 4:19 UTC (permalink / raw) To: Sergei Sharonov; +Cc: linux-mtd On Thu, 2005-09-29 at 17:45 +0000, Sergei Sharonov wrote: > Plese define usable. Yes, it does make flash available for read operations > fast but the first time you try to write to it you will be blocked for, I'd > say, anywhere from 5 minutes to a few hours depending on how the files > were written and assuming full 256 MByte filesystem. This has not been my experience. By usable I mean that a 500 MB filesystem with a couple hundred megabytes in use mounts and is usable for reads and writes in a few seconds. > Could you please provide time to first write for 256, 512 and 1024 MByte > full filesystems? I presume MB stands for MByte, not Mbit ;-) MB is megabyte. This is the time to mount a freshly erased partition: # time mount /mnt/mtdb2 real 0m 3.48s user 0m 0.00s sys 0m 1.78s I then exploded a kernel source tarball into the partition (this took about 2.5 minutes, but that accounts for the time to stream the tarball to my device via ethernet over USB. # df -h Filesystem Size Used Available Use% Mounted on /dev/mtdblock2 1020.0M 228.6M 791.4M 22% /mnt/mtdb2 # time umount /mnt/mtdb2 real 0m 0.73s user 0m 0.00s sys 0m 0.49s I was a little unsure about what you meant by "time to first write". How's this?: # time mount /mnt/mtdb2 && \ time echo "Hello world!" > /mnt/mtdb2/hello.txt real 0m 1.10s user 0m 0.00s sys 0m 1.03s real 0m 0.01s user 0m 0.00s sys 0m 0.00s # time umount /mnt/mtdb2 real 0m 0.56s user 0m 0.00s sys 0m 0.29s There is not any difference writing to an existing file: # time mount /mnt/mtdb2 && \ time echo "# some text" >> /mnt/mtdb2/linux-2.6.13.1/Makefile real 0m 1.10s user 0m 0.00s sys 0m 1.03s real 0m 0.01s user 0m 0.00s sys 0m 0.00s I then doubled the amount of data in the filesystem. # time cp -r /mnt/mtdb2/linux-2.6.13.1 /mnt/mtdb2/linux-2.6.13.1-2 real 2m 0.83s user 0m 5.17s sys 1m 7.80s # df -h Filesystem Size Used Available Use% Mounted on /dev/mtdblock2 1020.0M 431.2M 588.8M 42% /mnt/mtdb2 This yields a linear increase in unmount time, but in absolute terms it is still small. # time umount /mnt/mtdb2 real 0m 1.20s user 0m 0.00s sys 0m 0.78s Time to first write again. Mount time is worse: # time mount /mnt/mtdb2 && \ time echo "# some text" >> /mnt/mtdb2/linux-2.6.13.1/Makefile real 0m 3.60s user 0m 0.00s sys 0m 3.49s real 0m 0.01s user 0m 0.00s sys 0m 0.00s Let's double the amount of data in the filesystem again. You can see that the read and write times stay consistent: # time cp -r /mnt/mtdb2/linux-2.6.13.1 /mnt/mtdb2/linux-2.6.13.1-3 && \ time cp -r /mnt/mtdb2/linux-2.6.13.1 /mnt/mtdb2/linux-2.6.13.1-4 real 1m 59.50s user 0m 4.88s sys 1m 7.98s real 2m 2.69s user 0m 5.08s sys 1m 10.57s # df -h Filesystem Size Used Available Use% Mounted on /dev/mtdblock2 1020.0M 836.2M 183.8M 82% /mnt/mtdb2 Unmount time has stabilized. I've noticed that the garbage collector seems to add some variance to the unmount times. # time umount /mnt/mtdb2 real 0m 1.94s user 0m 0.00s sys 0m 1.21s TTFW take 3. Mount times get worse still: # time mount /mnt/mtdb2 && \ time echo "# some text" >> /mnt/mtdb2/linux-2.6.13.1/Makefile real 0m 16.74s user 0m 0.00s sys 0m 16.53s real 0m 0.01s user 0m 0.00s sys 0m 0.00s Hope that covers what you wanted to see. For the record, I am using the head of mtd cvs with the centralized summary patch applied. EBS and CS are both enabled. > Good point.. Can you estimate speedup due to the hw controller? We can get about 15 MB/s for raw reads and about 6 MB/s for raw writes. By raw, I mean reading and writing via an mtd character device to/from a raw partition. I do not know how these numbers compare to other flash access methods; but bit-banging a flash part directly would have to be considerably slower and probably very dependent on the CPU speed. My device has a 384 MHz PowerPC 405 CPU, by the way. Another advantage to this controller is that it does ECC in hardware and is capable of doing partial page reads. This means that even thought the nand flash part I am using has a 2048 byte page size, I can read as little as 256 bytes (ECC chunk size) at a time. One of the things that hurts jffs2 performance is that it very often reads small chunks that are only a fraction of the nand page size. This is why the nand_base driver caches pages in RAM. I implement a similar chunk caching scheme, but keep the chunks cached in my controller's buffers; this way I avoid extra copying of data. > Most embedded systems cannot rely on proper shutdown. So, IMHO CS is of > limited use in this respect. My device is usb powered, but has a small battery so that when the device looses bus power, it can shutdown cleanly. However, the thing to remember about centralized summary is that in the unclean shutdown cases, the mount time is no worse than jffs2 without CS. So even if you only expect the filesystem to be cleanly unmounted sometimes, CS can be worth it. Pete ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 4:19 ` Peter Grayson @ 2005-09-30 8:58 ` Artem B. Bityutskiy 2005-09-30 9:08 ` Artem B. Bityutskiy 2005-09-30 20:25 ` Peter Grayson 2005-09-30 21:15 ` Sergei Sharonov 1 sibling, 2 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-30 8:58 UTC (permalink / raw) To: Peter Grayson; +Cc: linux-mtd On Thu, 2005-09-29 at 22:19 -0600, Peter Grayson wrote: > I was a little unsure about what you meant by "time to first write". > How's this?: Could you please create a 200MB file 'dedekind' (or larger, the larger - the better), mount your FS, run 'time stat dedekind' and report how much time did that command stat took. Also, just after mount of a fully filled FS (again, the larger is your FS, the better), run top, and look how much time your cpu is busy by executing the GC thread. Basically, you can start writing only after the GC thread has stopped working. I am really curious about the results. And another fun is to create a directory with thousands of small (or empty) files, remount the FS and run stat on the directory or enter it. But this is not that fundamental problem and may be fixed. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 8:58 ` Artem B. Bityutskiy @ 2005-09-30 9:08 ` Artem B. Bityutskiy 2005-09-30 20:25 ` Peter Grayson 1 sibling, 0 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-09-30 9:08 UTC (permalink / raw) To: Peter Grayson; +Cc: linux-mtd On Fri, 2005-09-30 at 12:58 +0400, Artem B. Bityutskiy wrote: > And another fun is to create a directory with thousands of small (or > empty) files, remount the FS and run stat on the directory or enter it. > But this is not that fundamental problem and may be fixed. Oh, I'm actually lying, it is fundamental, but it may be relaxed greatly if we switch to balanced trees instead of lists in the internal directory representation. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 8:58 ` Artem B. Bityutskiy 2005-09-30 9:08 ` Artem B. Bityutskiy @ 2005-09-30 20:25 ` Peter Grayson 2005-10-01 7:01 ` Artem B. Bityutskiy 1 sibling, 1 reply; 37+ messages in thread From: Peter Grayson @ 2005-09-30 20:25 UTC (permalink / raw) To: dedekind; +Cc: linux-mtd On Fri, 2005-09-30 at 12:58 +0400, Artem B. Bityutskiy wrote: > Could you please create a 200MB file 'dedekind' (or larger, the larger - > the better), mount your FS, run 'time stat dedekind' and report how much > time did that command stat took. # time dd if=/dev/zero of=/mnt/mtdb2/dedekind bs=1M count=300 300+0 records in 300+0 records out real 1m 37.08s user 0m 0.01s sys 0m 40.64s # time umount /mnt/mtdb2 real 0m 0.45s user 0m 0.00s sys 0m 0.33s # time mount /mnt/mtdb2 && time stat /mnt/mtdb2/dedekind real 0m 0.18s user 0m 0.00s sys 0m 0.14s File: `/mnt/mtdb2/dedekind' Size: 314572800 Blocks: 614400 IO Block: 4096 regular file Device: 1f02h/7938d Inode: 2 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 17:01:29.000000000 -0700 Modify: 1969-12-31 17:03:06.000000000 -0700 Change: 1969-12-31 17:03:06.000000000 -0700 real 0m 9.50s user 0m 0.00s sys 0m 4.47s And again with a bigger file: # time dd if=/dev/zero of=/mnt/mtdb2/dedekind bs=1M count=600 600+0 records in 600+0 records out real 3m 6.93s user 0m 0.02s sys 1m 19.27s # time umount /mnt/mtdb2 real 0m 0.81s user 0m 0.00s sys 0m 0.58s # time mount /mnt/mtdb2 && time stat /mnt/mtdb2/dedekind real 0m 0.32s user 0m 0.00s sys 0m 0.26s File: `/mnt/mtdb2/dedekind' Size: 629145600 Blocks: 1228800 IO Block: 4096 regular file Device: 1f02h/7938d Inode: 3 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 17:19:56.000000000 -0700 Modify: 1969-12-31 17:19:56.000000000 -0700 Change: 1969-12-31 17:19:56.000000000 -0700 real 0m 18.72s user 0m 0.00s sys 0m 8.40s Yet bigger: # time dd if=/dev/zero of=/mnt/mtdb2/dedekind bs=1M count=900 900+0 records in 900+0 records out real 5m 28.55s user 0m 0.02s sys 1m 55.58s # time umount /mnt/mtdb2 real 0m 1.17s user 0m 0.00s sys 0m 0.83s # time mount /mnt/mtdb2 && time stat /mnt/mtdb2/dedekind real 0m 0.45s user 0m 0.00s sys 0m 0.36s File: `/mnt/mtdb2/dedekind' Size: 943718400 Blocks: 1843200 IO Block: 4096 regular file Device: 1f02h/7938d Inode: 4 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 17:30:46.000000000 -0700 Modify: 1969-12-31 17:30:46.000000000 -0700 Change: 1969-12-31 17:30:46.000000000 -0700 real 0m 28.31s user 0m 0.00s sys 0m 12.93s So, there seems to be a linear relationship between the time-to-stat and the file size. > Also, just after mount of a fully filled FS (again, the larger is your > FS, the better), run top, and look how much time your cpu is busy by > executing the GC thread. Basically, you can start writing only after the > GC thread has stopped working. I do not have the time to do this test right now. > I am really curious about the results. Me too! > And another fun is to create a directory with thousands of small (or > empty) files, remount the FS and run stat on the directory or enter it. > But this is not that fundamental problem and may be fixed. I'll try to make some time to do these tests next week. Pete ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 20:25 ` Peter Grayson @ 2005-10-01 7:01 ` Artem B. Bityutskiy 0 siblings, 0 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-10-01 7:01 UTC (permalink / raw) To: Peter Grayson; +Cc: linux-mtd Peter Grayson wrote: > # time dd if=/dev/zero of=/mnt/mtdb2/dedekind bs=1M count=300 Hmm, not bad with zero files, and now with if=/dev/urandom ? > So, there seems to be a linear relationship between the time-to-stat and > the file size. Neh, that's known fact and this is why we call it unscalable. > I do not have the time to do this test right now. That's not compulsory :-) That's for you, it may be interesting for you. > I'll try to make some time to do these tests next week. Yhank you! -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 4:19 ` Peter Grayson 2005-09-30 8:58 ` Artem B. Bityutskiy @ 2005-09-30 21:15 ` Sergei Sharonov 2005-09-30 23:22 ` Peter Grayson 1 sibling, 1 reply; 37+ messages in thread From: Sergei Sharonov @ 2005-09-30 21:15 UTC (permalink / raw) To: linux-mtd Peter, thanks for taking time to do the tests. > Hope that covers what you wanted to see. For the record, I am using the > head of mtd cvs with the centralized summary patch applied. EBS and CS > are both enabled. Aha!!! Of course, CS reduced your time to first write down to seconds. Could you repeat the test after the un-clean unmount (e.g. power off without sync)? I bet the numbers will be very different. Probably you also need to be writing to flash during the power off to force full scan on startup. > > Good point.. Can you estimate speedup due to the hw controller? > > We can get about 15 MB/s for raw reads and about 6 MB/s for raw writes. Hmm.. I was getting a read throughput jffs2 -> samba network fs of about 1.1 MB/s (180 MHz ARM9, linux 2.6.10, mtd 3/28/2005). I believe jffs2/mtd was the bottleneck since the throughput was 7.25x higher when files were comming from ext2/RAM. Looks like your hw controller is doing a good job. Not sure how overhead is split between mtd and jffs2. > By raw, I mean reading and writing via an mtd character device to/from a > raw partition. I do not know how these numbers compare to other flash > access methods; but bit-banging a flash part directly would have to be > considerably slower and probably very dependent on the CPU speed. My > device has a 384 MHz PowerPC 405 CPU, by the way. That helps too ;-) > > Most embedded systems cannot rely on proper shutdown. So, IMHO CS is of > > limited use in this respect. > > My device is usb powered, but has a small battery so that when the > device looses bus power, it can shutdown cleanly. Tricky, tricky, tricky ;-) Not everybody is that lucky to have a battery. Reminds me of a conversation with one "high reliability RTOS" vendor, you guess who. - Is your file system power fail safe? - Yes, of course. We do require that you use UPS though. > However, the thing to > remember about centralized summary is that in the unclean shutdown > cases, the mount time is no worse than jffs2 without CS. So even if you > only expect the filesystem to be cleanly unmounted sometimes, CS can be > worth it. As long as specification allows for an "occasional" 30 minutes startup time. Your hw controller does sound very interesting. What is the hardware implementation : asic, fpga? Is it available as a separate product (chip)? Sergei ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 21:15 ` Sergei Sharonov @ 2005-09-30 23:22 ` Peter Grayson 2005-10-01 7:43 ` Artem B. Bityutskiy 0 siblings, 1 reply; 37+ messages in thread From: Peter Grayson @ 2005-09-30 23:22 UTC (permalink / raw) To: Sergei Sharonov; +Cc: linux-mtd On Fri, 2005-09-30 at 21:15 +0000, Sergei Sharonov wrote: > Aha!!! Of course, CS reduced your time to first write down to seconds. > Could you repeat the test after the un-clean unmount (e.g. power off without > sync)? I bet the numbers will be very different. Probably you also need to > be writing to flash during the power off to force full scan on startup. I'll add this to my list of "things to try for the list". > Hmm.. I was getting a read throughput jffs2 -> samba network fs of about > 1.1 MB/s (180 MHz ARM9, linux 2.6.10, mtd 3/28/2005). I believe jffs2/mtd > was the bottleneck since the throughput was 7.25x higher when files were > comming from ext2/RAM. Looks like your hw controller is doing a good job. > Not sure how overhead is split between mtd and jffs2. The mtd layer does not really introduce any overhead; it accounts for the time it takes to get bytes in and out of the hardware. Jffs2, on the other hand, has a number of aspects that use RAM and cpu cycles. Also, the choices jffs2 makes affects how often it has to access flash. > Tricky, tricky, tricky ;-) > Not everybody is that lucky to have a battery. Reminds me of a conversation > with one "high reliability RTOS" vendor, you guess who. > - Is your file system power fail safe? > - Yes, of course. We do require that you use UPS though. > > > However, the thing to > > remember about centralized summary is that in the unclean shutdown > > cases, the mount time is no worse than jffs2 without CS. So even if you > > only expect the filesystem to be cleanly unmounted sometimes, CS can be > > worth it. > > As long as specification allows for an "occasional" 30 minutes startup time. Touche. > Your hw controller does sound very interesting. > What is the hardware implementation : asic, fpga? Is it available as a > separate product (chip)? This is a soft core in an fpga. My company may be willing to license its use. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-30 23:22 ` Peter Grayson @ 2005-10-01 7:43 ` Artem B. Bityutskiy 0 siblings, 0 replies; 37+ messages in thread From: Artem B. Bityutskiy @ 2005-10-01 7:43 UTC (permalink / raw) To: Peter Grayson; +Cc: linux-mtd On Fri, 2005-09-30 at 17:22 -0600, Peter Grayson wrote: > On Fri, 2005-09-30 at 21:15 +0000, Sergei Sharonov wrote: > > Aha!!! Of course, CS reduced your time to first write down to seconds. > > Could you repeat the test after the un-clean unmount (e.g. power off without > > sync)? I bet the numbers will be very different. Probably you also need to > > be writing to flash during the power off to force full scan on startup. > > I'll add this to my list of "things to try for the list". Mount should be anyway fast enough with EBS but w/o CS. But unfortunately: 1. your CPU will be busy for quite long time afterwords; 2. you will in general unable to write during that time - writers will be blocked (sometimes they may write, though). -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 8:21 ` Ferenc Havasi 2005-09-29 9:34 ` Jörn Engel @ 2005-09-29 10:15 ` hinko.kocevar 2005-09-29 12:01 ` Ferenc Havasi 2005-09-29 13:07 ` Jörn Engel 1 sibling, 2 replies; 37+ messages in thread From: hinko.kocevar @ 2005-09-29 10:15 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD Ferenc Havasi wrote: >>We found sumtool utility in tools dir and if my understanding is >>correct it is used to convert jffs2 image with no EBS to jffs2 image >>with EBS. >> >>Is this the only way to upgrade 'old' jffs2 partitions already in >>place (on the nand flash) to use EBS? Thing is, we have several system >>up and running in the wild and would like to upgrade jffs2 partitions >>as-painless-as-possible, preferably without complete reflash of the >>systems. > > > At this moment: yes, that is the only way. Ok, just to be on the right track. I guess we will have to come up with the solution using sumtool. > > Or you may try our other technique: CS (Centralized Summary). That > technique does not requie user space tool, and cause very fast mount > after any clean umount. All of our umounts are unclean - or said better application is not taking care of umount at all. System may be powered off anytime. Which raises another question: Will this impact mount time in long run? Eg. after more and more unclean (u)mounts should we expect significant performance degradation? regards, hinko ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:15 ` hinko.kocevar @ 2005-09-29 12:01 ` Ferenc Havasi 2005-09-29 13:07 ` Jörn Engel 1 sibling, 0 replies; 37+ messages in thread From: Ferenc Havasi @ 2005-09-29 12:01 UTC (permalink / raw) To: hinko.kocevar@cetrtapot.si; +Cc: Linux MTD hinko.kocevar@cetrtapot.si wrote: > All of our umounts are unclean - or said better application is not > taking care of umount at all. System may be powered off anytime. Which > raises another question: Will this impact mount time in long run? Eg. > after more and more unclean (u)mounts should we expect significant > performance degradation? Using EBS there should not be significant performance degradation even if there is no umount at all. Only CS needs clean umount. Ferenc ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Great jffs2 speedup 2005-09-29 10:15 ` hinko.kocevar 2005-09-29 12:01 ` Ferenc Havasi @ 2005-09-29 13:07 ` Jörn Engel 1 sibling, 0 replies; 37+ messages in thread From: Jörn Engel @ 2005-09-29 13:07 UTC (permalink / raw) To: hinko.kocevar@cetrtapot.si; +Cc: Linux MTD On Thu, 29 September 2005 12:15:49 +0200, hinko.kocevar@cetrtapot.si wrote: > > All of our umounts are unclean - or said better application is not > taking care of umount at all. System may be powered off anytime. Which > raises another question: Will this impact mount time in long run? Eg. > after more and more unclean (u)mounts should we expect significant > performance degradation? Only if your applications write to the fs in small chunks. Known problem, noone cared enough to actually fix it yet. If most writes are in the range of 4k, you're safe. Jörn -- My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. -- Edsger W. Dijkstra ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2005-10-01 7:44 UTC | newest] Thread overview: 37+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2005-09-28 9:36 Great jffs2 speedup hinko.kocevar 2005-09-28 10:00 ` Jörn Engel 2005-09-28 15:26 ` hinko.kocevar 2005-09-29 7:53 ` Jörn Engel 2005-09-30 12:26 ` hinko.kocevar 2005-09-29 8:21 ` Ferenc Havasi 2005-09-29 9:34 ` Jörn Engel 2005-09-29 9:44 ` Artem B. Bityutskiy 2005-09-29 9:52 ` Ferenc Havasi 2005-09-29 9:55 ` Jörn Engel 2005-09-29 9:59 ` Artem B. Bityutskiy 2005-09-29 10:12 ` Jörn Engel 2005-09-29 10:21 ` Artem B. Bityutskiy 2005-09-29 10:26 ` Jörn Engel 2005-09-29 11:34 ` Ferenc Havasi 2005-09-29 11:35 ` Jörn Engel 2005-09-29 11:45 ` Ferenc Havasi 2005-09-29 11:52 ` Jörn Engel 2005-09-29 12:39 ` Josh Boyer 2005-09-29 10:23 ` Ferenc Havasi 2005-09-29 10:29 ` Jörn Engel 2005-09-29 10:45 ` Artem B. Bityutskiy 2005-09-29 11:29 ` Ferenc Havasi 2005-09-29 11:32 ` Artem B. Bityutskiy 2005-09-29 16:10 ` Peter Grayson 2005-09-29 17:45 ` Sergei Sharonov 2005-09-30 4:19 ` Peter Grayson 2005-09-30 8:58 ` Artem B. Bityutskiy 2005-09-30 9:08 ` Artem B. Bityutskiy 2005-09-30 20:25 ` Peter Grayson 2005-10-01 7:01 ` Artem B. Bityutskiy 2005-09-30 21:15 ` Sergei Sharonov 2005-09-30 23:22 ` Peter Grayson 2005-10-01 7:43 ` Artem B. Bityutskiy 2005-09-29 10:15 ` hinko.kocevar 2005-09-29 12:01 ` Ferenc Havasi 2005-09-29 13:07 ` Jörn Engel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox