* JFFS2 empty file overhead questions
@ 2002-05-08 8:30 Jarkko Lavinen
2002-05-08 8:46 ` David Woodhouse
0 siblings, 1 reply; 3+ messages in thread
From: Jarkko Lavinen @ 2002-05-08 8:30 UTC (permalink / raw)
To: jffs-dev, MTD List; +Cc: David Woodhouse
Would anybody know exact JFFS2 file size overhead for empty file
and the data structures involved?
I tried to answer this by writing a simple test program that just
created as many empty files it could. The MTD partition size was 3904
KB. Initially after 5 spare block the used space is 320 and available
3584 KB.
My test program created a test directory and then in that one
directory creates files with 10 character long random names. I used
random names so that insertions into index tree would occur at random
places and the index tree should stay balanced and show average
insertion characteristics.
After creating 25531 the used space had grown to 3312 K. Initially
it had been 320K. This would suggest the empty file size overhead on
JFFS2 is (3312 - 320)*1024 / 25531 = 120.00 bytes.
So I assume the empty file overhead is 120 bytes on JFFS2. But where
that 120 bytes comes from?
I also tracked the file creation time. This showed linear first degree
dependence on the number of files created. On that particular device
empty file creation (file open) time can be presented as a function
t_c over the number of created files so far:
t_c(n) := 0.067 + 0.00013*n [seconds]
The file close time that followed was always 0.
After 25519 files I couldn't track space usage with "df" anymore because
it started to give out of memory message. After 26647 my test program
couldn't run anymore because of out of memory. Needles to say, but
my test program doesn't allocate memory nor leave files open when it
proceeds.
Jarkko Lavinen
PS. Test program available on request.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: JFFS2 empty file overhead questions
2002-05-08 8:30 JFFS2 empty file overhead questions Jarkko Lavinen
@ 2002-05-08 8:46 ` David Woodhouse
2002-05-14 8:48 ` Jarkko Lavinen
0 siblings, 1 reply; 3+ messages in thread
From: David Woodhouse @ 2002-05-08 8:46 UTC (permalink / raw)
To: Jarkko Lavinen; +Cc: jffs-dev, MTD List
jlavi@iki.fi said:
> After creating 25531 the used space had grown to 3312 K. Initially it
> had been 320K. This would suggest the empty file size overhead on
> JFFS2 is (3312 - 320)*1024 / 25531 = 120.00 bytes.
Sounds about right.
((sizeof(struct jffs2_raw_dirent) + strlen(name) + 3) & ~3)
+ sizeof(struct jffs2_raw_inode)
Where sizeof(jffs2_raw_dirent) == 40, sizeof(jffs2_raw_inode) == 68.
> I also tracked the file creation time. This showed linear first
> degree dependence on the number of files created. On that particular
> device empty file creation (file open) time can be presented as a
> function t_c over the number of created files so far:
That one I can probably settle without seeing a profile -- we look through a
single (but sorted) linked list in jffs2_lookup(). We could consider
optimising that if it's a problem, but I suspect it's not in the real world.
> After 25519 files I couldn't track space usage with "df" anymore
> because it started to give out of memory message. After 26647 my test
> program couldn't run anymore because of out of memory. Needles to say,
> but my test program doesn't allocate memory nor leave files open when
> it proceeds.
cat /proc/slabinfo /proc/meminfo /proc/sys/fs/inode-nr
--
dwmw2
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: JFFS2 empty file overhead questions
2002-05-08 8:46 ` David Woodhouse
@ 2002-05-14 8:48 ` Jarkko Lavinen
0 siblings, 0 replies; 3+ messages in thread
From: Jarkko Lavinen @ 2002-05-14 8:48 UTC (permalink / raw)
To: David Woodhouse
> > After 25519 files I couldn't track space usage with "df" anymore
> > because it started to give out of memory message. After 26647 my test
> > program couldn't run anymore because of out of memory.
> cat /proc/slabinfo /proc/meminfo /proc/sys/fs/inode-nr
After 25000 files created slabinfo and meminfo are following. Sys/fs/i-node-nr
was not available. This test was still with the old JFFS2 code from February.
I am now rerunning some of my tests with fresh code from CVS jffs2-2_4-branch.
slabinfo - version: 1.1
kmem_cache 62 68 112 2 2 1
tcp_tw_bucket 0 0 96 0 0 1
tcp_bind_bucket 0 0 32 0 0 1
tcp_open_request 0 0 64 0 0 1
inet_peer_cache 0 0 64 0 0 1
ip_fib_hash 0 0 32 0 0 1
ip_dst_cache 0 0 160 0 0 1
arp_cache 0 0 128 0 0 1
blkdev_requests 64 80 96 2 2 1
jffs2_inode_cache 25002 25012 20 148 148 1
jffs2_node_frag 0 0 20 0 0 1
jffs2_raw_node_ref 50063 50096 16 248 248 1
jffs2_tmp_dnode 0 0 12 0 0 1
jffs2_raw_inode 0 0 68 0 0 1
jffs2_raw_dirent 0 92 40 0 1 1
jffs2_full_dnode 1 202 16 1 1 1
dnotify cache 0 0 20 0 0 1
file lock cache 0 0 92 0 0 1
fasync cache 0 0 16 0 0 1
uid_cache 0 0 32 0 0 1
skbuff_head_cache 0 0 160 0 0 1
sock 2 9 832 1 1 2
sigqueue 0 29 132 0 1 1
cdev_cache 1 59 64 1 1 1
bdev_cache 2 59 64 1 1 1
mnt_cache 9 59 64 1 1 1
inode_cache 31 49 512 7 7 1
dentry_cache 29 60 128 2 2 1
filp 16 30 128 1 1 1
names_cache 0 1 4096 0 1 1
buffer_head 264 440 96 11 11 1
mm_struct 5 30 128 1 1 1
vm_area_struct 48 80 96 2 2 1
fs_cache 4 59 64 1 1 1
files_cache 4 9 416 1 1 1
signal_act 7 9 1312 3 3 1
pte-cache 15 16 2048 8 8 1
size-131072(DMA) 0 0 131072 0 0 32
size-131072 0 0 131072 0 0 32
size-65536(DMA) 0 0 65536 0 0 16
size-65536 0 0 65536 0 0 16
size-32768(DMA) 0 0 32768 0 0 8
size-32768 0 0 32768 0 0 8
size-16384(DMA) 0 0 16384 0 0 4
size-16384 0 0 16384 0 0 4
size-8192(DMA) 0 0 8192 0 0 2
size-8192 0 0 8192 0 0 2
size-4096(DMA) 0 0 4096 0 0 1
size-4096 2 2 4096 2 2 1
size-2048(DMA) 0 0 2048 0 0 1
size-2048 1 2 2048 1 1 1
size-1024(DMA) 0 0 1024 0 0 1
size-1024 1 4 1024 1 1 1
size-512(DMA) 0 0 512 0 0 1
size-512 9 16 512 2 2 1
size-256(DMA) 0 0 256 0 0 1
size-256 3 15 256 1 1 1
size-128(DMA) 0 0 128 0 0 1
size-128 52 60 128 2 2 1
size-64(DMA) 0 0 64 0 0 1
size-64 25009 25016 64 424 424 1
size-32(DMA) 0 0 32 0 0 1
size-32 32 113 32 1 1 1
total: used: free: shared: buffers: cached:
Mem: 7012352 6922240 90112 0 24576 2490368
Swap: 0 0 0
MemTotal: 6848 kB
MemFree: 88 kB
MemShared: 0 kB
Buffers: 24 kB
Cached: 2432 kB
SwapCached: 0 kB
Active: 2264 kB
Inactive: 680 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 6848 kB
LowFree: 88 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Jarkko Lavinen
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2002-05-14 8:50 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-05-08 8:30 JFFS2 empty file overhead questions Jarkko Lavinen
2002-05-08 8:46 ` David Woodhouse
2002-05-14 8:48 ` Jarkko Lavinen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox