Re: video archive on a microSD card

[Jaegeuk Kim <jaegeuk@kernel.org>]

On Thu, Aug 25, 2016 at 11:14:03PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> Thanks for all the help!
> 
> 24.08.2016, 00:12, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote:
> >>  Hi,
> >>
> >>  I ran the test over weekend and I think I have some interesting results.
> >>  I used 1 new SD card in one device and two fully utilized SD cards,
> >>  that have problems with write latency, on two oter devices.
> >>  I mounted all the cards with mode=lfs. The new SD card got utilized at 95% after some time.
> >>  Here is the current status after the archive is rotated for some time:
> >>
> >>  =====[ partition info(sda1). #0, RW]=====
> >>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 Resv:50)]
> >>
> >>  Utilization: 94% (6929763 valid blocks)
> >>    - Node: 8113 (Inode: 1255, Other: 6858)
> >>    - Data: 6921650
> >>    - Inline_xattr Inode: 0
> >>    - Inline_data Inode: 1
> >>    - Inline_dentry Inode: 0
> >>    - Orphan Inode: 0
> >>
> >>  Main area: 15112 segs, 7556 secs 7556 zones
> >>    - COLD data: 5306, 2653, 2653
> >>    - WARM data: 5233, 2616, 2616
> >>    - HOT data: 15100, 7550, 7550
> >>    - Dir dnode: 15097, 7548, 7548
> >>    - File dnode: 4701, 2350, 2350
> >>    - Indir nodes: 15105, 7552, 7552
> >>
> >>    - Valid: 97
> >>    - Dirty: 13798
> >>    - Prefree: 0
> >>    - Free: 1217 (604)
> >>
> >>  CP calls: 282 (BG: 0)
> >>  GC calls: 0 (BG: 0)
> >>    - data segments : 0 (0)
> >>    - node segments : 0 (0)
> >>  Try to move 0 blocks (BG: 0)
> >>    - data blocks : 0 (0)
> >>    - node blocks : 0 (0)
> >>
> >>  Extent Cache:
> >>    - Hit Count: L1-1:3084 L1-2:456 L2:0
> >>    - Hit Ratio: 4% (3540 / 84026)
> >>    - Inner Struct Count: tree: 1252(0), node: 0
> >>
> >>  Balancing F2FS Async:
> >>    - inmem: 0, wb_bios: 0
> >>    - nodes: 12 in 30
> >>    - dents: 3 in dirs: 2 ( 2)
> >>    - datas: 48 in files: 0
> >>    - meta: 9 in 34
> >>    - NATs: 10/ 249
> >>    - SITs: 13/ 15112
> >>    - free_nids: 1797
> >>
> >>  Distribution of User Blocks: [ valid | invalid | free ]
> >>    [-----------------------------------------------|--|-]
> >>
> >>  IPU: 0 blocks
> >>  SSR: 0 blocks in 0 segments
> >>  LFS: 912188 blocks in 1781 segments
> >>
> >>  BDF: 94, avg. vblocks: 996
> >>
> >>  Memory: 3604 KB
> >>    - static: 3270 KB
> >>    - cached: 78 KB
> >>    - paged : 256 KB
> >>
> >>  The interesting thing here is the very small number of valid and
> >>  a huge number of dirty sections. I don't understand this at all.
> >
> > This is the number of dirty segments, so it needs to consider section and
> > segment at the same time; a dirty section can consist of valid and free
> > segments.
> > How abouting testing 2MB-sized section which is the default option?
> 
> I tried what you said. Still the majority of segments are dirty for some reason:
> 
> =====[ partition info(sda). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
> 
> Utilization: 10% (3013093 valid blocks)
>   - Node: 3528 (Inode: 548, Other: 2980)
>   - Data: 3009565
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 0
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 59149 segs, 59149 secs 59149 zones
>   - COLD  data: 7183, 7183, 7183
>   - WARM  data: 6654, 6654, 6654
>   - HOT   data: 59134, 59134, 59134
>   - Dir   dnode: 59127, 59127, 59127
>   - File   dnode: 59125, 59125, 59125
>   - Indir nodes: 59129, 59129, 59129
> 
>   - Valid: 300
>   - Dirty: 6438
>   - Prefree: 0
>   - Free: 52411 (52411)
> 
> CP calls: 1023 (BG: 473)
> GC calls: 470 (BG: 470)
>   - data segments : 466 (466)
>   - node segments : 4 (4)
> Try to move 152221 blocks (BG: 152221)
>   - data blocks : 151417 (151417)
>   - node blocks : 804 (804)
> 
> Extent Cache:
>   - Hit Count: L1-1:6262 L1-2:0 L2:0
>   - Hit Ratio: 2% (6262 / 273606)
>   - Inner Struct Count: tree: 292(0), node: 8
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    0
>   - nodes:    0 in    0
>   - dents:    0 in dirs:   0 (   0)
>   - datas:    0 in files:   0
>   - meta:    0 in    0
>   - NATs:         0/       43
>   - SITs:         0/    59149
>   - free_nids:      3414
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----|--|-------------------------------------------]
> 
> IPU: 0 blocks
> SSR: 0 blocks in 0 segments
> LFS: 3691542 blocks in 7208 segments
> 
> BDF: 95, avg. vblocks: 444
> 
> Memory: 12662 KB
>   - static: 12597 KB
>   - cached: 64 KB
>   - paged : 0 KB
> 
> 
> But the archive is working perfectly as before.

Okay, so we need to gather more information about IO traces. :)

Could you get them by:

echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_write_bio/enable
echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_page_mbio/enable
echo 1 > /sys/kernel/debug/tracing/tracing_on
cat /sys/kernel/debug/tracing/trace_pipe

You can get a script in f2fs-tools.git/scripts/tracepoint.sh

Thanks,

> 
> >>  Still the archive is working perfectly. Also I see, that there GC is never
> >>  called, which is probably an indication of FS working exactly as
> >>  we expected.
> >>  Also the small number of cold sections does not make problems.
> >>  So, well, it works perfect so fat. But I don't understand everything here.
> >>  Is this expected?
> >
> > So, I'm in doubt that dirty sections consist of entirely valid or free segments.
> >
> >>  The other two SD cards were tested differently. On one of them I called
> >>  ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number
> >>  of dirty sectoins dropped considerably. It works fine so far.
> >
> > It makes sense that valid segments in dirty sections would be migrated to
> > different free sections.
> >
> >>  On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every
> >>  file in the archive. It works fine as well now. But the number of dirty sections
> >>  was still very high at the end of defragmentation. I don't understand this
> >>  as well.
> >
> > This is for defragementation to the given file, which would not move blocks in
> > order to decrease the number of dirty sections.
> > I think it's not necessary for this workload.
> 
> Yes, I'd prefer to not run manual GC or defragmentation. This was just a test.
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------