Re: near disk full performance (full 8TB)

[Marc Lehmann <schmorp@schmorp.de>]

On Fri, Oct 02, 2015 at 09:46:45AM -0700, Jaegeuk Kim <jaegeuk@kernel.org> wrote:
> Hmm, this is because of FS metadata flushes in background.
> I pushed one patch, can you get it through v3.18 branch?

I continued to write the same ~2TB data set to the disk, with the same
kernel module, giving me 85MB/s and 66MB/s throughput, respectively. This
was due to extended periods where write performance was around 30-40MB/s,
followed by shorter periods where it was >100MB/s. After using
"disable_ext_identify", this *seemed* to improve somewhat. I did this on
the theory that the zones near the end of the device (assuming f2fs would
roughly fill from the beginning) get larger, causing more pressure on
the disk (which has only 128MB of RAM to combine writes), but the result
wasn't conclusive either wayy.

For the fourth set, which wouldn't fully fit, I choose pdfs (larger
average filesize), and used the new kernel, either of which might have
helped.

I configured f2fs like this:

   echo 16 >ipu_policy
   echo 100 >min_ipu_util
   echo 100000 >reclaim_segments
   echo 1 >gc_idle
   echo   500 >gc_min_sleep_time
   echo 90000 >gc_max_sleep_time
   echo 30000 >gc_no_gc_sleep_time

Performance was ok'ish (as during the whole test) till about 200GB were
left out of 8.1 (metric) TB.

I started to make a trace around the 197GB mark:

http://data.plan9.de/f2fs.near_full.xz

status at beginning: http://ue.tst.eu/a4fc2a2522f3e372c7e92255cad1f3c3.txt
rsync was writing at this point, and I think you can see GC activity.

at 173851.953639, I ^S'ed rsync, which, due to -v, would cause it to pause
after a file. there was regular (probably GC) activity, but most of the
time, the disk was again idle, something I simply wouldn't expect from the
GC config (see next mail).

status after pausing rsync: http://ue.tst.eu/26c170d7d9f946d60926a5cdca814bbe.txt

I unpaused rsync at 174004.438302
status before unpausing rsync: http://ue.tst.eu/cc22fafa0efcb1cadae5a3849dff873b.txt

At 174186.324000, speed went down to ~2MB/s, and looking at the traces, it
seems f2fs is writing random 2MB segments, which would explain the speed.

I stopped at this point and started to prepare this mail. I could see
constant but very low activity afterwards, roughly every 30s.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp@schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\

------------------------------------------------------------------------------