From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jaegeuk Kim Subject: RE: [PATCH 00/16] f2fs: introduce flash-friendly file system Date: Fri, 12 Oct 2012 23:25:36 +0900 Message-ID: <1350051936.2299.29.camel@kjgkr> References: <415E76CC-A53D-4643-88AB-3D7D7DC56F98@dubeyko.com> <9DE65D03-D4EA-4B32-9C1D-1516EAE50E23@dubeyko.com> <1349553966.12699.132.camel@kjgkr> <50712AAA.5030807@gmail.com> <002201cda46e$88b84d30$9a28e790$%kim@samsung.com> <004101cda52e$72210e20$56632a60$%kim@samsung.com> <55A93BD0-CBCB-4707-A970-EB823EC54B2D@dubeyko.com> <006f01cda5ec$e63e9b60$b2bbd220$%kim@samsung.com> <1349855868.1889.87.camel@slavad-ubuntu> <009f01cda6cb$b8cb1020$2a613060$%kim@samsung.com> <1350045007.2026.97.camel@slavad-ubuntu> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Jaegeuk Kim , 'Marco Stornelli' , 'Al Viro' , tytso@mit.edu, gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, chur.lee@samsung.com, cm224.lee@samsung.com, jooyoung.hwang@samsung.com, linux-fsdevel@vger.kernel.org To: Vyacheslav Dubeyko Return-path: In-Reply-To: <1350045007.2026.97.camel@slavad-ubuntu> Sender: linux-kernel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org 2012-10-12 (=EA=B8=88), 16:30 +0400, Vyacheslav Dubeyko: > On Wed, 2012-10-10 at 18:43 +0900, Jaegeuk Kim wrote: > > [snip] > > > > How about the following scenario? > > > > 1. data "a" is newly written. > > > > 2. checkpoint "A" is done. > > > > 3. data "a" is truncated. > > > > 4. checkpoint "B" is done. > > > > > > > > If fs supports multiple snapshots like "A" and "B" to users, it= cannot reuse the space allocated by > > > > data "a" after checkpoint "B" even though data "a" is safely tr= uncated by checkpoint "B". > > > > This is because fs should keep data "a" to prepare a roll-back = to "A". > > > > So, even though user sees some free space, LFS may suffer from = cleaning due to the exhausted free > > > space. > > > > If users want to avoid this, they have to remove snapshots by t= hemselves. Or, maybe automatically? > > > > > > >=20 > > > I feel that here it exists some misunderstanding in checkpoint/sn= apshot terminology (especially, for > > > the NILFS2 case). It is possible that NILFS2 volume can contain o= nly checkpoints (if user doesn't > > > created any snapshot). You are right, snapshot cannot be deleted = because, in other word, user marked > > > this file system state as important point. But checkpoints can be= reclaimed easily. I can't see any > > > problem to reclaim free space from checkpoints in above-mentioned= scenario in the case of NILFS2. But > >=20 > > I meant that snapshot does checkpoint. > > And, the problem is related to real file system utilization managed= by NILFS2. > > [fs utilization to users] [fs utilization ma= naged by NILFS2] > > X - 1 X - 1 > > 1. new data "a" X X > > 2. snapshot "A" X X > > 3. truncate "a" X - 1 X > > 4. snapshot "B" X - 1 X > >=20 > > After this, user can see X-1, but the performance will be affected = by X. > > Until the snapshot "A" is removed, user will experience the perform= ance determined by X. > > Do I misunderstand? > >=20 >=20 > Ok. Maybe I have some misunderstanding but checkpoint and snapshot ar= e different things for me (especially, in the case of NILFS2). :-) >=20 > The most important is that f2fs has more efficient scheme of working = with checkpoints, from your point of view. If you are right then it is = very good. And I need to be more familiar with f2fs code. >=20 Ok, thanks. > [snip] > > > As I know, NILFS2 has Garbage Collector that removes checkpoints = automatically in background. But it > > > is possible also to force removing as checkpoints as snapshots by= hands with special utility using. As > >=20 > > If users may not want to remove the snapshots automatically, should= they configure not to do this too? > >=20 >=20 > As I know, NILFS2 doesn't delete snapshots automatically but checkpoi= nts - yes. Moreover, it exists nilfs_cleanerd.conf configuration file t= hat makes possible to manage by NILFS cleanerd daemon's behavior (min/m= ax number of clean segments, selection policy, check/clean intervals an= d so on). >=20 Ok. > [snip] > > > > IMHO, user does not need to know how many snapshots there exist= and track the fs utilization all the > > > time. > > > > (off list: I don't know why cleaning process should be tuned by= users.) > > > > > > >=20 > > > What do you plan to do in the case of users' complains about issu= es with free space reclaiming? If > > > user doesn't know about checkpoints and haven't any tools for acc= essing to checkpoints then how is it > > > possible to investigate issues with free space reclaiming on an u= ser side? > >=20 > > Could you explain why reclaiming free space is an issue? > > IMHO, that issue is caused by adopting multiple snapshots. > >=20 >=20 > I didn't mean that reclaiming free space is an issue. I hope that f2f= s > is stable but unfortunately it is not possible for any software to be > completely without bugs. So, anyway, f2fs users can have some issues > during using. One of the possible issue can be unexpected situation > with not reclaiming of free space. So, my question was about > possibility to investigate such bug on the user's side. From my point > of view, NILFS2 has very good utilities for such investigation. You mean fsck? Of course, we've implemented fsck tool also. But, why I didn't open it is that code is a mess. Another reason is that current fsck tool only checks the consistency of f2fs. Now we're still working on it to open. >=20 > [snip] > > > > In our experiments *also* on android phones, we've seen many ra= ndom patterns with frequent fsync > > > calls. > > > > We found that the main problem is database, and I think f2fs is= beneficial to this. > > >=20 > > > I think that database is not main use-case on Android phones. The= dominating use-case can be operation > > > by multimedia information and operations with small files, from m= y point of view. > > >=20 > > > So, it is possible to extract such key points from the shared pap= er: (1) file has complex structure; > > > (2) sequential access is not sequential; (3) auxiliary files domi= nate; (4) multiple threads perform > > > I/O. > > >=20 > > > I am afraid that random modification of different part of files a= nd I/O operations from multiple > > > threads can lead to significant fragmentation as file fragments a= s directory meta-information because > > > of garbage collection. > >=20 > > Could you explain in more detail? > >=20 >=20 > I mean that complex structure of modern files can lead to random modi= fication of small file's parts. > Moreover, such modifications can occur from multiple threads. > So, it means for me that Copy-On-Write policy can lead to file's cont= ent fragmentation. > Then GC can make additional fragmentation also. > But maybe I have some misunderstanding of f2fs internal techniques. >=20 Right. Random modification may cause data fragmentation due to COW in L= =46S. But, this is from the host side view only. If we consider FTL with file system adopting the in-place-update scheme= , eventually FTL should handle the fragmentation issue instead of file system. So, I think fragmentation is not a particular issue in LFS only. > With the best regards, > Vyacheslav Dubeyko. >=20 >=20 --=20 Jaegeuk Kim Samsung