[Bug 201911] New: High frequency of gc cause application database corrupt

[Bug 201911] New: High frequency of gc cause application database corrupt

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=201911

            Bug ID: 201911
           Summary: High frequency of gc cause application database
                    corrupt
           Product: File System
           Version: 2.5
    Kernel Version: 4.9
          Hardware: ARM
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: high
          Priority: P1
         Component: f2fs
          Assignee: filesystem_f2fs@kernel-bugs.kernel.org
          Reporter: liu_shuang_lin@163.com
        Regression: No

Hello, 

Android phone with kernel 4.9, f2fs version based on 2018-06-04 version with
commit description "f2fs: run fstrim asynchronously if runtime discard is on",

fill 2G space totaly, with a 4K small file each, and then delete 1G,

set gc_urgent_sleep_time to 300, and gc_urgent to 1.

then do monkey test,

40% cause application database corrupt, below is main log error info: 

----------------------------------------
11-25 21:42:27.874 21412 21412 W System.err: Application uncaught exception in
thread "main" com.tencent.wcdb.database.SQLiteDatabaseCorruptException:
database disk image is malformed (code 11, errno 0): 
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteConnection.nativeExecuteForLastInsertedRowId(Native
Method)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteConnection.executeForLastInsertedRowId(SourceFile:887)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteSession.executeForLastInsertedRowId(SourceFile:742)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteStatement.executeInsert(SourceFile:127)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteStatement.executeInsert(SourceFile:110)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteDatabase.insertWithOnConflict(SourceFile:1678)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.wcdb.database.SQLiteDatabase.insert(SourceFile:1528)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.cg.f.insert(SourceFile:645)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.cg.h.a(SourceFile:561)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.storage.bj.b(SourceFile:1583)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.modelmulti.h.<init>(SourceFile:120)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.ui.chatting.b.ah$1.run(SourceFile:176)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.sdk.platformtools.ao.run(SourceFile:127)
11-25 21:42:27.874 21412 21412 W System.err:    at
android.os.Handler.handleCallback(Handler.java:873)
11-25 21:42:27.874 21412 21412 W System.err:    at
android.os.Handler.dispatchMessage(Handler.java:99)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.tencent.mm.sdk.platformtools.aj.dispatchMessage(SourceFile:122)
11-25 21:42:27.874 21412 21412 W System.err:    at
android.os.Looper.loop(Looper.java:217)
11-25 21:42:27.874 21412 21412 W System.err:    at
android.app.ActivityThread.main(ActivityThread.java:7351)
11-25 21:42:27.874 21412 21412 W System.err:    at
java.lang.reflect.Method.invoke(Native Method)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:499)
11-25 21:42:27.874 21412 21412 W System.err:    at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:865)

----------------------------------------

in comparison, if decrease gc frequency, the corrupt frequency also decreased.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 201911] High frequency of gc cause application database corrupt

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=201911

Chao Yu (chao@kernel.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |chao@kernel.org

--- Comment #1 from Chao Yu (chao@kernel.org) ---
Sounds like that the problem was caused by GC, so could you merge below two
commits and retry the test?

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/commit/?h=linux-4.9.y&id=1d8aac3a98acaeb102083074b67da08ec9e9d440

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/commit/?h=linux-4.9.y&id=787485b8082e2271a01fab6f0bf267680e381bac

Or in order to make sure whether there is the same issue in last 4.9 version
kernel of AOSP, it will be better to use last f2fs codes in below branch:

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/log/?h=linux-4.9.y

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 201911] High frequency of gc cause application database corrupt

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=201911

--- Comment #2 from liushuanglin (liu_shuang_lin@163.com) ---
Hi, Yu, 

Tried two patches you mentioned, still reproduced this issue.

will try latest f2fs codes.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 201911] High frequency of gc cause application database corrupt

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=201911

Jaegeuk Kim (jaegeuk@kernel.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jaegeuk@kernel.org

--- Comment #3 from Jaegeuk Kim (jaegeuk@kernel.org) ---
I suspect it's caused by HW encryption.

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/commit/?h=linux-4.9.y&id=3d571474ab517bc05cd79e91735861c7e7918681

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=dev&id=9bf1a3f73927492c8be127b642197125e9d52be8

-- 
You are receiving this mail because:
You are watching the assignee of the bug.