From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.sourceforge.net (lists.sourceforge.net [216.105.38.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1AECAC54EE9 for ; Thu, 22 Sep 2022 11:14:10 +0000 (UTC) Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1obK9S-0004D7-8t; Thu, 22 Sep 2022 11:14:06 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1obK9P-0004Cx-IT for linux-f2fs-devel@lists.sourceforge.net; Thu, 22 Sep 2022 11:14:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=MIME-Version:Content-Transfer-Encoding:Content-Type :In-Reply-To:References:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=bXO91dWoXmsTkNgYA/uKLyXxkmrBFFSp5zrQ2+jV5LI=; b=bNz/3c3Ztb+QySIyA2k7W4K/JP K+ta+c3DErso5NsXILfRYFd5Rk0ZnbsHxRTE2IeKFAgsFtk3Rfi0mwh2Opuq9vjFpce8muWFwJuCz gFTsBqiAZXlUh4Lt65rD/0k5O7zrxaArtYCTR7RghMCxKxox2IsHLQv6ksuZ+9X0+34Y=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=MIME-Version:Content-Transfer-Encoding:Content-Type:In-Reply-To: References:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=bXO91dWoXmsTkNgYA/uKLyXxkmrBFFSp5zrQ2+jV5LI=; b=Os6znL0lDd0QLm1iWcOenNarp9 uTa978QqnLg62HQyqHhKqcmsh8x74hpQZyEWKhDlu2h/KKMHCkcDoXaXswcl06WdAnEVeaVMbK9cw FILWSn0qMsq1mlnifMD1QoGG/QSajI4KlAqMjE9KPUhrOKOlhUjdpquQmmLl387/SCRg=; Received: from szxga01-in.huawei.com ([45.249.212.187]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1obK9X-0008Lx-4E for linux-f2fs-devel@lists.sourceforge.net; Thu, 22 Sep 2022 11:14:04 +0000 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4MYCGt0PwJzlWrX; Thu, 22 Sep 2022 19:09:42 +0800 (CST) Received: from kwepemm000016.china.huawei.com (7.193.23.210) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 22 Sep 2022 19:13:50 +0800 Received: from kwepemm600014.china.huawei.com (7.193.23.54) by kwepemm000016.china.huawei.com (7.193.23.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 22 Sep 2022 19:13:50 +0800 Received: from kwepemm600014.china.huawei.com ([7.193.23.54]) by kwepemm600014.china.huawei.com ([7.193.23.54]) with mapi id 15.01.2375.031; Thu, 22 Sep 2022 19:13:50 +0800 To: Chao Yu , "jaegeuk@kernel.org" Thread-Topic: Reply: Reply: Reply: [PATCH -next 2/4] f2fs: extent cache: support extent for no-compressed file Thread-Index: AdjOX7sA/sf+cgDTRnGI5C4KfQ8JWP//koAA//9qlWA= Date: Thu, 22 Sep 2022 11:13:50 +0000 Message-ID: <4f2ac701853d4f89a2ff2a2451ba7695@huawei.com> References: <2bb4527d-6ba2-7e09-531b-021a0f513b18@kernel.org> In-Reply-To: <2bb4527d-6ba2-7e09-531b-021a0f513b18@kernel.org> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.174.177.246] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Headers-End: 1obK9X-0008Lx-4E Subject: [f2fs-dev] =?utf-8?b?562U5aSNOiBSZXBseTogUmVwbHk6IFJlcGx5OiBbUEFU?= =?utf-8?q?CH_-next_2/4=5D_f2fs=3A_extent_cache=3A_support_extent_for_no-c?= =?utf-8?q?ompressed_file?= X-BeenThere: linux-f2fs-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: zhangqilong via Linux-f2fs-devel Reply-To: zhangqilong Cc: "linux-f2fs-devel@lists.sourceforge.net" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-f2fs-devel-bounces@lists.sourceforge.net > > On 2022/9/22 17:32, zhangqilong wrote: > >> > >> On 2022/9/22 12:54, zhangqilong wrote: > >>>> On 2022/9/22 10:07, zhangqilong wrote: > >>>>>> On 2022/9/22 0:00, zhangqilong wrote: > >>>>>>>> On 2022/9/21 21:57, zhangqilong wrote: > >>>>>>>>>> On 2022/9/21 20:14, zhangqilong wrote: > >>>>>>>>>>>> On 2022/9/21 15:57, Zhang Qilong wrote: > >>>>>>>>>>>>> No-compressed file may suffer read performance issue > due > >> to > >>>> it > >>>>>>>>>>>>> can't use extent cache or the largest extent in inode > >>>>>>>>>>>>> can't covered other parts of continuous blocks in > readonly > >> format > >>>>>>>>>>>>> f2fs > >>>>>>>> image. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Now it won't build extent cacge tree for no-compressed > >>>>>>>>>>>>> file in readonly format f2fs image. > >>>>>>>>>>>>> > >>>>>>>>>>>>> For readonly format f2fs image, maybe the > no-compressed > >> file > >>>>>>>> don't > >>>>>>>>>>>>> have the largest extent, or it have more than one part > >> which > >>>>>>>>>>>>> have > >>>>>>>>>>>> > >>>>>>>>>>>> Why it can not have largest extent in f2fs_inode? > >>>>>>>>>>> > >>>>>>>>>>> The following several situations may occur: > >>>>>>>>>>> 1) Wrote w/o the extent when the filesystem is > read-write > >>>> fs. > >>>>>>>>>>> > >>>>>>>>>>> 2) Largest extent have been drop after being > >>>>>>>>>>> re-wrote, or it have > >>>>>>>>>> been split to smaller parts. > >>>>>>>>>>> > >>>>>>>>>>> 3) The largest extent only covered one part of > >>>>>>>>>>> continuous blocks, > >>>>>>>>>> like: > >>>>>>>>>>> |------parts 1(continuous blocks)-----|----not > >>>>>>>>>> continuous---|---------------------parts 2 (continuous > >>>>>>>>>> continuous---|blocks)-----------|---------| > >>>>>>>>>>> The largest extent is part 2, but other parts > >>>>>>>>>>> (like part1, ) can't be > >>>>>>>>>> mapped in readonly format f2fs image which should have > been > >>>>>>>> mapped. > >>>>>>>>>> > >>>>>>>>>> largest extent of non-compressed file should be updated > >> during > >>>>>>>>>> sload in a ro f2fs image? > >>>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> I am sorry, I don't fully understand what you mean. I want to > >>>>>>>>> show that the extent of file in readonly format f2fs image > >>>>>>>>> could not existed or > >>>>>>>> can't covered other parts that contain continuous blocks. > >>>>>>>> > >>>>>>>> Well, I mean the extent should be updated due to below flow? > >>>> during > >>>>>>>> the file was loaded into a formated f2fs image w/ ro feature. > >>>>>>>> > >>>>>>>> - f2fs_sload > >>>>>>>> - build_directory > >>>>>>>> - f2fs_build_file > >>>>>>>> > >>>>>>>> if (!c.compress.enabled || (c.feature & > >>>>>>>> cpu_to_le32(F2FS_FEATURE_RO))) > >>>>>>>> update_largest_extent(sbi, de->ino); > >>>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I get it. I think we could consider this flow. > >>>>>>> But it will change the metadata of the file, is it was a little > >>>> inappropriate? > >>>>>>> Maybe users does not want to do that during being loaded and > >>>>>>> will > >>>>>> refuse this change. > >>>>>> > >>>>>> I don't get it. > >>>>>> > >>>>>> IIUC, we can only load files into ro f2fs image w/ sload, and > >>>>>> sload has updated largest extent for file, or am I missing > something? > >>>>> > >>>>> Ah, maybe I misunderstood your idea just now. We could updated > >>>> largest > >>>>> extent for file when loading into ro f2fs image w/ sload, but it > >>>>> will not solve situations 3) issue for the no-compressed file that > >>>>> be > >>>> mentioned previously. > >>>> > >>>> Why case 3) can happen? The data blocks should be allocated > >>>> contiguously while sload loads file into image? > >>> > >>> That's what I thought before, it should allocated contiguously while > >>> sload loads file into image. But I found that it may not be > >>> completely continuous for the big file recently. It may contain > >>> several discontinuous parts. > >> > >> I doubt it was caused by hot/cold data separation strategy, but I > >> guess the strategy is not necessary for ro image, can you look into it? > > > > HI, > > > > "Looking into hot/cold data separation strategy" is a little difficult > > for me now but I will continuous attention and research it in the future. > > In addition, concurrent writing of multiple files also is likely to > > cause this issue, so I think it is valueable sometimes :). > > How can we write files into ro image? I means it is wrote when being sloaded to f2fs ro image. Thanks, > > Thanks, > > > > > Thanks, > > > >> > >> Thanks, > >> > >>> > >>> Thanks, > >>> > >>>> > >>>> Thanks, > >>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>>> Whether it exists or not, we will not change or update largest > >>>>>>>>> extent > >>>>>>>> during sload in a ro f2fs image. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> > >>>>>>>>>>>>> internally continuous blocks. So we add extent cache > tree > >>>>>>>>>>>>> for the no-compressed file in readonly format f2fs image. > >>>>>>>>>>>>> > >>>>>>>>>>>>> The cache policy is almost same with compressed file. > The > >>>>>>>>>>>>> difference is that, the no-compressed file part will set > >>>>>>>>>>>>> min-number of continuous blocks > F2FS_MIN_EXTENT_LEN > >> in > >>>>>> order > >>>>>>>> to > >>>>>>>>>>>>> reduce cache > >>>>>>>>>> fragmentation. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Signed-off-by: Zhang Qilong > > >>>>>>>>>>>>> --- > >>>>>>>>>>>>> fs/f2fs/extent_cache.c | 52 > >>>>>>>>>>>> ++++++++++++++++++++++++++++++++++-------- > >>>>>>>>>>>>> 1 file changed, 42 insertions(+), 10 deletions(-) > >>>>>>>>>>>>> > >>>>>>>>>>>>> diff --git a/fs/f2fs/extent_cache.c > >>>>>>>>>>>>> b/fs/f2fs/extent_cache.c index > >>>>>>>>>>>>> 387d53a61270..7e39381edca0 100644 > >>>>>>>>>>>>> --- a/fs/f2fs/extent_cache.c > >>>>>>>>>>>>> +++ b/fs/f2fs/extent_cache.c > >>>>>>>>>>>>> @@ -695,9 +695,12 @@ static void > >>>>>>>>>>>> f2fs_update_extent_tree_range_compressed(struct > inode > >>>>>> *inode, > >>>>>>>>>>>>> set_extent_info(&ei, fofs, blkaddr, llen); > >>>>>>>>>>>>> ei.c_len = c_len; > >>>>>>>>>>>>> > >>>>>>>>>>>>> - if (!__try_merge_extent_node(sbi, et, &ei, prev_en, > >>>>>> next_en)) > >>>>>>>>>>>>> + if (!__try_merge_extent_node(sbi, et, &ei, > prev_en, > >>>>>> next_en)) { > >>>>>>>>>>>>> + if (!c_len && llen < F2FS_MIN_EXTENT_LEN) > >>>>>>>>>>>>> + goto unlock_out; > >>>>>>>>>>>>> __insert_extent_tree(sbi, et, &ei, > >>>>>>>>>>>>> insert_p, insert_parent, > >> leftmost); > >>>>>>>>>>>>> + } > >>>>>>>>>>>>> unlock_out: > >>>>>>>>>>>>> write_unlock(&et->lock); > >>>>>>>>>>>>> } > >>>>>>>>>>>>> @@ -726,24 +729,53 @@ static unsigned int > >>>>>>>>>>>> f2fs_cluster_blocks_are_contiguous(struct > dnode_of_data > >>>> *dn) > >>>>>>>>>>>>> return compressed ? i - 1 : i; > >>>>>>>>>>>>> } > >>>>>>>>>>>>> > >>>>>>>>>>>>> +/* > >>>>>>>>>>>>> + * check whether normal file blocks are contiguous, and > >> add > >>>>>>>>>>>>> +extent cache > >>>>>>>>>>>>> + * entry only if remained blocks are logically and > >>>>>>>>>>>>> +physically > >>>>>>>>>> contiguous. > >>>>>>>>>>>>> + */ > >>>>>>>>>>>>> +static unsigned int > >>>> f2fs_normal_blocks_are_contiguous(struct > >>>>>>>>>>>>> +dnode_of_data *dn) { > >>>>>>>>>>>>> + int i = 0; > >>>>>>>>>>>>> + struct inode *inode = dn->inode; > >>>>>>>>>>>>> + block_t first_blkaddr = data_blkaddr(inode, > >> dn->node_page, > >>>>>>>>>>>>> + dn->ofs_in_node); > >>>>>>>>>>>>> + unsigned int max_blocks = > >>>>>> ADDRS_PER_PAGE(dn->node_page, > >>>>>>>>>> inode) > >>>>>>>>>>>>> + - dn->ofs_in_node; > >>>>>>>>>>>>> + > >>>>>>>>>>>>> + for (i = 1; i < max_blocks; i++) { > >>>>>>>>>>>>> + block_t blkaddr = data_blkaddr(inode, > >> dn->node_page, > >>>>>>>>>>>>> + dn->ofs_in_node + i); > >>>>>>>>>>>>> + > >>>>>>>>>>>>> + if (!__is_valid_data_blkaddr(blkaddr) || > >>>>>>>>>>>>> + first_blkaddr + i != blkaddr) > >>>>>>>>>>>>> + return i; > >>>>>>>>>>>>> + } > >>>>>>>>>>>>> + > >>>>>>>>>>>>> + return i; > >>>>>>>>>>>>> +} > >>>>>>>>>>>>> + > >>>>>>>>>>>>> void > f2fs_readonly_update_extent_cache(struct > >>>>>>>> dnode_of_data > >>>>>>>>>> *dn, > >>>>>>>>>>>>> pgoff_t index) > >>>>>>>>>>>>> { > >>>>>>>>>>>>> - unsigned int c_len = > >>>>>> f2fs_cluster_blocks_are_contiguous(dn); > >>>>>>>>>>>>> + unsigned int c_len = 0; > >>>>>>>>>>>>> + unsigned int llen = 0; > >>>>>>>>>>>>> block_t blkaddr; > >>>>>>>>>>>>> > >>>>>>>>>>>>> - if (!c_len) > >>>>>>>>>>>>> - return; > >>>>>>>>>>>>> - > >>>>>>>>>>>>> blkaddr = f2fs_data_blkaddr(dn); > >>>>>>>>>>>>> - if (blkaddr == COMPRESS_ADDR) > >>>>>>>>>>>>> - blkaddr = data_blkaddr(dn->inode, > >> dn->node_page, > >>>>>>>>>>>>> + if (f2fs_compressed_file(dn->inode)) { > >>>>>>>>>>>>> + c_len = > f2fs_cluster_blocks_are_contiguous(dn); > >>>>>>>>>>>>> + if (!c_len) > >>>>>>>>>>>>> + return; > >>>>>>>>>>>>> + llen = F2FS_I(dn->inode)->i_cluster_size; > >>>>>>>>>>>>> + if (blkaddr == COMPRESS_ADDR) > >>>>>>>>>>>>> + blkaddr = data_blkaddr(dn->inode, > >>>>>> dn->node_page, > >>>>>>>>>>>>> dn->ofs_in_node + 1); > >>>>>>>>>>>>> + } else { > >>>>>>>>>>>>> + llen = > f2fs_normal_blocks_are_contiguous(dn); > >>>>>>>>>>>>> + } > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>> f2fs_update_extent_tree_range_compressed(dn->inode, > >>>>>>>>>>>>> - index, blkaddr, > >>>>>>>>>>>>> - F2FS_I(dn->inode)->i_cluster_size, > >>>>>>>>>>>>> - c_len); > >>>>>>>>>>>>> + index, blkaddr, llen, c_len); > >>>>>>>>>>>>> } > >>>>>>>>>>>>> #endif > >>>>>>>>>>>>> _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel