From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZM540-0002cR-0P for qemu-devel@nongnu.org; Sun, 02 Aug 2015 22:01:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZM53v-00017I-EE for qemu-devel@nongnu.org; Sun, 02 Aug 2015 22:01:15 -0400 Received: from mail-ig0-x22c.google.com ([2607:f8b0:4001:c05::22c]:34010) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZM53v-000171-7V for qemu-devel@nongnu.org; Sun, 02 Aug 2015 22:01:11 -0400 Received: by igk11 with SMTP id 11so63133356igk.1 for ; Sun, 02 Aug 2015 19:01:10 -0700 (PDT) Date: Mon, 3 Aug 2015 10:01:00 +0800 From: Liu Yuan Message-ID: <20150803020100.GA4733@ubuntu-trusty> References: <1438142555-27011-1-git-send-email-namei.unix@gmail.com> <877fpj4kqw.wl%mitake@mitake-jiseki-PC> <20150729093135.GB22681@ubuntu-trusty> <20150730132744.GA11022@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150730132744.GA11022@localhost.localdomain> Subject: Re: [Qemu-devel] [sheepdog] [PATCH] sheepdog: fix overlapping metadata update List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jeff Cody Cc: Kevin Wolf , Teruaki Ishizaki , Hitoshi Mitake , Hitoshi Mitake , qemu-devel@nongnu.org, Vasiliy Tolstov , sheepdog-ng@googlegroups.com, morita.kazutaka@gmail.com, Stefan Hajnoczi , sheepdog@lists.wpkg.org On Thu, Jul 30, 2015 at 09:27:44AM -0400, Jeff Cody wrote: > On Thu, Jul 30, 2015 at 09:41:08AM +0300, Vasiliy Tolstov wrote: > > 2015-07-29 12:31 GMT+03:00 Liu Yuan : > > > Technically, it won't affect the performance because index updates are not range > > > but concrete in terms of underlying 4M block size. Only 2 or 3 indexes in a > > > range will be updated and 90+% updates will be only 1. So if 2 updates stride a > > > large range, it will actually worse the performance of sheepdog because many > > > additional unref of object will be executed by sheep internally. > > > > > > It is not a performance problem but more the right fix. Even with your patch, > > > updates of inode can overlap. You just don't allow overlapped requests go to > > > sheepdog, which is a overkill approach. IMHO, we should only adjust to avoid > > > the overlapped inode updates, which can be done easily and incrementally on top > > > of old code, rather than take on a complete new untested overkill mechanism. So > > > what we get from your patch? Covering the problem and lock every requests? > > > > > > Your patch actually fix nothing but just cover the problem by slowing down the > > > request and even with your patch, the problem still exists because inode updates > > > can overlap. Your commit log doesn't explain what is the real problem and why > > > your fix works. This is not your toy project that can commit whatever you want. > > > > > >> BTW, sheepdog project was already forked, why don't you fork the block > > >> driver, too? > > > > > > What makes you think you own the block driver? > > > > > > We forked the sheepdog project because it is low quality of code partly and mostly > > > some company tries to make it a private project. It is not as open source friendly > > > as before and that is the main reason Kazutaka and I chose to fork the sheepdog > > > project. But this doesn't mean we need to fork the QEMU project, it is an > > > open source project and not your home toy. > > > > > > Kazutaka and I are the biggest contributers of both sheepdog and QEMU sheepdog > > > block driver for years, so I think I am eligible to review the patch and > > > responsible to suggest the right fix. If you are pissed off when someone else > > > have other opinions, you can just fork the code and play with it at home or you > > > follow the rule of open source project. > > > > > > Jeff Cody, please be the judge, patch from Hitoshi solved my problem > > that i emailed in sheepdog list (i have test environment with 8 hosts > > on each 6 SSD disks and infiniband interconnect between hosts) before > > Hitoshi patch, massive writing to sheepdog storage breaks file system > > and corrupt it. > > After the patch i don't see issues. > > > > I'd rather see some sort consensus amongst Liu, Hitoshi, yourself, or > others more intimately familiar with sheepdog. > > Right now, we have Hitoshi's patch in the main git repo, slated for > 2.4 release (which is Monday). It sounds, from Liu's email, as this > may not fix the root cause. > > Vasiliy said he would test Liu's patch; if he can confirm this new > patch fix, then I would be inclined to use Liu's patch, based on the > detailed analysis of the issue in the commit message. > This is my performance comparison on top of latest QEMU with my latop with SSD. sheepdog cluster run with 3 nodes with '-n' to get best volume performance. QEMU command: qemu-system-x86_64 -m 1024 --enable-kvm \ -drive file=debian_squeeze_amd64_standard.qcow2,cache=writeback,if=virtio \ -drive file=sheepdog:test,if=virtio sheepdog:test is created as 'dog vdi create test 80G' I test both time for mkfs and iops for fio write. fio.conf: [global] ioengine=libaio direct=1 thread=1 norandommap=1 runtime=60 size=300M directory=/mnt [write4k-rand] stonewall group_reporting bs=4k rw=randwrite numjobs=8 iodepth=32 Resualt: ================================================ sheep formated with -c 2:1 (erasure coding) mkfs fio Yuan 0.069 4578 Hitosh 0.071 3722 sheep formarted with -c 2 (replication) mkfs fio Yuan 0.074 6873 Hitosh 0.081 6174 ================================================ Thanks, Yuan