From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE3ECC433F5 for ; Sun, 24 Apr 2022 08:00:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238622AbiDXID2 (ORCPT ); Sun, 24 Apr 2022 04:03:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236822AbiDXID1 (ORCPT ); Sun, 24 Apr 2022 04:03:27 -0400 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28F8B3524B; Sun, 24 Apr 2022 01:00:27 -0700 (PDT) Subject: Re: [PATCH v2 00/12] Improve Raid5 Lock Contention DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1650787222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gLLGzwk8fcXEWz+exNqSl7CbwMy3y1DnQlR3cQo6pOM=; b=kxd5j0CDABqAmfdF5I1+eE0CSu2jldOESNMCBSr5p2ePClJy+932zJZLSSyYPFhc8RpLLH sDVHG4pYUhd2jvjUF+y3sybLkSQgSooy1HP0Wi9/GzARheb7uyTNX1CDttD7pFpbkWFBIF UVXImihoFCh2w4CJNBGWmxXLUxg/kXI= To: Logan Gunthorpe , Xiao Ni Cc: open list , linux-raid , Song Liu , Christoph Hellwig , Stephen Bates , Martin Oliveira , David Sloan References: <20220420195425.34911-1-logang@deltatee.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Guoqing Jiang Message-ID: <4094aed9-d22d-d14f-07a7-5abe599beeab@linux.dev> Date: Sun, 24 Apr 2022 16:00:15 +0800 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-raid@vger.kernel.org On 4/22/22 12:02 AM, Logan Gunthorpe wrote: > > On 2022-04-21 02:45, Xiao Ni wrote: >> Could you share the commands to get the test result (lock contention >> and performance)? > Sure. The performance we were focused on was large block writes. So we > setup raid5 instances with varying number of disks and ran the following > fio script directly on the drive. > > [simple] > filename=/dev/md0 > ioengine=libaio > rw=write > direct=1 > size=8G > blocksize=2m > iodepth=16 > runtime=30s > time_based=1 > offset_increment=8G > numjobs=12 >  > (We also played around with tuning this but didn't find substantial > changes once the bottleneck was hit) Nice, I suppose other IO patterns keep the same performance as before. > We tuned md with parameters like: > > echo 4 > /sys/block/md0/md/group_thread_cnt > echo 8192 > /sys/block/md0/md/stripe_cache_size > > For lock contention stats, we just used lockstat[1]; roughly like: > > echo 1 > /proc/sys/kernel/lock_stat > fio test.fio > echo 0 > /proc/sys/kernel/lock_stat > cat /proc/lock_stat > > And compared the before and after. Thanks for your effort, besides the performance test, please try to run mdadm test suites to avoid regression. Thanks, Guoqing