From mboxrd@z Thu Jan 1 00:00:00 1970 From: hanjinke Subject: Re: [External] Re: [PATCH v3] blk-throtl: Introduce sync and async queues for blk-throtl Date: Tue, 10 Jan 2023 21:07:25 +0800 Message-ID: References: <20221226130505.7186-1-hanjinke.666@bytedance.com> <20230105161854.GA1259@blackbody.suse.cz> <20230106153813.4ttyuikzaagkk2sc@quack3> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=0b3RSYe8mQauTsHJUs6ykMcTgur4za2xftvbQNqIJa0=; b=RLUlg0vAPGTz4hMSALnOAbA95CQ2QhiAPu67CW1t5ndi5fYQvoHNRfl2s9mmbfxynC n5Hl26i2wnPo3Po7VVrtnQueuJ9LNMFdRz54SOR0UzQpuEPMsDS42Z/bO4BEb4COnlqZ Yhl/RGwxFnAkY+cH1otEhavspwfCl0FOYahc2kH+YxTr3TQTFwyb+jOHwd6AbOmhbT/k UrCTqXrRr5OpYjY6+L/mYlkYPUtgEqwUoG1lp5ywtifFjrif5++oGbE8puNRSkItokP8 ieNDdz1XkqTFKhhlST3mTca31Q50kVpZqKIC7YfgyJEYf+TFUH4gcW9m3mBKjOdd53dv s+jQ== In-Reply-To: List-ID: Content-Type: text/plain; charset="utf-8"; format="flowed" To: Tejun Heo Cc: Jan Kara , =?UTF-8?Q?Michal_Koutn=c3=bd?= , josef@toxicpanda.com, axboe@kernel.dk, cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yinxin.x@bytedance.com 在 2023/1/10 上午2:08, Tejun Heo 写道: > Hello, > > On Sat, Jan 07, 2023 at 12:44:35PM +0800, hanjinke wrote: >> For cost.model setting, We first use the tools iocost provided to test the >> benchmark model parameters of different types of disks online, and then save >> these benchmark parameters to a parametric Model Table. During the >> deployment process, pull and set the corresponding model parameters >> according to the type of disk. >> >> The setting of cost.qos should be considered slightly more，we need to make >> some compromises between overall disk throughput and io latency. >> The average disk utilization of the entire disk on a specific business and >> the RLA（if it is io sensitive） of key businesses will be taken as >> important input considerations. The cost.qos will be dynamically fine-tuned >> according to the health status monitoring of key businesses. > > Ah, I see. Do you use the latency targets and min/max ranges or just fixate > the vrate by setting min == max? Currently we use the former. > >> For cost.weight setting, high-priority services will gain greater >> advantages through weight settings to deal with a large number of io >> requests in a short period of time. It works fine as work-conservation >> of iocost works well according to our observation. > > Glad to hear. > >> These practices can be done better and I look forward to your better >> suggestions. > > It's still in progress but resctl-bench's iocost-tune benchmark is what > we're starting to use: > > https://github.com/facebookexperimental/resctl-demo/blob/main/resctl-bench/doc/iocost-tune.md > > The benchmark takes like 6 hours and what it does is probing the whole vrate > range looking for behavior inflection points given the scenario of > protecting a latency sensitive workload against memory leak. On completion, > it provides several solutions based on the behavior observed. > > The benchmark is destructive (to the content on the target ssd) and can be > tricky to set up. There's installable image to help setting up and running > the benchmark: > > https://github.com/iocost-benchmark/resctl-demo-image-recipe/actions > > The eventual goal is collecting these benchmark results in the following git > repo: > > https://github.com/iocost-benchmark/iocost-benchmarks > > which generates hwdb files describing all the found solution and make > systemd apply the appropriate configuration on boot automatically. > > It's still all a work in progress but hopefully we should be able to > configure iocost reasonably on boot on most SSDs. > > Thanks. > These methodologies are worthy of our study and will definitely help our future deployment of iocost. Thanks a lot. Thanks.