From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78811CD343C for ; Tue, 19 Sep 2023 05:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231214AbjISFVb (ORCPT ); Tue, 19 Sep 2023 01:21:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229960AbjISFVa (ORCPT ); Tue, 19 Sep 2023 01:21:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B90F1100 for ; Mon, 18 Sep 2023 22:20:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695100841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/yNKPxmsPCL5ZqRqn6QofYHN+lqEoZ/fz5W5giSNC8A=; b=L+fiw/FKkBBf6dHxOxfW2PCeBPs8Xtt0GwDL/8hrIfQNuOIdMCAjTAWxnl+61H1aNiqNAB NJKaf4BQ7WLbPrsn2W2i+gm3/4tqdhlyu1UQoOiW3NSaFxJBmdhXgrJZYRu6XwgMzzDq9w 9zkURPota1OsnuYHbiXEJNlYSpXYDf4= Received: from mail-oo1-f69.google.com (mail-oo1-f69.google.com [209.85.161.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-251-zU68rHTGPL2mB1OsuleJJw-1; Tue, 19 Sep 2023 01:20:36 -0400 X-MC-Unique: zU68rHTGPL2mB1OsuleJJw-1 Received: by mail-oo1-f69.google.com with SMTP id 006d021491bc7-55e16833517so7081342eaf.1 for ; Mon, 18 Sep 2023 22:20:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695100835; x=1695705635; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/yNKPxmsPCL5ZqRqn6QofYHN+lqEoZ/fz5W5giSNC8A=; b=uLf7WjxcD/jeVPLSypjP8h+u1UMegLJhCLOnbrDokNTPNQyknaGr0sEvDzyOiWsgW2 FE253wox7Wbsl9h1zJte8yyzDu2rzzSe2nu02KAIAvBw1X51xWhq/MrR7zmvXaCWhsJQ McnGMWeqjGNieO80ieVPAIddmWj9bmAb0niURdHbE8w8RWBaLYBkv/6d5j8sn1yvOV+C KwQ+dRKd/x5QEy86499gE+TJLorxdfWa9EIEwP0QKnhO9qI9yDwnjNAXXpntCfUpZw39 AlmjrwL267uLT0CrYH2jfCSajUU1f8LgNPet3jR5c6GdP9Q+AQV7+YSXJMThN9x293db 8GLw== X-Gm-Message-State: AOJu0Yzxsrod7fVH37IlC2VTP62DcfTAVKx75LkHjgFflq4mFjGlMLKI 7vZ42Trjp7BUPGMLpREdkM6C7KLF3tb1VXyC9OwfMLuYLkF5IQrpz7xPuzX5OfHRV30H31XkedP eoV7QdwwkmGvCt6bnPTrcmw== X-Received: by 2002:a4a:2a14:0:b0:576:8bd8:9ab5 with SMTP id k20-20020a4a2a14000000b005768bd89ab5mr11719450oof.1.1695100835321; Mon, 18 Sep 2023 22:20:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFyZiDIY8DWM8wmhxxSRadT5FLG918rWzNwRvzpvTFyy8RUOuciOkWDguCBKKGUuOdXuJNGTw== X-Received: by 2002:a4a:2a14:0:b0:576:8bd8:9ab5 with SMTP id k20-20020a4a2a14000000b005768bd89ab5mr11719421oof.1.1695100835016; Mon, 18 Sep 2023 22:20:35 -0700 (PDT) Received: from redhat.com ([2804:1b3:a803:677d:42e9:f426:9422:f020]) by smtp.gmail.com with ESMTPSA id z196-20020a4a49cd000000b005768a6a19f9sm5348310ooa.2.2023.09.18.22.20.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 22:20:34 -0700 (PDT) Date: Tue, 19 Sep 2023 02:20:24 -0300 From: Leonardo Bras To: Guo Ren Cc: paul.walmsley@sifive.com, anup@brainfault.org, peterz@infradead.org, mingo@redhat.com, will@kernel.org, palmer@rivosinc.com, longman@redhat.com, boqun.feng@gmail.com, tglx@linutronix.de, paulmck@kernel.org, rostedt@goodmis.org, rdunlap@infradead.org, catalin.marinas@arm.com, conor.dooley@microchip.com, xiaoguang.xing@sophgo.com, bjorn@rivosinc.com, alexghiti@rivosinc.com, keescook@chromium.org, greentime.hu@sifive.com, ajones@ventanamicro.com, jszhang@kernel.org, wefu@redhat.com, wuwei2016@iscas.ac.cn, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-csky@vger.kernel.org, Guo Ren Subject: Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support Message-ID: References: <20230910082911.3378782-1-guoren@kernel.org> <20230910082911.3378782-6-guoren@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-arch@vger.kernel.org On Sun, Sep 17, 2023 at 11:02:47PM +0800, Guo Ren wrote: > On Fri, Sep 15, 2023 at 5:08 PM Leonardo Bras wrote: > > > > On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote: > > > On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras wrote: > > > > > > > > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote: > > > > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras wrote: > > > > > > > > > > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote: > > > > > > > From: Guo Ren > > > > > > > > > > > > > > The requirements of qspinlock have been documented by commit: > > > > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size > > > > > > > atomics"). > > > > > > > > > > > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which > > > > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent > > > > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in > > > > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor > > > > > > > is the one. > > > > > > > > > > > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress > > > > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance > > > > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores): > > > > > > > > > > > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%): > > > > > > > queued_spinlock 0.5109/0.00 > > > > > > > ticket_spinlock 0.5814/0.00 > > > > > > > > > > > > > > perf futex/hash (+6.7%): > > > > > > > queued_spinlock 1444393 operations/sec (+- 0.09%) > > > > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%) > > > > > > > > > > > > > > perf futex/wake-parallel (+8.6%): > > > > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%) > > > > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%) > > > > > > > > > > > > > > perf futex/requeue (+4.2%): > > > > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%) > > > > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%) > > > > > > > > > > > > > > System Benchmarks (+6.4%) > > > > > > > queued_spinlock: > > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX > > > > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8 > > > > > > > Double-Precision Whetstone 55.0 182422.8 33167.8 > > > > > > > Execl Throughput 43.0 13116.6 3050.4 > > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8 > > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9 > > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9 > > > > > > > Pipe Throughput 12440.0 23058600.5 18535.9 > > > > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0 > > > > > > > Process Creation 126.0 12537.3 995.0 > > > > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9 > > > > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5 > > > > > > > System Call Overhead 15000.0 33308301.3 22205.5 > > > > > > > ======== > > > > > > > System Benchmarks Index Score 12426.1 > > > > > > > > > > > > > > ticket_spinlock: > > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX > > > > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2 > > > > > > > Double-Precision Whetstone 55.0 181921.0 33076.5 > > > > > > > Execl Throughput 43.0 12625.1 2936.1 > > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0 > > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3 > > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5 > > > > > > > Pipe Throughput 12440.0 20594018.7 16554.7 > > > > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8 > > > > > > > Process Creation 126.0 10798.4 857.0 > > > > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1 > > > > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3 > > > > > > > System Call Overhead 15000.0 30766778.4 20511.2 > > > > > > > ======== > > > > > > > System Benchmarks Index Score 11670.7 > > > > > > > > > > > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64 > > > > > > > cores platform than the ticket_lock. > > > > > > > > > > > > > > Signed-off-by: Guo Ren > > > > > > > Signed-off-by: Guo Ren > > > > > > > --- > > > > > > > arch/riscv/Kconfig | 16 ++++++++++++++++ > > > > > > > arch/riscv/include/asm/Kbuild | 3 ++- > > > > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++ > > > > > > > 3 files changed, 35 insertions(+), 1 deletion(-) > > > > > > > create mode 100644 arch/riscv/include/asm/spinlock.h > > > > > > > > > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > > > > > > index 2c346fe169c1..7f39bfc75744 100644 > > > > > > > --- a/arch/riscv/Kconfig > > > > > > > +++ b/arch/riscv/Kconfig > > > > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT > > > > > > > Specify the maximum number of NUMA Nodes available on the target > > > > > > > system. Increases memory reserved to accommodate various tables. > > > > > > > > > > > > > > +choice > > > > > > > + prompt "RISC-V spinlock type" > > > > > > > + default RISCV_TICKET_SPINLOCKS > > > > > > > + > > > > > > > +config RISCV_TICKET_SPINLOCKS > > > > > > > + bool "Using ticket spinlock" > > > > > > > + > > > > > > > +config RISCV_QUEUED_SPINLOCKS > > > > > > > + bool "Using queued spinlock" > > > > > > > + depends on SMP && MMU > > > > > > > + select ARCH_USE_QUEUED_SPINLOCKS > > > > > > > + help > > > > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee. > > > > > > > + Otherwise, stay at ticket-lock. > > > > > > > +endchoice > > > > > > > + > > > > > > > config RISCV_ALTERNATIVE > > > > > > > bool > > > > > > > depends on !XIP_KERNEL > > > > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild > > > > > > > index 504f8b7e72d4..a0dc85e4a754 100644 > > > > > > > --- a/arch/riscv/include/asm/Kbuild > > > > > > > +++ b/arch/riscv/include/asm/Kbuild > > > > > > > @@ -2,10 +2,11 @@ > > > > > > > generic-y += early_ioremap.h > > > > > > > generic-y += flat.h > > > > > > > generic-y += kvm_para.h > > > > > > > +generic-y += mcs_spinlock.h > > > > > > > generic-y += parport.h > > > > > > > -generic-y += spinlock.h > > > > > > > > > > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*()) > > > > > > and include the asm-generic headers of mcs_spinlock and qspinlock. > > > > > > > > > > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces, > > > > > > which seems the oposite of the above description (ticket spinlocks being > > > > > > the standard). > > > > > > > > > > > > Shouldn't ticket-spinlock.h also get included here? > > > > > > (Also, I am probably missing something, as I dont' see the use of > > > > > > mcs_spinlock here.) > > > > > No, because asm-generic/spinlock.h: > > > > > ... > > > > > #include > > > > > ... > > > > > > > > > > > > > But aren't you removing asm-generic/spinlock.h below ? > > > > -generic-y += spinlock.h > > > Yes, current is: > > > > > > arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h -> > > > include/asm-generic/ticket_spinlock.h > > > > I did a little reading on how generic-y works (which I was unaware): > > > > "If an architecture uses a verbatim copy of a header from > > include/asm-generic then this is listed in the file > > arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the > > build a wrapper include file is generated in the directory [...]" > > > > Oh, so you are removing the asm-generic/spinlock.h because it's link > > was replaced by a new asm/spinlock.h. > > > > You add qspinlock.h to generic-y because it's new in riscv, and add > > mcs_spinlock.h because it's needed by qspinlock.h. > > > > Ok, it makes sense now. > > > > Sorry about this noise. > > I was unaware of how generic-y worked, and (wrongly) > > assumed it was about including headers automatically in the build. > > > > > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS > > > +#include > > > +#include > > > +#else > > > +#include > > > +#endif > > > > > > So, you want me: > > > +#ifdef CONFIG_QUEUED_SPINLOCKS > > > +#include > > > +#else > > > +#include > > > +#endif > > > > > > +#include > > > > > > Right? > > > > No, I didn't mean that. > > I was just worried about the arch_spin_*() interfaces, but they should be > > fine. > > > > BTW, according to kernel doc on generic-y, shouldn't be a better idea to > > add 'ticket_spinlock.h' to generic-y, and include above as > > asm/ticket_spinlock.h? > > > > Or is generic-y reserved only for stuff which is indirectly included by > > other headers? > It's okay to add generic-y for ticket_spinlock.h, and I'm okay with > the following: > > +#ifdef CONFIG_QUEUED_SPINLOCKS > +#include > +#else > +#include > +#endif > > +#include It does look more intuitive, so I am glad it works for you :) FWIW: Reviewed-by: Leonardo Bras Thanks! Leo > > > > > Thanks! > > Leo > > > > > > > > > > > > > > > > > > > > > > generic-y += spinlock_types.h > > > > > > > generic-y += qrwlock.h > > > > > > > generic-y += qrwlock_types.h > > > > > > > +generic-y += qspinlock.h > > > > > > > generic-y += user.h > > > > > > > generic-y += vmlinux.lds.h > > > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h > > > > > > > new file mode 100644 > > > > > > > index 000000000000..c644a92d4548 > > > > > > > --- /dev/null > > > > > > > +++ b/arch/riscv/include/asm/spinlock.h > > > > > > > @@ -0,0 +1,17 @@ > > > > > > > +/* SPDX-License-Identifier: GPL-2.0 */ > > > > > > > + > > > > > > > +#ifndef __ASM_RISCV_SPINLOCK_H > > > > > > > +#define __ASM_RISCV_SPINLOCK_H > > > > > > > + > > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS > > > > > > > +#define _Q_PENDING_LOOPS (1 << 9) > > > > > > > +#endif > > > > > > > > > > > > Any reason the above define couldn't be merged on the ifdef below? > > > > > Easy for the next patch to modify. See Waiman's comment: > > > > > > > > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/ > > > > > > > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h > > > > > > index c644a92d4548..9eb3ad31e564 100644 > > > > > > --- a/arch/riscv/include/asm/spinlock.h > > > > > > +++ b/arch/riscv/include/asm/spinlock.h > > > > > > @@ -7,11 +7,94 @@ > > > > > > #define _Q_PENDING_LOOPS (1 << 9) > > > > > > #endif > > > > > > > > > > > > > > > > I see why you separated the _Q_PENDING_LOOPS out. > > > > > > > > > > > > > I see, should be fine then. > > > > > > > > Thanks! > > > > Leo > > > > > > > > > > > > > > > > > > > > > > + > > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS > > > > > > > +#include > > > > > > > +#include > > > > > > > +#else > > > > > > > +#include > > > > > > > +#endif > > > > > > > + > > > > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */ > > > > > > > -- > > > > > > > 2.36.1 > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > Leo > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best Regards > > > > > Guo Ren > > > > > > > > > > > > > > > > > > -- > > > Best Regards > > > Guo Ren > > > > > > > > -- > Best Regards > Guo Ren > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ACBD8CD3437 for ; Tue, 19 Sep 2023 05:20:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZKDtavy0/2Wv1+MTmHp1OPLMxEC+HCFY2Ku9JNM0JIM=; b=CD1IL9on4bMw9M nPp7cMbzgpxWbfILhiag/NpyLPcUD8Mu0JViRrO8gKDqw4o8YWe87gvc6PuGhIpMtOBxUyGv5S/eU TMl1yNcFo5rVjIPW/hWxco7lucUfkEZ/slsxmzIqgIuMbx1jG/mH7UT/cjcfJur5ngVvKFXnlfH4T rxsTx555dFW/o8H2UQNTHz+Z4lWmzZDnxwRjktundtZw6tVMm5se6xiCFj/YVYHmRtrMkJHA0E6rT FUqqn/1y7zIUfVL8uE4wqfJIWvogfrLfzXm+gHgV3HzK4OnqdS/U+aZ2h5Xqk5LEqtI8viG/DxdU0 veThg6XXnGFHnqstf/jw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiTAD-00H2G5-2u; Tue, 19 Sep 2023 05:20:45 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiTA9-00H2F2-2y for linux-riscv@lists.infradead.org; Tue, 19 Sep 2023 05:20:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695100840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/yNKPxmsPCL5ZqRqn6QofYHN+lqEoZ/fz5W5giSNC8A=; b=Wr4eFJ8nZw8cG2xTMaUzO/Yx1M4r9YOdLoKLDvK8zndeFvsOzXE9cyZa+rSPDvu+hDz1rM Jcizoo4Ne1pDoZuHdPgP3gOz4AEBedFC9EhtJHQ+0wwuO+BLcgdxiDjENf+nK/AmTOtsfm tm7IXDSmB3koLhXNCPnVCqIDoDZ1VEA= Received: from mail-oo1-f71.google.com (mail-oo1-f71.google.com [209.85.161.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-462-iBqpMvmqPoCW9TkcMYlrGw-1; Tue, 19 Sep 2023 01:20:36 -0400 X-MC-Unique: iBqpMvmqPoCW9TkcMYlrGw-1 Received: by mail-oo1-f71.google.com with SMTP id 006d021491bc7-573b8245f9bso7108807eaf.0 for ; Mon, 18 Sep 2023 22:20:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695100835; x=1695705635; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/yNKPxmsPCL5ZqRqn6QofYHN+lqEoZ/fz5W5giSNC8A=; b=Y84bJysAMvSVXI/H42giyIJmTVu7URfLdM35TQvqJ7zqgeWv1hgl6mnsyIF+aBhW4K 1SdSbxpepia4lG5eqthR0vcbuWKHTK8bfdiXddD1lcHzUC4PoH1sLckj+KvyLr1St5mU W9+81KhK/x1WhCbmCczZISfrtVIPzuO1E1QaG6zPNc1hRWefTK+ZI8ecpuU557x5Ksj1 m4HA13JOdxWgyUOtgrrBegFVeYPcNH9PcrbIGc0bWiOadEyfkR9KxhQYXgyan9wg6dQt hz8hKCZ8xBQeIpP4s8AQ8gXfxUD0S6PX0jHjSyW58OVVGhV4uDsFSmMeYza42amcFXvt vGFw== X-Gm-Message-State: AOJu0YyQ0Eh9uwTDspHFTS2y9++KdvJnRMgug4348QihBAKu4daiuYof cRXqBLW/55dsfcvv11gsePRwO3Mlylf31LawiZMaNyFgIC4HIh4OKTSCiRllZQEOu1t/1vzkXaA oXklalsm61uyVCmFbDb70RES+wZQf X-Received: by 2002:a4a:2a14:0:b0:576:8bd8:9ab5 with SMTP id k20-20020a4a2a14000000b005768bd89ab5mr11719453oof.1.1695100835322; Mon, 18 Sep 2023 22:20:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFyZiDIY8DWM8wmhxxSRadT5FLG918rWzNwRvzpvTFyy8RUOuciOkWDguCBKKGUuOdXuJNGTw== X-Received: by 2002:a4a:2a14:0:b0:576:8bd8:9ab5 with SMTP id k20-20020a4a2a14000000b005768bd89ab5mr11719421oof.1.1695100835016; Mon, 18 Sep 2023 22:20:35 -0700 (PDT) Received: from redhat.com ([2804:1b3:a803:677d:42e9:f426:9422:f020]) by smtp.gmail.com with ESMTPSA id z196-20020a4a49cd000000b005768a6a19f9sm5348310ooa.2.2023.09.18.22.20.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 22:20:34 -0700 (PDT) Date: Tue, 19 Sep 2023 02:20:24 -0300 From: Leonardo Bras To: Guo Ren Cc: paul.walmsley@sifive.com, anup@brainfault.org, peterz@infradead.org, mingo@redhat.com, will@kernel.org, palmer@rivosinc.com, longman@redhat.com, boqun.feng@gmail.com, tglx@linutronix.de, paulmck@kernel.org, rostedt@goodmis.org, rdunlap@infradead.org, catalin.marinas@arm.com, conor.dooley@microchip.com, xiaoguang.xing@sophgo.com, bjorn@rivosinc.com, alexghiti@rivosinc.com, keescook@chromium.org, greentime.hu@sifive.com, ajones@ventanamicro.com, jszhang@kernel.org, wefu@redhat.com, wuwei2016@iscas.ac.cn, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-csky@vger.kernel.org, Guo Ren Subject: Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support Message-ID: References: <20230910082911.3378782-1-guoren@kernel.org> <20230910082911.3378782-6-guoren@kernel.org> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_222042_108335_26C56022 X-CRM114-Status: GOOD ( 44.54 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gU3VuLCBTZXAgMTcsIDIwMjMgYXQgMTE6MDI6NDdQTSArMDgwMCwgR3VvIFJlbiB3cm90ZToK PiBPbiBGcmksIFNlcCAxNSwgMjAyMyBhdCA1OjA44oCvUE0gTGVvbmFyZG8gQnJhcyA8bGVvYnJh c0ByZWRoYXQuY29tPiB3cm90ZToKPiA+Cj4gPiBPbiBGcmksIFNlcCAxNSwgMjAyMyBhdCAxMDox MDoyNUFNICswODAwLCBHdW8gUmVuIHdyb3RlOgo+ID4gPiBPbiBUaHUsIFNlcCAxNCwgMjAyMyBh dCA1OjQz4oCvUE0gTGVvbmFyZG8gQnJhcyA8bGVvYnJhc0ByZWRoYXQuY29tPiB3cm90ZToKPiA+ ID4gPgo+ID4gPiA+IE9uIFRodSwgU2VwIDE0LCAyMDIzIGF0IDEyOjQ2OjU2UE0gKzA4MDAsIEd1 byBSZW4gd3JvdGU6Cj4gPiA+ID4gPiBPbiBUaHUsIFNlcCAxNCwgMjAyMyBhdCA0OjI54oCvQU0g TGVvbmFyZG8gQnJhcyA8bGVvYnJhc0ByZWRoYXQuY29tPiB3cm90ZToKPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gT24gU3VuLCBTZXAgMTAsIDIwMjMgYXQgMDQ6Mjg6NTlBTSAtMDQwMCwgZ3VvcmVu QGtlcm5lbC5vcmcgd3JvdGU6Cj4gPiA+ID4gPiA+ID4gRnJvbTogR3VvIFJlbiA8Z3VvcmVuQGxp bnV4LmFsaWJhYmEuY29tPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gVGhlIHJlcXVpcmVt ZW50cyBvZiBxc3BpbmxvY2sgaGF2ZSBiZWVuIGRvY3VtZW50ZWQgYnkgY29tbWl0Ogo+ID4gPiA+ ID4gPiA+IGE4YWQwN2U1MjQwYyAoImFzbS1nZW5lcmljOiBxc3BpbmxvY2s6IEluZGljYXRlIHRo ZSB1c2Ugb2YgbWl4ZWQtc2l6ZQo+ID4gPiA+ID4gPiA+IGF0b21pY3MiKS4KPiA+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPiA+IEFsdGhvdWdoIFJJU0MtViBJU0EgZ2l2ZXMgb3V0IGEgd2Vha2VyIGZv cndhcmQgZ3VhcmFudGVlIExSL1NDLCB3aGljaAo+ID4gPiA+ID4gPiA+IGRvZXNuJ3Qgc2F0aXNm eSB0aGUgcmVxdWlyZW1lbnRzIG9mIHFzcGlubG9jayBhYm92ZSwgaXQgd29uJ3QgcHJldmVudAo+ ID4gPiA+ID4gPiA+IHNvbWUgcmlzY3YgdmVuZG9ycyBmcm9tIGltcGxlbWVudGluZyBhIHN0cm9u ZyBmd2QgZ3VhcmFudGVlIExSL1NDIGluCj4gPiA+ID4gPiA+ID4gbWljcm9hcmNoaXRlY3R1cmUg dG8gbWF0Y2ggeGNoZ190YWlsIHJlcXVpcmVtZW50LiBULUhFQUQgQzl4eCBwcm9jZXNzb3IKPiA+ ID4gPiA+ID4gPiBpcyB0aGUgb25lLgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gV2UndmUg dGVzdGVkIHRoZSBwYXRjaCBvbiBTT1BIR08gc2cyMDQyICYgdGgxNTIwIGFuZCBwYXNzZWQgdGhl IHN0cmVzcwo+ID4gPiA+ID4gPiA+IHRlc3Qgb24gRmVkb3JhICYgVWJ1bnR1ICYgT3BlbkV1bGVy IC4uLiBIZXJlIGlzIHRoZSBwZXJmb3JtYW5jZQo+ID4gPiA+ID4gPiA+IGNvbXBhcmlzb24gYmV0 d2VlbiBxc3BpbmxvY2sgYW5kIHRpY2tldF9sb2NrIG9uIHNnMjA0MiAoNjQgY29yZXMpOgo+ID4g PiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gc3lzYmVuY2ggdGVzdD10aHJlYWRzIHRocmVhZHM9MzIg eWllbGRzPTEwMCBsb2NrPTggKCsxMy44JSk6Cj4gPiA+ID4gPiA+ID4gICBxdWV1ZWRfc3Bpbmxv Y2sgMC41MTA5LzAuMDAKPiA+ID4gPiA+ID4gPiAgIHRpY2tldF9zcGlubG9jayAwLjU4MTQvMC4w MAo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gcGVyZiBmdXRleC9oYXNoICgrNi43JSk6Cj4g PiA+ID4gPiA+ID4gICBxdWV1ZWRfc3BpbmxvY2sgMTQ0NDM5MyBvcGVyYXRpb25zL3NlYyAoKy0g MC4wOSUpCj4gPiA+ID4gPiA+ID4gICB0aWNrZXRfc3BpbmxvY2sgMTM1MzIxNSBvcGVyYXRpb25z L3NlYyAoKy0gMC4xNSUpCj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBwZXJmIGZ1dGV4L3dh a2UtcGFyYWxsZWwgKCs4LjYlKToKPiA+ID4gPiA+ID4gPiAgIHF1ZXVlZF9zcGlubG9jayAod2Fr aW5nIDEvNjQgdGhyZWFkcykgaW4gMC4wMjUzIG1zICgrLTIuOTAlKQo+ID4gPiA+ID4gPiA+ICAg dGlja2V0X3NwaW5sb2NrICh3YWtpbmcgMS82NCB0aHJlYWRzKSBpbiAwLjAyNzUgbXMgKCstMy4x MiUpCj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBwZXJmIGZ1dGV4L3JlcXVldWUgKCs0LjIl KToKPiA+ID4gPiA+ID4gPiAgIHF1ZXVlZF9zcGlubG9jayBSZXF1ZXVlZCA2NCBvZiA2NCB0aHJl YWRzIGluIDAuMDc4NSBtcyAoKy0wLjU1JSkKPiA+ID4gPiA+ID4gPiAgIHRpY2tldF9zcGlubG9j ayBSZXF1ZXVlZCA2NCBvZiA2NCB0aHJlYWRzIGluIDAuMDgxOCBtcyAoKy00LjEyJSkKPiA+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiA+IFN5c3RlbSBCZW5jaG1hcmtzICgrNi40JSkKPiA+ID4gPiA+ ID4gPiAgIHF1ZXVlZF9zcGlubG9jazoKPiA+ID4gPiA+ID4gPiAgICAgU3lzdGVtIEJlbmNobWFy a3MgSW5kZXggVmFsdWVzICAgICAgICAgICAgICAgQkFTRUxJTkUgICAgICAgUkVTVUxUICAgIElO REVYCj4gPiA+ID4gPiA+ID4gICAgIERocnlzdG9uZSAyIHVzaW5nIHJlZ2lzdGVyIHZhcmlhYmxl cyAgICAgICAgIDExNjcwMC4wICA2Mjg2MTM3NDUuNCAgNTM4NjUuOAo+ID4gPiA+ID4gPiA+ICAg ICBEb3VibGUtUHJlY2lzaW9uIFdoZXRzdG9uZSAgICAgICAgICAgICAgICAgICAgICAgNTUuMCAg ICAgMTgyNDIyLjggIDMzMTY3LjgKPiA+ID4gPiA+ID4gPiAgICAgRXhlY2wgVGhyb3VnaHB1dCAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDQzLjAgICAgICAxMzExNi42ICAgMzA1MC40 Cj4gPiA+ID4gPiA+ID4gICAgIEZpbGUgQ29weSAxMDI0IGJ1ZnNpemUgMjAwMCBtYXhibG9ja3Mg ICAgICAgICAgMzk2MC4wICAgIDc3NjIzMDYuMiAgMTk2MDEuOAo+ID4gPiA+ID4gPiA+ICAgICBG aWxlIENvcHkgMjU2IGJ1ZnNpemUgNTAwIG1heGJsb2NrcyAgICAgICAgICAgIDE2NTUuMCAgICAz NDE3NTU2LjggIDIwNjQ5LjkKPiA+ID4gPiA+ID4gPiAgICAgRmlsZSBDb3B5IDQwOTYgYnVmc2l6 ZSA4MDAwIG1heGJsb2NrcyAgICAgICAgICA1ODAwLjAgICAgNzQyNzk5NS43ICAxMjgwNi45Cj4g PiA+ID4gPiA+ID4gICAgIFBpcGUgVGhyb3VnaHB1dCAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAxMjQ0MC4wICAgMjMwNTg2MDAuNSAgMTg1MzUuOQo+ID4gPiA+ID4gPiA+ICAgICBQaXBl LWJhc2VkIENvbnRleHQgU3dpdGNoaW5nICAgICAgICAgICAgICAgICAgIDQwMDAuMCAgICAyODM1 NjE3LjcgICA3MDg5LjAKPiA+ID4gPiA+ID4gPiAgICAgUHJvY2VzcyBDcmVhdGlvbiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgMTI2LjAgICAgICAxMjUzNy4zICAgIDk5NS4wCj4gPiA+ ID4gPiA+ID4gICAgIFNoZWxsIFNjcmlwdHMgKDEgY29uY3VycmVudCkgICAgICAgICAgICAgICAg ICAgICA0Mi40ICAgICAgNTcwNTcuNCAgMTM0NTYuOQo+ID4gPiA+ID4gPiA+ICAgICBTaGVsbCBT Y3JpcHRzICg4IGNvbmN1cnJlbnQpICAgICAgICAgICAgICAgICAgICAgIDYuMCAgICAgICA3MzY3 LjEgIDEyMjc4LjUKPiA+ID4gPiA+ID4gPiAgICAgU3lzdGVtIENhbGwgT3ZlcmhlYWQgICAgICAg ICAgICAgICAgICAgICAgICAgIDE1MDAwLjAgICAzMzMwODMwMS4zICAyMjIwNS41Cj4gPiA+ID4g PiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA9PT09PT09PQo+ID4gPiA+ID4gPiA+ICAgICBTeXN0ZW0gQmVu Y2htYXJrcyBJbmRleCBTY29yZSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IDEyNDI2LjEKPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ICAgdGlja2V0X3NwaW5sb2NrOgo+ ID4gPiA+ID4gPiA+ICAgICBTeXN0ZW0gQmVuY2htYXJrcyBJbmRleCBWYWx1ZXMgICAgICAgICAg ICAgICBCQVNFTElORSAgICAgICBSRVNVTFQgICAgSU5ERVgKPiA+ID4gPiA+ID4gPiAgICAgRGhy eXN0b25lIDIgdXNpbmcgcmVnaXN0ZXIgdmFyaWFibGVzICAgICAgICAgMTE2NzAwLjAgIDYyNjU0 MTcwMS45ICA1MzY4OC4yCj4gPiA+ID4gPiA+ID4gICAgIERvdWJsZS1QcmVjaXNpb24gV2hldHN0 b25lICAgICAgICAgICAgICAgICAgICAgICA1NS4wICAgICAxODE5MjEuMCAgMzMwNzYuNQo+ID4g PiA+ID4gPiA+ICAgICBFeGVjbCBUaHJvdWdocHV0ICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgNDMuMCAgICAgIDEyNjI1LjEgICAyOTM2LjEKPiA+ID4gPiA+ID4gPiAgICAgRmlsZSBD b3B5IDEwMjQgYnVmc2l6ZSAyMDAwIG1heGJsb2NrcyAgICAgICAgICAzOTYwLjAgICAgNjU1Mzc5 Mi45ICAxNjU1MC4wCj4gPiA+ID4gPiA+ID4gICAgIEZpbGUgQ29weSAyNTYgYnVmc2l6ZSA1MDAg bWF4YmxvY2tzICAgICAgICAgICAgMTY1NS4wICAgIDMxODkyMzEuNiAgMTkyNzAuMwo+ID4gPiA+ ID4gPiA+ICAgICBGaWxlIENvcHkgNDA5NiBidWZzaXplIDgwMDAgbWF4YmxvY2tzICAgICAgICAg IDU4MDAuMCAgICA3MjIxMjc3LjAgIDEyNDUwLjUKPiA+ID4gPiA+ID4gPiAgICAgUGlwZSBUaHJv dWdocHV0ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDEyNDQwLjAgICAyMDU5NDAxOC43 ICAxNjU1NC43Cj4gPiA+ID4gPiA+ID4gICAgIFBpcGUtYmFzZWQgQ29udGV4dCBTd2l0Y2hpbmcg ICAgICAgICAgICAgICAgICAgNDAwMC4wICAgIDI1NzExMTcuNyAgIDY0MjcuOAo+ID4gPiA+ID4g PiA+ICAgICBQcm9jZXNzIENyZWF0aW9uICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAx MjYuMCAgICAgIDEwNzk4LjQgICAgODU3LjAKPiA+ID4gPiA+ID4gPiAgICAgU2hlbGwgU2NyaXB0 cyAoMSBjb25jdXJyZW50KSAgICAgICAgICAgICAgICAgICAgIDQyLjQgICAgICA1NzIyNy41ICAx MzQ5Ny4xCj4gPiA+ID4gPiA+ID4gICAgIFNoZWxsIFNjcmlwdHMgKDggY29uY3VycmVudCkgICAg ICAgICAgICAgICAgICAgICAgNi4wICAgICAgIDczMjkuMiAgMTIyMTUuMwo+ID4gPiA+ID4gPiA+ ICAgICBTeXN0ZW0gQ2FsbCBPdmVyaGVhZCAgICAgICAgICAgICAgICAgICAgICAgICAgMTUwMDAu MCAgIDMwNzY2Nzc4LjQgIDIwNTExLjIKPiA+ID4gPiA+ID4gPiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgID09PT09 PT09Cj4gPiA+ID4gPiA+ID4gICAgIFN5c3RlbSBCZW5jaG1hcmtzIEluZGV4IFNjb3JlICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMTE2NzAuNwo+ID4gPiA+ID4gPiA+Cj4g PiA+ID4gPiA+ID4gVGhlIHFzcGlubG9jayBoYXMgYSBzaWduaWZpY2FudCBpbXByb3ZlbWVudCBv biBTT1BIR08gU0cyMDQyIDY0Cj4gPiA+ID4gPiA+ID4gY29yZXMgcGxhdGZvcm0gdGhhbiB0aGUg dGlja2V0X2xvY2suCj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBTaWduZWQtb2ZmLWJ5OiBH dW8gUmVuIDxndW9yZW5Aa2VybmVsLm9yZz4KPiA+ID4gPiA+ID4gPiBTaWduZWQtb2ZmLWJ5OiBH dW8gUmVuIDxndW9yZW5AbGludXguYWxpYmFiYS5jb20+Cj4gPiA+ID4gPiA+ID4gLS0tCj4gPiA+ ID4gPiA+ID4gIGFyY2gvcmlzY3YvS2NvbmZpZyAgICAgICAgICAgICAgICB8IDE2ICsrKysrKysr KysrKysrKysKPiA+ID4gPiA+ID4gPiAgYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9LYnVpbGQgICAg IHwgIDMgKystCj4gPiA+ID4gPiA+ID4gIGFyY2gvcmlzY3YvaW5jbHVkZS9hc20vc3BpbmxvY2su aCB8IDE3ICsrKysrKysrKysrKysrKysrCj4gPiA+ID4gPiA+ID4gIDMgZmlsZXMgY2hhbmdlZCwg MzUgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQo+ID4gPiA+ID4gPiA+ICBjcmVhdGUgbW9k ZSAxMDA2NDQgYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9zcGlubG9jay5oCj4gPiA+ID4gPiA+ID4K PiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9LY29uZmlnIGIvYXJjaC9yaXNj di9LY29uZmlnCj4gPiA+ID4gPiA+ID4gaW5kZXggMmMzNDZmZTE2OWMxLi43ZjM5YmZjNzU3NDQg MTAwNjQ0Cj4gPiA+ID4gPiA+ID4gLS0tIGEvYXJjaC9yaXNjdi9LY29uZmlnCj4gPiA+ID4gPiA+ ID4gKysrIGIvYXJjaC9yaXNjdi9LY29uZmlnCj4gPiA+ID4gPiA+ID4gQEAgLTQ3MSw2ICs0NzEs MjIgQEAgY29uZmlnIE5PREVTX1NISUZUCj4gPiA+ID4gPiA+ID4gICAgICAgICBTcGVjaWZ5IHRo ZSBtYXhpbXVtIG51bWJlciBvZiBOVU1BIE5vZGVzIGF2YWlsYWJsZSBvbiB0aGUgdGFyZ2V0Cj4g PiA+ID4gPiA+ID4gICAgICAgICBzeXN0ZW0uICBJbmNyZWFzZXMgbWVtb3J5IHJlc2VydmVkIHRv IGFjY29tbW9kYXRlIHZhcmlvdXMgdGFibGVzLgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4g K2Nob2ljZQo+ID4gPiA+ID4gPiA+ICsgICAgIHByb21wdCAiUklTQy1WIHNwaW5sb2NrIHR5cGUi Cj4gPiA+ID4gPiA+ID4gKyAgICAgZGVmYXVsdCBSSVNDVl9USUNLRVRfU1BJTkxPQ0tTCj4gPiA+ ID4gPiA+ID4gKwo+ID4gPiA+ID4gPiA+ICtjb25maWcgUklTQ1ZfVElDS0VUX1NQSU5MT0NLUwo+ ID4gPiA+ID4gPiA+ICsgICAgIGJvb2wgIlVzaW5nIHRpY2tldCBzcGlubG9jayIKPiA+ID4gPiA+ ID4gPiArCj4gPiA+ID4gPiA+ID4gK2NvbmZpZyBSSVNDVl9RVUVVRURfU1BJTkxPQ0tTCj4gPiA+ ID4gPiA+ID4gKyAgICAgYm9vbCAiVXNpbmcgcXVldWVkIHNwaW5sb2NrIgo+ID4gPiA+ID4gPiA+ ICsgICAgIGRlcGVuZHMgb24gU01QICYmIE1NVQo+ID4gPiA+ID4gPiA+ICsgICAgIHNlbGVjdCBB UkNIX1VTRV9RVUVVRURfU1BJTkxPQ0tTCj4gPiA+ID4gPiA+ID4gKyAgICAgaGVscAo+ID4gPiA+ ID4gPiA+ICsgICAgICAgTWFrZSBzdXJlIHlvdXIgbWljcm8gYXJjaCBMTC9TQyBoYXMgYSBzdHJv bmcgZm9yd2FyZCBwcm9ncmVzcyBndWFyYW50ZWUuCj4gPiA+ID4gPiA+ID4gKyAgICAgICBPdGhl cndpc2UsIHN0YXkgYXQgdGlja2V0LWxvY2suCj4gPiA+ID4gPiA+ID4gK2VuZGNob2ljZQo+ID4g PiA+ID4gPiA+ICsKPiA+ID4gPiA+ID4gPiAgY29uZmlnIFJJU0NWX0FMVEVSTkFUSVZFCj4gPiA+ ID4gPiA+ID4gICAgICAgYm9vbAo+ID4gPiA+ID4gPiA+ICAgICAgIGRlcGVuZHMgb24gIVhJUF9L RVJORUwKPiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9L YnVpbGQgYi9hcmNoL3Jpc2N2L2luY2x1ZGUvYXNtL0tidWlsZAo+ID4gPiA+ID4gPiA+IGluZGV4 IDUwNGY4YjdlNzJkNC4uYTBkYzg1ZTRhNzU0IDEwMDY0NAo+ID4gPiA+ID4gPiA+IC0tLSBhL2Fy Y2gvcmlzY3YvaW5jbHVkZS9hc20vS2J1aWxkCj4gPiA+ID4gPiA+ID4gKysrIGIvYXJjaC9yaXNj di9pbmNsdWRlL2FzbS9LYnVpbGQKPiA+ID4gPiA+ID4gPiBAQCAtMiwxMCArMiwxMSBAQAo+ID4g PiA+ID4gPiA+ICBnZW5lcmljLXkgKz0gZWFybHlfaW9yZW1hcC5oCj4gPiA+ID4gPiA+ID4gIGdl bmVyaWMteSArPSBmbGF0LmgKPiA+ID4gPiA+ID4gPiAgZ2VuZXJpYy15ICs9IGt2bV9wYXJhLmgK PiA+ID4gPiA+ID4gPiArZ2VuZXJpYy15ICs9IG1jc19zcGlubG9jay5oCj4gPiA+ID4gPiA+ID4g IGdlbmVyaWMteSArPSBwYXJwb3J0LmgKPiA+ID4gPiA+ID4gPiAtZ2VuZXJpYy15ICs9IHNwaW5s b2NrLmgKPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gSUlVQyBoZXJlIHlvdSB0YWtlIHRoZSBhc20t Z2VuZXJpYy9zcGlubG9jay5oICh3aGljaCBkZWZpbmVzIGFyY2hfc3Bpbl8qKCkpCj4gPiA+ID4g PiA+IGFuZCBpbmNsdWRlIHRoZSBhc20tZ2VuZXJpYyBoZWFkZXJzIG9mIG1jc19zcGlubG9jayBh bmQgcXNwaW5sb2NrLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBJbiB0aGlzIGNhc2UsIHRoZSBx c3BpbmxvY2suaCB3aWxsIHByb3ZpZGUgdGhlIGFyY2hfc3Bpbl8qKCkgaW50ZXJmYWNlcywKPiA+ ID4gPiA+ID4gd2hpY2ggc2VlbXMgdGhlIG9wb3NpdGUgb2YgdGhlIGFib3ZlIGRlc2NyaXB0aW9u ICh0aWNrZXQgc3BpbmxvY2tzIGJlaW5nCj4gPiA+ID4gPiA+IHRoZSBzdGFuZGFyZCkuCj4gPiA+ ID4gPiA+Cj4gPiA+ID4gPiA+IFNob3VsZG4ndCB0aWNrZXQtc3BpbmxvY2suaCBhbHNvIGdldCBp bmNsdWRlZCBoZXJlPwo+ID4gPiA+ID4gPiAoQWxzbywgSSBhbSBwcm9iYWJseSBtaXNzaW5nIHNv bWV0aGluZywgYXMgSSBkb250JyBzZWUgdGhlIHVzZSBvZgo+ID4gPiA+ID4gPiBtY3Nfc3Bpbmxv Y2sgaGVyZS4pCj4gPiA+ID4gPiBObywgYmVjYXVzZSBhc20tZ2VuZXJpYy9zcGlubG9jay5oOgo+ ID4gPiA+ID4gLi4uCj4gPiA+ID4gPiAjaW5jbHVkZSA8YXNtLWdlbmVyaWMvdGlja2V0X3NwaW5s b2NrLmg+Cj4gPiA+ID4gPiAuLi4KPiA+ID4gPiA+Cj4gPiA+ID4KPiA+ID4gPiBCdXQgYXJlbid0 IHlvdSByZW1vdmluZyBhc20tZ2VuZXJpYy9zcGlubG9jay5oIGJlbG93ID8KPiA+ID4gPiAtZ2Vu ZXJpYy15ICs9IHNwaW5sb2NrLmgKPiA+ID4gWWVzLCBjdXJyZW50IGlzOgo+ID4gPgo+ID4gPiBh cmNoL3Jpc2N2L2luY2x1ZGUvYXNtL3NwaW5sb2NrLmggLT4gaW5jbHVkZS9hc20tZ2VuZXJpYy9z cGlubG9jay5oIC0+Cj4gPiA+IGluY2x1ZGUvYXNtLWdlbmVyaWMvdGlja2V0X3NwaW5sb2NrLmgK PiA+Cj4gPiBJIGRpZCBhIGxpdHRsZSByZWFkaW5nIG9uIGhvdyBnZW5lcmljLXkgd29ya3MgKHdo aWNoIEkgd2FzIHVuYXdhcmUpOgo+ID4KPiA+ICJJZiBhbiBhcmNoaXRlY3R1cmUgdXNlcyBhIHZl cmJhdGltIGNvcHkgb2YgYSBoZWFkZXIgZnJvbQo+ID4gaW5jbHVkZS9hc20tZ2VuZXJpYyB0aGVu IHRoaXMgaXMgbGlzdGVkIGluIHRoZSBmaWxlCj4gPiBhcmNoLyQoU1JDQVJDSCkvaW5jbHVkZS9h c20vS2J1aWxkIFsuLi5dIER1cmluZyB0aGUgcHJlcGFyZSBwaGFzZSBvZiB0aGUKPiA+IGJ1aWxk IGEgd3JhcHBlciBpbmNsdWRlIGZpbGUgaXMgZ2VuZXJhdGVkIGluIHRoZSBkaXJlY3RvcnkgWy4u Ll0iCj4gPgo+ID4gT2gsIHNvIHlvdSBhcmUgcmVtb3ZpbmcgdGhlIGFzbS1nZW5lcmljL3NwaW5s b2NrLmggYmVjYXVzZSBpdCdzIGxpbmsKPiA+IHdhcyByZXBsYWNlZCBieSBhIG5ldyBhc20vc3Bp bmxvY2suaC4KPiA+Cj4gPiBZb3UgYWRkIHFzcGlubG9jay5oIHRvIGdlbmVyaWMteSBiZWNhdXNl IGl0J3MgbmV3IGluIHJpc2N2LCBhbmQgYWRkCj4gPiBtY3Nfc3BpbmxvY2suaCBiZWNhdXNlIGl0 J3MgbmVlZGVkIGJ5IHFzcGlubG9jay5oLgo+ID4KPiA+IE9rLCBpdCBtYWtlcyBzZW5zZSBub3cu Cj4gPgo+ID4gU29ycnkgYWJvdXQgdGhpcyBub2lzZS4KPiA+IEkgd2FzIHVuYXdhcmUgb2YgaG93 IGdlbmVyaWMteSB3b3JrZWQsIGFuZCAod3JvbmdseSkKPiA+IGFzc3VtZWQgaXQgd2FzIGFib3V0 IGluY2x1ZGluZyBoZWFkZXJzIGF1dG9tYXRpY2FsbHkgaW4gdGhlIGJ1aWxkLgo+ID4KPiA+Cj4g PiA+Cj4gPiA+ICsjaWZkZWYgQ09ORklHX1FVRVVFRF9TUElOTE9DS1MKPiA+ID4gKyNpbmNsdWRl IDxhc20vcXNwaW5sb2NrLmg+Cj4gPiA+ICsjaW5jbHVkZSA8YXNtL3Fyd2xvY2suaD4KPiA+ID4g KyNlbHNlCj4gPiA+ICsjaW5jbHVkZSA8YXNtLWdlbmVyaWMvc3BpbmxvY2suaD4KPiA+ID4gKyNl bmRpZgo+ID4gPgo+ID4gPiBTbywgeW91IHdhbnQgbWU6Cj4gPiA+ICsjaWZkZWYgQ09ORklHX1FV RVVFRF9TUElOTE9DS1MKPiA+ID4gKyNpbmNsdWRlIDxhc20vcXNwaW5sb2NrLmg+Cj4gPiA+ICsj ZWxzZQo+ID4gPiArI2luY2x1ZGUgPGFzbS1nZW5lcmljL3RpY2tldF9zcGlubG9jay5oPgo+ID4g PiArI2VuZGlmCj4gPiA+Cj4gPiA+ICsjaW5jbHVkZSA8YXNtL3Fyd2xvY2suaD4KPiA+ID4KPiA+ ID4gUmlnaHQ/Cj4gPgo+ID4gTm8sIEkgZGlkbid0IG1lYW4gdGhhdC4KPiA+IEkgd2FzIGp1c3Qg d29ycmllZCBhYm91dCB0aGUgYXJjaF9zcGluXyooKSBpbnRlcmZhY2VzLCBidXQgdGhleSBzaG91 bGQgYmUKPiA+IGZpbmUuCj4gPgo+ID4gQlRXLCBhY2NvcmRpbmcgdG8ga2VybmVsIGRvYyBvbiBn ZW5lcmljLXksIHNob3VsZG4ndCBiZSBhIGJldHRlciBpZGVhIHRvCj4gPiBhZGQgJ3RpY2tldF9z cGlubG9jay5oJyB0byBnZW5lcmljLXksIGFuZCBpbmNsdWRlIGFib3ZlIGFzCj4gPiBhc20vdGlj a2V0X3NwaW5sb2NrLmg/Cj4gPgo+ID4gT3IgaXMgZ2VuZXJpYy15IHJlc2VydmVkIG9ubHkgZm9y IHN0dWZmIHdoaWNoIGlzIGluZGlyZWN0bHkgaW5jbHVkZWQgYnkKPiA+IG90aGVyIGhlYWRlcnM/ Cj4gSXQncyBva2F5IHRvIGFkZCBnZW5lcmljLXkgZm9yIHRpY2tldF9zcGlubG9jay5oLCBhbmQg SSdtIG9rYXkgd2l0aAo+IHRoZSBmb2xsb3dpbmc6Cj4gCj4gKyNpZmRlZiBDT05GSUdfUVVFVUVE X1NQSU5MT0NLUwo+ICsjaW5jbHVkZSA8YXNtL3FzcGlubG9jay5oPgo+ICsjZWxzZQo+ICsjaW5j bHVkZSA8YXNtL3RpY2tldF9zcGlubG9jay5oPgo+ICsjZW5kaWYKPiAKPiArI2luY2x1ZGUgPGFz bS9xcndsb2NrLmg+CgpJdCBkb2VzIGxvb2sgbW9yZSBpbnR1aXRpdmUsIHNvIEkgYW0gZ2xhZCBp dCB3b3JrcyBmb3IgeW91IDopCgpGV0lXOgpSZXZpZXdlZC1ieTogTGVvbmFyZG8gQnJhcyA8bGVv YnJhc0ByZWRoYXQuY29tPgoKVGhhbmtzIQpMZW8KCj4gCj4gPgo+ID4gVGhhbmtzIQo+ID4gTGVv Cj4gPgo+ID4gPgo+ID4gPiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gIGdlbmVyaWMteSAr PSBzcGlubG9ja190eXBlcy5oCj4gPiA+ID4gPiA+ID4gIGdlbmVyaWMteSArPSBxcndsb2NrLmgK PiA+ID4gPiA+ID4gPiAgZ2VuZXJpYy15ICs9IHFyd2xvY2tfdHlwZXMuaAo+ID4gPiA+ID4gPiA+ ICtnZW5lcmljLXkgKz0gcXNwaW5sb2NrLmgKPiA+ID4gPiA+ID4gPiAgZ2VuZXJpYy15ICs9IHVz ZXIuaAo+ID4gPiA+ID4gPiA+ICBnZW5lcmljLXkgKz0gdm1saW51eC5sZHMuaAo+ID4gPiA+ID4g PiA+IGRpZmYgLS1naXQgYS9hcmNoL3Jpc2N2L2luY2x1ZGUvYXNtL3NwaW5sb2NrLmggYi9hcmNo L3Jpc2N2L2luY2x1ZGUvYXNtL3NwaW5sb2NrLmgKPiA+ID4gPiA+ID4gPiBuZXcgZmlsZSBtb2Rl IDEwMDY0NAo+ID4gPiA+ID4gPiA+IGluZGV4IDAwMDAwMDAwMDAwMC4uYzY0NGE5MmQ0NTQ4Cj4g PiA+ID4gPiA+ID4gLS0tIC9kZXYvbnVsbAo+ID4gPiA+ID4gPiA+ICsrKyBiL2FyY2gvcmlzY3Yv aW5jbHVkZS9hc20vc3BpbmxvY2suaAo+ID4gPiA+ID4gPiA+IEBAIC0wLDAgKzEsMTcgQEAKPiA+ ID4gPiA+ID4gPiArLyogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAgKi8KPiA+ID4g PiA+ID4gPiArCj4gPiA+ID4gPiA+ID4gKyNpZm5kZWYgX19BU01fUklTQ1ZfU1BJTkxPQ0tfSAo+ ID4gPiA+ID4gPiA+ICsjZGVmaW5lIF9fQVNNX1JJU0NWX1NQSU5MT0NLX0gKPiA+ID4gPiA+ID4g PiArCj4gPiA+ID4gPiA+ID4gKyNpZmRlZiBDT05GSUdfUVVFVUVEX1NQSU5MT0NLUwo+ID4gPiA+ ID4gPiA+ICsjZGVmaW5lIF9RX1BFTkRJTkdfTE9PUFMgICAgICgxIDw8IDkpCj4gPiA+ID4gPiA+ ID4gKyNlbmRpZgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBBbnkgcmVhc29uIHRoZSBhYm92ZSBk ZWZpbmUgY291bGRuJ3QgYmUgbWVyZ2VkIG9uIHRoZSBpZmRlZiBiZWxvdz8KPiA+ID4gPiA+IEVh c3kgZm9yIHRoZSBuZXh0IHBhdGNoIHRvIG1vZGlmeS4gU2VlIFdhaW1hbidzIGNvbW1lbnQ6Cj4g PiA+ID4gPgo+ID4gPiA+ID4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtcmlzY3YvNGNj NzExM2EtMGU0ZS03NjNhLWNiYTItNzk2M2JjZDI2YzdhQHJlZGhhdC5jb20vCj4gPiA+ID4gPgo+ ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9zcGlubG9jay5o IGIvYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9zcGlubG9jay5oCj4gPiA+ID4gPiA+IGluZGV4IGM2 NDRhOTJkNDU0OC4uOWViM2FkMzFlNTY0IDEwMDY0NAo+ID4gPiA+ID4gPiAtLS0gYS9hcmNoL3Jp c2N2L2luY2x1ZGUvYXNtL3NwaW5sb2NrLmgKPiA+ID4gPiA+ID4gKysrIGIvYXJjaC9yaXNjdi9p bmNsdWRlL2FzbS9zcGlubG9jay5oCj4gPiA+ID4gPiA+IEBAIC03LDExICs3LDk0IEBACj4gPiA+ ID4gPiA+ICAgI2RlZmluZSBfUV9QRU5ESU5HX0xPT1BTICgxIDw8IDkpCj4gPiA+ID4gPiA+ICAg I2VuZGlmCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gSSBzZWUgd2h5IHlvdSBzZXBh cmF0ZWQgdGhlIF9RX1BFTkRJTkdfTE9PUFMgb3V0Lgo+ID4gPiA+ID4KPiA+ID4gPgo+ID4gPiA+ IEkgc2VlLCBzaG91bGQgYmUgZmluZSB0aGVuLgo+ID4gPiA+Cj4gPiA+ID4gVGhhbmtzIQo+ID4g PiA+IExlbwo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ICsKPiA+ ID4gPiA+ID4gPiArI2lmZGVmIENPTkZJR19RVUVVRURfU1BJTkxPQ0tTCj4gPiA+ID4gPiA+ID4g KyNpbmNsdWRlIDxhc20vcXNwaW5sb2NrLmg+Cj4gPiA+ID4gPiA+ID4gKyNpbmNsdWRlIDxhc20v cXJ3bG9jay5oPgo+ID4gPiA+ID4gPiA+ICsjZWxzZQo+ID4gPiA+ID4gPiA+ICsjaW5jbHVkZSA8 YXNtLWdlbmVyaWMvc3BpbmxvY2suaD4KPiA+ID4gPiA+ID4gPiArI2VuZGlmCj4gPiA+ID4gPiA+ ID4gKwo+ID4gPiA+ID4gPiA+ICsjZW5kaWYgLyogX19BU01fUklTQ1ZfU1BJTkxPQ0tfSCAqLwo+ ID4gPiA+ID4gPiA+IC0tCj4gPiA+ID4gPiA+ID4gMi4zNi4xCj4gPiA+ID4gPiA+ID4KPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gVGhhbmtzIQo+ID4gPiA+ID4gPiBMZW8KPiA+ID4gPiA+ID4KPiA+ ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gLS0KPiA+ID4gPiA+IEJlc3QgUmVnYXJkcwo+ID4g PiA+ID4gIEd1byBSZW4KPiA+ID4gPiA+Cj4gPiA+ID4KPiA+ID4KPiA+ID4KPiA+ID4gLS0KPiA+ ID4gQmVzdCBSZWdhcmRzCj4gPiA+ICBHdW8gUmVuCj4gPiA+Cj4gPgo+IAo+IAo+IC0tIAo+IEJl c3QgUmVnYXJkcwo+ICBHdW8gUmVuCj4gCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxp c3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0 aW5mby9saW51eC1yaXNjdgo=