From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 169D8CFB43C for ; Mon, 7 Oct 2024 18:01:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NZv5qbbJUkGv24unDjpJ20uMGTPMYfnrD1RMmIDanKk=; b=cvOEpQmCxjJiZjWje4eojcjoVx YiasbBVGvR2hT3M/aemgY0nGQtb2pSioP2shYPYDkTFzxhUdSU/xkyiyZ/o1z8uwRYCDhdBu8vaOA z/0NdDmYnsIlT3ZeI/D0tugVhosHd/eTxGUgXyCHH0ePEvojtrvWR4T23eaKnY79/gn3sXWQ12f2Q mDq2d825owKS5M6tnD2Lp3xV3vVrhgomQamieHKUTVGuJKDtfWVKdFlS39hYiZfSDctTzulE189bv dDIihkuNn+GkXzfXRIeUy+2bO50tMBYK/C2B84fgs+vM38e8PWyctktiihQvkbZnTtUU3C5Us/uv0 9p14aLkg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sxs3I-00000003PWP-1JP6; Mon, 07 Oct 2024 18:01:48 +0000 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1sxs0S-00000003P78-1OTx for linux-arm-kernel@lists.infradead.org; Mon, 07 Oct 2024 17:58:54 +0000 Received: by mail-qt1-x82d.google.com with SMTP id d75a77b69052e-4583083d05eso40522291cf.3 for ; Mon, 07 Oct 2024 10:58:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1728323930; x=1728928730; darn=lists.infradead.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=NZv5qbbJUkGv24unDjpJ20uMGTPMYfnrD1RMmIDanKk=; b=GuDI8a+5AAIrhgPZ8Qa1lWCNEHcGmQV1Q/DlVBCVcx1UEH69x1wxR96wSm/vL4XbLq sM59h9QZSVDNqPzu6FP6vRX3AZyIUo25sHbP8PdtEwdhmP0NjKyA9P2j/+CnhxHBOUST gEtPZYZ/yCFm2zbzMpdolMlK4VqX+lkMi5DVA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728323930; x=1728928730; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NZv5qbbJUkGv24unDjpJ20uMGTPMYfnrD1RMmIDanKk=; b=pR6E/T90qHZuDtVORNPLiMhr15PJQwdUGyN1r9T4U55A/lc0CeQoaMDtsrZkeulzqq Ddtrf7IN7AOEe3GWIOVTWm9jJ8gmIYmz48ok20RAhEk3FQS7jyS8h9Kz3eK/xkk9tliy 0Sh1N1E5mI6LBrHy1xFhk/3yb+c3M/uPn3ULv2FwBaJWaI11STVXNIkrLNZPUuwE6g/D dxonjqUtcColCenaVkm6cvmiWa8ksRRoXZ0Kgg1j0fPinoqHIhI4Dy7uuVQENEZxQdzV nGimGaHMwiJankUUQHId9ORQjkRpRNc1/SiXfGWcYZ11H32h0D4ROHnRqbwFK+0LXJ+W 3eOg== X-Forwarded-Encrypted: i=1; AJvYcCWBle0KYMBx0hpM9ZsY430oRQf0EYEQeyuNx2lmfgH4e+PIWbNNUT+iyYUExV67LjECuXRjMFJXJ0uvmK+JSQXq@lists.infradead.org X-Gm-Message-State: AOJu0Yw9w1daADeMtBpty7gQNQlL5dXW5pOf9DNtFhUYUGwANE5+4ATj Cv7RE3INVWTYb6lxdWA8yt65F1ya7zA4L35Pz63kIexXOHrHoOTsMYCotRQEkmk0OMpmNnLwOPQ oY/Vg X-Google-Smtp-Source: AGHT+IG+Fp5D/JcuEV8vkTjYnQODmiuNTAMoMah19yYazQ/VNwju3XNOspP8rkDijc8pJy/SYC2w6A== X-Received: by 2002:a05:6214:318b:b0:6cb:2f11:6b9 with SMTP id 6a1803df08f44-6cb9a308154mr191960916d6.23.1728323930322; Mon, 07 Oct 2024 10:58:50 -0700 (PDT) Received: from [10.69.69.40] ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cba46cd225sm27918646d6.19.2024.10.07.10.58.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 07 Oct 2024 10:58:49 -0700 (PDT) Message-ID: <1ad5c4e9-9f98-40ab-afa4-a7939781e8cc@broadcom.com> Date: Mon, 7 Oct 2024 10:58:47 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] firmware: arm_scmi: Queue in scmi layer for mailbox implementation To: Cristian Marussi , Sudeep Holla Cc: arm-scmi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, peng.fan@nxp.com, bcm-kernel-feedback-list@broadcom.com, florian.fainelli@broadcom.com References: <20241004221257.2888603-1-justin.chen@broadcom.com> Content-Language: en-US From: Justin Chen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241007_105852_502452_AF6EA9E8 X-CRM114-Status: GOOD ( 19.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 10/7/24 6:10 AM, Cristian Marussi wrote: > On Mon, Oct 07, 2024 at 02:04:10PM +0100, Sudeep Holla wrote: >> On Fri, Oct 04, 2024 at 03:12:57PM -0700, Justin Chen wrote: >>> The mailbox layer has its own queue. However this confuses the per >>> message timeouts since the clock starts ticking the moment the messages >>> get queued up. So all messages in the queue have there timeout clocks >>> ticking instead of only the message inflight. To fix this, lets move the >>> queue back into the SCMI layer. >>> >> >> I think this has come up in the past. We have avoided adding addition >> locking here as the mailbox layer takes care of it. Has anything changed >> recently ? > > I asked for an explanation in my reply (we crossed each other mails probably) > since it alredy came up in the past a few times and central locking seemed not > to be needed...here the difference is about the reason...Justin talks about > message timeouts related to the queueing process..so I asked to better > explain the detail (and the anbomaly observed) since it still does not > seem to me that even in this case the lock is needed....anyway I can > definitely be woring of course :D > Hello Cristian, Thanks for the response. I'll try to elaborate. When comparing SMC and mailbox transport, we noticed mailbox transport timesout much quicker when under load. Originally we thought this was the latency of the mailbox implementation, but after debugging we noticed a weird behavior. We saw SMCI transactions timing out before the mailbox even transmitted the message. This issue lies in the SCMI layer. drivers/firmware/arm_scmi/driver.c do_xfer() function. The fundamental issue is send_message() blocks for SMC transport, but doesn't block for mailbox transport. So if send_message() doesn't block we can have multiple messages waiting at scmi_wait_for_message_response(). SMC looks like this CPU #0 SCMI message 0 -> calls send_message() then calls scmi_wait_for_message_response(), timesout after 30ms. CPU #1 SCMI message 1 -> blocks at send_message() waiting for SCMI message 0 to complete. Mailbox looks like this CPU #0 SCMI message 0 -> calls send_message(), mailbox layer queues up message, mailbox layer sees no message is outgoing and sends it. CPU waits at scmi_wait_for_message_response(), timesout after 30ms CPU #1 SCMI message 1 -> calls send_message(), mailbox layer queues up message, mailbox layer sees message pending, hold message in queue. CPU waits at scmi_wait_for_message_response(), timesout after 30ms. Lets say if transport takes 25ms. The first message would succeed, the second message would timeout after 5ms. Hopefully this makes sense. Justin