From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2053A3446CF for ; Wed, 26 Nov 2025 19:50:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.178 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764186641; cv=none; b=qfNXaHblWUt3yg+MK1cRobtpdXnEstRUNcHs0xRas7tg0LDKGSInhPzRgS104OQ7sRrZS8bEovdYM4j4DJOsvbJlYObCxpsofOpsVErHqv+iPMazWOouAbgwkYnWd7mimzPZ/PWcq+U8AlVDo81XLToWUbfw5sIVAnGNh+vKoVs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764186641; c=relaxed/simple; bh=EZu0RBNe8CrCV1HEExrFp8bYx7PPxrtsOnul+gMh2p0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NBksUFt5IEuj2ui4AzvqUQakvColFQnRddQuTyuIG/KLhYR11PK/aSNH80EmvYqLGG8XCrDWNwwcOxZBXjMIuyT3zELVj3qQt1smMn+aIdMBqv0UPug+Tik4PbQfLPosJhV7RA+2vYlJSqcesYI6hZLLfQEtSDH4hAd81LDeN7o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=c4pYJds/; arc=none smtp.client-ip=209.85.160.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="c4pYJds/" Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-4edf1be4434so1332681cf.1 for ; Wed, 26 Nov 2025 11:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1764186638; x=1764791438; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KWelU42poZIxy2dr4VLVf29hmxsrX+k4eX1Z6rP2/c4=; b=c4pYJds/ZA7SWaJp9Pl5AV5A7TReMyhRDV3qelemvKecWdwN4ugaxZzVW7xdAhSVZ1 BoEg4UIYn623YVmBTeINUW0XAn9Oh8FuAvnQYkebZx6GXCCiLsd3/jE5E5zqGaLBe0Pe x37WjuGVLfBrXrFbzyzuIyAC/D2Xm8gwmzUyEG8k95TZ1ua4FiaxTfRKtUJFXhgrR2LG ornrE5YMd8DUGzHdCuUoZrvj3sH7TgiFGe6F3Dg0RvB1UWKsqoZdlpyx/9Z0RKnkiTc2 eHGcAw5E5qxM/qig2ohVfSbUObqiCtxU0e3s9xpk9C55m4T9U4ngQOrlAwX24FBqW+jR IctQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764186638; x=1764791438; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KWelU42poZIxy2dr4VLVf29hmxsrX+k4eX1Z6rP2/c4=; b=Ysmr37FrbBXF2ggBhMwgeCshPD77Yd5B1HqRMiveQ2VfctJkt65GX/zQePayvs0360 sIUsRHtjZGIC75ZxqZj9/hASiSzUV+n5aWkE8pT1CjafR2UJMRGS9ekay3W9PQMkBi6K PpjZGez6ON2grBkcU6bnreWCca1pNS14GT+Zgq+G12GWdYO5qfn1bvGJCxB0y+5XEmuV hLoKkJ+7cIaKT54+dYSXhl3pT/1Mh7TQduEi3XmDNEIpKD1AeFgFAWt+DUTJb5Nnljdp AUBdaElCzzIuNLnuACX9C9FHtT09aC+8Dq2fjApLai96knFNPUdkWgw58WbRkXJM12ca NsZA== X-Forwarded-Encrypted: i=1; AJvYcCXDfWKuQ6Z0sIfoDwvb5UdxZcYUlBP3MtM9Jq41oj8MPb4xLqEhAhOvsLidH8N9lT5IPGhpL3MqtnbR2z0=@vger.kernel.org X-Gm-Message-State: AOJu0YwVcVwbPaJIABfUEMRFsvXJ66x7bWpmiotxUrvtV4+De5mMLrvb EO9G35uPa4szzsAfLepvtuSBCACS7lPe7XfOZ7Z+1Tzc7dDMq0nJNb7f0R/bNxKLv2A= X-Gm-Gg: ASbGncsX+PszVBH93em3T1RuHSSOxZZCTcxzxY/GzRrZhGJLuW6+DB/Y6sksQ9ELQZo QB2liIm//ReN2o06Jr1IMg5RwxEmeAUF5UNj+2mmVDcB+tG+iVuIHEkq1dwt3QLJpNsIaCV2Tx+ Q2660cTc3EJnNtDnHPzQ+qw70D3YqPwxUYZjOrTBpv6fCSqc/Csv5aRq7TKWaKWyGnEpfNak+zy ySMwErwXQV2ZsCrVLSPqknNfZ0V9wnfAhqIjR6rQ1ZsX2DDz12fi2gMf6hR9zbXYXtORh0m6yq9 QB8T7cdkRQHQSd+Cfyb64qaHSDN4kbrP2Y7mPddE+ZpuL6lJYLD5xORqJiYLqMmSMiHSyJ4pYK1 M9M/YpB1CJonvE/dh7+ZWJCIgr+Pr3jmvkDu9jY7Rnyl4s272QSnaqiVjdzfKGfXd3HCKXJAAr+ utdZqCkbtenybFiesGKhlqMsB4jUL/IMttsaVdAunEPsBDcjCwv7mscShM X-Google-Smtp-Source: AGHT+IFU4nFii2n3WUxd0X+nyAHy/D3Snp9YG7y11dVkiaGuGYgBbTLNY7VTEwyRxnAIGt2I+DWuLA== X-Received: by 2002:ac8:7dc5:0:b0:4ee:1bc7:9d8d with SMTP id d75a77b69052e-4efbd9141d4mr87183481cf.17.1764186638014; Wed, 26 Nov 2025 11:50:38 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-47-55-120-4.dhcp-dynamic.fibreop.ns.bellaliant.net. [47.55.120.4]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4efbd411763sm40074121cf.17.2025.11.26.11.50.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Nov 2025 11:50:37 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1vOLXA-000000036QV-10ww; Wed, 26 Nov 2025 15:50:36 -0400 Date: Wed, 26 Nov 2025 15:50:36 -0400 From: Jason Gunthorpe To: Borislav Petkov Cc: iommu@lists.linux.dev, Vasant Hegde , Joerg Roedel , Suravee Suthikulpanit , Will Deacon , Robin Murphy , linux-kernel@vger.kernel.org Subject: Re: amd iommu: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-.... } 8 jiffies s: 113 root: 0x1/. Message-ID: <20251126195036.GA738503@ziepe.ca> References: <20251126143219.GBaScPc1b6RAC9ZfCv@fat_crate.local> <20251126160031.GGaSckH3SHrC8ckPa7@fat_crate.local> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20251126160031.GGaSckH3SHrC8ckPa7@fat_crate.local> On Wed, Nov 26, 2025 at 05:00:31PM +0100, Borislav Petkov wrote: > + Vasant. > > On Wed, Nov 26, 2025 at 03:32:19PM +0100, Borislav Petkov wrote: > > Hi, > > > > this is latest Linus + latest tip/master. Box is Zen3. CCing AMD IOMMU folks > > because the backtrace points to it. > > > > Ideas? > > > > [ 12.946913] (journald)[506]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy > > [ 12.948083] (journald)[506]: Successfully forked off '(sd-mkuserns)' as PID 507. > > [ 12.977579] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-.... } 8 jiffies s: 113 root: 0x1/. > > [ 12.983638] rcu: blocking rcu_node structures (internal RCU debug): l=1:0-15:0x1/. > > [ 12.983644] Sending NMI from CPU 1 to CPUs 0: > > [ 12.983652] NMI backtrace for cpu 0 > > [ 12.983655] CPU: 0 UID: 0 PID: 504 Comm: (modprobe) Not tainted 6.18.0-rc7+ #1 PREEMPT(voluntary) > > [ 12.983658] Hardware name: Supermicro Super Server/H12SSL-i, BIOS 2.5 09/08/2022 > > [ 12.983660] RIP: 0010:delay_halt_mwaitx+0x37/0x40 > > [ 12.983665] Code: 01 31 d2 89 d1 48 05 00 d0 3a 83 0f 01 fa b8 ff ff ff ff b9 02 00 00 00 48 39 c6 48 0f 46 c6 48 89 c3 b8 f0 00 00 00 0f 01 fb <5b> e9 ae 2b 1d ff 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 > > [ 12.983666] RSP: 0018:ffffc90000003d88 EFLAGS: 00000097 > > [ 12.983668] RAX: 00000000000000f0 RBX: 0000000000000b9b RCX: 0000000000000002 > > [ 12.983670] RDX: 0000000000000000 RSI: 0000000000000b9b RDI: 00000034e14b49dc > > [ 12.983671] RBP: 00000034e14b49dc R08: 000000000000006c R09: 0000000000000002 > > [ 12.983672] R10: 0000000000000050 R11: 0000000000000002 R12: 00000000000001a4 > > [ 12.983673] R13: 0000000000000002 R14: ffff888100064018 R15: ffff888100064000 > > [ 12.983674] FS: 00007f8ae7452e00(0000) GS:ffff889086e58000(0000) knlGS:0000000000000000 > > [ 12.983675] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 12.983676] CR2: 00007f8ae6c9a8a0 CR3: 0000000141913004 CR4: 0000000000770ef0 > > [ 12.983677] PKRU: 55555554 > > [ 12.983678] Call Trace: > > [ 12.983680] > > [ 12.983682] delay_halt+0x3b/0x60 > > [ 12.983685] iommu_completion_wait.part.0.isra.0+0xd3/0x100 > > [ 12.983693] domain_flush_complete+0x64/0xc0 > > [ 12.983696] amd_iommu_flush_iotlb_all+0x33/0x50 > > [ 12.983700] fq_flush_timeout+0x34/0xd0 This code in the AMD driver is spinning until the HW completes some work Maybe the HW has alot of work to do, or is slow, and this is legitimately taking a long time? I don't expect amd_iommu_flush_iotlb_all() to have a problem, that's pretty weird. Does it eventually boot? Jason