From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC13919DF9E for ; Tue, 22 Oct 2024 19:00:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623656; cv=none; b=WRy9+KHiQC+/QmyyvhtcXXyb1yqbkYCSNruE5mb1icRxBMQYp2Dl4VJ249v4vzXlvZbPNC5h32vgg3BnBgusq11BSCxGd9uKv94ITm+VLXat7XIDiRajABKsThR8i89OQD69a2j+TA/GdHoNKgWKkzXrNCfoEke2l0thEKlwyRQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623656; c=relaxed/simple; bh=ha7ePI1h8iQoHbzruRXiN0x3RDdtMaNlStfwZOXb78M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=O98b9RGogH2LxEGhEFpETFBHcmYuwZ0NzocKEArF94aiAM7NKQnEBmT+sekBBTIXGKRKvpDp3urpJlRUNiPQ1nnipJ1siCJTTIMQOOSNdoHYKdFzBYEdmJOSwgVBAd/JMtsiyOt+Q7XqNLlKLqy+5GDUzHRuim4tF/E10nZka2Q= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xUqhr7lZ; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xUqhr7lZ" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e2974759f5fso208793276.0 for ; Tue, 22 Oct 2024 12:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729623654; x=1730228454; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DrBrn35LjV4fle0zz8oDGlad/5uSTJzGLTSWOS1Px4s=; b=xUqhr7lZFnSmUfiPXod/taBrdL8iUwxIdxucne9INpjjeLO+laNnA2keROwswijqHA mndgF8xUTrGuEmilSG4flKvNWfETM30/EVjV8WOQ9nti/HBzAizSjBBHKc2GeJw+x4KW CXHxgEF9FHuaDT6r1caYHo7JxDFuDJyftkX+tBqup1tBacsIndjLUxVO/buQSFVbyDrY l6zYc4IHOT+Av5Zemb/XZaeaxPNDd9kBUZ1HFGid/yhGRaK5O4Bzd1nlF2wqbz1/DevO da4OCRmJR3QuC5Tl2CQ1Ba5kNdJzJJDOD4dSM1sWd5jGWxMiA2zN+8OC+ZsGGBgJRvJT I2aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729623654; x=1730228454; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DrBrn35LjV4fle0zz8oDGlad/5uSTJzGLTSWOS1Px4s=; b=tbkt9fKISYYtVsFqued6GeV/2XBD1tzYYK6no2qmy8k6s160w+KfsXMZuWoxklwqki Q0pqmKD+5rDFKo3dB3HZvWe3UXN0PW5vHiI8Nu/jYetfz4sLBh0b/YXoPih7KOWgLp0K CNprZgWPIMbMTVfc4bAdBgbqWG2P2gN47OveImxe1L8m5VxOKhm/R0k9Vck5ZSeiPD89 Y71te+9XsxLXQt07YZ9N55f3cex2vyEi67Mvowv6ij6oYeGVzMP3YvXOC18SvDblk1fx qA2W5pn7dh6JXsNH81SNjgLGh2tWwt0IMx08lJp/HAnfDi/y5dxBpQLoPPhmM/acrKI6 lTsw== X-Forwarded-Encrypted: i=1; AJvYcCUU+/Wfse3mnGDfyDdgwP2maw9h8v/m4nQGU+jzzyMOPRXEVk1A/BVvfH9ZItP5LGh5u43vZw==@lists.linux.dev X-Gm-Message-State: AOJu0YzJysQqASzgnilwD8KyGA0kqL20lj/u5rVYLWy1jB7DNVua0eIl Yz1YTEp4fAN4CopqZH3hGeD+7VQRZAJQlVebKnbrhwysIM53CTtolNnzIy1G5PQ+CMu7RZ009wb eMw== X-Google-Smtp-Source: AGHT+IHbl0Q/eeAVwKL16JD6RI9BME7YNfz+VrN478T5E+rYVmLa3hGPlShaVUdcv9jsDUrZk7bS7mixkfY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:aa12:0:b0:e2e:2c2e:277b with SMTP id 3f1490d57ef6-e2e2c2e316cmr19368276.3.1729623653976; Tue, 22 Oct 2024 12:00:53 -0700 (PDT) Date: Tue, 22 Oct 2024 12:00:52 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231002115723.175344-1-mlevitsk@redhat.com> <1d6044e0d71cd95c477e319d7e47819eee61a8fc.camel@redhat.com> Message-ID: Subject: Re: [PATCH v3 0/4] Allow AVIC's IPI virtualization to be optional From: Sean Christopherson To: Maxim Levitsky Cc: kvm@vger.kernel.org, Will Deacon , linux-kernel@vger.kernel.org, Borislav Petkov , Dave Hansen , x86@kernel.org, Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Joerg Roedel , Suravee Suthikulpanit , Robin Murphy , iommu@lists.linux.dev, Paolo Bonzini Content-Type: text/plain; charset="us-ascii" On Mon, Oct 21, 2024, Sean Christopherson wrote: > On Wed, Oct 04, 2023, Maxim Levitsky wrote: > > About the added 'vcpu->loaded' variable, I added it also because it is > > something that is long overdue to be added, I remember that in IPIv code > > there was also a need for this, and probalby more places in KVM can be > > refactored to take advantage of it, instead of various hacks. > > I don't view using the information from the Physical ID table as a hack. It very > explicitly uses the ir_list_lock to ensure that the pCPU that's programmed into > the IRTE is the pCPU on which the vCPU is loaded, and provides rather strict > ordering between task migration and device assignment. It's not a super hot path, > so I don't think lockless programming is justified. > > I also think we should keep IsRunning=1 when the vCPU is unloaded. That approach > won't run afoul of your concern with signaling the wrong pCPU, because KVM can > still keep the ID up-to-date, e.g. if the task is migrated when a pCPU is being > offlined. > > The motiviation for keeping IsRunning=1 is to avoid unnecessary VM-Exits and GA > log IRQs. E.g. if a vCPU exits to userspace, there's zero reason to force IPI > senders to exit, because KVM can't/won't notify userspace, and the pending virtual > interrupt will be processed on the next VMRUN. My only hesitation to keeping IsRunning=1 is that there could, in theory, be a noisy neighbor problem. E.g. if there is meaningful overhead when the CPU responds to the doorbell. Hrm, and if another vCPU is scheduled in on the same pCPU, that vCPU could end up processing a virtual interrupt in response to a doorbell intended for a different vCPU. The counter-argument to both concerns is that APICv Posted Interrupts have had a _worse_ version of that behavior for years, and no one has complained. KVM sets PID.SN only when a vCPU is _preempted_, and so devices (and now virtual IPIs) will send notification IRQs to pCPUs that aren't actively running the vCPU, or are running a different vCPU. The counter-counter-argument is that (a) IPI virtualization is a recent addition, and device posted interrupts are unlikely to be used in a CPU oversubscribed setup, and (b) Posted Interrupts are effectively rate-limited to a single "spurious" notification per vCPU, as notification IRQs are sent if and only if PID.ON=0. That said, while I'm somewhat less confident that keeping IsRunning=1 is desirable for all use cases than I was yesterday, I still think we should avoid tightly coupling it to whether or not the vCPU is loaded, because there are undoubtedly setups where it _is_ desirable, e.g. if vCPUs are pinned 1:1 to pCPUs.