From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78D5115EFA2 for ; Wed, 24 Apr 2024 14:39:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713969581; cv=none; b=m8EYBTHvXS5+PEukEXNcL6DTFQsG8JT/yuXoK0BbEcEPkTrjS30y7Nt9NaBHp4rn+Mwuv9jyRaF7HjT+cjCv6GgYXw4I/K/hh0Qg2olExWlD195ty4m3j7W4ef/9qbtMinNVUeVnB70TtfpvmycpW7p9aNCeYrTrGgQ3+mtBCU8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713969581; c=relaxed/simple; bh=T5BKJjSvvOmVV87OlPWfEi/EP0teOTg1hdzodl7DUC0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=p/UsAdpz4fTMX5CBNkaG/+IyvaIpbzP0UhfnOlgIFSXQJvD3I5Of+8p5vfM7XBVTI909ft/V725DN8b/dHPqnbqMEqHaLTrSVyJluMbQ3MncE5o3byrtB8pHUMZCqNp95ayBH2Cy3URCqnFVqrf1b6cOgWQU0flUGPnR3tNeMS8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mlNHTXKv; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mlNHTXKv" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61b76ab0b46so36111827b3.1 for ; Wed, 24 Apr 2024 07:39:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713969578; x=1714574378; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9iiaCPly7txqpV5xrsjcvYa7r1oSnb/9cSgMPGsvZnE=; b=mlNHTXKvEJaO1e6RZp3hmG6BMeGh24i7ng8xRCDLeEciSw5N3zALxjB4ZipnFYhJUG KkM0KJ6y8yRozodbm7heKurg7TcuvZu2XEIlx4Regl1Ge/9l+X+TgwdygxX2AcotS6wN QNfFSZiIjostSiPJGUZAufmMKahJ5m8KSDSQ5iOeS6RecA49tSR8kdmyfZ9kHaJ4HLpp dqUWvcfwkAIa4c3Mtl2BpgxjKpSc00uUcEagCdqTXDnZSU+7miUvANxedJjWv5A6w3T+ JVMYS+TtlTzba0jXdW1GarQOoeU2RG6wH52FvnAo1DPenmzfjHT2o5adMykbpd2nYoRI 8AlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713969578; x=1714574378; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9iiaCPly7txqpV5xrsjcvYa7r1oSnb/9cSgMPGsvZnE=; b=MdvlZJkMfx6x4mWjgMoDlKt4I3KDjyEUPKTXja5x2oYoPQE+SPfLaYU0Rz2GwXhYxj C/wSjgWUgpyAoA0TEC9ssFMXOzd+x5tfTqa4V/LJqxJ62i1AiC/AiArrjkpAsUS88SeC Cuwfj44sAS8n9TwFzITvkWdIghYhABLGyu1MnRncl721F8T2/TUiKKhd99AT373tUNsm g5Qf0RFV8ruDww1NUCChira7E80VMm+4jZ8BkJ6TgjwzVP/EYw/hrsbdZ6mER7wm4iul /EE62HTnicKhc4sZMydefYmWgNOtF+W3+9AakK1vDE9Udji8PPL8SgzjZQdxMSsNgSdf QkIw== X-Forwarded-Encrypted: i=1; AJvYcCUkoVZBKgNLvtk6JXDIwtyL4H6JrLWqi4pjZf4WtR4pKBjC0act0fepth6Mt5UNBROg37Hy7RqEnqTAsURXfbCb+4zA X-Gm-Message-State: AOJu0YwDJWpxFEvbRRXjaYPc6i6/bveBV7pgkIFgIyqXpV52y0Ei32Qa x9zWuhdqd9411sJoKsILjsvLNxhKtrAdhrPmeHlHay1zTeAIZi1iT8u8XY3WJPFTdyw5UyYz3sG ipQ== X-Google-Smtp-Source: AGHT+IFwI+NUBwgT4LFDfuVdz9Xo3ZIpDQDyL8ZT8aZdM2HzLnWPcne09NNC7ebB5WdlqxUK2nmaBs2zktg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:ccd8:0:b0:615:ca8:6058 with SMTP id o207-20020a0dccd8000000b006150ca86058mr525187ywd.5.1713969578601; Wed, 24 Apr 2024 07:39:38 -0700 (PDT) Date: Wed, 24 Apr 2024 07:39:37 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240409133959.2888018-1-pgonda@google.com> <20240409133959.2888018-5-pgonda@google.com> Message-ID: Subject: Re: [PATCH 4/6] Add GHCB allocations and helpers From: Sean Christopherson To: Peter Gonda Cc: linux-kernel@vger.kernel.org, Vishal Annapurve , Ackerley Tng , Paolo Bonzini , Claudio Imbrenda , Carlos Bilbao , Tom Lendacky , Michael Roth , kvm@vger.kernel.org, linux-kselftest@vger.kernel.org Content-Type: text/plain; charset="us-ascii" On Tue, Apr 23, 2024, Sean Christopherson wrote: > On Tue, Apr 09, 2024, Peter Gonda wrote: > > Add GHCB management functionality similar to the ucall management. > > Allows for selftest vCPUs to acquire GHCBs for their usage. > > Do we actually need a dedicated pool of GHCBs? The conundrum with ucalls is that > we have no place in the guest to store the pointer on all architectures. Or rather, > we were too lazy to find one. :-) > > But for SEV-ES, we have MSR_AMD64_SEV_ES_GHCB, and any test that clobbers that > obviously can't use ucalls anyways. Argh, but we can't get a value in that MSR > from the host. ... > Anyways, back to selftests. Since we apparently can't stuff MSR_AMD64_SEV_ES_GHCB > from host userspace, what if we instead use a trampoline? Instead having > vcpu_arch_set_entry_point() point directly at guest_code, point it at a trampoline > for SEV-ES guests, and then have the trampoline set MSR_AMD64_SEV_ES_GHCB to > the vCPU-specific GHCB before invoking guest_code(). > > Then we just need a register to stuff the GHCB into. Ah, and the actual guest > entry point. GPRs are already part of selftest's "ABI", since they're set by > vcpu_args_set(). And this is all 64-bit only, so we can use r10+. > > Ugh, the complication is that the trampoline would need to save/restore RAX, RCX, > and RDX in order to preserve the values from vcpu_args_set(), but that's just > annoying, not hard. And I think it'd be less painful overall than > having to create a GHCB pool? > > In rough pseudo-asm, something like this? > > static void vcpu_sev_es_guest_trampoline(void) > { > asm volatile( > "mov %%r15d, %%eax\n\t" > "shr %%r15, $32\n\t" > "mov %%r15d, %%eax\n\t" > "mov $MSR_AMD64_SEV_ES_GHCB, %%ecx\n\t" > > "jmp %%r14") > } Scratch using inline asm, it needs to be a proper asm subroutine, as it's possible the compiler could clobber GPRs before emitting the asm. But writing actual assembly code is probably a good thing. And we need assembly for TDX selftests, which forces vCPUs to start at the RESET vector[*]. Rather than add a TDX specific td_boot.S, we can add a common-ish entry.S to hold all of the CoCo entry points that need to be in assembly. Then I think we'll eventually end up with something like: diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index fd94a1bd82c9..03818d3c4669 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -597,7 +597,12 @@ void vcpu_arch_set_entry_point(struct kvm_vcpu *vcpu, void *guest_code) struct kvm_regs regs; vcpu_regs_get(vcpu, ®s); - regs.rip = (unsigned long) guest_code; + if () + regs.r14 = guest_code; + else if () + + else + regs.rip = (unsigned long) guest_code; vcpu_regs_set(vcpu, ®s); } @@ -635,6 +640,10 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id) vcpu_regs_get(vcpu, ®s); regs.rflags = regs.rflags | 0x2; regs.rsp = stack_vaddr; + if () { + regs.rip = vcpu_sev_es_guest_trampoline; + regs.r15 = (); + } vcpu_regs_set(vcpu, ®s); /* Setup the MP state */ --- [*] https://lore.kernel.org/all/20231212204647.2170650-6-sagis@google.com