From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 483E63358AE for ; Fri, 7 Nov 2025 17:37:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762537029; cv=none; b=TAquhGXiltovprJW0EAA/+zgZmrGsm/gv3Al/Z4rP8WNntSqJTxSjJ+7cPJn3zSdidbUOf7tCYge+69vvarKMybD42IinYoEFcc5BBFsrnsy6AMlTBViQZsWbkzB2w2irzScvi0rW6WAlgaC2qkgecnUf/50hPmdo53pWDdSmxk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762537029; c=relaxed/simple; bh=Lx3bFQ3NiP4uoTXJligat5YBVbYSgoMH3tTDsWSPIac=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=N9AshZzofWIP6JzP5ZNSOlnHLDSrE8L1SMAXGAaTsfoG/mhbt90twe5Wyb3U7kEFdbMKzt1fGot1X/JF3OY84VzVug7+NFs5ilSg9PUN9iHDc5SyyqIPc6UUSqvnuKRmdbu3xMx466RFvkX77B1Qgf4OEowAMAVoQe1dL5EGQ/o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iasIT9yN; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iasIT9yN" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-426ce339084so770561f8f.0 for ; Fri, 07 Nov 2025 09:37:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762537024; x=1763141824; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Zhz7gcsikgOjW44ogPO6UQeW+YT4AB9gKFb1aT581fI=; b=iasIT9yNyEvPFoZ+tIP8CMD4x32vXnXPfAD2PRCebxTyq5fCNsnuDoslDI46r9PIEz NTb5UJawktr3FVMV4ohBL9eQY+R8qaGvgUZa2Z4LdC2Rpkf9q96p25QqNfQ3uGQXq2OI 0MmWqZcN3osl8CqQKwRdH8YYPL+eCKSXudV96rXJbcQNC53UuJOwvxLZKBMx3hrx1AEa pieDXD4MvDArTv2K5GtBWH3JuoejKEOABc8dVF+2/QWGiwpmsZ1AGQDWyctTtEdObMpH dewvtm0NMAAQqjAvnZ7bYjNNZdkZyGBcVyOV4nekRs8VOiir6a+yTdedBgM6cbbydaHV CexQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762537024; x=1763141824; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Zhz7gcsikgOjW44ogPO6UQeW+YT4AB9gKFb1aT581fI=; b=YIyQ8KWi1NmS4kKi2iNVmxRZAxMTrvR9qfEElcoDdeWghILBiM7+ef3z8AQTgWZVyG LSgR0M9IdvuinlleQErIQHW4/sUxYOB8QCgwXcY+EH3DJBBvkH+8xhxJgpPVNo7p21YR 2BCDQwDUkgYVSezCxknF6Ibvib4pMxPI5TF51FT9aCtOFNGnKKcYVcRT4iXakmL4As/9 zDKqdfB392sU6CB6s80pBp+Hjfeq2aGQTccpj3Ckpe0z66vMPi+3AeA6BL0Z5RwJac5z BSDcOz67ZDirYKtkWkJa47h8csVBVGAVLTFwKQu8NINDdLQtD3c3tDSK3F4KEyMrxFlp m6hw== X-Forwarded-Encrypted: i=1; AJvYcCWeK1UlQ4bugzKHPsJS9QC3NKv6FWqtLhc3EuvtmurJ1VjjS3W3m8+UdJzdf/rmy/Y3H8/UH+k=@lists.linux.dev X-Gm-Message-State: AOJu0YweZrWY+nY872vtmyadyWfXh1N8l21WkUj7QkhGVRjCZ5BUgAod KZ33Yb1LJ5d5ZHmbuDu+4Q/ALruUqDlnOSF3/w1Nt+z7og6UwDumiCrnGF8J6YzDKov90ENm1BE +5ITtPchd1XDFPQ== X-Google-Smtp-Source: AGHT+IFJF2Ksc6u55CE+GD/K60cqff3jO8p1vj7rkxXgTU2hADlFy8fEI1U56A7kvOlzF5vcUeX5HHSZeZ5zgA== X-Received: from wrbfq11.prod.google.com ([2002:a05:6000:2a0b:b0:429:ccad:3298]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:178b:b0:429:d725:4114 with SMTP id ffacd0b85a97d-42ae5ac5374mr3668462f8f.39.1762537024246; Fri, 07 Nov 2025 09:37:04 -0800 (PST) Date: Fri, 07 Nov 2025 17:37:03 +0000 In-Reply-To: Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250924151101.2225820-1-patrick.roy@campus.lmu.de> X-Mailer: aerc 0.21.0 Message-ID: Subject: Re: [PATCH v7 00/12] Direct Map Removal Support for guest_memfd From: Brendan Jackman To: Brendan Jackman , Patrick Roy Cc: Patrick Roy , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Content-Type: text/plain; charset="UTF-8" On Fri Nov 7, 2025 at 3:54 PM UTC, Brendan Jackman wrote: > On Wed Sep 24, 2025 at 3:10 PM UTC, Patrick Roy wrote: >> From: Patrick Roy >> >> [ based on kvm/next ] >> >> Unmapping virtual machine guest memory from the host kernel's direct map is a >> successful mitigation against Spectre-style transient execution issues: If the >> kernel page tables do not contain entries pointing to guest memory, then any >> attempted speculative read through the direct map will necessarily be blocked >> by the MMU before any observable microarchitectural side-effects happen. This >> means that Spectre-gadgets and similar cannot be used to target virtual machine >> memory. Roughly 60% of speculative execution issues fall into this category [1, >> Table 1]. >> >> This patch series extends guest_memfd with the ability to remove its memory >> from the host kernel's direct map, to be able to attain the above protection >> for KVM guests running inside guest_memfd. >> >> Additionally, a Firecracker branch with support for these VMs can be found on >> GitHub [2]. >> >> For more details, please refer to the v5 cover letter [v5]. No >> substantial changes in design have taken place since. >> >> === Changes Since v6 === >> >> - Drop patch for passing struct address_space to ->free_folio(), due to >> possible races with freeing of the address_space. (Hugh) >> - Stop using PG_uptodate / gmem preparedness tracking to keep track of >> direct map state. Instead, use the lowest bit of folio->private. (Mike, David) >> - Do direct map removal when establishing mapping of gmem folio instead >> of at allocation time, due to impossibility of handling direct map >> removal errors in kvm_gmem_populate(). (Patrick) >> - Do TLB flushes after direct map removal, and provide a module >> parameter to opt out from them, and a new patch to export >> flush_tlb_kernel_range() to KVM. (Will) >> >> [1]: https://download.vusec.net/papers/quarantine_raid23.pdf >> [2]: https://github.com/firecracker-microvm/firecracker/tree/feature/secret-hiding > > I just got around to trying this out, I checked out this patchset using > its base-commit and grabbed the Firecracker branch. Things seem OK until > I set the secrets_free flag in the Firecracker config which IIUC makes > it set GUEST_MEMFD_FLAG_NO_DIRECT_MAP. > > If I set it, I find the guest doesn't show anything on the console. > Running it in a VM and attaching GDB suggests that it's entering the > guest repeatedly, it doesn't seem like the vCPU thread is stuck or > anything. I'm a bit clueless about how to debug that (so far, whenever > I've broken KVM, things always exploded very dramatically). I discovered that Firecracker has a GDB stub, so I can just attach to that and see what the guest is up to. The issue that the pvclock_vcpu_time_info in kvmclock is all zero: (gdb) backtrace #0 pvclock_tsc_khz (src=0xffffffff83a03000 ) at ../arch/x86/kernel/pvclock.c:28 #1 0xffffffff8109d137 in kvm_get_tsc_khz () at ../arch/x86/include/asm/kvmclock.h:11 #2 0xffffffff835c1842 in kvm_get_preset_lpj () at ../arch/x86/kernel/kvmclock.c:128 #3 kvmclock_init () at ../arch/x86/kernel/kvmclock.c:332 #4 0xffffffff835c1487 in kvm_init_platform () at ../arch/x86/kernel/kvm.c:982 #5 0xffffffff835a83df in setup_arch (cmdline_p=cmdline_p@entry=0xffffffff82e03f00) at ../arch/x86/kernel/setup.c:916 #6 0xffffffff83595a22 in start_kernel () at ../init/main.c:925 #7 0xffffffff835a7354 in x86_64_start_reservations ( real_mode_data=real_mode_data@entry=0x36326c0 ) at ../arch/x86/kernel/head64.c:507 #8 0xffffffff835a7466 in x86_64_start_kernel (real_mode_data=0x36326c0 ) at ../arch/x86/kernel/head64.c:488 #9 0xffffffff8103e7fd in secondary_startup_64 () at ../arch/x86/kernel/head_64.S:413 #10 0x0000000000000000 in ?? () (gdb) p *src $3 = {version = 0, pad0 = 0, tsc_timestamp = 0, system_time = 0, tsc_to_system_mul = 0, tsc_shift = 0 '\000', flags = 0 '\000', pad = "\000"} This causes a divide by zero in kvm_get_tsc_khz(). Probably the only reason I didn't see any console output is that I forgot to set earlyprintk, oops...