From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5EDD0C433F5 for ; Wed, 2 Mar 2022 13:46:56 +0000 (UTC) Received: from localhost ([::1]:44608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nPPJf-0004U0-C5 for qemu-devel@archiver.kernel.org; Wed, 02 Mar 2022 08:46:55 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47896) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nPO4F-0001c8-KF for qemu-devel@nongnu.org; Wed, 02 Mar 2022 07:26:55 -0500 Received: from [2607:f8b0:4864:20::52e] (port=41514 helo=mail-pg1-x52e.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nPO4D-0000h2-JD for qemu-devel@nongnu.org; Wed, 02 Mar 2022 07:26:55 -0500 Received: by mail-pg1-x52e.google.com with SMTP id o26so1484183pgb.8 for ; Wed, 02 Mar 2022 04:26:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2T+NCiBAVUTU368d4Yz1z1btdsU5CgR5N/ysNvhlN6k=; b=e6LI0lF1mguaCrofiTCr2Be66Z9DLbOOv/hv2ATMkGegZrlFCKNok3rnNBHrRPMNyS JyKXTr0lwUPSNGuxolZq9/U5/np3FbSgBcUglEszjUGTXcbwaKcGQ37t4b2OEar4UxKY UAhqiVMxDBb4pygDNPG+lXQMeQ8g8gqLhzK2iT23Wr2CKh3PO6nMnrd5Aw13IrpaBmll /lHlBRBbqJimIoToCUNHhhX0sSlJ0pHGdgf3WjrftbqwOF1Q+1/ryVVs+yaKEDqR597c AIGfYLXLfKyNya5dc2sRdJCt8xY+t9nkxn0xLs4ZiV6IFgOvySQx2kt5AoWFlut53yz0 p9Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2T+NCiBAVUTU368d4Yz1z1btdsU5CgR5N/ysNvhlN6k=; b=68dcpP4Wp1mMYqsT3tYHbDk4P+ffZf1z1zy9rdi//h2aI9KFw6ZearJYKTV4ctBmsa I48YB+jhdnWRBUYX9s/x/x6wwLSiOcos8B2uin/eJ7qxA7zMlB6mfu5yY+wSwqWgb5ai aoLNLAC9RssJRZgtG7cZU7qltuPulh554ylkrTcJfzCfrBCvbvVOgbgmU9TUzvXOz9IT geh5/GF34uSDEvoBKlEZSUeZpvm1ZsgWAzUy+RIQqLZh5SzSenycmkCk8yxbujrJ+ga8 eoyXUtJlL178jDKD/u2+ShJ48KKTQCvGmBIyFayK8imZWbHwe3LWeKFM47j20aXj03AD 4zrw== X-Gm-Message-State: AOAM530lHOl1Yf82UMDYhxqg8i3l+eu7lQgOwzinHrJVlop+intcGPWD ttmRHL1+linug6gR/46o8WzzAIQiugmYjg== X-Google-Smtp-Source: ABdhPJyHQiddXhD1g8aW96Wrtl3zhO4ZK/eIMZE/pnDdmXQI6qCjNNwtbfLAY1neW6Ji9qTzTsEaYw== X-Received: by 2002:a63:318f:0:b0:378:96fe:fdfb with SMTP id x137-20020a63318f000000b0037896fefdfbmr13231009pgx.120.1646224011903; Wed, 02 Mar 2022 04:26:51 -0800 (PST) Received: from localhost.localdomain (2001-b011-e000-59d7-a02b-4f1b-c415-11a0.dynamic-ip6.hinet.net. [2001:b011:e000:59d7:a02b:4f1b:c415:11a0]) by smtp.gmail.com with ESMTPSA id mu1-20020a17090b388100b001bedddf2000sm4912971pjb.14.2022.03.02.04.26.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 02 Mar 2022 04:26:51 -0800 (PST) From: Yan-Jie Wang To: qemu-devel@nongnu.org Subject: [PATCH v2 5/9] hvf: fix memory dirty-tracking Date: Wed, 2 Mar 2022 20:26:07 +0800 Message-Id: <20220302122611.15237-6-ubzeme@gmail.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220302122611.15237-1-ubzeme@gmail.com> References: <20220302122611.15237-1-ubzeme@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::52e (failed) Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=ubzeme@gmail.com; helo=mail-pg1-x52e.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Roman Bolshakov , Alexander Graf , Cameron Esfahani , Yan-Jie Wang Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Dirty-tracking in HVF is not properly implemented. On Intel Macs, Ubuntu ISO boot menu does not show properly. On Apple Silicon, using bochs-display may cause the guest crashes because the guest may uses load/store instructions on framebuffer which causes vmexits and the exception register does not contain enough information (ESR_EL2.ISV = 0) for QEMU to emulate the memory operation. The strategy to log the dirty pages is to write-protect the memory regions that are being dirty-tracked. When the guest is trapped to the host because of memory write, check whether the address being written is being dirty-tracked. If it is being dirty-tracked, restore the write permission of the page and mark the accessed page dirty, and resume the guest without increasing program counter, and then the same instruction will be execute again. This patch fixes the problem and make the dirty-tracking work properly. Buglink: https://bugs.launchpad.net/qemu/+bug/1827005 Signed-off-by: Yan-Jie Wang --- accel/hvf/hvf-mem.c | 62 ++++++++++++++++++++++++++++++++++++---- include/sysemu/hvf_int.h | 14 +-------- target/arm/hvf/hvf.c | 5 ++++ target/i386/hvf/hvf.c | 25 ++++------------ 4 files changed, 68 insertions(+), 38 deletions(-) diff --git a/accel/hvf/hvf-mem.c b/accel/hvf/hvf-mem.c index b8e9f30e4c..896e718374 100644 --- a/accel/hvf/hvf-mem.c +++ b/accel/hvf/hvf-mem.c @@ -30,9 +30,21 @@ #define HVF_NUM_SLOTS 32 +/* HVFSlot flags */ +#define HVF_SLOT_LOG (1 << 0) +#define HVF_SLOT_READONLY (1 << 1) + +typedef struct HVFSlot { + hwaddr start; + hwaddr size; /* 0 if the slot is free */ + hwaddr offset; /* offset within memory region */ + uint32_t flags; + MemoryRegion *region; +} HVFSlot; + static HVFSlot memslots[HVF_NUM_SLOTS]; -HVFSlot *hvf_find_overlap_slot(hwaddr start, hwaddr size) +static HVFSlot *hvf_find_overlap_slot(hwaddr start, hwaddr size) { HVFSlot *slot; int x; @@ -194,7 +206,7 @@ static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool on) static void hvf_log_start(MemoryListener *listener, MemoryRegionSection *section, int old, int new) { - if (old != 0) { + if (old == new) { return; } @@ -211,12 +223,12 @@ static void hvf_log_stop(MemoryListener *listener, hvf_set_dirty_tracking(section, 0); } -static void hvf_log_sync(MemoryListener *listener, +static void hvf_log_clear(MemoryListener *listener, MemoryRegionSection *section) { /* - * sync of dirty pages is handled elsewhere; just make sure we keep - * tracking the region. + * The dirty bits are being cleared. + * Make the section write-protected again. */ hvf_set_dirty_tracking(section, 1); } @@ -240,9 +252,47 @@ static MemoryListener hvf_memory_listener = { .region_del = hvf_region_del, .log_start = hvf_log_start, .log_stop = hvf_log_stop, - .log_sync = hvf_log_sync, + .log_clear = hvf_log_clear, }; + +/* + * The function is called when the guest is accessing memory causing vmexit. + * Check whether the guest can access the memory directly and + * also mark the accessed page being written dirty + * if the page is being dirty-tracked. + * + * Return true if the access is within the mapped region, + * otherwise return false. + */ +bool hvf_access_memory(hwaddr address, bool write) +{ + HVFSlot *slot; + hv_return_t ret; + hwaddr start, size; + + slot = hvf_find_overlap_slot(address, 1); + + if (!slot || (write && slot->flags & HVF_SLOT_READONLY)) { + /* MMIO or unmapped area, return false */ + return false; + } + + if (write && (slot->flags & HVF_SLOT_LOG)) { + /* The slot is being dirty-tracked. Mark the accessed page dirty. */ + start = address & qemu_real_host_page_mask; + size = qemu_real_host_page_size; + + memory_region_set_dirty(slot->region, + start - slot->start + slot->offset, size); + ret = hv_vm_protect(start, size, + HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC); + assert_hvf_ok(ret); + } + + return true; +} + void hvf_init_memslots(void) { memory_listener_register(&hvf_memory_listener, &address_space_memory); diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h index 0aafbc9357..16e5faf0ff 100644 --- a/include/sysemu/hvf_int.h +++ b/include/sysemu/hvf_int.h @@ -17,18 +17,6 @@ #include #endif -/* HVFSlot flags */ -#define HVF_SLOT_LOG (1 << 0) -#define HVF_SLOT_READONLY (1 << 1) - -typedef struct HVFSlot { - hwaddr start; - hwaddr size; /* 0 if the slot is free */ - hwaddr offset; /* offset within memory region */ - uint32_t flags; - MemoryRegion *region; -} HVFSlot; - typedef struct hvf_vcpu_caps { uint64_t vmx_cap_pinbased; uint64_t vmx_cap_procbased; @@ -58,11 +46,11 @@ int hvf_arch_init(void); int hvf_arch_init_vcpu(CPUState *cpu); void hvf_arch_vcpu_destroy(CPUState *cpu); int hvf_vcpu_exec(CPUState *); -HVFSlot *hvf_find_overlap_slot(hwaddr, hwaddr); int hvf_put_registers(CPUState *); int hvf_get_registers(CPUState *); void hvf_kick_vcpu_thread(CPUState *cpu); +bool hvf_access_memory(hwaddr address, bool write); void hvf_init_memslots(void); #endif diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c index 4d4ddab348..398ad50a29 100644 --- a/target/arm/hvf/hvf.c +++ b/target/arm/hvf/hvf.c @@ -1202,6 +1202,11 @@ int hvf_vcpu_exec(CPUState *cpu) break; } + if (iswrite && + hvf_access_memory(hvf_exit->exception.physical_address, 1)) { + break; + } + assert(isv); if (iswrite) { diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index 2ddb4fc825..c4c544dc54 100644 --- a/target/i386/hvf/hvf.c +++ b/target/i386/hvf/hvf.c @@ -113,7 +113,7 @@ void hvf_handle_io(CPUArchState *env, uint16_t port, void *buffer, } } -static bool ept_emulation_fault(HVFSlot *slot, uint64_t gpa, uint64_t ept_qual) +static bool ept_emulation_fault(uint64_t gpa, uint64_t ept_qual) { int read, write; @@ -129,14 +129,6 @@ static bool ept_emulation_fault(HVFSlot *slot, uint64_t gpa, uint64_t ept_qual) return false; } - if (write && slot) { - if (slot->flags & HVF_SLOT_LOG) { - memory_region_set_dirty(slot->region, gpa - slot->start, 1); - hv_vm_protect((hv_gpaddr_t)slot->start, (size_t)slot->size, - HV_MEMORY_READ | HV_MEMORY_WRITE); - } - } - /* * The EPT violation must have been caused by accessing a * guest-physical address that is a translation of a guest-linear @@ -147,14 +139,11 @@ static bool ept_emulation_fault(HVFSlot *slot, uint64_t gpa, uint64_t ept_qual) return false; } - if (!slot) { - return true; + if (hvf_access_memory(gpa, write)) { + return false; } - if (!memory_region_is_ram(slot->region) && - !(read && memory_region_is_romd(slot->region))) { - return true; - } - return false; + + return true; } void hvf_arch_vcpu_destroy(CPUState *cpu) @@ -469,7 +458,6 @@ int hvf_vcpu_exec(CPUState *cpu) /* Need to check if MMIO or unmapped fault */ case EXIT_REASON_EPT_FAULT: { - HVFSlot *slot; uint64_t gpa = rvmcs(cpu->hvf->fd, VMCS_GUEST_PHYSICAL_ADDRESS); if (((idtvec_info & VMCS_IDT_VEC_VALID) == 0) && @@ -477,9 +465,8 @@ int hvf_vcpu_exec(CPUState *cpu) vmx_set_nmi_blocking(cpu); } - slot = hvf_find_overlap_slot(gpa, 1); /* mmio */ - if (ept_emulation_fault(slot, gpa, exit_qual)) { + if (ept_emulation_fault(gpa, exit_qual)) { struct x86_decode decode; load_regs(cpu); -- 2.32.0 (Apple Git-132)