From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BF6FC4361A for ; Fri, 4 Dec 2020 16:58:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F0B322B3B for ; Fri, 4 Dec 2020 16:58:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F0B322B3B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1klEPG-0002QH-9K for qemu-devel@archiver.kernel.org; Fri, 04 Dec 2020 11:58:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43576) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klD8b-0007Yx-AA for qemu-devel@nongnu.org; Fri, 04 Dec 2020 10:36:49 -0500 Received: from smtp5-g21.free.fr ([2a01:e0c:1:1599::14]:33322) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klD8Z-0003ne-Hq for qemu-devel@nongnu.org; Fri, 04 Dec 2020 10:36:49 -0500 Received: from wise (unknown [77.159.208.71]) (Authenticated sender: stephane.duverger@free.fr) by smtp5-g21.free.fr (Postfix) with ESMTPSA id C8DC45FFB4; Fri, 4 Dec 2020 16:36:38 +0100 (CET) Date: Fri, 4 Dec 2020 16:36:35 +0100 From: Stephane Duverger To: qemu-devel@nongnu.org Subject: x86 TCG helpers clobbered registers Message-ID: <20201204153446.GA66154@wise> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Received-SPF: none client-ip=2a01:e0c:1:1599::14; envelope-from=stephane.duverger@free.fr; helo=smtp5-g21.free.fr X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, SPOOFED_FREEMAIL=1.497 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 04 Dec 2020 11:55:55 -0500 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hello, While looking at tcg/i386/tcg-target.c.inc:tcg_out_qemu_st(), I discovered that the TCG generates a call to a store helper at the end of the TB which is executed on TLB miss and get back to the remaining translated ops. I tried to mimick this behavior around the fast path (right between tcg_out_tlb_load() and tcg_out_qemu_st_direct()) to filter on memory store accesses. I know there is now TCG plugins for that purpose at TCG IR level, which every tcg-target might benefit. FWIW, my design choice was more led by the fact that I always work on an x86 host and plugins did not exist by the time. Anyway, the point is more related to generating a call to a helper at the TCG IR level (classic scenario), or later during tcg-target code generation (slow path for instance). The TCG when calling a helper knows that some registers will be call clobbered and as such must free them. This is what I observed in tcg_reg_alloc_call(): /* clobber call registers */ for (i = 0; i < TCG_TARGET_NB_REGS; i++) { if (tcg_regset_test_reg(tcg_target_call_clobber_regs, i)) { tcg_reg_free(s, i, allocated_regs); } } But in our case (ie. INDEX_op_qemu_st_i32), the TCG code path comes from: tcg_reg_alloc_op() tcg_out_op() tcg_out_qemu_st() Then tcg_out_tlb_load() will inject a 'jmp' to the slow path, whose generated code does not seem to take care of every call clobbered registers, if we look at tcg_out_qemu_st_slow_path(). First for an i386 (32bits) tcg-target, as expected, the helper arguments are injected into the stack. I noticed that 'esp' is not shifted down before stacking up the args, which might corrupt last stacked words. Second, for both 32/64 bits tcg-targets since all of the 'call clobbered' registers are not preserved, it may happen that depending on the code executed by the helper (and so generated by GCC) these registers will be clobbered (ie. R10 for x86-64). While this never happened for the slow path helper call, I observed that my guest had trouble running when filtering memory in the same fashion the slow path helper would be called. Conversely, if I push/pop all of the call clobbered regs around the call to the helper, everything runs as expected. Is this correct ? Am I missing something ? Thanks a lot in advance for your eagle eye on this :)