From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 292CECA9ED3 for ; Tue, 5 Nov 2019 01:17:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E687B2080F for ; Tue, 5 Nov 2019 01:17:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qdz0mbNB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E687B2080F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39952 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iRnTn-0000M9-10 for qemu-devel@archiver.kernel.org; Mon, 04 Nov 2019 20:17:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43788) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iRnSc-0008OZ-68 for qemu-devel@nongnu.org; Mon, 04 Nov 2019 20:16:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iRnSa-0008SI-Ca for qemu-devel@nongnu.org; Mon, 04 Nov 2019 20:16:41 -0500 Received: from mail-oi1-x230.google.com ([2607:f8b0:4864:20::230]:36948) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iRnSa-0008Q1-4g for qemu-devel@nongnu.org; Mon, 04 Nov 2019 20:16:40 -0500 Received: by mail-oi1-x230.google.com with SMTP id y194so16032941oie.4 for ; Mon, 04 Nov 2019 17:16:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5+VoSIUci//6fhE8bGmL4BERSeWlImzuc2U42mIdSMk=; b=qdz0mbNBqFRirIc83RXZywSOLa2PoQvvt6gJyoLOL9sAINknV/1iUyFkO0sPVuR6g8 TBr3aPUt3Rvbe5U+zwZlFyO8USoYZy0WU25SnhI74DAxqes+nMfsx+S5hUqLvl7/IIdw BrtF/dolVEsADcnvV4Sw8++6xHziBSbNK1w//Zd6uOAJRTlLwsUofjyEreVsOV5srQzh 6MYFwdYFf6mcoRbwEyuKvZhpixd72+euTUtTTBVJt4PxTI+GmMrnYWMUfi0IhQiOeNKP hGGGOv+KuYjweKSVid0oM8d59tSq5u5rSK2GcVAPqpUqHeaTONvBE6Ej5R4gjZKeSXIr ESFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5+VoSIUci//6fhE8bGmL4BERSeWlImzuc2U42mIdSMk=; b=eqG/ocDU9ZGrUILL9l0Oy2vGGH0nuDpWw0YYTP281v3ziG613/L5fct9ffGKoChujP 506xxyZHM0xS3DZh3pPQMRZa8KoY9hInCNHKfMU0aj4ZJFXxQBkm6tviB/ZaCOoWVBwb RybOmFoVEZXdxqGvBPAFTTMLH2Oc7aG9G8CsMtOWfmM572tjo9nP2q2meDmcjP4aa+DR 1makZX9sEIxW0SpZYlogVi+X7AVPcYxPBi0ximYuUcw1fiHrZD7pW4jTyKUp8wxHL5Nj wrfJGzMnL3wifDV7ir0hCw/PnYrrtQCSX9+4dVmf3qhO/ja/50SEbUzOIi5f7m76DcYg DuUg== X-Gm-Message-State: APjAAAWvHUny8asZlV5wIzEEKI+i3eh7itTzmAP9b8j2gKjEIKY/aT+h +rdqUgs1vjxkg1Kc1uJFOVn0mGvV5CYWrJHp5Ws= X-Google-Smtp-Source: APXvYqwdBAqqpEU2Ax4iNnrRCeOvZH3R7oW7MYjqQkOfg2erRp61GuOJi/Xl6EUw8TDHO0ZoD0D26R0ejGORnc1+Vdw= X-Received: by 2002:aca:ccd1:: with SMTP id c200mr1669964oig.157.1572916597912; Mon, 04 Nov 2019 17:16:37 -0800 (PST) MIME-Version: 1.0 References: <5DC05485.008EAA.00665@m12-12.163.com> <20191104114857.74fe9222@x1.home> In-Reply-To: <20191104114857.74fe9222@x1.home> From: Li Qiang Date: Tue, 5 Nov 2019 09:16:01 +0800 Message-ID: Subject: Re: Questions about the VFIO BAR region To: Alex Williamson Content-Type: multipart/alternative; boundary="000000000000147e2a05968f2fcd" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::230 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "eric.auger@redhat.com" , Li Qiang , Alex Williamson , "qemu-devel@nongnu.org" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --000000000000147e2a05968f2fcd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Alex Williamson =E4=BA=8E2019=E5=B9=B411=E6=9C= =885=E6=97=A5=E5=91=A8=E4=BA=8C =E4=B8=8A=E5=8D=882:49=E5=86=99=E9=81=93=EF= =BC=9A > On Tue, 5 Nov 2019 00:40:39 +0800 > Li Qiang wrote: > > > Hello Alex, Auger and all, > > > > I have a question about the VFIO virtual device BAR. > > > > In vfio_region_setup, it initialize a =E2=80=98region->mem=E2=80=99 MR = and set its ops > to =E2=80=98vfio_regions_ops=E2=80=99. > > In =E2=80=98vfio_region_mmap=E2=80=99, it maps the physical device=E2= =80=99s MMIO to QEMU=E2=80=99s > virtual address space > > as a raw MR =E2=80=98region->mmaps[i].mem=E2=80=99. > > And also it set the latter MR as a subregion of the first one. > > > > So when the guest accesses the BAR, it will direct go to the physical > device=E2=80=99s BAR. > > My question is here: > > When the qemu will use the =E2=80=98vfio_regions_ops=E2=80=99 to read/w= rite the BAR? > > Also whey in the last of =E2=80=98vfio_region_write/read=E2=80=99 we ne= ed to call > =E2=80=98vbasedev->ops->vfio_eoi(vbasedev);=E2=80=99? > > We support: > > a) sparse mmaps where the entire BAR is not covered by an mmap > Got. > b) quirks, which layer on top of the mmaps to provide virtualized > access > Do you mean like in 'vfio_probe_ati_bar4_quirk', register a high priority subregion of VFIORegion.mem. So when the guest write the BAR, vfio_regions_ops will be used. Here 'quirks' do you mean such things? static void vfio_probe_ati_bar4_quirk(VFIOPCIDevice *vdev, int nr) { VFIOQuirk *quirk; VFIOConfigWindowQuirk *window; ... memory_region_init_io(window->addr_mem, OBJECT(vdev), &vfio_generic_window_address_quirk, window, "vfio-ati-bar4-window-address-quirk", 4); memory_region_add_subregion_overlap(vdev->bars[nr].region.mem, window->address_offset, window->addr_mem, 1); ... } > c) INTx emulation which disables mmaps MRs in order to detect device > access as a generic mechanism for inferring interrupt > acknowledgment. > In the above two cases, in 'vfio_region_write/read' we always access the physical device's BAR. So as far as I can understand, the physical device(sometimes) will trigger interrupts. And the responsible of clear it will be by the 'guest'. So I can't understand why there calls 'vbasedev->ops->vfio_eoi'. Could you please give me an example. Thanks, Li Qiang > > The latter being the reason we call vfio_eoi. Thanks, > > Alex > > --000000000000147e2a05968f2fcd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
Alex Williamson <alex.williamson@redhat.com> =E4=BA=8E20= 19=E5=B9=B411=E6=9C=885=E6=97=A5=E5=91=A8=E4=BA=8C =E4=B8=8A=E5=8D=882:49= =E5=86=99=E9=81=93=EF=BC=9A
On Tue, 5 Nov 2019 00:40:39 +0800
Li Qiang <liq3ea@163= .com> wrote:

> Hello Alex, Auger and all,
>
> I have a question about the VFIO virtual device BAR.
>
> In vfio_region_setup, it initialize a =E2=80=98region->mem=E2=80=99= MR and set its ops to =E2=80=98vfio_regions_ops=E2=80=99.
> In =E2=80=98vfio_region_mmap=E2=80=99, it maps the physical device=E2= =80=99s MMIO to QEMU=E2=80=99s virtual address space
> as a raw MR =E2=80=98region->mmaps[i].mem=E2=80=99.
> And also it set the latter MR as a subregion of the first one.
>
> So when the guest accesses the BAR, it will direct go to the physical = device=E2=80=99s BAR.
> My question is here:
> When the qemu will use the =E2=80=98vfio_regions_ops=E2=80=99 to read/= write the BAR?
> Also whey in the last of =E2=80=98vfio_region_write/read=E2=80=99 we n= eed to call =E2=80=98vbasedev->ops->vfio_eoi(vbasedev);=E2=80=99?

We support:

=C2=A0a) sparse mmaps where the entire BAR is not covered by an mmap

Got.

=C2=A0
=C2=A0b) quirks, which layer on top of the mmaps to provide virtualized
=C2=A0 =C2=A0 access

Do you mean like i= n 'vfio_probe_ati_bar4_quirk', register a high priority subregion o= f VFIORegion.mem.
So when the guest write the BAR, vfio_regions_o= ps will be used. Here 'quirks' do you mean such things?
<= br>
static void vfio_probe_ati_bar4_quirk(VFIOPCIDevice *vdev, in= t nr)
{
=C2=A0 =C2=A0 VFIOQuirk *quirk;
=C2=A0 =C2=A0 VFIOConfigWi= ndowQuirk *window;

=C2=A0 =C2=A0 ...
=C2=A0 =C2=A0 memory_region_= init_io(window->addr_mem, OBJECT(vdev),
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &vfio_ge= neric_window_address_quirk, window,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "vfio-ati-bar4= -window-address-quirk", 4);
=C2=A0 =C2=A0 memory_region_add_subregi= on_overlap(vdev->bars[nr].region.mem,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 window->address_offset,
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 window->addr_mem= , 1);
=C2=A0 =C2=A0...
}

= =C2=A0
=C2=A0c) INTx emulation which disables mmaps MRs in order to detect device<= br> =C2=A0 =C2=A0 access as a generic mechanism for inferring interrupt
=C2=A0 =C2=A0 acknowledgment.

In the ab= ove two cases, in 'vfio_region_write/read' we always access the phy= sical device's BAR.
So as far as I can understand, the physic= al device(sometimes) will trigger interrupts. And the responsible of clear = it=C2=A0
will be by the 'guest'. So I can't understan= d why there calls 'vbasedev->ops->vfio_eoi'. Could you please= give me an
example.


Than= ks,
Li Qiang

=C2=A0

The latter being the reason we call vfio_eoi.=C2=A0 Thanks,

Alex

--000000000000147e2a05968f2fcd--