From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 634EDC3DA4A for ; Wed, 14 Aug 2024 16:09:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DAE186B0083; Wed, 14 Aug 2024 12:09:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D36F16B0085; Wed, 14 Aug 2024 12:09:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B896F6B0088; Wed, 14 Aug 2024 12:09:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8F00A6B0083 for ; Wed, 14 Aug 2024 12:09:13 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 159288110C for ; Wed, 14 Aug 2024 16:08:59 +0000 (UTC) X-FDA: 82451334798.23.11AB6DE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 9FD8412001F for ; Wed, 14 Aug 2024 16:08:56 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KMEtqdv4; spf=pass (imf29.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723651724; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eCQFSIeDwOTQt5HDmaLH+zjBp75O10MjJ4dyFVJRUrI=; b=lEcKmUFolSxjtjcgNkFBJK8awx+RUyC6owUiBGUPMvlYoet4Nh1pTyLegaYhurW1ZcAWKt EsAeqeopd7jeBKdH2vQrG22mqSJJoQemaJdTJ065gl/XYf7m9VSZw6a0WW2Rvx6/vYG3jR 1Whynyf6Q6omb2Y29wUQ6zo4utwa+NI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KMEtqdv4; spf=pass (imf29.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723651724; a=rsa-sha256; cv=none; b=F8IRYtxisWHimd5HnaupxdIfGPQU6ht/ihX97yWg63WFAc8XbWJGPEE25Chht7MRg3SWKV OFsqf3ERi1LmWTaJMxirN8mMzVNJbVm0FG+UdewCYkdOJ3Ruvfl6pQdcIo6Fz3pAEnPP2s AKdowFme+RLSms7/iNsuaomfCWFiqbs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723651736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eCQFSIeDwOTQt5HDmaLH+zjBp75O10MjJ4dyFVJRUrI=; b=KMEtqdv42QZnx55XCktqQaBm3uhJ68hlySzVfrD8/nUcttbh0bbuHVjEnpDTcjLJw2lqSt RgXFOh0qbhxD/ekeekyDPM6m+oA634lElasJyK4VlwiwHMwO+Y6MerxMt33q9W1zT49Pmb 61nqzsC9sAlPDbAiYQipNAEweExuT68= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-217-PeOyanyFNCWP6oE-_sV7jQ-1; Wed, 14 Aug 2024 12:08:52 -0400 X-MC-Unique: PeOyanyFNCWP6oE-_sV7jQ-1 Received: by mail-io1-f70.google.com with SMTP id ca18e2360f4ac-81fa58fbeceso7895739f.2 for ; Wed, 14 Aug 2024 09:08:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723651732; x=1724256532; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eCQFSIeDwOTQt5HDmaLH+zjBp75O10MjJ4dyFVJRUrI=; b=Vq5yMPf2TdsJbII8YC3zlGRebwg9UYMkNXADRZN0VykM32ngWsRXVhnzX+EbiiBzGN NwwOBedVh1XUkqK/ILIdY9LwmB64aCyvZrXPE+W8ULh/6AiXe6JR11QkVUI4lMTAa9nS 2XPx/wK3yIjm4m2lvkXC76/UWTnu18JJuTI0u2WnHVm9zpMUOz2Yy4s4HziadfzsxA6h kibcd9Z0QT9rR3qR2WMsDf5xlJbevQ+ZTJlbDYjTaugDMg7n3h5wLzW+hlDiZDd1H18G qyMd3VMInStmzTn+KoSIFTEvXLukyR/tlYsbK2YH4cNH9F0SzvObbB2ZHwczM91BAJM2 UJyw== X-Forwarded-Encrypted: i=1; AJvYcCWHoDf/K4Y1Xs0S6RAzhxeZszecY7ue+O/ZE6HJSY0rGR3xw6wqHJ8Tabcu8bm7gcv0OvswMJorbl4xb5VgXYpK6Ws= X-Gm-Message-State: AOJu0Yxl9rXvXLP20Ppa+xk6hJGspoo/0whap09pwAfVhkb2V9qZj4t8 s2YiFItUoZmyjdVmCfZHczdeftdXRNKX/uIOD4hCumNgplqkaPSeUVYlOAQEh40c0zdce+q+Fnz PYIE8ENQyL86uDjvA3gKoX1YyiQi8M/c30Q+BkA6p/Fpf+OEa X-Received: by 2002:a05:6602:6d19:b0:81f:9215:80da with SMTP id ca18e2360f4ac-824dacdaf5amr442806839f.4.1723651732025; Wed, 14 Aug 2024 09:08:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEyNuZ7oXWmfRZ0F1HaRl5CrVwWgXo73deaQcT2xIvtNUe5RT9qbnqyv76Wse5AYqDjEvk6VA== X-Received: by 2002:a05:6602:6d19:b0:81f:9215:80da with SMTP id ca18e2360f4ac-824dacdaf5amr442801939f.4.1723651731455; Wed, 14 Aug 2024 09:08:51 -0700 (PDT) Received: from redhat.com ([38.15.36.11]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4ca769102a4sm3320413173.18.2024.08.14.09.08.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Aug 2024 09:08:50 -0700 (PDT) Date: Wed, 14 Aug 2024 10:08:49 -0600 From: Alex Williamson To: Jason Gunthorpe Cc: Peter Xu , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sean Christopherson , Oscar Salvador , Axel Rasmussen , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Will Deacon , Gavin Shan , Paolo Bonzini , Zi Yan , Andrew Morton , Catalin Marinas , Ingo Molnar , Alistair Popple , Borislav Petkov , David Hildenbrand , Thomas Gleixner , kvm@vger.kernel.org, Dave Hansen , Yan Zhao Subject: Re: [PATCH 19/19] vfio/pci: Implement huge_fault support Message-ID: <20240814100849.601c14da.alex.williamson@redhat.com> In-Reply-To: <20240814132508.GM2032816@nvidia.com> References: <20240809160909.1023470-1-peterx@redhat.com> <20240809160909.1023470-20-peterx@redhat.com> <20240814132508.GM2032816@nvidia.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.43; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: hhjiaftefrd1m8kwk75xh45wd5t3e8co X-Rspamd-Queue-Id: 9FD8412001F X-Rspamd-Server: rspam11 X-HE-Tag: 1723651736-468763 X-HE-Meta: U2FsdGVkX1/ia0teNAyq//d0C2RUdso9dTsUawLz4MSHhDEUfjBkRNIVkJi7xu0afwQnCE6COV6tNy70HZvIX3ICayh/4Gts72cAU/BFyyhm07Lu+UMlzFXpS83pE3ccE7BxufKQt1odDck/iSSWQELNLqgOT/R6Vga7ANXMmucUG6r+LsEnbzYRQf4zHGCiN2vIfYEIAc1JXs/d5avmulGUJQOEQ1zpmQNl8URob4BktCII2ca8zVDMeYMJ1tOi1DEgfdUksQkX8koHBAHLpKlFcpjrV/whl0kH64MHMI/daUHVbjIWavbPItQ9A9083tnpcfEmwBEHWfLe2nPJQkkU/tpcC43re971/2X/quyt0x4MATYEanhbl9OspocjX7AFXHyXi5rmCU9+PvD+mEJPo2pRMS80bSav7ZEBi+Ygev1bdgF3q+hWcxpPZbDnl+yyKmk8DWEZNGulCpU8VTMCaHICyoNAL6Iyw2NWIxH/SMUjLKhJD7UoETHISapseBPA/oj4XkJ5u7UwVk4Jumaylcl5RprRjHMILfHzzJbcae7pyvwiAj52+M7cHM9jpBmSoEMt9mrVpBW0zCCs6rSu0jjwu1fqP0gWnw/4i1FlS676MaE2+eq0XlVKCaSetAKcpK36UoIzXfoMLzVpzEONSeOHTAD3lQ2h/r/0e6g8F+5Iizn3Q9Ud/oys6ufIVGHgglHBFdEUMaI7jSmqGSQHUSmxe5ql35EwRbZ4jv0jtXoO2oOcg+eBuj2K1eO+rOFUrR6LbTFkx390syNKwKs1gNoKlPuJX5xMYahED1nJKdJ6WkPXrLJi7ckabyX8TNmgjVcklVHlFJsWrJksR0hdLfXu6F/g59nk3x5/LIXCgHA2Ma6A+03a0kJoG+7X3zNFtDlbF7oAczO9ynMUmDdg9ymcNh6v5t4yCkek5af8mbCmKq8nzTcTewFrz0DExCDAnx6wM/O+wx1U7Iw avKDDszJ hJsg8BHUinPc3lDUnn7mmdAVie6gLXMIFHIemyYf5R1pxg9eFeTctm03QPF1ghaGy2er/L+FgcfLOdvu4zRvtgnkWmWNQxpuWF8zMlrj2Xc4kAe77VHiYa9s9GowDxON9hxkhYOStNIA4jyfSY3713tGpYS/lPqGyhCh3b799WDprGZSt5wIXz5pPHY6JLFoszBMzhFGJvDqtPIGpjrDGZSrryGjRz8+ajxTcvUFSqbfeLsL0Xrip6rFPfdCfLls4peHDA5LdCOx+hJAY0zH0yPo/WILJtflXfpTgHYR0WgnsibhsfVpUoXr1ig5Ju2CoBitFs+7x5xoXmTSoZpJPtoJud+zXTFjQ+2Cpf9gfP9DttXexT/0bfppdXpgyAfk+GXSj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 14 Aug 2024 10:25:08 -0300 Jason Gunthorpe wrote: > On Fri, Aug 09, 2024 at 12:09:09PM -0400, Peter Xu wrote: > > @@ -1672,30 +1679,49 @@ static vm_fault_t vfio_pci_mmap_fault(struct vm_fault *vmf) > > if (vdev->pm_runtime_engaged || !__vfio_pci_memory_enabled(vdev)) > > goto out_unlock; > > > > - ret = vmf_insert_pfn(vma, vmf->address, pfn + pgoff); > > - if (ret & VM_FAULT_ERROR) > > - goto out_unlock; > > - > > - /* > > - * Pre-fault the remainder of the vma, abort further insertions and > > - * supress error if fault is encountered during pre-fault. > > - */ > > - for (; addr < vma->vm_end; addr += PAGE_SIZE, pfn++) { > > - if (addr == vmf->address) > > - continue; > > - > > - if (vmf_insert_pfn(vma, addr, pfn) & VM_FAULT_ERROR) > > - break; > > + switch (order) { > > + case 0: > > + ret = vmf_insert_pfn(vma, vmf->address, pfn + pgoff); > > + break; > > +#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP > > + case PMD_ORDER: > > + ret = vmf_insert_pfn_pmd(vmf, __pfn_to_pfn_t(pfn + pgoff, > > + PFN_DEV), false); > > + break; > > +#endif > > +#ifdef CONFIG_ARCH_SUPPORTS_PUD_PFNMAP > > + case PUD_ORDER: > > + ret = vmf_insert_pfn_pud(vmf, __pfn_to_pfn_t(pfn + pgoff, > > + PFN_DEV), false); > > + break; > > +#endif > > I feel like this switch should be in some general function? > > vmf_insert_pfn_order(vmf, order, __pfn_to_pfn_t(pfn + pgoff, PFN_DEV), false); > > No reason to expose every driver to this when you've already got a > nice contract to have the driver work on the passed in order. > > What happens if the driver can't get a PFN that matches the requested > order? There was some alignment and size checking chopped from the previous reply that triggered a fallback, but in general PCI BARs are a power of two and naturally aligned, so there should always be an order aligned pfn. Thanks, Alex