Re: [SPDK] Can I disable hugepages if I have vfio and IOMMU enabled?

Re: [SPDK] Can I disable hugepages if I have vfio and IOMMU enabled?

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

On Thu, 2018-08-30 at 16:30 +0300, Itai Fiat Gecht wrote:
> Hello,
> 
> I'm new to SPDK. I have a question about the following manual page:
> http://www.spdk.io/doc/memory.html
> 
> The page says that:
> 
> "SPDK relies on DPDK to allocate pinned memory. On Linux, DPDK does this by
> allocating hugepages (by default, 2MiB). The Linux kernel treats hugepages
> differently than regular 4KiB pages. Specifically, the operating system will
> never change their physical location. This is not by intent, and so things
> could change in future versions, but it is true today and has been for a
> number
> of years (see the later section on the IOMMU for a future-proof solution)."
> 
> The page then explains about IOMMU, and concludes with saying:
> 
> "This is a future-proof, hardware-accelerated solution for performing DMA
> operations into and out of a user space process and forms the long-term
> foundation for SPDK and DPDK's memory management strategy. We highly recommend
> that applications are deployed using vfio and the IOMMU enabled, which is
> fully
> supported today."
> 
> Does this mean that if vfio and IOMMU are enabled I can disable hugepages?

The DPDK memory allocator still only attempts to allocate hugepage memory today
even with vfio and the IOMMU enabled. There is nothing in the hardware or
operating system preventing someone from reworking that memory allocator to
allow for regular 4K pages, but it would be a big undertaking.


> 
> Itai
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] Can I disable hugepages if I have vfio and IOMMU enabled?

[Itai Fiat Gecht]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

On Thu, Aug 30, 2018 at 8:36 PM, Walker, Benjamin
<benjamin.walker(a)intel.com> wrote:
> On Thu, 2018-08-30 at 16:30 +0300, Itai Fiat Gecht wrote:
>> Hello,
>>
>> I'm new to SPDK. I have a question about the following manual page:
>> http://www.spdk.io/doc/memory.html
>>
>> The page says that:
>>
>> "SPDK relies on DPDK to allocate pinned memory. On Linux, DPDK does this by
>> allocating hugepages (by default, 2MiB). The Linux kernel treats hugepages
>> differently than regular 4KiB pages. Specifically, the operating system will
>> never change their physical location. This is not by intent, and so things
>> could change in future versions, but it is true today and has been for a
>> number of years (see the later section on the IOMMU for a future-proof
>> solution)."
>>
>> The page then explains about IOMMU, and concludes with saying:
>>
>> "This is a future-proof, hardware-accelerated solution for performing DMA
>> operations into and out of a user space process and forms the long-term
>> foundation for SPDK and DPDK's memory management strategy. We highly recommend
>> that applications are deployed using vfio and the IOMMU enabled, which is
>> fully supported today."
>>
>> Does this mean that if vfio and IOMMU are enabled I can disable hugepages?
>
> The DPDK memory allocator still only attempts to allocate hugepage memory today
> even with vfio and the IOMMU enabled. There is nothing in the hardware or
> operating system preventing someone from reworking that memory allocator to
> allow for regular 4K pages, but it would be a big undertaking.

Okay. Thank you for your reply!

[SPDK] Can I disable hugepages if I have vfio and IOMMU enabled?

[Itai Fiat Gecht]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

Hello,

I'm new to SPDK. I have a question about the following manual page:
http://www.spdk.io/doc/memory.html

The page says that:

"SPDK relies on DPDK to allocate pinned memory. On Linux, DPDK does this by
allocating hugepages (by default, 2MiB). The Linux kernel treats hugepages
differently than regular 4KiB pages. Specifically, the operating system will
never change their physical location. This is not by intent, and so things
could change in future versions, but it is true today and has been for a number
of years (see the later section on the IOMMU for a future-proof solution)."

The page then explains about IOMMU, and concludes with saying:

"This is a future-proof, hardware-accelerated solution for performing DMA
operations into and out of a user space process and forms the long-term
foundation for SPDK and DPDK's memory management strategy. We highly recommend
that applications are deployed using vfio and the IOMMU enabled, which is fully
supported today."

Does this mean that if vfio and IOMMU are enabled I can disable hugepages?

Itai