* [SPDK] Failed to remap 2 MB pages
@ 2017-08-01 13:01 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 7+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2017-08-01 13:01 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 470 bytes --]
Hi All,
I have a system with 256GB. Initially I was giving 16GB for the SPDK to work. Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I am getting the error Failed to remap 2 MB pages. Before the map_all_hugepages there is a call find_physaddrs and that passes. So can I assume that physical memory is available and failure is due to some other reason. Has anyone else observed it and aware of why this is happening.
Thanking You,
Param.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 8:17 Wodkowski, PawelX
0 siblings, 0 replies; 7+ messages in thread
From: Wodkowski, PawelX @ 2017-08-02 8:17 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 977 bytes --]
Can you send full log when failure occur.
Pawel
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
> Kumaraparameshwaran Rathnavel
> Sent: Tuesday, August 1, 2017 3:02 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Failed to remap 2 MB pages
>
> Hi All,
>
> I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
> Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
> am getting the error Failed to remap 2 MB pages. Before the
> map_all_hugepages there is a call find_physaddrs and that passes. So can I
> assume that physical memory is available and failure is due to some other
> reason. Has anyone else observed it and aware of why this is happening.
>
> Thanking You,
> Param.
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 9:53 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 7+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2017-08-02 9:53 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1701 bytes --]
Hi Pawel,
This is the error.
Thanking You,
Param.
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 40 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: Failed to remap 2 MB pages
> PANIC in rte_eal_init():
> Cannot init memory
> On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com> wrote:
>
> Can you send full log when failure occur.
>
> Pawel
>
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
>> Kumaraparameshwaran Rathnavel
>> Sent: Tuesday, August 1, 2017 3:02 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>> Subject: [SPDK] Failed to remap 2 MB pages
>>
>> Hi All,
>>
>> I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
>> Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
>> am getting the error Failed to remap 2 MB pages. Before the
>> map_all_hugepages there is a call find_physaddrs and that passes. So can I
>> assume that physical memory is available and failure is due to some other
>> reason. Has anyone else observed it and aware of why this is happening.
>>
>> Thanking You,
>> Param.
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 4675 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 11:02 Wodkowski, PawelX
0 siblings, 0 replies; 7+ messages in thread
From: Wodkowski, PawelX @ 2017-08-02 11:02 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2821 bytes --]
Yeah, the same message. CONFIG_RTE_MAX_MEMSEG is not big enough for 16GB. Try rebuilding DPDK and SPDK with CONFIG_RTE_MAX_MEMSEG=512 or 1024. (Just grep for CONFIG_RTE_MAX_MEMSEG).
BTW: which version of SPDK are you using? In current master or 17.07 error message from SPDK should be something like this:
Starting DPDK 17.05.0 initialization...
[ DPDK EAL parameters: vhost -c 2 -m 16384 --master-lcore=1 --file-prefix=spdk_pid33057 ]
EAL: Detected 48 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Can only reserve 6375 pages from 9942 requested
Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
Please either increase it or request less amount of memory.
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
Failed to initialize DPDK
Pawel
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 11:53 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Hi Pawel,
This is the error.
Thanking You,
Param.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 40 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Failed to remap 2 MB pages
PANIC in rte_eal_init():
Cannot init memory
On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
Can you send full log when failure occur.
Pawel
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Tuesday, August 1, 2017 3:02 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: [SPDK] Failed to remap 2 MB pages
Hi All,
I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
am getting the error Failed to remap 2 MB pages. Before the
map_all_hugepages there is a call find_physaddrs and that passes. So can I
assume that physical memory is available and failure is due to some other
reason. Has anyone else observed it and aware of why this is happening.
Thanking You,
Param.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 11402 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 12:35 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 7+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2017-08-02 12:35 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3606 bytes --]
Thanks Pawel.
I am using an old version of DPDK and SPDK. I will try increasing the CONFIG_RTR_MAX_MEMSEG.
Should we add it to the config file?
Thanking You,
Param.
> On 02-Aug-2017, at 4:32 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com> wrote:
>
> Yeah, the same message. CONFIG_RTE_MAX_MEMSEG is not big enough for 16GB. Try rebuilding DPDK and SPDK with CONFIG_RTE_MAX_MEMSEG=512 or 1024. (Just grep for CONFIG_RTE_MAX_MEMSEG).
>
> BTW: which version of SPDK are you using? In current master or 17.07 error message from SPDK should be something like this:
>
> Starting DPDK 17.05.0 initialization...
> [ DPDK EAL parameters: vhost -c 2 -m 16384 --master-lcore=1 --file-prefix=spdk_pid33057 ]
> EAL: Detected 48 lcore(s)
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: Can only reserve 6375 pages from 9942 requested
> Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
> Please either increase it or request less amount of memory.
> EAL: FATAL: Cannot init memory
>
> EAL: Cannot init memory
>
> Failed to initialize DPDK
>
> Pawel
>
> <>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Wednesday, August 2, 2017 11:53 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Failed to remap 2 MB pages
>
> Hi Pawel,
>
> This is the error.
>
> Thanking You,
> Param.
>
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 40 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: Failed to remap 2 MB pages
> PANIC in rte_eal_init():
> Cannot init memory
>
>
> On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com <mailto:pawelx.wodkowski(a)intel.com>> wrote:
>
> Can you send full log when failure occur.
>
> Pawel
>
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of
> Kumaraparameshwaran Rathnavel
> Sent: Tuesday, August 1, 2017 3:02 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: [SPDK] Failed to remap 2 MB pages
>
> Hi All,
>
> I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
> Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
> am getting the error Failed to remap 2 MB pages. Before the
> map_all_hugepages there is a call find_physaddrs and that passes. So can I
> assume that physical memory is available and failure is due to some other
> reason. Has anyone else observed it and aware of why this is happening.
>
> Thanking You,
> Param.
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 18564 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 12:44 Wodkowski, PawelX
0 siblings, 0 replies; 7+ messages in thread
From: Wodkowski, PawelX @ 2017-08-02 12:44 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3665 bytes --]
Just grep DPDK files for CONFIG_RTR_MAX_MEMSEG. You will find several places. Change all of them.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 2:35 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Thanks Pawel.
I am using an old version of DPDK and SPDK. I will try increasing the CONFIG_RTR_MAX_MEMSEG.
Should we add it to the config file?
Thanking You,
Param.
On 02-Aug-2017, at 4:32 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
Yeah, the same message. CONFIG_RTE_MAX_MEMSEG is not big enough for 16GB. Try rebuilding DPDK and SPDK with CONFIG_RTE_MAX_MEMSEG=512 or 1024. (Just grep for CONFIG_RTE_MAX_MEMSEG).
BTW: which version of SPDK are you using? In current master or 17.07 error message from SPDK should be something like this:
Starting DPDK 17.05.0 initialization...
[ DPDK EAL parameters: vhost -c 2 -m 16384 --master-lcore=1 --file-prefix=spdk_pid33057 ]
EAL: Detected 48 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Can only reserve 6375 pages from 9942 requested
Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
Please either increase it or request less amount of memory.
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
Failed to initialize DPDK
Pawel
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 11:53 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Hi Pawel,
This is the error.
Thanking You,
Param.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 40 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Failed to remap 2 MB pages
PANIC in rte_eal_init():
Cannot init memory
On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
Can you send full log when failure occur.
Pawel
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Tuesday, August 1, 2017 3:02 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: [SPDK] Failed to remap 2 MB pages
Hi All,
I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
am getting the error Failed to remap 2 MB pages. Before the
map_all_hugepages there is a call find_physaddrs and that passes. So can I
assume that physical memory is available and failure is due to some other
reason. Has anyone else observed it and aware of why this is happening.
Thanking You,
Param.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 15059 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SPDK] Failed to remap 2 MB pages
@ 2017-08-02 13:58 Walker, Benjamin
0 siblings, 0 replies; 7+ messages in thread
From: Walker, Benjamin @ 2017-08-02 13:58 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4553 bytes --]
One other thing to try is to simply reboot the machine, then reserve the 32GB right away. DPDK's memory manager works by gathering the hugepages into physically continuous memory "segments", but it has a limited number of total segments that it will create (256 by default). I don't know why the number of segments is limited - probably because the list of segments is just held in an array. Either way, if you reboot and then immediately reserve the pages (or change your boot options to reserve the pages automatically), you're much more likely to get nice physically contiguous chunks so you don't hit the max segments issue.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Wodkowski, PawelX
Sent: Wednesday, August 2, 2017 5:44 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Just grep DPDK files for CONFIG_RTR_MAX_MEMSEG. You will find several places. Change all of them.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 2:35 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Thanks Pawel.
I am using an old version of DPDK and SPDK. I will try increasing the CONFIG_RTR_MAX_MEMSEG.
Should we add it to the config file?
Thanking You,
Param.
On 02-Aug-2017, at 4:32 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
Yeah, the same message. CONFIG_RTE_MAX_MEMSEG is not big enough for 16GB. Try rebuilding DPDK and SPDK with CONFIG_RTE_MAX_MEMSEG=512 or 1024. (Just grep for CONFIG_RTE_MAX_MEMSEG).
BTW: which version of SPDK are you using? In current master or 17.07 error message from SPDK should be something like this:
Starting DPDK 17.05.0 initialization...
[ DPDK EAL parameters: vhost -c 2 -m 16384 --master-lcore=1 --file-prefix=spdk_pid33057 ]
EAL: Detected 48 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Can only reserve 6375 pages from 9942 requested
Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
Please either increase it or request less amount of memory.
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
Failed to initialize DPDK
Pawel
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 11:53 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Hi Pawel,
This is the error.
Thanking You,
Param.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 40 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Failed to remap 2 MB pages
PANIC in rte_eal_init():
Cannot init memory
On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
Can you send full log when failure occur.
Pawel
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Tuesday, August 1, 2017 3:02 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: [SPDK] Failed to remap 2 MB pages
Hi All,
I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
am getting the error Failed to remap 2 MB pages. Before the
map_all_hugepages there is a call find_physaddrs and that passes. So can I
assume that physical memory is available and failure is due to some other
reason. Has anyone else observed it and aware of why this is happening.
Thanking You,
Param.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 17927 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-08-02 13:58 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-01 13:01 [SPDK] Failed to remap 2 MB pages Kumaraparameshwaran Rathnavel
-- strict thread matches above, loose matches on Subject: below --
2017-08-02 8:17 Wodkowski, PawelX
2017-08-02 9:53 Kumaraparameshwaran Rathnavel
2017-08-02 11:02 Wodkowski, PawelX
2017-08-02 12:35 Kumaraparameshwaran Rathnavel
2017-08-02 12:44 Wodkowski, PawelX
2017-08-02 13:58 Walker, Benjamin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox