* rte_mbuf: documentation, meta-data, and inconsistencies
@ 2014-08-29 0:00 daniel chapiesky
[not found] ` <CAJnwcoPgydbE9BOZCkMejcd4BpLpw7E3gtYnrdCWs0744qJTkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 4+ messages in thread
From: daniel chapiesky @ 2014-08-29 0:00 UTC (permalink / raw)
To: dev-VfR2kkLFssw
rte_muf: Inconsistency in the programmer's guide or the code.....?
---------------------------------------------------------
>From the DPDK 1.7.0 programmer's guide we read:
"For a newly allocated mbuf, the area at which the data begins in the
message buffer
is RTE_PKTMBUF_HEADROOM bytes after the beginning of the buffer, which
is cache
aligned."
In the file ./lib/librte_mbuf/rte_mbuf.c and function rte_pktmbuf_init() we
find:
m->pkt.data = (char*) m->buf_addr + RTE_MIN(RTE_PKTMBUF_HEADROOM,
m->buf_len);
Where RTE_PKTMBUF_HEADROOM is configured to be 128 bytes from the file
./config/common_linuxapp:
CONFIG_RTE_PKTMBUF_HEADROOM=128
Question 1:
Does the above invocation of RTE_MIN() cause the programmer's guide
to be inaccurate?
(Saying "in practice it does not matter..." is not an answer)
Question 2:
Why is "packet metadata" implicitly sharing the same bytes of the mbuf
headroom area,
instead of being explicitly kept from being overwritten by a call to
rte_pktmbuf_prepend(pkt, 100);
presuming my metadata is at least 32 bytes long????
The above command would place the packet data area starting at the last
4 bytes of my metadata....
(Saying "you can change CONFIG_RTE_PKTMBUF_HEADROOM at compile time" is
not an answer because it
effects all mbufs not just the one I with my meta-data...)
Question 3:
Why do we write:
#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +
RTE_PKTMBUF_HEADROOM)
rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
rte_pktmbuf_pool_init, NULL,
rte_pktmbuf_init, NULL,
socketid, flags);
instead of:
#define MBUF_SIZE (2048 + sizeof(struct rte_mbuf))
rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
rte_pktmbuf_pool_init, NULL,
rte_pktmbuf_init, NULL,
socketid, flags);
and have rte_pktmbuf_init() enforce ( + RTE_PKTMBUF_HEADROOM )
automatically?
------------------------------
My reason for asking the above questions stems from my work on a Packet
Framework
Pipeline.
I came to understand that there really isn't an *explicit* declaration
of "allocate this much meta-data space for each packet".
The programmer's guide and mbuf.h describe of headroom and tailroom as a
place to edit
the packet data. An example would be to wrap the packet for tunnelling by
prepending
and appending the data area to be large enought to contain a new header and
checksum.
In fact, while the programmer's guide mentions packet meta-data a few
times, there is no
section which actually describes ****how to make and access**** your very
own packet meta-data.
This addition would be very nice.
The tables in the Packet Framework Pipeline examples all use keys injected
into the meta-data
of the mbuf at RX time to compare a rule against that key (explicitly
stating the "offset
into packet meta-data") and not allowing "offset into packet data".
I actually like this setup because it allows the meta-data key to be
securely analyzed
and copied from the packet data - keeping malformed packets out of the
decision making
process of the tables.
But, in the end, sharing the meta-data area with the packet headroom seems
to be a very
bad idea.
Sincerely,
Daniel Chapiesky
^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <CAJnwcoPgydbE9BOZCkMejcd4BpLpw7E3gtYnrdCWs0744qJTkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: rte_mbuf: documentation, meta-data, and inconsistencies [not found] ` <CAJnwcoPgydbE9BOZCkMejcd4BpLpw7E3gtYnrdCWs0744qJTkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2014-08-29 3:09 ` Matthew Hall 2014-08-29 23:22 ` rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw 1 sibling, 0 replies; 4+ messages in thread From: Matthew Hall @ 2014-08-29 3:09 UTC (permalink / raw) To: daniel chapiesky; +Cc: dev-VfR2kkLFssw On Thu, Aug 28, 2014 at 08:00:59PM -0400, daniel chapiesky wrote: > But, in the end, sharing the meta-data area with the packet headroom seems > to be a very > bad idea. > > Sincerely, > > Daniel Chapiesky You might have picked a good time to inquire about it as some of the Intel guys are making patches to clean up rte_mbuf during the last couple of weeks as we speak. Matthew. ^ permalink raw reply [flat|nested] 4+ messages in thread
* rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time. [not found] ` <CAJnwcoPgydbE9BOZCkMejcd4BpLpw7E3gtYnrdCWs0744qJTkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2014-08-29 3:09 ` Matthew Hall @ 2014-08-29 23:22 ` zimeiw [not found] ` <14f9ed02.118f5.1482414316f.Coremail.zimeiw-9Onoh4P/yGk@public.gmane.org> 1 sibling, 1 reply; 4+ messages in thread From: zimeiw @ 2014-08-29 23:22 UTC (permalink / raw) To: dev-VfR2kkLFssw hi, Running primary process, it is successful. $ sudo ./simple_mp -c 1 -n 1 --socket-mem=64 --proc-type=primary EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Setting up memory... EAL: Ask a virtual area of 0x26000000 bytes EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000) EAL: Ask a virtual area of 0x2200000 bytes EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000) EAL: Ask a virtual area of 0x1a00000 bytes EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000) EAL: Ask a virtual area of 0x2200000 bytes EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffca600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffca200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000) EAL: Requesting 32 pages of size 2MB from socket 0 EAL: TSC frequency is ~2793683 KHz EAL: Master core 0 is ready (tid=f7fe4800) EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:01.0 not managed by UIO driver, skipping EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:06.0 not managed by UIO driver, skipping proc primary APP: Finished Process Init. simple_mp > Running secondary process, it is faied for memzone <RG_MP_log_history> already exists. $ sudo ./simple_mp -c 1 -n 1 --proc-type=secondary --socket-mem=1 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Setting up memory... EAL: Analysing 358 files EAL: Mapped segment 0 of size 0x4000000 EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists RING: Cannot reserve memory EAL: TSC frequency is ~2793681 KHz EAL: Master core 0 is ready (tid=f7fe4800) EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: Cannot find resource for device EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: Cannot find resource for device proc secandary APP: Finished Process Init. simple_mp > ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <14f9ed02.118f5.1482414316f.Coremail.zimeiw-9Onoh4P/yGk@public.gmane.org>]
* Re: rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time. [not found] ` <14f9ed02.118f5.1482414316f.Coremail.zimeiw-9Onoh4P/yGk@public.gmane.org> @ 2014-08-30 14:22 ` Zhang, Jerry 0 siblings, 0 replies; 4+ messages in thread From: Zhang, Jerry @ 2014-08-30 14:22 UTC (permalink / raw) To: zimeiw, dev-VfR2kkLFssw@public.gmane.org Hi, 'memzone <RG_MP_log_history> already exists' is not an error for the secondary DPDK process. DPDK EAL initialization creates a mempool for log history. This mempool creation includes creating a ring based on the memzone named RG_MP_log_history. For the secondary DPDK process, if the mempool for log history creation fails because the memzone RG_MP_log_history exits, it looks up the exited mempool and reuse it. Please refer to the implementation of rte_eal_common_log_init() function. From the logs, it seems your secondary DPDK process has initialized successfully. int rte_eal_common_log_init(FILE *default_log) { STAILQ_INIT(&log_history); /* reserve RTE_LOG_HISTORY*2 elements, so we can dump and * keep logging during this time */ log_history_mp = rte_mempool_create(LOG_HISTORY_MP_NAME, RTE_LOG_HISTORY*2, LOG_ELT_SIZE, 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0); if ((log_history_mp == NULL) && ((log_history_mp = rte_mempool_lookup(LOG_HISTORY_MP_NAME)) == NULL)){ RTE_LOG(ERR, EAL, "%s(): cannot create log_history mempool\n", __func__); return -1; } default_log_stream = default_log; rte_openlog_stream(default_log); return 0; } -----Original Message----- From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of zimeiw Sent: Saturday, August 30, 2014 7:23 AM To: dev-VfR2kkLFssw@public.gmane.org Subject: [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time. hi, Running primary process, it is successful. $ sudo ./simple_mp -c 1 -n 1 --socket-mem=64 --proc-type=primary EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Setting up memory... EAL: Ask a virtual area of 0x26000000 bytes EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000) EAL: Ask a virtual area of 0x2200000 bytes EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000) EAL: Ask a virtual area of 0x1a00000 bytes EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000) EAL: Ask a virtual area of 0x2200000 bytes EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffca600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffca200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000) EAL: Requesting 32 pages of size 2MB from socket 0 EAL: TSC frequency is ~2793683 KHz EAL: Master core 0 is ready (tid=f7fe4800) EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:01.0 not managed by UIO driver, skipping EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:06.0 not managed by UIO driver, skipping proc primary APP: Finished Process Init. simple_mp > Running secondary process, it is faied for memzone <RG_MP_log_history> already exists. $ sudo ./simple_mp -c 1 -n 1 --proc-type=secondary --socket-mem=1 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Setting up memory... EAL: Analysing 358 files EAL: Mapped segment 0 of size 0x4000000 EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists RING: Cannot reserve memory EAL: TSC frequency is ~2793681 KHz EAL: Master core 0 is ready (tid=f7fe4800) EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: Cannot find resource for device EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: Cannot find resource for device proc secandary APP: Finished Process Init. simple_mp > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-08-30 14:22 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-08-29 0:00 rte_mbuf: documentation, meta-data, and inconsistencies daniel chapiesky [not found] ` <CAJnwcoPgydbE9BOZCkMejcd4BpLpw7E3gtYnrdCWs0744qJTkg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2014-08-29 3:09 ` Matthew Hall 2014-08-29 23:22 ` rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw [not found] ` <14f9ed02.118f5.1482414316f.Coremail.zimeiw-9Onoh4P/yGk@public.gmane.org> 2014-08-30 14:22 ` Zhang, Jerry
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).