* Re: [PATCH v4] nfp: report link speed using hardware info
From: Alejandro Lucero @ 2016-12-19 10:32 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev
In-Reply-To: <724e9785-3499-ac5a-bf7a-8093a649780c@intel.com>
On Mon, Dec 19, 2016 at 10:24 AM, Ferruh Yigit <ferruh.yigit@intel.com>
wrote:
> On 12/16/2016 5:10 PM, Alejandro Lucero wrote:
> > Previous reported speed was hardcoded because there was not firmware
> > support for getting this information. This change needs to support old
> > firmware versions, keeping with the hardcoded report, and the new
> > versions, where the firmware makes that information available.
> >
> > v4: Make conditional simple and more ellaborated commit comment.
> > v3: remove unsed macro
> > v2: use RTE_DIM instead of own macro
> >
> > Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
> > ---
> > drivers/net/nfp/nfp_net.c | 27 +++++++++++++++++++++++++--
> > drivers/net/nfp/nfp_net_ctrl.h | 11 +++++++++++
> > 2 files changed, 36 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> <...>
> > @@ -831,8 +842,20 @@ static void nfp_net_read_mac(struct nfp_net_hw *hw)
> > link.link_status = ETH_LINK_UP;
> >
> > link.link_duplex = ETH_LINK_FULL_DUPLEX;
> > - /* Other cards can limit the tx and rx rate per VF */
> > - link.link_speed = ETH_SPEED_NUM_40G;
> > +
> > + nn_link_status = (nn_link_status >> NFP_NET_CFG_STS_LINK_RATE_SHIFT)
> &
> > + NFP_NET_CFG_STS_LINK_RATE_MASK;
> > +
> > + if ((NFD_CFG_MAJOR_VERSION_of(hw->ver) < 4) ||
> > + ((NFD_CFG_MINOR_VERSION_of(hw->ver) == 4) &&
> > + (NFD_CFG_MINOR_VERSION_of(hw->ver) == 0)))
> > + link.link_speed = ETH_SPEED_NUM_40G;
> > + else {
> > + if (nn_link_status >= RTE_DIM(ls_to_ethtool)
>
> This is not compiling fine, missing parenthesis.
>
Sorry about that. It was so simple the change I did not do any test.
>
> > + link.link_speed = ETH_SPEED_NUM_NONE;
> > + else
> > + link.link_speed = ls_to_ethtool[nn_link_status];
> > + }
> >
> > if (old.link_status != link.link_status) {
> > nfp_net_dev_atomic_write_link_status(dev, &link);
> <...>
>
>
^ permalink raw reply
* Re: No packets received if burst is too small in rte_eth_rx_burst
From: Bruce Richardson @ 2016-12-19 10:25 UTC (permalink / raw)
To: tom.barbette; +Cc: dev
In-Reply-To: <1289578237.19546607.1481971405418.JavaMail.zimbra@ulg.ac.be>
On Sat, Dec 17, 2016 at 11:43:25AM +0100, tom.barbette@ulg.ac.be wrote:
> Hi,
>
> Your comments made me saw the line "PMD: i40e_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0)."
>
> The problem was probably that I was under this limit... Is there a way to get that limit through a function or something?
>
> With 16.04 I received sometimes 5 or 7 packets with a burst_size of 4 which respects this limit. I see that "[dpdk-dev] net/i40e: fix out-of-bounds writes during vector Rx" fixed that, as the limit was in fact 32 no matter the message.
>
> At the end, what should be the minimal rx burst size? How to find it at runtime for any NIC? I imagine that vector rx will create a problem if I give a burst size of 1 even with a recent DPDK version, right?
>
Sadly, there doesn't appear to be any way to discover this, and the i40e
driver requires at least a burst size of 4 even with the latest DPDK.
>From i40e_rxtx_vec_sse.c:
243 /* nb_pkts has to be floor-aligned to RTE_I40E_DESCS_PER_LOOP */
244 nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, RTE_I40E_DESCS_PER_LOOP);
245
I think in this case the gap is not so much having a discovery mechanism
to determine min burst size, but rather a driver gap so as to allow some
form of slower-path fallback when we get below min-size bursts for the
vector driver.
/Bruce
> Thanks,
> Tom
>
> Tom Barbette
> PhD Student @ Université de Liège
>
> Office 1/13
> Bâtiment B37
> Quartier Polytech
> Allée de la découverte, 12
> 4000 Liège
>
> 04/366 91 75
> 0479/60 94 63
>
>
> ----- Mail original -----
> De: "Bruce Richardson" <bruce.richardson@intel.com>
> À: "tom barbette" <tom.barbette@ulg.ac.be>
> Cc: dev@dpdk.org
> Envoyé: Mercredi 14 Décembre 2016 17:52:21
> Objet: Re: [dpdk-dev] No packets received if burst is too small in rte_eth_rx_burst
>
> On Wed, Dec 14, 2016 at 04:13:53PM +0100, tom.barbette@ulg.ac.be wrote:
> > Hi list,
> >
> > Between 2.2.0 and 16.04 (up to at least 16.07.2 if not current), with the XL710 controller I do not get any packet when calling rte_eth_rx_burst if nb_pkts is too small. I would say smaller than 32. The input rate is not big, if that helps. But It should definitely get at least one packet per second.
> >
> > Any ideas? Is that a bug or expected behaviour? Could be caused by other ABI changes?
> >
> Does this issue still occur even if you disable the vector driver in
> your build-time configuration?
>
> /Bruce
^ permalink raw reply
* Re: [PATCH v4] nfp: report link speed using hardware info
From: Ferruh Yigit @ 2016-12-19 10:24 UTC (permalink / raw)
To: Alejandro Lucero, dev
In-Reply-To: <1481908242-845-1-git-send-email-alejandro.lucero@netronome.com>
On 12/16/2016 5:10 PM, Alejandro Lucero wrote:
> Previous reported speed was hardcoded because there was not firmware
> support for getting this information. This change needs to support old
> firmware versions, keeping with the hardcoded report, and the new
> versions, where the firmware makes that information available.
>
> v4: Make conditional simple and more ellaborated commit comment.
> v3: remove unsed macro
> v2: use RTE_DIM instead of own macro
>
> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
> ---
> drivers/net/nfp/nfp_net.c | 27 +++++++++++++++++++++++++--
> drivers/net/nfp/nfp_net_ctrl.h | 11 +++++++++++
> 2 files changed, 36 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
<...>
> @@ -831,8 +842,20 @@ static void nfp_net_read_mac(struct nfp_net_hw *hw)
> link.link_status = ETH_LINK_UP;
>
> link.link_duplex = ETH_LINK_FULL_DUPLEX;
> - /* Other cards can limit the tx and rx rate per VF */
> - link.link_speed = ETH_SPEED_NUM_40G;
> +
> + nn_link_status = (nn_link_status >> NFP_NET_CFG_STS_LINK_RATE_SHIFT) &
> + NFP_NET_CFG_STS_LINK_RATE_MASK;
> +
> + if ((NFD_CFG_MAJOR_VERSION_of(hw->ver) < 4) ||
> + ((NFD_CFG_MINOR_VERSION_of(hw->ver) == 4) &&
> + (NFD_CFG_MINOR_VERSION_of(hw->ver) == 0)))
> + link.link_speed = ETH_SPEED_NUM_40G;
> + else {
> + if (nn_link_status >= RTE_DIM(ls_to_ethtool)
This is not compiling fine, missing parenthesis.
> + link.link_speed = ETH_SPEED_NUM_NONE;
> + else
> + link.link_speed = ls_to_ethtool[nn_link_status];
> + }
>
> if (old.link_status != link.link_status) {
> nfp_net_dev_atomic_write_link_status(dev, &link);
<...>
^ permalink raw reply
* Re: [PATCH v2 06/25] app/testpmd: implement basic support for rte_flow
From: Adrien Mazarguil @ 2016-12-19 10:19 UTC (permalink / raw)
To: Xing, Beilei; +Cc: dev@dpdk.org, Pei, Yulong
In-Reply-To: <94479800C636CB44BD422CB454846E013157433C@SHSMSX101.ccr.corp.intel.com>
Hi Beilei,
On Mon, Dec 19, 2016 at 08:37:20AM +0000, Xing, Beilei wrote:
> Hi Adrien,
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Adrien Mazarguil
> > Sent: Saturday, December 17, 2016 12:25 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v2 06/25] app/testpmd: implement basic
> > support for rte_flow
> >
> > Add basic management functions for the generic flow API (validate, create,
> > destroy, flush, query and list). Flow rule objects and properties are arranged
> > in lists associated with each port.
> >
> > Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > +/** Create flow rule. */
> > +int
> > +port_flow_create(portid_t port_id,
> > + const struct rte_flow_attr *attr,
> > + const struct rte_flow_item *pattern,
> > + const struct rte_flow_action *actions) {
> > + struct rte_flow *flow;
> > + struct rte_port *port;
> > + struct port_flow *pf;
> > + uint32_t id;
> > + struct rte_flow_error error;
> > +
>
> I think there should be memset for error here, e.g. memset(&error, 0, sizeof(struct rte_flow_error));
> Since both cause and message may be NULL regardless of the error type, if there's no error.cause and error.message returned from PMD, Segmentation fault will happen in port_flow_complain.
> PS: This issue doesn't happen if add "export EXTRA_CFLAGS=' -g O0'" when compiling.
Actually, PMDs must fill the error structure only in case of error if the
application provides one, it's not optional. I didn't initialize this
structure for this reason.
I suggest we initialize it with a known poisoning value for debugging
purposes though, to make it fail every time. Does it sound reasonable?
> > + flow = rte_flow_create(port_id, attr, pattern, actions, &error);
> > + if (!flow)
> > + return port_flow_complain(&error);
> > + port = &ports[port_id];
> > + if (port->flow_list) {
> > + if (port->flow_list->id == UINT32_MAX) {
> > + printf("Highest rule ID is already assigned, delete"
> > + " it first");
> > + rte_flow_destroy(port_id, flow, NULL);
> > + return -ENOMEM;
> > + }
> > + id = port->flow_list->id + 1;
> > + } else
> > + id = 0;
> > + pf = port_flow_new(attr, pattern, actions);
> > + if (!pf) {
> > + int err = rte_errno;
> > +
> > + printf("Cannot allocate flow: %s\n", rte_strerror(err));
> > + rte_flow_destroy(port_id, flow, NULL);
> > + return -err;
> > + }
> > + pf->next = port->flow_list;
> > + pf->id = id;
> > + pf->flow = flow;
> > + port->flow_list = pf;
> > + printf("Flow rule #%u created\n", pf->id);
> > + return 0;
> > +}
> > +
--
Adrien Mazarguil
6WIND
^ permalink raw reply
* [PATCH v4] vmxnet3: fix Rx deadlock
From: Stefan Puiu @ 2016-12-19 9:40 UTC (permalink / raw)
To: dev; +Cc: yongwang, mac_leehk, Stefan Puiu
In-Reply-To: <1481902617-16050-1-git-send-email-stefan.puiu@gmail.com>
Our use case is that we have an app that needs to keep mbufs around
for a while. We've seen cases when calling vmxnet3_post_rx_bufs() from
vmxet3_recv_pkts(), it might not succeed to add any mbufs to any RX
descriptors (where it returns -err). Since there are no mbufs that the
virtual hardware can use, no packets will be received after this; the
driver won't refill the mbuf after this so it gets stuck in this
state. I call this a deadlock for lack of a better term - the virtual
HW waits for free mbufs, while the app waits for the hardware to
notify it for data (by flipping the generation bit on the used Rx
descriptors). Note that after this, the app can't recover.
This fix is a rework of this patch by Marco Lee:
http://dpdk.org/dev/patchwork/patch/6575/. I had to forward port
it, address review comments and also reverted the allocation
failure handling to the first version of the patch
(http://dpdk.org/ml/archives/dev/2015-July/022079.html), since
that's the only approach that seems to work, and seems to be what
other drivers are doing (I checked ixgbe and em). Reusing the mbuf
that's getting passed to the application doesn't seem to make
sense, and it was causing weird issues in our app. Also, reusing
rxm without checking if it's NULL could cause the code to crash.
Signed-off-by: Stefan Puiu <stefan.puiu@gmail.com>
---
v4:
* no change, just added sign-off
v3:
* rework description after review, explain how HW signals receipt of
packets
v2:
* address review comments, reworded description a bit
drivers/net/vmxnet3/vmxnet3_rxtx.c | 39 ++++++++++++++++++++++++++++++++++++--
1 file changed, 37 insertions(+), 2 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index b109168..93db10f 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -518,6 +518,32 @@
return nb_tx;
}
+static inline void
+vmxnet3_renew_desc(vmxnet3_rx_queue_t *rxq, uint8_t ring_id,
+ struct rte_mbuf *mbuf)
+{
+ uint32_t val = 0;
+ struct vmxnet3_cmd_ring *ring = &rxq->cmd_ring[ring_id];
+ struct Vmxnet3_RxDesc *rxd =
+ (struct Vmxnet3_RxDesc *)(ring->base + ring->next2fill);
+ vmxnet3_buf_info_t *buf_info = &ring->buf_info[ring->next2fill];
+
+ if (ring_id == 0)
+ val = VMXNET3_RXD_BTYPE_HEAD;
+ else
+ val = VMXNET3_RXD_BTYPE_BODY;
+
+ buf_info->m = mbuf;
+ buf_info->len = (uint16_t)(mbuf->buf_len - RTE_PKTMBUF_HEADROOM);
+ buf_info->bufPA = rte_mbuf_data_dma_addr_default(mbuf);
+
+ rxd->addr = buf_info->bufPA;
+ rxd->btype = val;
+ rxd->len = buf_info->len;
+ rxd->gen = ring->gen;
+
+ vmxnet3_cmd_ring_adv_next2fill(ring);
+}
/*
* Allocates mbufs and clusters. Post rx descriptors with buffer details
* so that device can receive packets in those buffers.
@@ -657,9 +683,18 @@
}
while (rcd->gen == rxq->comp_ring.gen) {
+ struct rte_mbuf *newm;
+
if (nb_rx >= nb_pkts)
break;
+ newm = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(newm == NULL)) {
+ PMD_RX_LOG(ERR, "Error allocating mbuf");
+ rxq->stats.rx_buf_alloc_failure++;
+ break;
+ }
+
idx = rcd->rxdIdx;
ring_idx = (uint8_t)((rcd->rqID == rxq->qid1) ? 0 : 1);
rxd = (Vmxnet3_RxDesc *)rxq->cmd_ring[ring_idx].base + idx;
@@ -759,8 +794,8 @@
VMXNET3_INC_RING_IDX_ONLY(rxq->cmd_ring[ring_idx].next2comp,
rxq->cmd_ring[ring_idx].size);
- /* It's time to allocate some new buf and renew descriptors */
- vmxnet3_post_rx_bufs(rxq, ring_idx);
+ /* It's time to renew descriptors */
+ vmxnet3_renew_desc(rxq, ring_idx, newm);
if (unlikely(rxq->shared->ctrl.updateRxProd)) {
VMXNET3_WRITE_BAR0_REG(hw, rxprod_reg[ring_idx] + (rxq->queue_id * VMXNET3_REG_ALIGN),
rxq->cmd_ring[ring_idx].next2fill);
--
1.9.1
^ permalink raw reply related
* Re: [PATCH v5 00/29] Support VFD and DPDK PF + kernel VF on i40e
From: Chen, Jing D @ 2016-12-19 9:01 UTC (permalink / raw)
To: Vincent JARDIN, Yigit, Ferruh, dev@dpdk.org; +Cc: Wu, Jingjing, Zhang, Helin
In-Reply-To: <5846f66b-9a83-faa6-3de1-c7ae12236201@6wind.com>
Hi, Vincent,
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vincent JARDIN
> Sent: Saturday, December 17, 2016 4:36 AM
> To: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org
> Cc: Wu, Jingjing <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v5 00/29] Support VFD and DPDK PF + kernel VF on
> i40e
>
> Le 16/12/2016 à 20:02, Ferruh Yigit a écrit :
> > As we need to support the scenario kernel PF + DPDK VF,
> > DPDK follows the interface between kernel PF + kernel VF.
>
> Please, it has to be proven that DPDK provides the same interface that
> the kernel. It seems insane to duplicate kernel's PF into the DPDK
> without a strong guarantee that it is a strict same behaviour. For instance,
> - what will happen when the DPDK's PF will have a new feature which
> is not into the kernel?
> - what will happen when the Kernel's PF will have a feature with
> different capabilities that the kernel?
>
> Please, make it clear before. Currently, for me it is a nack of this serie.
>
Let me try to explain.
When we DPDK developed i40e drivers several years ago, we found the API and
data structures defined in shared code for VF and PF device communication can
satisfy DPDK's requirements, so we just followed that API. At that time, we'll try
to satisfy 3 scenarios.
1. Linux PF + Linux VF : it's not our scope.
2. Linux PF + DPDK VF: there is no problems observed since we use same API.
3. DPDK PF + DPDK VF: that's fully under control.
The only scenario was not considered is DPDK PF + Linux VF.
Since then, both Linux and DPDK keep developing code. Then, we found it's
necessary to extend VF capability (Like promiscuous mode). It will be hard to
ask Linux PF to support that service considering upstream effort in Linux world.
So, supporting it in scenario of DPDK PF + DPDK VF became best candidate. But
how can VF identify who is serving it then decide if extended request can be dispatched?
So, DPDK PF will send a special version number for this purpose.
As time passed by, we realized there some use case for DPDK PF + Linux VF and it's not
supported yet. Then, we found our implementation in DPDK PF is not complete since we
only had considered how to serve DPDK VF at that time. So, we need some improvement
on the PF host driver. Besides that, 'send a special version' to VF doesn't work now since
Linux VF can't interpret the version info. So, we behave the same as Linux PF driver with
sacrifice of extended function in DPDK VF.
Let me summary the chang here.
1. We shared the same interface with Linux driver from beginning.
2. This change will support both Linux VF and DPDK VF.
3. We'll find a way for DPDK VF identifying who is serving it so it can use extended function
in future release.
^ permalink raw reply
* Re: [PATCH v2 06/25] app/testpmd: implement basic support for rte_flow
From: Xing, Beilei @ 2016-12-19 8:37 UTC (permalink / raw)
To: Adrien Mazarguil, dev@dpdk.org; +Cc: Pei, Yulong
In-Reply-To: <3318a43c9e105caaf4d5b13d6e4fcce774fb522f.1481903839.git.adrien.mazarguil@6wind.com>
Hi Adrien,
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Adrien Mazarguil
> Sent: Saturday, December 17, 2016 12:25 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v2 06/25] app/testpmd: implement basic
> support for rte_flow
>
> Add basic management functions for the generic flow API (validate, create,
> destroy, flush, query and list). Flow rule objects and properties are arranged
> in lists associated with each port.
>
> Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> +/** Create flow rule. */
> +int
> +port_flow_create(portid_t port_id,
> + const struct rte_flow_attr *attr,
> + const struct rte_flow_item *pattern,
> + const struct rte_flow_action *actions) {
> + struct rte_flow *flow;
> + struct rte_port *port;
> + struct port_flow *pf;
> + uint32_t id;
> + struct rte_flow_error error;
> +
I think there should be memset for error here, e.g. memset(&error, 0, sizeof(struct rte_flow_error));
Since both cause and message may be NULL regardless of the error type, if there's no error.cause and error.message returned from PMD, Segmentation fault will happen in port_flow_complain.
PS: This issue doesn't happen if add "export EXTRA_CFLAGS=' -g O0'" when compiling.
Thanks
Beilei
> + flow = rte_flow_create(port_id, attr, pattern, actions, &error);
> + if (!flow)
> + return port_flow_complain(&error);
> + port = &ports[port_id];
> + if (port->flow_list) {
> + if (port->flow_list->id == UINT32_MAX) {
> + printf("Highest rule ID is already assigned, delete"
> + " it first");
> + rte_flow_destroy(port_id, flow, NULL);
> + return -ENOMEM;
> + }
> + id = port->flow_list->id + 1;
> + } else
> + id = 0;
> + pf = port_flow_new(attr, pattern, actions);
> + if (!pf) {
> + int err = rte_errno;
> +
> + printf("Cannot allocate flow: %s\n", rte_strerror(err));
> + rte_flow_destroy(port_id, flow, NULL);
> + return -err;
> + }
> + pf->next = port->flow_list;
> + pf->id = id;
> + pf->flow = flow;
> + port->flow_list = pf;
> + printf("Flow rule #%u created\n", pf->id);
> + return 0;
> +}
> +
^ permalink raw reply
* Re: Virtio driver lockup in KVM
From: Maxime Coquelin @ 2016-12-19 8:29 UTC (permalink / raw)
To: Sreenaath Vasudevan, dev
In-Reply-To: <CANNY8mk5XM1tRyPyEpo4ogypaQKCArqCLjDDK6pK2-DpCDnMdA@mail.gmail.com>
Hi Sreenaath,
On 12/19/2016 09:21 AM, Sreenaath Vasudevan wrote:
> Hi
> I am running a DPDK application with virtio driver and ran in to driver rx
> lockup issue.
Are you running vhost-user DPDK backend on host or Kernel backend?
> The application seems to be running fine for couple of hours before it
> stops receiving packets on the interface. Transmission of packets seems to
> work fine on the same interface while reception of packets stop.
> Has anyone run in to anything similar?
Not that I am aware, I would suggest you recompile DPDK with debug,
attach to the DPDK process when it happens and check what PMd threads
are doing.
>
> Following is the environment
> Application runnign dpdk-2.2
> VM running on KVM hypervisor using VIRTIO driver
>
Cheers,
Maxime
^ permalink raw reply
* Virtio driver lockup in KVM
From: Sreenaath Vasudevan @ 2016-12-19 8:21 UTC (permalink / raw)
To: dev
Hi
I am running a DPDK application with virtio driver and ran in to driver rx
lockup issue.
The application seems to be running fine for couple of hours before it
stops receiving packets on the interface. Transmission of packets seems to
work fine on the same interface while reception of packets stop.
Has anyone run in to anything similar?
Following is the environment
Application runnign dpdk-2.2
VM running on KVM hypervisor using VIRTIO driver
--
regards
sreenaath
^ permalink raw reply
* Re: [PATCH 1/4] eal/common: introduce rte_memset on IA platform
From: Yuanhan Liu @ 2016-12-19 6:27 UTC (permalink / raw)
To: Yang, Zhiyong
Cc: Richardson, Bruce, Ananyev, Konstantin, Thomas Monjalon,
dev@dpdk.org, De Lara Guarch, Pablo, Wang, Zhihong
In-Reply-To: <E182254E98A5DA4EB1E657AC7CB9BD2A3EB59F21@BGSMSX101.gar.corp.intel.com>
On Fri, Dec 16, 2016 at 10:19:43AM +0000, Yang, Zhiyong wrote:
> > > I run the same virtio/vhost loopback tests without NIC.
> > > I can see the throughput drop when running choosing functions at run
> > > time compared to original code as following on the same platform(my
> > machine is haswell)
> > > Packet size perf drop
> > > 64 -4%
> > > 256 -5.4%
> > > 1024 -5%
> > > 1500 -2.5%
> > > Another thing, I run the memcpy_perf_autotest, when N= <128, the
> > > rte_memcpy perf gains almost disappears When choosing functions at run
> > > time. For N=other numbers, the perf gains will become narrow.
> > >
> > How narrow. How significant is the improvement that we gain from having to
> > maintain our own copy of memcpy. If the libc version is nearly as good we
> > should just use that.
> >
> > /Bruce
>
> Zhihong sent a patch about rte_memcpy, From the patch,
> we can see the optimization job for memcpy will bring obvious perf improvements
> than glibc for DPDK.
Just a clarification: it's better than the __original DPDK__ rte_memcpy
but not the glibc one. That makes me think have any one tested the memcpy
with big packets? Does the one from DPDK outweigh the one from glibc,
even for big packets?
--yliu
> http://www.dpdk.org/dev/patchwork/patch/17753/
> git log as following:
> This patch is tested on Ivy Bridge, Haswell and Skylake, it provides
> up to 20% gain for Virtio Vhost PVP traffic, with packet size ranging
> from 64 to 1500 bytes.
>
> thanks
> Zhiyong
^ permalink raw reply
* [PATCH 2/2] net/ixgbe: calculate correct number of received packets for ARM NEON-version vPMD
From: Jianbo Liu @ 2016-12-19 6:09 UTC (permalink / raw)
To: dev, helin.zhang, konstantin.ananyev, jerin.jacob; +Cc: Jianbo Liu
In-Reply-To: <1482127758-4904-1-git-send-email-jianbo.liu@linaro.org>
vPMD will check 4 descriptors in one time, but the statuses are not consistent
because the memory allocated for RX descriptors is cacheable huagepage.
This patch is to calculate the number of received packets by scanning DD bit
sequentially, and stops when meeting the first packet with DD bit unset.
Signed-off-by: Jianbo Liu <jianbo.liu@linaro.org>
---
drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
index f96cc85..0b1338d 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c
@@ -196,7 +196,6 @@ _recv_raw_pkts_vec(struct ixgbe_rx_queue *rxq, struct rte_mbuf **rx_pkts,
struct ixgbe_rx_entry *sw_ring;
uint16_t nb_pkts_recd;
int pos;
- uint64_t var;
uint8x16_t shuf_msk = {
0xFF, 0xFF,
0xFF, 0xFF, /* skip 32 bits pkt_type */
@@ -255,6 +254,7 @@ _recv_raw_pkts_vec(struct ixgbe_rx_queue *rxq, struct rte_mbuf **rx_pkts,
uint64x2_t mbp1, mbp2;
uint8x16_t staterr;
uint16x8_t tmp;
+ uint32_t var = 0;
uint32_t stat;
/* B.1 load 1 mbuf point */
@@ -349,11 +349,19 @@ _recv_raw_pkts_vec(struct ixgbe_rx_queue *rxq, struct rte_mbuf **rx_pkts,
vst1q_u8((uint8_t *)&rx_pkts[pos]->rx_descriptor_fields1,
pkt_mb1);
+ stat &= IXGBE_VPMD_DESC_DD_MASK;
+
/* C.4 calc avaialbe number of desc */
- var = __builtin_popcount(stat & IXGBE_VPMD_DESC_DD_MASK);
- nb_pkts_recd += var;
- if (likely(var != RTE_IXGBE_DESCS_PER_LOOP))
+ if (likely(var != IXGBE_VPMD_DESC_DD_MASK)) {
+ while (stat & 0x01) {
+ ++var;
+ stat = stat >> 8;
+ }
+ nb_pkts_recd += var;
break;
+ } else {
+ nb_pkts_recd += RTE_IXGBE_DESCS_PER_LOOP;
+ }
}
/* Update our internal tail pointer */
--
2.4.11
^ permalink raw reply related
* [PATCH 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function
From: Jianbo Liu @ 2016-12-19 6:09 UTC (permalink / raw)
To: dev, helin.zhang, konstantin.ananyev, jerin.jacob; +Cc: Jianbo Liu
To get better performance, Rx bulk alloc recv function will scan 8 descriptors
in one time, but the statuses are not consistent on ARM platform because
the memory allocated for Rx descriptors is cacheable hugepages.
This patch is to calculate the number of received packets by scanning DD bit
sequentially, and stops when meeting the first packet with DD bit unset.
Signed-off-by: Jianbo Liu <jianbo.liu@linaro.org>
---
drivers/net/ixgbe/ixgbe_rxtx.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index b2d9f45..2866bdb 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -1402,17 +1402,21 @@ ixgbe_rx_scan_hw_ring(struct ixgbe_rx_queue *rxq)
for (i = 0; i < RTE_PMD_IXGBE_RX_MAX_BURST;
i += LOOK_AHEAD, rxdp += LOOK_AHEAD, rxep += LOOK_AHEAD) {
/* Read desc statuses backwards to avoid race condition */
- for (j = LOOK_AHEAD-1; j >= 0; --j)
+ for (j = LOOK_AHEAD - 1; j >= 0; --j) {
s[j] = rte_le_to_cpu_32(rxdp[j].wb.upper.status_error);
-
- for (j = LOOK_AHEAD - 1; j >= 0; --j)
pkt_info[j] = rte_le_to_cpu_32(rxdp[j].wb.lower.
lo_dword.data);
+ }
+
+ rte_smp_rmb();
/* Compute how many status bits were set */
nb_dd = 0;
for (j = 0; j < LOOK_AHEAD; ++j)
- nb_dd += s[j] & IXGBE_RXDADV_STAT_DD;
+ if (s[j] & IXGBE_RXDADV_STAT_DD)
+ ++nb_dd;
+ else
+ break;
nb_rx += nb_dd;
--
2.4.11
^ permalink raw reply related
* Re: [PATCH 09/32] lib/ether: add rte_device in rte_eth_dev
From: Hemant Agrawal @ 2016-12-19 5:30 UTC (permalink / raw)
To: Ferruh Yigit, dev@dpdk.org
Cc: thomas.monjalon@6wind.com, Richardson, Bruce,
shreyansh.jain@nxp.com
In-Reply-To: <4ced5dc6-c2d7-a4d9-b7a1-29476efd9791@intel.com>
On 12/15/2016 8:11 PM, Ferruh Yigit wrote:
> On 12/7/2016 6:41 AM, Hemant Agrawal wrote:
>> On 12/7/2016 1:18 AM, Ferruh Yigit wrote:
>>> On 12/4/2016 6:17 PM, Hemant Agrawal wrote:
>>>> Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>>>> ---
>>>> lib/librte_ether/rte_ethdev.h | 1 +
>>>> 1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
>>>> index 3c45a1f..6f5673f 100644
>>>> --- a/lib/librte_ether/rte_ethdev.h
>>>> +++ b/lib/librte_ether/rte_ethdev.h
>>>> @@ -1626,6 +1626,7 @@ struct rte_eth_dev {
>>>> eth_rx_burst_t rx_pkt_burst; /**< Pointer to PMD receive function. */
>>>> eth_tx_burst_t tx_pkt_burst; /**< Pointer to PMD transmit function. */
>>>> struct rte_eth_dev_data *data; /**< Pointer to device data */
>>>> + struct rte_device *device;
>>>
>>> I believe this change should not be part of a PMD patchset. This change
>>> is more generic than the PMD.
>>>
>>> Won't Shreyansh's patch already do this?
>>
>> I agree that this patch is not a fit for this PMD patchset, Shreyansh's
>> patch is not yet doing it. He will be taking care of it next.
>>
>> So till Shreyansh provide the support, we need it.
>
> If you need it, what do you think sending this as a separate patch? And
> when accepted, your driver can use it?
>
I will prefer to keep this patch as the first patch in my patchset. If
Shreyansh's patch come on time, we can easily remove it.
>>
>>>
>>>> const struct eth_driver *driver;/**< Driver for this device */
>>>> const struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */
>>>> struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
>>>>
>>>
>>>
>>
>>
>
>
^ permalink raw reply
* Re: [PATCH 02/32] drivers/common: introducing dpaa2 mc driver
From: Hemant Agrawal @ 2016-12-19 5:27 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, thomas.monjalon, bruce.richardson, shreyansh.jain,
Cristian Sovaiala
In-Reply-To: <20161215060453.GA19354@localhost.localdomain>
On 12/15/2016 11:34 AM, Jerin Jacob wrote:
> On Sun, Dec 04, 2016 at 11:46:57PM +0530, Hemant Agrawal wrote:
>> This patch intoduces the DPAA2 MC(Management complex Driver)
>>
>> This driver is common to be used by various DPAA2 net, crypto
>> and other drivers
>>
>> Signed-off-by: Cristian Sovaiala <cristian.sovaiala@nxp.com>
>> [Hemant:rebase and conversion to library for DPDK]
>> Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>> +#ifndef _FSL_MC_SYS_H
>> +#define _FSL_MC_SYS_H
>> +
>> +#ifdef __linux_driver__
>> +
>> +#include <linux/errno.h>
>> +#include <asm/io.h>
>> +#include <linux/slab.h>
>> +
>> +struct fsl_mc_io {
>> + void *regs;
>> +};
>> +
>> +#ifndef ENOTSUP
>> +#define ENOTSUP 95
>> +#endif
>> +
>> +#define ioread64(_p) readq(_p)
>> +#define iowrite64(_v, _p) writeq(_v, _p)
>> +
>> +#else /* __linux_driver__ */
>> +
>> +#include <stdio.h>
>> +#include <libio.h>
>> +#include <stdint.h>
>> +#include <errno.h>
>> +#include <sys/uio.h>
>> +#include <linux/byteorder/little_endian.h>
>> +
>> +#define cpu_to_le64(x) __cpu_to_le64(x)
>> +#ifndef dmb
>> +#define dmb() {__asm__ __volatile__("" : : : "memory"); }
>> +#endif
>
> Better to use DPDK macros here.
>
>> +#define __iormb() dmb()
>> +#define __iowmb() dmb()
>> +#define __arch_getq(a) (*(volatile unsigned long *)(a))
>> +#define __arch_putq(v, a) (*(volatile unsigned long *)(a) = (v))
>> +#define __arch_putq32(v, a) (*(volatile unsigned int *)(a) = (v))
>> +#define readq(c) \
>> + ({ uint64_t __v = __arch_getq(c); __iormb(); __v; })
>> +#define writeq(v, c) \
>> + ({ uint64_t __v = v; __iowmb(); __arch_putq(__v, c); __v; })
>> +#define writeq32(v, c) \
>> + ({ uint32_t __v = v; __iowmb(); __arch_putq32(__v, c); __v; })
>> +#define ioread64(_p) readq(_p)
>> +#define iowrite64(_v, _p) writeq(_v, _p)
>> +#define iowrite32(_v, _p) writeq32(_v, _p)
>
> Hopefully, we can clean all this once rte_read32 and rte_write32 becomes
> mainline
>
> http://dpdk.org/dev/patchwork/patch/17935/
I agree, We will update it as your other patch progresses.
>> +#define __iomem
>> +
>> +struct fsl_mc_io {
>> + void *regs;
>> +};
>> +
>> +#ifndef ENOTSUP
>> +#define ENOTSUP 95
>> +#endif
>> +
>> +/*GPP is supposed to use MC commands with low priority*/
>> +#define CMD_PRI_LOW 0 /*!< Low Priority command indication */
>> +
>> +struct mc_command;
>> +
>> +int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd);
>> +
>> +#endif /* __linux_driver__ */
>> +
>> +#endif /* _FSL_MC_SYS_H */
>> +
>> +/** User space framework uses MC Portal in shared mode. Following change
>> +* introduces lock in MC FLIB
>> +*/
>> +
>> +/**
>> +* The mc_spinlock_t type.
>> +*/
>> +typedef struct {
>> + volatile int locked; /**< lock status 0 = unlocked, 1 = locked */
>> +} mc_spinlock_t;
>> +
>> +/**
>> +* A static spinlock initializer.
>> +*/
>> +static mc_spinlock_t mc_portal_lock = { 0 };
>> +
>> +static inline void mc_pause(void) {}
>> +
>> +static inline void mc_spinlock_lock(mc_spinlock_t *sl)
>> +{
>> + while (__sync_lock_test_and_set(&sl->locked, 1))
>> + while (sl->locked)
>> + mc_pause();
>> +}
>> +
>> +static inline void mc_spinlock_unlock(mc_spinlock_t *sl)
>> +{
>> + __sync_lock_release(&sl->locked);
>> +}
>> +
>
> DPDK spinlock can be used here.
>
Yes!
^ permalink raw reply
* Re: [PATCH v3 0/6] libeventdev API and northbound implementation
From: Shreyansh Jain @ 2016-12-19 5:16 UTC (permalink / raw)
To: Jerin Jacob
Cc: dev, thomas.monjalon, bruce.richardson, hemant.agrawal, gage.eads,
harry.van.haaren
In-Reply-To: <1482070895-32491-1-git-send-email-jerin.jacob@caviumnetworks.com>
My mail reader (thunderbird) is showing this series as a thread of "eal:
postpone vdev initialization" patch series. Is it just me?
If it is a wrong 'in-reply-to', I think it should be corrected or people
might not be able to search for this under right thread.
On Sunday 18 December 2016 07:51 PM, Jerin Jacob wrote:
> As previously discussed in RFC v1 [1], RFC v2 [2], with changes
> described in [3] (also pasted below), here is the first non-draft series
> for this new API.
>
> [1] http://dpdk.org/ml/archives/dev/2016-August/045181.html
> [2] http://dpdk.org/ml/archives/dev/2016-October/048592.html
> [3] http://dpdk.org/ml/archives/dev/2016-October/048196.html
>
> v2..v3:
>
> 1) Changed struct rte_event layout more aligment balanced(Harry, Jerin)
> 2) Changed event_ptr type to void* from uintptr_t(Bruce)
> 3) Changed ev[] as const in rte_event_enqueue_burst to disallow
> drivers from modifying the events passed in(Bruce)
> 4) Removed queue memory allocation from common code as some drivers may not need
> it(Bruce)
> 5) Removed "struct rte_event_queue_link" and replaced with queues and priorities
> in the link and link_get API to avoid one redirection to use the API(Bruce)
>
> v1..v2:
> 1) Remove unnecessary header files from rte_eventdev.h(Thomas)
> 2) Removed PMD driver name(EVENTDEV_NAME_SKELETON_PMD) from rte_eventdev.h(Thomas)
> 3) Removed different #define for different priority schemes. Changed to
> one event device RTE_EVENT_DEV_PRIORITY_* priority (Bruce)
> 4) add const to rte_event_dev_configure(), rte_event_queue_setup(),
> rte_event_port_setup(), rte_event_port_link()(Bruce)
> 5) Fixed missing dev argument in dev->schedule() function(Bruce)
> 6) Changed \see to @see in doxgen comments(Thomas)
> 7) Added additional text in specification to clarify the queue depth(Thomas)
> 8) Changed wait to timeout across the specification(Thomas)
> 9) Added longer explanation for RTE_EVENT_OP_NEW and RTE_EVENT_OP_FORWARD(Thomas)
> 10) Fixed issue with RTE_EVENT_OP_RELEASE doxgen formatting(Thomas)
> 11) Changed to RTE_EVENT_DEV_CFG_FLAG_ from RTE_EVENT_DEV_CFG_(Thomas)
> 12) Changed to EVENT_QUEUE_CFG_FLAG_ from EVENT_QUEUE_CFG_(Thomas)
> 13) s/RTE_EVENT_TYPE_CORE/RTE_EVENT_TYPE_CPU/(Thomas, Gage)
> 14) Removed non burst API and kept only the burst API in the API specification
> (Thomas, Bruce, Harry, Jerin)
> -- Driver interface has non burst API, selection of the non burst API is based
> on num_objects == 1
> 15) sizeeof(struct rte_event) was not 16 in v1. Fixed it in v2
> -- reduced the width of event_type to 4bit to save space for future change
> -- introduced impl_opaque for implementation specific opaque data(Harry),
> Something useful for HW driver too, in the context of removal the need for sepeare
> release API.
> -- squashed other element size and provided enough space to impl_opaque(Jerin)
> -- added RTE_BUILD_BUG_ON(sizeof(struct rte_event) != 16); check
> 16) add union of uint64_t in the second element in struct rte_event to
> make sure the structure has 16byte address all arch(Thomas)
> 17) Fixed invalid check of nb_atomic_order_sequences in implementation(Gage)
> 18) s/EDEV_LOG_ERR/RTE_EDEV_LOG_ERR(Thomas)
> 19) s/rte_eventdev_pmd_/rte_event_pmd_/(Bruce)
> 20) added fine details of distributed vs centralized scheduling information
> in the specification and introduced RTE_EVENT_DEV_CAP_FLAG_DISTRIBUTED_SCHED
> flag(Gage)
> 21)s/RTE_EVENT_QUEUE_CFG_FLAG_SINGLE_CONSUMER/RTE_EVENT_QUEUE_CFG_FLAG_SINGLE_LINK (Jerin)
> to remove the confusion to between another producer and consumer in sw eventdev driver
> 22) Northbound api implementation patch spited to more logical patches(Thomas)
>
> Changes since RFC v2:
>
> - Updated the documentation to define the need for this library[Jerin]
> - Added RTE_EVENT_QUEUE_CFG_*_ONLY configuration parameters in
> struct rte_event_queue_conf to enable optimized sw implementation [Bruce]
> - Introduced RTE_EVENT_OP* ops [Bruce]
> - Added nb_event_queue_flows,nb_event_port_dequeue_depth, nb_event_port_enqueue_depth
> in rte_event_dev_configure() like ethdev and crypto library[Jerin]
> - Removed rte_event_release() and replaced with RTE_EVENT_OP_RELEASE ops to
> reduce fast path APIs and it is redundant too[Jerin]
> - In the view of better application portability, Removed pin_event
> from rte_event_enqueue as it is just hint and Intel/NXP can not support it[Jerin]
> - Added rte_event_port_links_get()[Jerin]
> - Added rte_event_dev_dump[Harry]
>
> Notes:
>
> - This patch set is check-patch clean with an exception that
> 03/06 has one WARNING:MACRO_WITH_FLOW_CONTROL
> - Looking forward to getting additional maintainers for libeventdev
>
> TODO:
> 1) Create user guide
>
> Jerin Jacob (6):
> eventdev: introduce event driven programming model
> eventdev: define southbound driver interface
> eventdev: implement the northbound APIs
> eventdev: implement PMD registration functions
> event/skeleton: add skeleton eventdev driver
> app/test: unit test case for eventdev APIs
>
> MAINTAINERS | 5 +
> app/test/Makefile | 2 +
> app/test/test_eventdev.c | 778 +++++++++++
> config/common_base | 14 +
> doc/api/doxy-api-index.md | 1 +
> doc/api/doxy-api.conf | 1 +
> drivers/Makefile | 1 +
> drivers/event/Makefile | 36 +
> drivers/event/skeleton/Makefile | 55 +
> .../skeleton/rte_pmd_skeleton_event_version.map | 4 +
> drivers/event/skeleton/skeleton_eventdev.c | 518 +++++++
> drivers/event/skeleton/skeleton_eventdev.h | 68 +
> lib/Makefile | 1 +
> lib/librte_eal/common/include/rte_log.h | 1 +
> lib/librte_eventdev/Makefile | 57 +
> lib/librte_eventdev/rte_eventdev.c | 1220 +++++++++++++++++
> lib/librte_eventdev/rte_eventdev.h | 1407 ++++++++++++++++++++
> lib/librte_eventdev/rte_eventdev_pmd.h | 511 +++++++
> lib/librte_eventdev/rte_eventdev_version.map | 39 +
> mk/rte.app.mk | 5 +
> 20 files changed, 4724 insertions(+)
> create mode 100644 app/test/test_eventdev.c
> create mode 100644 drivers/event/Makefile
> create mode 100644 drivers/event/skeleton/Makefile
> create mode 100644 drivers/event/skeleton/rte_pmd_skeleton_event_version.map
> create mode 100644 drivers/event/skeleton/skeleton_eventdev.c
> create mode 100644 drivers/event/skeleton/skeleton_eventdev.h
> create mode 100644 lib/librte_eventdev/Makefile
> create mode 100644 lib/librte_eventdev/rte_eventdev.c
> create mode 100644 lib/librte_eventdev/rte_eventdev.h
> create mode 100644 lib/librte_eventdev/rte_eventdev_pmd.h
> create mode 100644 lib/librte_eventdev/rte_eventdev_version.map
>
^ permalink raw reply
* Re: [PATCH] app/testpmd: fix invalid port ID
From: Wu, Jingjing @ 2016-12-19 1:24 UTC (permalink / raw)
To: Lu, Wenzhuo, dev@dpdk.org; +Cc: Lu, Wenzhuo, Chen, Jing D, stable@dpdk.org
In-Reply-To: <1481613068-92744-1-git-send-email-wenzhuo.lu@intel.com>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> Sent: Tuesday, December 13, 2016 3:11 PM
> To: dev@dpdk.org
> Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Chen, Jing D
> <jing.d.chen@intel.com>; stable@dpdk.org
> Subject: [dpdk-dev] [PATCH] app/testpmd: fix invalid port ID
>
> Some CLIs don't check the input port ID, it may cause segmentation fault
> (core dumped).
>
> Fixes: 425781ff5afe ("app/testpmd: add ixgbe VF management")
>
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> Signed-off-by: Chen Jing D(Mark) <jing.d.chen@intel.com>
> ---
> CC: stable@dpdk.org
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
^ permalink raw reply
* Re: ConnectX4 100GbE - Compilation problem
From: Olga Shern @ 2016-12-18 21:00 UTC (permalink / raw)
To: george.dit@gmail.com; +Cc: dev@dpdk.org
In-Reply-To: <CAN9HtFDbmtCZ1=2EUCG4BqLcSSmaFVZG4DRU7YNnWex4TdECBg@mail.gmail.com>
Hi George,
You are right, this is the expected behavior.
Every new DPDK version that is released supported on top of latest available MLNX_OFED and this is documented in the RN.
For DPDK 16.07 you will need to use MLNX_OFED 3.3
Best Regards,
Olga
From: george.dit@gmail.com [mailto:george.dit@gmail.com]
Sent: Sunday, December 18, 2016 6:06 PM
To: Olga Shern <olgas@mellanox.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] ConnectX4 100GbE - Compilation problem
Hi again,
I have a follow up question. I noticed that when I compile DPDK with CONFIG_RTE_LIBRTE_MLX5_PMD=y, the compilation fate depends upon the OFED version that I have.
To become more clear, with DPDK 16.11 and OFED 3.4-2.0.0.0 it compiles, but DPDK 16.07 and the same OFED fails with:
== Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
In file included from /opt/dpdk/drivers/net/mlx5/mlx5.h:41:0,
from /opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:65:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c: In function ‘mlx5_mpw_new’:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:911:9: error: ‘MLX5_OPCODE_LSO_MPW’ undeclared (first use in this function)
MLX5_OPCODE_LSO_MPW);
^
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:911:9: note: each undeclared identifier is reported only once for each function it appears in
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c: In function ‘mlx5_mpw_inline_new’:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:1110:13: error: ‘MLX5_OPCODE_LSO_MPW’ undeclared (first use in this function)
MLX5_OPCODE_LSO_MPW);
^
/opt/dpdk/mk/internal/rte.compile-pre.mk:138<http://rte.compile-pre.mk:138>: recipe for target 'mlx5_rxtx.o' failed
make[6]: *** [mlx5_rxtx.o] Error 1
/opt/dpdk/mk/rte.subdir.mk:61<http://rte.subdir.mk:61>: recipe for target 'mlx5' failed
make[5]: *** [mlx5] Error 2
/opt/dpdk/mk/rte.subdir.mk:61<http://rte.subdir.mk:61>: recipe for target 'net' failed
make[4]: *** [net] Error 2
/opt/dpdk/mk/rte.sdkbuild.mk:78<http://rte.sdkbuild.mk:78>: recipe for target 'drivers' failed
make[3]: *** [drivers] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:123<http://rte.sdkroot.mk:123>: recipe for target 'all' failed
make[2]: *** [all] Error 2
/opt/dpdk/mk/rte.sdkinstall.mk:84<http://rte.sdkinstall.mk:84>: recipe for target 'pre_install' failed
make[1]: *** [pre_install] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:98<http://rte.sdkroot.mk:98>: recipe for target 'install' failed
make: *** [install] Error 2
Is there a way to have DPDK compiled with MLX5 support across a variety of DPDK versions (i.e, from 2.2.0 up to 16.11)?
Thanks in advance and best regards,
Georgios
On Sun, Dec 18, 2016 at 8:58 AM, <george.dit@gmail.com<mailto:george.dit@gmail.com>> wrote:
Hi Olga,
That was the issue indeed, thank you very much!
Best regards,
Georgios
On Sat, Dec 17, 2016 at 10:26 PM, Olga Shern <olgas@mellanox.com<mailto:olgas@mellanox.com>> wrote:
Hi Georgios,
Did you install MLNX_OFED?
You probably missing libmlx5-devel package
Best Regards,
Olga
-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of george.dit@gmail.com<mailto:george.dit@gmail.com>
Sent: Friday, December 16, 2016 4:46 PM
To: Georgios Katsikas <george.dit@gmail.com<mailto:george.dit@gmail.com>>; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] ConnectX4 100GbE - Compilation problem
Hi all,
I am coming back to you regarding the Mellanox Connectx4 100Gbps DPDK driver.
We exchanged some e-mails last summer and I managed to compile the DPDK-based Mellanox driver using DPDK 16.07 on top of Ubuntu server 16.04.01.
A few days ago, I installed a fresh Ubuntu server 16.04.1 on my machine and attempted to re-install DPDK.
This time, DPDK does not compile because of the following error:
\== Build drivers/net/mlx5
CC mlx5.o
In file included from /opt/dpdk/drivers/net/mlx5/mlx5.h:67:0,
from /opt/dpdk/drivers/net/mlx5/mlx5.c:66:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.h:46:32: fatal error:
infiniband/mlx5_hw.h: No such file or directory compilation terminated.
/opt/dpdk/mk/internal/rte.compile-pre.mk:138<http://rte.compile-pre.mk:138>: recipe for target 'mlx5.o'
failed
make[6]: *** [mlx5.o] Error 1
/opt/dpdk/mk/rte.subdir.mk:61<http://rte.subdir.mk:61>: recipe for target 'mlx5' failed
make[5]: *** [mlx5] Error 2
/opt/dpdk/mk/rte.subdir.mk:61<http://rte.subdir.mk:61>: recipe for target 'net' failed
make[4]: *** [net] Error 2
/opt/dpdk/mk/rte.sdkbuild.mk:78<http://rte.sdkbuild.mk:78>: recipe for target 'drivers' failed
make[3]: *** [drivers] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:126<http://rte.sdkroot.mk:126>: recipe for target 'all' failed
make[2]: *** [all] Error 2
/opt/dpdk/mk/rte.sdkinstall.mk:85<http://rte.sdkinstall.mk:85>: recipe for target 'pre_install' failed
make[1]: *** [pre_install] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:101<http://rte.sdkroot.mk:101>: recipe for target 'install' failed
make: *** [install] Error 2
[ ERROR] [ CheckStatus] Command make install failed
I tried 2 different kernels (4.4 and 4.8) both compiled from sources and I also tried 3 different DPDk versions (16.04/07/11) but they all get stuck to this point.
The way I configure DPDK is via config/common_base, where I set CONFIG_RTE_LIBRTE_MLX5_PMD=y.
The file infiniband/mlx5_hw.h does not exist in my system although I have installed libmlx5.
Do you know why does this happen now?
Thanks in advance and best regards,
Georgios
On Thu, Aug 18, 2016 at 7:35 PM, <george.dit@gmail.com<mailto:george.dit@gmail.com>> wrote:
> Hi Adrien,
>
> Thanks for the prompt reply!
> You are right, I didn't go via the DPDK route, in the hope that
> Mellanox will provide the exact source and configuration.
> DPDK 16.07 from dpdk.org<http://dpdk.org> works like a charm and my NIC is in PMD mode,
> thanks a lot for your guidance!
>
> Best regards,
> Georgios
>
> On Thu, Aug 18, 2016 at 6:05 PM, Adrien Mazarguil <
> adrien.mazarguil@6wind.com<mailto:adrien.mazarguil@6wind.com>> wrote:
>
>> Hi George,
>>
>> On Thu, Aug 18, 2016 at 05:41:38PM +0200, george.dit@gmail.com<mailto:george.dit@gmail.com> wrote:
>> > Hi,
>> >
>> > I have a single port Mellanox ConnectX 4 100GbE NIC and I want to
>> > test
>> its
>> > Rx/Tx capabilites using DPDK.
>> > My system runs a Linux kernel 4.4 compiled from sources.
>> >
>> > I found the PMD driver for this NIC as provided by Mellanox here
>> > <http://www.mellanox.com/page/products_dyn?product_family=20
>> 9&mtag=pmd_for_dpdk>
>> > .
>> > Following this
>> > <http://www.mellanox.com/related-docs/prod_software/MLNX_
>> DPDK_Quick_Start_Guide_v2.2_2.7.pdf>
>> > guideline, I put my NIC in Ethernet mode, configured the 3 options
>> > regarding mlx5 in the config/common_linuxapp file and applied 'make
>> install
>> > T=x86_64-native-linuxapp-gcc'.
>>
>> Please note this is a third party package maintained by Mellanox,
>> therefore this mailing list is not the right place to discuss related
>> errors, unless they can be reproduced with a version downloaded from
>> dpdk.org<http://dpdk.org>.
>>
>> > Then, I stumbled upon a compilation problem in the mlx4 module.
>> > The compiler's output is as follows:
>> >
>> > == Build drivers/net/mlx4
>> > CC mlx4.o
>> > In file included from /usr/include/linux/if.h:31:0,
>> > from /opt/MLNX_DPDK_2.2_2.7/drivers
>> /net/mlx4/mlx4.c:57:
>> > /usr/include/linux/hdlc/ioctl.h:73:14: error: ‘IFNAMSIZ’ undeclared
>> here
>> > (not in a function)
>> > char master[IFNAMSIZ]; /* Name of master FRAD device */
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:612:53: error:
>> > ‘struct ifreq’ declared inside parameter list [-Werror]
>> > priv_ifreq(const struct priv *priv, int req, struct ifreq *ifr)
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:612:53: error: its
>> scope is
>> > only this definition or declaration, which is probably not what you
>> > want [-Werror]
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> ‘priv_ifreq’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:619:32: error:
>> dereferencing
>> > pointer to incomplete type ‘struct ifreq’
>> > if (priv_get_ifname(priv, &ifr->ifr_name) == 0)
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> ‘rxq_setup’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:3735:29: error:
>> > unused parameter ‘inactive’ [-Werror=unused-parameter]
>> > unsigned int socket, int inactive, const struct rte_eth_rxconf
>> *conf,
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> > ‘mlx4_link_update_unlocked’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4712:15: error:
>> > storage
>> size
>> > of ‘ifr’ isn’t known
>> > struct ifreq ifr;
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4724:43: error: ‘IFF_UP’
>> > undeclared (first use in this function)
>> > dev_link.link_status = ((ifr.ifr_flags & IFF_UP) &&
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4724:43: note: each
>> > undeclared identifier is reported only once for each function it
>> appears in
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4725:22: error:
>> > ‘IFF_RUNNING’ undeclared (first use in this function)
>> > (ifr.ifr_flags & IFF_RUNNING));
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4712:15: error:
>> > unused variable ‘ifr’ [-Werror=unused-variable]
>> > struct ifreq ifr;
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> > ‘mlx4_dev_get_flow_ctrl’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4880:15: error:
>> > storage
>> size
>> > of ‘ifr’ isn’t known
>> > struct ifreq ifr;
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4880:15: error:
>> > unused variable ‘ifr’ [-Werror=unused-variable]
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> > ‘mlx4_dev_set_flow_ctrl’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4930:15: error:
>> > storage
>> size
>> > of ‘ifr’ isn’t known
>> > struct ifreq ifr;
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4930:15: error:
>> > unused variable ‘ifr’ [-Werror=unused-variable]
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> ‘priv_get_mac’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5184:15: error:
>> > storage
>> size
>> > of ‘request’ isn’t known
>> > struct ifreq request;
>> > ^
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5184:15: error:
>> > unused variable ‘request’ [-Werror=unused-variable]
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> > ‘mlx4_pci_devinit’:
>> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5725:25: error: ‘IFF_UP’
>> > undeclared (first use in this function)
>> > priv_set_flags(priv, ~IFF_UP, IFF_UP);
>> > ^
>> > cc1: all warnings being treated as errors
>> > /opt/MLNX_DPDK_2.2_2.7/mk/internal/rte.compile-pre.mk:126<http://rte.compile-pre.mk:126>: recipe
>> > for target 'mlx4.o' failed
>> >
>> > Iwould appreciate any suggestions and guidance.
>>
>> Well fortunately these errors are also present in v2.2.0 and should
>> have been addressed since v16.07 by the following commit:
>>
>> http://dpdk.org/browse/dpdk/commit/?id=d06c608c013c36711e7a
>> 693b3fece68a93ae4369
>>
>> You can either upgrade to v16.07, back-port this commit yourself or
>> wait for an update from Mellanox.
>>
>> --
>> Adrien Mazarguil
>> 6WIND
>>
>
>
>
> --
> Georgios Katsikas
> Ph.D. Student and Research Assistant
> Network Systems Lab (NSL)
>
>
>
> *E-Mail:* george <george.katsikas@imdea.org<mailto:george.katsikas@imdea.org>>.dit@gmail.com<mailto:dit@gmail.com>
> *Web Site:* http://www.di.uoa.gr/~katsikas/
> <http://people.networks.imdea.org/~george_katsikas/index.html>
>
--
Georgios Katsikas
Ph.D. Student and Research Assistant
Network Systems Lab (NSL)
*E-Mail:* george <george.katsikas@imdea.org<mailto:george.katsikas@imdea.org>>.dit@gmail.com<mailto:dit@gmail.com>
*Web Site:* http://www.di.uoa.gr/~katsikas/ <http://people.networks.imdea.org/~george_katsikas/index.html>
--
Georgios Katsikas
Ph.D. Student and Research Assistant
Network Systems Lab (NSL)
[http://www.kth.se/polopoly_fs/1.77259!/image/logo-main-2013.png]
E-Mail: george<mailto:george.katsikas@imdea.org>.dit@gmail.com<mailto:dit@gmail.com>
Web Site: http://www.di.uoa.gr/~katsikas/<http://people.networks.imdea.org/~george_katsikas/index.html>
--
Georgios Katsikas
Ph.D. Student and Research Assistant
Network Systems Lab (NSL)
[http://www.kth.se/polopoly_fs/1.77259!/image/logo-main-2013.png]
E-Mail: george<mailto:george.katsikas@imdea.org>.dit@gmail.com<mailto:dit@gmail.com>
Web Site: http://www.di.uoa.gr/~katsikas/<http://people.networks.imdea.org/~george_katsikas/index.html>
^ permalink raw reply
* Re: [PATCH] doc: fix required tools list layout
From: Mcnamara, John @ 2016-12-18 20:50 UTC (permalink / raw)
To: Baruch Siach; +Cc: dev@dpdk.org
In-Reply-To: <20161218191100.nzgn2w7465qnk26v@tarshish>
> -----Original Message-----
> From: Baruch Siach [mailto:baruch@tkos.co.il]
> Sent: Sunday, December 18, 2016 7:11 PM
> To: Mcnamara, John <john.mcnamara@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] doc: fix required tools list layout
>
> Hi John,
>
> On Thu, Dec 15, 2016 at 03:09:32PM +0000, Mcnamara, John wrote:
> > > -----Original Message-----
> > > From: Baruch Siach [mailto:baruch at tkos.co.il]
> > > Sent: Tuesday, December 13, 2016 10:04 AM
> > > To: dev at dpdk.org
> > > Cc: Mcnamara, John <john.mcnamara at intel.com>; David Marchand
> > > <david.marchand at 6wind.com>; Baruch Siach <baruch at tkos.co.il>
> > > Subject: [PATCH] doc: fix required tools list layout
> > >
> > > The Python requirement should appear in the bullet list.
> > >
> > > Signed-off-by: Baruch Siach <baruch at tkos.co.il>
> > > ---
> > > doc/guides/linux_gsg/sys_reqs.rst | 4 +---
> > > 1 file changed, 1 insertion(+), 3 deletions(-)
> > >
> > > diff --git a/doc/guides/linux_gsg/sys_reqs.rst
> > > b/doc/guides/linux_gsg/sys_reqs.rst
> > > index 3d743421595a..621cc9ddaef6 100644
> > > --- a/doc/guides/linux_gsg/sys_reqs.rst
> > > +++ b/doc/guides/linux_gsg/sys_reqs.rst
> > > @@ -84,9 +84,7 @@ Compilation of the DPDK
> > > x86_x32 ABI is currently supported with distribution packages
> > > only on Ubuntu
> > > higher than 13.10 or recent Debian distribution. The only
> > > supported compiler is gcc 4.9+.
> > >
> > > -.. note::
> > > -
> > > - Python, version 2.6 or 2.7, to use various helper scripts
> included in
> > > the DPDK package.
> > > +* Python, version 2.6 or 2.7, to use various helper scripts
> included in
> > > the DPDK package.
> >
> > In addition to this change the note on the previous item should be
> > indented to the level of the bullet item. It is probably worth making
> > that change at the same time.
>
> All items are equally aligned as far as I can see. The 32bit on 64bit
> requirement bullets are sub-items of the previous item. Am I missing
> anything?
Hi Baruch,
The note should be indented to the level of the first level bullet item text rather
than the margin since it is a note on that particular item and not a general note.
Like this:
* Additional packages required for 32-bit compilation on 64-bit systems are:
* glibc.i686, libgcc.i686, libstdc++.i686 and glibc-devel.i686 for Intel i686/x86_64;
* glibc.ppc64, libgcc.ppc64, libstdc++.ppc64 and glibc-devel.ppc64 for IBM ppc_64;
.. note::
x86_x32 ABI is currently supported with distribution packages only on Ubuntu
higher than 13.10 or recent Debian distribution. The only supported compiler is gcc 4.9+.
If you generate the html before and after you will see the difference.
>
> > Also, the Python version should probably say 2.7+ and 3.2+ if this
> > patch is
> > accepted:
> >
> > http://dpdk.org/dev/patchwork/patch/17775/
> >
> > However, since that change hasn't been acked/merged yet you can leave
> > that part of your patch as it is and I'll fix the version numbers in
> > the other patch.
>
> Note that your updated patch[1] conflicts with this one.
>
Yes. :-)
It also conflicts with Thomas' patch to move the directories. I'll rebase based
on whatever order the patches are applied in.
John
^ permalink raw reply
* Re: [PATCH] doc: fix required tools list layout
From: Baruch Siach @ 2016-12-18 19:11 UTC (permalink / raw)
To: Mcnamara, John; +Cc: dev
In-Reply-To: <B27915DBBA3421428155699D51E4CFE202686088@IRSMSX103.ger.corp.intel.com>
Hi John,
On Thu, Dec 15, 2016 at 03:09:32PM +0000, Mcnamara, John wrote:
> > -----Original Message-----
> > From: Baruch Siach [mailto:baruch at tkos.co.il]
> > Sent: Tuesday, December 13, 2016 10:04 AM
> > To: dev at dpdk.org
> > Cc: Mcnamara, John <john.mcnamara at intel.com>; David Marchand
> > <david.marchand at 6wind.com>; Baruch Siach <baruch at tkos.co.il>
> > Subject: [PATCH] doc: fix required tools list layout
> >
> > The Python requirement should appear in the bullet list.
> >
> > Signed-off-by: Baruch Siach <baruch at tkos.co.il>
> > ---
> > doc/guides/linux_gsg/sys_reqs.rst | 4 +---
> > 1 file changed, 1 insertion(+), 3 deletions(-)
> >
> > diff --git a/doc/guides/linux_gsg/sys_reqs.rst
> > b/doc/guides/linux_gsg/sys_reqs.rst
> > index 3d743421595a..621cc9ddaef6 100644
> > --- a/doc/guides/linux_gsg/sys_reqs.rst
> > +++ b/doc/guides/linux_gsg/sys_reqs.rst
> > @@ -84,9 +84,7 @@ Compilation of the DPDK
> > x86_x32 ABI is currently supported with distribution packages only on
> > Ubuntu
> > higher than 13.10 or recent Debian distribution. The only supported
> > compiler is gcc 4.9+.
> >
> > -.. note::
> > -
> > - Python, version 2.6 or 2.7, to use various helper scripts included in
> > the DPDK package.
> > +* Python, version 2.6 or 2.7, to use various helper scripts included in
> > the DPDK package.
>
> In addition to this change the note on the previous item should be indented
> to the level of the bullet item. It is probably worth making that change at
> the same time.
All items are equally aligned as far as I can see. The 32bit on 64bit
requirement bullets are sub-items of the previous item. Am I missing anything?
> Also, the Python version should probably say 2.7+ and 3.2+ if this patch is
> accepted:
>
> http://dpdk.org/dev/patchwork/patch/17775/
>
> However, since that change hasn't been acked/merged yet you can leave that
> part of your patch as it is and I'll fix the version numbers in the other
> patch.
Note that your updated patch[1] conflicts with this one.
[1] http://dpdk.org/dev/patchwork/patch/18152/
baruch
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch@tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -
^ permalink raw reply
* Re: ConnectX4 100GbE - Compilation problem
From: george.dit @ 2016-12-18 16:06 UTC (permalink / raw)
To: Olga Shern; +Cc: dev@dpdk.org
In-Reply-To: <CAN9HtFC0O9FF4SeBM5HKBBgy+Ma09EyGdjqpbfFZSweCA20YzQ@mail.gmail.com>
Hi again,
I have a follow up question. I noticed that when I compile DPDK with
CONFIG_RTE_LIBRTE_MLX5_PMD=y, the compilation fate depends upon the OFED
version that I have.
To become more clear, with DPDK 16.11 and OFED 3.4-2.0.0.0 it compiles, but
DPDK 16.07 and the same OFED fails with:
== Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
In file included from /opt/dpdk/drivers/net/mlx5/mlx5.h:41:0,
from /opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:65:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c: In function ‘mlx5_mpw_new’:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:911:9: error: ‘MLX5_OPCODE_LSO_MPW’
undeclared (first use in this function)
MLX5_OPCODE_LSO_MPW);
^
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:911:9: note: each undeclared
identifier is reported only once for each function it appears in
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c: In function ‘mlx5_mpw_inline_new’:
/opt/dpdk/drivers/net/mlx5/mlx5_rxtx.c:1110:13: error:
‘MLX5_OPCODE_LSO_MPW’ undeclared (first use in this function)
MLX5_OPCODE_LSO_MPW);
^
/opt/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target
'mlx5_rxtx.o' failed
make[6]: *** [mlx5_rxtx.o] Error 1
/opt/dpdk/mk/rte.subdir.mk:61: recipe for target 'mlx5' failed
make[5]: *** [mlx5] Error 2
/opt/dpdk/mk/rte.subdir.mk:61: recipe for target 'net' failed
make[4]: *** [net] Error 2
/opt/dpdk/mk/rte.sdkbuild.mk:78: recipe for target 'drivers' failed
make[3]: *** [drivers] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:123: recipe for target 'all' failed
make[2]: *** [all] Error 2
/opt/dpdk/mk/rte.sdkinstall.mk:84: recipe for target 'pre_install' failed
make[1]: *** [pre_install] Error 2
/opt/dpdk/mk/rte.sdkroot.mk:98: recipe for target 'install' failed
make: *** [install] Error 2
Is there a way to have DPDK compiled with MLX5 support across a variety of
DPDK versions (i.e, from 2.2.0 up to 16.11)?
Thanks in advance and best regards,
Georgios
On Sun, Dec 18, 2016 at 8:58 AM, <george.dit@gmail.com> wrote:
> Hi Olga,
>
> That was the issue indeed, thank you very much!
>
> Best regards,
> Georgios
>
> On Sat, Dec 17, 2016 at 10:26 PM, Olga Shern <olgas@mellanox.com> wrote:
>
>> Hi Georgios,
>>
>> Did you install MLNX_OFED?
>> You probably missing libmlx5-devel package
>>
>> Best Regards,
>> Olga
>>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of george.dit@gmail.com
>> Sent: Friday, December 16, 2016 4:46 PM
>> To: Georgios Katsikas <george.dit@gmail.com>; dev@dpdk.org
>> Subject: Re: [dpdk-dev] ConnectX4 100GbE - Compilation problem
>>
>> Hi all,
>>
>> I am coming back to you regarding the Mellanox Connectx4 100Gbps DPDK
>> driver.
>> We exchanged some e-mails last summer and I managed to compile the
>> DPDK-based Mellanox driver using DPDK 16.07 on top of Ubuntu server
>> 16.04.01.
>>
>> A few days ago, I installed a fresh Ubuntu server 16.04.1 on my machine
>> and attempted to re-install DPDK.
>> This time, DPDK does not compile because of the following error:
>> \== Build drivers/net/mlx5
>> CC mlx5.o
>> In file included from /opt/dpdk/drivers/net/mlx5/mlx5.h:67:0,
>> from /opt/dpdk/drivers/net/mlx5/mlx5.c:66:
>> /opt/dpdk/drivers/net/mlx5/mlx5_rxtx.h:46:32: fatal error:
>> infiniband/mlx5_hw.h: No such file or directory compilation terminated.
>> /opt/dpdk/mk/internal/rte.compile-pre.mk:138: recipe for target 'mlx5.o'
>> failed
>> make[6]: *** [mlx5.o] Error 1
>> /opt/dpdk/mk/rte.subdir.mk:61: recipe for target 'mlx5' failed
>> make[5]: *** [mlx5] Error 2
>> /opt/dpdk/mk/rte.subdir.mk:61: recipe for target 'net' failed
>> make[4]: *** [net] Error 2
>> /opt/dpdk/mk/rte.sdkbuild.mk:78: recipe for target 'drivers' failed
>> make[3]: *** [drivers] Error 2
>> /opt/dpdk/mk/rte.sdkroot.mk:126: recipe for target 'all' failed
>> make[2]: *** [all] Error 2
>> /opt/dpdk/mk/rte.sdkinstall.mk:85: recipe for target 'pre_install' failed
>> make[1]: *** [pre_install] Error 2
>> /opt/dpdk/mk/rte.sdkroot.mk:101: recipe for target 'install' failed
>> make: *** [install] Error 2
>> [ ERROR] [ CheckStatus] Command make install failed
>>
>> I tried 2 different kernels (4.4 and 4.8) both compiled from sources and
>> I also tried 3 different DPDk versions (16.04/07/11) but they all get stuck
>> to this point.
>> The way I configure DPDK is via config/common_base, where I set
>> CONFIG_RTE_LIBRTE_MLX5_PMD=y.
>> The file infiniband/mlx5_hw.h does not exist in my system although I have
>> installed libmlx5.
>>
>> Do you know why does this happen now?
>>
>> Thanks in advance and best regards,
>> Georgios
>>
>> On Thu, Aug 18, 2016 at 7:35 PM, <george.dit@gmail.com> wrote:
>>
>> > Hi Adrien,
>> >
>> > Thanks for the prompt reply!
>> > You are right, I didn't go via the DPDK route, in the hope that
>> > Mellanox will provide the exact source and configuration.
>> > DPDK 16.07 from dpdk.org works like a charm and my NIC is in PMD mode,
>> > thanks a lot for your guidance!
>> >
>> > Best regards,
>> > Georgios
>> >
>> > On Thu, Aug 18, 2016 at 6:05 PM, Adrien Mazarguil <
>> > adrien.mazarguil@6wind.com> wrote:
>> >
>> >> Hi George,
>> >>
>> >> On Thu, Aug 18, 2016 at 05:41:38PM +0200, george.dit@gmail.com wrote:
>> >> > Hi,
>> >> >
>> >> > I have a single port Mellanox ConnectX 4 100GbE NIC and I want to
>> >> > test
>> >> its
>> >> > Rx/Tx capabilites using DPDK.
>> >> > My system runs a Linux kernel 4.4 compiled from sources.
>> >> >
>> >> > I found the PMD driver for this NIC as provided by Mellanox here
>> >> > <http://www.mellanox.com/page/products_dyn?product_family=20
>> >> 9&mtag=pmd_for_dpdk>
>> >> > .
>> >> > Following this
>> >> > <http://www.mellanox.com/related-docs/prod_software/MLNX_
>> >> DPDK_Quick_Start_Guide_v2.2_2.7.pdf>
>> >> > guideline, I put my NIC in Ethernet mode, configured the 3 options
>> >> > regarding mlx5 in the config/common_linuxapp file and applied 'make
>> >> install
>> >> > T=x86_64-native-linuxapp-gcc'.
>> >>
>> >> Please note this is a third party package maintained by Mellanox,
>> >> therefore this mailing list is not the right place to discuss related
>> >> errors, unless they can be reproduced with a version downloaded from
>> >> dpdk.org.
>> >>
>> >> > Then, I stumbled upon a compilation problem in the mlx4 module.
>> >> > The compiler's output is as follows:
>> >> >
>> >> > == Build drivers/net/mlx4
>> >> > CC mlx4.o
>> >> > In file included from /usr/include/linux/if.h:31:0,
>> >> > from /opt/MLNX_DPDK_2.2_2.7/drivers
>> >> /net/mlx4/mlx4.c:57:
>> >> > /usr/include/linux/hdlc/ioctl.h:73:14: error: ‘IFNAMSIZ’ undeclared
>> >> here
>> >> > (not in a function)
>> >> > char master[IFNAMSIZ]; /* Name of master FRAD device */
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:612:53: error:
>> >> > ‘struct ifreq’ declared inside parameter list [-Werror]
>> >> > priv_ifreq(const struct priv *priv, int req, struct ifreq *ifr)
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:612:53: error: its
>> >> scope is
>> >> > only this definition or declaration, which is probably not what you
>> >> > want [-Werror]
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> ‘priv_ifreq’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:619:32: error:
>> >> dereferencing
>> >> > pointer to incomplete type ‘struct ifreq’
>> >> > if (priv_get_ifname(priv, &ifr->ifr_name) == 0)
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> ‘rxq_setup’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:3735:29: error:
>> >> > unused parameter ‘inactive’ [-Werror=unused-parameter]
>> >> > unsigned int socket, int inactive, const struct rte_eth_rxconf
>> >> *conf,
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> > ‘mlx4_link_update_unlocked’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4712:15: error:
>> >> > storage
>> >> size
>> >> > of ‘ifr’ isn’t known
>> >> > struct ifreq ifr;
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4724:43: error:
>> ‘IFF_UP’
>> >> > undeclared (first use in this function)
>> >> > dev_link.link_status = ((ifr.ifr_flags & IFF_UP) &&
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4724:43: note: each
>> >> > undeclared identifier is reported only once for each function it
>> >> appears in
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4725:22: error:
>> >> > ‘IFF_RUNNING’ undeclared (first use in this function)
>> >> > (ifr.ifr_flags & IFF_RUNNING));
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4712:15: error:
>> >> > unused variable ‘ifr’ [-Werror=unused-variable]
>> >> > struct ifreq ifr;
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> > ‘mlx4_dev_get_flow_ctrl’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4880:15: error:
>> >> > storage
>> >> size
>> >> > of ‘ifr’ isn’t known
>> >> > struct ifreq ifr;
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4880:15: error:
>> >> > unused variable ‘ifr’ [-Werror=unused-variable]
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> > ‘mlx4_dev_set_flow_ctrl’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4930:15: error:
>> >> > storage
>> >> size
>> >> > of ‘ifr’ isn’t known
>> >> > struct ifreq ifr;
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:4930:15: error:
>> >> > unused variable ‘ifr’ [-Werror=unused-variable]
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> ‘priv_get_mac’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5184:15: error:
>> >> > storage
>> >> size
>> >> > of ‘request’ isn’t known
>> >> > struct ifreq request;
>> >> > ^
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5184:15: error:
>> >> > unused variable ‘request’ [-Werror=unused-variable]
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c: In function
>> >> > ‘mlx4_pci_devinit’:
>> >> > /opt/MLNX_DPDK_2.2_2.7/drivers/net/mlx4/mlx4.c:5725:25: error:
>> ‘IFF_UP’
>> >> > undeclared (first use in this function)
>> >> > priv_set_flags(priv, ~IFF_UP, IFF_UP);
>> >> > ^
>> >> > cc1: all warnings being treated as errors
>> >> > /opt/MLNX_DPDK_2.2_2.7/mk/internal/rte.compile-pre.mk:126: recipe
>> >> > for target 'mlx4.o' failed
>> >> >
>> >> > Iwould appreciate any suggestions and guidance.
>> >>
>> >> Well fortunately these errors are also present in v2.2.0 and should
>> >> have been addressed since v16.07 by the following commit:
>> >>
>> >> http://dpdk.org/browse/dpdk/commit/?id=d06c608c013c36711e7a
>> >> 693b3fece68a93ae4369
>> >>
>> >> You can either upgrade to v16.07, back-port this commit yourself or
>> >> wait for an update from Mellanox.
>> >>
>> >> --
>> >> Adrien Mazarguil
>> >> 6WIND
>> >>
>> >
>> >
>> >
>> > --
>> > Georgios Katsikas
>> > Ph.D. Student and Research Assistant
>> > Network Systems Lab (NSL)
>> >
>> >
>> >
>> > *E-Mail:* george <george.katsikas@imdea.org>.dit@gmail.com
>> > *Web Site:* http://www.di.uoa.gr/~katsikas/
>> > <http://people.networks.imdea.org/~george_katsikas/index.html>
>> >
>>
>>
>>
>> --
>> Georgios Katsikas
>> Ph.D. Student and Research Assistant
>> Network Systems Lab (NSL)
>>
>>
>>
>> *E-Mail:* george <george.katsikas@imdea.org>.dit@gmail.com
>> *Web Site:* http://www.di.uoa.gr/~katsikas/ <
>> http://people.networks.imdea.org/~george_katsikas/index.html>
>>
>
>
>
> --
> Georgios Katsikas
> Ph.D. Student and Research Assistant
> Network Systems Lab (NSL)
>
>
>
> *E-Mail:* george <george.katsikas@imdea.org>.dit@gmail.com
> *Web Site:* http://www.di.uoa.gr/~katsikas/
> <http://people.networks.imdea.org/~george_katsikas/index.html>
>
--
Georgios Katsikas
Ph.D. Student and Research Assistant
Network Systems Lab (NSL)
*E-Mail:* george <george.katsikas@imdea.org>.dit@gmail.com
*Web Site:* http://www.di.uoa.gr/~katsikas/
<http://people.networks.imdea.org/~george_katsikas/index.html>
^ permalink raw reply
* [PATCH v3 3/3] doc: add required python versions to docs
From: John McNamara @ 2016-12-18 14:32 UTC (permalink / raw)
To: dev; +Cc: mkletzan, thomas.monjalon, nhorman, John McNamara
In-Reply-To: <1481212265-10229-1-git-send-email-john.mcnamara@intel.com>
Add a requirement to support both Python 2 and 3 to the
DPDK Python Coding Standards and Getting started Guide.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
---
doc/guides/contributing/coding_style.rst | 3 ++-
doc/guides/linux_gsg/sys_reqs.rst | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/doc/guides/contributing/coding_style.rst b/doc/guides/contributing/coding_style.rst
index 1eb67f3..4163960 100644
--- a/doc/guides/contributing/coding_style.rst
+++ b/doc/guides/contributing/coding_style.rst
@@ -690,6 +690,7 @@ Control Statements
Python Code
-----------
-All python code should be compliant with `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
+All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+`PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
The ``pep8`` tool can be used for testing compliance with the guidelines.
diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index 3d74342..9653a13 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -86,7 +86,7 @@ Compilation of the DPDK
.. note::
- Python, version 2.6 or 2.7, to use various helper scripts included in the DPDK package.
+ Python, version 2.7+ or 3.2+, to use various helper scripts included in the DPDK package.
**Optional Tools:**
--
2.7.4
^ permalink raw reply related
* [PATCH v3 2/3] app: make python apps python2/3 compliant
From: John McNamara @ 2016-12-18 14:32 UTC (permalink / raw)
To: dev; +Cc: mkletzan, thomas.monjalon, nhorman, John McNamara
In-Reply-To: <1481212265-10229-1-git-send-email-john.mcnamara@intel.com>
Make all the DPDK Python apps work with Python 2 or 3 to
allow them to work with whatever is the system default.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
---
app/cmdline_test/cmdline_test.py | 26 ++++++++++++------------
app/cmdline_test/cmdline_test_data.py | 2 --
app/test/autotest.py | 10 ++++-----
app/test/autotest_data.py | 2 --
app/test/autotest_runner.py | 37 ++++++++++++++++------------------
app/test/autotest_test_funcs.py | 2 --
tools/cpu_layout.py | 38 ++++++++++++++++++-----------------
tools/dpdk-devbind.py | 2 +-
tools/dpdk-pmdinfo.py | 14 +++++++------
9 files changed, 64 insertions(+), 69 deletions(-)
diff --git a/app/cmdline_test/cmdline_test.py b/app/cmdline_test/cmdline_test.py
index 4729987..229f71f 100755
--- a/app/cmdline_test/cmdline_test.py
+++ b/app/cmdline_test/cmdline_test.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
# BSD LICENSE
#
@@ -32,7 +32,7 @@
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Script that runs cmdline_test app and feeds keystrokes into it.
-
+from __future__ import print_function
import cmdline_test_data
import os
import pexpect
@@ -81,38 +81,38 @@ def runHistoryTest(child):
# the path to cmdline_test executable is supplied via command-line.
if len(sys.argv) < 2:
- print "Error: please supply cmdline_test app path"
+ print("Error: please supply cmdline_test app path")
sys.exit(1)
test_app_path = sys.argv[1]
if not os.path.exists(test_app_path):
- print "Error: please supply cmdline_test app path"
+ print("Error: please supply cmdline_test app path")
sys.exit(1)
child = pexpect.spawn(test_app_path)
-print "Running command-line tests..."
+print("Running command-line tests...")
for test in cmdline_test_data.tests:
- print (test["Name"] + ":").ljust(30),
+ testname = (test["Name"] + ":").ljust(30)
try:
runTest(child, test)
- print "PASS"
+ print(testname, "PASS")
except:
- print "FAIL"
- print child
+ print(testname, "FAIL")
+ print(child)
sys.exit(1)
# since last test quits the app, run new instance
child = pexpect.spawn(test_app_path)
-print ("History fill test:").ljust(30),
+testname = ("History fill test:").ljust(30)
try:
runHistoryTest(child)
- print "PASS"
+ print(testname, "PASS")
except:
- print "FAIL"
- print child
+ print(testname, "FAIL")
+ print(child)
sys.exit(1)
child.close()
sys.exit(0)
diff --git a/app/cmdline_test/cmdline_test_data.py b/app/cmdline_test/cmdline_test_data.py
index 3ce6cbc..28dfefe 100644
--- a/app/cmdline_test/cmdline_test_data.py
+++ b/app/cmdline_test/cmdline_test_data.py
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-
# BSD LICENSE
#
# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
diff --git a/app/test/autotest.py b/app/test/autotest.py
index 3a00538..5c19a02 100644
--- a/app/test/autotest.py
+++ b/app/test/autotest.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
# BSD LICENSE
#
@@ -32,15 +32,15 @@
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Script that uses either test app or qemu controlled by python-pexpect
-
+from __future__ import print_function
import autotest_data
import autotest_runner
import sys
def usage():
- print"Usage: autotest.py [test app|test iso image]",
- print "[target] [whitelist|-blacklist]"
+ print("Usage: autotest.py [test app|test iso image] ",
+ "[target] [whitelist|-blacklist]")
if len(sys.argv) < 3:
usage()
@@ -63,7 +63,7 @@ def usage():
cmdline = "%s -c f -n 4" % (sys.argv[1])
-print cmdline
+print(cmdline)
runner = autotest_runner.AutotestRunner(cmdline, target, test_blacklist,
test_whitelist)
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 0cf4cfd..0cd598b 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -1,5 +1,3 @@
-#!/usr/bin/env python
-
# BSD LICENSE
#
# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index 55b63a8..fc882ec 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-
# BSD LICENSE
#
# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
@@ -271,15 +269,16 @@ def __process_results(self, results):
total_time = int(cur_time - self.start)
# print results, test run time and total time since start
- print ("%s:" % test_name).ljust(30),
- print result_str.ljust(29),
- print "[%02dm %02ds]" % (test_time / 60, test_time % 60),
+ result = ("%s:" % test_name).ljust(30)
+ result += result_str.ljust(29)
+ result += "[%02dm %02ds]" % (test_time / 60, test_time % 60)
# don't print out total time every line, it's the same anyway
if i == len(results) - 1:
- print "[%02dm %02ds]" % (total_time / 60, total_time % 60)
+ print(result,
+ "[%02dm %02ds]" % (total_time / 60, total_time % 60))
else:
- print ""
+ print(result)
# if test failed and it wasn't a "start" test
if test_result < 0 and not i == 0:
@@ -294,7 +293,7 @@ def __process_results(self, results):
f = open("%s_%s_report.rst" %
(self.target, test_name), "w")
except IOError:
- print "Report for %s could not be created!" % test_name
+ print("Report for %s could not be created!" % test_name)
else:
with f:
f.write(report)
@@ -360,12 +359,10 @@ def run_all_tests(self):
try:
# create table header
- print ""
- print "Test name".ljust(30),
- print "Test result".ljust(29),
- print "Test".center(9),
- print "Total".center(9)
- print "=" * 80
+ print("")
+ print("Test name".ljust(30), "Test result".ljust(29),
+ "Test".center(9), "Total".center(9))
+ print("=" * 80)
# make a note of tests start time
self.start = time.time()
@@ -407,11 +404,11 @@ def run_all_tests(self):
total_time = int(cur_time - self.start)
# print out summary
- print "=" * 80
- print "Total run time: %02dm %02ds" % (total_time / 60,
- total_time % 60)
+ print("=" * 80)
+ print("Total run time: %02dm %02ds" % (total_time / 60,
+ total_time % 60))
if self.fails != 0:
- print "Number of failed tests: %s" % str(self.fails)
+ print("Number of failed tests: %s" % str(self.fails))
# write summary to logfile
self.logfile.write("Summary\n")
@@ -420,8 +417,8 @@ def run_all_tests(self):
self.logfile.write("Failed tests: ".ljust(
15) + "%i\n" % self.fails)
except:
- print "Exception occurred"
- print sys.exc_info()
+ print("Exception occurred")
+ print(sys.exc_info())
self.fails = 1
# drop logs from all executions to a logfile
diff --git a/app/test/autotest_test_funcs.py b/app/test/autotest_test_funcs.py
index c482ea8..1c5f390 100644
--- a/app/test/autotest_test_funcs.py
+++ b/app/test/autotest_test_funcs.py
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-
# BSD LICENSE
#
# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
diff --git a/tools/cpu_layout.py b/tools/cpu_layout.py
index ccc22ec..0e049a6 100755
--- a/tools/cpu_layout.py
+++ b/tools/cpu_layout.py
@@ -1,4 +1,5 @@
-#! /usr/bin/python
+#!/usr/bin/env python
+
#
# BSD LICENSE
#
@@ -31,7 +32,7 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
-
+from __future__ import print_function
import sys
sockets = []
@@ -55,7 +56,7 @@
for core in core_details:
for field in ["processor", "core id", "physical id"]:
if field not in core:
- print "Error getting '%s' value from /proc/cpuinfo" % field
+ print("Error getting '%s' value from /proc/cpuinfo" % field)
sys.exit(1)
core[field] = int(core[field])
@@ -68,29 +69,30 @@
core_map[key] = []
core_map[key].append(core["processor"])
-print "============================================================"
-print "Core and Socket Information (as reported by '/proc/cpuinfo')"
-print "============================================================\n"
-print "cores = ", cores
-print "sockets = ", sockets
-print ""
+print("============================================================")
+print("Core and Socket Information (as reported by '/proc/cpuinfo')")
+print("============================================================\n")
+print("cores = ", cores)
+print("sockets = ", sockets)
+print("")
max_processor_len = len(str(len(cores) * len(sockets) * 2 - 1))
max_core_map_len = max_processor_len * 2 + len('[, ]') + len('Socket ')
max_core_id_len = len(str(max(cores)))
-print " ".ljust(max_core_id_len + len('Core ')),
+output = " ".ljust(max_core_id_len + len('Core '))
for s in sockets:
- print "Socket %s" % str(s).ljust(max_core_map_len - len('Socket ')),
-print ""
+ output += " Socket %s" % str(s).ljust(max_core_map_len - len('Socket '))
+print(output)
-print " ".ljust(max_core_id_len + len('Core ')),
+output = " ".ljust(max_core_id_len + len('Core '))
for s in sockets:
- print "--------".ljust(max_core_map_len),
-print ""
+ output += " --------".ljust(max_core_map_len)
+ output += " "
+print(output)
for c in cores:
- print "Core %s" % str(c).ljust(max_core_id_len),
+ output = "Core %s" % str(c).ljust(max_core_id_len)
for s in sockets:
- print str(core_map[(s, c)]).ljust(max_core_map_len),
- print ""
+ output += " " + str(core_map[(s, c)]).ljust(max_core_map_len)
+ print(output)
diff --git a/tools/dpdk-devbind.py b/tools/dpdk-devbind.py
index 4f51a4b..e057b87 100755
--- a/tools/dpdk-devbind.py
+++ b/tools/dpdk-devbind.py
@@ -1,4 +1,4 @@
-#! /usr/bin/python
+#! /usr/bin/env python
#
# BSD LICENSE
#
diff --git a/tools/dpdk-pmdinfo.py b/tools/dpdk-pmdinfo.py
index 3d3ad7d..d4e51aa 100755
--- a/tools/dpdk-pmdinfo.py
+++ b/tools/dpdk-pmdinfo.py
@@ -1,9 +1,11 @@
#!/usr/bin/env python
+
# -------------------------------------------------------------------------
#
# Utility to dump PMD_INFO_STRING support from an object file
#
# -------------------------------------------------------------------------
+from __future__ import print_function
import json
import os
import platform
@@ -54,7 +56,7 @@ def addDevice(self, deviceStr):
self.devices[devID] = Device(deviceStr)
def report(self):
- print self.ID, self.name
+ print(self.ID, self.name)
for id, dev in self.devices.items():
dev.report()
@@ -80,7 +82,7 @@ def __init__(self, deviceStr):
self.subdevices = {}
def report(self):
- print "\t%s\t%s" % (self.ID, self.name)
+ print("\t%s\t%s" % (self.ID, self.name))
for subID, subdev in self.subdevices.items():
subdev.report()
@@ -126,7 +128,7 @@ def __init__(self, vendor, device, name):
self.name = name
def report(self):
- print "\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name)
+ print("\t\t%s\t%s\t%s" % (self.vendorID, self.deviceID, self.name))
class PCIIds:
@@ -154,7 +156,7 @@ def reportVendors(self):
"""Reports the vendors
"""
for vid, v in self.vendors.items():
- print v.ID, v.name
+ print(v.ID, v.name)
def report(self, vendor=None):
"""
@@ -185,7 +187,7 @@ def findDate(self, content):
def parse(self):
if len(self.contents) < 1:
- print "data/%s-pci.ids not found" % self.date
+ print("data/%s-pci.ids not found" % self.date)
else:
vendorID = ""
deviceID = ""
@@ -432,7 +434,7 @@ def process_dt_needed_entries(self):
for tag in dynsec.iter_tags():
if tag.entry.d_tag == 'DT_NEEDED':
- rc = tag.needed.find("librte_pmd")
+ rc = tag.needed.find(b"librte_pmd")
if (rc != -1):
library = search_file(tag.needed,
runpath + ":" + ldlibpath +
--
2.7.4
^ permalink raw reply related
* [PATCH v3 1/3] app: make python apps pep8 compliant
From: John McNamara @ 2016-12-18 14:32 UTC (permalink / raw)
To: dev; +Cc: mkletzan, thomas.monjalon, nhorman, John McNamara
In-Reply-To: <1481212265-10229-1-git-send-email-john.mcnamara@intel.com>
Make all DPDK python application compliant with the PEP8 standard
to allow for consistency checking of patches and to allow further
refactoring.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
---
app/cmdline_test/cmdline_test.py | 81 +-
app/cmdline_test/cmdline_test_data.py | 401 +++++-----
app/test/autotest.py | 40 +-
app/test/autotest_data.py | 831 +++++++++++----------
app/test/autotest_runner.py | 739 +++++++++---------
app/test/autotest_test_funcs.py | 479 ++++++------
doc/guides/conf.py | 9 +-
examples/ip_pipeline/config/diagram-generator.py | 13 +-
.../ip_pipeline/config/pipeline-to-core-mapping.py | 11 +-
tools/cpu_layout.py | 55 +-
tools/dpdk-devbind.py | 23 +-
tools/dpdk-pmdinfo.py | 61 +-
12 files changed, 1376 insertions(+), 1367 deletions(-)
diff --git a/app/cmdline_test/cmdline_test.py b/app/cmdline_test/cmdline_test.py
index 8efc5ea..4729987 100755
--- a/app/cmdline_test/cmdline_test.py
+++ b/app/cmdline_test/cmdline_test.py
@@ -33,16 +33,21 @@
# Script that runs cmdline_test app and feeds keystrokes into it.
-import sys, pexpect, string, os, cmdline_test_data
+import cmdline_test_data
+import os
+import pexpect
+import sys
+
#
# function to run test
#
-def runTest(child,test):
- child.send(test["Sequence"])
- if test["Result"] == None:
- return 0
- child.expect(test["Result"],1)
+def runTest(child, test):
+ child.send(test["Sequence"])
+ if test["Result"] is None:
+ return 0
+ child.expect(test["Result"], 1)
+
#
# history test is a special case
@@ -57,57 +62,57 @@ def runTest(child,test):
# This is a self-contained test, it needs only a pexpect child
#
def runHistoryTest(child):
- # find out history size
- child.sendline(cmdline_test_data.CMD_GET_BUFSIZE)
- child.expect("History buffer size: \\d+", timeout=1)
- history_size = int(child.after[len(cmdline_test_data.BUFSIZE_TEMPLATE):])
- i = 0
+ # find out history size
+ child.sendline(cmdline_test_data.CMD_GET_BUFSIZE)
+ child.expect("History buffer size: \\d+", timeout=1)
+ history_size = int(child.after[len(cmdline_test_data.BUFSIZE_TEMPLATE):])
+ i = 0
- # fill the history with numbers
- while i < history_size / 10:
- # add 1 to prevent from parsing as octals
- child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
- # the app will simply print out the number
- child.expect(str(i + 100000000), timeout=1)
- i += 1
- # scroll back history
- child.send(cmdline_test_data.UP * (i + 2) + cmdline_test_data.ENTER)
- child.expect("100000000", timeout=1)
+ # fill the history with numbers
+ while i < history_size / 10:
+ # add 1 to prevent from parsing as octals
+ child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
+ # the app will simply print out the number
+ child.expect(str(i + 100000000), timeout=1)
+ i += 1
+ # scroll back history
+ child.send(cmdline_test_data.UP * (i + 2) + cmdline_test_data.ENTER)
+ child.expect("100000000", timeout=1)
# the path to cmdline_test executable is supplied via command-line.
if len(sys.argv) < 2:
- print "Error: please supply cmdline_test app path"
- sys.exit(1)
+ print "Error: please supply cmdline_test app path"
+ sys.exit(1)
test_app_path = sys.argv[1]
if not os.path.exists(test_app_path):
- print "Error: please supply cmdline_test app path"
- sys.exit(1)
+ print "Error: please supply cmdline_test app path"
+ sys.exit(1)
child = pexpect.spawn(test_app_path)
print "Running command-line tests..."
for test in cmdline_test_data.tests:
- print (test["Name"] + ":").ljust(30),
- try:
- runTest(child,test)
- print "PASS"
- except:
- print "FAIL"
- print child
- sys.exit(1)
+ print (test["Name"] + ":").ljust(30),
+ try:
+ runTest(child, test)
+ print "PASS"
+ except:
+ print "FAIL"
+ print child
+ sys.exit(1)
# since last test quits the app, run new instance
child = pexpect.spawn(test_app_path)
print ("History fill test:").ljust(30),
try:
- runHistoryTest(child)
- print "PASS"
+ runHistoryTest(child)
+ print "PASS"
except:
- print "FAIL"
- print child
- sys.exit(1)
+ print "FAIL"
+ print child
+ sys.exit(1)
child.close()
sys.exit(0)
diff --git a/app/cmdline_test/cmdline_test_data.py b/app/cmdline_test/cmdline_test_data.py
index b1945a5..3ce6cbc 100644
--- a/app/cmdline_test/cmdline_test_data.py
+++ b/app/cmdline_test/cmdline_test_data.py
@@ -33,8 +33,6 @@
# collection of static data
-import sys
-
# keycode constants
CTRL_A = chr(1)
CTRL_B = chr(2)
@@ -95,217 +93,220 @@
# and expected output (if any).
tests = [
-# test basic commands
- {"Name" : "command test 1",
- "Sequence" : "ambiguous first" + ENTER,
- "Result" : CMD1},
- {"Name" : "command test 2",
- "Sequence" : "ambiguous second" + ENTER,
- "Result" : CMD2},
- {"Name" : "command test 3",
- "Sequence" : "ambiguous ambiguous" + ENTER,
- "Result" : AMBIG},
- {"Name" : "command test 4",
- "Sequence" : "ambiguous ambiguous2" + ENTER,
- "Result" : AMBIG},
+ # test basic commands
+ {"Name": "command test 1",
+ "Sequence": "ambiguous first" + ENTER,
+ "Result": CMD1},
+ {"Name": "command test 2",
+ "Sequence": "ambiguous second" + ENTER,
+ "Result": CMD2},
+ {"Name": "command test 3",
+ "Sequence": "ambiguous ambiguous" + ENTER,
+ "Result": AMBIG},
+ {"Name": "command test 4",
+ "Sequence": "ambiguous ambiguous2" + ENTER,
+ "Result": AMBIG},
- {"Name" : "invalid command test 1",
- "Sequence" : "ambiguous invalid" + ENTER,
- "Result" : BAD_ARG},
-# test invalid commands
- {"Name" : "invalid command test 2",
- "Sequence" : "invalid" + ENTER,
- "Result" : NOT_FOUND},
- {"Name" : "invalid command test 3",
- "Sequence" : "ambiguousinvalid" + ENTER2,
- "Result" : NOT_FOUND},
+ {"Name": "invalid command test 1",
+ "Sequence": "ambiguous invalid" + ENTER,
+ "Result": BAD_ARG},
+ # test invalid commands
+ {"Name": "invalid command test 2",
+ "Sequence": "invalid" + ENTER,
+ "Result": NOT_FOUND},
+ {"Name": "invalid command test 3",
+ "Sequence": "ambiguousinvalid" + ENTER2,
+ "Result": NOT_FOUND},
-# test arrows and deletes
- {"Name" : "arrows & delete test 1",
- "Sequence" : "singlebad" + LEFT*2 + CTRL_B + DEL*3 + ENTER,
- "Result" : SINGLE},
- {"Name" : "arrows & delete test 2",
- "Sequence" : "singlebad" + LEFT*5 + RIGHT + CTRL_F + DEL*3 + ENTER,
- "Result" : SINGLE},
+ # test arrows and deletes
+ {"Name": "arrows & delete test 1",
+ "Sequence": "singlebad" + LEFT*2 + CTRL_B + DEL*3 + ENTER,
+ "Result": SINGLE},
+ {"Name": "arrows & delete test 2",
+ "Sequence": "singlebad" + LEFT*5 + RIGHT + CTRL_F + DEL*3 + ENTER,
+ "Result": SINGLE},
-# test backspace
- {"Name" : "backspace test",
- "Sequence" : "singlebad" + BKSPACE*3 + ENTER,
- "Result" : SINGLE},
+ # test backspace
+ {"Name": "backspace test",
+ "Sequence": "singlebad" + BKSPACE*3 + ENTER,
+ "Result": SINGLE},
-# test goto left and goto right
- {"Name" : "goto left test",
- "Sequence" : "biguous first" + CTRL_A + "am" + ENTER,
- "Result" : CMD1},
- {"Name" : "goto right test",
- "Sequence" : "biguous fir" + CTRL_A + "am" + CTRL_E + "st" + ENTER,
- "Result" : CMD1},
+ # test goto left and goto right
+ {"Name": "goto left test",
+ "Sequence": "biguous first" + CTRL_A + "am" + ENTER,
+ "Result": CMD1},
+ {"Name": "goto right test",
+ "Sequence": "biguous fir" + CTRL_A + "am" + CTRL_E + "st" + ENTER,
+ "Result": CMD1},
-# test goto words
- {"Name" : "goto left word test",
- "Sequence" : "ambiguous st" + ALT_B + "fir" + ENTER,
- "Result" : CMD1},
- {"Name" : "goto right word test",
- "Sequence" : "ambig first" + CTRL_A + ALT_F + "uous" + ENTER,
- "Result" : CMD1},
+ # test goto words
+ {"Name": "goto left word test",
+ "Sequence": "ambiguous st" + ALT_B + "fir" + ENTER,
+ "Result": CMD1},
+ {"Name": "goto right word test",
+ "Sequence": "ambig first" + CTRL_A + ALT_F + "uous" + ENTER,
+ "Result": CMD1},
-# test removing words
- {"Name" : "remove left word 1",
- "Sequence" : "single invalid" + CTRL_W + ENTER,
- "Result" : SINGLE},
- {"Name" : "remove left word 2",
- "Sequence" : "single invalid" + ALT_BKSPACE + ENTER,
- "Result" : SINGLE},
- {"Name" : "remove right word",
- "Sequence" : "single invalid" + ALT_B + ALT_D + ENTER,
- "Result" : SINGLE},
+ # test removing words
+ {"Name": "remove left word 1",
+ "Sequence": "single invalid" + CTRL_W + ENTER,
+ "Result": SINGLE},
+ {"Name": "remove left word 2",
+ "Sequence": "single invalid" + ALT_BKSPACE + ENTER,
+ "Result": SINGLE},
+ {"Name": "remove right word",
+ "Sequence": "single invalid" + ALT_B + ALT_D + ENTER,
+ "Result": SINGLE},
-# test kill buffer (copy and paste)
- {"Name" : "killbuffer test 1",
- "Sequence" : "ambiguous" + CTRL_A + CTRL_K + " first" + CTRL_A + CTRL_Y + ENTER,
- "Result" : CMD1},
- {"Name" : "killbuffer test 2",
- "Sequence" : "ambiguous" + CTRL_A + CTRL_K + CTRL_Y*26 + ENTER,
- "Result" : NOT_FOUND},
+ # test kill buffer (copy and paste)
+ {"Name": "killbuffer test 1",
+ "Sequence": "ambiguous" + CTRL_A + CTRL_K + " first" + CTRL_A +
+ CTRL_Y + ENTER,
+ "Result": CMD1},
+ {"Name": "killbuffer test 2",
+ "Sequence": "ambiguous" + CTRL_A + CTRL_K + CTRL_Y*26 + ENTER,
+ "Result": NOT_FOUND},
-# test newline
- {"Name" : "newline test",
- "Sequence" : "invalid" + CTRL_C + "single" + ENTER,
- "Result" : SINGLE},
+ # test newline
+ {"Name": "newline test",
+ "Sequence": "invalid" + CTRL_C + "single" + ENTER,
+ "Result": SINGLE},
-# test redisplay (nothing should really happen)
- {"Name" : "redisplay test",
- "Sequence" : "single" + CTRL_L + ENTER,
- "Result" : SINGLE},
+ # test redisplay (nothing should really happen)
+ {"Name": "redisplay test",
+ "Sequence": "single" + CTRL_L + ENTER,
+ "Result": SINGLE},
-# test autocomplete
- {"Name" : "autocomplete test 1",
- "Sequence" : "si" + TAB + ENTER,
- "Result" : SINGLE},
- {"Name" : "autocomplete test 2",
- "Sequence" : "si" + TAB + "_" + TAB + ENTER,
- "Result" : SINGLE_LONG},
- {"Name" : "autocomplete test 3",
- "Sequence" : "in" + TAB + ENTER,
- "Result" : NOT_FOUND},
- {"Name" : "autocomplete test 4",
- "Sequence" : "am" + TAB + ENTER,
- "Result" : BAD_ARG},
- {"Name" : "autocomplete test 5",
- "Sequence" : "am" + TAB + "fir" + TAB + ENTER,
- "Result" : CMD1},
- {"Name" : "autocomplete test 6",
- "Sequence" : "am" + TAB + "fir" + TAB + TAB + ENTER,
- "Result" : CMD1},
- {"Name" : "autocomplete test 7",
- "Sequence" : "am" + TAB + "fir" + TAB + " " + TAB + ENTER,
- "Result" : CMD1},
- {"Name" : "autocomplete test 8",
- "Sequence" : "am" + TAB + " am" + TAB + " " + ENTER,
- "Result" : AMBIG},
- {"Name" : "autocomplete test 9",
- "Sequence" : "am" + TAB + "inv" + TAB + ENTER,
- "Result" : BAD_ARG},
- {"Name" : "autocomplete test 10",
- "Sequence" : "au" + TAB + ENTER,
- "Result" : NOT_FOUND},
- {"Name" : "autocomplete test 11",
- "Sequence" : "au" + TAB + "1" + ENTER,
- "Result" : AUTO1},
- {"Name" : "autocomplete test 12",
- "Sequence" : "au" + TAB + "2" + ENTER,
- "Result" : AUTO2},
- {"Name" : "autocomplete test 13",
- "Sequence" : "au" + TAB + "2" + TAB + ENTER,
- "Result" : AUTO2},
- {"Name" : "autocomplete test 14",
- "Sequence" : "au" + TAB + "2 " + TAB + ENTER,
- "Result" : AUTO2},
- {"Name" : "autocomplete test 15",
- "Sequence" : "24" + TAB + ENTER,
- "Result" : "24"},
+ # test autocomplete
+ {"Name": "autocomplete test 1",
+ "Sequence": "si" + TAB + ENTER,
+ "Result": SINGLE},
+ {"Name": "autocomplete test 2",
+ "Sequence": "si" + TAB + "_" + TAB + ENTER,
+ "Result": SINGLE_LONG},
+ {"Name": "autocomplete test 3",
+ "Sequence": "in" + TAB + ENTER,
+ "Result": NOT_FOUND},
+ {"Name": "autocomplete test 4",
+ "Sequence": "am" + TAB + ENTER,
+ "Result": BAD_ARG},
+ {"Name": "autocomplete test 5",
+ "Sequence": "am" + TAB + "fir" + TAB + ENTER,
+ "Result": CMD1},
+ {"Name": "autocomplete test 6",
+ "Sequence": "am" + TAB + "fir" + TAB + TAB + ENTER,
+ "Result": CMD1},
+ {"Name": "autocomplete test 7",
+ "Sequence": "am" + TAB + "fir" + TAB + " " + TAB + ENTER,
+ "Result": CMD1},
+ {"Name": "autocomplete test 8",
+ "Sequence": "am" + TAB + " am" + TAB + " " + ENTER,
+ "Result": AMBIG},
+ {"Name": "autocomplete test 9",
+ "Sequence": "am" + TAB + "inv" + TAB + ENTER,
+ "Result": BAD_ARG},
+ {"Name": "autocomplete test 10",
+ "Sequence": "au" + TAB + ENTER,
+ "Result": NOT_FOUND},
+ {"Name": "autocomplete test 11",
+ "Sequence": "au" + TAB + "1" + ENTER,
+ "Result": AUTO1},
+ {"Name": "autocomplete test 12",
+ "Sequence": "au" + TAB + "2" + ENTER,
+ "Result": AUTO2},
+ {"Name": "autocomplete test 13",
+ "Sequence": "au" + TAB + "2" + TAB + ENTER,
+ "Result": AUTO2},
+ {"Name": "autocomplete test 14",
+ "Sequence": "au" + TAB + "2 " + TAB + ENTER,
+ "Result": AUTO2},
+ {"Name": "autocomplete test 15",
+ "Sequence": "24" + TAB + ENTER,
+ "Result": "24"},
-# test history
- {"Name" : "history test 1",
- "Sequence" : "invalid" + ENTER + "single" + ENTER + "invalid" + ENTER + UP + CTRL_P + ENTER,
- "Result" : SINGLE},
- {"Name" : "history test 2",
- "Sequence" : "invalid" + ENTER + "ambiguous first" + ENTER + "invalid" + ENTER + "single" + ENTER + UP * 3 + CTRL_N + DOWN + ENTER,
- "Result" : SINGLE},
+ # test history
+ {"Name": "history test 1",
+ "Sequence": "invalid" + ENTER + "single" + ENTER + "invalid" +
+ ENTER + UP + CTRL_P + ENTER,
+ "Result": SINGLE},
+ {"Name": "history test 2",
+ "Sequence": "invalid" + ENTER + "ambiguous first" + ENTER + "invalid" +
+ ENTER + "single" + ENTER + UP * 3 + CTRL_N + DOWN + ENTER,
+ "Result": SINGLE},
-#
-# tests that improve coverage
-#
+ #
+ # tests that improve coverage
+ #
-# empty space tests
- {"Name" : "empty space test 1",
- "Sequence" : RIGHT + LEFT + CTRL_B + CTRL_F + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 2",
- "Sequence" : BKSPACE + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 3",
- "Sequence" : CTRL_E*2 + CTRL_A*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 4",
- "Sequence" : ALT_F*2 + ALT_B*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 5",
- "Sequence" : " " + CTRL_E*2 + CTRL_A*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 6",
- "Sequence" : " " + CTRL_A + ALT_F*2 + ALT_B*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 7",
- "Sequence" : " " + CTRL_A + CTRL_D + CTRL_E + CTRL_D + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 8",
- "Sequence" : " space" + CTRL_W*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 9",
- "Sequence" : " space" + ALT_BKSPACE*2 + ENTER,
- "Result" : PROMPT},
- {"Name" : "empty space test 10",
- "Sequence" : " space " + CTRL_A + ALT_D*3 + ENTER,
- "Result" : PROMPT},
+ # empty space tests
+ {"Name": "empty space test 1",
+ "Sequence": RIGHT + LEFT + CTRL_B + CTRL_F + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 2",
+ "Sequence": BKSPACE + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 3",
+ "Sequence": CTRL_E*2 + CTRL_A*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 4",
+ "Sequence": ALT_F*2 + ALT_B*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 5",
+ "Sequence": " " + CTRL_E*2 + CTRL_A*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 6",
+ "Sequence": " " + CTRL_A + ALT_F*2 + ALT_B*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 7",
+ "Sequence": " " + CTRL_A + CTRL_D + CTRL_E + CTRL_D + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 8",
+ "Sequence": " space" + CTRL_W*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 9",
+ "Sequence": " space" + ALT_BKSPACE*2 + ENTER,
+ "Result": PROMPT},
+ {"Name": "empty space test 10",
+ "Sequence": " space " + CTRL_A + ALT_D*3 + ENTER,
+ "Result": PROMPT},
-# non-printable char tests
- {"Name" : "non-printable test 1",
- "Sequence" : chr(27) + chr(47) + ENTER,
- "Result" : PROMPT},
- {"Name" : "non-printable test 2",
- "Sequence" : chr(27) + chr(128) + ENTER*7,
- "Result" : PROMPT},
- {"Name" : "non-printable test 3",
- "Sequence" : chr(27) + chr(91) + chr(127) + ENTER*6,
- "Result" : PROMPT},
+ # non-printable char tests
+ {"Name": "non-printable test 1",
+ "Sequence": chr(27) + chr(47) + ENTER,
+ "Result": PROMPT},
+ {"Name": "non-printable test 2",
+ "Sequence": chr(27) + chr(128) + ENTER*7,
+ "Result": PROMPT},
+ {"Name": "non-printable test 3",
+ "Sequence": chr(27) + chr(91) + chr(127) + ENTER*6,
+ "Result": PROMPT},
-# miscellaneous tests
- {"Name" : "misc test 1",
- "Sequence" : ENTER,
- "Result" : PROMPT},
- {"Name" : "misc test 2",
- "Sequence" : "single #comment" + ENTER,
- "Result" : SINGLE},
- {"Name" : "misc test 3",
- "Sequence" : "#empty line" + ENTER,
- "Result" : PROMPT},
- {"Name" : "misc test 4",
- "Sequence" : " single " + ENTER,
- "Result" : SINGLE},
- {"Name" : "misc test 5",
- "Sequence" : "single#" + ENTER,
- "Result" : SINGLE},
- {"Name" : "misc test 6",
- "Sequence" : 'a' * 257 + ENTER,
- "Result" : NOT_FOUND},
- {"Name" : "misc test 7",
- "Sequence" : "clear_history" + UP*5 + DOWN*5 + ENTER,
- "Result" : PROMPT},
- {"Name" : "misc test 8",
- "Sequence" : "a" + HELP + CTRL_C,
- "Result" : PROMPT},
- {"Name" : "misc test 9",
- "Sequence" : CTRL_D*3,
- "Result" : None},
+ # miscellaneous tests
+ {"Name": "misc test 1",
+ "Sequence": ENTER,
+ "Result": PROMPT},
+ {"Name": "misc test 2",
+ "Sequence": "single #comment" + ENTER,
+ "Result": SINGLE},
+ {"Name": "misc test 3",
+ "Sequence": "#empty line" + ENTER,
+ "Result": PROMPT},
+ {"Name": "misc test 4",
+ "Sequence": " single " + ENTER,
+ "Result": SINGLE},
+ {"Name": "misc test 5",
+ "Sequence": "single#" + ENTER,
+ "Result": SINGLE},
+ {"Name": "misc test 6",
+ "Sequence": 'a' * 257 + ENTER,
+ "Result": NOT_FOUND},
+ {"Name": "misc test 7",
+ "Sequence": "clear_history" + UP*5 + DOWN*5 + ENTER,
+ "Result": PROMPT},
+ {"Name": "misc test 8",
+ "Sequence": "a" + HELP + CTRL_C,
+ "Result": PROMPT},
+ {"Name": "misc test 9",
+ "Sequence": CTRL_D*3,
+ "Result": None},
]
diff --git a/app/test/autotest.py b/app/test/autotest.py
index b9fd6b6..3a00538 100644
--- a/app/test/autotest.py
+++ b/app/test/autotest.py
@@ -33,44 +33,46 @@
# Script that uses either test app or qemu controlled by python-pexpect
-import sys, autotest_data, autotest_runner
-
+import autotest_data
+import autotest_runner
+import sys
def usage():
- print"Usage: autotest.py [test app|test iso image]",
- print "[target] [whitelist|-blacklist]"
+ print"Usage: autotest.py [test app|test iso image]",
+ print "[target] [whitelist|-blacklist]"
if len(sys.argv) < 3:
- usage()
- sys.exit(1)
+ usage()
+ sys.exit(1)
target = sys.argv[2]
-test_whitelist=None
-test_blacklist=None
+test_whitelist = None
+test_blacklist = None
# get blacklist/whitelist
if len(sys.argv) > 3:
- testlist = sys.argv[3].split(',')
- testlist = [test.lower() for test in testlist]
- if testlist[0].startswith('-'):
- testlist[0] = testlist[0].lstrip('-')
- test_blacklist = testlist
- else:
- test_whitelist = testlist
+ testlist = sys.argv[3].split(',')
+ testlist = [test.lower() for test in testlist]
+ if testlist[0].startswith('-'):
+ testlist[0] = testlist[0].lstrip('-')
+ test_blacklist = testlist
+ else:
+ test_whitelist = testlist
-cmdline = "%s -c f -n 4"%(sys.argv[1])
+cmdline = "%s -c f -n 4" % (sys.argv[1])
print cmdline
-runner = autotest_runner.AutotestRunner(cmdline, target, test_blacklist, test_whitelist)
+runner = autotest_runner.AutotestRunner(cmdline, target, test_blacklist,
+ test_whitelist)
for test_group in autotest_data.parallel_test_group_list:
- runner.add_parallel_test_group(test_group)
+ runner.add_parallel_test_group(test_group)
for test_group in autotest_data.non_parallel_test_group_list:
- runner.add_non_parallel_test_group(test_group)
+ runner.add_non_parallel_test_group(test_group)
num_fails = runner.run_all_tests()
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 9e8fd94..0cf4cfd 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
# BSD LICENSE
#
@@ -36,12 +36,14 @@
from glob import glob
from autotest_test_funcs import *
+
# quick and dirty function to find out number of sockets
def num_sockets():
- result = len(glob("/sys/devices/system/node/node*"))
- if result == 0:
- return 1
- return result
+ result = len(glob("/sys/devices/system/node/node*"))
+ if result == 0:
+ return 1
+ return result
+
# Assign given number to each socket
# e.g. 32 becomes 32,32 or 32,32,32,32
@@ -51,420 +53,419 @@ def per_sockets(num):
# groups of tests that can be run in parallel
# the grouping has been found largely empirically
parallel_test_group_list = [
-
-{
- "Prefix": "group_1",
- "Memory" : per_sockets(8),
- "Tests" :
- [
- {
- "Name" : "Cycles autotest",
- "Command" : "cycles_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Timer autotest",
- "Command" : "timer_autotest",
- "Func" : timer_autotest,
- "Report" : None,
- },
- {
- "Name" : "Debug autotest",
- "Command" : "debug_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Errno autotest",
- "Command" : "errno_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Meter autotest",
- "Command" : "meter_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Common autotest",
- "Command" : "common_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Resource autotest",
- "Command" : "resource_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "group_2",
- "Memory" : "16",
- "Tests" :
- [
- {
- "Name" : "Memory autotest",
- "Command" : "memory_autotest",
- "Func" : memory_autotest,
- "Report" : None,
- },
- {
- "Name" : "Read/write lock autotest",
- "Command" : "rwlock_autotest",
- "Func" : rwlock_autotest,
- "Report" : None,
- },
- {
- "Name" : "Logs autotest",
- "Command" : "logs_autotest",
- "Func" : logs_autotest,
- "Report" : None,
- },
- {
- "Name" : "CPU flags autotest",
- "Command" : "cpuflags_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Version autotest",
- "Command" : "version_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "EAL filesystem autotest",
- "Command" : "eal_fs_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "EAL flags autotest",
- "Command" : "eal_flags_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Hash autotest",
- "Command" : "hash_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ],
-},
-{
- "Prefix": "group_3",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "LPM autotest",
- "Command" : "lpm_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "LPM6 autotest",
- "Command" : "lpm6_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Memcpy autotest",
- "Command" : "memcpy_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Memzone autotest",
- "Command" : "memzone_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "String autotest",
- "Command" : "string_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Alarm autotest",
- "Command" : "alarm_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "group_4",
- "Memory" : per_sockets(128),
- "Tests" :
- [
- {
- "Name" : "PCI autotest",
- "Command" : "pci_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Malloc autotest",
- "Command" : "malloc_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Multi-process autotest",
- "Command" : "multiprocess_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Mbuf autotest",
- "Command" : "mbuf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Per-lcore autotest",
- "Command" : "per_lcore_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Ring autotest",
- "Command" : "ring_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "group_5",
- "Memory" : "32",
- "Tests" :
- [
- {
- "Name" : "Spinlock autotest",
- "Command" : "spinlock_autotest",
- "Func" : spinlock_autotest,
- "Report" : None,
- },
- {
- "Name" : "Byte order autotest",
- "Command" : "byteorder_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "TAILQ autotest",
- "Command" : "tailq_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Command-line autotest",
- "Command" : "cmdline_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Interrupts autotest",
- "Command" : "interrupt_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "group_6",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "Function reentrancy autotest",
- "Command" : "func_reentrancy_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Mempool autotest",
- "Command" : "mempool_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Atomics autotest",
- "Command" : "atomic_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Prefetch autotest",
- "Command" : "prefetch_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" :"Red autotest",
- "Command" : "red_autotest",
- "Func" :default_autotest,
- "Report" :None,
- },
- ]
-},
-{
- "Prefix" : "group_7",
- "Memory" : "64",
- "Tests" :
- [
- {
- "Name" : "PMD ring autotest",
- "Command" : "ring_pmd_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" : "Access list control autotest",
- "Command" : "acl_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- {
- "Name" :"Sched autotest",
- "Command" : "sched_autotest",
- "Func" :default_autotest,
- "Report" :None,
- },
- ]
-},
+ {
+ "Prefix": "group_1",
+ "Memory": per_sockets(8),
+ "Tests":
+ [
+ {
+ "Name": "Cycles autotest",
+ "Command": "cycles_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Timer autotest",
+ "Command": "timer_autotest",
+ "Func": timer_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Debug autotest",
+ "Command": "debug_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Errno autotest",
+ "Command": "errno_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Meter autotest",
+ "Command": "meter_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Common autotest",
+ "Command": "common_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Resource autotest",
+ "Command": "resource_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "group_2",
+ "Memory": "16",
+ "Tests":
+ [
+ {
+ "Name": "Memory autotest",
+ "Command": "memory_autotest",
+ "Func": memory_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Read/write lock autotest",
+ "Command": "rwlock_autotest",
+ "Func": rwlock_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Logs autotest",
+ "Command": "logs_autotest",
+ "Func": logs_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "CPU flags autotest",
+ "Command": "cpuflags_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Version autotest",
+ "Command": "version_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "EAL filesystem autotest",
+ "Command": "eal_fs_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "EAL flags autotest",
+ "Command": "eal_flags_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Hash autotest",
+ "Command": "hash_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ],
+ },
+ {
+ "Prefix": "group_3",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "LPM autotest",
+ "Command": "lpm_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "LPM6 autotest",
+ "Command": "lpm6_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Memcpy autotest",
+ "Command": "memcpy_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Memzone autotest",
+ "Command": "memzone_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "String autotest",
+ "Command": "string_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Alarm autotest",
+ "Command": "alarm_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "group_4",
+ "Memory": per_sockets(128),
+ "Tests":
+ [
+ {
+ "Name": "PCI autotest",
+ "Command": "pci_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Malloc autotest",
+ "Command": "malloc_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Multi-process autotest",
+ "Command": "multiprocess_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Mbuf autotest",
+ "Command": "mbuf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Per-lcore autotest",
+ "Command": "per_lcore_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Ring autotest",
+ "Command": "ring_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "group_5",
+ "Memory": "32",
+ "Tests":
+ [
+ {
+ "Name": "Spinlock autotest",
+ "Command": "spinlock_autotest",
+ "Func": spinlock_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Byte order autotest",
+ "Command": "byteorder_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "TAILQ autotest",
+ "Command": "tailq_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Command-line autotest",
+ "Command": "cmdline_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Interrupts autotest",
+ "Command": "interrupt_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "group_6",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "Function reentrancy autotest",
+ "Command": "func_reentrancy_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Mempool autotest",
+ "Command": "mempool_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Atomics autotest",
+ "Command": "atomic_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Prefetch autotest",
+ "Command": "prefetch_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Red autotest",
+ "Command": "red_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "group_7",
+ "Memory": "64",
+ "Tests":
+ [
+ {
+ "Name": "PMD ring autotest",
+ "Command": "ring_pmd_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Access list control autotest",
+ "Command": "acl_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ {
+ "Name": "Sched autotest",
+ "Command": "sched_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
]
# tests that should not be run when any other tests are running
non_parallel_test_group_list = [
-{
- "Prefix" : "kni",
- "Memory" : "512",
- "Tests" :
- [
- {
- "Name" : "KNI autotest",
- "Command" : "kni_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "mempool_perf",
- "Memory" : per_sockets(256),
- "Tests" :
- [
- {
- "Name" : "Mempool performance autotest",
- "Command" : "mempool_perf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "memcpy_perf",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "Memcpy performance autotest",
- "Command" : "memcpy_perf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "hash_perf",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "Hash performance autotest",
- "Command" : "hash_perf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix" : "power",
- "Memory" : "16",
- "Tests" :
- [
- {
- "Name" : "Power autotest",
- "Command" : "power_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix" : "power_acpi_cpufreq",
- "Memory" : "16",
- "Tests" :
- [
- {
- "Name" : "Power ACPI cpufreq autotest",
- "Command" : "power_acpi_cpufreq_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix" : "power_kvm_vm",
- "Memory" : "16",
- "Tests" :
- [
- {
- "Name" : "Power KVM VM autotest",
- "Command" : "power_kvm_vm_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
-{
- "Prefix": "timer_perf",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "Timer performance autotest",
- "Command" : "timer_perf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
+ {
+ "Prefix": "kni",
+ "Memory": "512",
+ "Tests":
+ [
+ {
+ "Name": "KNI autotest",
+ "Command": "kni_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "mempool_perf",
+ "Memory": per_sockets(256),
+ "Tests":
+ [
+ {
+ "Name": "Mempool performance autotest",
+ "Command": "mempool_perf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "memcpy_perf",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "Memcpy performance autotest",
+ "Command": "memcpy_perf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "hash_perf",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "Hash performance autotest",
+ "Command": "hash_perf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "power",
+ "Memory": "16",
+ "Tests":
+ [
+ {
+ "Name": "Power autotest",
+ "Command": "power_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "power_acpi_cpufreq",
+ "Memory": "16",
+ "Tests":
+ [
+ {
+ "Name": "Power ACPI cpufreq autotest",
+ "Command": "power_acpi_cpufreq_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "power_kvm_vm",
+ "Memory": "16",
+ "Tests":
+ [
+ {
+ "Name": "Power KVM VM autotest",
+ "Command": "power_kvm_vm_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
+ {
+ "Prefix": "timer_perf",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "Timer performance autotest",
+ "Command": "timer_perf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
-#
-# Please always make sure that ring_perf is the last test!
-#
-{
- "Prefix": "ring_perf",
- "Memory" : per_sockets(512),
- "Tests" :
- [
- {
- "Name" : "Ring performance autotest",
- "Command" : "ring_perf_autotest",
- "Func" : default_autotest,
- "Report" : None,
- },
- ]
-},
+ #
+ # Please always make sure that ring_perf is the last test!
+ #
+ {
+ "Prefix": "ring_perf",
+ "Memory": per_sockets(512),
+ "Tests":
+ [
+ {
+ "Name": "Ring performance autotest",
+ "Command": "ring_perf_autotest",
+ "Func": default_autotest,
+ "Report": None,
+ },
+ ]
+ },
]
diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index 21d3be2..55b63a8 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -33,20 +33,29 @@
# The main logic behind running autotests in parallel
-import multiprocessing, subprocess, sys, pexpect, re, time, os, StringIO, csv
+import StringIO
+import csv
+import multiprocessing
+import pexpect
+import re
+import subprocess
+import sys
+import time
# wait for prompt
+
+
def wait_prompt(child):
- try:
- child.sendline()
- result = child.expect(["RTE>>", pexpect.TIMEOUT, pexpect.EOF],
- timeout = 120)
- except:
- return False
- if result == 0:
- return True
- else:
- return False
+ try:
+ child.sendline()
+ result = child.expect(["RTE>>", pexpect.TIMEOUT, pexpect.EOF],
+ timeout=120)
+ except:
+ return False
+ if result == 0:
+ return True
+ else:
+ return False
# run a test group
# each result tuple in results list consists of:
@@ -60,363 +69,363 @@ def wait_prompt(child):
# this function needs to be outside AutotestRunner class
# because otherwise Pool won't work (or rather it will require
# quite a bit of effort to make it work).
-def run_test_group(cmdline, test_group):
- results = []
- child = None
- start_time = time.time()
- startuplog = None
-
- # run test app
- try:
- # prepare logging of init
- startuplog = StringIO.StringIO()
-
- print >>startuplog, "\n%s %s\n" % ("="*20, test_group["Prefix"])
- print >>startuplog, "\ncmdline=%s" % cmdline
-
- child = pexpect.spawn(cmdline, logfile=startuplog)
-
- # wait for target to boot
- if not wait_prompt(child):
- child.close()
-
- results.append((-1, "Fail [No prompt]", "Start %s" % test_group["Prefix"],
- time.time() - start_time, startuplog.getvalue(), None))
-
- # mark all tests as failed
- for test in test_group["Tests"]:
- results.append((-1, "Fail [No prompt]", test["Name"],
- time.time() - start_time, "", None))
- # exit test
- return results
-
- except:
- results.append((-1, "Fail [Can't run]", "Start %s" % test_group["Prefix"],
- time.time() - start_time, startuplog.getvalue(), None))
-
- # mark all tests as failed
- for t in test_group["Tests"]:
- results.append((-1, "Fail [Can't run]", t["Name"],
- time.time() - start_time, "", None))
- # exit test
- return results
-
- # startup was successful
- results.append((0, "Success", "Start %s" % test_group["Prefix"],
- time.time() - start_time, startuplog.getvalue(), None))
-
- # parse the binary for available test commands
- binary = cmdline.split()[0]
- stripped = 'not stripped' not in subprocess.check_output(['file', binary])
- if not stripped:
- symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
- avail_cmds = re.findall('test_register_(\w+)', symbols)
-
- # run all tests in test group
- for test in test_group["Tests"]:
-
- # create log buffer for each test
- # in multiprocessing environment, the logging would be
- # interleaved and will create a mess, hence the buffering
- logfile = StringIO.StringIO()
- child.logfile = logfile
-
- result = ()
-
- # make a note when the test started
- start_time = time.time()
-
- try:
- # print test name to log buffer
- print >>logfile, "\n%s %s\n" % ("-"*20, test["Name"])
-
- # run test function associated with the test
- if stripped or test["Command"] in avail_cmds:
- result = test["Func"](child, test["Command"])
- else:
- result = (0, "Skipped [Not Available]")
-
- # make a note when the test was finished
- end_time = time.time()
-
- # append test data to the result tuple
- result += (test["Name"], end_time - start_time,
- logfile.getvalue())
-
- # call report function, if any defined, and supply it with
- # target and complete log for test run
- if test["Report"]:
- report = test["Report"](self.target, log)
-
- # append report to results tuple
- result += (report,)
- else:
- # report is None
- result += (None,)
- except:
- # make a note when the test crashed
- end_time = time.time()
-
- # mark test as failed
- result = (-1, "Fail [Crash]", test["Name"],
- end_time - start_time, logfile.getvalue(), None)
- finally:
- # append the results to the results list
- results.append(result)
-
- # regardless of whether test has crashed, try quitting it
- try:
- child.sendline("quit")
- child.close()
- # if the test crashed, just do nothing instead
- except:
- # nop
- pass
-
- # return test results
- return results
-
+def run_test_group(cmdline, test_group):
+ results = []
+ child = None
+ start_time = time.time()
+ startuplog = None
+
+ # run test app
+ try:
+ # prepare logging of init
+ startuplog = StringIO.StringIO()
+
+ print >>startuplog, "\n%s %s\n" % ("=" * 20, test_group["Prefix"])
+ print >>startuplog, "\ncmdline=%s" % cmdline
+
+ child = pexpect.spawn(cmdline, logfile=startuplog)
+
+ # wait for target to boot
+ if not wait_prompt(child):
+ child.close()
+
+ results.append((-1,
+ "Fail [No prompt]",
+ "Start %s" % test_group["Prefix"],
+ time.time() - start_time,
+ startuplog.getvalue(),
+ None))
+
+ # mark all tests as failed
+ for test in test_group["Tests"]:
+ results.append((-1, "Fail [No prompt]", test["Name"],
+ time.time() - start_time, "", None))
+ # exit test
+ return results
+
+ except:
+ results.append((-1,
+ "Fail [Can't run]",
+ "Start %s" % test_group["Prefix"],
+ time.time() - start_time,
+ startuplog.getvalue(),
+ None))
+
+ # mark all tests as failed
+ for t in test_group["Tests"]:
+ results.append((-1, "Fail [Can't run]", t["Name"],
+ time.time() - start_time, "", None))
+ # exit test
+ return results
+
+ # startup was successful
+ results.append((0, "Success", "Start %s" % test_group["Prefix"],
+ time.time() - start_time, startuplog.getvalue(), None))
+
+ # parse the binary for available test commands
+ binary = cmdline.split()[0]
+ stripped = 'not stripped' not in subprocess.check_output(['file', binary])
+ if not stripped:
+ symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+ avail_cmds = re.findall('test_register_(\w+)', symbols)
+
+ # run all tests in test group
+ for test in test_group["Tests"]:
+
+ # create log buffer for each test
+ # in multiprocessing environment, the logging would be
+ # interleaved and will create a mess, hence the buffering
+ logfile = StringIO.StringIO()
+ child.logfile = logfile
+
+ result = ()
+
+ # make a note when the test started
+ start_time = time.time()
+
+ try:
+ # print test name to log buffer
+ print >>logfile, "\n%s %s\n" % ("-" * 20, test["Name"])
+
+ # run test function associated with the test
+ if stripped or test["Command"] in avail_cmds:
+ result = test["Func"](child, test["Command"])
+ else:
+ result = (0, "Skipped [Not Available]")
+
+ # make a note when the test was finished
+ end_time = time.time()
+
+ # append test data to the result tuple
+ result += (test["Name"], end_time - start_time,
+ logfile.getvalue())
+
+ # call report function, if any defined, and supply it with
+ # target and complete log for test run
+ if test["Report"]:
+ report = test["Report"](self.target, log)
+
+ # append report to results tuple
+ result += (report,)
+ else:
+ # report is None
+ result += (None,)
+ except:
+ # make a note when the test crashed
+ end_time = time.time()
+
+ # mark test as failed
+ result = (-1, "Fail [Crash]", test["Name"],
+ end_time - start_time, logfile.getvalue(), None)
+ finally:
+ # append the results to the results list
+ results.append(result)
+
+ # regardless of whether test has crashed, try quitting it
+ try:
+ child.sendline("quit")
+ child.close()
+ # if the test crashed, just do nothing instead
+ except:
+ # nop
+ pass
+
+ # return test results
+ return results
# class representing an instance of autotests run
class AutotestRunner:
- cmdline = ""
- parallel_test_groups = []
- non_parallel_test_groups = []
- logfile = None
- csvwriter = None
- target = ""
- start = None
- n_tests = 0
- fails = 0
- log_buffers = []
- blacklist = []
- whitelist = []
-
-
- def __init__(self, cmdline, target, blacklist, whitelist):
- self.cmdline = cmdline
- self.target = target
- self.blacklist = blacklist
- self.whitelist = whitelist
-
- # log file filename
- logfile = "%s.log" % target
- csvfile = "%s.csv" % target
-
- self.logfile = open(logfile, "w")
- csvfile = open(csvfile, "w")
- self.csvwriter = csv.writer(csvfile)
-
- # prepare results table
- self.csvwriter.writerow(["test_name","test_result","result_str"])
-
-
-
- # set up cmdline string
- def __get_cmdline(self, test):
- cmdline = self.cmdline
-
- # append memory limitations for each test
- # otherwise tests won't run in parallel
- if not "i686" in self.target:
- cmdline += " --socket-mem=%s"% test["Memory"]
- else:
- # affinitize startup so that tests don't fail on i686
- cmdline = "taskset 1 " + cmdline
- cmdline += " -m " + str(sum(map(int,test["Memory"].split(","))))
-
- # set group prefix for autotest group
- # otherwise they won't run in parallel
- cmdline += " --file-prefix=%s"% test["Prefix"]
-
- return cmdline
-
-
-
- def add_parallel_test_group(self,test_group):
- self.parallel_test_groups.append(test_group)
-
- def add_non_parallel_test_group(self,test_group):
- self.non_parallel_test_groups.append(test_group)
-
-
- def __process_results(self, results):
- # this iterates over individual test results
- for i, result in enumerate(results):
-
- # increase total number of tests that were run
- # do not include "start" test
- if i > 0:
- self.n_tests += 1
-
- # unpack result tuple
- test_result, result_str, test_name, \
- test_time, log, report = result
-
- # get total run time
- cur_time = time.time()
- total_time = int(cur_time - self.start)
-
- # print results, test run time and total time since start
- print ("%s:" % test_name).ljust(30),
- print result_str.ljust(29),
- print "[%02dm %02ds]" % (test_time / 60, test_time % 60),
-
- # don't print out total time every line, it's the same anyway
- if i == len(results) - 1:
- print "[%02dm %02ds]" % (total_time / 60, total_time % 60)
- else:
- print ""
-
- # if test failed and it wasn't a "start" test
- if test_result < 0 and not i == 0:
- self.fails += 1
-
- # collect logs
- self.log_buffers.append(log)
-
- # create report if it exists
- if report:
- try:
- f = open("%s_%s_report.rst" % (self.target,test_name), "w")
- except IOError:
- print "Report for %s could not be created!" % test_name
- else:
- with f:
- f.write(report)
-
- # write test result to CSV file
- if i != 0:
- self.csvwriter.writerow([test_name, test_result, result_str])
-
-
-
-
- # this function iterates over test groups and removes each
- # test that is not in whitelist/blacklist
- def __filter_groups(self, test_groups):
- groups_to_remove = []
-
- # filter out tests from parallel test groups
- for i, test_group in enumerate(test_groups):
-
- # iterate over a copy so that we could safely delete individual tests
- for test in test_group["Tests"][:]:
- test_id = test["Command"]
-
- # dump tests are specified in full e.g. "Dump_mempool"
- if "_autotest" in test_id:
- test_id = test_id[:-len("_autotest")]
-
- # filter out blacklisted/whitelisted tests
- if self.blacklist and test_id in self.blacklist:
- test_group["Tests"].remove(test)
- continue
- if self.whitelist and test_id not in self.whitelist:
- test_group["Tests"].remove(test)
- continue
-
- # modify or remove original group
- if len(test_group["Tests"]) > 0:
- test_groups[i] = test_group
- else:
- # remember which groups should be deleted
- # put the numbers backwards so that we start
- # deleting from the end, not from the beginning
- groups_to_remove.insert(0, i)
-
- # remove test groups that need to be removed
- for i in groups_to_remove:
- del test_groups[i]
-
- return test_groups
-
-
-
- # iterate over test groups and run tests associated with them
- def run_all_tests(self):
- # filter groups
- self.parallel_test_groups = \
- self.__filter_groups(self.parallel_test_groups)
- self.non_parallel_test_groups = \
- self.__filter_groups(self.non_parallel_test_groups)
-
- # create a pool of worker threads
- pool = multiprocessing.Pool(processes=1)
-
- results = []
-
- # whatever happens, try to save as much logs as possible
- try:
-
- # create table header
- print ""
- print "Test name".ljust(30),
- print "Test result".ljust(29),
- print "Test".center(9),
- print "Total".center(9)
- print "=" * 80
-
- # make a note of tests start time
- self.start = time.time()
-
- # assign worker threads to run test groups
- for test_group in self.parallel_test_groups:
- result = pool.apply_async(run_test_group,
- [self.__get_cmdline(test_group), test_group])
- results.append(result)
-
- # iterate while we have group execution results to get
- while len(results) > 0:
-
- # iterate over a copy to be able to safely delete results
- # this iterates over a list of group results
- for group_result in results[:]:
-
- # if the thread hasn't finished yet, continue
- if not group_result.ready():
- continue
-
- res = group_result.get()
-
- self.__process_results(res)
-
- # remove result from results list once we're done with it
- results.remove(group_result)
-
- # run non_parallel tests. they are run one by one, synchronously
- for test_group in self.non_parallel_test_groups:
- group_result = run_test_group(self.__get_cmdline(test_group), test_group)
-
- self.__process_results(group_result)
-
- # get total run time
- cur_time = time.time()
- total_time = int(cur_time - self.start)
-
- # print out summary
- print "=" * 80
- print "Total run time: %02dm %02ds" % (total_time / 60, total_time % 60)
- if self.fails != 0:
- print "Number of failed tests: %s" % str(self.fails)
-
- # write summary to logfile
- self.logfile.write("Summary\n")
- self.logfile.write("Target: ".ljust(15) + "%s\n" % self.target)
- self.logfile.write("Tests: ".ljust(15) + "%i\n" % self.n_tests)
- self.logfile.write("Failed tests: ".ljust(15) + "%i\n" % self.fails)
- except:
- print "Exception occured"
- print sys.exc_info()
- self.fails = 1
-
- # drop logs from all executions to a logfile
- for buf in self.log_buffers:
- self.logfile.write(buf.replace("\r",""))
-
- log_buffers = []
-
- return self.fails
+ cmdline = ""
+ parallel_test_groups = []
+ non_parallel_test_groups = []
+ logfile = None
+ csvwriter = None
+ target = ""
+ start = None
+ n_tests = 0
+ fails = 0
+ log_buffers = []
+ blacklist = []
+ whitelist = []
+
+ def __init__(self, cmdline, target, blacklist, whitelist):
+ self.cmdline = cmdline
+ self.target = target
+ self.blacklist = blacklist
+ self.whitelist = whitelist
+
+ # log file filename
+ logfile = "%s.log" % target
+ csvfile = "%s.csv" % target
+
+ self.logfile = open(logfile, "w")
+ csvfile = open(csvfile, "w")
+ self.csvwriter = csv.writer(csvfile)
+
+ # prepare results table
+ self.csvwriter.writerow(["test_name", "test_result", "result_str"])
+
+ # set up cmdline string
+ def __get_cmdline(self, test):
+ cmdline = self.cmdline
+
+ # append memory limitations for each test
+ # otherwise tests won't run in parallel
+ if "i686" not in self.target:
+ cmdline += " --socket-mem=%s" % test["Memory"]
+ else:
+ # affinitize startup so that tests don't fail on i686
+ cmdline = "taskset 1 " + cmdline
+ cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))
+
+ # set group prefix for autotest group
+ # otherwise they won't run in parallel
+ cmdline += " --file-prefix=%s" % test["Prefix"]
+
+ return cmdline
+
+ def add_parallel_test_group(self, test_group):
+ self.parallel_test_groups.append(test_group)
+
+ def add_non_parallel_test_group(self, test_group):
+ self.non_parallel_test_groups.append(test_group)
+
+ def __process_results(self, results):
+ # this iterates over individual test results
+ for i, result in enumerate(results):
+
+ # increase total number of tests that were run
+ # do not include "start" test
+ if i > 0:
+ self.n_tests += 1
+
+ # unpack result tuple
+ test_result, result_str, test_name, \
+ test_time, log, report = result
+
+ # get total run time
+ cur_time = time.time()
+ total_time = int(cur_time - self.start)
+
+ # print results, test run time and total time since start
+ print ("%s:" % test_name).ljust(30),
+ print result_str.ljust(29),
+ print "[%02dm %02ds]" % (test_time / 60, test_time % 60),
+
+ # don't print out total time every line, it's the same anyway
+ if i == len(results) - 1:
+ print "[%02dm %02ds]" % (total_time / 60, total_time % 60)
+ else:
+ print ""
+
+ # if test failed and it wasn't a "start" test
+ if test_result < 0 and not i == 0:
+ self.fails += 1
+
+ # collect logs
+ self.log_buffers.append(log)
+
+ # create report if it exists
+ if report:
+ try:
+ f = open("%s_%s_report.rst" %
+ (self.target, test_name), "w")
+ except IOError:
+ print "Report for %s could not be created!" % test_name
+ else:
+ with f:
+ f.write(report)
+
+ # write test result to CSV file
+ if i != 0:
+ self.csvwriter.writerow([test_name, test_result, result_str])
+
+ # this function iterates over test groups and removes each
+ # test that is not in whitelist/blacklist
+ def __filter_groups(self, test_groups):
+ groups_to_remove = []
+
+ # filter out tests from parallel test groups
+ for i, test_group in enumerate(test_groups):
+
+ # iterate over a copy so that we could safely delete individual
+ # tests
+ for test in test_group["Tests"][:]:
+ test_id = test["Command"]
+
+ # dump tests are specified in full e.g. "Dump_mempool"
+ if "_autotest" in test_id:
+ test_id = test_id[:-len("_autotest")]
+
+ # filter out blacklisted/whitelisted tests
+ if self.blacklist and test_id in self.blacklist:
+ test_group["Tests"].remove(test)
+ continue
+ if self.whitelist and test_id not in self.whitelist:
+ test_group["Tests"].remove(test)
+ continue
+
+ # modify or remove original group
+ if len(test_group["Tests"]) > 0:
+ test_groups[i] = test_group
+ else:
+ # remember which groups should be deleted
+ # put the numbers backwards so that we start
+ # deleting from the end, not from the beginning
+ groups_to_remove.insert(0, i)
+
+ # remove test groups that need to be removed
+ for i in groups_to_remove:
+ del test_groups[i]
+
+ return test_groups
+
+ # iterate over test groups and run tests associated with them
+ def run_all_tests(self):
+ # filter groups
+ self.parallel_test_groups = \
+ self.__filter_groups(self.parallel_test_groups)
+ self.non_parallel_test_groups = \
+ self.__filter_groups(self.non_parallel_test_groups)
+
+ # create a pool of worker threads
+ pool = multiprocessing.Pool(processes=1)
+
+ results = []
+
+ # whatever happens, try to save as much logs as possible
+ try:
+
+ # create table header
+ print ""
+ print "Test name".ljust(30),
+ print "Test result".ljust(29),
+ print "Test".center(9),
+ print "Total".center(9)
+ print "=" * 80
+
+ # make a note of tests start time
+ self.start = time.time()
+
+ # assign worker threads to run test groups
+ for test_group in self.parallel_test_groups:
+ result = pool.apply_async(run_test_group,
+ [self.__get_cmdline(test_group),
+ test_group])
+ results.append(result)
+
+ # iterate while we have group execution results to get
+ while len(results) > 0:
+
+ # iterate over a copy to be able to safely delete results
+ # this iterates over a list of group results
+ for group_result in results[:]:
+
+ # if the thread hasn't finished yet, continue
+ if not group_result.ready():
+ continue
+
+ res = group_result.get()
+
+ self.__process_results(res)
+
+ # remove result from results list once we're done with it
+ results.remove(group_result)
+
+ # run non_parallel tests. they are run one by one, synchronously
+ for test_group in self.non_parallel_test_groups:
+ group_result = run_test_group(
+ self.__get_cmdline(test_group), test_group)
+
+ self.__process_results(group_result)
+
+ # get total run time
+ cur_time = time.time()
+ total_time = int(cur_time - self.start)
+
+ # print out summary
+ print "=" * 80
+ print "Total run time: %02dm %02ds" % (total_time / 60,
+ total_time % 60)
+ if self.fails != 0:
+ print "Number of failed tests: %s" % str(self.fails)
+
+ # write summary to logfile
+ self.logfile.write("Summary\n")
+ self.logfile.write("Target: ".ljust(15) + "%s\n" % self.target)
+ self.logfile.write("Tests: ".ljust(15) + "%i\n" % self.n_tests)
+ self.logfile.write("Failed tests: ".ljust(
+ 15) + "%i\n" % self.fails)
+ except:
+ print "Exception occurred"
+ print sys.exc_info()
+ self.fails = 1
+
+ # drop logs from all executions to a logfile
+ for buf in self.log_buffers:
+ self.logfile.write(buf.replace("\r", ""))
+
+ return self.fails
diff --git a/app/test/autotest_test_funcs.py b/app/test/autotest_test_funcs.py
index 14cffd0..c482ea8 100644
--- a/app/test/autotest_test_funcs.py
+++ b/app/test/autotest_test_funcs.py
@@ -33,257 +33,272 @@
# Test functions
-import sys, pexpect, time, os, re
+import pexpect
# default autotest, used to run most tests
# waits for "Test OK"
+
+
def default_autotest(child, test_name):
- child.sendline(test_name)
- result = child.expect(["Test OK", "Test Failed",
- "Command not found", pexpect.TIMEOUT], timeout = 900)
- if result == 1:
- return -1, "Fail"
- elif result == 2:
- return -1, "Fail [Not found]"
- elif result == 3:
- return -1, "Fail [Timeout]"
- return 0, "Success"
+ child.sendline(test_name)
+ result = child.expect(["Test OK", "Test Failed",
+ "Command not found", pexpect.TIMEOUT], timeout=900)
+ if result == 1:
+ return -1, "Fail"
+ elif result == 2:
+ return -1, "Fail [Not found]"
+ elif result == 3:
+ return -1, "Fail [Timeout]"
+ return 0, "Success"
# autotest used to run dump commands
# just fires the command
+
+
def dump_autotest(child, test_name):
- child.sendline(test_name)
- return 0, "Success"
+ child.sendline(test_name)
+ return 0, "Success"
# memory autotest
# reads output and waits for Test OK
+
+
def memory_autotest(child, test_name):
- child.sendline(test_name)
- regexp = "phys:0x[0-9a-f]*, len:([0-9]*), virt:0x[0-9a-f]*, socket_id:[0-9]*"
- index = child.expect([regexp, pexpect.TIMEOUT], timeout = 180)
- if index != 0:
- return -1, "Fail [Timeout]"
- size = int(child.match.groups()[0], 16)
- if size <= 0:
- return -1, "Fail [Bad size]"
- index = child.expect(["Test OK", "Test Failed",
- pexpect.TIMEOUT], timeout = 10)
- if index == 1:
- return -1, "Fail"
- elif index == 2:
- return -1, "Fail [Timeout]"
- return 0, "Success"
+ child.sendline(test_name)
+ regexp = "phys:0x[0-9a-f]*, len:([0-9]*), virt:0x[0-9a-f]*, " \
+ "socket_id:[0-9]*"
+ index = child.expect([regexp, pexpect.TIMEOUT], timeout=180)
+ if index != 0:
+ return -1, "Fail [Timeout]"
+ size = int(child.match.groups()[0], 16)
+ if size <= 0:
+ return -1, "Fail [Bad size]"
+ index = child.expect(["Test OK", "Test Failed",
+ pexpect.TIMEOUT], timeout=10)
+ if index == 1:
+ return -1, "Fail"
+ elif index == 2:
+ return -1, "Fail [Timeout]"
+ return 0, "Success"
+
def spinlock_autotest(child, test_name):
- i = 0
- ir = 0
- child.sendline(test_name)
- while True:
- index = child.expect(["Test OK",
- "Test Failed",
- "Hello from core ([0-9]*) !",
- "Hello from within recursive locks from ([0-9]*) !",
- pexpect.TIMEOUT], timeout = 5)
- # ok
- if index == 0:
- break
-
- # message, check ordering
- elif index == 2:
- if int(child.match.groups()[0]) < i:
- return -1, "Fail [Bad order]"
- i = int(child.match.groups()[0])
- elif index == 3:
- if int(child.match.groups()[0]) < ir:
- return -1, "Fail [Bad order]"
- ir = int(child.match.groups()[0])
-
- # fail
- elif index == 4:
- return -1, "Fail [Timeout]"
- elif index == 1:
- return -1, "Fail"
-
- return 0, "Success"
+ i = 0
+ ir = 0
+ child.sendline(test_name)
+ while True:
+ index = child.expect(["Test OK",
+ "Test Failed",
+ "Hello from core ([0-9]*) !",
+ "Hello from within recursive locks "
+ "from ([0-9]*) !",
+ pexpect.TIMEOUT], timeout=5)
+ # ok
+ if index == 0:
+ break
+
+ # message, check ordering
+ elif index == 2:
+ if int(child.match.groups()[0]) < i:
+ return -1, "Fail [Bad order]"
+ i = int(child.match.groups()[0])
+ elif index == 3:
+ if int(child.match.groups()[0]) < ir:
+ return -1, "Fail [Bad order]"
+ ir = int(child.match.groups()[0])
+
+ # fail
+ elif index == 4:
+ return -1, "Fail [Timeout]"
+ elif index == 1:
+ return -1, "Fail"
+
+ return 0, "Success"
+
def rwlock_autotest(child, test_name):
- i = 0
- child.sendline(test_name)
- while True:
- index = child.expect(["Test OK",
- "Test Failed",
- "Hello from core ([0-9]*) !",
- "Global write lock taken on master core ([0-9]*)",
- pexpect.TIMEOUT], timeout = 10)
- # ok
- if index == 0:
- if i != 0xffff:
- return -1, "Fail [Message is missing]"
- break
-
- # message, check ordering
- elif index == 2:
- if int(child.match.groups()[0]) < i:
- return -1, "Fail [Bad order]"
- i = int(child.match.groups()[0])
-
- # must be the last message, check ordering
- elif index == 3:
- i = 0xffff
-
- elif index == 4:
- return -1, "Fail [Timeout]"
-
- # fail
- else:
- return -1, "Fail"
-
- return 0, "Success"
+ i = 0
+ child.sendline(test_name)
+ while True:
+ index = child.expect(["Test OK",
+ "Test Failed",
+ "Hello from core ([0-9]*) !",
+ "Global write lock taken on master "
+ "core ([0-9]*)",
+ pexpect.TIMEOUT], timeout=10)
+ # ok
+ if index == 0:
+ if i != 0xffff:
+ return -1, "Fail [Message is missing]"
+ break
+
+ # message, check ordering
+ elif index == 2:
+ if int(child.match.groups()[0]) < i:
+ return -1, "Fail [Bad order]"
+ i = int(child.match.groups()[0])
+
+ # must be the last message, check ordering
+ elif index == 3:
+ i = 0xffff
+
+ elif index == 4:
+ return -1, "Fail [Timeout]"
+
+ # fail
+ else:
+ return -1, "Fail"
+
+ return 0, "Success"
+
def logs_autotest(child, test_name):
- i = 0
- child.sendline(test_name)
-
- log_list = [
- "TESTAPP1: error message",
- "TESTAPP1: critical message",
- "TESTAPP2: critical message",
- "TESTAPP1: error message",
- ]
-
- for log_msg in log_list:
- index = child.expect([log_msg,
- "Test OK",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 10)
-
- if index == 3:
- return -1, "Fail [Timeout]"
- # not ok
- elif index != 0:
- return -1, "Fail"
-
- index = child.expect(["Test OK",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 10)
-
- return 0, "Success"
+ child.sendline(test_name)
+
+ log_list = [
+ "TESTAPP1: error message",
+ "TESTAPP1: critical message",
+ "TESTAPP2: critical message",
+ "TESTAPP1: error message",
+ ]
+
+ for log_msg in log_list:
+ index = child.expect([log_msg,
+ "Test OK",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=10)
+
+ if index == 3:
+ return -1, "Fail [Timeout]"
+ # not ok
+ elif index != 0:
+ return -1, "Fail"
+
+ index = child.expect(["Test OK",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=10)
+
+ return 0, "Success"
+
def timer_autotest(child, test_name):
- i = 0
- child.sendline(test_name)
-
- index = child.expect(["Start timer stress tests",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 5)
-
- if index == 1:
- return -1, "Fail"
- elif index == 2:
- return -1, "Fail [Timeout]"
-
- index = child.expect(["Start timer stress tests 2",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 5)
-
- if index == 1:
- return -1, "Fail"
- elif index == 2:
- return -1, "Fail [Timeout]"
-
- index = child.expect(["Start timer basic tests",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 5)
-
- if index == 1:
- return -1, "Fail"
- elif index == 2:
- return -1, "Fail [Timeout]"
-
- prev_lcore_timer1 = -1
-
- lcore_tim0 = -1
- lcore_tim1 = -1
- lcore_tim2 = -1
- lcore_tim3 = -1
-
- while True:
- index = child.expect(["TESTTIMER: ([0-9]*): callback id=([0-9]*) count=([0-9]*) on core ([0-9]*)",
- "Test OK",
- "Test Failed",
- pexpect.TIMEOUT], timeout = 10)
-
- if index == 1:
- break
-
- if index == 2:
- return -1, "Fail"
- elif index == 3:
- return -1, "Fail [Timeout]"
-
- try:
- t = int(child.match.groups()[0])
- id = int(child.match.groups()[1])
- cnt = int(child.match.groups()[2])
- lcore = int(child.match.groups()[3])
- except:
- return -1, "Fail [Cannot parse]"
-
- # timer0 always expires on the same core when cnt < 20
- if id == 0:
- if lcore_tim0 == -1:
- lcore_tim0 = lcore
- elif lcore != lcore_tim0 and cnt < 20:
- return -1, "Fail [lcore != lcore_tim0 (%d, %d)]"%(lcore, lcore_tim0)
- if cnt > 21:
- return -1, "Fail [tim0 cnt > 21]"
-
- # timer1 each time expires on a different core
- if id == 1:
- if lcore == lcore_tim1:
- return -1, "Fail [lcore == lcore_tim1 (%d, %d)]"%(lcore, lcore_tim1)
- lcore_tim1 = lcore
- if cnt > 10:
- return -1, "Fail [tim1 cnt > 30]"
-
- # timer0 always expires on the same core
- if id == 2:
- if lcore_tim2 == -1:
- lcore_tim2 = lcore
- elif lcore != lcore_tim2:
- return -1, "Fail [lcore != lcore_tim2 (%d, %d)]"%(lcore, lcore_tim2)
- if cnt > 30:
- return -1, "Fail [tim2 cnt > 30]"
-
- # timer0 always expires on the same core
- if id == 3:
- if lcore_tim3 == -1:
- lcore_tim3 = lcore
- elif lcore != lcore_tim3:
- return -1, "Fail [lcore_tim3 changed (%d -> %d)]"%(lcore, lcore_tim3)
- if cnt > 30:
- return -1, "Fail [tim3 cnt > 30]"
-
- # must be 2 different cores
- if lcore_tim0 == lcore_tim3:
- return -1, "Fail [lcore_tim0 (%d) == lcore_tim3 (%d)]"%(lcore_tim0, lcore_tim3)
-
- return 0, "Success"
+ child.sendline(test_name)
+
+ index = child.expect(["Start timer stress tests",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=5)
+
+ if index == 1:
+ return -1, "Fail"
+ elif index == 2:
+ return -1, "Fail [Timeout]"
+
+ index = child.expect(["Start timer stress tests 2",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=5)
+
+ if index == 1:
+ return -1, "Fail"
+ elif index == 2:
+ return -1, "Fail [Timeout]"
+
+ index = child.expect(["Start timer basic tests",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=5)
+
+ if index == 1:
+ return -1, "Fail"
+ elif index == 2:
+ return -1, "Fail [Timeout]"
+
+ lcore_tim0 = -1
+ lcore_tim1 = -1
+ lcore_tim2 = -1
+ lcore_tim3 = -1
+
+ while True:
+ index = child.expect(["TESTTIMER: ([0-9]*): callback id=([0-9]*) "
+ "count=([0-9]*) on core ([0-9]*)",
+ "Test OK",
+ "Test Failed",
+ pexpect.TIMEOUT], timeout=10)
+
+ if index == 1:
+ break
+
+ if index == 2:
+ return -1, "Fail"
+ elif index == 3:
+ return -1, "Fail [Timeout]"
+
+ try:
+ id = int(child.match.groups()[1])
+ cnt = int(child.match.groups()[2])
+ lcore = int(child.match.groups()[3])
+ except:
+ return -1, "Fail [Cannot parse]"
+
+ # timer0 always expires on the same core when cnt < 20
+ if id == 0:
+ if lcore_tim0 == -1:
+ lcore_tim0 = lcore
+ elif lcore != lcore_tim0 and cnt < 20:
+ return -1, "Fail [lcore != lcore_tim0 (%d, %d)]" \
+ % (lcore, lcore_tim0)
+ if cnt > 21:
+ return -1, "Fail [tim0 cnt > 21]"
+
+ # timer1 each time expires on a different core
+ if id == 1:
+ if lcore == lcore_tim1:
+ return -1, "Fail [lcore == lcore_tim1 (%d, %d)]" \
+ % (lcore, lcore_tim1)
+ lcore_tim1 = lcore
+ if cnt > 10:
+ return -1, "Fail [tim1 cnt > 30]"
+
+ # timer0 always expires on the same core
+ if id == 2:
+ if lcore_tim2 == -1:
+ lcore_tim2 = lcore
+ elif lcore != lcore_tim2:
+ return -1, "Fail [lcore != lcore_tim2 (%d, %d)]" \
+ % (lcore, lcore_tim2)
+ if cnt > 30:
+ return -1, "Fail [tim2 cnt > 30]"
+
+ # timer0 always expires on the same core
+ if id == 3:
+ if lcore_tim3 == -1:
+ lcore_tim3 = lcore
+ elif lcore != lcore_tim3:
+ return -1, "Fail [lcore_tim3 changed (%d -> %d)]" \
+ % (lcore, lcore_tim3)
+ if cnt > 30:
+ return -1, "Fail [tim3 cnt > 30]"
+
+ # must be 2 different cores
+ if lcore_tim0 == lcore_tim3:
+ return -1, "Fail [lcore_tim0 (%d) == lcore_tim3 (%d)]" \
+ % (lcore_tim0, lcore_tim3)
+
+ return 0, "Success"
+
def ring_autotest(child, test_name):
- child.sendline(test_name)
- index = child.expect(["Test OK", "Test Failed",
- pexpect.TIMEOUT], timeout = 2)
- if index == 1:
- return -1, "Fail"
- elif index == 2:
- return -1, "Fail [Timeout]"
-
- child.sendline("set_watermark test 100")
- child.sendline("dump_ring test")
- index = child.expect([" watermark=100",
- pexpect.TIMEOUT], timeout = 1)
- if index != 0:
- return -1, "Fail [Bad watermark]"
-
- return 0, "Success"
+ child.sendline(test_name)
+ index = child.expect(["Test OK", "Test Failed",
+ pexpect.TIMEOUT], timeout=2)
+ if index == 1:
+ return -1, "Fail"
+ elif index == 2:
+ return -1, "Fail [Timeout]"
+
+ child.sendline("set_watermark test 100")
+ child.sendline("dump_ring test")
+ index = child.expect([" watermark=100",
+ pexpect.TIMEOUT], timeout=1)
+ if index != 0:
+ return -1, "Fail [Bad watermark]"
+
+ return 0, "Success"
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 29e8efb..34c62de 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -58,7 +58,8 @@
html_show_copyright = False
highlight_language = 'none'
-version = subprocess.check_output(['make', '-sRrC', '../../', 'showversion']).decode('utf-8').rstrip()
+version = subprocess.check_output(['make', '-sRrC', '../../', 'showversion'])
+version = version.decode('utf-8').rstrip()
release = version
master_doc = 'index'
@@ -94,6 +95,7 @@
'preamble': latex_preamble
}
+
# Override the default Latex formatter in order to modify the
# code/verbatim blocks.
class CustomLatexFormatter(LatexFormatter):
@@ -117,12 +119,12 @@ def __init__(self, **options):
("tools/devbind", "dpdk-devbind",
"check device status and bind/unbind them from drivers", "", 8)]
-######## :numref: fallback ########
+
+# ####### :numref: fallback ########
# The following hook functions add some simple handling for the :numref:
# directive for Sphinx versions prior to 1.3.1. The functions replace the
# :numref: reference with a link to the target (for all Sphinx doc types).
# It doesn't try to label figures/tables.
-
def numref_role(reftype, rawtext, text, lineno, inliner):
"""
Add a Sphinx role to handle numref references. Note, we can't convert
@@ -136,6 +138,7 @@ def numref_role(reftype, rawtext, text, lineno, inliner):
internal=True)
return [newnode], []
+
def process_numref(app, doctree, from_docname):
"""
Process the numref nodes once the doctree has been built and prior to
diff --git a/examples/ip_pipeline/config/diagram-generator.py b/examples/ip_pipeline/config/diagram-generator.py
index 6b7170b..1748833 100755
--- a/examples/ip_pipeline/config/diagram-generator.py
+++ b/examples/ip_pipeline/config/diagram-generator.py
@@ -36,7 +36,8 @@
# the DPDK ip_pipeline application.
#
# The input configuration file is translated to an output file in DOT syntax,
-# which is then used to create the image file using graphviz (www.graphviz.org).
+# which is then used to create the image file using graphviz
+# (www.graphviz.org).
#
from __future__ import print_function
@@ -94,6 +95,7 @@
# SOURCEx | SOURCEx | SOURCEx | PIPELINEy | SOURCEx
# SINKx | SINKx | PIPELINEy | SINKx | SINKx
+
#
# Parse the input configuration file to detect the graph nodes and edges
#
@@ -321,16 +323,17 @@ def process_config_file(cfgfile):
#
print('Creating image file "%s" ...' % imgfile)
if os.system('which dot > /dev/null'):
- print('Error: Unable to locate "dot" executable.' \
- 'Please install the "graphviz" package (www.graphviz.org).')
+ print('Error: Unable to locate "dot" executable.'
+ 'Please install the "graphviz" package (www.graphviz.org).')
return
os.system(dot_cmd)
if __name__ == '__main__':
- parser = argparse.ArgumentParser(description=\
- 'Create diagram for IP pipeline configuration file.')
+ parser = argparse.ArgumentParser(description='Create diagram for IP '
+ 'pipeline configuration '
+ 'file.')
parser.add_argument(
'-f',
diff --git a/examples/ip_pipeline/config/pipeline-to-core-mapping.py b/examples/ip_pipeline/config/pipeline-to-core-mapping.py
index c2050b8..7a4eaa2 100755
--- a/examples/ip_pipeline/config/pipeline-to-core-mapping.py
+++ b/examples/ip_pipeline/config/pipeline-to-core-mapping.py
@@ -39,15 +39,14 @@
#
from __future__ import print_function
-import sys
-import errno
-import os
-import re
+from collections import namedtuple
+import argparse
import array
+import errno
import itertools
+import os
import re
-import argparse
-from collections import namedtuple
+import sys
# default values
enable_stage0_traceout = 1
diff --git a/tools/cpu_layout.py b/tools/cpu_layout.py
index d38d0b5..ccc22ec 100755
--- a/tools/cpu_layout.py
+++ b/tools/cpu_layout.py
@@ -38,40 +38,40 @@
cores = []
core_map = {}
-fd=open("/proc/cpuinfo")
+fd = open("/proc/cpuinfo")
lines = fd.readlines()
fd.close()
core_details = []
core_lines = {}
for line in lines:
- if len(line.strip()) != 0:
- name, value = line.split(":", 1)
- core_lines[name.strip()] = value.strip()
- else:
- core_details.append(core_lines)
- core_lines = {}
+ if len(line.strip()) != 0:
+ name, value = line.split(":", 1)
+ core_lines[name.strip()] = value.strip()
+ else:
+ core_details.append(core_lines)
+ core_lines = {}
for core in core_details:
- for field in ["processor", "core id", "physical id"]:
- if field not in core:
- print "Error getting '%s' value from /proc/cpuinfo" % field
- sys.exit(1)
- core[field] = int(core[field])
+ for field in ["processor", "core id", "physical id"]:
+ if field not in core:
+ print "Error getting '%s' value from /proc/cpuinfo" % field
+ sys.exit(1)
+ core[field] = int(core[field])
- if core["core id"] not in cores:
- cores.append(core["core id"])
- if core["physical id"] not in sockets:
- sockets.append(core["physical id"])
- key = (core["physical id"], core["core id"])
- if key not in core_map:
- core_map[key] = []
- core_map[key].append(core["processor"])
+ if core["core id"] not in cores:
+ cores.append(core["core id"])
+ if core["physical id"] not in sockets:
+ sockets.append(core["physical id"])
+ key = (core["physical id"], core["core id"])
+ if key not in core_map:
+ core_map[key] = []
+ core_map[key].append(core["processor"])
print "============================================================"
print "Core and Socket Information (as reported by '/proc/cpuinfo')"
print "============================================================\n"
-print "cores = ",cores
+print "cores = ", cores
print "sockets = ", sockets
print ""
@@ -81,15 +81,16 @@
print " ".ljust(max_core_id_len + len('Core ')),
for s in sockets:
- print "Socket %s" % str(s).ljust(max_core_map_len - len('Socket ')),
+ print "Socket %s" % str(s).ljust(max_core_map_len - len('Socket ')),
print ""
+
print " ".ljust(max_core_id_len + len('Core ')),
for s in sockets:
- print "--------".ljust(max_core_map_len),
+ print "--------".ljust(max_core_map_len),
print ""
for c in cores:
- print "Core %s" % str(c).ljust(max_core_id_len),
- for s in sockets:
- print str(core_map[(s,c)]).ljust(max_core_map_len),
- print ""
+ print "Core %s" % str(c).ljust(max_core_id_len),
+ for s in sockets:
+ print str(core_map[(s, c)]).ljust(max_core_map_len),
+ print ""
diff --git a/tools/dpdk-devbind.py b/tools/dpdk-devbind.py
index f1d374d..4f51a4b 100755
--- a/tools/dpdk-devbind.py
+++ b/tools/dpdk-devbind.py
@@ -93,10 +93,10 @@ def usage():
Unbind a device (Equivalent to \"-b none\")
--force:
- By default, network devices which are used by Linux - as indicated by having
- routes in the routing table - cannot be modified. Using the --force
- flag overrides this behavior, allowing active links to be forcibly
- unbound.
+ By default, network devices which are used by Linux - as indicated by
+ having routes in the routing table - cannot be modified. Using the
+ --force flag overrides this behavior, allowing active links to be
+ forcibly unbound.
WARNING: This can lead to loss of network connection and should be used
with caution.
@@ -151,7 +151,7 @@ def find_module(mod):
# check for a copy based off current path
tools_dir = dirname(abspath(sys.argv[0]))
- if (tools_dir.endswith("tools")):
+ if tools_dir.endswith("tools"):
base_dir = dirname(tools_dir)
find_out = check_output(["find", base_dir, "-name", mod + ".ko"])
if len(find_out) > 0: # something matched
@@ -249,7 +249,7 @@ def get_nic_details():
dev = {}
dev_lines = check_output(["lspci", "-Dvmmn"]).splitlines()
for dev_line in dev_lines:
- if (len(dev_line) == 0):
+ if len(dev_line) == 0:
if dev["Class"][0:2] == NETWORK_BASE_CLASS:
# convert device and vendor ids to numbers, then add to global
dev["Vendor"] = int(dev["Vendor"], 16)
@@ -315,8 +315,8 @@ def get_crypto_details():
dev = {}
dev_lines = check_output(["lspci", "-Dvmmn"]).splitlines()
for dev_line in dev_lines:
- if (len(dev_line) == 0):
- if (dev["Class"][0:2] == CRYPTO_BASE_CLASS):
+ if len(dev_line) == 0:
+ if dev["Class"][0:2] == CRYPTO_BASE_CLASS:
# convert device and vendor ids to numbers, then add to global
dev["Vendor"] = int(dev["Vendor"], 16)
dev["Device"] = int(dev["Device"], 16)
@@ -513,7 +513,8 @@ def display_devices(title, dev_list, extra_params=None):
for dev in dev_list:
if extra_params is not None:
strings.append("%s '%s' %s" % (dev["Slot"],
- dev["Device_str"], extra_params % dev))
+ dev["Device_str"],
+ extra_params % dev))
else:
strings.append("%s '%s'" % (dev["Slot"], dev["Device_str"]))
# sort before printing, so that the entries appear in PCI order
@@ -532,7 +533,7 @@ def show_status():
# split our list of network devices into the three categories above
for d in devices.keys():
- if (NETWORK_BASE_CLASS in devices[d]["Class"]):
+ if NETWORK_BASE_CLASS in devices[d]["Class"]:
if not has_driver(d):
no_drv.append(devices[d])
continue
@@ -555,7 +556,7 @@ def show_status():
no_drv = []
for d in devices.keys():
- if (CRYPTO_BASE_CLASS in devices[d]["Class"]):
+ if CRYPTO_BASE_CLASS in devices[d]["Class"]:
if not has_driver(d):
no_drv.append(devices[d])
continue
diff --git a/tools/dpdk-pmdinfo.py b/tools/dpdk-pmdinfo.py
index 3db9819..3d3ad7d 100755
--- a/tools/dpdk-pmdinfo.py
+++ b/tools/dpdk-pmdinfo.py
@@ -4,52 +4,20 @@
# Utility to dump PMD_INFO_STRING support from an object file
#
# -------------------------------------------------------------------------
+import json
import os
+import platform
+import string
import sys
+from elftools.common.exceptions import ELFError
+from elftools.common.py3compat import (byte2int, bytes2str, str2bytes)
+from elftools.elf.elffile import ELFFile
from optparse import OptionParser
-import string
-import json
-import platform
# For running from development directory. It should take precedence over the
# installed pyelftools.
sys.path.insert(0, '.')
-
-from elftools import __version__
-from elftools.common.exceptions import ELFError
-from elftools.common.py3compat import (
- ifilter, byte2int, bytes2str, itervalues, str2bytes)
-from elftools.elf.elffile import ELFFile
-from elftools.elf.dynamic import DynamicSection, DynamicSegment
-from elftools.elf.enums import ENUM_D_TAG
-from elftools.elf.segments import InterpSegment
-from elftools.elf.sections import SymbolTableSection
-from elftools.elf.gnuversions import (
- GNUVerSymSection, GNUVerDefSection,
- GNUVerNeedSection,
-)
-from elftools.elf.relocation import RelocationSection
-from elftools.elf.descriptions import (
- describe_ei_class, describe_ei_data, describe_ei_version,
- describe_ei_osabi, describe_e_type, describe_e_machine,
- describe_e_version_numeric, describe_p_type, describe_p_flags,
- describe_sh_type, describe_sh_flags,
- describe_symbol_type, describe_symbol_bind, describe_symbol_visibility,
- describe_symbol_shndx, describe_reloc_type, describe_dyn_tag,
- describe_ver_flags,
-)
-from elftools.elf.constants import E_FLAGS
-from elftools.dwarf.dwarfinfo import DWARFInfo
-from elftools.dwarf.descriptions import (
- describe_reg_name, describe_attr_value, set_global_machine_arch,
- describe_CFI_instructions, describe_CFI_register_rule,
- describe_CFI_CFA_rule,
-)
-from elftools.dwarf.constants import (
- DW_LNS_copy, DW_LNS_set_file, DW_LNE_define_file)
-from elftools.dwarf.callframe import CIE, FDE
-
raw_output = False
pcidb = None
@@ -326,7 +294,7 @@ def parse_pmd_info_string(self, mystring):
for i in optional_pmd_info:
try:
print("%s: %s" % (i['tag'], pmdinfo[i['id']]))
- except KeyError as e:
+ except KeyError:
continue
if (len(pmdinfo["pci_ids"]) != 0):
@@ -475,7 +443,7 @@ def process_dt_needed_entries(self):
with open(library, 'rb') as file:
try:
libelf = ReadElf(file, sys.stdout)
- except ELFError as e:
+ except ELFError:
print("%s is no an ELF file" % library)
continue
libelf.process_dt_needed_entries()
@@ -491,7 +459,7 @@ def scan_autoload_path(autoload_path):
try:
dirs = os.listdir(autoload_path)
- except OSError as e:
+ except OSError:
# Couldn't read the directory, give up
return
@@ -503,10 +471,10 @@ def scan_autoload_path(autoload_path):
try:
file = open(dpath, 'rb')
readelf = ReadElf(file, sys.stdout)
- except ELFError as e:
+ except ELFError:
# this is likely not an elf file, skip it
continue
- except IOError as e:
+ except IOError:
# No permission to read the file, skip it
continue
@@ -531,7 +499,7 @@ def scan_for_autoload_pmds(dpdk_path):
file = open(dpdk_path, 'rb')
try:
readelf = ReadElf(file, sys.stdout)
- except ElfError as e:
+ except ElfError:
if raw_output is False:
print("Unable to parse %s" % file)
return
@@ -557,7 +525,7 @@ def main(stream=None):
global raw_output
global pcidb
- pcifile_default = "./pci.ids" # for unknown OS's assume local file
+ pcifile_default = "./pci.ids" # For unknown OS's assume local file
if platform.system() == 'Linux':
pcifile_default = "/usr/share/hwdata/pci.ids"
elif platform.system() == 'FreeBSD':
@@ -577,7 +545,8 @@ def main(stream=None):
"to get vendor names from",
default=pcifile_default, metavar="FILE")
optparser.add_option("-t", "--table", dest="tblout",
- help="output information on hw support as a hex table",
+ help="output information on hw support as a "
+ "hex table",
action='store_true')
optparser.add_option("-p", "--plugindir", dest="pdir",
help="scan dpdk for autoload plugins",
--
2.7.4
^ permalink raw reply related
* [PATCH v3 0/3] app: make python apps python2/3 compliant
From: John McNamara @ 2016-12-18 14:32 UTC (permalink / raw)
To: dev; +Cc: mkletzan, thomas.monjalon, nhorman, John McNamara
In-Reply-To: <1481212265-10229-1-git-send-email-john.mcnamara@intel.com>
These patches refactor the DPDK Python applications to make them Python 2/3
compatible.
In order to do this the patchset starts by making the apps PEP8 compliant in
accordance with the DPDK Coding guidelines:
http://dpdk.org/doc/guides/contributing/coding_style.html#python-code
Implementing PEP8 and Python 2/3 compliance means that we can check all future
Python patches for consistency. Python 2/3 support also makes downstream
packaging easier as more distros move to Python 3 as the system python.
See the previous discussion about Python2/3 compatibilty here:
http://dpdk.org/ml/archives/dev/2016-December/051683.html
V3: * Squash shebang patch into Python 3 patch.
* Only add /usr/bin/env shebang line to code that is executable.
V2: * Fix broken rebase.
John McNamara (3):
app: make python apps pep8 compliant
app: make python apps python2/3 compliant
doc: add required python versions to docs
app/cmdline_test/cmdline_test.py | 87 ++-
app/cmdline_test/cmdline_test_data.py | 403 +++++-----
app/test/autotest.py | 46 +-
app/test/autotest_data.py | 831 ++++++++++-----------
app/test/autotest_runner.py | 740 +++++++++---------
app/test/autotest_test_funcs.py | 481 ++++++------
doc/guides/conf.py | 9 +-
doc/guides/contributing/coding_style.rst | 3 +-
doc/guides/linux_gsg/sys_reqs.rst | 2 +-
examples/ip_pipeline/config/diagram-generator.py | 13 +-
.../ip_pipeline/config/pipeline-to-core-mapping.py | 11 +-
tools/cpu_layout.py | 79 +-
tools/dpdk-devbind.py | 25 +-
tools/dpdk-pmdinfo.py | 75 +-
14 files changed, 1405 insertions(+), 1400 deletions(-)
--
2.7.4
^ permalink raw reply
* [PATCH v2 3/3] doc: add required python versions to docs
From: John McNamara @ 2016-12-18 14:25 UTC (permalink / raw)
To: dev; +Cc: mkletzan, thomas.monjalon, nhorman, John McNamara
In-Reply-To: <1481212265-10229-1-git-send-email-john.mcnamara@intel.com>
Add a requirement to support both Python 2 and 3 to the
DPDK Python Coding Standards and Getting started Guide.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
---
doc/guides/contributing/coding_style.rst | 3 ++-
doc/guides/linux_gsg/sys_reqs.rst | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/doc/guides/contributing/coding_style.rst b/doc/guides/contributing/coding_style.rst
index 1eb67f3..4163960 100644
--- a/doc/guides/contributing/coding_style.rst
+++ b/doc/guides/contributing/coding_style.rst
@@ -690,6 +690,7 @@ Control Statements
Python Code
-----------
-All python code should be compliant with `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
+All Python code should work with Python 2.7+ and 3.2+ and be compliant with
+`PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
The ``pep8`` tool can be used for testing compliance with the guidelines.
diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index 3d74342..9653a13 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -86,7 +86,7 @@ Compilation of the DPDK
.. note::
- Python, version 2.6 or 2.7, to use various helper scripts included in the DPDK package.
+ Python, version 2.7+ or 3.2+, to use various helper scripts included in the DPDK package.
**Optional Tools:**
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox