Re: Can not init NIC after merge to DPDK 1.7 problem

Re: Can not init NIC after merge to DPDK 1.7 problem

[Sanford, Robert]

We ran into a similar problem when migrating to 1.7.
Here are the subtle flags, in dpdk/mk/rte.app.mk, that we needed:

	LDLIBS += --whole-archive
	...
	LDLIBS += --no-whole-archive

This apparently tells the linker to pull in whole archive(s), even if it
thinks that we don't need all objects.




--
Regards,
Robert


>Hi:
>
>We are using our own Makefile in building dpdk program. Recently we are
>working on upgrading from DPDK 1.3 to DPDK 1.7. I found the
>rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete
>rte_ixgbe_pmd_init calls. But after that, our dpdk program could not
>correctly find the NIC anymore. After digging into it a little more, I
>found the code dose not correctly register the driver type we are using,
>which is ixgbe.
>To isolate the problem, I hacked a smal example l3fwd, and only have the
>main.c file like this for my testing purpose.
>
>#include <rte_config.h>
>#include <rte_eal.h>
>
>#include "main.h"
>
>int
>MAIN(int argc, char **argv)
>{
>        /* init EAL */
>        int ret = rte_eal_init(argc, argv);
>        printf("ret %d\n", ret);
>        return 0;
>}
>
>I found if I use the Makefile provided in the example, the program will
>find the ixgbe NIC. But if I just use these 2 commands to compile and
>link it. It will not find the ixgbe NIC.
>
>gcc -I../../x86_64-native-linuxapp-gcc/include
>-L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
>gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal
>-lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash
>-lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool
>-lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond
>-lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt
>
>Can someone share some light on what is magic of the dpdk Makefile to
>correctly register the NIC type?
>
>Thank you so much.
>Xingbo Wang

Re: Can not init NIC after merge to DPDK 1.7 problem

[Wang, Shawn]

This does resolve the problem.

Thank you so much.

________________________________________
From: Sanford, Robert [rsanford-JqFfY2XvxFXQT0dZR+AlfA@public.gmane.org]
Sent: Tuesday, September 23, 2014 2:50 PM
To: Wang, Shawn; dev-VfR2kkLFssw@public.gmane.org
Subject: Re: [dpdk-dev] Can not init NIC after merge to DPDK 1.7 problem

We ran into a similar problem when migrating to 1.7.
Here are the subtle flags, in dpdk/mk/rte.app.mk, that we needed:

        LDLIBS += --whole-archive
        ...
        LDLIBS += --no-whole-archive

This apparently tells the linker to pull in whole archive(s), even if it
thinks that we don't need all objects.




--
Regards,
Robert


>Hi:
>
>We are using our own Makefile in building dpdk program. Recently we are
>working on upgrading from DPDK 1.3 to DPDK 1.7. I found the
>rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete
>rte_ixgbe_pmd_init calls. But after that, our dpdk program could not
>correctly find the NIC anymore. After digging into it a little more, I
>found the code dose not correctly register the driver type we are using,
>which is ixgbe.
>To isolate the problem, I hacked a smal example l3fwd, and only have the
>main.c file like this for my testing purpose.
>
>#include <rte_config.h>
>#include <rte_eal.h>
>
>#include "main.h"
>
>int
>MAIN(int argc, char **argv)
>{
>        /* init EAL */
>        int ret = rte_eal_init(argc, argv);
>        printf("ret %d\n", ret);
>        return 0;
>}
>
>I found if I use the Makefile provided in the example, the program will
>find the ixgbe NIC. But if I just use these 2 commands to compile and
>link it. It will not find the ixgbe NIC.
>
>gcc -I../../x86_64-native-linuxapp-gcc/include
>-L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
>gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal
>-lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash
>-lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool
>-lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond
>-lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt
>
>Can someone share some light on what is magic of the dpdk Makefile to
>correctly register the NIC type?
>
>Thank you so much.
>Xingbo Wang

Can not init NIC after merge to DPDK 1.7 problem

[Wang, Shawn]

Hi:

We are using our own Makefile in building dpdk program. Recently we are working on upgrading from DPDK 1.3 to DPDK 1.7. I found the rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete rte_ixgbe_pmd_init calls. But after that, our dpdk program could not correctly find the NIC anymore. After digging into it a little more, I found the code dose not correctly register the driver type we are using, which is ixgbe.
To isolate the problem, I hacked a smal example l3fwd, and only have the main.c file like this for my testing purpose.

#include <rte_config.h>
#include <rte_eal.h>

#include "main.h"

int
MAIN(int argc, char **argv)
{
        /* init EAL */
        int ret = rte_eal_init(argc, argv);
        printf("ret %d\n", ret);
        return 0;
}

I found if I use the Makefile provided in the example, the program will find the ixgbe NIC. But if I just use these 2 commands to compile and link it. It will not find the ixgbe NIC.

gcc -I../../x86_64-native-linuxapp-gcc/include -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash -lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool -lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt

Can someone share some light on what is magic of the dpdk Makefile to correctly register the NIC type?

Thank you so much.
Xingbo Wang

Re: Can not init NIC after merge to DPDK 1.7 problem

[Neil Horman]

On Tue, Sep 23, 2014 at 06:53:57PM +0000, Wang, Shawn wrote:
> Hi:
> 
> We are using our own Makefile in building dpdk program. Recently we are working on upgrading from DPDK 1.3 to DPDK 1.7. I found the rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete rte_ixgbe_pmd_init calls. But after that, our dpdk program could not correctly find the NIC anymore. After digging into it a little more, I found the code dose not correctly register the driver type we are using, which is ixgbe.
> To isolate the problem, I hacked a smal example l3fwd, and only have the main.c file like this for my testing purpose.
> 
> #include <rte_config.h>
> #include <rte_eal.h>
> 
> #include "main.h"
> 
> int
> MAIN(int argc, char **argv)
> {
>         /* init EAL */
>         int ret = rte_eal_init(argc, argv);
>         printf("ret %d\n", ret);
>         return 0;
> }
> 
> I found if I use the Makefile provided in the example, the program will find the ixgbe NIC. But if I just use these 2 commands to compile and link it. It will not find the ixgbe NIC.
> 
> gcc -I../../x86_64-native-linuxapp-gcc/include -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
> gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash -lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool -lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt
> 
> Can someone share some light on what is magic of the dpdk Makefile to correctly register the NIC type?
> 
> Thank you so much.
> Xingbo Wang
> 

I'm not really sure why you would strip out the Makefiles to dpdk, but I suppose
thats not the germaine question.

First, how are you building the DPDK?  As a set of shared libraries, or as a set
of static archives?  If you're building shared libraries, you need to pass
-shared to gcc, or the constructors will get stripped out using your command
line above.  There might be some other options that escape me, but you can find
out for sure by using the packaged makefiles and running make V=1 to see all the
passed options in the link stage

Secondly, when you say register the NIC type, do you mean that you don't see the
NIC get registered with dpdk, or you don't see an instance of the NIC created?
If its the former, you need to confirm that by running a debugger and looking at
what elements are on the device_list after your applications starts.  If its the
latter, that may well be a config error, as you may need to pass the --whitelist
option on the command line to trigger a device probe.

Neil

Re: Can not init NIC after merge to DPDK 1.7 problem

[Keunhong Lee]

In DPDK1.7, PMDs are loaded by __constructor__ functions.
As there is no direct reference to each PMD driver structures, linker may
not link the PMD library.
You should use -Wl,--whole-archive and -Wl,--no-whole-archive for linking
DPDK applications.
See http://dpdk.org/browse/dpdk/tree/mk/rte.app.mk for further information.

Keunhong.



2014-09-24 4:12 GMT+09:00 Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org>:

> On Tue, Sep 23, 2014 at 06:53:57PM +0000, Wang, Shawn wrote:
> > Hi:
> >
> > We are using our own Makefile in building dpdk program. Recently we are
> working on upgrading from DPDK 1.3 to DPDK 1.7. I found the
> rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete
> rte_ixgbe_pmd_init calls. But after that, our dpdk program could not
> correctly find the NIC anymore. After digging into it a little more, I
> found the code dose not correctly register the driver type we are using,
> which is ixgbe.
> > To isolate the problem, I hacked a smal example l3fwd, and only have the
> main.c file like this for my testing purpose.
> >
> > #include <rte_config.h>
> > #include <rte_eal.h>
> >
> > #include "main.h"
> >
> > int
> > MAIN(int argc, char **argv)
> > {
> >         /* init EAL */
> >         int ret = rte_eal_init(argc, argv);
> >         printf("ret %d\n", ret);
> >         return 0;
> > }
> >
> > I found if I use the Makefile provided in the example, the program will
> find the ixgbe NIC. But if I just use these 2 commands to compile and link
> it. It will not find the ixgbe NIC.
> >
> > gcc -I../../x86_64-native-linuxapp-gcc/include
> -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
> > gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal
> -lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash
> -lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool
> -lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond
> -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt
> >
> > Can someone share some light on what is magic of the dpdk Makefile to
> correctly register the NIC type?
> >
> > Thank you so much.
> > Xingbo Wang
> >
>
> I'm not really sure why you would strip out the Makefiles to dpdk, but I
> suppose
> thats not the germaine question.
>
> First, how are you building the DPDK?  As a set of shared libraries, or as
> a set
> of static archives?  If you're building shared libraries, you need to pass
> -shared to gcc, or the constructors will get stripped out using your
> command
> line above.  There might be some other options that escape me, but you can
> find
> out for sure by using the packaged makefiles and running make V=1 to see
> all the
> passed options in the link stage
>
> Secondly, when you say register the NIC type, do you mean that you don't
> see the
> NIC get registered with dpdk, or you don't see an instance of the NIC
> created?
> If its the former, you need to confirm that by running a debugger and
> looking at
> what elements are on the device_list after your applications starts.  If
> its the
> latter, that may well be a config error, as you may need to pass the
> --whitelist
> option on the command line to trigger a device probe.
>
> Neil
>
>

Re: Can not init NIC after merge to DPDK 1.7 problem

[Matthew Hall]

On Tue, Sep 23, 2014 at 06:53:57PM +0000, Wang, Shawn wrote:
> Can someone share some light on what is magic of the dpdk Makefile to correctly register the NIC type?

I had the same problem as a guy who began using it before the auto-reg, 
stopped a while, and began again after.

You have to pass the following GNU LD option:

--whole-archive

Matthew.