Build broken with COMBINE_LIBS=y

[Li Wei <lw@cn.fujitsu.com>]

Hi list,

After drivers separation, the following building error was encountered,
it seems the build system build lib/ first and link it into libintel_dpdk.a
and then drivers/ got compiled, so the symbols in drivers never got linked
into libintel_dpdk.a.

I guess we need add some dependence on drivers/ on libintel_dpdk.a target,
but I'm not familiar with the build system :(

Error messages as follow:

[...]
== Build app
== Build app/test
  CC commands.o
  CC test.o
  CC test_pci.o
  CC test_prefetch.o
  CC test_byteorder.o
  CC test_per_lcore.o
  CC test_atomic.o
  CC test_malloc.o
  CC test_cycles.o
  CC test_spinlock.o
  CC test_memory.o
  CC test_memzone.o
  CC test_ring.o
  CC test_ring_perf.o
  CC test_pmd_perf.o
  CC test_table.o
  CC test_table_pipeline.o
  CC test_table_tables.o
  CC test_table_ports.o
  CC test_table_combined.o
  CC test_table_acl.o
  CC test_rwlock.o
  CC test_timer.o
  CC test_timer_perf.o
  CC test_mempool.o
  CC test_mempool_perf.o
  CC test_mbuf.o
  CC test_logs.o
  CC test_memcpy.o
  CC test_memcpy_perf.o
  CC test_hash.o
  CC test_hash_perf.o
  CC test_lpm.o
  CC test_lpm6.o
  CC test_debug.o
  CC test_errno.o
  CC test_tailq.o
  CC test_string_fns.o
  CC test_cpuflags.o
  CC test_mp_secondary.o
  CC test_eal_flags.o
  CC test_eal_fs.o
  CC test_alarm.o
  CC test_interrupts.o
  CC test_version.o
  CC test_func_reentrancy.o
  CC test_cmdline.o
  CC test_cmdline_num.o
  CC test_cmdline_etheraddr.o
  CC test_cmdline_portlist.o
  CC test_cmdline_ipaddr.o
  CC test_cmdline_cirbuf.o
  CC test_cmdline_string.o
  CC test_cmdline_lib.o
  CC test_red.o
  CC test_sched.o
  CC test_meter.o
  CC test_kni.o
  CC test_power.o
  CC test_power_acpi_cpufreq.o
  CC test_power_kvm_vm.o
  CC test_common.o
  CC test_distributor.o
  CC test_distributor_perf.o
  CC test_reorder.o
  CC test_devargs.o
  CC virtual_pmd.o
  CC packet_burst_generator.o
  CC test_acl.o
  CC test_link_bonding.o
  CC test_link_bonding_mode4.o
  CC test_pmd_ring.o
  CC test_kvargs.o
  LD test
test_link_bonding.o: In function `test_add_slave_to_bonded_device':
test_link_bonding.c:(.text+0x7ca): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0x7e2): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x807): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_remove_slave_from_bonded_device':
test_link_bonding.c:(.text+0x8cf): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.c:(.text+0x8ed): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.o: In function `test_get_slaves_from_bonded_device':
test_link_bonding.c:(.text+0xa1f): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xa3e): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xa59): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xa79): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xa90): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xaac): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_set_bonded_port_initialization_mac_assignment':
test_link_bonding.c:(.text+0xbd0): undefined reference to `rte_eth_bond_create'
test_link_bonding.c:(.text+0xc7e): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0xca0): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xcbf): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.c:(.text+0xddb): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0xed3): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.c:(.text+0xef5): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.o: In function `test_add_already_bonded_slave_to_bonded_device':
test_link_bonding.c:(.text+0x11fa): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x1244): undefined reference to `rte_eth_bond_create'
test_link_bonding.c:(.text+0x1269): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.o: In function `test_create_bonded_device_with_invalid_params':
test_link_bonding.c:(.text+0x131d): undefined reference to `rte_eth_bond_create'
test_link_bonding.c:(.text+0x1341): undefined reference to `rte_eth_bond_create'
test_link_bonding.c:(.text+0x1362): undefined reference to `rte_eth_bond_create'
test_link_bonding.o: In function `test_create_bonded_device':
test_link_bonding.c:(.text+0x13e9): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.c:(.text+0x1405): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x1421): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x144d): undefined reference to `rte_eth_bond_create'
test_link_bonding.o: In function `test_stop_bonded_device':
test_link_bonding.c:(.text+0x158d): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x15b0): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_start_bonded_device':
test_link_bonding.c:(.text+0x1694): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x16b7): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x16d0): undefined reference to `rte_eth_bond_mode_get'
test_link_bonding.c:(.text+0x16e9): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_add_slave_to_invalid_bonded_device':
test_link_bonding.c:(.text+0x19e1): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0x19ff): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.o: In function `test_remove_slave_from_invalid_bonded_device':
test_link_bonding.c:(.text+0x1a76): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.c:(.text+0x1a99): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.o: In function `initialize_bonded_device_with_slaves.part.1':
test_link_bonding.c:(.text+0x24c8): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x24fe): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0x251a): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x2596): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.o: In function `remove_slaves_and_stop_bonded_device':
test_link_bonding.c:(.text+0x27e1): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.c:(.text+0x27ff): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x2880): undefined reference to `rte_eth_bond_mac_address_reset'
test_link_bonding.o: In function `test_broadcast_verify_mac_assignment':
test_link_bonding.c:(.text+0x29cf): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0x2b3d): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.o: In function `test_tlb_verify_slave_link_status_change_failover':
test_link_bonding.c:(.text+0x3c41): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x3c6b): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x3c80): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.c:(.text+0x3cbf): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x3d1c): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x3d31): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_tlb_verify_promiscuous_enable_disable':
test_link_bonding.c:(.text+0x4108): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_tlb_verify_mac_assignment':
test_link_bonding.c:(.text+0x43bc): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0x44eb): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.o: In function `test_tlb_rx_burst':
test_link_bonding.c:(.text+0x4838): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_balance_verify_mac_assignment':
test_link_bonding.c:(.text+0x510c): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0x523b): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.o: In function `test_balance_xmit_policy_configuration':
test_link_bonding.c:(.text+0x5898): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0x58ae): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0x58c4): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0x58d8): undefined reference to `rte_eth_bond_xmit_policy_get'
test_link_bonding.c:(.text+0x58f1): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0x5905): undefined reference to `rte_eth_bond_xmit_policy_get'
test_link_bonding.c:(.text+0x591b): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0x592f): undefined reference to `rte_eth_bond_xmit_policy_get'
test_link_bonding.c:(.text+0x593e): undefined reference to `rte_eth_bond_xmit_policy_get'
test_link_bonding.o: In function `test_activebackup_verify_mac_assignment':
test_link_bonding.c:(.text+0x5b6c): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0x5c9b): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.o: In function `test_activebackup_verify_promiscuous_enable_disable':
test_link_bonding.c:(.text+0x5f98): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_activebackup_rx_burst':
test_link_bonding.c:(.text+0x61c7): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_roundrobin_verify_mac_assignment':
test_link_bonding.c:(.text+0x65af): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0x671d): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.o: In function `test_roundrobin_verfiy_polling_slave_link_status_change':
test_link_bonding.c:(.text+0x73d7): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0x75fb): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.c:(.text+0x7616): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding.o: In function `test_status_interrupt':
test_link_bonding.c:(.text+0x7880): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x794d): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_set_bonding_mode':
test_link_bonding.c:(.text+0x7c3e): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.c:(.text+0x7c54): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.c:(.text+0x7c6a): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.c:(.text+0x7c7e): undefined reference to `rte_eth_bond_mode_get'
test_link_bonding.c:(.text+0x7c91): undefined reference to `rte_eth_bond_mode_get'
test_link_bonding.c:(.text+0x7ca5): undefined reference to `rte_eth_bond_mode_get'
test_link_bonding.o: In function `balance_l34_tx_burst.constprop.6':
test_link_bonding.c:(.text+0x8152): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.o: In function `balance_l23_tx_burst':
test_link_bonding.c:(.text+0x8642): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.o: In function `test_balance_l2_tx_burst':
test_link_bonding.c:(.text+0x8b17): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.o: In function `test_adding_slave_after_bonded_device_started':
test_link_bonding.c:(.text+0x8faf): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.o: In function `test_broadcast_verify_slave_link_status_change_behaviour':
test_link_bonding.c:(.text+0x90b1): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x90ce): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x9114): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_add_remove_multiple_slaves_to_from_bonded_device':
test_link_bonding.c:(.text+0x9553): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0x956f): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x9594): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_roundrobin_verify_slave_link_status_change_behaviour':
test_link_bonding.c:(.text+0x9712): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0x972f): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0x9768): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_balance_tx_burst_slave_tx_fail':
test_link_bonding.c:(.text+0xa085): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.o: In function `test_activebackup_tx_burst':
test_link_bonding.c:(.text+0xa660): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_set_explicit_bonded_mac':
test_link_bonding.c:(.text+0xa8ee): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.c:(.text+0xa905): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.c:(.text+0xa91b): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.c:(.text+0xa932): undefined reference to `rte_eth_bond_mac_address_set'
test_link_bonding.c:(.text+0xa96a): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xa9a4): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0xa9c5): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xaa8f): undefined reference to `rte_eth_bond_mac_address_reset'
test_link_bonding.c:(.text+0xaaa3): undefined reference to `rte_eth_bond_mac_address_reset'
test_link_bonding.c:(.text+0xaac4): undefined reference to `rte_eth_bond_mac_address_reset'
test_link_bonding.o: In function `test_set_primary_slave':
test_link_bonding.c:(.text+0xb400): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xb43a): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding.c:(.text+0xb456): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xb4ba): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding.c:(.text+0xb4d6): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0xb4ec): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0xb50e): undefined reference to `rte_eth_bond_primary_set'
test_link_bonding.c:(.text+0xb522): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.c:(.text+0xb5f3): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.c:(.text+0xb607): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.c:(.text+0xb628): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.o: In function `test_balance_verify_slave_link_status_change_behaviour':
test_link_bonding.c:(.text+0xbeab): undefined reference to `rte_eth_bond_xmit_policy_set'
test_link_bonding.c:(.text+0xbec7): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xbee4): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xbf2a): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xc0ac): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.o: In function `test_activebackup_verify_slave_link_status_change_failover':
test_link_bonding.c:(.text+0xc535): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding.c:(.text+0xc552): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xc574): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding.c:(.text+0xc5b3): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xc600): undefined reference to `rte_eth_bond_active_slaves_get'
test_link_bonding.c:(.text+0xc625): undefined reference to `rte_eth_bond_primary_get'
test_link_bonding_mode4.o: In function `check_environment':
test_link_bonding_mode4.c:(.text+0x54c): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding_mode4.c:(.text+0x587): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding_mode4.c:(.text+0x692): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding_mode4.o: In function `remove_slaves_and_stop_bonded_device':
test_link_bonding_mode4.c:(.text+0x736): undefined reference to `rte_eth_bond_slaves_get'
test_link_bonding_mode4.c:(.text+0x7c3): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding_mode4.c:(.text+0x82d): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding_mode4.c:(.text+0x897): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding_mode4.c:(.text+0x8fd): undefined reference to `rte_eth_bond_slave_remove'
test_link_bonding_mode4.o: In function `initialize_bonded_device_with_slaves.constprop.6':
test_link_bonding_mode4.c:(.text+0xbfa): undefined reference to `rte_eth_bond_slave_add'
test_link_bonding_mode4.c:(.text+0xc98): undefined reference to `rte_eth_bond_8023ad_setup'
test_link_bonding_mode4.o: In function `bond_handshake':
test_link_bonding_mode4.c:(.text+0x1cb4): undefined reference to `rte_eth_bond_8023ad_conf_get'
test_link_bonding_mode4.o: In function `test_mode4_expired':
test_link_bonding_mode4.c:(.text+0x205f): undefined reference to `rte_eth_bond_8023ad_conf_get'
test_link_bonding_mode4.c:(.text+0x20a9): undefined reference to `rte_eth_bond_8023ad_setup'
test_link_bonding_mode4.o: In function `test_mode4_tx_burst':
test_link_bonding_mode4.c:(.text+0x28af): undefined reference to `rte_eth_bond_8023ad_conf_get'
test_link_bonding_mode4.o: In function `test_mode4_marker':
test_link_bonding_mode4.c:(.text+0x2eef): undefined reference to `rte_eth_bond_8023ad_conf_get'
test_link_bonding_mode4.o: In function `test_setup':
test_link_bonding_mode4.c:(.text+0x378b): undefined reference to `rte_eth_bond_mode_get'
test_link_bonding_mode4.c:(.text+0x37a1): undefined reference to `rte_eth_bond_mode_set'
test_link_bonding_mode4.c:(.text+0x37f9): undefined reference to `rte_eth_from_rings'
test_link_bonding_mode4.c:(.text+0x3827): undefined reference to `rte_eth_bond_create'
test_link_bonding_mode4.o: In function `test_mode4_rx':
test_link_bonding_mode4.c:(.text+0x441d): undefined reference to `rte_eth_bond_8023ad_conf_get'
collect2: error: ld returned 1 exit status
/home/liwei/src/dpdk/mk/rte.app.mk:195: recipe for target 'test' failed
make[3]: *** [test] Error 1
/home/liwei/src/dpdk/mk/rte.subdir.mk:61: recipe for target 'test' failed
make[2]: *** [test] Error 2
/home/liwei/src/dpdk/mk/rte.sdkbuild.mk:93: recipe for target 'app' failed
make[1]: *** [app] Error 2
/home/liwei/src/dpdk/mk/rte.sdkroot.mk:124: recipe for target 'all' failed
make: *** [all] Error 2