From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yoann Juet Subject: bnx2x + SR-IOV, no internal L2 switching Date: Wed, 12 Feb 2014 14:33:55 +0100 Message-ID: <52FB7843.6050601@univ-nantes.fr> Reply-To: yoann.juet@univ-nantes.fr Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms030102030106030608000602" To: netdev@vger.kernel.org Return-path: Received: from smtptls1-lmb.cpub.univ-nantes.fr ([193.52.103.110]:54927 "EHLO smtp-tls.univ-nantes.fr" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751354AbaBLNlm (ORCPT ); Wed, 12 Feb 2014 08:41:42 -0500 Received: from localhost (localhost [127.0.0.1]) by smtp-tls.univ-nantes.fr (Postfix) with ESMTP id 1E95FC03696 for ; Wed, 12 Feb 2014 14:33:02 +0100 (CET) Received: from smtp-tls.univ-nantes.fr ([127.0.0.1]) by localhost (smtptls1-lmb.cpub.univ-nantes.fr [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QokfC19wx8UP for ; Wed, 12 Feb 2014 14:33:02 +0100 (CET) Received: from [IPv6:2001:660:7220:0:7016:a7c0:40d1:fe9a] (unknown [IPv6:2001:660:7220:0:7016:a7c0:40d1:fe9a]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-tls.univ-nantes.fr (Postfix) with ESMTPSA id EF78DC03694 for ; Wed, 12 Feb 2014 14:33:01 +0100 (CET) Sender: netdev-owner@vger.kernel.org List-ID: This is a cryptographically signed message in MIME format. --------------ms030102030106030608000602 Content-Type: multipart/mixed; boundary="------------000300020504010907090109" This is a multi-part message in MIME format. --------------000300020504010907090109 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Hi all, I'm conducting experiments on SR-IOV with Broadcom and Intel cards on=20 debian/unstable with KVM hypervisor. On Broadcom cards (bnx2x module,=20 BCM57810 devices), Virtual Functions (VFs) get running, Virtual Machines = attached to such VFs inherit network connectivity with excellent=20 performance. However, VMs attached to VFs on the Broadcom Physical Functions (PFs)=20 behave like they were connected to an ancient hub, not a L2 switch. It=20 is as if there was no internal L2 switching on the Broadcom card to=20 process VF <-> VF or VF <-> PF communications. As a result, a VM sees=20 all inbound/outbound traffic from/to others VMs as well as traffic=20 destined to the PF (for instance, the physical ethX has an IP address). On the other hand, everything works like a charm with Intel cards (ixgbe = module, 82599EB devices). Traffic between VFs or VF/PF is switched=20 internally by the card. I found very little literature about SR-IOV on Broadcom devices. I=20 wonder if it's a normal behaviour, a misconfiguration on my side or=20 perhaps a firmware/driver bug. Have you seen this issue before ? --- Kernel 3.12.9 (same behaviour with kernels 3.10.x) driver: bnx2x firmware-version: 7.8.17 Debian/unstable libvirt 1.2.1 QEMU 1.7.0 --- Best regards, --=20 Universit=E9 de Nantes - Direction des Syst=E8mes d'Information --------------000300020504010907090109 Content-Type: text/x-vcard; charset=utf-8; name="yoann_juet.vcf" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="yoann_juet.vcf" begin:vcard fn:Yoann Juet n:Juet;Yoann org;quoted-printable;quoted-printable:Direction des Syst=3DC3=3DA8mes d'I= nformation;P=3DC3=3DB4le R=3DC3=3DA9seau adr;quoted-printable:BP 92208;;2 rue de la Houssini=3DC3=3DA8re;Nantes Ce= dex 3;;44322;France email;internet:yoann.juet@univ-nantes.fr tel;work:02.53.48.49.26 tel;fax:02.53.48.49.09 tel;cell:06.73.15.42.19 version:2.1 end:vcard --------------000300020504010907090109-- --------------ms030102030106030608000602 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJcDCC BKUwggONoAMCAQICEQCaw9r4AWwhPTK3X2MyL6H8MA0GCSqGSIb3DQEBBQUAMDsxCzAJBgNV BAYTAk5MMQ8wDQYDVQQKEwZURVJFTkExGzAZBgNVBAMTElRFUkVOQSBQZXJzb25hbCBDQTAe Fw0xMTA4MTcwMDAwMDBaFw0xNDA4MTYyMzU5NTlaMGcxCzAJBgNVBAYTAkZSMR0wGwYDVQQK ExRVTklWRVJTSVRFIERFIE5BTlRFUzETMBEGA1UEAxMKWW9hbm4gSlVFVDEkMCIGCSqGSIb3 DQEJAhYVanVldC15QHVuaXYtbmFudGVzLmZyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEAzT4cyMjT4IaRAql635iNoxjb9ticpoS8Qauq9qGkV2HzDOqjI5SIc0nlU1EOWnbO 05cewhUbOcM1YunHBSahw5cltsC3Uu4D2XAVQ5vD9VmS9kV2eBn3T9vSB1Yz7r7xSPOhHN50 8qspRwgRExu1tFANqG7TMJbFtN5iG22fiQkNzlVLDV3vgB6B9eJaAayfVlZSq+QUMv2dp6AI 8g7Z1tInb7EWVMbB17d7rSfGXJnQHN9wlRduAg2r+PfHcH2Z6Bxmxl66GABxH9keZxdnfo6G ph0/l0FFj7VwitXEU7V1kTuq7+vZc5qAT/mcSPO471+KzfU14AgEc8qvALV6iQIDAQABo4IB djCCAXIwHwYDVR0jBBgwFoAUY01DWhlIP8RGwQK6v+4O5YK3ZqYwHQYDVR0OBBYEFKPfL4a9 soFXobME5dHUc30NI4iCMA4GA1UdDwEB/wQEAwIFoDAMBgNVHRMBAf8EAjAAMB0GA1UdJQQW MBQGCCsGAQUFBwMEBggrBgEFBQcDAjAYBgNVHSAEETAPMA0GCysGAQQBsjEBAgIdMD8GA1Ud HwQ4MDYwNKAyoDCGLmh0dHA6Ly9jcmwudGNzLnRlcmVuYS5vcmcvVEVSRU5BUGVyc29uYWxD QS5jcmwwcgYIKwYBBQUHAQEEZjBkMDoGCCsGAQUFBzAChi5odHRwOi8vY3J0LnRjcy50ZXJl bmEub3JnL1RFUkVOQVBlcnNvbmFsQ0EuY3J0MCYGCCsGAQUFBzABhhpodHRwOi8vb2NzcC50 Y3MudGVyZW5hLm9yZzAkBgNVHREEHTAbgRlZb2Fubi5KdWV0QHVuaXYtbmFudGVzLmZyMA0G CSqGSIb3DQEBBQUAA4IBAQCsH20G5ntaV5rBAKu+TjhdseCOkRFhExE6W+ImhXJb1aKFLtnt epvS6pTenXbBp+8OTjOXDJ9JpmA6sbGpy8FYIThiDiRLVMf1kVgU4ECDo2RE6AHtnRJ645Dw +fvXbFef9u+0aGehKREb6OVdl/6x9DFQvVm/bj9hQtXPusCXYUe1b/ZCbUKFQSfmXwFMLmFb U64YhP3YRczYaznadT8Cm0INpaYZJatUan74iufwiC71toLGOEnupbQGOLSHalBdAy+u9WGl 5Nm+oE+R+m8tqbRFUn9GSiswSi7RQXxk4O/qOev3/aF+cG4W4D5DCVBsKqTpFcu1fgzdzWQE IxRTMIIEwzCCA6ugAwIBAgIQc/5X+t+4xQiBe2a5a/At7zANBgkqhkiG9w0BAQUFADCBrjEL MAkGA1UEBhMCVVMxCzAJBgNVBAgTAlVUMRcwFQYDVQQHEw5TYWx0IExha2UgQ2l0eTEeMBwG A1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMSEwHwYDVQQLExhodHRwOi8vd3d3LnVzZXJ0 cnVzdC5jb20xNjA0BgNVBAMTLVVUTi1VU0VSRmlyc3QtQ2xpZW50IEF1dGhlbnRpY2F0aW9u IGFuZCBFbWFpbDAeFw0wOTA1MTgwMDAwMDBaFw0yODEyMzEyMzU5NTlaMDsxCzAJBgNVBAYT Ak5MMQ8wDQYDVQQKEwZURVJFTkExGzAZBgNVBAMTElRFUkVOQSBQZXJzb25hbCBDQTCCASIw DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMgV2fUzaiOhkA3PuwVEw6sfSjWFGiGFoE/4 8EDiSkOb/luxsL+0V9x1gEFLZBr2209vj9AlRTX56stK+vva0+1FiBGUNuTMqA3vxT037RZ7 48KVnlgzfyL7+P/s5r7brgplJSKH2m+Ei0boQIYoP79WCOJK6YOi6SL7Lfq2KB+RwcNx+1PA K06kDKFunVXt7OEkhzoI4g0c5MRN0Msn+oRk5tGXnVYkW5O7KS5D4kFqPeJKZfyaX1qGh7yH x3mAlplRxpAPZJGZfRDdrCwDKuF4ZP7OPU70K1ARS9FY1JsD8H/1O1OwU0P7xS/EBCkaOqzX VenRDpXROzF/eE+uKS0CAwEAAaOCAU0wggFJMB8GA1UdIwQYMBaAFImCZ33EnSZwAEu0UEh8 3j2uBG59MB0GA1UdDgQWBBRjTUNaGUg/xEbBArq/7g7lgrdmpjAOBgNVHQ8BAf8EBAMCAQYw EgYDVR0TAQH/BAgwBgEB/wIBADAYBgNVHSAEETAPMA0GCysGAQQBsjEBAgIdMFgGA1UdHwRR ME8wTaBLoEmGR2h0dHA6Ly9jcmwudXNlcnRydXN0LmNvbS9VVE4tVVNFUkZpcnN0LUNsaWVu dEF1dGhlbnRpY2F0aW9uYW5kRW1haWwuY3JsMG8GCCsGAQUFBwEBBGMwYTA4BggrBgEFBQcw AoYsaHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VUTkFBQUNsaWVudF9DQS5jcnQwJQYIKwYB BQUHMAGGGWh0dHA6Ly9vY3NwLnVzZXJ0cnVzdC5jb20wDQYJKoZIhvcNAQEFBQADggEBAAYr qVMvE9xcORbMhp9eTHty++yNVYpemgr/U0x29AY9UM9X8KLPr5mMqv6gGXY+mQMy+nWDOIq7 a5qlDBOy1Bt25pQuZ5hZ45FsApCanMhgS1WryohajSvlaZUDB9HUDvwkIi5ZsWOkX+3ZI3Lk nM46XGwfT6kAyR3++n9FLbYuhN0PJ6BZGE7VdiVF9JkmedtvnyP3Q7srDwSjgSYst3s1+T13 X0Ah5n8dpZZavdDLFjpsu2GLiv0EOUQKyyzhy84uEJga2+CT7UlkZAggn7ejUPCi3cq0xnwM PedeFdwnhuQ6O1JaF6upBlMrnQlzZBwBw/0w0ocDb+QVA3o5X2gxggMKMIIDBgIBATBQMDsx CzAJBgNVBAYTAk5MMQ8wDQYDVQQKEwZURVJFTkExGzAZBgNVBAMTElRFUkVOQSBQZXJzb25h bCBDQQIRAJrD2vgBbCE9MrdfYzIvofwwCQYFKw4DAhoFAKCCAY8wGAYJKoZIhvcNAQkDMQsG CSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwMjEyMTMzMzU1WjAjBgkqhkiG9w0BCQQx FgQUyLIaseacFmBPsmMemRSMsbvHo9YwXwYJKwYBBAGCNxAEMVIwUDA7MQswCQYDVQQGEwJO TDEPMA0GA1UEChMGVEVSRU5BMRswGQYDVQQDExJURVJFTkEgUGVyc29uYWwgQ0ECEQCaw9r4 AWwhPTK3X2MyL6H8MGEGCyqGSIb3DQEJEAILMVKgUDA7MQswCQYDVQQGEwJOTDEPMA0GA1UE ChMGVEVSRU5BMRswGQYDVQQDExJURVJFTkEgUGVyc29uYWwgQ0ECEQCaw9r4AWwhPTK3X2My L6H8MGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG 9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcN AwICASgwDQYJKoZIhvcNAQEBBQAEggEAffoVQ/tAOLLsph9UTzIimDh5QPN3oGhmK3lARJ6K 4DaK6a/mh069JpufmdeROZh0Cq5+790ytprBK/klr5Bc5q83MVuMfZLiQqbfumqxqDrO7e+M AxVZJO/rO1tT9qJmKrgKTXQoi8bzQxobsbY84wHkD8uqlr6runPcxoHE/YGGhtLrVYRwMqIR zx5ox4KaU+4xhH4cpZgOYXCzccgkXq2jZV573MFSE1BOzqVn867Gp1YfveQbu9pwa7kWE1od l/f1Y9GE8vsJJU50pe17TivEFv8D9oJ6Xel70cHaDgyi/GIPKLfhwKojegrRWns6UHI1Z0A8 rZGHdT7WfHuqygAAAAAAAA== --------------ms030102030106030608000602--