From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Thu, 12 Jan 2017 07:29:58 -0800 (PST) From: shlemisto@gmail.com Message-Id: <1d63eddd-e9b2-4023-8e0c-6bc658633b9a@googlegroups.com> In-Reply-To: <422f314a-de5e-06a0-33cd-a6d894db1d61@intel.com> References: <360341483979222@web10o.yandex.ru> <7784bee8-663d-bd89-2d2b-ef47081153d1@intel.com> <6664bf72-a4bb-4081-0fb9-13c66adbec4e@intel.com> <422f314a-de5e-06a0-33cd-a6d894db1d61@intel.com> Subject: Re: Problem with ntb_transport + ntb_netdev MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_233_1731089330.1484234999032" To: linux-ntb List-ID: ------=_Part_233_1731089330.1484234999032 Content-Type: multipart/alternative; boundary="----=_Part_234_1144932586.1484234999033" ------=_Part_234_1144932586.1484234999033 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I built this kernel. Unfortunately link doesn't train. ntb_hw_intel_1 - 1'st side ntb_hw_intel_2 - 2'nd side =D1=81=D1=80=D0=B5=D0=B4=D0=B0, 11 =D1=8F=D0=BD=D0=B2=D0=B0=D1=80=D1=8F 201= 7 =D0=B3., 19:45:05 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0= =D1=82=D0=B5=D0=BB=D1=8C dave.jiang =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0= =BB: > > > > On 01/11/2017 09:42 AM, Dave Jiang wrote:=20 > >=20 > >=20 > > On 01/11/2017 03:08 AM, shle...@gmail.com wrote:=20 > >>> Do pings in the other direction work?=20 > >> Similarly=20 > >>> How are the NTBs connected?=20 > >> B2B=20 > >>> And which distro are you running?=20 > >> Ubuntu 16.10 with kernel 4.10.0-rc1 from git=20 > >>> Sometimes=20 > >> Network Manager will yank the IP for no reason if you don't configure= =20 > it=20 > >> correctly and thus your interface can't route.=20 > >> I stop network-manager service before load the modules and set IP=20 > manually.=20 > >>=20 > >>> Also can you dump the stats in debugfs?=20 > >> ntb_hw_intel_info_1 - one side=20 > >> ntb_hw_intel_info_2 - second side=20 > >> ntb_transport_info_1 - one side=20 > >> ntb_transport_info_2 - second side=20 > >=20 > > Your secondary BAR2 size is not the same as your primary. They both nee= d=20 > > to be 22. Also all the upstream memory miss is concerning. You should= =20 > > have none. You may have signal integrity issues. But fix the bar size= =20 > > first and see if that helps.=20 > > Also, if you are using Broadwell Xeon D, you should use the kernel from= =20 > here and not upstream.=20 > https://github.com/davejiang/linux=20 > branch: davejiang/ntb=20 > > > > >=20 > >>=20 > >> =D1=81=D1=80=D0=B5=D0=B4=D0=B0, 11 =D1=8F=D0=BD=D0=B2=D0=B0=D1=80=D1= =8F 2017 =D0=B3., 1:24:17 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2= =D0=B0=D1=82=D0=B5=D0=BB=D1=8C dave.jiang=20 > =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:=20 > >>=20 > >> On 01/10/2017 03:03 PM, Jon Mason wrote:=20 > >> > Adding the NTB mailing list=20 > >> >=20 > >> > On Mon, Jan 9, 2017 at 11:27 AM, perplexus >> > wrote:=20 > >> >> Hello.=20 > >> >>=20 > >> >> I'm working with NTB (kernel version 4.10) on Xeon D-1527 CPU= =20 > and=20 > >> have a problem with couple ntb_transport + ntb_netdev.=20 > >> >>=20 > >> >> On the local side I run:=20 > >> >> sudo /etc/init.d/irqbalance stop=20 > >> >> sudo modprobe ntb_transport dyndbg=3D+p=20 > >> >> sudo modprobe ntb_netdev dyndbg=3D+p=20 > >> >> sudo ifconfig eth0 192.168.1.1 up=20 > >> >>=20 > >> >> On the remote side I run:=20 > >> >> sudo /etc/init.d/irqbalance stop=20 > >> >> sudo modprobe ntb_transport dyndbg=3D+p=20 > >> >> sudo modprobe ntb_netdev dyndbg=3D+p=20 > >> >> sudo ifconfig eth0 192.168.1.2 up=20 > >> >>=20 > >> >> Along with these steps I run ping 192.168.1.2 and get about 2-3= =20 > >> successfull transmitted/received packets then get 'Destination hos= t=20 > >> unreacheble'.=20 > >> >>=20 > >> >=20 > >> > Do pings in the other direction work?=20 > >> >=20 > >> How are the NTBs connected? Where did the get the hardware? Also= =20 > can=20 > >> you=20 > >> dump the stats in debugfs? And which distro are you running?=20 > Sometimes=20 > >> Network Manager will yank the IP for no reason if you don't=20 > >> configure it=20 > >> correctly and thus your interface can't route.=20 > >>=20 > >> --=20 > >> You received this message because you are subscribed to the Google=20 > >> Groups "linux-ntb" group.=20 > >> To unsubscribe from this group and stop receiving emails from it, send= =20 > >> an email to linux-ntb+...@googlegroups.com =20 > >> >.=20 > >> To post to this group, send email to linu...@googlegroups.com=20 > =20 > >> >.=20 > >> To view this discussion on the web visit=20 > >>=20 > https://groups.google.com/d/msgid/linux-ntb/c18c3dc2-d242-4352-946e-931ff= 13e1d28%40googlegroups.com=20 > >> < > https://groups.google.com/d/msgid/linux-ntb/c18c3dc2-d242-4352-946e-931ff= 13e1d28%40googlegroups.com?utm_medium=3Demail&utm_source=3Dfooter>.=20 > > >> For more options, visit https://groups.google.com/d/optout.=20 > >=20 > ------=_Part_234_1144932586.1484234999033 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
I built this kernel.
Unfortunately link doesn't tr= ain.

ntb_hw_= intel_1=C2=A0- 1'st side
ntb_hw_intel_2=C2=A0- 2'nd side

=D1=81= =D1=80=D0=B5=D0=B4=D0=B0, 11 =D1=8F=D0=BD=D0=B2=D0=B0=D1=80=D1=8F 2017 =D0= =B3., 19:45:05 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE=D0=B2=D0=B0=D1=82= =D0=B5=D0=BB=D1=8C dave.jiang =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:

On 01/11/2017 09:42 AM, Dave Jiang wrote:
>=20
>=20
> On 01/11/2017 03:08 AM, shle...@gmail.com wrote:
>>> Do pings in the other direction work?=20
>> Similarly
>>> How are the NTBs connected?
>> B2B
>>> And which distro are you running?
>> Ubuntu 16.10 with kernel 4.10.0-rc1 from git
>>> =C2=A0Sometimes=20
>> Network Manager will yank the IP for no reason if you don'= t configure it=20
>> correctly and thus your interface can't route.
>> I stop network-manager service before load the modules and set= IP manually.
>>
>>> Also can you dump the stats in debugfs?
>> ntb_hw_intel_info_1 <http://pastebin.com/eyMZzSik> - o= ne side
>> ntb_hw_intel_info_2 <http://pastebin.com/vF94zLBk> - s= econd side
>> ntb_transport_info_1 <http://pastebin.com/Ktq1yAqa> - = one side
>> ntb_transport_info_2 <http://pastebin.com/w8r08ezB> - = second side
>=20
> Your secondary BAR2 size is not the same as your primary. They bot= h need
> to be 22. Also all the upstream memory miss is concerning. You sho= uld
> have none. You may have signal integrity issues. But fix the bar s= ize
> first and see if that helps.

Also, if you are using Broadwell Xeon D, you should use the kernel from
here and not upstream.
https://github.com/davejiang/linux
branch: davejiang/ntb



>=20
>>
>> =D1=81=D1=80=D0=B5=D0=B4=D0=B0, 11 =D1=8F=D0=BD=D0=B2=D0=B0=D1= =80=D1=8F 2017 =D0=B3., 1:24:17 UTC+3 =D0=BF=D0=BE=D0=BB=D1=8C=D0=B7=D0=BE= =D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C dave.jiang =D0=BD=D0=B0=D0=BF=D0=B8=D1= =81=D0=B0=D0=BB:
>>
>> =C2=A0 =C2=A0 On 01/10/2017 03:03 PM, Jon Mason wrote:
>> =C2=A0 =C2=A0 > Adding the NTB mailing list
>> =C2=A0 =C2=A0 >
>> =C2=A0 =C2=A0 > On Mon, Jan 9, 2017 at 11:27 AM, perplexus = <perp...@ya.ru
>> =C2=A0 =C2=A0 <javascript:>> wrote:
>> =C2=A0 =C2=A0 >> Hello.
>> =C2=A0 =C2=A0 >>
>> =C2=A0 =C2=A0 >> I'm working with NTB (kernel versio= n 4.10) on Xeon D-1527 CPU and
>> =C2=A0 =C2=A0 have a problem with couple ntb_transport + ntb_n= etdev.
>> =C2=A0 =C2=A0 >>
>> =C2=A0 =C2=A0 >> On the local side I run:
>> =C2=A0 =C2=A0 >> sudo /etc/init.d/irqbalance stop
>> =C2=A0 =C2=A0 >> sudo modprobe ntb_transport dyndbg=3D+p
>> =C2=A0 =C2=A0 >> sudo modprobe ntb_netdev dyndbg=3D+p
>> =C2=A0 =C2=A0 >> sudo ifconfig eth0 192.168.1.1 up
>> =C2=A0 =C2=A0 >>
>> =C2=A0 =C2=A0 >> On the remote side I run:
>> =C2=A0 =C2=A0 >> sudo /etc/init.d/irqbalance stop
>> =C2=A0 =C2=A0 >> sudo modprobe ntb_transport dyndbg=3D+p
>> =C2=A0 =C2=A0 >> sudo modprobe ntb_netdev dyndbg=3D+p
>> =C2=A0 =C2=A0 >> sudo ifconfig eth0 192.168.1.2 up
>> =C2=A0 =C2=A0 >>
>> =C2=A0 =C2=A0 >> Along with these steps I run ping 192.1= 68.1.2 and get about 2-3
>> =C2=A0 =C2=A0 successfull transmitted/received packets then ge= t 'Destination host
>> =C2=A0 =C2=A0 unreacheble'.
>> =C2=A0 =C2=A0 >>
>> =C2=A0 =C2=A0 >
>> =C2=A0 =C2=A0 > Do pings in the other direction work?
>> =C2=A0 =C2=A0 >
>> =C2=A0 =C2=A0 How are the NTBs connected? Where did the get th= e hardware? Also can
>> =C2=A0 =C2=A0 you
>> =C2=A0 =C2=A0 dump the stats in debugfs? And which distro are = you running? Sometimes
>> =C2=A0 =C2=A0 Network Manager will yank the IP for no reason i= f you don't
>> =C2=A0 =C2=A0 configure it
>> =C2=A0 =C2=A0 correctly and thus your interface can't rout= e.
>>
>> --=20
>> You received this message because you are subscribed to the Go= ogle
>> Groups "linux-ntb" group.
>> To unsubscribe from this group and stop receiving emails from = it, send
>> an email to linux-ntb+...@googlegroups.com
>> <mailto:linux-ntb+...@googlegroups.com>.
>> To post to this group, send email to linu...@googlegroups.com
>> <mailto:linu...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/linux-ntb/c18c3dc2-d242-4352-946e-931ff13e1d28%40googlegroups.com
>> <https:/= /groups.google.com/d/msgid/linux-ntb/c18c3dc2-d242-4352-946e-931f= f13e1d28%40googlegroups.com?utm_medium=3Demail&utm_source=3Df= ooter>.
>> For more options, visit https://group= s.google.com/d/optout.
>=20
------=_Part_234_1144932586.1484234999033-- ------=_Part_233_1731089330.1484234999032--