[dpdk-dev] Forcing inlining for igb_uio and kni

[dpdk-dev] Forcing inlining for igb_uio and kni

[David Marchand]

I noticed that OVS CI [1] patches the dpdk sources to force some inlining
parameters and get kni and igb_uio to build fine.

Looking at it in dpdk, meson support dropped this.
In the makefiles, I can't find a reason in the git history (we go back to
1.3.0rX version).

[dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
max-inline-insns-single=100
kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
max-inline-insns-single=50
[dmarchan@dmarchan dpdk]$ git blame origin/master --
kernel/linux/igb_uio/Makefile |grep max-inline-insns-single
13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel
 2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param
max-inline-insns-single=100
[dmarchan@dmarchan dpdk]$ git blame origin/master --
kernel/linux/kni/Makefile |grep max-inline-insns-single
3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel            2012-12-20
00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param
max-inline-insns-single=50

Is there a valid reason to keep this?
1: https://github.com/openvswitch/ovs/blob/master/.travis/linux-build.sh#L81


-- 
David Marchand

Re: [dpdk-dev] Forcing inlining for igb_uio and kni

[Ilya Maximets]

On 11.06.2019 11:45, David Marchand wrote:
> I noticed that OVS CI [1] patches the dpdk sources to force some inlining parameters and get kni and igb_uio to build fine.
> 
> Looking at it in dpdk, meson support dropped this.
> In the makefiles, I can't find a reason in the git history (we go back to 1.3.0rX version).
> 
> [dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
> kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param max-inline-insns-single=100
> kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param max-inline-insns-single=50
> [dmarchan@dmarchan dpdk]$ git blame origin/master -- kernel/linux/igb_uio/Makefile |grep max-inline-insns-single
> 13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel            2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param max-inline-insns-single=100
> [dmarchan@dmarchan dpdk]$ git blame origin/master -- kernel/linux/kni/Makefile |grep max-inline-insns-single
> 3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel            2012-12-20 00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param max-inline-insns-single=50
> 
> Is there a valid reason to keep this?
> 1: https://github.com/openvswitch/ovs/blob/master/.travis/linux-build.sh#L81 <https://protect2.fireeye.com/url?k=b7de159c45b1fa79.b7df9ed3-c4806461f28ecaf5&u=https://github.com/openvswitch/ovs/blob/master/.travis/linux-build.sh#L81>

Hi, David.
I don't know the reason for these in OVS travis config, But we don't
need to know them, actually. I have a patch to drop all the kernel
related stuff from the DPDK build in OVS Travis checks, just didn't
send it yet. Will send soon, probably.
If (or when) OVS Travis will migrate to meson, I'd suggest to just
use '-Denable_kmods=false'.

Best regards, Ilya Maximets.

Re: [dpdk-dev] Forcing inlining for igb_uio and kni

[David Marchand]

On Tue, Jun 11, 2019 at 11:31 AM Ilya Maximets <i.maximets@samsung.com>
wrote:

> On 11.06.2019 11:45, David Marchand wrote:
> > I noticed that OVS CI [1] patches the dpdk sources to force some
> inlining parameters and get kni and igb_uio to build fine.
> >
> > Looking at it in dpdk, meson support dropped this.
> > In the makefiles, I can't find a reason in the git history (we go back
> to 1.3.0rX version).
> >
> > [dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
> > kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
> max-inline-insns-single=100
> > kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
> max-inline-insns-single=50
> > [dmarchan@dmarchan dpdk]$ git blame origin/master --
> kernel/linux/igb_uio/Makefile |grep max-inline-insns-single
> > 13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel
>  2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param
> max-inline-insns-single=100
> > [dmarchan@dmarchan dpdk]$ git blame origin/master --
> kernel/linux/kni/Makefile |grep max-inline-insns-single
> > 3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel
>  2012-12-20 00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param
> max-inline-insns-single=50
> >
> > Is there a valid reason to keep this?
> > 1:
> https://github.com/openvswitch/ovs/blob/master/.travis/linux-build.sh#L81
> <
> https://protect2.fireeye.com/url?k=b7de159c45b1fa79.b7df9ed3-c4806461f28ecaf5&u=https://github.com/openvswitch/ovs/blob/master/.travis/linux-build.sh#L81
> >
>
> Hi, David.
> I don't know the reason for these in OVS travis config, But we don't
> need to know them, actually. I have a patch to drop all the kernel
> related stuff from the DPDK build in OVS Travis checks, just didn't
> send it yet. Will send soon, probably.
>

I had this in mind since we don't need those kmods in the CI.
Thanks Ilya.


The question on dpdk side remains open :-).

-- 
David Marchand

Re: [dpdk-dev] Forcing inlining for igb_uio and kni

[Bruce Richardson]

On Tue, Jun 11, 2019 at 11:44:01AM +0200, David Marchand wrote:
>    On Tue, Jun 11, 2019 at 11:31 AM Ilya Maximets
>    <[1]i.maximets@samsung.com> wrote:
> 
>      On 11.06.2019 11:45, David Marchand wrote:
>      > I noticed that OVS CI [1] patches the dpdk sources to force some
>      inlining parameters and get kni and igb_uio to build fine.
>      >
>      > Looking at it in dpdk, meson support dropped this.
>      > In the makefiles, I can't find a reason in the git history (we go
>      back to 1.3.0rX version).
>      >
>      > [dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
>      > kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
>      max-inline-insns-single=100
>      > kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param
>      max-inline-insns-single=50
>      > [dmarchan@dmarchan dpdk]$ git blame origin/master --
>      kernel/linux/igb_uio/Makefile |grep max-inline-insns-single
>      > 13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel
>       2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param
>      max-inline-insns-single=100
>      > [dmarchan@dmarchan dpdk]$ git blame origin/master --
>      kernel/linux/kni/Makefile |grep max-inline-insns-single
>      > 3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel
>      2012-12-20 00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param
>      max-inline-insns-single=50
>      >
>      > Is there a valid reason to keep this?
>      > 1:
>      [2]https://github.com/openvswitch/ovs/blob/master/.travis/linux-buil
>      d.sh#L81
>      <[3]https://protect2.fireeye.com/url?k=b7de159c45b1fa79.b7df9ed3-c48
>      06461f28ecaf5&u=https://github.com/openvswitch/ovs/blob/master/.trav
>      is/linux-build.sh#L81>
>      Hi, David.
>      I don't know the reason for these in OVS travis config, But we don't
>      need to know them, actually. I have a patch to drop all the kernel
>      related stuff from the DPDK build in OVS Travis checks, just didn't
>      send it yet. Will send soon, probably.
> 
>    I had this in mind since we don't need those kmods in the CI.
>    Thanks Ilya.
>    The question on dpdk side remains open :-).
>    --
>    David Marchand

I know that previously we did have issues with the modules not compiling
due to errors about maximum levels of inlines. Whether that was because of
the compiler, or the kernel makefiles at the time, I'm not sure. The kmods
seem to build fine for me now without the parameters, and the fact that
there has never been a problem reported with building using meson either,
probably indicates that the extra compiler flags can be dropped.

/Bruce

Re: [dpdk-dev] Forcing inlining for igb_uio and kni

[Stephen Hemminger]

On Tue, 11 Jun 2019 11:18:22 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Tue, Jun 11, 2019 at 11:44:01AM +0200, David Marchand wrote:
> >    On Tue, Jun 11, 2019 at 11:31 AM Ilya Maximets
> >    <[1]i.maximets@samsung.com> wrote:
> > 
> >      On 11.06.2019 11:45, David Marchand wrote:  
> >      > I noticed that OVS CI [1] patches the dpdk sources to force some  
> >      inlining parameters and get kni and igb_uio to build fine.  
> >      >
> >      > Looking at it in dpdk, meson support dropped this.
> >      > In the makefiles, I can't find a reason in the git history (we go  
> >      back to 1.3.0rX version).  
> >      >
> >      > [dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
> >      > kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param  
> >      max-inline-insns-single=100  
> >      > kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param  
> >      max-inline-insns-single=50  
> >      > [dmarchan@dmarchan dpdk]$ git blame origin/master --  
> >      kernel/linux/igb_uio/Makefile |grep max-inline-insns-single  
> >      > 13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel  
> >       2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param
> >      max-inline-insns-single=100  
> >      > [dmarchan@dmarchan dpdk]$ git blame origin/master --  
> >      kernel/linux/kni/Makefile |grep max-inline-insns-single  
> >      > 3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel  
> >      2012-12-20 00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param
> >      max-inline-insns-single=50  
> >      >
> >      > Is there a valid reason to keep this?
> >      > 1:  
> >      [2]https://github.com/openvswitch/ovs/blob/master/.travis/linux-buil
> >      d.sh#L81
> >      <[3]https://protect2.fireeye.com/url?k=b7de159c45b1fa79.b7df9ed3-c48
> >      06461f28ecaf5&u=https://github.com/openvswitch/ovs/blob/master/.trav
> >      is/linux-build.sh#L81>
> >      Hi, David.
> >      I don't know the reason for these in OVS travis config, But we don't
> >      need to know them, actually. I have a patch to drop all the kernel
> >      related stuff from the DPDK build in OVS Travis checks, just didn't
> >      send it yet. Will send soon, probably.
> > 
> >    I had this in mind since we don't need those kmods in the CI.
> >    Thanks Ilya.
> >    The question on dpdk side remains open :-).
> >    --
> >    David Marchand  
> 
> I know that previously we did have issues with the modules not compiling
> due to errors about maximum levels of inlines. Whether that was because of
> the compiler, or the kernel makefiles at the time, I'm not sure. The kmods
> seem to build fine for me now without the parameters, and the fact that
> there has never been a problem reported with building using meson either,
> probably indicates that the extra compiler flags can be dropped.
> 
> /Bruce

I really doubt forcing inlining makes any difference to the performance.

Re: [dpdk-dev] Forcing inlining for igb_uio and kni

[Bruce Richardson]

On Tue, Jun 11, 2019 at 07:25:09AM -0700, Stephen Hemminger wrote:
> On Tue, 11 Jun 2019 11:18:22 +0100
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > On Tue, Jun 11, 2019 at 11:44:01AM +0200, David Marchand wrote:
> > >    On Tue, Jun 11, 2019 at 11:31 AM Ilya Maximets
> > >    <[1]i.maximets@samsung.com> wrote:
> > > 
> > >      On 11.06.2019 11:45, David Marchand wrote:  
> > >      > I noticed that OVS CI [1] patches the dpdk sources to force some  
> > >      inlining parameters and get kni and igb_uio to build fine.  
> > >      >
> > >      > Looking at it in dpdk, meson support dropped this.
> > >      > In the makefiles, I can't find a reason in the git history (we go  
> > >      back to 1.3.0rX version).  
> > >      >
> > >      > [dmarchan@dmarchan dpdk]$ git grep max-inline-insns-single
> > >      > kernel/linux/igb_uio/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param  
> > >      max-inline-insns-single=100  
> > >      > kernel/linux/kni/Makefile:MODULE_CFLAGS += -I$(SRCDIR) --param  
> > >      max-inline-insns-single=50  
> > >      > [dmarchan@dmarchan dpdk]$ git blame origin/master --  
> > >      kernel/linux/igb_uio/Makefile |grep max-inline-insns-single  
> > >      > 13dc56a6 lib/librte_eal/linuxapp/igb_uio/Makefile (Intel  
> > >       2012-12-20 00:00:00 +0100 15) MODULE_CFLAGS += -I$(SRCDIR) --param
> > >      max-inline-insns-single=100  
> > >      > [dmarchan@dmarchan dpdk]$ git blame origin/master --  
> > >      kernel/linux/kni/Makefile |grep max-inline-insns-single  
> > >      > 3fc5ca2f lib/librte_eal/linuxapp/kni/Makefile (Intel  
> > >      2012-12-20 00:00:00 +0100 14) MODULE_CFLAGS += -I$(SRCDIR) --param
> > >      max-inline-insns-single=50  
> > >      >
> > >      > Is there a valid reason to keep this?
> > >      > 1:  
> > >      [2]https://github.com/openvswitch/ovs/blob/master/.travis/linux-buil
> > >      d.sh#L81
> > >      <[3]https://protect2.fireeye.com/url?k=b7de159c45b1fa79.b7df9ed3-c48
> > >      06461f28ecaf5&u=https://github.com/openvswitch/ovs/blob/master/.trav
> > >      is/linux-build.sh#L81>
> > >      Hi, David.
> > >      I don't know the reason for these in OVS travis config, But we don't
> > >      need to know them, actually. I have a patch to drop all the kernel
> > >      related stuff from the DPDK build in OVS Travis checks, just didn't
> > >      send it yet. Will send soon, probably.
> > > 
> > >    I had this in mind since we don't need those kmods in the CI.
> > >    Thanks Ilya.
> > >    The question on dpdk side remains open :-).
> > >    --
> > >    David Marchand  
> > 
> > I know that previously we did have issues with the modules not compiling
> > due to errors about maximum levels of inlines. Whether that was because of
> > the compiler, or the kernel makefiles at the time, I'm not sure. The kmods
> > seem to build fine for me now without the parameters, and the fact that
> > there has never been a problem reported with building using meson either,
> > probably indicates that the extra compiler flags can be dropped.
> > 
> > /Bruce
> 
> I really doubt forcing inlining makes any difference to the performance.

I agree, especially for uio module. This was not a "perf" issue, but a
"just own't compile" issue, IIRC. :-)

That being said, given that this is just a line in two makefiles, the rule
about "if it ain't broke" may apply....