[dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[Juraj Linkeš]

Tests requiring hugepages do not work outside of VM environment because
of security limitations. Enable aarch64 builds which run tests to run in
a VM to avoid these limitations.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 .travis.yml | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index d6eeab371..d73d3e4c5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -88,7 +88,10 @@ jobs:
     arch: arm64
     compiler: gcc
   - env: DEF_LIB="shared" RUN_TESTS=1
-    arch: arm64
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
     compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
@@ -103,5 +106,8 @@ jobs:
     arch: arm64
     compiler: clang
   - env: DEF_LIB="shared" RUN_TESTS=1
-    arch: arm64
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
     compiler: clang
-- 
2.20.1

Re: [dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[Aaron Conole]

Juraj Linkeš <juraj.linkes@pantheon.tech> writes:

> Tests requiring hugepages do not work outside of VM environment because
> of security limitations. Enable aarch64 builds which run tests to run in
> a VM to avoid these limitations.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---

Cool!

Globally we have dist: set to 'bionic'.  Does it make sense to upgrade all
the distributions to 'focal' ?  What is the rationale for not using
'bionic'?

>  .travis.yml | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index d6eeab371..d73d3e4c5 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -88,7 +88,10 @@ jobs:
>      arch: arm64
>      compiler: gcc
>    - env: DEF_LIB="shared" RUN_TESTS=1
> -    arch: arm64
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
>      compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
> @@ -103,5 +106,8 @@ jobs:
>      arch: arm64
>      compiler: clang
>    - env: DEF_LIB="shared" RUN_TESTS=1
> -    arch: arm64
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
>      compiler: clang

Re: [dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[Juraj Linkeš]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Tuesday, August 25, 2020 4:18 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> maicolgabriel@hotmail.com; dev@dpdk.org; Kevin Laatz
> <kevin.laatz@intel.com>
> Subject: Re: [PATCH] ci: enable vm env for aarch64 builds
> 
> Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
> 
> > Tests requiring hugepages do not work outside of VM environment
> > because of security limitations. Enable aarch64 builds which run tests
> > to run in a VM to avoid these limitations.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > ---
> 
> Cool!
> 
> Globally we have dist: set to 'bionic'.  Does it make sense to upgrade all the
> distributions to 'focal' ?  What is the rationale for not using 'bionic'?
> 

I tried bionic, but it ran with focal anyway, so the rationale is 'not supported for arm vm environments'. This was released for early adopters, so they may expand the support to bionic later.

> >  .travis.yml | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/.travis.yml b/.travis.yml index d6eeab371..d73d3e4c5
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -88,7 +88,10 @@ jobs:
> >      arch: arm64
> >      compiler: gcc
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> > -    arch: arm64
> > +    dist: focal
> > +    arch: arm64-graviton2
> > +    virt: vm
> > +    group: edge
> >      compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: arm64
> > @@ -103,5 +106,8 @@ jobs:
> >      arch: arm64
> >      compiler: clang
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> > -    arch: arm64
> > +    dist: focal
> > +    arch: arm64-graviton2
> > +    virt: vm
> > +    group: edge
> >      compiler: clang
> 

Re: [dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[Juraj Linkeš]

> -----Original Message-----
> From: Juraj Linkeš
> Sent: Wednesday, August 26, 2020 8:14 AM
> To: 'Aaron Conole' <aconole@redhat.com>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> maicolgabriel@hotmail.com; dev@dpdk.org; Kevin Laatz
> <kevin.laatz@intel.com>
> Subject: RE: [PATCH] ci: enable vm env for aarch64 builds
> 
> 
> 
> > -----Original Message-----
> > From: Aaron Conole <aconole@redhat.com>
> > Sent: Tuesday, August 25, 2020 4:18 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > maicolgabriel@hotmail.com; dev@dpdk.org; Kevin Laatz
> > <kevin.laatz@intel.com>
> > Subject: Re: [PATCH] ci: enable vm env for aarch64 builds
> >
> > Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
> >
> > > Tests requiring hugepages do not work outside of VM environment
> > > because of security limitations. Enable aarch64 builds which run
> > > tests to run in a VM to avoid these limitations.
> > >
> > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > > ---
> >
> > Cool!
> >
> > Globally we have dist: set to 'bionic'.  Does it make sense to upgrade
> > all the distributions to 'focal' ?  What is the rationale for not using 'bionic'?
> >
> 
> I tried bionic, but it ran with focal anyway, so the rationale is 'not supported for
> arm vm environments'. This was released for early adopters, so they may
> expand the support to bionic later.
> 

Couple more things. Since this new VM environment runs only on Focal, we want to leave the old jobs and add these new jobs, so we'll have bionic without hugepages and focal with hugepages. The reason is that the same test behave differently when run on a hungepage and non-hugepage environments. Does that make sense? Maybe having a non-hugepage job for just one compiler would be enough?

The other thing is switching to focal for all jobs. The difference would be in different lib and compiler versions but I don't know whether we want to dig deep into that or we just want to go with the latest or what's the most important thing to consider. How did you decide in the past?

> > >  .travis.yml | 10 ++++++++--
> > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/.travis.yml b/.travis.yml index d6eeab371..d73d3e4c5
> > > 100644
> > > --- a/.travis.yml
> > > +++ b/.travis.yml
> > > @@ -88,7 +88,10 @@ jobs:
> > >      arch: arm64
> > >      compiler: gcc
> > >    - env: DEF_LIB="shared" RUN_TESTS=1
> > > -    arch: arm64
> > > +    dist: focal
> > > +    arch: arm64-graviton2
> > > +    virt: vm
> > > +    group: edge
> > >      compiler: gcc
> > >    - env: DEF_LIB="shared" BUILD_DOCS=1
> > >      arch: arm64
> > > @@ -103,5 +106,8 @@ jobs:
> > >      arch: arm64
> > >      compiler: clang
> > >    - env: DEF_LIB="shared" RUN_TESTS=1
> > > -    arch: arm64
> > > +    dist: focal
> > > +    arch: arm64-graviton2
> > > +    virt: vm
> > > +    group: edge
> > >      compiler: clang
> >

[dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests

[Juraj Linkeš]

Tests requiring hugepages do not work outside of VM environment because
of security limitations. Add aarch64 builds which run tests to run in
a VM to avoid these limitations. Leave non-hugepage environments since
the tests may produce different results in hugepage and non-hugepage
environments.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 .travis.yml | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index d6eeab371..5e12db23b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -90,6 +90,12 @@ jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: arm64
     compiler: gcc
+  - env: DEF_LIB="shared" RUN_TESTS=1
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
+    compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
     compiler: gcc
@@ -105,3 +111,9 @@ jobs:
   - env: DEF_LIB="shared" RUN_TESTS=1
     arch: arm64
     compiler: clang
+  - env: DEF_LIB="shared" RUN_TESTS=1
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
+    compiler: clang
-- 
2.20.1

Re: [dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests

[Ruifeng Wang]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Juraj Linke?
> Sent: Friday, August 28, 2020 7:46 PM
> To: thomas@monjalon.net; david.marchand@redhat.com;
> aconole@redhat.com; maicolgabriel@hotmail.com
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests
> 
> Tests requiring hugepages do not work outside of VM environment because
> of security limitations. Add aarch64 builds which run tests to run in a VM to
> avoid these limitations. Leave non-hugepage environments since the tests
> may produce different results in hugepage and non-hugepage environments.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  .travis.yml | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/.travis.yml b/.travis.yml
> index d6eeab371..5e12db23b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -90,6 +90,12 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: arm64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS=1
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
> +    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -105,3 +111,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: arm64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS=1
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
> +    compiler: clang
> --
> 2.20.1

Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

Re: [dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests

[Aaron Conole]

Juraj Linkeš <juraj.linkes@pantheon.tech> writes:

> Tests requiring hugepages do not work outside of VM environment because
> of security limitations. Add aarch64 builds which run tests to run in
> a VM to avoid these limitations. Leave non-hugepage environments since
> the tests may produce different results in hugepage and non-hugepage
> environments.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---

I'm okay with the change, but I wonder - does it make sense to have the
non-vm build around with these?  The builds you're adding cover all the
cases of the container build, and add the additional hugepage tests.

Maybe we should drop the other tests?  It can be a follow up patch, WDYT?

Acked-by: Aaron Conole <aconole@redhat.com>

>  .travis.yml | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index d6eeab371..5e12db23b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -90,6 +90,12 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: arm64
>      compiler: gcc
> +  - env: DEF_LIB="shared" RUN_TESTS=1
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
> +    compiler: gcc
>    - env: DEF_LIB="shared" BUILD_DOCS=1
>      arch: arm64
>      compiler: gcc
> @@ -105,3 +111,9 @@ jobs:
>    - env: DEF_LIB="shared" RUN_TESTS=1
>      arch: arm64
>      compiler: clang
> +  - env: DEF_LIB="shared" RUN_TESTS=1
> +    dist: focal
> +    arch: arm64-graviton2
> +    virt: vm
> +    group: edge
> +    compiler: clang

Re: [dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests

[Juraj Linkeš]

> -----Original Message-----
> From: Aaron Conole <aconole@redhat.com>
> Sent: Friday, September 11, 2020 3:25 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> maicolgabriel@hotmail.com; dev@dpdk.org
> Subject: Re: [PATCH v2] ci: add vm jobs for aarch64 builds with tests
> 
> Juraj Linkeš <juraj.linkes@pantheon.tech> writes:
> 
> > Tests requiring hugepages do not work outside of VM environment
> > because of security limitations. Add aarch64 builds which run tests to
> > run in a VM to avoid these limitations. Leave non-hugepage
> > environments since the tests may produce different results in hugepage
> > and non-hugepage environments.
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > ---
> 
> I'm okay with the change, but I wonder - does it make sense to have the non-vm
> build around with these?  The builds you're adding cover all the cases of the
> container build, and add the additional hugepage tests.
> 

Well, the commit message explains why I left the non-vm builds. The test cases may be the same, but the environment is not. We were getting different results when running the same tests with and without huge pages.

> Maybe we should drop the other tests?  It can be a follow up patch, WDYT?

We could, but we'd want to add to run all tests with hugepages both enabled and not enabled. That would be a bigger change I think, but it should be possible, right?

> 
> Acked-by: Aaron Conole <aconole@redhat.com>
> 
> >  .travis.yml | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/.travis.yml b/.travis.yml index d6eeab371..5e12db23b
> > 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -90,6 +90,12 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: arm64
> >      compiler: gcc
> > +  - env: DEF_LIB="shared" RUN_TESTS=1
> > +    dist: focal
> > +    arch: arm64-graviton2
> > +    virt: vm
> > +    group: edge
> > +    compiler: gcc
> >    - env: DEF_LIB="shared" BUILD_DOCS=1
> >      arch: arm64
> >      compiler: gcc
> > @@ -105,3 +111,9 @@ jobs:
> >    - env: DEF_LIB="shared" RUN_TESTS=1
> >      arch: arm64
> >      compiler: clang
> > +  - env: DEF_LIB="shared" RUN_TESTS=1
> > +    dist: focal
> > +    arch: arm64-graviton2
> > +    virt: vm
> > +    group: edge
> > +    compiler: clang
> 

Re: [dpdk-dev] [PATCH v2] ci: add vm jobs for aarch64 builds with tests

[David Marchand]

On Fri, Aug 28, 2020 at 1:45 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>
> Tests requiring hugepages do not work outside of VM environment because
> of security limitations. Add aarch64 builds which run tests to run in
> a VM to avoid these limitations. Leave non-hugepage environments since
> the tests may produce different results in hugepage and non-hugepage
> environments.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Aaron Conole <aconole@redhat.com>


Applied, thanks.


-- 
David Marchand

[dpdk-dev] [PATCH] ci: enable vm env for aarch64 builds

[Juraj Linkeš]

Tests requiring hugepages do not work outside of VM environment because
of security limitations. Enable aarch64 builds which run tests to run in
a VM to avoid these limitations.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 .travis.yml | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index d6eeab371..d73d3e4c5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -88,7 +88,10 @@ jobs:
     arch: arm64
     compiler: gcc
   - env: DEF_LIB="shared" RUN_TESTS=1
-    arch: arm64
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
     compiler: gcc
   - env: DEF_LIB="shared" BUILD_DOCS=1
     arch: arm64
@@ -103,5 +106,8 @@ jobs:
     arch: arm64
     compiler: clang
   - env: DEF_LIB="shared" RUN_TESTS=1
-    arch: arm64
+    dist: focal
+    arch: arm64-graviton2
+    virt: vm
+    group: edge
     compiler: clang
-- 
2.20.1