Suggestion to use python-parameterized for QA

Suggestion to use python-parameterized for QA

[Vyacheslav Yurkov]

Hi guys,
I have a suggestion to add https://pypi.org/project/parameterized/ as a 
required package for QA selftests. Together with python unittest module 
it can be useful to increase code coverage and reduce duplication in 
case you want to do the same test with different input values.

Anything speaks against it?

Cheers,
Vyacheslav

Re: [OE-core] Suggestion to use python-parameterized for QA

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

It would be helpful to provide an example of improvements that can be
enabled with it.

Alex

On Mon, 15 Nov 2021 at 20:34, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> Hi guys,
> I have a suggestion to add https://pypi.org/project/parameterized/ as a
> required package for QA selftests. Together with python unittest module
> it can be useful to increase code coverage and reduce duplication in
> case you want to do the same test with different input values.
>
> Anything speaks against it?
>
> Cheers,
> Vyacheslav
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#158309):
> https://lists.openembedded.org/g/openembedded-core/message/158309
> Mute This Topic: https://lists.openembedded.org/mt/87078063/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 1888 bytes --]

Re: [OE-core] Suggestion to use python-parameterized for QA

[Vyacheslav Yurkov]

This is still work-in-progress, but I'm working on overlayfs-etc class, 
which would have different behavior depending on an input variable. You 
can see the example here

https://github.com/UVV-gh/openembedded-core/commit/ea424efb7780dc3c85e329b36f4a675b56e86877#diff-9980b9625344e11f15451ab7c97fb1f137e94b68fa27df737c576b75e313996aR215

Regardless of the input parameter the test would be the same and should 
work.

What do you think?

Vyacheslav

On 15.11.2021 20:38, Alexander Kanavin wrote:
> It would be helpful to provide an example of improvements that can be 
> enabled with it.
>
> Alex

Re: [OE-core] Suggestion to use python-parameterized for QA

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 896 bytes --]

I can't understand what is supposed to happen, perhaps because it's
incomplete? There's

@parameterized.expand([
("original", True),
("preinit", False),
])

but where is that used?

Alex

On Mon, 15 Nov 2021 at 20:53, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> This is still work-in-progress, but I'm working on overlayfs-etc class,
> which would have different behavior depending on an input variable. You
> can see the example here
>
>
> https://github.com/UVV-gh/openembedded-core/commit/ea424efb7780dc3c85e329b36f4a675b56e86877#diff-9980b9625344e11f15451ab7c97fb1f137e94b68fa27df737c576b75e313996aR215
>
> Regardless of the input parameter the test would be the same and should
> work.
>
> What do you think?
>
> Vyacheslav
>
> On 15.11.2021 20:38, Alexander Kanavin wrote:
> > It would be helpful to provide an example of improvements that can be
> > enabled with it.
> >
> > Alex
>
>

[-- Attachment #2: Type: text/html, Size: 3252 bytes --]

Re: [OE-core] Suggestion to use python-parameterized for QA

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

It's substituted further down into
def test_sbin_init(self, _, origInit):

parameterized.expand generates two tests out of it. The string parameter 
is used as a suffix for the test, the rest is used as input parameters, 
i.e. the first test would be called with origInit=True, the second one 
with origInit=False.

The sample output looks like this:

2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - 
overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original: PASSED 
(295.73s)
2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS - 
overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit: PASSED 
(177.97s)

Vyacheslav

On 15.11.2021 21:20, Alexander Kanavin wrote:
> I can't understand what is supposed to happen, perhaps because it's 
> incomplete? There's
>
> @parameterized.expand([
>
> ("original", True),
>
> ("preinit", False),
>
> ])
>
> but where is that used?
>
> Alex
>

[-- Attachment #2: Type: text/html, Size: 4189 bytes --]

Re: [OE-core] Suggestion to use python-parameterized for QA

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

Can't we achieve the same by abstracting the common part into a function
and defining the two tests explicitly?

Alex

On Mon, 15 Nov 2021 at 21:26, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> It's substituted further down into
> def test_sbin_init(self, _, origInit):
>
> parameterized.expand generates two tests out of it. The string parameter
> is used as a suffix for the test, the rest is used as input parameters,
> i.e. the first test would be called with origInit=True, the second one with
> origInit=False.
>
> The sample output looks like this:
>
> 2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -
> overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original: PASSED
> (295.73s)
> 2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -
> overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit: PASSED
> (177.97s)
>
> Vyacheslav
>
> On 15.11.2021 21:20, Alexander Kanavin wrote:
>
> I can't understand what is supposed to happen, perhaps because it's
> incomplete? There's
>
> @parameterized.expand([
> ("original", True),
> ("preinit", False),
> ])
>
> but where is that used?
>
> Alex
>
>
>

[-- Attachment #2: Type: text/html, Size: 3037 bytes --]

Re: [OE-core] Suggestion to use python-parameterized for QA

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 1051 bytes --]

Yes, technically it would be the same. Thanks for the hint.

Vyacheslav

On 15.11.2021 21:56, Alexander Kanavin wrote:
> Can't we achieve the same by abstracting the common part into a 
> function and defining the two tests explicitly?
>
> Alex
>
> On Mon, 15 Nov 2021 at 21:26, Vyacheslav Yurkov <uvv.mail@gmail.com> 
> wrote:
>
>     It's substituted further down into
>     def test_sbin_init(self, _, origInit):
>
>     parameterized.expand generates two tests out of it. The string
>     parameter is used as a suffix for the test, the rest is used as
>     input parameters, i.e. the first test would be called with
>     origInit=True, the second one with origInit=False.
>
>     The sample output looks like this:
>
>     2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -
>     overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_0_original:
>     PASSED (295.73s)
>     2021-11-15 20:21:15,740 - oe-selftest - INFO - RESULTS -
>     overlayfs.OverlayFSEtcRunTimeTests.test_sbin_init_1_preinit:
>     PASSED (177.97s)
>
>     Vyacheslav
>

[-- Attachment #2: Type: text/html, Size: 2380 bytes --]