Python tests

Python tests

[Lukasz Florczak]

Hi Coly,
I want to write some mdadm tests for assemblation and incremental
regarding duplicated array names in config and I'd like to do it in
python. I've seen that some time ago[1] you said that you could try to
integrate the python tests framework into the mdadm ci. I was wondering
how is it going? Do you need any help with this subject?

Thanks,
Lukasz

[1] https://marc.info/?l=linux-raid&m=165277539509464&w=2

Re: Python tests

[Coly Li]

> 2022年8月24日 16:15，Lukasz Florczak <lukasz.florczak@linux.intel.com> 写道：
> 
> Hi Coly,
> I want to write some mdadm tests for assemblation and incremental
> regarding duplicated array names in config and I'd like to do it in
> python. I've seen that some time ago[1] you said that you could try to
> integrate the python tests framework into the mdadm ci. I was wondering
> how is it going? Do you need any help with this subject?
> 
> Thanks,
> Lukasz
> 
> [1] https://marc.info/?l=linux-raid&m=165277539509464&w=2

Hi Lukasz,

Now I just make some of the existed mdadm test scripts running, which are copied from upstream mdadm. There won’t be any conflict for the python testing code between you and me, because now I am just studying Python again and not do any useful thing yet.

As I said if no one works on the testing framework, I will do it, but it may take time. How about posting out the python code once you make it, then let’s put it into mdadm-test to test mdadm-CI, and improve whole things step by step.

Thanks.

Coly Li

Re: Python tests

[Logan Gunthorpe]

Hi,

On 2022-08-24 10:23, Coly Li wrote:
> 
> 
>> 2022年8月24日 16:15，Lukasz Florczak <lukasz.florczak@linux.intel.com> 写道：
>>
>> Hi Coly,
>> I want to write some mdadm tests for assemblation and incremental
>> regarding duplicated array names in config and I'd like to do it in
>> python. I've seen that some time ago[1] you said that you could try to
>> integrate the python tests framework into the mdadm ci. I was wondering
>> how is it going? Do you need any help with this subject?
>>
>> Thanks,
>> Lukasz
>>
>> [1] https://marc.info/?l=linux-raid&m=165277539509464&w=2
> 
> Hi Lukasz,
> 
> Now I just make some of the existed mdadm test scripts running, which are copied from upstream mdadm. There won’t be any conflict for the python testing code between you and me, because now I am just studying Python again and not do any useful thing yet.
> 
> As I said if no one works on the testing framework, I will do it, but it may take time. How about posting out the python code once you make it, then let’s put it into mdadm-test to test mdadm-CI, and improve whole things step by step.
> 

I'm not sure if this is of use to anyone but we are very slowly growing
a testing framework written mostly in python. Its focused on raid5 at
the moment and still is a fairly sizable mess, but we've caught a lot of
bugs with it and continue to run it, clean it up and make improvements.

https://github.com/Eideticom/raid5-tests/

The 'md.py 'file provides a nice interface to setup an array based on
ram, loop or block devices and provides methods to degrade, recover or
grow the array. 'test3' does grow/degrade tests while running IO,
'test_all' runs all the tests with an array of different settings.

Feel free to use anything from it that you may find useful.

Logan

Re: Python tests

[Hannes Reinecke]

On 8/24/22 22:40, Logan Gunthorpe wrote:
> Hi,
> 
> On 2022-08-24 10:23, Coly Li wrote:
>>
>>
>>> 2022年8月24日 16:15，Lukasz Florczak <lukasz.florczak@linux.intel.com> 写道：
>>>
>>> Hi Coly,
>>> I want to write some mdadm tests for assemblation and incremental
>>> regarding duplicated array names in config and I'd like to do it in
>>> python. I've seen that some time ago[1] you said that you could try to
>>> integrate the python tests framework into the mdadm ci. I was wondering
>>> how is it going? Do you need any help with this subject?
>>>
>>> Thanks,
>>> Lukasz
>>>
>>> [1] https://marc.info/?l=linux-raid&m=165277539509464&w=2
>>
>> Hi Lukasz,
>>
>> Now I just make some of the existed mdadm test scripts running, which are copied from upstream mdadm. There won’t be any conflict for the python testing code between you and me, because now I am just studying Python again and not do any useful thing yet.
>>
>> As I said if no one works on the testing framework, I will do it, but it may take time. How about posting out the python code once you make it, then let’s put it into mdadm-test to test mdadm-CI, and improve whole things step by step.
>>
> 
> I'm not sure if this is of use to anyone but we are very slowly growing
> a testing framework written mostly in python. Its focused on raid5 at
> the moment and still is a fairly sizable mess, but we've caught a lot of
> bugs with it and continue to run it, clean it up and make improvements.
> 
> https://github.com/Eideticom/raid5-tests/
> 
> The 'md.py 'file provides a nice interface to setup an array based on
> ram, loop or block devices and provides methods to degrade, recover or
> grow the array. 'test3' does grow/degrade tests while running IO,
> 'test_all' runs all the tests with an array of different settings.
> 
> Feel free to use anything from it that you may find useful.
> 
When developing 'md_monitor' (https://github.com/hreinecke/md_monitor) 
I've also created an extensive testsuite for it.
The one thing which I found particularly painful is error handling once 
mdadm fails. It's really hard to figure out _what_ went wrong, and more 
often than not mdadm simply locked up on me (try to stall I/O on one 
component device while md is running and you are in a world of pain).

That's when I started to split mdadm into a library, as then we could 
have a python binding and the life would so much more fun.
So maybe I should resurrect that patchset.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman

Re: Python tests

[Mariusz Tkaczyk]

On Thu, 25 Aug 2022 08:15:14 +0200
Hannes Reinecke <hare@suse.de> wrote:

> On 8/24/22 22:40, Logan Gunthorpe wrote:
> > Hi,
> > 
> > On 2022-08-24 10:23, Coly Li wrote:  
> >>
> >>  
> >>> 2022年8月24日 16:15，Lukasz Florczak <lukasz.florczak@linux.intel.com>
> >>> 写道：
> >>>
> >>> Hi Coly,
> >>> I want to write some mdadm tests for assemblation and incremental
> >>> regarding duplicated array names in config and I'd like to do it in
> >>> python. I've seen that some time ago[1] you said that you could try to
> >>> integrate the python tests framework into the mdadm ci. I was wondering
> >>> how is it going? Do you need any help with this subject?
> >>>
> >>> Thanks,
> >>> Lukasz
> >>>
> >>> [1] https://marc.info/?l=linux-raid&m=165277539509464&w=2  
> >>
> >> Hi Lukasz,
> >>
> >> Now I just make some of the existed mdadm test scripts running, which are
> >> copied from upstream mdadm. There won’t be any conflict for the python
> >> testing code between you and me, because now I am just studying Python
> >> again and not do any useful thing yet.
> >>
> >> As I said if no one works on the testing framework, I will do it, but it
> >> may take time. How about posting out the python code once you make it,
> >> then let’s put it into mdadm-test to test mdadm-CI, and improve whole
> >> things step by step. 
> > 
> > I'm not sure if this is of use to anyone but we are very slowly growing
> > a testing framework written mostly in python. Its focused on raid5 at
> > the moment and still is a fairly sizable mess, but we've caught a lot of
> > bugs with it and continue to run it, clean it up and make improvements.
> > 
> > https://github.com/Eideticom/raid5-tests/
> > 
> > The 'md.py 'file provides a nice interface to setup an array based on
> > ram, loop or block devices and provides methods to degrade, recover or
> > grow the array. 'test3' does grow/degrade tests while running IO,
> > 'test_all' runs all the tests with an array of different settings.
> > 
> > Feel free to use anything from it that you may find useful.
> >   
> When developing 'md_monitor' (https://github.com/hreinecke/md_monitor) 
> I've also created an extensive testsuite for it.
> The one thing which I found particularly painful is error handling once 
> mdadm fails. It's really hard to figure out _what_ went wrong, and more 
> often than not mdadm simply locked up on me (try to stall I/O on one 
> component device while md is running and you are in a world of pain).
> 
> That's when I started to split mdadm into a library, as then we could 
> have a python binding and the life would so much more fun.
> So maybe I should resurrect that patchset.
> 
Hi Hannes,

Will be great if you can handle this. Beside library part you did huge code
rearrangement and divides one big mdadm.h header into separate ones which is
also great improvement. mdadm.h is still growing.

It is hard to do this separately, one by one due to cross references between
files.

Thanks,
Mariusz

Re: Python tests

[Lukasz Florczak]

On Fri, 26 Aug 2022 10:12:42 +0200, Mariusz Tkaczyk
<mariusz.tkaczyk@linux.intel.com> wrote:

> On Thu, 25 Aug 2022 08:15:14 +0200
> Hannes Reinecke <hare@suse.de> wrote:
> 
> > On 8/24/22 22:40, Logan Gunthorpe wrote:  
> > > Hi,
> > > 
> > > On 2022-08-24 10:23, Coly Li wrote:    
> > >>
> > >>    
> > >>> 2022年8月24日 16:15，Lukasz Florczak
> > >>> <lukasz.florczak@linux.intel.com> 写道：
> > >>>
> > >>> Hi Coly,
> > >>> I want to write some mdadm tests for assemblation and
> > >>> incremental regarding duplicated array names in config and I'd
> > >>> like to do it in python. I've seen that some time ago[1] you
> > >>> said that you could try to integrate the python tests framework
> > >>> into the mdadm ci. I was wondering how is it going? Do you need
> > >>> any help with this subject?
> > >>>
> > >>> Thanks,
> > >>> Lukasz
> > >>>
> > >>> [1] https://marc.info/?l=linux-raid&m=165277539509464&w=2    
> > >>
> > >> Hi Lukasz,
> > >>
> > >> Now I just make some of the existed mdadm test scripts running,
> > >> which are copied from upstream mdadm. There won’t be any
> > >> conflict for the python testing code between you and me, because
> > >> now I am just studying Python again and not do any useful thing
> > >> yet.
> > >>
> > >> As I said if no one works on the testing framework, I will do
> > >> it, but it may take time. How about posting out the python code
> > >> once you make it, then let’s put it into mdadm-test to test
> > >> mdadm-CI, and improve whole things step by step.   
> > > 
> > > I'm not sure if this is of use to anyone but we are very slowly
> > > growing a testing framework written mostly in python. Its focused
> > > on raid5 at the moment and still is a fairly sizable mess, but
> > > we've caught a lot of bugs with it and continue to run it, clean
> > > it up and make improvements.
> > > 
> > > https://github.com/Eideticom/raid5-tests/
> > > 
> > > The 'md.py 'file provides a nice interface to setup an array
> > > based on ram, loop or block devices and provides methods to
> > > degrade, recover or grow the array. 'test3' does grow/degrade
> > > tests while running IO, 'test_all' runs all the tests with an
> > > array of different settings.
> > > 
> > > Feel free to use anything from it that you may find useful.
> > >     
> > When developing 'md_monitor'
> > (https://github.com/hreinecke/md_monitor) I've also created an
> > extensive testsuite for it. The one thing which I found
> > particularly painful is error handling once mdadm fails. It's
> > really hard to figure out _what_ went wrong, and more often than
> > not mdadm simply locked up on me (try to stall I/O on one component
> > device while md is running and you are in a world of pain).
> > 
> > That's when I started to split mdadm into a library, as then we
> > could have a python binding and the life would so much more fun.
> > So maybe I should resurrect that patchset.
> >   
> Hi Hannes,
> 
> Will be great if you can handle this. Beside library part you did
> huge code rearrangement and divides one big mdadm.h header into
> separate ones which is also great improvement. mdadm.h is still
> growing.
> 
> It is hard to do this separately, one by one due to cross references
> between files.
> 
> Thanks,
> Mariusz

Hi,
I'm writing with a little update. I was trying to work with python
libraries pytest and ctypes. They look promising to me as with them
you can load shared objects and create unit tests. Unfortunately I
didn't manage to compile mdadm sources as library yet.
I don't think I can manage to work more on this topic anymore due to
structural changes on my team at work, so this must be postponed from
my side. Maybe one of my collegues will try to take this subject
further in the near future. 

Thanks,
Lukasz