[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Lukasz Majewski]

This commit adds test script for testing if any commit has introduced
regression to the DFU.

It uses md5 to test if sent and received file is correct.
The test detailed description is available at DESCRIPTION.TXT file.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
---
 test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++
 test/dfu/dfu_gadget_test.sh |   86 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 113 insertions(+)
 create mode 100644 test/dfu/DESCRIPTION.TXT
 create mode 100755 test/dfu/dfu_gadget_test.sh

diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
new file mode 100644
index 0000000..48da06b
--- /dev/null
+++ b/test/dfu/DESCRIPTION.TXT
@@ -0,0 +1,27 @@
+DFU TEST CASE DESCRIPTION:
+
+For running test script one needs to create:
+./log and ./bkp
+
+One also need to generate with dd following files:
+
+dat_*
+
+(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img  dat_4095B.img
+dat_4096B.img  dat_4097B.img  dat_63B.img  dat_64B.img  dat_65B.img
+dat_960.img  dat_97M.img)
+
+The size is only important (so if=/dev/urandom), especially corner cases of
+DFU EP0's size of packet (64B). This is very helpful for fixing UDC driver
+regressions.
+
+Moreover on a target device the "dfu_alt_info" env variable should be extended
+to have "dfu_test.bin fat 0 6;" \ entry [1].
+One can use fat, ext4 or any other supported file system.
+
+As a prerequisite one must also create proper partition with file system on it.
+
+Example usage:
+./dfu_gadget_test.sh 11
+
+where 11 is the mumber of alt setting corresponding to entry [1].
\ No newline at end of file
diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
new file mode 100755
index 0000000..ebde2ff
--- /dev/null
+++ b/test/dfu/dfu_gadget_test.sh
@@ -0,0 +1,86 @@
+#! /bin/bash
+set -e # any command return not equal to zero
+clear
+
+DIR=./
+SUFFIX=img
+RCV_DIR=rcv/
+LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
+
+cleanup () {
+    rm -rf $RCV_DIR
+}
+
+die () {
+	printf "   \33[31m FAILED \33[0m \n"
+	cleanup
+	exit 1
+}
+
+calculate_md5sum () {
+    MD5SUM=`md5sum $1`
+    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
+    echo "md5sum:"$MD5SUM
+}
+
+dfu_test_file () {
+    printf "\33[32m========================================================================================= \33[0m\n"
+    printf "File:\33[32m %s \33[0m\n" $1
+
+    # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1
+    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "TX: "
+    calculate_md5sum $1
+
+    MD5_TX=$MD5SUM
+
+    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
+
+    # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1
+    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "RX: "
+    calculate_md5sum $N_FILE
+    MD5_RX=$MD5SUM
+
+    if [ "$MD5_TX" == "$MD5_RX" ]
+	then
+	printf "   \33[32m -------> OK \33[0m \n"
+	else
+	printf "   \33[31m -------> FAILED \33[0m \n"
+	cleanup
+	exit 1
+    fi
+
+#    echo $N_FILE
+}
+
+printf "\33[32m========================================================================================= \33[0m\n"
+echo "DFU EP0 transmission test program"
+echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
+echo "@ -> TRATS # dfu mmc 0"
+mkdir -p $RCV_DIR
+touch $LOG_FILE
+
+if [ $# -eq 0 ]
+then
+	printf "   \33[31m Please pass alt setting number!!  \33[0m \n"
+	exit 0
+fi
+
+TARGET_ALT_SETTING=$1
+
+if [ -n "$2" ]
+then
+	dfu_test_file $2
+else
+	for file in $DIR*.$SUFFIX
+	do
+	    dfu_test_file $file
+	done
+fi
+
+cleanup
+
+exit 0
\ No newline at end of file
-- 
1.7.10.4

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Simon Glass]

Hi Lukasz,

On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote:
> This commit adds test script for testing if any commit has introduced
> regression to the DFU.
>
> It uses md5 to test if sent and received file is correct.
> The test detailed description is available at DESCRIPTION.TXT file.
>
> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> ---
>  test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++

This sounds like an important and useful test, but I have a few
comments (coming from someone with little knowledge of dfu)

Would README be a better name?

>  test/dfu/dfu_gadget_test.sh |   86 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 113 insertions(+)
>  create mode 100644 test/dfu/DESCRIPTION.TXT
>  create mode 100755 test/dfu/dfu_gadget_test.sh
>
> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
> new file mode 100644
> index 0000000..48da06b
> --- /dev/null
> +++ b/test/dfu/DESCRIPTION.TXT
> @@ -0,0 +1,27 @@
> +DFU TEST CASE DESCRIPTION:
> +
> +For running test script one needs to create:
> +./log and ./bkp
> +
> +One also need to generate with dd following files:
> +
> +dat_*
> +
> +(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img  dat_4095B.img
> +dat_4096B.img  dat_4097B.img  dat_63B.img  dat_64B.img  dat_65B.img
> +dat_960.img  dat_97M.img)

How do you do that? Is this intended for use with a particular board?
I think you are missing some info here.

> +
> +The size is only important (so if=/dev/urandom), especially corner cases of
> +DFU EP0's size of packet (64B). This is very helpful for fixing UDC driver
> +regressions.
> +
> +Moreover on a target device the "dfu_alt_info" env variable should be extended
> +to have "dfu_test.bin fat 0 6;" \ entry [1].

What is [1] ?

> +One can use fat, ext4 or any other supported file system.
> +
> +As a prerequisite one must also create proper partition with file system on it.
> +
> +Example usage:
> +./dfu_gadget_test.sh 11
> +
> +where 11 is the mumber of alt setting corresponding to entry [1].
> \ No newline at end of file

I'm really none-the-wiser as to how this works - I think you need to
add a bit more detail.

> diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
> new file mode 100755
> index 0000000..ebde2ff
> --- /dev/null
> +++ b/test/dfu/dfu_gadget_test.sh
> @@ -0,0 +1,86 @@
> +#! /bin/bash
> +set -e # any command return not equal to zero
> +clear
> +
> +DIR=./
> +SUFFIX=img
> +RCV_DIR=rcv/
> +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> +
> +cleanup () {
> +    rm -rf $RCV_DIR
> +}
> +
> +die () {
> +       printf "   \33[31m FAILED \33[0m \n"

Perhaps define $RED or whatever at the top and use those here?

> +       cleanup
> +       exit 1
> +}
> +
> +calculate_md5sum () {
> +    MD5SUM=`md5sum $1`
> +    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> +    echo "md5sum:"$MD5SUM
> +}
> +
> +dfu_test_file () {

I suggest putting the argument in a variable here, instead of using $1
throughout the function.

> +    printf "\33[32m========================================================================================= \33[0m\n"
> +    printf "File:\33[32m %s \33[0m\n" $1
> +
> +    # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1

Remove this line?

> +    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
> +
> +    echo -n "TX: "
> +    calculate_md5sum $1
> +
> +    MD5_TX=$MD5SUM
> +
> +    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> +
> +    # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1

And again

> +    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
> +
> +    echo -n "RX: "
> +    calculate_md5sum $N_FILE
> +    MD5_RX=$MD5SUM
> +
> +    if [ "$MD5_TX" == "$MD5_RX" ]
> +       then
> +       printf "   \33[32m -------> OK \33[0m \n"
> +       else

outdent

> +       printf "   \33[31m -------> FAILED \33[0m \n"
> +       cleanup
> +       exit 1
> +    fi
> +
> +#    echo $N_FILE
> +}
> +
> +printf "\33[32m========================================================================================= \33[0m\n"
> +echo "DFU EP0 transmission test program"
> +echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
> +echo "@ -> TRATS # dfu mmc 0"
> +mkdir -p $RCV_DIR
> +touch $LOG_FILE
> +
> +if [ $# -eq 0 ]
> +then
> +       printf "   \33[31m Please pass alt setting number!!  \33[0m \n"
> +       exit 0
> +fi
> +
> +TARGET_ALT_SETTING=$1
> +
> +if [ -n "$2" ]
> +then
> +       dfu_test_file $2
> +else
> +       for file in $DIR*.$SUFFIX
> +       do
> +           dfu_test_file $file
> +       done
> +fi
> +
> +cleanup
> +
> +exit 0
> \ No newline at end of file

Why not?

> --
> 1.7.10.4
>

Regards,
Simon

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Lukasz Majewski]

Hi Simon,

> Hi Lukasz,
> 
> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote:
> > This commit adds test script for testing if any commit has
> > introduced regression to the DFU.
> >
> > It uses md5 to test if sent and received file is correct.
> > The test detailed description is available at DESCRIPTION.TXT file.
> >
> > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> > ---
> >  test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++
> 
> This sounds like an important and useful test, but I have a few
> comments (coming from someone with little knowledge of dfu)
> 
> Would README be a better name?

I think it will be better.

> 
> >  test/dfu/dfu_gadget_test.sh |   86
> > +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113
> > insertions(+) create mode 100644 test/dfu/DESCRIPTION.TXT
> >  create mode 100755 test/dfu/dfu_gadget_test.sh
> >
> > diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
> > new file mode 100644
> > index 0000000..48da06b
> > --- /dev/null
> > +++ b/test/dfu/DESCRIPTION.TXT
> > @@ -0,0 +1,27 @@
> > +DFU TEST CASE DESCRIPTION:
> > +
> > +For running test script one needs to create:
> > +./log and ./bkp
> > +
> > +One also need to generate with dd following files:
> > +
> > +dat_*
> > +
> > +(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img
> > dat_4095B.img +dat_4096B.img  dat_4097B.img  dat_63B.img
> > dat_64B.img  dat_65B.img +dat_960.img  dat_97M.img)
> 
> How do you do that? 

I generate them with using dd if=/dev/urandom of=/dev/null bs=XX
count=YY

Generation of those test images can be easily scripted.

> Is this intended for use with a particular board?

No. Those values shall be applicable to all boards which use DFU. For
example, since DFU uses the 64B as packet size, it is important to test
+1/-1 size. With those tests one can also test UDC driver for
regression.

Other file sizes (like 33MiB or 97MiB) are picked randomly to check if
transferring large or very large files work.

> I think you are missing some info here.
> 
> > +
> > +The size is only important (so if=/dev/urandom), especially corner
> > cases of +DFU EP0's size of packet (64B). This is very helpful for
> > fixing UDC driver +regressions.
> > +
> > +Moreover on a target device the "dfu_alt_info" env variable should
> > be extended +to have "dfu_test.bin fat 0 6;" \ entry [1].
> 
> What is [1] ?

[1] is the entry: "dfu_test.bin fat 0 6;", which should be added to the
"dfu_alt_info" environment variable to define one special alt setting
for testing purposes. 

> 
> > +One can use fat, ext4 or any other supported file system.
> > +
> > +As a prerequisite one must also create proper partition with file
> > system on it. +
> > +Example usage:
> > +./dfu_gadget_test.sh 11
> > +
> > +where 11 is the mumber of alt setting corresponding to entry [1].
> > \ No newline at end of file
> 
> I'm really none-the-wiser as to how this works - I think you need to
> add a bit more detail.

Ok. 

In short - the parameter which the script ./dfu_gadget_test.sh accepts
is 11 (therefore we must type ./dfu_gadget_test.sh 11

> 
> > diff --git a/test/dfu/dfu_gadget_test.sh
> > b/test/dfu/dfu_gadget_test.sh new file mode 100755
> > index 0000000..ebde2ff
> > --- /dev/null
> > +++ b/test/dfu/dfu_gadget_test.sh
> > @@ -0,0 +1,86 @@
> > +#! /bin/bash
> > +set -e # any command return not equal to zero
> > +clear
> > +
> > +DIR=./
> > +SUFFIX=img
> > +RCV_DIR=rcv/
> > +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> > +
> > +cleanup () {
> > +    rm -rf $RCV_DIR
> > +}
> > +
> > +die () {
> > +       printf "   \33[31m FAILED \33[0m \n"
> 
> Perhaps define $RED or whatever at the top and use those here?

Good point, thanks.

> 
> > +       cleanup
> > +       exit 1
> > +}
> > +
> > +calculate_md5sum () {
> > +    MD5SUM=`md5sum $1`
> > +    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> > +    echo "md5sum:"$MD5SUM
> > +}
> > +
> > +dfu_test_file () {
> 
> I suggest putting the argument in a variable here, instead of using $1
> throughout the function.

OK.

> 
> > +    printf
> > "\33[32m=========================================================================================
> > \33[0m\n"
> > +    printf "File:\33[32m %s \33[0m\n" $1
> > +
> > +    # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1
> 
> Remove this line?

Yes, agree. This line seems like a remain after testing.

> 
> > +    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die
> > $? +
> > +    echo -n "TX: "
> > +    calculate_md5sum $1
> > +
> > +    MD5_TX=$MD5SUM
> > +
> > +    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> > +
> > +    # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1
> 
> And again

OK.

> 
> > +    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1
> > || die $? +
> > +    echo -n "RX: "
> > +    calculate_md5sum $N_FILE
> > +    MD5_RX=$MD5SUM
> > +
> > +    if [ "$MD5_TX" == "$MD5_RX" ]
> > +       then
> > +       printf "   \33[32m -------> OK \33[0m \n"
> > +       else
> 
> outdent
> 
> > +       printf "   \33[31m -------> FAILED \33[0m \n"
> > +       cleanup
> > +       exit 1
> > +    fi
> > +
> > +#    echo $N_FILE
> > +}
> > +
> > +printf
> > "\33[32m=========================================================================================
> > \33[0m\n" +echo "DFU EP0 transmission test program" +echo "Trouble
> > shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" +echo
> > "@ -> TRATS # dfu mmc 0" +mkdir -p $RCV_DIR
> > +touch $LOG_FILE
> > +
> > +if [ $# -eq 0 ]
> > +then
> > +       printf "   \33[31m Please pass alt setting number!!  \33[0m
> > \n"
> > +       exit 0
> > +fi
> > +
> > +TARGET_ALT_SETTING=$1
> > +
> > +if [ -n "$2" ]
> > +then
> > +       dfu_test_file $2
> > +else
> > +       for file in $DIR*.$SUFFIX
> > +       do
> > +           dfu_test_file $file
> > +       done
> > +fi
> > +
> > +cleanup
> > +
> > +exit 0
> > \ No newline at end of file
> 
> Why not?

OK.

> 
> > --
> > 1.7.10.4
> >
> 
> Regards,
> Simon



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Stephen Warren]

On 05/23/2014 06:51 PM, Simon Glass wrote:
> Hi Lukasz,
> 
> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote:
>> This commit adds test script for testing if any commit has introduced
>> regression to the DFU.
>>
>> It uses md5 to test if sent and received file is correct.
>> The test detailed description is available at DESCRIPTION.TXT file.
>>
>> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
>> ---
>>  test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++

>> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
>> new file mode 100644
>> index 0000000..48da06b
>> --- /dev/null
>> +++ b/test/dfu/DESCRIPTION.TXT
>> @@ -0,0 +1,27 @@
>> +DFU TEST CASE DESCRIPTION:
>> +
>> +For running test script one needs to create:
>> +./log and ./bkp
>> +
>> +One also need to generate with dd following files:
>> +
>> +dat_*
>> +
>> +(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img  dat_4095B.img
>> +dat_4096B.img  dat_4097B.img  dat_63B.img  dat_64B.img  dat_65B.img
>> +dat_960.img  dat_97M.img)
> 
> How do you do that? Is this intended for use with a particular board?
> I think you are missing some info here.

I think the test script should generate its own test files, so the user
can simply run it without having to manually set up a bunch of stuff first.

One important test-case is files that are larger than
CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be 97MB?
If so, that's probably covered. Testing large sizes that both are and
are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be useful.

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Simon Glass]

Hi,

On 29 May 2014 09:58, Stephen Warren <swarren@wwwdotorg.org> wrote:
> On 05/23/2014 06:51 PM, Simon Glass wrote:
>> Hi Lukasz,
>>
>> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote:
>>> This commit adds test script for testing if any commit has introduced
>>> regression to the DFU.
>>>
>>> It uses md5 to test if sent and received file is correct.
>>> The test detailed description is available at DESCRIPTION.TXT file.
>>>
>>> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
>>> ---
>>>  test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++
>
>>> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
>>> new file mode 100644
>>> index 0000000..48da06b
>>> --- /dev/null
>>> +++ b/test/dfu/DESCRIPTION.TXT
>>> @@ -0,0 +1,27 @@
>>> +DFU TEST CASE DESCRIPTION:
>>> +
>>> +For running test script one needs to create:
>>> +./log and ./bkp
>>> +
>>> +One also need to generate with dd following files:
>>> +
>>> +dat_*
>>> +
>>> +(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img  dat_4095B.img
>>> +dat_4096B.img  dat_4097B.img  dat_63B.img  dat_64B.img  dat_65B.img
>>> +dat_960.img  dat_97M.img)
>>
>> How do you do that? Is this intended for use with a particular board?
>> I think you are missing some info here.
>
> I think the test script should generate its own test files, so the user
> can simply run it without having to manually set up a bunch of stuff first.

Agreed. As a general rule it would be good if the tests would run
through all possible options and not require user involvement. I'm not
sure if that is possible in this case though.

>
> One important test-case is files that are larger than
> CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be 97MB?
> If so, that's probably covered. Testing large sizes that both are and
> are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be useful.

Regards,
Simon

[U-Boot] [PATCH] test:dfu: Add test script for testing DFU regression

[Lukasz Majewski]

Hi Simon,

> Hi,
> 
> On 29 May 2014 09:58, Stephen Warren <swarren@wwwdotorg.org> wrote:
> > On 05/23/2014 06:51 PM, Simon Glass wrote:
> >> Hi Lukasz,
> >>
> >> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com>
> >> wrote:
> >>> This commit adds test script for testing if any commit has
> >>> introduced regression to the DFU.
> >>>
> >>> It uses md5 to test if sent and received file is correct.
> >>> The test detailed description is available at DESCRIPTION.TXT
> >>> file.
> >>>
> >>> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> >>> ---
> >>>  test/dfu/DESCRIPTION.TXT    |   27 ++++++++++++++
> >
> >>> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT
> >>> new file mode 100644
> >>> index 0000000..48da06b
> >>> --- /dev/null
> >>> +++ b/test/dfu/DESCRIPTION.TXT
> >>> @@ -0,0 +1,27 @@
> >>> +DFU TEST CASE DESCRIPTION:
> >>> +
> >>> +For running test script one needs to create:
> >>> +./log and ./bkp
> >>> +
> >>> +One also need to generate with dd following files:
> >>> +
> >>> +dat_*
> >>> +
> >>> +(e.g. dat_127B.img  dat_128B.img  dat_129B.img  dat_33M.img
> >>> dat_4095B.img +dat_4096B.img  dat_4097B.img  dat_63B.img
> >>> dat_64B.img  dat_65B.img +dat_960.img  dat_97M.img)
> >>
> >> How do you do that? Is this intended for use with a particular
> >> board? I think you are missing some info here.
> >
> > I think the test script should generate its own test files, so the
> > user can simply run it without having to manually set up a bunch of
> > stuff first.
> 
> Agreed. As a general rule it would be good if the tests would run
> through all possible options and not require user involvement. I'm not
> sure if that is possible in this case though.

I could provide a special script to automatize the work. Either in the
script itself or via configuration file it could be possible to specify
files' size to be used. Running this script would assure that each user
would have the same environment. 

> 
> >
> > One important test-case is files that are larger than
> > CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be
> > 97MB? If so, that's probably covered. Testing large sizes that both
> > are and are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be
> > useful.

Yes, this is the case.

> 
> Regards,
> Simon



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH v2] test:dfu: Add test scripts for testing DFU regression

[Lukasz Majewski]

This commit adds test scripts for testing if any commit has introduced
regression to the DFU subsystem.

It uses md5 to test if sent and received file is correct.
The test detailed description is available at README file.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
---
Changes for v2:
- Rename DESCRIPTION.TXT to README
- Introduction of COLOUR_* variables to hold code necessary to change
  console color
- File to create initial setup - dfu_gadget_test_init.sh has been added
- Test files are now automatically generated with the above script
---
 test/dfu/README                  |   27 ++++++++++++
 test/dfu/dfu_gadget_test.sh      |   88 ++++++++++++++++++++++++++++++++++++++
 test/dfu/dfu_gadget_test_init.sh |   34 +++++++++++++++
 3 files changed, 149 insertions(+)
 create mode 100644 test/dfu/README
 create mode 100755 test/dfu/dfu_gadget_test.sh
 create mode 100755 test/dfu/dfu_gadget_test_init.sh

diff --git a/test/dfu/README b/test/dfu/README
new file mode 100644
index 0000000..f8f5a43
--- /dev/null
+++ b/test/dfu/README
@@ -0,0 +1,27 @@
+DFU TEST CASE DESCRIPTION:
+
+The prerequisites for running this script are assured by
+dfu_gadget_test_init.sh script.
+In this file user is able to generate their own set of test files by altering
+the default set of TEST_FILES_SIZES variable
+
+Moreover, on a target device the "dfu_alt_info" env variable should be extended
+to have "dfu_test.bin fat 0 6;" \ entry ([1]). For reference please consult the
+config file for TRATS/TRATS2 devices (./include/configs/trats{2}.h)
+
+One can use fat, ext4 or any other supported file system, which can be
+created in a convenient way with exporting partitions via UMS (ums 0 mmc 0)
+and using standard tools on host (like mkfs.ext4).
+
+Example usage:
+1. On the target:
+   env default -a
+   dfu 0 mmc 0
+2. On the host:
+   ./dfu_gadget_test.sh 11
+
+where 11 is the mumber of alt setting corresponding to entry [1].
+
+The number of the alt setting entry can be obtained with dfu-util -l command.
+In its output one should look for the 'name="dfu_test1.bin"' and corresponding
+alt=11.
diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
new file mode 100755
index 0000000..8abd8e7
--- /dev/null
+++ b/test/dfu/dfu_gadget_test.sh
@@ -0,0 +1,88 @@
+#! /bin/bash
+set -e # any command return if not equal to zero
+clear
+
+COLOR_RED="\33[31m"
+COLOR_GREEN="\33[32m"
+COLOR_DEFAULT="\33[0m"
+
+DIR=./
+SUFFIX=img
+RCV_DIR=rcv/
+LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
+
+./dfu_gadget_test_init.sh
+
+cleanup () {
+    rm -rf $RCV_DIR
+}
+
+die () {
+	printf "   $COLOR_RED FAILED $COLOR_DEFAULT \n"
+	cleanup
+	exit 1
+}
+
+calculate_md5sum () {
+    MD5SUM=`md5sum $1`
+    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
+    echo "md5sum:"$MD5SUM
+}
+
+dfu_test_file () {
+    printf "$COLOR_GREEN ========================================================================================= $COLOR_DEFAULT\n"
+    printf "File:$COLOR_GREEN %s $COLOR_DEFAULT\n" $1
+
+    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "TX: "
+    calculate_md5sum $1
+
+    MD5_TX=$MD5SUM
+
+    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
+
+    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "RX: "
+    calculate_md5sum $N_FILE
+    MD5_RX=$MD5SUM
+
+    if [ "$MD5_TX" == "$MD5_RX" ]; then
+	printf "   $COLOR_GREEN -------> OK $COLOR_DEFAULT \n"
+    else
+	printf "   $COLOR_RED -------> FAILED $COLOR_DEFAULT \n"
+	cleanup
+	exit 1
+    fi
+
+}
+
+printf "$COLOR_GREEN========================================================================================= $COLOR_DEFAULT\n"
+echo "DFU EP0 transmission test program"
+echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
+echo "@ -> TRATS2 # dfu 0 mmc 0"
+mkdir -p $RCV_DIR
+touch $LOG_FILE
+
+if [ $# -eq 0 ]
+then
+	printf "   $COLOR_RED Please pass alt setting number!!  $COLOR_DEFAULT \n"
+	exit 0
+fi
+
+TARGET_ALT_SETTING=$1
+
+if [ -n "$2" ]
+then
+	dfu_test_file $2
+else
+	for file in $DIR*.$SUFFIX
+	do
+	    dfu_test_file $file
+	done
+fi
+
+cleanup
+
+exit 0
diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh
new file mode 100755
index 0000000..ea2f715
--- /dev/null
+++ b/test/dfu/dfu_gadget_test_init.sh
@@ -0,0 +1,34 @@
+#! /bin/bash
+set -e # any command return if not equal to zero
+clear
+
+COLOR_RED="\33[31m"
+COLOR_GREEN="\33[32m"
+COLOR_DEFAULT="\33[0m"
+
+LOG_DIR="./log"
+BKP_DIR="./bkp"
+
+TEST_FILES_SIZES="127 128 129 8M 4095 4096 4097 63 64 65 960"
+
+printf "Init script for generating data necessary for DFU test script"
+
+if [ ! -d $LOG_DIR ]; then
+    `mkdir $LOG_DIR`
+fi
+
+if [ ! -d $BKP_DIR ]; then
+    `mkdir $BKP_DIR`
+fi
+
+for size in $TEST_FILES_SIZES
+do
+    FILE="./dat_$size.img"
+    if [ ! -f $FILE ]; then
+	dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1 > /dev/null 2>&1 || exit $?
+    fi
+done
+
+printf "$COLOR_GREEN OK $COLOR_DEFAULT \n"
+
+exit 0
-- 
1.7.10.4

[U-Boot] [PATCH v2] test:dfu: Add test scripts for testing DFU regression

[Simon Glass]

Hi Lukasz,

On 3 June 2014 03:54, Lukasz Majewski <l.majewski@samsung.com> wrote:
> This commit adds test scripts for testing if any commit has introduced
> regression to the DFU subsystem.
>
> It uses md5 to test if sent and received file is correct.
> The test detailed description is available at README file.
>
> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> ---
> Changes for v2:
> - Rename DESCRIPTION.TXT to README
> - Introduction of COLOUR_* variables to hold code necessary to change
>   console color
> - File to create initial setup - dfu_gadget_test_init.sh has been added
> - Test files are now automatically generated with the above script

Looks good, but a few comments below.

> ---
>  test/dfu/README                  |   27 ++++++++++++
>  test/dfu/dfu_gadget_test.sh      |   88 ++++++++++++++++++++++++++++++++++++++
>  test/dfu/dfu_gadget_test_init.sh |   34 +++++++++++++++
>  3 files changed, 149 insertions(+)
>  create mode 100644 test/dfu/README
>  create mode 100755 test/dfu/dfu_gadget_test.sh
>  create mode 100755 test/dfu/dfu_gadget_test_init.sh
>
> diff --git a/test/dfu/README b/test/dfu/README
> new file mode 100644
> index 0000000..f8f5a43
> --- /dev/null
> +++ b/test/dfu/README
> @@ -0,0 +1,27 @@
> +DFU TEST CASE DESCRIPTION:
> +
> +The prerequisites for running this script are assured by
> +dfu_gadget_test_init.sh script.
> +In this file user is able to generate their own set of test files by altering
> +the default set of TEST_FILES_SIZES variable
> +
> +Moreover, on a target device the "dfu_alt_info" env variable should be extended
> +to have "dfu_test.bin fat 0 6;" \ entry ([1]). For reference please consult the
> +config file for TRATS/TRATS2 devices (./include/configs/trats{2}.h)
> +
> +One can use fat, ext4 or any other supported file system, which can be
> +created in a convenient way with exporting partitions via UMS (ums 0 mmc 0)
> +and using standard tools on host (like mkfs.ext4).
> +
> +Example usage:
> +1. On the target:
> +   env default -a
> +   dfu 0 mmc 0
> +2. On the host:
> +   ./dfu_gadget_test.sh 11
> +
> +where 11 is the mumber of alt setting corresponding to entry [1].
> +
> +The number of the alt setting entry can be obtained with dfu-util -l command.
> +In its output one should look for the 'name="dfu_test1.bin"' and corresponding
> +alt=11.
> diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
> new file mode 100755
> index 0000000..8abd8e7
> --- /dev/null
> +++ b/test/dfu/dfu_gadget_test.sh
> @@ -0,0 +1,88 @@
> +#! /bin/bash
> +set -e # any command return if not equal to zero
> +clear
> +
> +COLOR_RED="\33[31m"
> +COLOR_GREEN="\33[32m"
> +COLOR_DEFAULT="\33[0m"

Which side of the pond are you? :-) I think it should be COLOUR in U-Boot.

> +
> +DIR=./
> +SUFFIX=img
> +RCV_DIR=rcv/
> +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> +
> +./dfu_gadget_test_init.sh
> +
> +cleanup () {
> +    rm -rf $RCV_DIR
> +}
> +
> +die () {
> +       printf "   $COLOR_RED FAILED $COLOR_DEFAULT \n"
> +       cleanup
> +       exit 1
> +}
> +
> +calculate_md5sum () {
> +    MD5SUM=`md5sum $1`
> +    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> +    echo "md5sum:"$MD5SUM
> +}
> +
> +dfu_test_file () {
> +    printf "$COLOR_GREEN ========================================================================================= $COLOR_DEFAULT\n"
> +    printf "File:$COLOR_GREEN %s $COLOR_DEFAULT\n" $1
> +
> +    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
> +
> +    echo -n "TX: "
> +    calculate_md5sum $1
> +
> +    MD5_TX=$MD5SUM
> +
> +    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> +
> +    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
> +
> +    echo -n "RX: "
> +    calculate_md5sum $N_FILE
> +    MD5_RX=$MD5SUM
> +
> +    if [ "$MD5_TX" == "$MD5_RX" ]; then
> +       printf "   $COLOR_GREEN -------> OK $COLOR_DEFAULT \n"
> +    else
> +       printf "   $COLOR_RED -------> FAILED $COLOR_DEFAULT \n"
> +       cleanup
> +       exit 1
> +    fi
> +
> +}
> +
> +printf "$COLOR_GREEN========================================================================================= $COLOR_DEFAULT\n"
> +echo "DFU EP0 transmission test program"
> +echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
> +echo "@ -> TRATS2 # dfu 0 mmc 0"
> +mkdir -p $RCV_DIR
> +touch $LOG_FILE
> +
> +if [ $# -eq 0 ]
> +then
> +       printf "   $COLOR_RED Please pass alt setting number!!  $COLOR_DEFAULT \n"
> +       exit 0
> +fi
> +
> +TARGET_ALT_SETTING=$1
> +
> +if [ -n "$2" ]
> +then
> +       dfu_test_file $2

Where is $2 mentioned? in your example usage?

> +else
> +       for file in $DIR*.$SUFFIX
> +       do
> +           dfu_test_file $file
> +       done
> +fi
> +
> +cleanup
> +
> +exit 0
> diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh
> new file mode 100755
> index 0000000..ea2f715
> --- /dev/null
> +++ b/test/dfu/dfu_gadget_test_init.sh

Should this script be run by the main script in the default case? You
could have a flag to force it to use existing files.

> @@ -0,0 +1,34 @@
> +#! /bin/bash
> +set -e # any command return if not equal to zero
> +clear
> +
> +COLOR_RED="\33[31m"
> +COLOR_GREEN="\33[32m"
> +COLOR_DEFAULT="\33[0m"
> +
> +LOG_DIR="./log"
> +BKP_DIR="./bkp"
> +
> +TEST_FILES_SIZES="127 128 129 8M 4095 4096 4097 63 64 65 960"
> +
> +printf "Init script for generating data necessary for DFU test script"
> +
> +if [ ! -d $LOG_DIR ]; then
> +    `mkdir $LOG_DIR`
> +fi
> +
> +if [ ! -d $BKP_DIR ]; then
> +    `mkdir $BKP_DIR`
> +fi
> +
> +for size in $TEST_FILES_SIZES
> +do
> +    FILE="./dat_$size.img"
> +    if [ ! -f $FILE ]; then
> +       dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1 > /dev/null 2>&1 || exit $?
> +    fi
> +done
> +
> +printf "$COLOR_GREEN OK $COLOR_DEFAULT \n"
> +
> +exit 0
> --
> 1.7.10.4
>

Regards,
Simon

[U-Boot] [PATCH v2] test:dfu: Add test scripts for testing DFU regression

[Lukasz Majewski]

Hi Simon,

> Hi Lukasz,
> 
> On 3 June 2014 03:54, Lukasz Majewski <l.majewski@samsung.com> wrote:
> > This commit adds test scripts for testing if any commit has
> > introduced regression to the DFU subsystem.
> >
> > It uses md5 to test if sent and received file is correct.
> > The test detailed description is available at README file.
> >
> > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> > ---
> > Changes for v2:
> > - Rename DESCRIPTION.TXT to README
> > - Introduction of COLOUR_* variables to hold code necessary to
> > change console color
> > - File to create initial setup - dfu_gadget_test_init.sh has been
> > added
> > - Test files are now automatically generated with the above script
> 
> Looks good, but a few comments below.
> 
> > ---
> >  test/dfu/README                  |   27 ++++++++++++
> >  test/dfu/dfu_gadget_test.sh      |   88
> > ++++++++++++++++++++++++++++++++++++++
> > test/dfu/dfu_gadget_test_init.sh |   34 +++++++++++++++ 3 files
> > changed, 149 insertions(+) create mode 100644 test/dfu/README
> >  create mode 100755 test/dfu/dfu_gadget_test.sh
> >  create mode 100755 test/dfu/dfu_gadget_test_init.sh
> >
> > diff --git a/test/dfu/README b/test/dfu/README
> > new file mode 100644
> > index 0000000..f8f5a43
> > --- /dev/null
> > +++ b/test/dfu/README
> > @@ -0,0 +1,27 @@
> > +DFU TEST CASE DESCRIPTION:
> > +
> > +The prerequisites for running this script are assured by
> > +dfu_gadget_test_init.sh script.
> > +In this file user is able to generate their own set of test files
> > by altering +the default set of TEST_FILES_SIZES variable
> > +
> > +Moreover, on a target device the "dfu_alt_info" env variable
> > should be extended +to have "dfu_test.bin fat 0 6;" \ entry ([1]).
> > For reference please consult the +config file for TRATS/TRATS2
> > devices (./include/configs/trats{2}.h) +
> > +One can use fat, ext4 or any other supported file system, which
> > can be +created in a convenient way with exporting partitions via
> > UMS (ums 0 mmc 0) +and using standard tools on host (like
> > mkfs.ext4). +
> > +Example usage:
> > +1. On the target:
> > +   env default -a
> > +   dfu 0 mmc 0
> > +2. On the host:
> > +   ./dfu_gadget_test.sh 11
> > +
> > +where 11 is the mumber of alt setting corresponding to entry [1].
> > +
> > +The number of the alt setting entry can be obtained with dfu-util
> > -l command. +In its output one should look for the
> > 'name="dfu_test1.bin"' and corresponding +alt=11.
> > diff --git a/test/dfu/dfu_gadget_test.sh
> > b/test/dfu/dfu_gadget_test.sh new file mode 100755
> > index 0000000..8abd8e7
> > --- /dev/null
> > +++ b/test/dfu/dfu_gadget_test.sh
> > @@ -0,0 +1,88 @@
> > +#! /bin/bash
> > +set -e # any command return if not equal to zero
> > +clear
> > +
> > +COLOR_RED="\33[31m"
> > +COLOR_GREEN="\33[32m"
> > +COLOR_DEFAULT="\33[0m"
> 
> Which side of the pond are you? :-) I think it should be COLOUR in
> U-Boot.

This script is run on the host. Could you be more specific.

> 
> > +
> > +DIR=./
> > +SUFFIX=img
> > +RCV_DIR=rcv/
> > +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> > +
> > +./dfu_gadget_test_init.sh
> > +
> > +cleanup () {
> > +    rm -rf $RCV_DIR
> > +}
> > +
> > +die () {
> > +       printf "   $COLOR_RED FAILED $COLOR_DEFAULT \n"
> > +       cleanup
> > +       exit 1
> > +}
> > +
> > +calculate_md5sum () {
> > +    MD5SUM=`md5sum $1`
> > +    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> > +    echo "md5sum:"$MD5SUM
> > +}
> > +
> > +dfu_test_file () {
> > +    printf "$COLOR_GREEN
> > =========================================================================================
> > $COLOR_DEFAULT\n"
> > +    printf "File:$COLOR_GREEN %s $COLOR_DEFAULT\n" $1
> > +
> > +    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die
> > $? +
> > +    echo -n "TX: "
> > +    calculate_md5sum $1
> > +
> > +    MD5_TX=$MD5SUM
> > +
> > +    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> > +
> > +    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1
> > || die $? +
> > +    echo -n "RX: "
> > +    calculate_md5sum $N_FILE
> > +    MD5_RX=$MD5SUM
> > +
> > +    if [ "$MD5_TX" == "$MD5_RX" ]; then
> > +       printf "   $COLOR_GREEN -------> OK $COLOR_DEFAULT \n"
> > +    else
> > +       printf "   $COLOR_RED -------> FAILED $COLOR_DEFAULT \n"
> > +       cleanup
> > +       exit 1
> > +    fi
> > +
> > +}
> > +
> > +printf
> > "$COLOR_GREEN=========================================================================================
> > $COLOR_DEFAULT\n" +echo "DFU EP0 transmission test program" +echo
> > "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC
> > driver" +echo "@ -> TRATS2 # dfu 0 mmc 0" +mkdir -p $RCV_DIR
> > +touch $LOG_FILE
> > +
> > +if [ $# -eq 0 ]
> > +then
> > +       printf "   $COLOR_RED Please pass alt setting number!!
> > $COLOR_DEFAULT \n"
> > +       exit 0
> > +fi
> > +
> > +TARGET_ALT_SETTING=$1
> > +
> > +if [ -n "$2" ]
> > +then
> > +       dfu_test_file $2
> 
> Where is $2 mentioned? in your example usage?

I think that I might overlooked it.

> 
> > +else
> > +       for file in $DIR*.$SUFFIX
> > +       do
> > +           dfu_test_file $file
> > +       done
> > +fi
> > +
> > +cleanup
> > +
> > +exit 0
> > diff --git a/test/dfu/dfu_gadget_test_init.sh
> > b/test/dfu/dfu_gadget_test_init.sh new file mode 100755
> > index 0000000..ea2f715
> > --- /dev/null
> > +++ b/test/dfu/dfu_gadget_test_init.sh
> 
> Should this script be run by the main script in the default case? You
> could have a flag to force it to use existing files.
> 
> > @@ -0,0 +1,34 @@
> > +#! /bin/bash
> > +set -e # any command return if not equal to zero
> > +clear
> > +
> > +COLOR_RED="\33[31m"
> > +COLOR_GREEN="\33[32m"
> > +COLOR_DEFAULT="\33[0m"
> > +
> > +LOG_DIR="./log"
> > +BKP_DIR="./bkp"
> > +
> > +TEST_FILES_SIZES="127 128 129 8M 4095 4096 4097 63 64 65 960"
> > +
> > +printf "Init script for generating data necessary for DFU test
> > script" +
> > +if [ ! -d $LOG_DIR ]; then
> > +    `mkdir $LOG_DIR`
> > +fi
> > +
> > +if [ ! -d $BKP_DIR ]; then
> > +    `mkdir $BKP_DIR`
> > +fi
> > +
> > +for size in $TEST_FILES_SIZES
> > +do
> > +    FILE="./dat_$size.img"
> > +    if [ ! -f $FILE ]; then
> > +       dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1
> > > /dev/null 2>&1 || exit $?
> > +    fi
> > +done
> > +
> > +printf "$COLOR_GREEN OK $COLOR_DEFAULT \n"
> > +
> > +exit 0
> > --
> > 1.7.10.4
> >
> 
> Regards,
> Simon



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH v2] test:dfu: Add test scripts for testing DFU regression

[Simon Glass]

Hi Lukasz,

On 12 June 2014 05:00, Lukasz Majewski <l.majewski@samsung.com> wrote:
[snip]

>> > +COLOR_RED="\33[31m"
>> > +COLOR_GREEN="\33[32m"
>> > +COLOR_DEFAULT="\33[0m"
>>
>> Which side of the pond are you? :-) I think it should be COLOUR in
>> U-Boot.
>
> This script is run on the host. Could you be more specific.

I just mean that you should spell it COLOUR, not COLOR as COLOUR is
the spelling that U-Boot uses, so far as I understand it.

Regards,
Simon

[U-Boot] [PATCH v2] test:dfu: Add test scripts for testing DFU regression

[Lukasz Majewski]

Hi Simon,

> Hi Lukasz,
> 
> On 3 June 2014 03:54, Lukasz Majewski <l.majewski@samsung.com> wrote:
> > This commit adds test scripts for testing if any commit has
> > introduced regression to the DFU subsystem.
> >
> > It uses md5 to test if sent and received file is correct.
> > The test detailed description is available at README file.
> >
> > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> > ---
> > Changes for v2:
> > - Rename DESCRIPTION.TXT to README
> > - Introduction of COLOUR_* variables to hold code necessary to
> > change console color
> > - File to create initial setup - dfu_gadget_test_init.sh has been
> > added
> > - Test files are now automatically generated with the above script
> 
> Looks good, but a few comments below.
> 
> > ---
> >  test/dfu/README                  |   27 ++++++++++++
> >  test/dfu/dfu_gadget_test.sh      |   88
> > ++++++++++++++++++++++++++++++++++++++
> > test/dfu/dfu_gadget_test_init.sh |   34 +++++++++++++++ 3 files
> > changed, 149 insertions(+) create mode 100644 test/dfu/README
> >  create mode 100755 test/dfu/dfu_gadget_test.sh
> >  create mode 100755 test/dfu/dfu_gadget_test_init.sh
> >
> > diff --git a/test/dfu/README b/test/dfu/README
> > new file mode 100644
> > index 0000000..f8f5a43
> > --- /dev/null
> > +++ b/test/dfu/README
> > @@ -0,0 +1,27 @@
> > +DFU TEST CASE DESCRIPTION:
> > +
> > +The prerequisites for running this script are assured by
> > +dfu_gadget_test_init.sh script.
> > +In this file user is able to generate their own set of test files
> > by altering +the default set of TEST_FILES_SIZES variable
> > +
> > +Moreover, on a target device the "dfu_alt_info" env variable
> > should be extended +to have "dfu_test.bin fat 0 6;" \ entry ([1]).
> > For reference please consult the +config file for TRATS/TRATS2
> > devices (./include/configs/trats{2}.h) +
> > +One can use fat, ext4 or any other supported file system, which
> > can be +created in a convenient way with exporting partitions via
> > UMS (ums 0 mmc 0) +and using standard tools on host (like
> > mkfs.ext4). +
> > +Example usage:
> > +1. On the target:
> > +   env default -a
> > +   dfu 0 mmc 0
> > +2. On the host:
> > +   ./dfu_gadget_test.sh 11
> > +
> > +where 11 is the mumber of alt setting corresponding to entry [1].
> > +
> > +The number of the alt setting entry can be obtained with dfu-util
> > -l command. +In its output one should look for the
> > 'name="dfu_test1.bin"' and corresponding +alt=11.
> > diff --git a/test/dfu/dfu_gadget_test.sh
> > b/test/dfu/dfu_gadget_test.sh new file mode 100755
> > index 0000000..8abd8e7
> > --- /dev/null
> > +++ b/test/dfu/dfu_gadget_test.sh
> > @@ -0,0 +1,88 @@
> > +#! /bin/bash
> > +set -e # any command return if not equal to zero
> > +clear
> > +
> > +COLOR_RED="\33[31m"
> > +COLOR_GREEN="\33[32m"
> > +COLOR_DEFAULT="\33[0m"
> 
> Which side of the pond are you? :-) I think it should be COLOUR in
> U-Boot.

Ok, I will change the spelling from COLOR to COLOUR.

> 
> > +
> > +DIR=./
> > +SUFFIX=img
> > +RCV_DIR=rcv/
> > +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> > +
> > +./dfu_gadget_test_init.sh
> > +
> > +cleanup () {
> > +    rm -rf $RCV_DIR
> > +}
> > +
> > +die () {
> > +       printf "   $COLOR_RED FAILED $COLOR_DEFAULT \n"
> > +       cleanup
> > +       exit 1
> > +}
> > +
> > +calculate_md5sum () {
> > +    MD5SUM=`md5sum $1`
> > +    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> > +    echo "md5sum:"$MD5SUM
> > +}
> > +
> > +dfu_test_file () {
> > +    printf "$COLOR_GREEN
> > =========================================================================================
> > $COLOR_DEFAULT\n"
> > +    printf "File:$COLOR_GREEN %s $COLOR_DEFAULT\n" $1
> > +
> > +    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die
> > $? +
> > +    echo -n "TX: "
> > +    calculate_md5sum $1
> > +
> > +    MD5_TX=$MD5SUM
> > +
> > +    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> > +
> > +    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1
> > || die $? +
> > +    echo -n "RX: "
> > +    calculate_md5sum $N_FILE
> > +    MD5_RX=$MD5SUM
> > +
> > +    if [ "$MD5_TX" == "$MD5_RX" ]; then
> > +       printf "   $COLOR_GREEN -------> OK $COLOR_DEFAULT \n"
> > +    else
> > +       printf "   $COLOR_RED -------> FAILED $COLOR_DEFAULT \n"
> > +       cleanup
> > +       exit 1
> > +    fi
> > +
> > +}
> > +
> > +printf
> > "$COLOR_GREEN=========================================================================================
> > $COLOR_DEFAULT\n" +echo "DFU EP0 transmission test program" +echo
> > "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC
> > driver" +echo "@ -> TRATS2 # dfu 0 mmc 0" +mkdir -p $RCV_DIR
> > +touch $LOG_FILE
> > +
> > +if [ $# -eq 0 ]
> > +then
> > +       printf "   $COLOR_RED Please pass alt setting number!!
> > $COLOR_DEFAULT \n"
> > +       exit 0
> > +fi
> > +
> > +TARGET_ALT_SETTING=$1
> > +
> > +if [ -n "$2" ]
> > +then
> > +       dfu_test_file $2
> 
> Where is $2 mentioned? in your example usage?

$2 is for the case when one wants to test a particular file. I will add
this to the patch description.

> 
> > +else
> > +       for file in $DIR*.$SUFFIX
> > +       do
> > +           dfu_test_file $file
> > +       done
> > +fi
> > +
> > +cleanup
> > +
> > +exit 0
> > diff --git a/test/dfu/dfu_gadget_test_init.sh
> > b/test/dfu/dfu_gadget_test_init.sh new file mode 100755
> > index 0000000..ea2f715
> > --- /dev/null
> > +++ b/test/dfu/dfu_gadget_test_init.sh
> 
> Should this script be run by the main script in the default case? You
> could have a flag to force it to use existing files.

This script (dfu_gadget_test_init.sh) at [1] checks if file which it
intends to create already exists.

Is this the behaviour which you would like to see in this script?

> 
> > @@ -0,0 +1,34 @@
> > +#! /bin/bash
> > +set -e # any command return if not equal to zero
> > +clear
> > +
> > +COLOR_RED="\33[31m"
> > +COLOR_GREEN="\33[32m"
> > +COLOR_DEFAULT="\33[0m"
> > +
> > +LOG_DIR="./log"
> > +BKP_DIR="./bkp"
> > +
> > +TEST_FILES_SIZES="127 128 129 8M 4095 4096 4097 63 64 65 960"
> > +
> > +printf "Init script for generating data necessary for DFU test
> > script" +
> > +if [ ! -d $LOG_DIR ]; then
> > +    `mkdir $LOG_DIR`
> > +fi
> > +
> > +if [ ! -d $BKP_DIR ]; then
> > +    `mkdir $BKP_DIR`
> > +fi
> > +
> > +for size in $TEST_FILES_SIZES
> > +do
> > +    FILE="./dat_$size.img"
> > +    if [ ! -f $FILE ]; then
		^^^^^^^^^^ [1]

> > +       dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1
> > > /dev/null 2>&1 || exit $?
> > +    fi
> > +done
> > +
> > +printf "$COLOR_GREEN OK $COLOR_DEFAULT \n"
> > +
> > +exit 0
> > --
> > 1.7.10.4
> >
> 
> Regards,
> Simon



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH v3] test:dfu: Add test scripts for testing DFU regression

[Lukasz Majewski]

This commit adds test scripts for testing if any commit has introduced
regression to the DFU subsystem.

It uses md5 to test if sent and received file is correct.
The test detailed description is available at README file.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
---
Changes for v3:
- Rename COLOR with COLOUR
- Add README description for the optional second parameter passed to
  dfu_gadget_test.sh

Changes for v2:
- Rename DESCRIPTION.TXT to README
- Introduction of COLOUR_* variables to hold code necessary to change
  console color
- File to create initial setup - dfu_gadget_test_init.sh has been added
- Test files are now automatically generated with the above script
---
 test/dfu/README                  | 30 ++++++++++++++
 test/dfu/dfu_gadget_test.sh      | 88 ++++++++++++++++++++++++++++++++++++++++
 test/dfu/dfu_gadget_test_init.sh | 34 ++++++++++++++++
 3 files changed, 152 insertions(+)
 create mode 100644 test/dfu/README
 create mode 100755 test/dfu/dfu_gadget_test.sh
 create mode 100755 test/dfu/dfu_gadget_test_init.sh

diff --git a/test/dfu/README b/test/dfu/README
new file mode 100644
index 0000000..f8c1a74
--- /dev/null
+++ b/test/dfu/README
@@ -0,0 +1,30 @@
+DFU TEST CASE DESCRIPTION:
+
+The prerequisites for running this script are assured by dfu_gadget_test_init.sh.
+In this file user is able to generate their own set of test files by altering
+the default set of TEST_FILES_SIZES variable.
+The dfu_gadget_test_init.sh would generate test images only if they are not
+already generated.
+
+Moreover, on a target device the "dfu_alt_info" env variable should be extended
+to have "dfu_test.bin fat 0 6;" \ entry ([1]). For reference please consult the
+config file for TRATS/TRATS2 devices (./include/configs/trats{2}.h)
+
+One can use fat, ext4 or any other supported file system, which can be
+created in a convenient way with exporting partitions via UMS (ums 0 mmc 0)
+and using standard tools on host (like mkfs.ext4).
+
+Example usage:
+1. On the target:
+   env default -a
+   dfu 0 mmc 0
+2. On the host:
+   ./dfu_gadget_test.sh 11 [test_file]
+
+where 11 is the mumber of alt setting corresponding to entry [1] and [test_file]
+is an optional parameter, with which one can explicitly indicate the test file
+to be used.
+
+The number of the alt setting entry can be obtained with dfu-util -l command.
+In its output one should look for the 'name="dfu_test1.bin"' and corresponding
+alt=11.
diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
new file mode 100755
index 0000000..4a848c8
--- /dev/null
+++ b/test/dfu/dfu_gadget_test.sh
@@ -0,0 +1,88 @@
+#! /bin/bash
+set -e # any command return if not equal to zero
+clear
+
+COLOUR_RED="\33[31m"
+COLOUR_GREEN="\33[32m"
+COLOUR_DEFAULT="\33[0m"
+
+DIR=./
+SUFFIX=img
+RCV_DIR=rcv/
+LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
+
+./dfu_gadget_test_init.sh
+
+cleanup () {
+    rm -rf $RCV_DIR
+}
+
+die () {
+	printf "   $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
+	cleanup
+	exit 1
+}
+
+calculate_md5sum () {
+    MD5SUM=`md5sum $1`
+    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
+    echo "md5sum:"$MD5SUM
+}
+
+dfu_test_file () {
+    printf "$COLOUR_GREEN ========================================================================================= $COLOUR_DEFAULT\n"
+    printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
+
+    dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "TX: "
+    calculate_md5sum $1
+
+    MD5_TX=$MD5SUM
+
+    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
+
+    dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
+
+    echo -n "RX: "
+    calculate_md5sum $N_FILE
+    MD5_RX=$MD5SUM
+
+    if [ "$MD5_TX" == "$MD5_RX" ]; then
+	printf "   $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
+    else
+	printf "   $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
+	cleanup
+	exit 1
+    fi
+
+}
+
+printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n"
+echo "DFU EP0 transmission test program"
+echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
+echo "@ -> TRATS2 # dfu 0 mmc 0"
+mkdir -p $RCV_DIR
+touch $LOG_FILE
+
+if [ $# -eq 0 ]
+then
+	printf "   $COLOUR_RED Please pass alt setting number!!  $COLOUR_DEFAULT \n"
+	exit 0
+fi
+
+TARGET_ALT_SETTING=$1
+
+if [ -n "$2" ]
+then
+	dfu_test_file $2
+else
+	for file in $DIR*.$SUFFIX
+	do
+	    dfu_test_file $file
+	done
+fi
+
+cleanup
+
+exit 0
diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh
new file mode 100755
index 0000000..b319ed0
--- /dev/null
+++ b/test/dfu/dfu_gadget_test_init.sh
@@ -0,0 +1,34 @@
+#! /bin/bash
+set -e # any command return if not equal to zero
+clear
+
+COLOUR_RED="\33[31m"
+COLOUR_GREEN="\33[32m"
+COLOUR_DEFAULT="\33[0m"
+
+LOG_DIR="./log"
+BKP_DIR="./bkp"
+
+TEST_FILES_SIZES="127 128 129 8M 4095 4096 4097 63 64 65 960"
+
+printf "Init script for generating data necessary for DFU test script"
+
+if [ ! -d $LOG_DIR ]; then
+    `mkdir $LOG_DIR`
+fi
+
+if [ ! -d $BKP_DIR ]; then
+    `mkdir $BKP_DIR`
+fi
+
+for size in $TEST_FILES_SIZES
+do
+    FILE="./dat_$size.img"
+    if [ ! -f $FILE ]; then
+	dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1 > /dev/null 2>&1 || exit $?
+    fi
+done
+
+printf "$COLOUR_GREEN OK $COLOUR_DEFAULT \n"
+
+exit 0
-- 
2.0.0.rc2

[U-Boot] [PATCH v3] test:dfu: Add test scripts for testing DFU regression

[Lukasz Majewski]

Hi Lukasz,

> This commit adds test scripts for testing if any commit has introduced
> regression to the DFU subsystem.
> 
> It uses md5 to test if sent and received file is correct.
> The test detailed description is available at README file.
> 
> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>

Applied to u-boot-dfu

Thanks for the patch.
-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group