[U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing

[Lukasz Majewski <l.majewski@samsung.com>]

Hi Stephen,

> From: Stephen Warren <swarren@nvidia.com>
> 
> Enhance the UMS test to optionally mount a partition and read/write a
> file to it, validating that the content written and read back are
> identical.
> 
> This enhancement is backwards-compatible; old boardenv contents that
> don't define the new configuration data will cause the test code to
> perform as before.
> 
> test/ums/ is deleted since the Python test now performs the same
> testing that it did.
> 
> The code is also re-written to make use of the recently added utility
> module, and split it up into nested functions so the overall logic of
> the test process can be followed more easily without the details
> cluttering the code.
> 
> Cc: Lukasz Majewski <l.majewski@samsung.com>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
>  test/py/tests/test_ums.py   | 212
> +++++++++++++++++++++++++++++++++++---------
> test/ums/README             |  30 ------- test/ums/ums_gadget_test.sh
> | 183 -------------------------------------- 3 files changed, 169
> insertions(+), 256 deletions(-) delete mode 100644 test/ums/README
>  delete mode 100755 test/ums/ums_gadget_test.sh
> 
> diff --git a/test/py/tests/test_ums.py b/test/py/tests/test_ums.py
> index a137221c7a5b..cb6e5ef8c20b 100644
> --- a/test/py/tests/test_ums.py
> +++ b/test/py/tests/test_ums.py
> @@ -7,8 +7,11 @@
>  # should be enhanced to validate disk IO.
>  
>  import os
> +import os.path
>  import pytest
> +import re
>  import time
> +import u_boot_utils
>  
>  '''
>  Note: This test relies on:
> @@ -17,13 +20,36 @@ a) boardenv_* to contain configuration values to
> define which USB ports are available for testing. Without this, this
> test will be automatically skipped. For example:
>  
> +# Leave this list empty if you have no block_devs below with writable
> +# partitions defined.
> +env__mount_points = (
> +    "/mnt/ubtest-mnt-p2371-2180-na",
> +)
> +
>  env__usb_dev_ports = (
> -    {'tgt_usb_ctlr': '0', 'host_ums_dev_node':
> '/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'},
> +    {
> +        "tgt_usb_ctlr": "0",
> +        "host_ums_dev_node":
> "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
> +    },
>  )
>  
>  env__block_devs = (
> -    {'type': 'mmc', 'id': '0'}, # eMMC; always present
> -    {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged
> one in
> +    # eMMC; always present
> +    {
> +        "type": "mmc",
> +        "id": "0",
> +        # The following two properties are optional.
> +        # If present, the partition will be mounted and a file
> written-to and
> +        # read-from it. If missing, only a simple block read test
> will be
> +        # performed.
> +        "writable_fs_partition": 1,
> +        "writable_fs_subdir": "tmp/",
> +    },
> +    # SD card; present since I plugged one in
> +    {
> +        "type": "mmc",
> +        "id": "1"
> +    },
>  )
>  
>  b) udev rules to set permissions on devices nodes, so that sudo is
> not @@ -34,41 +60,15 @@ ACTION=="add", SUBSYSTEM=="block",
> SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="66 (You may wish to change
> the group ID instead of setting the permissions wide open. All that
> matters is that the user ID running the test can access the device.)
> -'''
>  
> -def open_ums_device(host_ums_dev_node):
> -    '''Attempt to open a device node, returning either the opened
> file handle,
> -    or None on any error.'''
> +c) /etc/fstab entries to allow the block device to be mounted
> without requiring +root permissions. For example:
>  
> -    try:
> -        return open(host_ums_dev_node, 'rb')
> -    except:
> -        return None
> -
> -def wait_for_ums_device(host_ums_dev_node):
> -    '''Continually attempt to open the device node exported by the
> "ums"
> -    command, and either return the opened file handle, or raise an
> exception
> -    after a timeout.'''
> -
> -    for i in xrange(100):
> -        fh = open_ums_device(host_ums_dev_node)
> -        if fh:
> -            return fh
> -        time.sleep(0.1)
> -    raise Exception('UMS device did not appear')
> -
> -def wait_for_ums_device_gone(host_ums_dev_node):
> -    '''Continually attempt to open the device node exported by the
> "ums"
> -    command, and either return once the device has disappeared, or
> raise an
> -    exception if it does not before a timeout occurs.'''
> -
> -    for i in xrange(100):
> -        fh = open_ums_device(host_ums_dev_node)
> -        if not fh:
> -            return
> -        fh.close()
> -        time.sleep(0.1)
> -    raise Exception('UMS device did not disappear')
> +/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0-part1 /mnt/ubtest-mnt-p2371-2180-na
> ext4 noauto,user,nosuid,nodev +
> +This entry is only needed if any block_devs above contain a
> +writable_fs_partition value.
> +'''
>  
>  @pytest.mark.buildconfigspec('cmd_usb_mass_storage')
>  def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
> @@ -76,6 +76,14 @@ def test_ums(u_boot_console, env__usb_dev_port,
> env__block_devs): device when "ums" is running, and this device must
> disappear when "ums" is aborted.'''
>  
> +    have_writable_fs_partition = 'writable_fs_partition' in
> env__block_devs[0]
> +    if not have_writable_fs_partition:
> +        # If 'writable_fs_subdir' is missing, we'll skip all parts
> of the
> +        # testing which mount filesystems.
> +        u_boot_console.log.warning(
> +            'boardenv missing "writable_fs_partition"; ' +
> +            'UMS testing will be limited.')
> +
>      tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr']
>      host_ums_dev_node = env__usb_dev_port['host_ums_dev_node']
>  
> @@ -84,11 +92,129 @@ def test_ums(u_boot_console, env__usb_dev_port,
> env__block_devs): # device list here. We'll test each block device
> somewhere else. tgt_dev_type = env__block_devs[0]['type']
>      tgt_dev_id = env__block_devs[0]['id']
> +    if have_writable_fs_partition:
> +        mount_point =
> u_boot_console.config.env['env__mount_points'][0]
> +        mount_subdir = env__block_devs[0]['writable_fs_subdir']
> +        part_num = env__block_devs[0]['writable_fs_partition']
> +        host_ums_part_node = '%s-part%d' % (host_ums_dev_node,
> part_num)
> +    else:
> +        host_ums_part_node = host_ums_dev_node
> +
> +    test_f = u_boot_utils.PersistentRandomFile(u_boot_console,
> 'ums.bin',
> +        1024 * 1024);
> +    if have_writable_fs_partition:
> +        mounted_test_fn = mount_point + '/' + mount_subdir +
> test_f.fn +
> +    def start_ums():
> +        '''Start U-Boot's ums shell command.
> +
> +        This also waits for the host-side USB enumeration process to
> complete. +
> +        Args:
> +            None.
> +
> +        Returns:
> +            Nothing.
> +        '''
> +
> +        u_boot_console.log.action(
> +            'Starting long-running U-Boot ums shell command')
> +        cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type,
> tgt_dev_id)
> +        u_boot_console.run_command(cmd, wait_for_prompt=False)
> +        u_boot_console.wait_for(re.compile('UMS: LUN.*[\r\n]'))
> +        fh =
> u_boot_utils.wait_until_open_succeeds(host_ums_part_node)
> +        u_boot_console.log.action('Reading raw data from UMS device')
> +        fh.read(4096)
> +        fh.close()
> +
> +    def mount():
> +        '''Mount the block device that U-Boot exports.
> +
> +        Args:
> +            None.
> +
> +        Returns:
> +            Nothing.
> +        '''
>  
> -    cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
> -    u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False)
> -    fh = wait_for_ums_device(host_ums_dev_node)
> -    fh.read(4096)
> -    fh.close()
> -    u_boot_console.ctrlc()
> -    wait_for_ums_device_gone(host_ums_dev_node)
> +        u_boot_console.log.action('Mounting exported UMS device')
> +        cmd = ('/bin/mount', host_ums_part_node)
> +        u_boot_utils.run_and_log(u_boot_console, cmd)
> +
> +    def umount(ignore_errors):
> +        '''Unmount the block device that U-Boot exports.
> +
> +        Args:
> +            ignore_errors: Ignore any errors. This is useful if an
> error has
> +                already been detected, and the code is performing
> best-effort
> +                cleanup. In this case, we do not want to mask the
> original
> +                error by "honoring" any new errors.
> +
> +        Returns:
> +            Nothing.
> +        '''
> +
> +        u_boot_console.log.action('Unmounting UMS device')
> +        cmd = ('/bin/umount', host_ums_part_node)
> +        u_boot_utils.run_and_log(u_boot_console, cmd, ignore_errors)
> +
> +    def stop_ums(ignore_errors):
> +        '''Stop U-Boot's ums shell command from executing.
> +
> +        This also waits for the host-side USB de-enumeration process
> to
> +        complete.
> +
> +        Args:
> +            ignore_errors: Ignore any errors. This is useful if an
> error has
> +                already been detected, and the code is performing
> best-effort
> +                cleanup. In this case, we do not want to mask the
> original
> +                error by "honoring" any new errors.
> +
> +        Returns:
> +            Nothing.
> +        '''
> +
> +        u_boot_console.log.action(
> +            'Stopping long-running U-Boot ums shell command')
> +        u_boot_console.ctrlc()
> +        u_boot_utils.wait_until_file_open_fails(host_ums_part_node,
> +            ignore_errors)
> +
> +    ignore_cleanup_errors = True
> +    try:
> +        start_ums()
> +        if not have_writable_fs_partition:
> +            # Skip filesystem-based testing if not configured
> +            return
> +        try:
> +            mount()
> +            u_boot_console.log.action('Writing test file via UMS')
> +            cmd = ('rm', '-f', mounted_test_fn)
> +            u_boot_utils.run_and_log(u_boot_console, cmd)
> +            if os.path.exists(mounted_test_fn):
> +                raise Exception('Could not rm target UMS test file')
> +            cmd = ('cp', test_f.abs_fn, mounted_test_fn)
> +            u_boot_utils.run_and_log(u_boot_console, cmd)
> +            ignore_cleanup_errors = False
> +        finally:
> +            umount(ignore_errors=ignore_cleanup_errors)
> +    finally:
> +        stop_ums(ignore_errors=ignore_cleanup_errors)
> +
> +    ignore_cleanup_errors = True
> +    try:
> +        start_ums()
> +        try:
> +            mount()
> +            u_boot_console.log.action('Reading test file back via
> UMS')
> +            read_back_hash =
> u_boot_utils.md5sum_file(mounted_test_fn)
> +            cmd = ('rm', '-f', mounted_test_fn)
> +            u_boot_utils.run_and_log(u_boot_console, cmd)
> +            ignore_cleanup_errors = False
> +        finally:
> +            umount(ignore_errors=ignore_cleanup_errors)
> +    finally:
> +        stop_ums(ignore_errors=ignore_cleanup_errors)
> +
> +    written_hash = test_f.content_hash
> +    assert(written_hash == read_back_hash)
> diff --git a/test/ums/README b/test/ums/README
> deleted file mode 100644
> index c80fbfefbf52..000000000000
> --- a/test/ums/README
> +++ /dev/null
> @@ -1,30 +0,0 @@
> -UMS test script.
> -
> -ums_gadget_test.sh
> -==================
> -
> -Example usage:
> -1. On the target:
> -   create UMS exportable partitions (with e.g. gpt write), or
> specify a
> -   partition number (PART_NUM) as "-" to use the entire device
> -   ums 0 mmc 0
> -2. On the host:
> -   sudo test/ums/ums_gadget_test.sh VID PID PART_NUM [-f
> FILE_SYSTEM] [test_file]
> -   e.g. sudo test/ums/ums_gadget_test.sh 0525 a4a5 6 -f
> vfat ./dat_14M.img -
> -... where:
> -    VID - UMS device USB Vendor ID
> -    PID - UMS device USB Product ID
> -    PART_NUM - is the partition number on which UMS operates or "-"
> to use the
> -	       whole device
> -
> -Information about available partitions on the target one can read
> with using -the 'mmc part' or 'part list' commands.
> -
> -The partition num (PART_NUM) can be specified as '-' for using the
> whole device. -
> -The [-f FILE_SYSTEM] optional switch allows for formatting target
> partition to -FILE_SYSTEM.
> -
> -The last, optional [test_file] parameter is for specifying the exact
> test file -to use.
> diff --git a/test/ums/ums_gadget_test.sh b/test/ums/ums_gadget_test.sh
> deleted file mode 100755
> index 9da486b266ce..000000000000
> --- a/test/ums/ums_gadget_test.sh
> +++ /dev/null
> @@ -1,183 +0,0 @@
> -#! /bin/bash
> -
> -# Copyright (C) 2014 Samsung Electronics
> -# Lukasz Majewski <l.majewski@samsung.com>
> -#
> -# UMS operation test script
> -#
> -# SPDX-License-Identifier:	GPL-2.0+
> -
> -clear
> -
> -COLOUR_RED="\33[31m"
> -COLOUR_GREEN="\33[32m"
> -COLOUR_ORANGE="\33[33m"
> -COLOUR_DEFAULT="\33[0m"
> -
> -DIR=./
> -SUFFIX=img
> -RCV_DIR=rcv/
> -LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> -
> -cd `dirname $0`
> -../dfu/dfu_gadget_test_init.sh 33M 97M
> -
> -cleanup () {
> -    rm -rf $RCV_DIR $MNT_DIR
> -}
> -
> -control_c()
> -# run if user hits control-c
> -{
> -	echo -en "\n*** CTRL+C ***\n"
> -	umount $MNT_DIR
> -	cleanup
> -	exit 0
> -}
> -
> -# trap keyboard interrupt (control-c)
> -trap control_c SIGINT
> -
> -die () {
> -    printf "   $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
> -    cleanup
> -    exit 1
> -}
> -
> -calculate_md5sum () {
> -    MD5SUM=`md5sum $1`
> -    MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> -    echo "md5sum:"$MD5SUM
> -}
> -
> -ums_test_file () {
> -    printf
> "$COLOUR_GREEN=========================================================================================
> $COLOUR_DEFAULT\n"
> -    printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
> -
> -    mount /dev/$MEM_DEV $MNT_DIR
> -    if [ -f $MNT_DIR/dat_* ]; then
> -	rm $MNT_DIR/dat_*
> -    fi
> -
> -    cp ./$1 $MNT_DIR
> -
> -    while true; do
> -	umount $MNT_DIR > /dev/null 2>&1
> -	if [ $? -eq 0 ]; then
> -	    break
> -	fi
> -	printf "$COLOUR_ORANGE\tSleeping to wait for
> umount...$COLOUR_DEFAULT\n"
> -	sleep 1
> -    done
> -
> -    echo -n "TX: "
> -    calculate_md5sum $1
> -
> -    MD5_TX=$MD5SUM
> -    sleep 1
> -    N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> -
> -    mount /dev/$MEM_DEV $MNT_DIR
> -    cp $MNT_DIR/$1 $N_FILE || die $?
> -    rm $MNT_DIR/$1
> -    umount $MNT_DIR
> -
> -    echo -n "RX: "
> -    calculate_md5sum $N_FILE
> -    MD5_RX=$MD5SUM
> -
> -    if [ "$MD5_TX" == "$MD5_RX" ]; then
> -	printf "   $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
> -    else
> -	printf "   $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
> -	cleanup
> -	exit 1
> -    fi
> -}
> -
> -printf
> "$COLOUR_GREEN=========================================================================================
> $COLOUR_DEFAULT\n" -echo "U-boot UMS test program" -
> -if [ $EUID -ne 0 ]; then
> -   echo "You must be root to do this." 1>&2
> -   exit 100
> -fi
> -
> -if [ $# -lt 3 ]; then
> -    echo "Wrong number of arguments"
> -    echo "Example:"
> -    echo "sudo ./ums_gadget_test.sh VID PID PART_NUM [-f ext4]
> [test_file]"
> -    die
> -fi
> -
> -MNT_DIR="/mnt/tmp-ums-test"
> -
> -VID=$1; shift
> -PID=$1; shift
> -PART_NUM=$1; shift
> -
> -if [ "$1" == "-f" ]; then
> -    shift
> -    FS_TO_FORMAT=$1; shift
> -fi
> -
> -TEST_FILE=$1
> -
> -for f in `find /sys -type f -name idProduct`; do
> -     d=`dirname ${f}`
> -     if [ `cat ${d}/idVendor` != "${VID}" ]; then
> -	 continue
> -     fi
> -     if [ `cat ${d}/idProduct` != "${PID}" ]; then
> -	 continue
> -     fi
> -     USB_DEV=${d}
> -     break
> -done
> -
> -if [ -z "${USB_DEV}" ]; then
> -     echo "Connect target"
> -     echo "e.g. ums 0 mmc 0"
> -     exit 1
> -fi
> -
> -MEM_DEV=`find $USB_DEV -type d -name "sd[a-z]" | awk -F/ '{print
> $(NF)}' -` -
> -mkdir -p $RCV_DIR
> -if [ ! -d $MNT_DIR ]; then
> -    mkdir -p $MNT_DIR
> -fi
> -
> -if [ "$PART_NUM" == "-" ]; then
> -    PART_NUM=""
> -fi
> -MEM_DEV=$MEM_DEV$PART_NUM
> -
> -if [ -n "$FS_TO_FORMAT" ]; then
> -    echo -n "Formatting partition /dev/$MEM_DEV to $FS_TO_FORMAT"
> -    mkfs -t $FS_TO_FORMAT /dev/$MEM_DEV > /dev/null 2>&1
> -    if [ $? -eq 0 ]; then
> -	printf " $COLOUR_GREEN DONE $COLOUR_DEFAULT \n"
> -    else
> -	die
> -    fi
> -fi
> -
> -printf "Mount: /dev/$MEM_DEV \n"
> -
> -if [ -n "$TEST_FILE" ]; then
> -    if [ ! -e $TEST_FILE ]; then
> -	echo "No file: $TEST_FILE"
> -	die
> -    fi
> -    ums_test_file $TEST_FILE
> -else
> -    for file in $DIR*.$SUFFIX
> -    do
> -	ums_test_file $file
> -    done
> -fi
> -
> -cleanup
> -
> -exit 0

Acked-by: Lukasz Majewski <l.majewski@samsung.com>

Stephen, thanks for converting DFU and UMS to pytest code.

-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group