[LTP] [PATCH 1/2] scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

[LTP] [PATCH 1/2] scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

[Konstantin Khorenko via ltp]

Sometimes we got failures like:
 growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288
 growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
 gf217       1  TFAIL  :  growfiles.c:134: Test failed

Which looked strange as partial write is something usual and valid.
It turned out that lio_write_buffer() has the code cycle writes in case
of a partial write happens, but it anyway returns the amount of bytes
written by the LAST write.

And upper growfile() consider the returned amount from
lio_write_buffer() to be less than it tried to write and fails the
testcase.

Fix lio_write_buffer() to always return total bytes written, even in
case partial writes.

Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/tlibio.c b/lib/tlibio.c
index cc110d1c9..8298e2de9 100644
--- a/lib/tlibio.c
+++ b/lib/tlibio.c
@@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_written = 0;/* as we cycle writes in case of partial writes, */
+				/* we have to report up total bytes written */
 	char *io_type;		/* Holds string of type of io */
 	int omethod = method;
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
@@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_written += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);
 
-					return ret;
+					return totally_written + ret;
 				}
 			}
 			wait4sync_io(fd, 0);
-- 
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 2/2] scsi_debug/tlibio/lio_read_buffer: Always return total amount of read bytes

[Konstantin Khorenko via ltp]

This issue behind this patch is noticed while fixing the
lio_write_buffer().

Here in lio_read_buffer() we have similar situation: in case of a
partial read, we cycle, but lio_read_buffer() returns back the amount of
bytes read during the last read() call while it's expected to return the
whole amount of read bytes.

Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
 lib/tlibio.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/tlibio.c b/lib/tlibio.c
index 8298e2de9..604663307 100644
--- a/lib/tlibio.c
+++ b/lib/tlibio.c
@@ -1112,6 +1112,8 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 		long wrd)	/* to allow future features, use zero for now */
 {
 	int ret = 0;		/* syscall return or used to get random method */
+	int totally_read = 0;	/* as we cycle reads in case of partial reads, */
+				/* we have to report up total bytes read */
 	char *io_type;		/* Holds string of type of io */
 	int listio_cmd;		/* Holds the listio/lio_listio cmd */
 	int omethod = method;
@@ -1323,13 +1325,14 @@ int lio_read_buffer(int fd,	/* open file descriptor */
 						fd, size, ret);
 					size -= ret;
 					buffer += ret;
+					totally_read += ret;
 				} else {
 					if (Debug_level > 1)
 						printf
 						    ("DEBUG %s/%d: read completed without error (ret %d)\n",
 						     __FILE__, __LINE__, ret);
 
-					return ret;
+					return totally_read + ret;
 				}
 			}
 			wait4sync_io(fd, 1);
-- 
2.31.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/2] scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

[Petr Vorel]

Hi Konstantin,

> Sometimes we got failures like:
>  growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288
>  growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
>  gf217       1  TFAIL  :  growfiles.c:134: Test failed

I wonder on which circumstances do you get this error. Running on container?

> Which looked strange as partial write is something usual and valid.
> It turned out that lio_write_buffer() has the code cycle writes in case
> of a partial write happens, but it anyway returns the amount of bytes
> written by the LAST write.

> And upper growfile() consider the returned amount from
> lio_write_buffer() to be less than it tried to write and fails the
> testcase.

> Fix lio_write_buffer() to always return total bytes written, even in
> case partial writes.

> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
> ---
>  lib/tlibio.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)

> diff --git a/lib/tlibio.c b/lib/tlibio.c
> index cc110d1c9..8298e2de9 100644
> --- a/lib/tlibio.c
> +++ b/lib/tlibio.c
> @@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
>  		long wrd)	/* to allow future features, use zero for now */
>  {
>  	int ret = 0;		/* syscall return or used to get random method */
> +	int totally_written = 0;/* as we cycle writes in case of partial writes, */
> +				/* we have to report up total bytes written */

Nit: I'd do multiline comment (much more readable):

	/* as we cycle writes in case of partial writes,
	 * we have to report up total bytes written */
	int totally_written = 0;

Otherwise LGTM.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

>  	char *io_type;		/* Holds string of type of io */
>  	int omethod = method;
>  	int listio_cmd;		/* Holds the listio/lio_listio cmd */
> @@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
>  						fd, size, ret);
>  					size -= ret;
>  					buffer += ret;
> +					totally_written += ret;
>  				} else {
>  					if (Debug_level > 1)
>  						printf
>  						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
>  						     __FILE__, __LINE__, ret);

BTW growfiles.c and other files based on tlibio.c would deserve big cleanup and
rewrite into new API.

Kind regards,
Petr

> -					return ret;
> +					return totally_written + ret;
>  				}
>  			}
>  			wait4sync_io(fd, 0);

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 2/2] scsi_debug/tlibio/lio_read_buffer: Always return total amount of read bytes

[Petr Vorel]

> This issue behind this patch is noticed while fixing the
> lio_write_buffer().

> Here in lio_read_buffer() we have similar situation: in case of a
> partial read, we cycle, but lio_read_buffer() returns back the amount of
> bytes read during the last read() call while it's expected to return the
> whole amount of read bytes.

> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
> ---
>  lib/tlibio.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)

> diff --git a/lib/tlibio.c b/lib/tlibio.c
> index 8298e2de9..604663307 100644
> --- a/lib/tlibio.c
> +++ b/lib/tlibio.c
> @@ -1112,6 +1112,8 @@ int lio_read_buffer(int fd,	/* open file descriptor */
>  		long wrd)	/* to allow future features, use zero for now */
>  {
>  	int ret = 0;		/* syscall return or used to get random method */
> +	int totally_read = 0;	/* as we cycle reads in case of partial reads, */
> +				/* we have to report up total bytes read */

Again, multiline comment would be more readable:

	/* as we cycle reads in case of partial reads,
	 * we have to report up total bytes read */
	int totally_read = 0;

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

>  	char *io_type;		/* Holds string of type of io */
>  	int listio_cmd;		/* Holds the listio/lio_listio cmd */
>  	int omethod = method;
> @@ -1323,13 +1325,14 @@ int lio_read_buffer(int fd,	/* open file descriptor */
>  						fd, size, ret);
>  					size -= ret;
>  					buffer += ret;
> +					totally_read += ret;
>  				} else {
>  					if (Debug_level > 1)
>  						printf
>  						    ("DEBUG %s/%d: read completed without error (ret %d)\n",
>  						     __FILE__, __LINE__, ret);

> -					return ret;
> +					return totally_read + ret;
>  				}
>  			}
>  			wait4sync_io(fd, 1);

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/2] scsi_debug/tlibio/lio_write_buffer: Always return total amount of written bytes

[Konstantin Khorenko via ltp]

On 17.05.2023 02:23, Petr Vorel wrote:
> Hi Konstantin,
> 
>> Sometimes we got failures like:
>>   growfiles(gf217): 65884 growfiles.c/2262: 104203 tlibio.c/744 write(3, buf, 5000) returned=2288
>>   growfiles(gf217): 65884 growfiles.c/1765: 104203 Hit max errors value of 1
>>   gf217       1  TFAIL  :  growfiles.c:134: Test failed
> 
> I wonder on which circumstances do you get this error. Running on container?

Yes, your guess is correct. :) Our QA runs ltp in Containers and VMs under high load,
i mean when many Containers/VMs host on a physical Node and run in parallel different tests.

>> Which looked strange as partial write is something usual and valid.
>> It turned out that lio_write_buffer() has the code cycle writes in case
>> of a partial write happens, but it anyway returns the amount of bytes
>> written by the LAST write.
> 
>> And upper growfile() consider the returned amount from
>> lio_write_buffer() to be less than it tried to write and fails the
>> testcase.
> 
>> Fix lio_write_buffer() to always return total bytes written, even in
>> case partial writes.
> 
>> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
>> ---
>>   lib/tlibio.c | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
>> diff --git a/lib/tlibio.c b/lib/tlibio.c
>> index cc110d1c9..8298e2de9 100644
>> --- a/lib/tlibio.c
>> +++ b/lib/tlibio.c
>> @@ -539,6 +539,8 @@ int lio_write_buffer(int fd,		/* open file descriptor */
>>   		long wrd)	/* to allow future features, use zero for now */
>>   {
>>   	int ret = 0;		/* syscall return or used to get random method */
>> +	int totally_written = 0;/* as we cycle writes in case of partial writes, */
>> +				/* we have to report up total bytes written */
> 
> Nit: I'd do multiline comment (much more readable):

No problem, will redo and resend.

Thank you for the review!


Best regards,
Konstantin Khorenko

> 
> 	/* as we cycle writes in case of partial writes,
> 	 * we have to report up total bytes written */
> 	int totally_written = 0;
> 
> Otherwise LGTM.
> 
> Reviewed-by: Petr Vorel <pvorel@suse.cz>
> 
>>   	char *io_type;		/* Holds string of type of io */
>>   	int omethod = method;
>>   	int listio_cmd;		/* Holds the listio/lio_listio cmd */
>> @@ -745,13 +747,14 @@ int lio_write_buffer(int fd,		/* open file descriptor */
>>   						fd, size, ret);
>>   					size -= ret;
>>   					buffer += ret;
>> +					totally_written += ret;
>>   				} else {
>>   					if (Debug_level > 1)
>>   						printf
>>   						    ("DEBUG %s/%d: write completed without error (ret %d)\n",
>>   						     __FILE__, __LINE__, ret);
> 
> BTW growfiles.c and other files based on tlibio.c would deserve big cleanup and
> rewrite into new API.
> 
> Kind regards,
> Petr
> 
>> -					return ret;
>> +					return totally_written + ret;
>>   				}
>>   			}
>>   			wait4sync_io(fd, 0);

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp