dummy file read periodically for external USB Hard Disk

dummy file read periodically for external USB Hard Disk

[loody]

hi all:
    we met a USB Hard Disk that will go to suspend if host stop
sending scsi read command over 5mins.
    To save the IO, kernel will keep the file in page cache as much as
he can and under this circumstances, the read command may disappear
for a while longer enough to cause the device suspend.

    is there any kernel config or module parameter can do the dummy
read periodically, even the sector 0 (MBR) is fine.
    or is there any kernel api I can use to read sector 0(MBR) maybe
every 4mins?

appreciate your help in advance,

Re: dummy file read periodically for external USB Hard Disk

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

On Mon, 7 Jul 2014 22:42:29 +0800 loody <miloody@gmail.com> wrote:

> hi all:
>     we met a USB Hard Disk that will go to suspend if host stop
> sending scsi read command over 5mins.
>     To save the IO, kernel will keep the file in page cache as much as
> he can and under this circumstances, the read command may disappear
> for a while longer enough to cause the device suspend.
> 
>     is there any kernel config or module parameter can do the dummy
> read periodically, even the sector 0 (MBR) is fine.
>     or is there any kernel api I can use to read sector 0(MBR) maybe
> every 4mins?
> 
> appreciate your help in advance,
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Open the device with O_DIRECT and read a block every 4 minutes.
That should keep it awake.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: dummy file read periodically for external USB Hard Disk

[Dave Jones]

On Tue, Jul 08, 2014 at 01:01:27PM +1000, NeilBrown wrote:
 > On Mon, 7 Jul 2014 22:42:29 +0800 loody <miloody@gmail.com> wrote:
 > 
 > > hi all:
 > >     we met a USB Hard Disk that will go to suspend if host stop
 > > sending scsi read command over 5mins.
 > >     To save the IO, kernel will keep the file in page cache as much as
 > > he can and under this circumstances, the read command may disappear
 > > for a while longer enough to cause the device suspend.
 > > 
 > >     is there any kernel config or module parameter can do the dummy
 > > read periodically, even the sector 0 (MBR) is fine.
 > >     or is there any kernel api I can use to read sector 0(MBR) maybe
 > > every 4mins?
 > 
 > Open the device with O_DIRECT and read a block every 4 minutes.
 > That should keep it awake.

modifying /sys/module/usbcore/parameters/autosuspend might also have the
desired effect without the need for periodic reads.

	Dave

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi NeilBrown:
>
> Open the device with O_DIRECT and read a block every 4 minutes.
> That should keep it awake.
Could we do that within kernel module?

appreciate your kind help,

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi NeilBrown:
we use below c source but there is no read command firing from usb
host to device.
except O_DIRECT, is there any flag we need to use?
appreciate all your kind help,

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

char message[] = "/mnt/usb/4854344154343452/
test.txt";
int main()
{
   int fd;
   char buffer[5];
   int count = 0;
   char *buf="1234567890";
   if((fd=open(message,O_CREAT|O_TRUNC|O_RDWR|O_DIRECT, 0777))<0)
   {
       perror("open");
       return -1;
   }
   printf("fd=%d\n", fd);
   write(fd, buf, strlen(buf));
   while(1){
       lseek(fd,0,SEEK_SET);
       sleep(3);
       count = read(fd, buffer, 3);
       printf("count=%d,%x,%x,%x\n", count,buffer[0],buffer[1],buffer[2]);
   }

}

2014-07-08 11:01 GMT+08:00 NeilBrown <neilb@suse.de>:
> On Mon, 7 Jul 2014 22:42:29 +0800 loody <miloody@gmail.com> wrote:
>
>> hi all:
>>     we met a USB Hard Disk that will go to suspend if host stop
>> sending scsi read command over 5mins.
>>     To save the IO, kernel will keep the file in page cache as much as
>> he can and under this circumstances, the read command may disappear
>> for a while longer enough to cause the device suspend.
>>
>>     is there any kernel config or module parameter can do the dummy
>> read periodically, even the sector 0 (MBR) is fine.
>>     or is there any kernel api I can use to read sector 0(MBR) maybe
>> every 4mins?
>>
>> appreciate your help in advance,
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> Open the device with O_DIRECT and read a block every 4 minutes.
> That should keep it awake.
>
> NeilBrown



-- 
Regards,

Re: dummy file read periodically for external USB Hard Disk

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

On Wed, 9 Jul 2014 22:13:28 +0800 loody <miloody@gmail.com> wrote:

> hi NeilBrown:
> we use below c source but there is no read command firing from usb
> host to device.
> except O_DIRECT, is there any flag we need to use?
> appreciate all your kind help,
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> 
> char message[] = "/mnt/usb/4854344154343452/
> test.txt";
> int main()
> {
>    int fd;
>    char buffer[5];
>    int count = 0;
>    char *buf="1234567890";
>    if((fd=open(message,O_CREAT|O_TRUNC|O_RDWR|O_DIRECT, 0777))<0)

I said "Open the device with O_DIRECT".  You are opening a file in the
filesystem which is mounted from the device.  That is a different thing.

>    {
>        perror("open");
>        return -1;
>    }
>    printf("fd=%d\n", fd);
>    write(fd, buf, strlen(buf));
>    while(1){
>        lseek(fd,0,SEEK_SET);
>        sleep(3);
>        count = read(fd, buffer, 3);

Did you do any research to understand how O_DIRECT works?
I recommend the man page for "open(2)".

You need to read thoroughly, but towards the end it says:

       Under Linux 2.4, transfer sizes, and the alignment of the user buffer and  the
       file  offset  must all be multiples of the logical block size of the file sys-
       tem.  Under Linux 2.6, alignment to 512-byte boundaries suffices.

Neither your buffer nor your IO size is 512-byte aligned.

NeilBrown


>        printf("count=%d,%x,%x,%x\n", count,buffer[0],buffer[1],buffer[2]);
>    }
> 
> }
> 
> 2014-07-08 11:01 GMT+08:00 NeilBrown <neilb@suse.de>:
> > On Mon, 7 Jul 2014 22:42:29 +0800 loody <miloody@gmail.com> wrote:
> >
> >> hi all:
> >>     we met a USB Hard Disk that will go to suspend if host stop
> >> sending scsi read command over 5mins.
> >>     To save the IO, kernel will keep the file in page cache as much as
> >> he can and under this circumstances, the read command may disappear
> >> for a while longer enough to cause the device suspend.
> >>
> >>     is there any kernel config or module parameter can do the dummy
> >> read periodically, even the sector 0 (MBR) is fine.
> >>     or is there any kernel api I can use to read sector 0(MBR) maybe
> >> every 4mins?
> >>
> >> appreciate your help in advance,
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> > Open the device with O_DIRECT and read a block every 4 minutes.
> > That should keep it awake.
> >
> > NeilBrown
> 
> 
> 


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi NeilBrown:

2014-07-10 8:47 GMT+08:00 NeilBrown <neilb@suse.de>:
> On Wed, 9 Jul 2014 22:13:28 +0800 loody <miloody@gmail.com> wrote:
>
>> hi NeilBrown:
>> we use below c source but there is no read command firing from usb
>> host to device.
>> except O_DIRECT, is there any flag we need to use?
>> appreciate all your kind help,
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <sys/types.h>
>> #include <sys/stat.h>
>> #include <fcntl.h>
>>
>> char message[] = "/mnt/usb/4854344154343452/
>> test.txt";
>> int main()
>> {
>>    int fd;
>>    char buffer[5];
>>    int count = 0;
>>    char *buf="1234567890";
>>    if((fd=open(message,O_CREAT|O_TRUNC|O_RDWR|O_DIRECT, 0777))<0)
>
> I said "Open the device with O_DIRECT".  You are opening a file in the
> filesystem which is mounted from the device.  That is a different thing.
sorry for misunderstanding your explanation.

>
>>    {
>>        perror("open");
>>        return -1;
>>    }
>>    printf("fd=%d\n", fd);
>>    write(fd, buf, strlen(buf));
>>    while(1){
>>        lseek(fd,0,SEEK_SET);
>>        sleep(3);
>>        count = read(fd, buffer, 3);
>
> Did you do any research to understand how O_DIRECT works?
> I recommend the man page for "open(2)".
>
> You need to read thoroughly, but towards the end it says:
>
>        Under Linux 2.4, transfer sizes, and the alignment of the user buffer and  the
>        file  offset  must all be multiples of the logical block size of the file sys-
>        tem.  Under Linux 2.6, alignment to 512-byte boundaries suffices.
>
> Neither your buffer nor your IO size is 512-byte aligned.
I follow your suggestion and try to read /dev/sda or /dev/sda1 for
512Bytes like below.
But the read back count is -1, that mean the read is not successful.
I try to open both with "O_DIRECT" or "O_DIRECT|O_RDONLY" but all of
them get read back count are -1.
Does that mean block device node not support system read command?
appreciate your help,

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

char message[] = "/dev/sda1";
int main()
{
   int fd;
   char buffer[1024];
   int count;
   //if((fd=open(message,O_DIRECT))<0)
   if((fd=open(message,O_DIRECT|O_RDONLY))<0)
   {
       perror("open");
       return -1;
   }
   printf("fd=%d\n", fd);
   while(1){
       sleep(3);
       count = read(fd, buffer, 512); //read back fail
       printf("count=%d,%x,%x,%x\n", count,buffer[0],buffer[1],buffer[2]);
   }
close(fd);
}

Re: dummy file read periodically for external USB Hard Disk

[Randy Dunlap]

On 07/11/14 02:04, loody wrote:
> hi NeilBrown:
> 
> 2014-07-10 8:47 GMT+08:00 NeilBrown <neilb@suse.de>:
>> On Wed, 9 Jul 2014 22:13:28 +0800 loody <miloody@gmail.com> wrote:
>>
>>> hi NeilBrown:
>>> we use below c source but there is no read command firing from usb
>>> host to device.
>>> except O_DIRECT, is there any flag we need to use?
>>> appreciate all your kind help,
>>>
>>> #include <stdio.h>
>>> #include <stdlib.h>
>>> #include <sys/types.h>
>>> #include <sys/stat.h>
>>> #include <fcntl.h>
>>>
>>> char message[] = "/mnt/usb/4854344154343452/
>>> test.txt";
>>> int main()
>>> {
>>>    int fd;
>>>    char buffer[5];
>>>    int count = 0;
>>>    char *buf="1234567890";
>>>    if((fd=open(message,O_CREAT|O_TRUNC|O_RDWR|O_DIRECT, 0777))<0)
>>
>> I said "Open the device with O_DIRECT".  You are opening a file in the
>> filesystem which is mounted from the device.  That is a different thing.
> sorry for misunderstanding your explanation.
> 
>>
>>>    {
>>>        perror("open");
>>>        return -1;
>>>    }
>>>    printf("fd=%d\n", fd);
>>>    write(fd, buf, strlen(buf));
>>>    while(1){
>>>        lseek(fd,0,SEEK_SET);
>>>        sleep(3);
>>>        count = read(fd, buffer, 3);
>>
>> Did you do any research to understand how O_DIRECT works?
>> I recommend the man page for "open(2)".
>>
>> You need to read thoroughly, but towards the end it says:
>>
>>        Under Linux 2.4, transfer sizes, and the alignment of the user buffer and  the
>>        file  offset  must all be multiples of the logical block size of the file sys-
>>        tem.  Under Linux 2.6, alignment to 512-byte boundaries suffices.
>>
>> Neither your buffer nor your IO size is 512-byte aligned.
> I follow your suggestion and try to read /dev/sda or /dev/sda1 for
> 512Bytes like below.
> But the read back count is -1, that mean the read is not successful.
> I try to open both with "O_DIRECT" or "O_DIRECT|O_RDONLY" but all of
> them get read back count are -1.
> Does that mean block device node not support system read command?
> appreciate your help,

It means that the <buffer> still is not aligned to a 512-byte boundary.
Try
   char buffer[1024] __attribute__ ((aligned(1024)));

Well, that is aligned to 1024 bytes, not 512, but whatever.

> 
> #define _GNU_SOURCE
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> 
> char message[] = "/dev/sda1";
> int main()
> {
>    int fd;
>    char buffer[1024];
>    int count;
>    //if((fd=open(message,O_DIRECT))<0)
>    if((fd=open(message,O_DIRECT|O_RDONLY))<0)
>    {
>        perror("open");
>        return -1;
>    }
>    printf("fd=%d\n", fd);
>    while(1){
>        sleep(3);
>        count = read(fd, buffer, 512); //read back fail
>        printf("count=%d,%x,%x,%x\n", count,buffer[0],buffer[1],buffer[2]);
>    }
> close(fd);
> }


-- 
~Randy

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi Randy and All:

>
> It means that the <buffer> still is not aligned to a 512-byte boundary.
> Try
>    char buffer[1024] __attribute__ ((aligned(1024)));
>
> Well, that is aligned to 1024 bytes, not 512, but whatever.
the suggestion you gave to me works ^^
is there any alignment check in kernel FS such that it will directly
output to device instead of getting data from page cache?

appreciate all your help,

Re: dummy file read periodically for external USB Hard Disk

[Randy Dunlap]

On 07/18/2014 10:31 AM, loody wrote:
> hi Randy and All:
> 
>>
>> It means that the <buffer> still is not aligned to a 512-byte boundary.
>> Try
>>    char buffer[1024] __attribute__ ((aligned(1024)));
>>
>> Well, that is aligned to 1024 bytes, not 512, but whatever.
> the suggestion you gave to me works ^^
> is there any alignment check in kernel FS such that it will directly
> output to device instead of getting data from page cache?

Not sure that I understand your question, but I think that using
O_DIRECT in the open() call attempts to do what you are asking.

Please read 'man open' and search for O_DIRECT.

-- 
~Randy

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi Randy:
>
> Not sure that I understand your question, but I think that using
> O_DIRECT in the open() call attempts to do what you are asking.
>
> Please read 'man open' and search for O_DIRECT.
sorry for making you confused.
I just want to know why O_DIRECT

-- 
Regards,

Re: dummy file read periodically for external USB Hard Disk

[loody]

hi Randy:

2014-07-19 2:13 GMT+08:00 Randy Dunlap <rdunlap@infradead.org>:
> On 07/18/2014 10:31 AM, loody wrote:
>> hi Randy and All:
>>
>>>
>>> It means that the <buffer> still is not aligned to a 512-byte boundary.
>>> Try
>>>    char buffer[1024] __attribute__ ((aligned(1024)));
>>>
>>> Well, that is aligned to 1024 bytes, not 512, but whatever.
>> the suggestion you gave to me works ^^
>> is there any alignment check in kernel FS such that it will directly
>> output to device instead of getting data from page cache?
>
> Not sure that I understand your question, but I think that using
> O_DIRECT in the open() call attempts to do what you are asking.
>
> Please read 'man open' and search for O_DIRECT.
sorry for making you confused. ^^
I just want to know why O_DIRECT is not enough and need to announce
the buffer size as 512-byte boundary.
Is there any checking mechanism for 512-byte when O_DIRECT flag apply?

appreciate all your kind help,

Re: dummy file read periodically for external USB Hard Disk

[Randy Dunlap]

On 07/19/2014 04:02 AM, loody wrote:
> hi Randy:
> 
> 2014-07-19 2:13 GMT+08:00 Randy Dunlap <rdunlap@infradead.org>:
>> On 07/18/2014 10:31 AM, loody wrote:
>>> hi Randy and All:
>>>
>>>>
>>>> It means that the <buffer> still is not aligned to a 512-byte boundary.
>>>> Try
>>>>    char buffer[1024] __attribute__ ((aligned(1024)));
>>>>
>>>> Well, that is aligned to 1024 bytes, not 512, but whatever.
>>> the suggestion you gave to me works ^^
>>> is there any alignment check in kernel FS such that it will directly
>>> output to device instead of getting data from page cache?
>>
>> Not sure that I understand your question, but I think that using
>> O_DIRECT in the open() call attempts to do what you are asking.
>>
>> Please read 'man open' and search for O_DIRECT.
> sorry for making you confused. ^^
> I just want to know why O_DIRECT is not enough and need to announce
> the buffer size as 512-byte boundary.
> 
> Is there any checking mechanism for 512-byte when O_DIRECT flag apply?

Sorry, I have no idea.

-- 
~Randy