when are buffers/caches flushed?

when are buffers/caches flushed?

[Steven Scholz]

Hi there,

According to an (old) man page of sync(2)

        According to  the  standard  specification  (e.g.,  SVID),
        sync()  schedules  the  writes,  but may return before the
        actual writing is done.   However,  since  version  1.3.20
        Linux  does actually wait.  (This still does not guarantee
        data integrity: modern disks have large caches.)

How about recent kernels? Does sync() block until buffers are flushed?
How can I find out if the disk caches are actually flushed?

I want to make sure that all data is flushed to my disk drive before powering 
down the system.

Thanks.

--
Steven

Re: when are buffers/caches flushed?

[Tejun Heo]

Steven Scholz wrote:
> Hi there,
> 
> According to an (old) man page of sync(2)
> 
>        According to  the  standard  specification  (e.g.,  SVID),
>        sync()  schedules  the  writes,  but may return before the
>        actual writing is done.   However,  since  version  1.3.20
>        Linux  does actually wait.  (This still does not guarantee
>        data integrity: modern disks have large caches.)
> 
> How about recent kernels? Does sync() block until buffers are flushed?
> How can I find out if the disk caches are actually flushed?
> 
> I want to make sure that all data is flushed to my disk drive before 
> powering down the system.
> 
> Thanks.
> 

  All disk caches are flushed before shutdown via the following path.

kernel/sys.c::sys_reboot()
kernel/drivers/base/power/shutdown.c::device_shutdown()
  driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown()
drivers/ide/ide-disk.c::ide_cacheflush_p()
drivers/ide/ide-disk.c::do_idedisk_flushcache()

  And, AFAIK, sync() doesn't flush disk caches.

-- 
tejun

Re: when are buffers/caches flushed?

[Steven Scholz]

Tejun Heo wrote:

> Steven Scholz wrote:
> 
>> Hi there,
>>
>> According to an (old) man page of sync(2)
>>
>>        According to  the  standard  specification  (e.g.,  SVID),
>>        sync()  schedules  the  writes,  but may return before the
>>        actual writing is done.   However,  since  version  1.3.20
>>        Linux  does actually wait.  (This still does not guarantee
>>        data integrity: modern disks have large caches.)
>>
>> How about recent kernels? Does sync() block until buffers are flushed?
>> How can I find out if the disk caches are actually flushed?
>>
>> I want to make sure that all data is flushed to my disk drive before 
>> powering down the system.
>>
>> Thanks.
>>
> 
>  All disk caches are flushed before shutdown via the following path.
> 
> kernel/sys.c::sys_reboot()
> kernel/drivers/base/power/shutdown.c::device_shutdown()
>  driver specific ->shutdown callback, for ide disks, the path is
> drivers/ide/ide-disk.c::ide_device_shutdown()
> drivers/ide/ide-disk.c::ide_cacheflush_p()
> drivers/ide/ide-disk.c::do_idedisk_flushcache()
> 
>  And, AFAIK, sync() doesn't flush disk caches.

How about umount?

--
Steven




-- 
Steven Scholz

imc Measurement & Control               imc Meßsysteme GmbH
Voltastr. 5                             Voltastr. 5
13355 Berlin                            13355 Berlin
Germany                                 Deutschland

Re: when are buffers/caches flushed?

[Tejun Heo]

Steven Scholz wrote:
> Tejun Heo wrote:
> 
>> Steven Scholz wrote:
>>
>>> Hi there,
>>>
>>> According to an (old) man page of sync(2)
>>>
>>>        According to  the  standard  specification  (e.g.,  SVID),
>>>        sync()  schedules  the  writes,  but may return before the
>>>        actual writing is done.   However,  since  version  1.3.20
>>>        Linux  does actually wait.  (This still does not guarantee
>>>        data integrity: modern disks have large caches.)
>>>
>>> How about recent kernels? Does sync() block until buffers are flushed?
>>> How can I find out if the disk caches are actually flushed?
>>>
>>> I want to make sure that all data is flushed to my disk drive before 
>>> powering down the system.
>>>
>>> Thanks.
>>>
>>
>>  All disk caches are flushed before shutdown via the following path.
>>
>> kernel/sys.c::sys_reboot()
>> kernel/drivers/base/power/shutdown.c::device_shutdown()
>>  driver specific ->shutdown callback, for ide disks, the path is
>> drivers/ide/ide-disk.c::ide_device_shutdown()
>> drivers/ide/ide-disk.c::ide_cacheflush_p()
>> drivers/ide/ide-disk.c::do_idedisk_flushcache()
>>
>>  And, AFAIK, sync() doesn't flush disk caches.
> 
> 
> How about umount?
> 

[CC'ing Bartlomiej (Hi!)]

  Hmmm, umount doesn't.  I think maybe adding cache flushing to sync and 
umount can be helpful.

  And one more thing, ide-disk doesn't flush cache when shutting down. 
It flushes only when rebooting.

from ide_device_shutdown()...

#ifdef	CONFIG_ALPHA
	/* On Alpha, halt(8) doesn't actually turn the machine off,
	   it puts you into the sort of firmware monitor. Typically,
	   it's used to boot another kernel image, so it's not much
	   different from reboot(8). Therefore, we don't need to
	   spin down the disk in this case, especially since Alpha
	   firmware doesn't handle disks in standby mode properly.
	   On the other hand, it's reasonably safe to turn the power
	   off when the shutdown process reaches the firmware prompt,
	   as the firmware initialization takes rather long time -
	   at least 10 seconds, which should be sufficient for
	   the disk to expire its write cache. */
	if (system_state != SYSTEM_POWER_OFF) {
#else
	if (system_state == SYSTEM_RESTART) {
#endif
		ide_cacheflush_p(drive);
		return;
	}

  It seems that the if clause are there due to some historic thing.  The 
comment suggests that the body in the if clause spun down the drive 
previously, and we needed the if to avoid spinning down unnecessarily, 
and the if is left there after we changed the body to flush cache.

  Bartlomiej, am I missing something?

-- 
tejun

Re: when are buffers/caches flushed?

[Steven Scholz]

Tejun,

>>>> How about recent kernels? Does sync() block until buffers are flushed?
>>>> How can I find out if the disk caches are actually flushed?
>>>>
>>>> I want to make sure that all data is flushed to my disk drive before 
>>>> powering down the system.
>>>
>>>  All disk caches are flushed before shutdown via the following path.
>>>
>>> kernel/sys.c::sys_reboot()
>>> kernel/drivers/base/power/shutdown.c::device_shutdown()
>>>  driver specific ->shutdown callback, for ide disks, the path is
>>> drivers/ide/ide-disk.c::ide_device_shutdown()
>>> drivers/ide/ide-disk.c::ide_cacheflush_p()
>>> drivers/ide/ide-disk.c::do_idedisk_flushcache()
>>>
>>>  And, AFAIK, sync() doesn't flush disk caches.
>>
>> How about umount?
>>
> 
> [CC'ing Bartlomiej (Hi!)]
> 
>  Hmmm, umount doesn't.  I think maybe adding cache flushing to sync and 
> umount can be helpful.

Ehm. So you're saying umount does _not_ flush the disk caches? BUT it will flush 
the the fs buffers, right?

>  And one more thing, ide-disk doesn't flush cache when shutting down. It 
> flushes only when rebooting.

How about ide-cs then? When I unregister a drive?

--
Steven

Re: when are buffers/caches flushed?

[Tejun Heo]

Steven Scholz wrote:
> Tejun,
> 
>>>>> How about recent kernels? Does sync() block until buffers are flushed?
>>>>> How can I find out if the disk caches are actually flushed?
>>>>>
>>>>> I want to make sure that all data is flushed to my disk drive 
>>>>> before powering down the system.
>>>>
>>>>
>>>>  All disk caches are flushed before shutdown via the following path.
>>>>
>>>> kernel/sys.c::sys_reboot()
>>>> kernel/drivers/base/power/shutdown.c::device_shutdown()
>>>>  driver specific ->shutdown callback, for ide disks, the path is
>>>> drivers/ide/ide-disk.c::ide_device_shutdown()
>>>> drivers/ide/ide-disk.c::ide_cacheflush_p()
>>>> drivers/ide/ide-disk.c::do_idedisk_flushcache()
>>>>
>>>>  And, AFAIK, sync() doesn't flush disk caches.
>>>
>>>
>>> How about umount?
>>>
>>
>> [CC'ing Bartlomiej (Hi!)]
>>
>>  Hmmm, umount doesn't.  I think maybe adding cache flushing to sync 
>> and umount can be helpful.
> 
> 
> Ehm. So you're saying umount does _not_ flush the disk caches? BUT it 
> will flush the the fs buffers, right?

  Yeap, but it seems that currently ide-disk seems to flush only on 
reboot not on halt.  As soon as Bartlomiej confirms it, I'll submit a 
patch or he'll fix it.

> 
>>  And one more thing, ide-disk doesn't flush cache when shutting down. 
>> It flushes only when rebooting.
> 
> 
> How about ide-cs then? When I unregister a drive?
> 

  ide-cs is low-level driver, it just registers ide device with upper 
ide midlayer and ide-disk handles all the rest, so it's all the same there.

-- 
tejun

Re: when are buffers/caches flushed?

[Bartlomiej Zolnierkiewicz]

Hi,

On 8/18/05, Tejun Heo <htejun@gmail.com> wrote:
> Steven Scholz wrote:
> > Tejun Heo wrote:
> >
> >> Steven Scholz wrote:
> >>
> >>> Hi there,
> >>>
> >>> According to an (old) man page of sync(2)
> >>>
> >>>        According to  the  standard  specification  (e.g.,  SVID),
> >>>        sync()  schedules  the  writes,  but may return before the
> >>>        actual writing is done.   However,  since  version  1.3.20
> >>>        Linux  does actually wait.  (This still does not guarantee
> >>>        data integrity: modern disks have large caches.)
> >>>
> >>> How about recent kernels? Does sync() block until buffers are flushed?
> >>> How can I find out if the disk caches are actually flushed?
> >>>
> >>> I want to make sure that all data is flushed to my disk drive before
> >>> powering down the system.
> >>>
> >>> Thanks.
> >>>
> >>
> >>  All disk caches are flushed before shutdown via the following path.
> >>
> >> kernel/sys.c::sys_reboot()
> >> kernel/drivers/base/power/shutdown.c::device_shutdown()
> >>  driver specific ->shutdown callback, for ide disks, the path is
> >> drivers/ide/ide-disk.c::ide_device_shutdown()
> >> drivers/ide/ide-disk.c::ide_cacheflush_p()
> >> drivers/ide/ide-disk.c::do_idedisk_flushcache()
> >>
> >>  And, AFAIK, sync() doesn't flush disk caches.
> >
> >
> > How about umount?
> >
> 
> [CC'ing Bartlomiej (Hi!)]
> 
>   Hmmm, umount doesn't.  I think maybe adding cache flushing to sync and
> umount can be helpful.

I think that cache is flushed on umount through idedisk_release().

>   And one more thing, ide-disk doesn't flush cache when shutting down.
> It flushes only when rebooting.

Please see ide_disk_remove().
 
> from ide_device_shutdown()...
> 
> #ifdef  CONFIG_ALPHA
>         /* On Alpha, halt(8) doesn't actually turn the machine off,
>            it puts you into the sort of firmware monitor. Typically,
>            it's used to boot another kernel image, so it's not much
>            different from reboot(8). Therefore, we don't need to
>            spin down the disk in this case, especially since Alpha
>            firmware doesn't handle disks in standby mode properly.
>            On the other hand, it's reasonably safe to turn the power
>            off when the shutdown process reaches the firmware prompt,
>            as the firmware initialization takes rather long time -
>            at least 10 seconds, which should be sufficient for
>            the disk to expire its write cache. */
>         if (system_state != SYSTEM_POWER_OFF) {
> #else
>         if (system_state == SYSTEM_RESTART) {
> #endif
>                 ide_cacheflush_p(drive);
>                 return;
>         }
> 
>   It seems that the if clause are there due to some historic thing.  The
> comment suggests that the body in the if clause spun down the drive
> previously, and we needed the if to avoid spinning down unnecessarily,
> and the if is left there after we changed the body to flush cache.

It is a correct workaround for Alpha firmware, please see the original patch:
http://linux.bkbits.net:8080/linux-2.6/cset@40bec934PKpDSgX_q3NCTdKXxyk6bw?nav=index.html|src/|src/drivers|src/drivers/ide|related/drivers/ide/ide-disk.c

The thing is that we want to avoid calling dev->bus->suspend() on
system_state == SYSTEM_HALT.

Bartlomiej

Re: when are buffers/caches flushed?

[Tejun Heo]

On Thu, Aug 18, 2005 at 02:24:25PM +0200, Bartlomiej Zolnierkiewicz wrote:
> Hi,
> 
> On 8/18/05, Tejun Heo <htejun@gmail.com> wrote:
> > Steven Scholz wrote:
> > > Tejun Heo wrote:
> > >
> > >> Steven Scholz wrote:
> > >>
> > >>> Hi there,
> > >>>
> > >>> According to an (old) man page of sync(2)
> > >>>
> > >>>        According to  the  standard  specification  (e.g.,  SVID),
> > >>>        sync()  schedules  the  writes,  but may return before the
> > >>>        actual writing is done.   However,  since  version  1.3.20
> > >>>        Linux  does actually wait.  (This still does not guarantee
> > >>>        data integrity: modern disks have large caches.)
> > >>>
> > >>> How about recent kernels? Does sync() block until buffers are flushed?
> > >>> How can I find out if the disk caches are actually flushed?
> > >>>
> > >>> I want to make sure that all data is flushed to my disk drive before
> > >>> powering down the system.
> > >>>
> > >>> Thanks.
> > >>>
> > >>
> > >>  All disk caches are flushed before shutdown via the following path.
> > >>
> > >> kernel/sys.c::sys_reboot()
> > >> kernel/drivers/base/power/shutdown.c::device_shutdown()
> > >>  driver specific ->shutdown callback, for ide disks, the path is
> > >> drivers/ide/ide-disk.c::ide_device_shutdown()
> > >> drivers/ide/ide-disk.c::ide_cacheflush_p()
> > >> drivers/ide/ide-disk.c::do_idedisk_flushcache()
> > >>
> > >>  And, AFAIK, sync() doesn't flush disk caches.
> > >
> > >
> > > How about umount?
> > >
> > 
> > [CC'ing Bartlomiej (Hi!)]
> > 
> >   Hmmm, umount doesn't.  I think maybe adding cache flushing to sync and
> > umount can be helpful.
> 
> I think that cache is flushed on umount through idedisk_release().
> 

 Ah... right.  I was testing while another partition is still mounted.

> >   And one more thing, ide-disk doesn't flush cache when shutting down.
> > It flushes only when rebooting.
> 
> Please see ide_disk_remove().
>  

 Hmmm, disks are not removed on shutdown, so it's not called on
shutdown.

> > from ide_device_shutdown()...
> > 
> > #ifdef  CONFIG_ALPHA
> >         /* On Alpha, halt(8) doesn't actually turn the machine off,
> >            it puts you into the sort of firmware monitor. Typically,
> >            it's used to boot another kernel image, so it's not much
> >            different from reboot(8). Therefore, we don't need to
> >            spin down the disk in this case, especially since Alpha
> >            firmware doesn't handle disks in standby mode properly.
> >            On the other hand, it's reasonably safe to turn the power
> >            off when the shutdown process reaches the firmware prompt,
> >            as the firmware initialization takes rather long time -
> >            at least 10 seconds, which should be sufficient for
> >            the disk to expire its write cache. */
> >         if (system_state != SYSTEM_POWER_OFF) {
> > #else
> >         if (system_state == SYSTEM_RESTART) {
> > #endif
> >                 ide_cacheflush_p(drive);
> >                 return;
> >         }
> > 
> >   It seems that the if clause are there due to some historic thing.  The
> > comment suggests that the body in the if clause spun down the drive
> > previously, and we needed the if to avoid spinning down unnecessarily,
> > and the if is left there after we changed the body to flush cache.
> 
> It is a correct workaround for Alpha firmware, please see the original patch:
> http://linux.bkbits.net:8080/linux-2.6/cset@40bec934PKpDSgX_q3NCTdKXxyk6bw?nav=index.html|src/|src/drivers|src/drivers/ide|related/drivers/ide/ide-disk.c
> 
> The thing is that we want to avoid calling dev->bus->suspend() on
> system_state == SYSTEM_HALT.

 Oh.. right.  We're calling either cacheflush_p or suspend and as
suspending implies flushing, we should be okay.  Sorry about
unnecessary noise.

 Steven, ide-disk's are correctly flushed on shutdown and umount.
Again, sorry about the confusion.

 Thanks.

-- 
tejun

Re: when are buffers/caches flushed?

[Bartlomiej Zolnierkiewicz]

On 8/18/05, Tejun Heo <htejun@gmail.com> wrote:
> 
>  Hmmm, disks are not removed on shutdown, so it's not called on
> shutdown.

Yep, I got confused that you meant driver removal.

> > > from ide_device_shutdown()...

above is called on device shutdown

Thanks,
Bartlomiej