dm-cache bug when using the cleaner?

dm-cache bug when using the cleaner?

[Darrick J. Wong]

Hi there,

I've been playing around with writeback mode in dmcache with my shiny new
3.9-rc1 kernel.

# dmsetup table moocache
0 67108864 cache 253:0 253:1 8:0 512 1 writeback default 4 random_threshold 8 sequential_threshold 512
# dmsetup status moocache
0 67108864 cache 14/2048 3967 54586 263549 190982 0 0 2000 51 0 2 migration_threshold 204800 4 random_threshold 8 sequential_threshold 512

I think this means there are 51 dirty cache blocks?  I'd like to flush them out
to the spinny disk:

# dmsetup suspend moocache
# echo '0 67108864 cache 253:0 253:1 8:0 512 0 cleaner 0' | dmsetup reload moocache
# dmsetup resume moocache

Ok, now let's wait for it to finish...

# dmsetup wait moocache
# dmsetup status moocache
0 67108864 cache 14/2048 3967 54586 263583 190982 0 0 2000 0 0 2 migration_threshold 204800 0

Now that dirty field goes from 51 back to 0, like I'd expected.  However, if I
then put the old policy back:

# dmsetup suspend moocache
# echo '0 67108864 cache 253:0 253:1 8:0 512 1 writeback default 4 random_threshold 8 sequential_threshold 512' | dmsetup reload moocache
# dmsetup resume moocache

# dmsetup status moocache
0 67108864 cache 14/2048 3967 54586 263553 190982 0 0 2000 51 0 2 migration_threshold 204800 4 random_threshold 8 sequential_threshold 512

That's odd, there are still 51 dirty cache blocks?  Am I missing something?  Is
there something about suspend/resume that causes something to get reset?  Do I
even need the suspend/resume pair?  (I'm pretty sure I do; the table doesn't
change if I don't.)  I get the same dirtyblocks==51 results even if I remove
and (re)create moocache.

--D

Re: dm-cache bug when using the cleaner?

[thornber]

Hi Darrick,

You're using it correctly and have found a bug, thanks!  This patch fixes it:

  https://github.com/jthornber/linux-2.6/commit/6bbc70ab77ab828f1fecd8a3f4776b344eedef24

This is the test scenario I use (tweaked to catch your issue).

  def wait_for_all_clean(cache)
    cache.event_tracker.wait(cache) do |cache|
      status = CacheStatus.new(cache)
      STDERR.puts "#{status.nr_dirty} dirty blocks"
      status.nr_dirty == 0
    end
  end

  def test_cleaner_policy
    with_standard_cache(:format => true) do |cache|
      git_prepare(cache, :ext4)

      cache.pause do
        table = cache.active_table
        table.targets[0].args[5] = 'cleaner'
        cache.load(table)
      end

      wait_for_all_clean(cache)

      cache.pause do
        table = cache.active_table
        table.targets[0].args[5] = 'mq'
        cache.load(table)
      end

      status = CacheStatus.new(cache)
      assert_equal(0, status.nr_dirty)
    end

    # We should be able to use the origin directly now                                                                       
    with_standard_linear do |origin|
      fs = FS::file_system(:ext4, origin)
      fs.with_mount('./kernel_builds', :discard => true) do
        # triggers fsck                                                                                                      
      end
    end
  end

Re: dm-cache bug when using the cleaner?

[Darrick J. Wong]

On Tue, Mar 05, 2013 at 04:52:49PM +0000, thornber@redhat.com wrote:
> Hi Darrick,
> 
> You're using it correctly and have found a bug, thanks!  This patch fixes it:
> 
>   https://github.com/jthornber/linux-2.6/commit/6bbc70ab77ab828f1fecd8a3f4776b344eedef24

Seems to fix my problem, thank you!

--D

> This is the test scenario I use (tweaked to catch your issue).
> 
>   def wait_for_all_clean(cache)
>     cache.event_tracker.wait(cache) do |cache|
>       status = CacheStatus.new(cache)
>       STDERR.puts "#{status.nr_dirty} dirty blocks"
>       status.nr_dirty == 0
>     end
>   end
> 
>   def test_cleaner_policy
>     with_standard_cache(:format => true) do |cache|
>       git_prepare(cache, :ext4)
> 
>       cache.pause do
>         table = cache.active_table
>         table.targets[0].args[5] = 'cleaner'
>         cache.load(table)
>       end
> 
>       wait_for_all_clean(cache)
> 
>       cache.pause do
>         table = cache.active_table
>         table.targets[0].args[5] = 'mq'
>         cache.load(table)
>       end
> 
>       status = CacheStatus.new(cache)
>       assert_equal(0, status.nr_dirty)
>     end
> 
>     # We should be able to use the origin directly now                                                                       
>     with_standard_linear do |origin|
>       fs = FS::file_system(:ext4, origin)
>       fs.with_mount('./kernel_builds', :discard => true) do
>         # triggers fsck                                                                                                      
>       end
>     end
>   end