Re: [PATCH] Btrfs: throttle delayed refs better

[Josef Bacik <jbacik@fb.com>]

On 02/05/2014 05:57 PM, Johannes Hirte wrote:
> On Wed, 5 Feb 2014 16:46:57 -0500
> Josef Bacik <jbacik@fb.com> wrote:
>
>> On 02/05/2014 04:42 PM, Johannes Hirte wrote:
>>> On Wed, 5 Feb 2014 14:36:39 -0500
>>> Josef Bacik <jbacik@fb.com> wrote:
>>>
>>>> On 02/05/2014 02:30 PM, Johannes Hirte wrote:
>>>>> On Wed, 5 Feb 2014 14:00:57 -0500
>>>>> Josef Bacik <jbacik@fb.com> wrote:
>>>>>
>>>>>> On 02/05/2014 12:34 PM, Johannes Hirte wrote:
>>>>>>> On Wed, 5 Feb 2014 10:49:15 -0500
>>>>>>> Josef Bacik <jbacik@fb.com> wrote:
>>>>>>>
>>>>>>>> Ok none of those make sense which makes me think it may be the
>>>>>>>> ktime bits, instead of un-applying the whole patch could you
>>>>>>>> just comment out the parts
>>>>>>>>
>>>>>>>>              ktime_t start = ktime_get();
>>>>>>>>
>>>>>>>> and
>>>>>>>>
>>>>>>>>              if (actual_count > 0) {
>>>>>>>>                      u64 runtime =
>>>>>>>> ktime_to_ns(ktime_sub(ktime_get(), start)); u64 avg;
>>>>>>>>
>>>>>>>>                      /*
>>>>>>>>                       * We weigh the current average higher than
>>>>>>>> our current runtime
>>>>>>>>                       * to avoid large swings in the average.
>>>>>>>>                       */
>>>>>>>>                      spin_lock(&delayed_refs->lock);
>>>>>>>>                      avg = fs_info->avg_delayed_ref_runtime * 3
>>>>>>>> + runtime; avg = div64_u64(avg, 4);
>>>>>>>>                      fs_info->avg_delayed_ref_runtime = avg;
>>>>>>>>                      spin_unlock(&delayed_refs->lock);
>>>>>>>>              }
>>>>>>>>
>>>>>>>> in __btrfs_run_delayed_refs and see if that makes the problem
>>>>>>>> stop? If it does will you try chris's for-linus branch to see
>>>>>>>> if it still reproduces there?  Maybe some patch changed
>>>>>>>> ktime_get() in -rc1 that is causing issues and we're just now
>>>>>>>> exposing it. Thanks,
>>>>>>> With the ktime bits disabled, I wasn't able to reproduce the
>>>>>>> problem anymore. With Chris' for-linus branch it took longer but
>>>>>>> still appeared.
>>>>>>>
>>>>>> Ok can you send your .config, maybe there's some weird time bug
>>>>>> being exposed.  What kind of CPU do you have?  Thanks,
>>>>>>
>>>>>> Josef
>>>>> It's a Core i5-540M, dualcore + hyperthreading
>>>> Ok while I'm doing this can you change
>>>> btrfs_should_throttle_delayed_refs to _always_ return 1, still with
>>>> all the ktime stuff commented out, and see if that causes the
>>>> problem to happen?  Thanks,
>>> Yes it does. Same behavior as without ktime stuff commented out.
>>>
>> Ok perfect, can you send me a btrfs fi df of that volume, and do you
>> have any snapshots or anything?  Thanks,
> btrfs fi df /
> Data, single: total=220.01GiB, used=210.85GiB
> System, DUP: total=8.00MiB, used=32.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=4.00GiB, used=2.93GiB
> Metadata, single: total=8.00MiB, used=0.00
>
> No snapshots but several subvolumes. / itself is a seperate subvolume
> and subvol 0 only contains the other subvolumes (5 at moment). qgroups
> aren't enabled.
>
> mount options are noatime,inode_cache, if this matters
>
I've managed to reproduce on one of my test boxes at work, I'll get to 
the bottom of this.  Thanks,

Josef