System crash while scrubbing on 3.18.x

System crash while scrubbing on 3.18.x

[Cameron Berkenpas]

Hello,

Sorry for the long email...

I've found my system locks up when scrubbing with 3.18.x, but not with 
3.17.8 across 2 systems.

I have the following BTRFS partitions on system 1:
/  (128GiB, 49GiB used on SSD)
/home (4.2TiB, 624GB used on HDD RAID volume)

I have the following BTRFS partitions on system 2:
/ (196GiB, 17GiB used on HDD RAID volume)
/home (7.1TiB, 2.9TiB used on HDD RAID volume)

My OS is Netrunner 15 (which 98% Kubuntu) on system 1, and up-to-date 
debian testing on system 2.

I've never encountered a lock up while scrubbing /. Just with /home.

The systems never lock up immediately, but takes some time. VERY rarely 
I'll see the lockup when the scrub is at <100GiB completed. Typically it 
happens somewhere between 200-350GiB. A few times it's gone beyond 
500GiB. This is probably why I've never encountered the issue with /, 
it's just not big enough on either system.

Both systems were otherwise idle while performing the scrubs that 
crashed the systems.

/home is on a partition on a RAID10 volume on a 3ware 9740-4i controller 
with 4x 3TB disks on system 1. On system 2, it's the same controller but 
with 4x 4TB disks (and / on system 2 is a partition on the same RAID 
volume rather than a separate disk). Both systems have 32GiB memory, and 
the otherwise the hardware is pretty different between the systems (AMD 
Vs. Intel, etc).

I suspect that the RAID controller probably isn't relevant. Both arrays 
and their drives are healthy.

I've also encountered the issue on a freshly formatted filesystem with 
my data copied from a backup on system 1.

I've tried tried scrubbing with btrfs-progs 3.17 (installed from the 
distribution repos on both systems), and btrfs-progs from git (using tag 
v3.18.x). Neither version made a difference.

In case this is helpful to anyone, here's how I've discovered the issue:
I decided to test btrfs with bcache on system 1 to see if the stability 
had improved since I'd tried bcache+btrfs about a year ago. I backed up 
/home on system 1 and then freshly formatted it and set it to use 
bcache. I was running Linux 3.18.8 and encountered the problem that I've 
described above. I assumed the bcache+btrfs combination was still broken 
so I formatted the system again (this time still using btrfs, but 
without bcache) and copied all my files back. I encountered the same 
issue without bcache. Realizing the issue wasn't bcache related, I did 
ANOTHER format, this time back to bcache+btrfs.

 From here in my testing, I found that system 2 (which has no bcache) 
also crashed when scrubbing with Linux 3.18.8. I decided to try 3.17.8 
on system 1 (since 3.18.8 seemed to be the common denominator between 
the 2 systems), found that fixed the issue, and then downgraded system 2 
to use 3.17.8 as well, which also fixed the issue there.

(Note: At one point I also tried Linux 3.18.7 and 3.18.5, however, those 
kernels are affected by the scrub/crash issue as well.)

I found something else interesting when I tested against Linux 3.19.0. 
With 3.19.0, the bcache system always crashes fairly early in the scrub 
(<100GiB), but the non-bcache system has no issues. This suggests my 
problem with 3.19.0 is a bcache+btrfs issue (or simply an issue with 
bcache).

I'm not sure if bcache is relevant to the BTRFS devs at this point, but 
I thought I'd put that there for anyone who might find that information 
useful.

To summarize:
I've tested with 2 systems, and scrubbing caused crashes occurred on 
both with Linux 3.18.8, but not with 3.17.8 for both systems
I've tested 1 system with and without bcache, and bcache made no 
difference between Linux 3.17.8 and 3.18.8.
I've tested with 3.19.0, and I crash when scrubbing on the bcache 
system, but not the non-bcache system.

Thanks!

-Cameron

Re: System crash while scrubbing on 3.18.x

[Liu Bo]

On Thu, Mar 05, 2015 at 05:18:21PM -0800, Cameron Berkenpas wrote:
> Hello,
> 
> Sorry for the long email...
> 
> I've found my system locks up when scrubbing with 3.18.x, but not
> with 3.17.8 across 2 systems.
> 
> I have the following BTRFS partitions on system 1:
> /  (128GiB, 49GiB used on SSD)
> /home (4.2TiB, 624GB used on HDD RAID volume)
> 
> I have the following BTRFS partitions on system 2:
> / (196GiB, 17GiB used on HDD RAID volume)
> /home (7.1TiB, 2.9TiB used on HDD RAID volume)
> 
> My OS is Netrunner 15 (which 98% Kubuntu) on system 1, and
> up-to-date debian testing on system 2.
> 
> I've never encountered a lock up while scrubbing /. Just with /home.
> 
> The systems never lock up immediately, but takes some time. VERY
> rarely I'll see the lockup when the scrub is at <100GiB completed.
> Typically it happens somewhere between 200-350GiB. A few times it's
> gone beyond 500GiB. This is probably why I've never encountered the
> issue with /, it's just not big enough on either system.
> 
> Both systems were otherwise idle while performing the scrubs that
> crashed the systems.
> 
> /home is on a partition on a RAID10 volume on a 3ware 9740-4i
> controller with 4x 3TB disks on system 1. On system 2, it's the same
> controller but with 4x 4TB disks (and / on system 2 is a partition
> on the same RAID volume rather than a separate disk). Both systems
> have 32GiB memory, and the otherwise the hardware is pretty
> different between the systems (AMD Vs. Intel, etc).
> 
> I suspect that the RAID controller probably isn't relevant. Both
> arrays and their drives are healthy.
> 
> I've also encountered the issue on a freshly formatted filesystem
> with my data copied from a backup on system 1.
> 
> I've tried tried scrubbing with btrfs-progs 3.17 (installed from the
> distribution repos on both systems), and btrfs-progs from git (using
> tag v3.18.x). Neither version made a difference.
> 
> In case this is helpful to anyone, here's how I've discovered the issue:
> I decided to test btrfs with bcache on system 1 to see if the
> stability had improved since I'd tried bcache+btrfs about a year
> ago. I backed up /home on system 1 and then freshly formatted it and
> set it to use bcache. I was running Linux 3.18.8 and encountered the
> problem that I've described above. I assumed the bcache+btrfs
> combination was still broken so I formatted the system again (this
> time still using btrfs, but without bcache) and copied all my files
> back. I encountered the same issue without bcache. Realizing the
> issue wasn't bcache related, I did ANOTHER format, this time back to
> bcache+btrfs.
> 
> From here in my testing, I found that system 2 (which has no bcache)
> also crashed when scrubbing with Linux 3.18.8. I decided to try
> 3.17.8 on system 1 (since 3.18.8 seemed to be the common denominator
> between the 2 systems), found that fixed the issue, and then
> downgraded system 2 to use 3.17.8 as well, which also fixed the
> issue there.
> 
> (Note: At one point I also tried Linux 3.18.7 and 3.18.5, however,
> those kernels are affected by the scrub/crash issue as well.)
> 
> I found something else interesting when I tested against Linux
> 3.19.0. With 3.19.0, the bcache system always crashes fairly early
> in the scrub (<100GiB), but the non-bcache system has no issues.
> This suggests my problem with 3.19.0 is a bcache+btrfs issue (or
> simply an issue with bcache).
> 
> I'm not sure if bcache is relevant to the BTRFS devs at this point,
> but I thought I'd put that there for anyone who might find that
> information useful.
> 
> To summarize:
> I've tested with 2 systems, and scrubbing caused crashes occurred on
> both with Linux 3.18.8, but not with 3.17.8 for both systems
> I've tested 1 system with and without bcache, and bcache made no
> difference between Linux 3.17.8 and 3.18.8.
> I've tested with 3.19.0, and I crash when scrubbing on the bcache
> system, but not the non-bcache system.

Better to have some stacks about the scrub crash.

Thanks,

-liubo

Re: System crash while scrubbing on 3.18.x

[Cameron Berkenpas]

Hello,

I should have mentioned this earlier. I get absolutely nothing in the 
logs. I can't look at dmesg either as the system is completely unresponsive.

On 03/06/2015 04:30 AM, Liu Bo wrote:
> On Thu, Mar 05, 2015 at 05:18:21PM -0800, Cameron Berkenpas wrote:
>> Hello,
>>
>> Sorry for the long email...
>>
>> I've found my system locks up when scrubbing with 3.18.x, but not
>> with 3.17.8 across 2 systems.
>>
>> I have the following BTRFS partitions on system 1:
>> /  (128GiB, 49GiB used on SSD)
>> /home (4.2TiB, 624GB used on HDD RAID volume)
>>
>> I have the following BTRFS partitions on system 2:
>> / (196GiB, 17GiB used on HDD RAID volume)
>> /home (7.1TiB, 2.9TiB used on HDD RAID volume)
>>
>> My OS is Netrunner 15 (which 98% Kubuntu) on system 1, and
>> up-to-date debian testing on system 2.
>>
>> I've never encountered a lock up while scrubbing /. Just with /home.
>>
>> The systems never lock up immediately, but takes some time. VERY
>> rarely I'll see the lockup when the scrub is at <100GiB completed.
>> Typically it happens somewhere between 200-350GiB. A few times it's
>> gone beyond 500GiB. This is probably why I've never encountered the
>> issue with /, it's just not big enough on either system.
>>
>> Both systems were otherwise idle while performing the scrubs that
>> crashed the systems.
>>
>> /home is on a partition on a RAID10 volume on a 3ware 9740-4i
>> controller with 4x 3TB disks on system 1. On system 2, it's the same
>> controller but with 4x 4TB disks (and / on system 2 is a partition
>> on the same RAID volume rather than a separate disk). Both systems
>> have 32GiB memory, and the otherwise the hardware is pretty
>> different between the systems (AMD Vs. Intel, etc).
>>
>> I suspect that the RAID controller probably isn't relevant. Both
>> arrays and their drives are healthy.
>>
>> I've also encountered the issue on a freshly formatted filesystem
>> with my data copied from a backup on system 1.
>>
>> I've tried tried scrubbing with btrfs-progs 3.17 (installed from the
>> distribution repos on both systems), and btrfs-progs from git (using
>> tag v3.18.x). Neither version made a difference.
>>
>> In case this is helpful to anyone, here's how I've discovered the issue:
>> I decided to test btrfs with bcache on system 1 to see if the
>> stability had improved since I'd tried bcache+btrfs about a year
>> ago. I backed up /home on system 1 and then freshly formatted it and
>> set it to use bcache. I was running Linux 3.18.8 and encountered the
>> problem that I've described above. I assumed the bcache+btrfs
>> combination was still broken so I formatted the system again (this
>> time still using btrfs, but without bcache) and copied all my files
>> back. I encountered the same issue without bcache. Realizing the
>> issue wasn't bcache related, I did ANOTHER format, this time back to
>> bcache+btrfs.
>>
>>  From here in my testing, I found that system 2 (which has no bcache)
>> also crashed when scrubbing with Linux 3.18.8. I decided to try
>> 3.17.8 on system 1 (since 3.18.8 seemed to be the common denominator
>> between the 2 systems), found that fixed the issue, and then
>> downgraded system 2 to use 3.17.8 as well, which also fixed the
>> issue there.
>>
>> (Note: At one point I also tried Linux 3.18.7 and 3.18.5, however,
>> those kernels are affected by the scrub/crash issue as well.)
>>
>> I found something else interesting when I tested against Linux
>> 3.19.0. With 3.19.0, the bcache system always crashes fairly early
>> in the scrub (<100GiB), but the non-bcache system has no issues.
>> This suggests my problem with 3.19.0 is a bcache+btrfs issue (or
>> simply an issue with bcache).
>>
>> I'm not sure if bcache is relevant to the BTRFS devs at this point,
>> but I thought I'd put that there for anyone who might find that
>> information useful.
>>
>> To summarize:
>> I've tested with 2 systems, and scrubbing caused crashes occurred on
>> both with Linux 3.18.8, but not with 3.17.8 for both systems
>> I've tested 1 system with and without bcache, and bcache made no
>> difference between Linux 3.17.8 and 3.18.8.
>> I've tested with 3.19.0, and I crash when scrubbing on the bcache
>> system, but not the non-bcache system.
> Better to have some stacks about the scrub crash.
>
> Thanks,
>
> -liubo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: System crash while scrubbing on 3.18.x

[Cameron Berkenpas]

Minor update:

I've now tested 4.0-rc4 on systems. Both systems lockup during a scrub.

I've also tested a laptop with 4.0-rc4 and I have no issues. It could be 
that I "only" have 221GB on it, that it's an SSD, and/or the lack of a 
3ware controller.

Within the next few days, I hope to have the time to test on a btrfs 
RAID1 system I have (btrfs RAID) as well.

-Cameron

On 03/06/2015 12:03 PM, Cameron Berkenpas wrote:
> Hello,
>
> I should have mentioned this earlier. I get absolutely nothing in the 
> logs. I can't look at dmesg either as the system is completely 
> unresponsive.
>
> On 03/06/2015 04:30 AM, Liu Bo wrote:
>> On Thu, Mar 05, 2015 at 05:18:21PM -0800, Cameron Berkenpas wrote:
>>> Hello,
>>>
>>> Sorry for the long email...
>>>
>>> I've found my system locks up when scrubbing with 3.18.x, but not
>>> with 3.17.8 across 2 systems.
>>>
>>> I have the following BTRFS partitions on system 1:
>>> /  (128GiB, 49GiB used on SSD)
>>> /home (4.2TiB, 624GB used on HDD RAID volume)
>>>
>>> I have the following BTRFS partitions on system 2:
>>> / (196GiB, 17GiB used on HDD RAID volume)
>>> /home (7.1TiB, 2.9TiB used on HDD RAID volume)
>>>
>>> My OS is Netrunner 15 (which 98% Kubuntu) on system 1, and
>>> up-to-date debian testing on system 2.
>>>
>>> I've never encountered a lock up while scrubbing /. Just with /home.
>>>
>>> The systems never lock up immediately, but takes some time. VERY
>>> rarely I'll see the lockup when the scrub is at <100GiB completed.
>>> Typically it happens somewhere between 200-350GiB. A few times it's
>>> gone beyond 500GiB. This is probably why I've never encountered the
>>> issue with /, it's just not big enough on either system.
>>>
>>> Both systems were otherwise idle while performing the scrubs that
>>> crashed the systems.
>>>
>>> /home is on a partition on a RAID10 volume on a 3ware 9740-4i
>>> controller with 4x 3TB disks on system 1. On system 2, it's the same
>>> controller but with 4x 4TB disks (and / on system 2 is a partition
>>> on the same RAID volume rather than a separate disk). Both systems
>>> have 32GiB memory, and the otherwise the hardware is pretty
>>> different between the systems (AMD Vs. Intel, etc).
>>>
>>> I suspect that the RAID controller probably isn't relevant. Both
>>> arrays and their drives are healthy.
>>>
>>> I've also encountered the issue on a freshly formatted filesystem
>>> with my data copied from a backup on system 1.
>>>
>>> I've tried tried scrubbing with btrfs-progs 3.17 (installed from the
>>> distribution repos on both systems), and btrfs-progs from git (using
>>> tag v3.18.x). Neither version made a difference.
>>>
>>> In case this is helpful to anyone, here's how I've discovered the 
>>> issue:
>>> I decided to test btrfs with bcache on system 1 to see if the
>>> stability had improved since I'd tried bcache+btrfs about a year
>>> ago. I backed up /home on system 1 and then freshly formatted it and
>>> set it to use bcache. I was running Linux 3.18.8 and encountered the
>>> problem that I've described above. I assumed the bcache+btrfs
>>> combination was still broken so I formatted the system again (this
>>> time still using btrfs, but without bcache) and copied all my files
>>> back. I encountered the same issue without bcache. Realizing the
>>> issue wasn't bcache related, I did ANOTHER format, this time back to
>>> bcache+btrfs.
>>>
>>>  From here in my testing, I found that system 2 (which has no bcache)
>>> also crashed when scrubbing with Linux 3.18.8. I decided to try
>>> 3.17.8 on system 1 (since 3.18.8 seemed to be the common denominator
>>> between the 2 systems), found that fixed the issue, and then
>>> downgraded system 2 to use 3.17.8 as well, which also fixed the
>>> issue there.
>>>
>>> (Note: At one point I also tried Linux 3.18.7 and 3.18.5, however,
>>> those kernels are affected by the scrub/crash issue as well.)
>>>
>>> I found something else interesting when I tested against Linux
>>> 3.19.0. With 3.19.0, the bcache system always crashes fairly early
>>> in the scrub (<100GiB), but the non-bcache system has no issues.
>>> This suggests my problem with 3.19.0 is a bcache+btrfs issue (or
>>> simply an issue with bcache).
>>>
>>> I'm not sure if bcache is relevant to the BTRFS devs at this point,
>>> but I thought I'd put that there for anyone who might find that
>>> information useful.
>>>
>>> To summarize:
>>> I've tested with 2 systems, and scrubbing caused crashes occurred on
>>> both with Linux 3.18.8, but not with 3.17.8 for both systems
>>> I've tested 1 system with and without bcache, and bcache made no
>>> difference between Linux 3.17.8 and 3.18.8.
>>> I've tested with 3.19.0, and I crash when scrubbing on the bcache
>>> system, but not the non-bcache system.
>> Better to have some stacks about the scrub crash.
>>
>> Thanks,
>>
>> -liubo
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe 
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: System crash while scrubbing on 3.18.x

[Cameron Berkenpas]

Hello,

I've figured out at least part of the issue.

I had blk-mq enabled by default (SCSI_MQ_DEFAULT=y). I've since set 
SCSI_MQ_DEFAULT=n, and now I'm completely stable with 4.0-rc4, 3.18.9, 
and presumably all the others. This is on both the bcache and non-bcache 
systems.

All of my systems with btrfs and SCSI_MQ_DEFAULT=y are stable unless 
they're using a 3ware 9750 controller (which I believe is the 3w-sas 
driver from the top of my head). So either there's a bug with 
blk-mq+3w-sas+btrfs, or just blk-mq+3w-sas. I suspect the latter, but if 
I can find the time, I'll try to test with another filesystem.

I suspect other controllers/drivers could be affected as well.

Are any btrfs devs interested in me looking into this further?

As a side note, in spite of all the crashes I've had through out my 
testing etc, bcache+btrfs hasn't given me any issues like I've seen in 
the past. This is just a few weeks worth of testing, but it seems 
there's at least some sort of basic stability with bcache+btrfs now.

-Cameron

On 03/16/2015 10:19 AM, Cameron Berkenpas wrote:
> Minor update:
>
> I've now tested 4.0-rc4 on systems. Both systems lockup during a scrub.
>
> I've also tested a laptop with 4.0-rc4 and I have no issues. It could 
> be that I "only" have 221GB on it, that it's an SSD, and/or the lack 
> of a 3ware controller.
>
> Within the next few days, I hope to have the time to test on a btrfs 
> RAID1 system I have (btrfs RAID) as well.
>
> -Cameron
>
> On 03/06/2015 12:03 PM, Cameron Berkenpas wrote:
>> Hello,
>>
>> I should have mentioned this earlier. I get absolutely nothing in the 
>> logs. I can't look at dmesg either as the system is completely 
>> unresponsive.
>>
>> On 03/06/2015 04:30 AM, Liu Bo wrote:
>>> On Thu, Mar 05, 2015 at 05:18:21PM -0800, Cameron Berkenpas wrote:
>>>> Hello,
>>>>
>>>> Sorry for the long email...
>>>>
>>>> I've found my system locks up when scrubbing with 3.18.x, but not
>>>> with 3.17.8 across 2 systems.
>>>>
>>>> I have the following BTRFS partitions on system 1:
>>>> /  (128GiB, 49GiB used on SSD)
>>>> /home (4.2TiB, 624GB used on HDD RAID volume)
>>>>
>>>> I have the following BTRFS partitions on system 2:
>>>> / (196GiB, 17GiB used on HDD RAID volume)
>>>> /home (7.1TiB, 2.9TiB used on HDD RAID volume)
>>>>
>>>> My OS is Netrunner 15 (which 98% Kubuntu) on system 1, and
>>>> up-to-date debian testing on system 2.
>>>>
>>>> I've never encountered a lock up while scrubbing /. Just with /home.
>>>>
>>>> The systems never lock up immediately, but takes some time. VERY
>>>> rarely I'll see the lockup when the scrub is at <100GiB completed.
>>>> Typically it happens somewhere between 200-350GiB. A few times it's
>>>> gone beyond 500GiB. This is probably why I've never encountered the
>>>> issue with /, it's just not big enough on either system.
>>>>
>>>> Both systems were otherwise idle while performing the scrubs that
>>>> crashed the systems.
>>>>
>>>> /home is on a partition on a RAID10 volume on a 3ware 9740-4i
>>>> controller with 4x 3TB disks on system 1. On system 2, it's the same
>>>> controller but with 4x 4TB disks (and / on system 2 is a partition
>>>> on the same RAID volume rather than a separate disk). Both systems
>>>> have 32GiB memory, and the otherwise the hardware is pretty
>>>> different between the systems (AMD Vs. Intel, etc).
>>>>
>>>> I suspect that the RAID controller probably isn't relevant. Both
>>>> arrays and their drives are healthy.
>>>>
>>>> I've also encountered the issue on a freshly formatted filesystem
>>>> with my data copied from a backup on system 1.
>>>>
>>>> I've tried tried scrubbing with btrfs-progs 3.17 (installed from the
>>>> distribution repos on both systems), and btrfs-progs from git (using
>>>> tag v3.18.x). Neither version made a difference.
>>>>
>>>> In case this is helpful to anyone, here's how I've discovered the 
>>>> issue:
>>>> I decided to test btrfs with bcache on system 1 to see if the
>>>> stability had improved since I'd tried bcache+btrfs about a year
>>>> ago. I backed up /home on system 1 and then freshly formatted it and
>>>> set it to use bcache. I was running Linux 3.18.8 and encountered the
>>>> problem that I've described above. I assumed the bcache+btrfs
>>>> combination was still broken so I formatted the system again (this
>>>> time still using btrfs, but without bcache) and copied all my files
>>>> back. I encountered the same issue without bcache. Realizing the
>>>> issue wasn't bcache related, I did ANOTHER format, this time back to
>>>> bcache+btrfs.
>>>>
>>>>  From here in my testing, I found that system 2 (which has no bcache)
>>>> also crashed when scrubbing with Linux 3.18.8. I decided to try
>>>> 3.17.8 on system 1 (since 3.18.8 seemed to be the common denominator
>>>> between the 2 systems), found that fixed the issue, and then
>>>> downgraded system 2 to use 3.17.8 as well, which also fixed the
>>>> issue there.
>>>>
>>>> (Note: At one point I also tried Linux 3.18.7 and 3.18.5, however,
>>>> those kernels are affected by the scrub/crash issue as well.)
>>>>
>>>> I found something else interesting when I tested against Linux
>>>> 3.19.0. With 3.19.0, the bcache system always crashes fairly early
>>>> in the scrub (<100GiB), but the non-bcache system has no issues.
>>>> This suggests my problem with 3.19.0 is a bcache+btrfs issue (or
>>>> simply an issue with bcache).
>>>>
>>>> I'm not sure if bcache is relevant to the BTRFS devs at this point,
>>>> but I thought I'd put that there for anyone who might find that
>>>> information useful.
>>>>
>>>> To summarize:
>>>> I've tested with 2 systems, and scrubbing caused crashes occurred on
>>>> both with Linux 3.18.8, but not with 3.17.8 for both systems
>>>> I've tested 1 system with and without bcache, and bcache made no
>>>> difference between Linux 3.17.8 and 3.18.8.
>>>> I've tested with 3.19.0, and I crash when scrubbing on the bcache
>>>> system, but not the non-bcache system.
>>> Better to have some stacks about the scrub crash.
>>>
>>> Thanks,
>>>
>>> -liubo
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe 
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: System crash while scrubbing on 3.18.x

[Duncan]

Cameron Berkenpas posted on Tue, 17 Mar 2015 10:26:10 -0700 as excerpted:

> I've figured out at least part of the issue.
> 
> I had blk-mq enabled by default (SCSI_MQ_DEFAULT=y). I've since set
> SCSI_MQ_DEFAULT=n, and now I'm completely stable with 4.0-rc4, 3.18.9,
> and presumably all the others. This is on both the bcache and non-bcache
> systems.
> 
> All of my systems with btrfs and SCSI_MQ_DEFAULT=y are stable unless
> they're using a 3ware 9750 controller (which I believe is the 3w-sas
> driver from the top of my head). So either there's a bug with
> blk-mq+3w-sas+btrfs, or just blk-mq+3w-sas. I suspect the latter, but if
> I can find the time, I'll try to test with another filesystem.
> 
> I suspect other controllers/drivers could be affected as well.
> 
> Are any btrfs devs interested in me looking into this further?
> 
> As a side note, in spite of all the crashes I've had through out my
> testing etc, bcache+btrfs hasn't given me any issues like I've seen in
> the past. This is just a few weeks worth of testing, but it seems
> there's at least some sort of basic stability with bcache+btrfs now.

Wow.  Thanks!

I'm not a dev so won't attempt to answer whether they have further 
interest or not, but thanks for keeping with this until you found the 
scsi-mq and 32-sas connection and reported it.  It'll very possibly be of 
use to someone else later, and meanwhile, that's one less instability-
mystery hanging over btrfs alone, now, which surely lets all regulars 
here rest easier. =:^)

And that despite all the crashes during testing, bcache+btrfs didn't 
demonstrate any filesystem instabilities, demonstrates how far both of 
those components has come, each on its own and working together, as 
well.  I know we get bcache+btrfs questions from time to time and the 
stability here even under repeated crashes triggered by unrelated factors 
is something I'm sure many will find reassuring. =:^)

So thanks, again! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman