drawbacks of non-ECC RAM

drawbacks of non-ECC RAM

[valleysmail-lol5]

I'd like to know if there are drawbacks in using btrfs with non-ECC RAM instead of using ext4 with non-ECC RAM. I know that some features of btrfs may rely on ECC RAM but is the chance of data corruption or even a damaged filesystem higher than when i use ext4 instead of btrfs?
I want to know this because i would like to use the snapshot feature of btrfs and ext4 does not support that. I will not use btrfs for fixing silent data corruption nor for using RAID like features or encryption. ZFS however checks files in the background (even if i don't want) and if it thinks there is an error it will fix it and i cannot disable this feature. So errors in RAM may corrupt my files or even more.


If you have some information regarding this topic or some good links i would be very happy! =)

Re: drawbacks of non-ECC RAM

[Austin S Hemmelgarn]

On 01/17/2014 01:33 PM, valleysmail-lol5@yahoo.de wrote:
> 
> 
> I'd like to know if there are drawbacks in using btrfs with non-ECC
> RAM instead of using ext4 with non-ECC RAM. I know that some
> features of btrfs may rely on ECC RAM but is the chance of data
> corruption or even a damaged filesystem higher than when i use ext4
> instead of btrfs? I want to know this because i would like to use
> the snapshot feature of btrfs and ext4 does not support that. I
> will not use btrfs for fixing silent data corruption nor for using
> RAID like features or encryption. ZFS however checks files in the
> background (even if i don't want) and if it thinks there is an
> error it will fix it and i cannot disable this feature. So errors
> in RAM may corrupt my files or even more.
> 
> 
> If you have some information regarding this topic or some good
> links i would be very happy! =)

Theoretically, there should be little difference, and it should in
fact be slightly safer to use BTRFS than EXT4 in general because the
data structures for BTRFS are slightly more resilient, and it is smart
enough to check the on-disk data if stuff in memory looks damaged.

AFAIK, none of the features in BTRFS depend on using ECC RAM, and
BTRFS's error recovery is entirely independent of whether the system
RAM is ECC or not.  The aren't really any BTRFS specific drawbacks to
using non-ECC ram.

If you're really that paranoid about data corruption, you really
should be using both ECC RAM and a RAID setup.  Unless your hardware
is damaged (or you plan on sending your computer to space or running
it near a poorly shielded nuclear reactor), the chance of data in ram
not matching that on-disk is very low.

As an entirely secondary thought, BTRFS is almost always faster on
commodity hardware than ZFS.  ZFS was designed to be used on high-end
server systems with dozens of disks on multiple RAID controllers, so
it's a lot less efficient in most other use cases than BTRFS.

Re: drawbacks of non-ECC RAM

[Duncan]

valleysmail-lol5@yahoo.de posted on Fri, 17 Jan 2014 18:33:35 +0000 as
excerpted:

> I'd like to know if there are drawbacks in using btrfs with non-ECC RAM
> instead of using ext4 with non-ECC RAM. I know that some features of
> btrfs may rely on ECC RAM 

Crossed signals somewhere, as that's entirely incorrect.  Btrfs does have 
date integrity checksumming as one of its features -- one which I'm using 
here BTW with raid1 mode so there's two copies of everything, on 
different devices, in case one goes bad -- but BTRFS IN NO WAY REQUIRES 
ECC RAM.  ECC RAM is a hardware solution to a conceptually similar data 
integrity problem in memory to the problem btrfs addresses as filesystem 
software for non-volatile storage, but the two shouldn't be confused for 
each other or conflated at all -- they're entirely separate in practice, 
and one is hardware while the other is software.

> [...] but is the chance of data corruption or even a
> damaged filesystem higher than when i use ext4 instead of btrfs?

The chances of data corruption or a damaged filesystem are certainly not 
higher on btrfs for that reason, because as I said, ECC memory is 
entirely unrelated to btrfs data checksumming and integrity measures.

However, btrfs /is/ a rather less mature filesystem than ext4 ATM, and is 
in fact just now beginning to stabilize.  As such, there are still bugs 
being actively fixed, and as the warning in mkfs.btrfs says when you 
create a filesystem, it's still critical to keep non-btrfs backups of 
anything you value, AND actually test that you can recover from them, 
because at this point, you just might hit one of those still existing 
corner-case bugs and may very well need to do just that, recover from 
backup!

If that's concerning to you, then it's likely that btrfs isn't a good 
choice as your filesystem at this point.  Perhaps in a year or so, as 
2014 really does look to be the year that btrfs finally begins to really 
stabilize, but not right now.

> I want to know this because i would like to use the snapshot feature of
> btrfs and ext4 does not support that. I will not use btrfs for fixing
> silent data corruption nor for using RAID like features or encryption.
> ZFS however checks files in the background (even if i don't want) and if
> it thinks there is an error it will fix it and i cannot disable this
> feature.

> So errors in RAM may corrupt my files or even more.

While it's possible to disable the checksumming in btrfs, and in fact 
that and COW (copy-on-write) are recommended to be disabled for specific 
use-cases (large "internal-write" files such as VM images, large 
databases, and large pre-allocated files then filled in as the file gets 
written, such as many bittorrent clients create) it's definitely *NOT* 
recommended in general.

And in fact, disabling COW would disable efficient snapshots usage as 
well, so it's probably not what you want.

If you're after snapshotting ability without checksumming, I'd suggest 
looking into something like LVM, which has snapshotting as one of its 
major features, but without checksumming/data-integrity.  LVM is a 
relatively mature technology that a number of distros use by default, and 
you can use the ext4 that you're already familiar with and that is 
reasonably mature as well, on top of LVM.

However, I really think you need to do a bit more research, because it 
appears you are rather confused about filesystem checksumming and data 
integrity at the filesystem level, and are in fact conflating it with the 
entirely separate hardware ECC data integrity at memory level.  The two 
are entirely distinct, and data-integrity at the filesystem level 
certainly isn't the bugbear you make it out to be, but rather, a vital 
part of how some filesystems work, particularly as storage sizes reach 
farther and farther into the multi-terabyte range.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: drawbacks of non-ECC RAM

[Justus Seifert]

[-- Attachment #1.1: Type: text/plain, Size: 1642 bytes --]

On 18.01.2014 00:18, Duncan wrote:
> valleysmail-lol5@yahoo.de posted on Fri, 17 Jan 2014 18:33:35 +0000 as
> excerpted:
> 
>> I'd like to know if there are drawbacks in using btrfs with non-ECC RAM
>> instead of using ext4 with non-ECC RAM. I know that some features of
>> btrfs may rely on ECC RAM 
> 
> Crossed signals somewhere, as that's entirely incorrect.  Btrfs does have 
> date integrity checksumming as one of its features -- one which I'm using 
> here BTW with raid1 mode so there's two copies of everything, on 
> different devices, in case one goes bad -- but BTRFS IN NO WAY REQUIRES 
> ECC RAM.  ECC RAM is a hardware solution to a conceptually similar data 
> integrity problem in memory to the problem btrfs addresses as filesystem 
> software for non-volatile storage, but the two shouldn't be confused for 
> each other or conflated at all -- they're entirely separate in practice, 
> and one is hardware while the other is software.

To be fair: Both problems are essentially hardware problems where data
gets corrupted, but the solutions are different.  ECC is only used in
systems where the additional cost for the hardware, energy (registered
ECC-RAM uses more electricity than register-less non-ECC-RAM), and space
(ECC-RAM is physically larger) is less than the cost of recovery in case
of soft errors caused by non-ECC-RAM.  ECC-RAM does not offer a memory
performance advantage.  See Wikipedia for more info about registered RAM
and ECC-RAM.
Raid1 uses more power and space as well, but also adds read performance.
 So _there are reasons to use Raid1 besides resilience_.

> […]


[-- Attachment #1.2: justus_seifert.vcf --]
[-- Type: text/x-vcard, Size: 206 bytes --]

begin:vcard
fn:Justus Seifert
n:Seifert;Justus
adr:;;;Dresden;Saxony;;Germany
email;internet:justus.seifert@dergleichrichter.de
tel;cell:+4915730640509
x-mozilla-html:FALSE
version:2.1
end:vcard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: drawbacks of non-ECC RAM

[Fajar A. Nugraha]

On Sat, Jan 18, 2014 at 1:33 AM, valleysmail-lol5@yahoo.de
<valleysmail-lol5@yahoo.de> wrote:
>
>
>
> I'd like to know if there are drawbacks in using btrfs with non-ECC RAM instead of using ext4 with non-ECC RAM.

Non-ECC RAM can cause problems no matter what fs you use.

> I know that some features of btrfs may rely on ECC RAM but is the chance of data corruption or even a damaged filesystem higher than when i use ext4 instead of btrfs?

Not really.

In the past the occurence of corrupted btrfs report on this list
(regardless of RAM) is somewhat high, but I don't see much of it in
recent versions.

> I want to know this because i would like to use the snapshot feature of btrfs and ext4 does not support that. I will not use btrfs for fixing silent data corruption nor for using RAID like features or encryption. ZFS however checks files in the background (even if i don't want)

zfs does not "checks files in the background" by default. When
checksum is enabled (the default option), zfs only checks file
integrity when you access it, and when you run the "scrub" command. It
does not run background scrubs automatically.

AFAIK btrfs behaves the same way.

> and if it thinks there is an error it will fix it and i cannot disable this feature. So errors in RAM may corrupt my files or even more.

You can disable checksum on both btrfs and zfs. See
https://btrfs.wiki.kernel.org/index.php/FAQ#Can_data_checksumming_be_turned_off.3F
,  https://btrfs.wiki.kernel.org/index.php/Mount_options

-- 
Fajar