Re: BTRFS on production: NVR 16+ IP Cameras

[Anand Jain <anand.jain@oracle.com>]

On 11/16/2018 03:51 AM, Nikolay Borisov wrote:
> 
> 
> On 15.11.18 г. 20:39 ч., Juan Alberto Cirez wrote:
>> Is BTRFS mature enough to be deployed on a production system to underpin
>> the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)?
>>
>> Based on our limited experience with BTRFS (1+ year) under the above
>> scenario the answer seems to be no; but I wanted you ask the community
>> at large for their experience before making a final decision to hold off
>> on deploying BTRFS on production systems.
>>
>> Let us be clear: We think BTRFS has great potential, and as it matures
>> we will continue to watch its progress, so that at some future point we
>> can return to using it.
>>
>> The issue has been the myriad of problems we have encountered when
>> deploying BTRFS as the storage fs for the NVR/VMS in cases were the
>> camera count exceeds 10: Corrupted file systems, sudden read-only file
>> system, re-balance kernel panics, broken partitions, etc.
> 
> What version of the file system, how many of those issues (if any) have
> you reported to the upstream mailing list? I don't remember seeing any
> reports from you.
> 
>>
>> One specific case saw the btrfs drive pool mounted under the /var
>> partition so that upon installation the btrfs pool contained all files
>> under /var; /lib/mysql as well as the video storage. Needless to say
>> this was a catastrophe...
> 
> BTRFS is not suitable for random workloads, such as databases for
> example. In those cases it's recommended, at the very least, to disable
> COW on the database files. Note, disabling CoW doesn't preclude you from
> creating snapshots, it just means that not every 4k write (for example)
> will result in a CoW operation.
> 
>>
>> At the other end of the spectrum is a use case where ONLY the video
>> storage was on the btrfs pool; but in that case, the btrfs pool became
>> read-only suddenly and would not re-mount as rw despite all the recovery
>> trick btrfs-tools could throw at it. This, of course prevented the
>> NVR/VMS from recording any footage.
> 
> Again, you give absolutely no information about how you have configured
> the filesystem or versions of the software used.
> 
>>
>> So, again, the question is: is BTRFS mature enough to be used in such
>> use case and if so, what approach can be used to mitigate such issues.

  And..

   What is the blocksize used by the application to write into btrfs?
   And what type of write IO? async, directio or fsync?

Thanks, Anand


>>
>> Thank you all for your assistance
>>
>> -Ryan-