Re: [LSF/MM TOPIC] Software RAID Support for NV-DIMM

[Bob Liu <bob.liu@oracle.com>]

On 2/16/19 1:39 PM, Dave Chinner wrote:
> On Sat, Feb 16, 2019 at 04:31:33PM +1100, Dave Chinner wrote:
>> On Fri, Feb 15, 2019 at 10:57:12AM +0100, Johannes Thumshirn wrote:
>>> (This is a joint proposal with Hannes Reinecke)
>>>
>>> Servers with NV-DIMM are slowly emerging in data centers but one key feature
>>> for reliability of these systems hasn't been addressed up to now, data
>>> redundancy.
>>>
>>> While it would be best to solve this issue in the memory controller of the CPU
>>> itself, I don't see this coming in the next few years. This puts us as the OS
>>> in the burden to create the redundant copies of data for the users.
>>>
>>> If we leave of the DAX support Linux' software RAID implementations (MD,
>>> device-mapper and BTRFS RAID) do already work on top of pmem devices, but they
>>> are incompatible with DAX.
>>>
>>> In this session Hannes and I would like to discuss eventual ways how we as an
>>> operating system can mitigate these issues for our users.
>>
>> We've supported this since mid 2018 and commit ba23cba9b3bd ("fs:
>> allow per-device dax status checking for filesystems"). That is,
>> we can have DAX on the XFS RT device indepently of the data device.
>>
>> That is, you set up pmem in three segments - two small identical
>> segments start get mirrored with RAID1 as the data device, and
>> the remainder as a block device that is dax capable set up as the
>> XFS realtime device. Set the RTINHERIT bit on the root directory at
>> mkfs time ("-d rtinherit=1") and then all the data goes to the DAX
>> capable realtime device, and all the metadata goes to the software
>> raided pmem block devices that aren't DAX capable.
>>
>> Problem already solved, yes?
> 
> Sorry, this was meant to be a reply to Dan's email commenting about
> some people needing mirrored metadata, not the parent that was
> talking about whole device RAID...
> 
> i.e. mirrored metadata w/ FS-DAX for data should already be a solved
> problem...
> 

Indeed, here is the v2 version about mirrored metadata retry.
https://marc.info/?l=linux-block&m=155005161104512&w=2
Appreciate any reviews, thank you!

- Bob