From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754730AbbIBPj5 (ORCPT <rfc822;w@1wt.eu>);
	Wed, 2 Sep 2015 11:39:57 -0400
Received: from mga14.intel.com ([192.55.52.115]:51771 "EHLO mga14.intel.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1752484AbbIBPj4 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Wed, 2 Sep 2015 11:39:56 -0400
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.17,454,1437462000"; 
   d="scan'208";a="781265390"
Subject: Re: [PATCH] dax, pmem: add support for msync
To: Boaz Harrosh <boaz@plexistor.com>, Dave Chinner <david@fromorbit.com>,
        Ross Zwisler <ross.zwisler@linux.intel.com>,
        Christoph Hellwig <hch@lst.de>, linux-kernel@vger.kernel.org,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        Andrew Morton <akpm@osdl.org>, "H. Peter Anvin" <hpa@zytor.com>,
        Hugh Dickins <hughd@google.com>, Ingo Molnar <mingo@redhat.com>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
        linux-nvdimm@ml01.01.org, Matthew Wilcox <willy@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org
References: <1441047584-14664-1-git-send-email-ross.zwisler@linux.intel.com>
 <20150831233803.GO3902@dastard> <20150901070608.GA5482@lst.de>
 <20150901222120.GQ3902@dastard> <20150902031945.GA8916@linux.intel.com>
 <20150902051711.GS3902@dastard> <55E6CF15.4070105@plexistor.com>
 <55E70653.4090302@linux.intel.com> <55E7132E.104@plexistor.com>
From: Dave Hansen <dave.hansen@linux.intel.com>
Message-ID: <55E7184B.3020104@linux.intel.com>
Date: Wed, 2 Sep 2015 08:39:55 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.2.0
MIME-Version: 1.0
In-Reply-To: <55E7132E.104@plexistor.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 09/02/2015 08:18 AM, Boaz Harrosh wrote:
> On 09/02/2015 05:23 PM, Dave Hansen wrote:
>> > I'd be curious what the cost is in practice.  Do you have any actual
>> > numbers of the cost of doing it this way?
>> > 
>> > Even if the instruction is a "noop", I'd really expect the overhead to
>> > really add up for a tens-of-gigabytes mapping, no matter how much the
>> > CPU optimizes it.
> What tens-of-gigabytes mapping? I have yet to encounter an application
> that does that. Our tests show that usually the mmaps are small.

We are going to have 2-socket systems with 6TB of persistent memory in
them.  I think it's important to design this mechanism so that it scales
to memory sizes like that and supports large mmap()s.

I'm not sure the application you've seen thus far are very
representative of what we want to design for.

> I can send you a micro benchmark results of an mmap vs direct-io random
> write. Our code will jump over holes in the file BTW, but I'll ask to also
> run it with falloc that will make all blocks allocated.

I'm really just more curious about actual clflush performance on large
ranges.  I'm curious how good the CPU is at optimizing it.