From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:54977)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <aavati@redhat.com>) id 1TJTH1-0005Nh-Tj
	for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:02:21 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <aavati@redhat.com>) id 1TJTGy-0003vV-28
	for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:02:19 -0400
Received: from mx1.redhat.com ([209.132.183.28]:19448)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <aavati@redhat.com>) id 1TJTGx-0003uW-Qv
	for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:02:16 -0400
Message-ID: <506C7CD5.6090407@redhat.com>
Date: Wed, 03 Oct 2012 10:58:45 -0700
From: Anand Avati <aavati@redhat.com>
MIME-Version: 1.0
References: <20120924091008.GJ18470@in.ibm.com>
	<20120924091340.GN18470@in.ibm.com> <506C5EB0.50508@redhat.com>
In-Reply-To: <506C5EB0.50508@redhat.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [Qemu-devel] O_DIRECT on glusterfs (was Re: [PATCH v9 4/4]
 block: Support GlusterFS as a QEMU block backend)
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>, Anthony Liguori <aliguori@us.ibm.com>, Vijay Bellur <vbellur@redhat.com>, Stefan Hajnoczi <stefanha@gmail.com>, Harsh Bora <harsh@linux.vnet.ibm.com>, Amar Tumballi <amarts@redhat.com>, qemu-devel@nongnu.org, "Richard W.M. Jones" <rjones@redhat.com>, Blue Swirl <blauwirbel@gmail.com>, Avi Kivity <avi@redhat.com>, bharata@linux.vnet.ibm.com, Daniel Veillard <veillard@redhat.com>

On 10/03/2012 08:50 AM, Paolo Bonzini wrote:
> Il 24/09/2012 11:13, Bharata B Rao ha scritto:
>> +
>> +    if ((bdrv_flags&  BDRV_O_NOCACHE)) {
>> +        open_flags |= O_DIRECT;
>> +    }
>> +
>
> If I understand correctly what I was told, this prevents the brick
> server from using its own buffer cache.  This is quite different from
> what we do for example over NFS (where the host does no caching, but
> nothing prevents it on the remote server).
>
> I think these 3 lines should be removed.  We're bypassing the host
> buffer cache just by virtue of using a userspace driver, and that's what
> cache=none cares about.
>
> Paolo

O_DIRECT also has an effect on the behavior of the "client side" (the 
part within the qemu) of Gluster stack as well. I presume the intention 
of O_DIRECT is to minimize use of memory (whether as host' page cache or 
buffered data in user space). To that end it is a good idea to leave 
O_DIRECT flag set.

The behavior of whether gluster bricks need to get the O_DIRECT 
propagated or not is a different issue. We are exploring the possibility 
of not sending O_DIRECT flag over the wire to mimic NFS behavior. That 
would be independent of the qemu block driver setting the open flag.

Avati