public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: Ido Shamay <idos@dev.mellanox.co.il>
To: David Daney <ddaney.cavm@gmail.com>,
	linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
	linux-rdma@vger.kernel.org,
	"David S. Miller" <davem@davemloft.net>
Cc: Roland Dreier <roland@kernel.org>,
	Sean Hefty <sean.hefty@intel.com>,
	Hal Rosenstock <hal.rosenstock@gmail.com>,
	Amir Vadai <amirv@mellanox.com>,
	Or Gerlitz <ogerlitz@mellanox.com>,
	Yishai Hadas <yishaih@mellanox.com>,
	Matan Barak <matanb@mellanox.com>,
	Majd Dibbiny <majd@mellanox.com>,
	Jack Morgenstein <jackm@dev.mellanox.co.il>,
	Moni Shoua <monis@mellanox.com>,
	Eugenia Emantayev <eugenia@mellanox.co.il>,
	Saeed Mahameed <saeedm@mellanox.com>,
	Yuval Atias <yuvala@mellanox.com>,
	Maor Gottlieb <maorg@mellanox.com>,
	David Daney <david.daney@cavium.com>,
	liranl@mellanox.com, gdror@mellanox.com
Subject: Re: [PATCH RFC] net/mlx4: Remove improper usage of dma_alloc_coherent().
Date: Mon, 13 Apr 2015 00:33:24 +0300	[thread overview]
Message-ID: <552AE4A4.30005@dev.mellanox.co.il> (raw)
In-Reply-To: <55243362.5040207@dev.mellanox.co.il>

On 4/7/2015 10:43 PM, Ido Shamay wrote:
> On 4/7/2015 2:00 AM, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> The dma_alloc_coherent() function returns a virtual address which can
>> be used for coherent access to the underlying memory.  On some
>> architectures, like arm64, undefined behavior results if this memory is
>> also accessed via virtual mappings that are not coherent. Because of
>> their undefined nature, operations like virt_to_page() return garbage
>> when passed virtual addresses obtained from dma_alloc_coherent().  Any
>> subsequent mappings via vmap() of the garbage page values are unusable
>> and result in bad things like bus errors (synchronous aborts in ARM64
>> speak).
>>
>> The MLX4 driver contains code that does the equivalent of:
>>
>>    vmap(virt_to_page(dma_alloc_coherent))
>>
>> This results in an OOPs when the device is opened.
>>
>> To fix this...
>>
>> Always use result of dma_alloc_coherent() directly.
> Hi David,
>
> I'm not sure this solution is good enough for the common case(s).
> Typical allocation size will be around 64KB (with default 1K ring size).
> We can't rely on the system to always provide us with that amount of 
> contiguous memory.
>
> Current code allocation scheme is more robust, max_direct is typically 
> 2 * PAGE_SIZE,
> so pages from order 1 are far more available then higher order.
>
> I need to check why the code is written as it is today,  and not as in 
> this RFC (which is much more trivial).
> I'll continue to investigate tomorrow, will get back with some answers.
>
> Ido

Acked-by: Ido Shamay <idos@mellanox.com>
Thanks David, this is good for us

>> Remove 'max_direct' parameter to mlx4_buf_alloc(), as it is unused,
>> and adjust all callers.
>>
>> Remove mlx4_en_map_buffer() and mlx4_en_unmap_buffer() as they now do
>> nothing, and adjust all callers.
>>
>> Remove 'page_list' element from struct mlx4_buf as it is unused.
>>
>> Signed-off-by: David Daney <david.daney@cavium.com>

  reply	other threads:[~2015-04-12 21:33 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-06 23:00 [PATCH RFC] net/mlx4: Remove improper usage of dma_alloc_coherent() David Daney
2015-04-07 19:43 ` Ido Shamay
2015-04-12 21:33   ` Ido Shamay [this message]
2015-04-29 14:13     ` Ido Shamay
2015-04-30  0:07       ` Daney, David

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=552AE4A4.30005@dev.mellanox.co.il \
    --to=idos@dev.mellanox.co.il \
    --cc=amirv@mellanox.com \
    --cc=davem@davemloft.net \
    --cc=david.daney@cavium.com \
    --cc=ddaney.cavm@gmail.com \
    --cc=eugenia@mellanox.co.il \
    --cc=gdror@mellanox.com \
    --cc=hal.rosenstock@gmail.com \
    --cc=jackm@dev.mellanox.co.il \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=liranl@mellanox.com \
    --cc=majd@mellanox.com \
    --cc=maorg@mellanox.com \
    --cc=matanb@mellanox.com \
    --cc=monis@mellanox.com \
    --cc=netdev@vger.kernel.org \
    --cc=ogerlitz@mellanox.com \
    --cc=roland@kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=sean.hefty@intel.com \
    --cc=yishaih@mellanox.com \
    --cc=yuvala@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox