From: Ido Shamay <idos@dev.mellanox.co.il>
To: David Daney <ddaney.cavm@gmail.com>,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-rdma@vger.kernel.org,
"David S. Miller" <davem@davemloft.net>
Cc: Roland Dreier <roland@kernel.org>,
Sean Hefty <sean.hefty@intel.com>,
Hal Rosenstock <hal.rosenstock@gmail.com>,
Amir Vadai <amirv@mellanox.com>,
Or Gerlitz <ogerlitz@mellanox.com>,
Yishai Hadas <yishaih@mellanox.com>,
Matan Barak <matanb@mellanox.com>,
Majd Dibbiny <majd@mellanox.com>,
Jack Morgenstein <jackm@dev.mellanox.co.il>,
Moni Shoua <monis@mellanox.com>,
Eugenia Emantayev <eugenia@mellanox.co.il>,
Saeed Mahameed <saeedm@mellanox.com>,
Yuval Atias <yuvala@mellanox.com>,
Maor Gottlieb <maorg@mellanox.com>,
David Daney <david.daney@cavium.com>,
liranl@mellanox.com, gdror@mellanox.com
Subject: Re: [PATCH RFC] net/mlx4: Remove improper usage of dma_alloc_coherent().
Date: Mon, 13 Apr 2015 00:33:24 +0300 [thread overview]
Message-ID: <552AE4A4.30005@dev.mellanox.co.il> (raw)
In-Reply-To: <55243362.5040207@dev.mellanox.co.il>
On 4/7/2015 10:43 PM, Ido Shamay wrote:
> On 4/7/2015 2:00 AM, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> The dma_alloc_coherent() function returns a virtual address which can
>> be used for coherent access to the underlying memory. On some
>> architectures, like arm64, undefined behavior results if this memory is
>> also accessed via virtual mappings that are not coherent. Because of
>> their undefined nature, operations like virt_to_page() return garbage
>> when passed virtual addresses obtained from dma_alloc_coherent(). Any
>> subsequent mappings via vmap() of the garbage page values are unusable
>> and result in bad things like bus errors (synchronous aborts in ARM64
>> speak).
>>
>> The MLX4 driver contains code that does the equivalent of:
>>
>> vmap(virt_to_page(dma_alloc_coherent))
>>
>> This results in an OOPs when the device is opened.
>>
>> To fix this...
>>
>> Always use result of dma_alloc_coherent() directly.
> Hi David,
>
> I'm not sure this solution is good enough for the common case(s).
> Typical allocation size will be around 64KB (with default 1K ring size).
> We can't rely on the system to always provide us with that amount of
> contiguous memory.
>
> Current code allocation scheme is more robust, max_direct is typically
> 2 * PAGE_SIZE,
> so pages from order 1 are far more available then higher order.
>
> I need to check why the code is written as it is today, and not as in
> this RFC (which is much more trivial).
> I'll continue to investigate tomorrow, will get back with some answers.
>
> Ido
Acked-by: Ido Shamay <idos@mellanox.com>
Thanks David, this is good for us
>> Remove 'max_direct' parameter to mlx4_buf_alloc(), as it is unused,
>> and adjust all callers.
>>
>> Remove mlx4_en_map_buffer() and mlx4_en_unmap_buffer() as they now do
>> nothing, and adjust all callers.
>>
>> Remove 'page_list' element from struct mlx4_buf as it is unused.
>>
>> Signed-off-by: David Daney <david.daney@cavium.com>
next prev parent reply other threads:[~2015-04-12 21:33 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-06 23:00 [PATCH RFC] net/mlx4: Remove improper usage of dma_alloc_coherent() David Daney
2015-04-07 19:43 ` Ido Shamay
2015-04-12 21:33 ` Ido Shamay [this message]
2015-04-29 14:13 ` Ido Shamay
2015-04-30 0:07 ` Daney, David
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=552AE4A4.30005@dev.mellanox.co.il \
--to=idos@dev.mellanox.co.il \
--cc=amirv@mellanox.com \
--cc=davem@davemloft.net \
--cc=david.daney@cavium.com \
--cc=ddaney.cavm@gmail.com \
--cc=eugenia@mellanox.co.il \
--cc=gdror@mellanox.com \
--cc=hal.rosenstock@gmail.com \
--cc=jackm@dev.mellanox.co.il \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=liranl@mellanox.com \
--cc=majd@mellanox.com \
--cc=maorg@mellanox.com \
--cc=matanb@mellanox.com \
--cc=monis@mellanox.com \
--cc=netdev@vger.kernel.org \
--cc=ogerlitz@mellanox.com \
--cc=roland@kernel.org \
--cc=saeedm@mellanox.com \
--cc=sean.hefty@intel.com \
--cc=yishaih@mellanox.com \
--cc=yuvala@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox