From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sagi Grimberg Subject: Re: RDMA Read: Local protection error Date: Thu, 26 May 2016 19:48:44 +0300 Message-ID: <574728EC.9040802@grimberg.me> References: <1A4F4C32-CE5A-44D9-9BFE-0E1F8D5DF44D@oracle.com> <57238F8C.70505@sandisk.com> <57277B63.8030506@sandisk.com> <6BBFD126-877C-4638-BB91-ABF715E29326@oracle.com> <1AFD636B-09FC-4736-B1C5-D1D9FA0B97B0@oracle.com> <8a3276bf-f716-3dca-9d54-369fc3bdcc39@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Chuck Lever , Bart Van Assche , Yishai Hadas Cc: Yishai Hadas , linux-rdma , Or Gerlitz , Joonsoo Kim , Haggai Eran , Majd Dibbiny List-Id: linux-rdma@vger.kernel.org >>>> When debugging is disabled, kzalloc returns page-aligned >>>> addresses: >>> >>> Is it defined some where that regular kzalloc/kmalloc guaranties to >>> return a page-aligned address as you see in your testing ? if so the >>> debug mode should behave the same. Otherwise we can consider using any >>> flag allocation that can force that if such exists. >>> Let's get other people's input here. >> >> My understanding is that the fact that k[mz]alloc() returns a page-aligned buffer if the allocation size is > PAGE_SIZE / 2 is a side effect of the implementation and not something callers of that function should rely on. I think the only assumption k[mz]alloc() callers should rely on is that the allocated memory respects ARCH_KMALLOC_MINALIGN. > > I agree. mlx4_alloc_priv_pages() is carefully designed to > correct the alignment of the buffer, so it already assumes > that it is not getting a page-aligned buffer. > > The alignment isn't the problem here, though. It's that > the buffer contains a page-boundary. That is guaranteed > to be the case for HCAs that support more than 512 > sges, so that will have to be addressed (at least in > mlx5). rrr... I think we should make the pages allocations dma coherent in order to fix that... Nice catch Chunk. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html