From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752657AbbE0N0T (ORCPT ); Wed, 27 May 2015 09:26:19 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:27139 "EHLO mailout2.w1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751322AbbE0N0R (ORCPT ); Wed, 27 May 2015 09:26:17 -0400 X-AuditID: cbfec7f4-f79c56d0000012ee-70-5565c5f69c6b Message-id: <5565C5F5.1000000@samsung.com> Date: Wed, 27 May 2015 15:26:13 +0200 From: Marek Szyprowski User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-version: 1.0 To: Laurent Pinchart , linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, Laura Abbott , Arnd Bergmann , Mitchel Humpherys , Joreg Roedel , Will Deacon , Grant Likely , Robin Murphy , Thierry Reding , Greg Kroah-Hartman Subject: Re: [RFC/PATCH 0/9] IOMMU probe deferral support References: <1431644410-2997-1-git-send-email-laurent.pinchart+renesas@ideasonboard.com> In-reply-to: <1431644410-2997-1-git-send-email-laurent.pinchart+renesas@ideasonboard.com> Content-type: text/plain; charset=utf-8; format=flowed Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprKIsWRmVeSWpSXmKPExsVy+t/xK7rfjqaGGuy5ZWnxd9IxdosDf3Yw WjQvXs9msWC/tUXn7A3sFts7Z7BbTGy6w2Kx6fE1VovLu+awWaydv43N4uCHJ6wWP3fNY7F4 +fEEiwOvx5OD85g81sxbw+jx+9ckRo/Lfb1MHjtn3WX3mN0xk9XjzrU9bB77565h99i8pN5j 8o3ljB6fN8kFcEdx2aSk5mSWpRbp2yVwZVybv5CtYIFuxZmpLUwNjEtUuhg5OCQETCTmHCzp YuQEMsUkLtxbz9bFyMUhJLCUUeLazJPsIAkhgeeMEqueqoLYvAJaEnPevWYCsVkEVCX23bkL VsMmYCjR9baLDcQWFYiR6NvaywxRLyjxY/I9FhBbRCBBonPVUWaQBcwCW5glZvV8ARskLGAp 0fm4nxFiWZTEwZu/wWxOgWiJRa33wIYyC5hJfHl5mBXClpfYvOYt8wRGgVlIdsxCUjYLSdkC RuZVjKKppckFxUnpuYZ6xYm5xaV56XrJ+bmbGCGR9WUH4+JjVocYBTgYlXh4D0inhgqxJpYV V+YeYpTgYFYS4Z26FyjEm5JYWZValB9fVJqTWnyIUZqDRUmcd+6u9yFCAumJJanZqakFqUUw WSYOTqkGRs+Fv46H/2tg2Pnw1Jdppz9vmLLKZtPmErf9/6sUlrucXvdQL/q/xSm9zzxRB5tn B2Y4n5qzc6PpZqfL4Z9LxXpqQ78/2r5iDcO/O4/infQD125+99V6QXnZKvZTfzdyTRRWyFZb sVm2foEUS6X0rEeSdT5Pb2r/yHLZ1Ha7kql7ni3z5xJXc28lluKMREMt5qLiRABo3NpPqAIA AA== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, On 2015-05-15 01:00, Laurent Pinchart wrote: > This patch series attempts to implement support for deferring probe of both > IOMMU drivers and bus master drivers. > > The relationship between bus masters and IOMMUs creates a strong ordering > during initialization of devices. As in the general case IOMMUs are hidden > behind the DMA mapping API, IOMMU support relies on the automatic setup of DMA > operations without any direct intervention of bus master drivers. > > DMA operations are set up when platform devices are added to the system. This > requires IOMMUs to be available at that time. On systems where ordering of > device add and probe can't be guaranteed (such as, but not limited to, > DT-based systems) this caused incorrect DMA operation setup. This has been > addressed by a patches series [1] that introduced a DT-based early > registration mechanism for IOMMUs. > > However, that mechanism fails to address all issues. Various dependencies > exist between IOMMU devices and other devices, in particular on clocks and on > power domains (as mentioned by Marek in [2]). While there are mechanisms to > handle some of them without probe deferral (for instance by using the > OF_DECLARE macros to register clock drivers), generalizing those mechanisms > would essentially recreate a probe ordering mechanism similar to link order > probe ordering and couldn't really scale. > > Additionally, IOMMUs could also be present hot-pluggable devices and depend on > resources that are thus hot-plugged. OF_DECLARE wouldn't help in that case. > For all those reasons probe deferral for IOMMUs has been considered as desired > if it can be implemented cleanly. For more in-depth information see [3]. > > This RFC series is a first attempt at implementing IOMMU probe deferral > support cleanly. > > The core idea is to move setup of DMA operations from device add time to > device probe time, implemented in patch 6/9. It could be possible to move > setup of other DMA parameters (namely masks and offset) to probe time as well, > but that change would be more intrusive and has a higher risk of introducing > regressions. For that reason I've decided to keep DMA masks and offset setup > at device add time and thus split DMA configuration in masks and operations > (patch 5/9). This can be revisited if we decide that the DMA mapping API > shouldn't require masks and offset to be set before probe time. > > Patch 8/9 then defers probe of bus master drivers when required IOMMUs are not > available yet. This requires knowing when a failed IOMMU lookup should be > considered as permanent or temporary. I've reused the OF_DECLARE_IOMMU for > this purpose, considering that the presence of a driver compatible with the > IOMMU DT node indicates that the failure is temporary and probing of the bus > master device should be deferred. > > Note that only IOMMU drivers using the recent .of_xlate() mechanism for > DT-based IOMMU reference can cause probe deferral of bus master devices. The > .add_device() mechanism isn't supported in this case. > > As an example I've converted the ipmmu-vmsa driver to the new API in patch 9/9. > > At this point many enhancements are possible, but I'd like to receive feedback > on the proposed approach before basing more patches on this series. One > particular point I would like to address (or see being addressed) in the > future is the use of struct iommu_ops with of_iommu_get_ops() and > of_iommu_set_ops(). I believe we should introduct a struct iommu and register > IOMMU instances instead of IOMMU operations. That should bring us one step > closer to removing bus_set_iommu(). I've checked this patchset with my recent updates to Exynos IOMMU driver. It works fine. Then I removed my exynos_iommu_of_setup() callback and relied only on proper initialization from platform device probe() of all iommu controllers. The driver also worked fine in such case. The other change I had to make to get everything working was removal of some hackery in dma-mapping internal structures in Exynos DRM driver, but this is completely different story. You can add the following tag for the of/iommu/dma-mapping patches: Tested-by: Marek Szyprowski > [1] http://www.spinics.net/lists/arm-kernel/msg382787.html > [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-February/323238.html > [3] https://lkml.org/lkml/2015/2/16/345 > > Laurent Pinchart (9): > arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() > arm: dma-mapping: Support IOMMU mappings spanning the full 32 bits > range > of: dma: Move range size workaround to of_dma_get_range() > of: dma: Make of_dma_deconfigure() public > of: dma: Split of_configure_dma() into mask and ops configuration > drivers: platform: Configure dma operations at probe time > iommu: of: Document the of_iommu_configure() function > iommu: of: Handle IOMMU lookup failure with deferred probing or error > iommu/ipmmu-vmsa: Use DT-based instantiation > > arch/arm/include/asm/dma-iommu.h | 2 +- > arch/arm/mm/dma-mapping.c | 21 +++-- > drivers/base/platform.c | 9 ++ > drivers/iommu/ipmmu-vmsa.c | 189 +++++++++++++-------------------------- > drivers/iommu/of_iommu.c | 29 +++++- > drivers/of/address.c | 20 ++++- > drivers/of/device.c | 77 ++++++++++------ > drivers/of/of_pci.c | 3 +- > drivers/of/platform.c | 16 ++-- > include/linux/of_device.h | 14 ++- > 10 files changed, 195 insertions(+), 185 deletions(-) Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland