From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757272Ab0CaTvc (ORCPT ); Wed, 31 Mar 2010 15:51:32 -0400 Received: from out02.mta.xmission.com ([166.70.13.232]:55294 "EHLO out02.mta.xmission.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755628Ab0CaTvb (ORCPT ); Wed, 31 Mar 2010 15:51:31 -0400 To: Neil Horman Cc: Vivek Goyal , iommu@lists.linux-foundation.org, joerg.roedel@amd.com, hbabu@us.ibm.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] amd iommu: force flush of iommu prior during shutdown References: <20100331152417.GB13406@hmsreliant.think-freely.org> <20100331155430.GF14011@redhat.com> <20100331182824.GC13406@hmsreliant.think-freely.org> <20100331191811.GD13406@hmsreliant.think-freely.org> From: ebiederm@xmission.com (Eric W. Biederman) Date: Wed, 31 Mar 2010 12:51:25 -0700 In-Reply-To: <20100331191811.GD13406@hmsreliant.think-freely.org> (Neil Horman's message of "Wed\, 31 Mar 2010 15\:18\:11 -0400") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-XM-SPF: eid=;;;mid=;;;hst=in01.mta.xmission.com;;;ip=76.21.114.89;;;frm=ebiederm@xmission.com;;;spf=neutral X-SA-Exim-Connect-IP: 76.21.114.89 X-SA-Exim-Rcpt-To: nhorman@tuxdriver.com, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, hbabu@us.ibm.com, joerg.roedel@amd.com, iommu@lists.linux-foundation.org, vgoyal@redhat.com X-SA-Exim-Mail-From: ebiederm@xmission.com X-SA-Exim-Scanned: No (on in01.mta.xmission.com); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Neil Horman writes: > On Wed, Mar 31, 2010 at 11:57:46AM -0700, Eric W. Biederman wrote: >> Neil Horman writes: >> >> > On Wed, Mar 31, 2010 at 11:54:30AM -0400, Vivek Goyal wrote: >> >> >> So this call amd_iommu_flush_all_devices() will be able to tell devices >> >> that don't do any more DMAs and hence it is safe to reprogram iommu >> >> mapping entries. >> >> >> > It blocks the cpu until any pending DMA operations are complete. Hmm, as I >> > think about it, there is still a small possibility that a device like a NIC >> > which has several buffers pre-dma-mapped could start a new dma before we >> > completely disabled the iommu, althought thats small. I never saw that in my >> > testing, but hitting that would be fairly difficult I think, since its literally >> > just a few hundred cycles between the flush and the actual hardware disable >> > operation. >> > >> > According to this though: >> > http://support.amd.com/us/Processor_TechDocs/34434-IOMMU-Rev_1.26_2-11-09.pdf >> > That window could be closed fairly easily, but simply disabling read and write >> > permissions for each device table entry prior to calling flush. If we do that, >> > then flush the device table, any subsequently started dma operation would just >> > get noted in the error log, which we could ignore, since we're abot to boot to >> > the kdump kernel anyway. >> > >> > Would you like me to respin w/ that modification? >> >> Disabling permissions on all devices sounds good for the new virtualization >> capable iommus. I think older iommus will still be challenged. I think >> on x86 we have simply been able to avoid using those older iommus. >> >> I like the direction you are going but please let's put this in a >> paranoid iommu enable routine. >> > You mean like initialize the device table so that all devices are default > disabled on boot, and then selectively enable them (perhaps during a > device_attach)? I can give that a spin. That sounds good. Eric