From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arvind R Subject: Re: [Solved] Nouveau on dom0 Date: Wed, 10 Mar 2010 18:20:42 +0530 Message-ID: References: <20100303181303.GA21078@phenom.dumpdata.com> <20100304182518.GB20263@phenom.dumpdata.com> <20100305202309.GA15454@phenom.dumpdata.com> <20100308175127.GF4568@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20100308175127.GF4568@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Mon, Mar 8, 2010 at 11:21 PM, Konrad Rzeszutek Wilk wrote: > On Sun, Mar 07, 2010 at 05:26:12AM +0530, Arvind R wrote: >> On Sun, Mar 7, 2010 at 2:29 AM, Arvind R wrote: >> > On Sat, Mar 6, 2010 at 1:46 PM, Arvind R wrote: >> >> On Sat, Mar 6, 2010 at 1:53 AM, Konrad Rzeszutek Wilk >> >> wrote: >> >>> On Fri, Mar 05, 2010 at 01:16:13PM +0530, Arvind R wrote: >> >>>> On Thu, Mar 4, 2010 at 11:55 PM, Konrad Rzeszutek Wilk >> >>>> wrote: >> >>>> > On Thu, Mar 04, 2010 at 02:47:58PM +0530, Arvind R wrote: >> >>>> >> On Wed, Mar 3, 2010 at 11:43 PM, Konrad Rzeszutek Wilk >> >>>> >> wrote: >> >> >>> (FYI, look at >> >>> http://git.kernel.org/?p=3Dlinux/kernel/git/konrad/xen.git;a=3Dcommi= t;h=3De84db8b7136d1b4a393dbd982201d0c5a3794333) >> >> THAT SOLVED THE FAULTING; OUT_RING now completes under Xen. > > That is great! Thanks for doing all the hard-work in digging through the > code. > > > So this means you got graphics on the screen? Or at least that Kernel > Mode Setting and the DRM parts show fancy graphics during boot? AT LAST, yes! Patch: (after aboout 600 reboots!) diff -Naur nouveau-kernel.orig/drivers/gpu/drm/ttm/ttm_bo_vm.c nouveau-kernel.new/drivers/gpu/drm/ttm/ttm_bo_vm.c --- nouveau-kernel.orig/drivers/gpu/drm/ttm/ttm_bo_vm.c 2010-01-27 10:19:28.000000000 +0530 +++ nouveau-kernel.new/drivers/gpu/drm/ttm/ttm_bo_vm.c 2010-03-10 17:28:59.000000000 +0530 @@ -271,7 +271,10 @@ */ vma->vm_private_data =3D bo; - vma->vm_flags |=3D VM_RESERVED | VM_IO | VM_MIXEDMAP | VM_DONTEXPAN= D; + vma->vm_flags |=3D VM_RESERVED | VM_MIXEDMAP | VM_DONTEXPAND; + if (!((bo->mem.placement & TTM_PL_MASK_MEM) & TTM_PL_FLAG_TT)) + vma->vm_flags |=3D VM_IO; + vma->vm_page_prot =3D vma_get_vm_prot(vma->vm_flags); return 0; out_unref: ttm_bo_unref(&bo); The previous patch worked for memory-space exported to user via mmap. That worked for the pushbuf, but not for mode-setting (I guess). The ensuing crashes were hard - no logs, nothing. So had to devise ways of forcing log-writing before crashing (and praying). The located iomem problem and had search code for appropriate condition. And setting the vm_page_prot IS important! Nouveau does kernel-modesetting only. The framebuffer device uses channel 1 and is as regular a framebuffer as any other. 2D graphics operations use channel 2 (xf86-video-nouveau). 3D graphics (gallium) use a channel for every 3D window. There are 128 channels, 0 and 127 being reserved. Every channel has a dma-engine which is user triggered thro' pushbuffer rings. Every DMA has a 1MiB VRAM space which forms one of the targets of DMA ops - the other being in the opaque GPU-space. The BO encapsualtes the virtual-address space of the user VM. and the GPU-DMA is provided a constructed PageTable that is consistent with the kernel view= of that space. The GEM_NEW ioctl sets up the whole space-management machinery, the user-space is mmaped out, and the operations triggered thro the pushbuf= . > But to answer your question, the DMA address is actually the MFN > (machine frame number) which is bitshifted by twelve and an offset > added. The debug patch I provided gets that from the > > PTE value: > > =A0 =A0 =A0 =A0if (xen_domain()) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 phys =3D (pte_mfn(*pte) << PAGE_SHIFT) + of= fset; > > The 'phys' now has the physical address that PCI bus (and the video > card) would utilize to request data to. Please keep in mind that the > 'pte_mfn' is a special Xen function. Normally one would do 'pte'. > > There is a layer of indirection in the Linux pvops kernel that makes > this a bit funny. Mainly most of the time you get something called GPFN > which is a psedu-physical MFN. Then there is a translation of PFN to > MFN (or vice-versa). For pages that are being utilized for PCI devices > (and that have _PAGE_IOMAP PTE flag set), the GPFN is actually the MFN, > while for the rest (like the pages allocated by the mmap and then > stitched up in the ttm_bo_fault handler), it is the PFN. > > .. back to the DMA part. When kernel subsystems do DMA they go through a > PCI DMA API. This API has things such as 'dma_map_page', which through > layers of indirection calls the Xen SWIOTLB layer. The Xen SWIOTLB is > smart enough (actually, the enligthen.c) to distinguish if the page has > _PAGE_IOMAP set or not and to figure out if the PTE has a MFN or PFN. > > Hopefully I've not confused this matter :-( On the contrary, a neat essence of the matter - only wish it was clear to m= e a month ago:-( YAHOO! (just a simple shout)