Re: TTM and AGP conflicts

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Jan 03, 2012 at 05:43:40PM -0500, Jerome Glisse wrote:
> On Fri, Dec 23, 2011 at 01:19:43AM +0000, James Simmons wrote:
> > 
> > > > Hi!
> > > >
> > > >        I updated the openchrome tree and while testing on the AGP system
> > > > discovered some interesting problems with the new TTM changes. The
> > > > problems center around the ttm_tt_[un]populate which I modeled after the
> > > > radeon and nouveau driver.
> > > >        First problem I noticed was on a AGP system that my ttm_tt_populate
> > > > function would oops. Tracking it down I found the problem was the
> > > > ttm_agp_tt_create calls ttm_tt_init instead of ttm_dma_tt_init so once my
> > > > ttm_tt_populate function would attempt to touch the dma_address it would
> > > > oops. The second issue is the assumption of the cast for struct ttm_tt in
> > > > both the populate and unpopulate function. For the AGP case the proper
> > > > case would be to ttm_agp_backend.
> > > >        At this point my assumption is the ttm_bo_move function has to be
> > > > rewritten to handle the AGP case to avoid calling ttm_tt_bind and in all
> > > > cases ttm_tt_bind needs to be avoided. Looking at the radeon and nouveau
> > > > drivers I don't see that testing happening. Am I just missing something?
> > > 
> > > Happens on AGP radeons as well:
> > > https://bugs.freedesktop.org/show_bug.cgi?id=43719
> > 
> > 	So I'm not crazy, so this needs to be fixed. Here is what my 
> > understanding of the TTM layer is. My impression is that struct ttm_bo_driver
> > handles multiple domains, AGP, pcie etc and in each method you have to 
> > handle each specific domain you support. Also *move gives the impression of
> > moving between these different domains. 
> > 	Where as for struct ttm_backend_func was more for allocating from
> > a specific domain. Also I never saw a clear way to handle multiple backends. 
> > For example my AGP systems can also do pci dma as well. 
> 
> > _______________________________________________
> > dri-devel mailing list
> > dri-devel@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/dri-devel
> 
> Attached is patch to fix this, so sorry about that, i must have lost my
> agp change along the way when working on the patchset. This patch is not
> extensively tested, i will do more testing tomorrow on more gpu, might
> even found an nvidia agp i can try. Again sorry for breaking this.

Hey Jerome,

Was going to look at this week and see how it performs before (and after)
the squash ttm bind+populate operation. Any thoughts of what benchmarks I
should run?

Fyi, I've some NVidia AGP cards. We both live in Boston so could meet up.

And I could ask you about the patches I sent - not clear if you want to me
squash the patch 2 and 3 together or just redo them diffrently.


> 
> Cheers,
> Jerome

> >From 99a6eee89a43bce155a93ab6d71ea041aac10680 Mon Sep 17 00:00:00 2001
> From: Jerome Glisse <jglisse@redhat.com>
> Date: Tue, 3 Jan 2012 17:37:37 -0500
> Subject: [PATCH] ttm: fix agp since ttm tt rework
> 
> ttm tt rework modified the way we allocate and populate the
> ttm_tt structure, the AGP side was missing some bit to properly
> work. Fix those and fix radeon and nouveau AGP support.
> 
> Tested on radeon only so far.
> 
> Signed-off-by: Jerome Glisse <jglisse@redhat.com>
> ---
>  drivers/gpu/drm/nouveau/nouveau_bo.c  |   13 +++++++++++++
>  drivers/gpu/drm/radeon/radeon_ttm.c   |   11 +++++++++++
>  drivers/gpu/drm/ttm/ttm_agp_backend.c |   17 +++++++++++++++++
>  drivers/gpu/drm/ttm/ttm_tt.c          |    2 ++
>  include/drm/ttm/ttm_bo_driver.h       |    2 ++
>  5 files changed, 45 insertions(+), 0 deletions(-)
> 
> diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c
> index 8cf4a48..724b41a 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_bo.c
> +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
> @@ -1066,6 +1066,12 @@ nouveau_ttm_tt_populate(struct ttm_tt *ttm)
>  	dev_priv = nouveau_bdev(ttm->bdev);
>  	dev = dev_priv->dev;
>  
> +#if __OS_HAS_AGP
> +	if (dev_priv->gart_info.type == NOUVEAU_GART_AGP) {
> +		return ttm_agp_tt_populate(ttm);
> +	}
> +#endif
> +
>  #ifdef CONFIG_SWIOTLB
>  	if (swiotlb_nr_tbl()) {
>  		return ttm_dma_populate((void *)ttm, dev->dev);
> @@ -1105,6 +1111,13 @@ nouveau_ttm_tt_unpopulate(struct ttm_tt *ttm)
>  	dev_priv = nouveau_bdev(ttm->bdev);
>  	dev = dev_priv->dev;
>  
> +#if __OS_HAS_AGP
> +	if (dev_priv->gart_info.type == NOUVEAU_GART_AGP) {
> +		ttm_agp_tt_unpopulate(ttm);
> +		return;
> +	}
> +#endif
> +
>  #ifdef CONFIG_SWIOTLB
>  	if (swiotlb_nr_tbl()) {
>  		ttm_dma_unpopulate((void *)ttm, dev->dev);
> diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
> index b0ebaf8..dc73d77 100644
> --- a/drivers/gpu/drm/radeon/radeon_ttm.c
> +++ b/drivers/gpu/drm/radeon/radeon_ttm.c
> @@ -588,6 +588,11 @@ static int radeon_ttm_tt_populate(struct ttm_tt *ttm)
>  		return 0;
>  
>  	rdev = radeon_get_rdev(ttm->bdev);
> +#if __OS_HAS_AGP
> +	if (rdev->flags & RADEON_IS_AGP) {
> +		return ttm_agp_tt_populate(ttm);
> +	}
> +#endif
>  
>  #ifdef CONFIG_SWIOTLB
>  	if (swiotlb_nr_tbl()) {
> @@ -624,6 +629,12 @@ static void radeon_ttm_tt_unpopulate(struct ttm_tt *ttm)
>  	unsigned i;
>  
>  	rdev = radeon_get_rdev(ttm->bdev);
> +#if __OS_HAS_AGP
> +	if (rdev->flags & RADEON_IS_AGP) {
> +		ttm_agp_tt_unpopulate(ttm);
> +		return;
> +	}
> +#endif
>  
>  #ifdef CONFIG_SWIOTLB
>  	if (swiotlb_nr_tbl()) {
> diff --git a/drivers/gpu/drm/ttm/ttm_agp_backend.c b/drivers/gpu/drm/ttm/ttm_agp_backend.c
> index 14ebd36..747c141 100644
> --- a/drivers/gpu/drm/ttm/ttm_agp_backend.c
> +++ b/drivers/gpu/drm/ttm/ttm_agp_backend.c
> @@ -31,6 +31,7 @@
>  
>  #include "ttm/ttm_module.h"
>  #include "ttm/ttm_bo_driver.h"
> +#include "ttm/ttm_page_alloc.h"
>  #ifdef TTM_HAS_AGP
>  #include "ttm/ttm_placement.h"
>  #include <linux/agp_backend.h>
> @@ -97,6 +98,7 @@ static void ttm_agp_destroy(struct ttm_tt *ttm)
>  
>  	if (agp_be->mem)
>  		ttm_agp_unbind(ttm);
> +	ttm_tt_fini(ttm);
>  	kfree(agp_be);
>  }
>  
> @@ -129,4 +131,19 @@ struct ttm_tt *ttm_agp_tt_create(struct ttm_bo_device *bdev,
>  }
>  EXPORT_SYMBOL(ttm_agp_tt_create);
>  
> +int ttm_agp_tt_populate(struct ttm_tt *ttm)
> +{
> +	if (ttm->state != tt_unpopulated)
> +		return 0;
> +
> +	return ttm_pool_populate(ttm);
> +}
> +EXPORT_SYMBOL(ttm_agp_tt_populate);
> +
> +void ttm_agp_tt_unpopulate(struct ttm_tt *ttm)
> +{
> +	ttm_pool_unpopulate(ttm);
> +}
> +EXPORT_SYMBOL(ttm_agp_tt_unpopulate);
> +
>  #endif
> diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c
> index 58e1fa1..2f75d20 100644
> --- a/drivers/gpu/drm/ttm/ttm_tt.c
> +++ b/drivers/gpu/drm/ttm/ttm_tt.c
> @@ -191,6 +191,7 @@ int ttm_tt_init(struct ttm_tt *ttm, struct ttm_bo_device *bdev,
>  	ttm->page_flags = page_flags;
>  	ttm->dummy_read_page = dummy_read_page;
>  	ttm->state = tt_unpopulated;
> +	ttm->swap_storage = NULL;
>  
>  	ttm_tt_alloc_page_directory(ttm);
>  	if (!ttm->pages) {
> @@ -222,6 +223,7 @@ int ttm_dma_tt_init(struct ttm_dma_tt *ttm_dma, struct ttm_bo_device *bdev,
>  	ttm->page_flags = page_flags;
>  	ttm->dummy_read_page = dummy_read_page;
>  	ttm->state = tt_unpopulated;
> +	ttm->swap_storage = NULL;
>  
>  	INIT_LIST_HEAD(&ttm_dma->pages_list);
>  	ttm_dma_tt_alloc_page_directory(ttm_dma);
> diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h
> index 2be8891..d43e892 100644
> --- a/include/drm/ttm/ttm_bo_driver.h
> +++ b/include/drm/ttm/ttm_bo_driver.h
> @@ -1030,6 +1030,8 @@ extern struct ttm_tt *ttm_agp_tt_create(struct ttm_bo_device *bdev,
>  					struct agp_bridge_data *bridge,
>  					unsigned long size, uint32_t page_flags,
>  					struct page *dummy_read_page);
> +int ttm_agp_tt_populate(struct ttm_tt *ttm);
> +void ttm_agp_tt_unpopulate(struct ttm_tt *ttm);
>  #endif
>  
>  #endif
> -- 
> 1.7.5.4
> 

> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel