PPC Lockup (ati-pcigart-branch)

PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

[CC'ing linuxppc-dev, hopefully someone there knows what might be up...]

This is where it dies:


        /* FIXME: We should really have a kernel call for this...
         */
        entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
                                    GFP_KERNEL,
                                    PAGE_KERNEL);

        printk("Checkpoint 2\n");


Checkpoint 2 is never reached, the machine is absolutely dead.


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Michel Dänzer wrote:
>
> [CC'ing linuxppc-dev, hopefully someone there knows what might be up...]
>
> This is where it dies:
>
>         /* FIXME: We should really have a kernel call for this...
>          */
>         entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
>                                     GFP_KERNEL,
>                                     PAGE_KERNEL);

This isn't very much information, but only one thing can really
be wrong........

What is the value of 'pages'?  I suspect it is huge (and perhaps
wrong).  The GFP_KERNEL flag will cause vmalloc() to wait for pages
to become available (i.e. it will swap other things out).  If this
value in incorrect, this call will wait forever for pages that are
never going to arrive.  Worse, it is going to keep sucking up pages
and holding them, so nothing else is going to run either.  Is "pages"
really the number of pages, or a size that was never converted to pages?

...and for the 'FIXME' comment, you want a function that does
what? vmalloc?  Why don't you just call it (or one of the more
appropriate variants if necessary)?


	-- Dan

--

	I like MMUs because I don't have a real life.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] Re: PPC Lockup (ati-pcigart-branch)

[Gareth Hughes]

Dan Malek wrote:
>
> Michel Dänzer wrote:
> >
> > [CC'ing linuxppc-dev, hopefully someone there knows what might be up...]
> >
> > This is where it dies:
> >
> >         /* FIXME: We should really have a kernel call for this...
> >          */
> >         entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
> >                                     GFP_KERNEL,
> >                                     PAGE_KERNEL);
>
> This isn't very much information, but only one thing can really
> be wrong........
>
> What is the value of 'pages'?  I suspect it is huge (and perhaps
> wrong).  The GFP_KERNEL flag will cause vmalloc() to wait for pages
> to become available (i.e. it will swap other things out).  If this
> value in incorrect, this call will wait forever for pages that are
> never going to arrive.  Worse, it is going to keep sucking up pages
> and holding them, so nothing else is going to run either.  Is "pages"
> really the number of pages, or a size that was never converted to pages?

Pages is definitely the number of pages (at least when I wrote the
code...).

> ...and for the 'FIXME' comment, you want a function that does
> what? vmalloc?  Why don't you just call it (or one of the more
> appropriate variants if necessary)?

This was originally when I was passing a flag to make the memory
uncached.  This wasn't needed, and we really should be using
vmalloc_32(...) instead (which will result in exactly the same code, but
it is cleaner).

-- Gareth

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] Re: PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

Gareth Hughes wrote:

> Dan Malek wrote:
>
>> Michel Dänzer wrote:
>>
>>> [CC'ing linuxppc-dev, hopefully someone there knows what might be up...]
>>>
>>> This is where it dies:
>>>
>>>         /* FIXME: We should really have a kernel call for this...
>>>          */
>>>         entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
>>>                                     GFP_KERNEL,
>>>                                     PAGE_KERNEL);
>>
>> This isn't very much information, but only one thing can really
>> be wrong........
>>
>> What is the value of 'pages'?  I suspect it is huge (and perhaps
>> wrong).  The GFP_KERNEL flag will cause vmalloc() to wait for pages
>> to become available (i.e. it will swap other things out).  If this
>> value in incorrect, this call will wait forever for pages that are
>> never going to arrive.  Worse, it is going to keep sucking up pages
>> and holding them, so nothing else is going to run either.  Is "pages"
>> really the number of pages, or a size that was never converted to pages?
>
>
> Pages is definitely the number of pages (at least when I wrote the
> code...).

It is definitely the number of pages.  Also this function works
correctly on an ia32, I'll post the file where this code is executed in
case there is a simple error which the PPC people would see.

>
>
>> ...and for the 'FIXME' comment, you want a function that does
>> what? vmalloc?  Why don't you just call it (or one of the more
>> appropriate variants if necessary)?
>
>
> This was originally when I was passing a flag to make the memory
> uncached.  This wasn't needed, and we really should be using
> vmalloc_32(...) instead (which will result in exactly the same code, but
> it is cleaner).

I agree, I haven't gotten around to code cleanup on this branch yet though.

-Jeff
(The function in question is drm_sg_alloc starting on line 64.)


[-- Attachment #2: scatter.c --]
[-- Type: text/plain, Size: 5607 bytes --]

/* scatter.c -- IOCTLs to manage scatter/gather memory -*- linux-c -*-
 * Created: Mon Dec 18 23:20:54 2000 by gareth@valinux.com
 *
 * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California.
 * All Rights Reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
 * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
 * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 *
 * Authors:
 *	Gareth Hughes <gareth@valinux.com>
 *
 */

#define __NO_VERSION__
#include <linux/config.h>
#include "drmP.h"
#include <linux/wrapper.h>

#define DEBUG_SCATTER 1

static void drm_sg_cleanup( drm_sg_mem_t *entry )
{
	int i;

	for ( i = 0 ; i < entry->pages ; i++ ) {
		ClearPageReserved( entry->pagelist[i] );
	}

	vfree( entry->virtual );

	drm_free( entry->pagelist,
		  entry->pages * sizeof(*entry->pagelist),
		  DRM_MEM_PAGES );
	drm_free( entry,
		  sizeof(*entry),
		  DRM_MEM_SGLISTS );
}

static inline long usec( void )
{
   struct timeval tv;

   do_gettimeofday( &tv );

   return (tv.tv_sec & 0x7ff) * 1000000 + tv.tv_usec;
}

int drm_sg_alloc( struct inode *inode, struct file *filp,
		  unsigned int cmd, unsigned long arg )
{
	drm_file_t *priv = filp->private_data;
	drm_device_t *dev = priv->dev;
	drm_scatter_gather_t request;
	drm_sg_mem_t *entry;
	unsigned long pages;
	pgd_t *pgd;
	pmd_t *pmd;
	pte_t *pte;
	int i, j;
	DRM_INFO( "%s\n", __FUNCTION__ );

	if ( dev->sg ) return -EINVAL;

	if ( copy_from_user( &request,
			     (drm_scatter_gather_t *)arg,
			     sizeof(request) ) )
		return -EFAULT;

	entry = drm_alloc( sizeof(*entry), DRM_MEM_SGLISTS );
	if ( !entry ) return -ENOMEM;

   	memset( entry, 0, sizeof(*entry) );

	pages = (request.size + PAGE_SIZE - 1) / PAGE_SIZE;
	DRM_DEBUG( "sg size=%ld pages=%ld\n", request.size, pages );

	entry->pages = pages;
	entry->pagelist = drm_alloc( pages * sizeof(*entry->pagelist),
				     DRM_MEM_PAGES );
	if ( !entry->pagelist ) {
		drm_free( entry, sizeof(*entry), DRM_MEM_SGLISTS );
		return -ENOMEM;
	}

	/* FIXME: We should really have a kernel call for this...
	 */
	entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
				    GFP_KERNEL,
				    PAGE_KERNEL);
	if ( !entry->virtual ) {
		drm_free( entry->pagelist,
			  entry->pages * sizeof(*entry->pagelist),
			  DRM_MEM_PAGES );
		drm_free( entry,
			  sizeof(*entry),
			  DRM_MEM_SGLISTS );
		return -ENOMEM;
	}

	/* This also forces the mapping of COW pages, so our page list
	 * will be valid.  Please don't remove it...
	 */
	memset( entry->virtual, 0, pages << PAGE_SHIFT );

	entry->handle = (unsigned long)entry->virtual;

	DRM_DEBUG( "sg alloc handle  = %08lx\n", entry->handle );
	DRM_DEBUG( "sg alloc virtual = %p\n", entry->virtual );

	for ( i = entry->handle, j = 0 ; j < pages ; i += PAGE_SIZE, j++ ) {
		pgd = pgd_offset_k( i );
		pmd = pmd_offset( pgd, i );
		pte = pte_offset( pmd, i );

		entry->pagelist[j]= pte_page( *pte );
		SetPageReserved( entry->pagelist[j] );

		if ( j < 16 ) {
			DRM_DEBUG("0x%08lx (page %lu) => 0x%08lx\n",
				  i, j,
				  (unsigned long)entry->pagelist[j]->virtual);
		}
	}

	request.handle = entry->handle;

	if ( copy_to_user( (drm_scatter_gather_t *)arg,
			   &request,
			   sizeof(request) ) ) {
		drm_sg_cleanup( entry );
		return -EFAULT;
	}

	dev->sg = entry;

#ifdef DEBUG_SCATTER
	/* Verify that each page points to its virtual address, and vice
	 * versa.
	 */
	{
	int error = 0;

	for(i = 0; i < pages; i++) {
		unsigned long *tmp;

		tmp = (unsigned long *)entry->pagelist[i]->virtual;
		for(j = 0; j < PAGE_SIZE / sizeof(unsigned long); j++, tmp++) {
			*tmp = 0xcafebabe;
		}
		tmp = (unsigned long *)((u8 *)entry->virtual +
					(PAGE_SIZE * i));
		for(j = 0; j < PAGE_SIZE / sizeof(unsigned long); j++, tmp++) {
			if(*tmp != 0xcafebabe && error == 0) {
				error = 1;
				printk("Scatter allocation error, pagelist"
				       " does not match virtual mapping\n");
			}
		}
		tmp = (unsigned long *)entry->pagelist[i]->virtual;
		for(j = 0; j < PAGE_SIZE / sizeof(unsigned long); j++, tmp++) {
			*tmp = 0;
		}
	}
	if(error == 0) printk("Scatter allocation matches pagelist\n");
	}
#endif

	return 0;
}

int drm_sg_free( struct inode *inode, struct file *filp,
		 unsigned int cmd, unsigned long arg )
{
	drm_file_t *priv = filp->private_data;
	drm_device_t *dev = priv->dev;
	drm_scatter_gather_t request;
	drm_sg_mem_t *entry;

	if ( copy_from_user( &request,
			     (drm_scatter_gather_t *)arg,
			     sizeof(request) ) )
		return -EFAULT;

	entry = dev->sg;
	dev->sg = NULL;

	if ( !entry || entry->handle != request.handle ) return -EINVAL;

	DRM_INFO( "sg free virtual  = %p\n", entry->virtual );

	drm_sg_cleanup( entry );

	return 0;
}

Re: [Dri-devel] Re: PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Gareth Hughes wrote:

> Pages is definitely the number of pages (at least when I wrote the
> code...).

Hmmm....vmalloc() definitely works fine, we use it all of the
time in the kernel.  The only way it would hang is if you call
it too soon, or if the number of pages was too large.

> .....and we really should be using
> vmalloc_32(...) instead (which will result in exactly the same code, but
> it is cleaner).

It's no longer the same.  Linux has the "highmem" configuration now,
and vmalloc() allows that to be used, while vmalloc32() does not.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] Re: PPC Lockup (ati-pcigart-branch)

[Gareth Hughes]

Dan Malek wrote:
>
> Hmmm....vmalloc() definitely works fine, we use it all of the
> time in the kernel.  The only way it would hang is if you call
> it too soon, or if the number of pages was too large.

Yes.

> > .....and we really should be using
> > vmalloc_32(...) instead (which will result in exactly the same code, but
> > it is cleaner).
>
> It's no longer the same.  Linux has the "highmem" configuration now,
> and vmalloc() allows that to be used, while vmalloc32() does not.

If you look, we've just included the vmalloc_32() code inline, using
__vmalloc().  We're not using the regular highmem vmalloc().

-- Gareth

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Michel Dänzer wrote:

> [CC'ing linuxppc-dev, hopefully someone there knows what might be up...]
>
> This is where it dies:
>
>
>         /* FIXME: We should really have a kernel call for this...
>          */
>         entry->virtual = __vmalloc( (pages << PAGE_SHIFT),
>                                     GFP_KERNEL,
>                                     PAGE_KERNEL);
>
>         printk("Checkpoint 2\n");
>
>
> Checkpoint 2 is never reached, the machine is absolutely dead.
>
>
> Michel
>
>
Since your running through klogd, this might not be 'exactly' where you
die.  I would be TERRIBLY surprised if you died here.  One thing you
have to remember about kernel debugging is that you can't always trust
what is in the log.  Here is a technique that works for me when I'm
unsure of a piece of code:

Do what I suggested first, if it finds a likely problem, then great
(printk's on every line of a function).
When you get a general area where there is a failure, comment out every
line of code after that.
Line by line put the code back in.
This will help you find the hang.

This is a very common way of debugging kernel code with hangs.  There
are other alternatives.

Setup a serial console, and turn of syslogd/klogd.  This will give you
almost immediate information if you keep the output small (i.e., no
logging of lots of loop iterations or other high output stuff.)  This is
one useful tool for kernel debugging.

If through the serial console you get an Oops message, you can use
ksymoops to find out what section of code was executing right before the
Oops.  You have to do objdump usually to figure out which line of C code
caused the problem, unless the logic of the asm makes it obvious.
(i.e., I only do those types of operations on line X.)

I dunno if the kgdb patch works on powerpc, but you might try it.  You
could single step through this code since your not at a point where
interrupts are disabled.

Unfortunately kdb (built-in kernel debugger) does not work on the
powerpc.  It is REALLY useful when your debugging code when interrupts
are disabled, but it only works on ia64 and ia32 currently.  I
personally like it alot more then kgdb, even though it has some
limitations kgdb doesn't.

Hope this helps,
-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Benjamin Herrenschmidt]

>
>Since your running through klogd, this might not be 'exactly' where you
>die.  I would be TERRIBLY surprised if you died here.  One thing you
>have to remember about kernel debugging is that you can't always trust
>what is in the log.  Here is a technique that works for me when I'm
>unsure of a piece of code:

Additionally, on PPCs, you have another technique, which is to use
prom_print or xmon_printf() (the latest one support full printf semantics
while the first one only supports a simple string).

If you have compiled the kernel with early boot text support, then those
function will go to a small text rendering engine that blasts things
directly to the screen. For that to work, you must have:

 - The framebuffer reachable (depends on the state of the card)
 - The same video mode/depth current than the one you had when it booted.
If you are using an fbdev, the recent aty128fb should properly "update"
the screen informations of the early boot text engine on a mode switch.

When debugging kernel code on a G4 mac, the best thing to do is to have a
stealth serial port. This is a small device that replace the internal
modem and provides a real "Apple-SCC" serial port. With that, you can
then enable xmon to use it (arch/ppc/xmon/start.c) and benefit from the
in-kernel low level debugger and xmon_printf. Those can be used at any
time, interrupt level, etc... and rely on so few kernel resources that
they can usually still be used when the rest of the kernel is dead.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Chris Emerson]

[dri-devel removed]

On Friday, 19 Jan 2001, Benjamin Herrenschmidt wrote:

[requirements for prom_print() and xmon_printf()]
[snip]
>  - The same video mode/depth current than the one you had when it
> booted.

Does that mean the mode/depth set by OF, or the one set by the vmode
kernel cmdline option?

> If you are using an fbdev, the recent aty128fb should properly "update"
> the screen informations of the early boot text engine on a mode switch.

Ok.

> When debugging kernel code on a G4 mac, the best thing to do is to have a
> stealth serial port.

Is it possible to use xmon on a G4 (350MHz AGP) without one of these
stealth serial ports?  I don't currently do enough kernel debugging to
really make it worth getting one along with the necessary cables to
plug something into it.

I'm currently running 2.4.1pre7 rsynced from the bk tree.

Cheers,

Chris
--
Chris Emerson, obsessed Cambridge juggler
E-mail: cemerson@chiark.greenend.org.uk
Web page: http://www.chiark.greenend.org.uk/~cemerson/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Benjamin Herrenschmidt]

>
>[dri-devel removed]
>
>On Friday, 19 Jan 2001, Benjamin Herrenschmidt wrote:
>
>[requirements for prom_print() and xmon_printf()]
>[snip]
>>  - The same video mode/depth current than the one you had when it
>> booted.
>
>Does that mean the mode/depth set by OF, or the one set by the vmode
>kernel cmdline option?

The one set by OF. If the video driver supports it (that should be the
case of most drivers used on ppc with 2.4), any video mode set either via
cmdline, fbset, etc... or X if X is using fbdev's.

>Is it possible to use xmon on a G4 (350MHz AGP) without one of these
>stealth serial ports?  I don't currently do enough kernel debugging to
>really make it worth getting one along with the necessary cables to
>plug something into it.

With a cable from the G4's modem to another modem, and some hackery
(already present in arch/ppc/xmon/start.c, I suggest you look at that
file), it's possible but tricky.

>I'm currently running 2.4.1pre7 rsynced from the bk tree.

pre8 is already there ;)

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Chris Emerson]

On Friday, 19 Jan 2001, Benjamin Herrenschmidt wrote:

> >Does that mean the mode/depth set by OF, or the one set by the
> >vmode kernel cmdline option?
>
> The one set by OF. If the video driver supports it (that should be
> the case of most drivers used on ppc with 2.4), any video mode set
> either via cmdline, fbset, etc... or X if X is using fbdev's.

Ok, I think I meet the criteria then.  I'll do some experimentation.

> With a cable from the G4's modem to another modem, and some hackery
> (already present in arch/ppc/xmon/start.c, I suggest you look at that
> file), it's possible but tricky.

Ok.  Is there a fundamental reason which prevents it being possible to
use the USB keyboard, or just hard work?

> >I'm currently running 2.4.1pre7 rsynced from the bk tree.
>
> pre8 is already there ;)

Well, unless there are any vital fixes, I'll hold off for a few days.
:-)

Cheers,

Chris
--
Chris Emerson, obsessed Cambridge juggler
E-mail: cemerson@chiark.greenend.org.uk
Web page: http://www.chiark.greenend.org.uk/~cemerson/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Benjamin Herrenschmidt]

>Ok.  Is there a fundamental reason which prevents it being possible to
>use the USB keyboard, or just hard work?
>
>> >I'm currently running 2.4.1pre7 rsynced from the bk tree.
>>
>> pre8 is already there ;)
>
>Well, unless there are any vital fixes, I'll hold off for a few days.
>:-)

Very hard work indeed :) You'd have to write a mecanism to let most of
the OHCI controller code to run in a polled way from within the
debugger... quite nasty.

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michael Schmitz]

> Very hard work indeed :) You'd have to write a mecanism to let most of
> the OHCI controller code to run in a polled way from within the
> debugger... quite nasty.

Why? How's that different from ADB keyboards? It just won't work when
interrupts are dead ;-)

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Benjamin Herrenschmidt]

>> Very hard work indeed :) You'd have to write a mecanism to let most of
>> the OHCI controller code to run in a polled way from within the
>> debugger... quite nasty.
>
>Why? How's that different from ADB keyboards? It just won't work when
>interrupts are dead ;-)

xmon can poll the keyboard without any interrupts on Cuda & PMU machines
by sending the appropriate Cuda or PMU command to the chip and doing
polled IOs. On USB, it's also possible to poll the interrupt status
register of the OHCI chip. But it's a lot more painful since you have to
run most of the USB stack this way.

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michael Schmitz]

> >> Very hard work indeed :) You'd have to write a mecanism to let most of
> >> the OHCI controller code to run in a polled way from within the
> >> debugger... quite nasty.
> >
> >Why? How's that different from ADB keyboards? It just won't work when
> >interrupts are dead ;-)
>
> xmon can poll the keyboard without any interrupts on Cuda & PMU machines
> by sending the appropriate Cuda or PMU command to the chip and doing
> polled IOs. On USB, it's also possible to poll the interrupt status

Yuck. Thanks for the clarification ... I always thought ADB was too messy
for this but should have checked the source...

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Benjamin Herrenschmidt wrote:
>
> >
> >Since your running through klogd, this might not be 'exactly' where you
> >die.  I would be TERRIBLY surprised if you died here.  One thing you
> >have to remember about kernel debugging is that you can't always trust
> >what is in the log.  Here is a technique that works for me when I'm
> >unsure of a piece of code:
>
> Additionally, on PPCs, you have another technique, which is to use
> prom_print or xmon_printf() (the latest one support full printf semantics
> while the first one only supports a simple string).

xmon might really be helpful, but the crash is after the X server blanks the
display, so I wonder if xmon output would be visible?


Thanks to Jeff and everyone for the tips on how to debug this BTW.

Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Michel Dänzer wrote:

> xmon might really be helpful, but the crash is after the X server blanks the
> display, so I wonder if xmon output would be visible?

Doesn't look like. I realized that when I thought the machine seems dead, it
is in fact at the xmon prompt, but the display is black. Hitting 'x' makes it
continue.

I've narrowed down the problem by modifying the code like this:

        for ( i = entry->handle, j = 0 ; j < pages ; i += PAGE_SIZE, j++ ) {
                printk("i: %08lx\n", i);
                pgd = pgd_offset_k( i );
                printk("pgd: %08lx\n", pgd);
                pmd = pmd_offset( pgd, i );
                printk("pmd: %08lx\n", pmd);
                pte = pte_offset( pmd, i );
                printk("pte: %08lx\n", pte);

                if (!pte)
                {
                        printk("D'oh!\n");
                        return -ENOMEM;
                }

                entry->pagelist[j]= pte_page( *pte );
                printk("Checkpoint 5\n");
                SetPageReserved( entry->pagelist[j] );
                printk("Checkpoint 6\n");

                if ( j < 16 ) {
                        DRM_DEBUG("0x%08lx (page %lu) => 0x%08lx\n",
                                  i, j,
                                  (unsigned long)entry->pagelist[j]->virtual);
                }
        }


The kernel output is as follows:

[drm] drm_sg_alloc
i: ca292000
pgd: c014dca0
pmd: c014dca0
pte: 00000a48
Oops: kernel access of bad area, sig: 11
NIP: C981EE58 XER: 20000000 LR: C981EE50 SP: C5857DD0 REGS: c5857d20 TRAP:
0300
MSR: 00009032 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 11
DAR: 00000A48, DSISR: 40000000
TASK = c5856000[227] 'XFree86' Last syscall: 54
last math c784e000 last altivec 00000000
GPR00: C981EE50 C5857DD0 C5856000 0000000E 00001032 00000001 C0190000 00000000
GPR08: 00000001 C0150000 00000000 C5857D10 20822824 101D5EAC 00000000 C01503DC
GPR16: C0190000 C9820000 C9820000 C9825BD4 C5857DE0 C9820000 7FFFFA58 00000800
GPR24: 00000000 C5EA1C40 00000000 C014E000 00000000 00000A48 00000A48 CA292000
Call backtrace:
C981EE50 C980ECD0 C004D7DC C000411C 00000007 1005449C 10320D8C
106226EC 106231A8 1061D474 1008D098 10025B68 1008C5B4 0FE6FCC8
00000000


So it's not the __vmalloc indeed. I hope this rings any bells...


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Come on, someone has to know what's going on here... :)

For the Linux/PPC developers: the purpose of this code is to walk a virtual
memory range and look up the page struct for each page. I spent some time on
the train this morning trying to figure out what might be wrong but couldn't
find anything obvious.

> I've narrowed down the problem by modifying the code like this:
>
>         for ( i = entry->handle, j = 0 ; j < pages ; i += PAGE_SIZE, j++ ) {
>                 printk("i: %08lx\n", i);
>                 pgd = pgd_offset_k( i );
>                 printk("pgd: %08lx\n", pgd);
>                 pmd = pmd_offset( pgd, i );
>                 printk("pmd: %08lx\n", pmd);
>                 pte = pte_offset( pmd, i );
>                 printk("pte: %08lx\n", pte);
>
>                 entry->pagelist[j]= pte_page( *pte );
>                 printk("Checkpoint 5\n");
>                 SetPageReserved( entry->pagelist[j] );
>                 printk("Checkpoint 6\n");
>
>                 if ( j < 16 ) {
>                         DRM_DEBUG("0x%08lx (page %lu) => 0x%08lx\n",
>                                   i, j,
>                                   (unsigned
> long)entry->pagelist[j]->virtual);
>                 }
>         }

[...]

> [drm] drm_sg_alloc
> i: ca292000
> pgd: c014dca0
> pmd: c014dca0
> pte: 00000a48
> Oops: kernel access of bad area, sig: 11

Looking at pgd/pmd, pte seems fishy for a pointer. Any reason why this code
shouldn't work on PPC?


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Michel Dänzer wrote:

> > [drm] drm_sg_alloc
> > i: ca292000
> > pgd: c014dca0
> > pmd: c014dca0
> > pte: 00000a48
> > Oops: kernel access of bad area, sig: 11
>
> Looking at pgd/pmd, pte seems fishy for a pointer. Any reason why this code
> shouldn't work on PPC?

There is no pte table and so there is nothing mapped at that address, you
can use (pgd|pmd|pte)_present() to test for that.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Roman Zippel wrote:

> > > [drm] drm_sg_alloc
> > > i: ca292000
> > > pgd: c014dca0
> > > pmd: c014dca0
> > > pte: 00000a48
> > > Oops: kernel access of bad area, sig: 11
> >
> > Looking at pgd/pmd, pte seems fishy for a pointer. Any reason why this
> > code shouldn't work on PPC?
>
> There is no pte table and so there is nothing mapped at that address, you
> can use (pgd|pmd|pte)_present() to test for that.

You are saying we should use *_present() to check each of these before using
*_offset to get the pointer? And what if *_present() is false?


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Michel Dänzer wrote:

> > There is no pte table and so there is nothing mapped at that address, you
> > can use (pgd|pmd|pte)_present() to test for that.
>
> You are saying we should use *_present() to check each of these before using
> *_offset to get the pointer? And what if *_present() is false?

It depends what you what you want to do. If you just want to do a reverse
lookup, yes, you have to check the value with *_present.
But you can also map something at that address, then you had to use
*_alloc(), but leave that better to ioremap/vmalloc. (Look for map_page in
arch/ppc/mm/init.c for how it can be done.)

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

...I don't know who wrote:
>
> > > There is no pte table and so there is nothing mapped at that address,

No, there is a pte table there, you just didn't get to printing
anything from it......

Roman Zipple wrote:

> It depends what you what you want to do.

I am really bad at guessing today....someone is going to have to
tell me what you are trying to do.  From the few lines of code I have
see posted, you are doing:

	1. Allocating some kernel virtual address and backing that
		with pages in real memory.

	2. Trying to find something in the kernel page tables, that I
		guessed wrong was a physical address of the pages.

Then, Jeff mentioned something about mapping user space pages, but
virt_to_bus/bus_to_virt aren't going to do that on PowerPC. I'm working
on some functions that will, but they aren't there for all systems yet.


> But you can also map something at that address, then you had to use
> *_alloc(), but leave that better to ioremap/vmalloc. (Look for map_page in
> arch/ppc/mm/init.c for how it can be done.)

Err...ahhhh...I don't know if I would go looking in that file for
examples.  I would prefer to understand the problem we are trying to
solve, and perhaps write some functions to call if necessary.  Scattering
code from functions in this file into other places may not be a good
thing.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Dan Malek wrote:
>
> ...I don't know who wrote:
> >
> > > > There is no pte table and so there is nothing mapped at that address,
>
> No, there is a pte table there, you just didn't get to printing
> anything from it......

Why? Why does pte_offset yield a bogus pointer?


> > It depends what you what you want to do.
>
> I am really bad at guessing today....someone is going to have to
> tell me what you are trying to do.  From the few lines of code I have
> see posted, you are doing:
>
>         1. Allocating some kernel virtual address and backing that
>                 with pages in real memory.
>
>         2. Trying to find something in the kernel page tables, that I
>                 guessed wrong was a physical address of the pages.

We want the struct page for each page of the allocated virtual region. The
code apparently works on i386 (and alpha?).


> Then, Jeff mentioned something about mapping user space pages, but
> virt_to_bus/bus_to_virt aren't going to do that on PowerPC. I'm working
> on some functions that will, but they aren't there for all systems yet.

Hmm, hopefully this doesn't keep the PCI GART from working on PPC...


I've tried catching NULL pointers and *_present(), still the same problem. But
I'm not sure if I used them correctly.


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Dan Malek wrote:

> ...I don't know who wrote:
>
>>>> There is no pte table and so there is nothing mapped at that address,
>>>
>
> No, there is a pte table there, you just didn't get to printing
> anything from it......
>
> Roman Zipple wrote:
>
>> It depends what you what you want to do.
>
>
> I am really bad at guessing today....someone is going to have to
> tell me what you are trying to do.  From the few lines of code I have
> see posted, you are doing:
>
> 	1. Allocating some kernel virtual address and backing that
> 		with pages in real memory.
>
> 	2. Trying to find something in the kernel page tables, that I
> 		guessed wrong was a physical address of the pages.


Okay let me try and explain things a little better.

We need a virtually contiguous area of memory addressable by the
kernel.  We also need to know what physical pages actually make up this
virtually contiguous area.  Currently there is no kernel function to do
this explicitly (I'm probably going to add one to 2.5 for some other
work I'm doing though.)  So we call vmalloc_32 and get a virtual
address.  We then take that virtual address and grab the list pages that
the kernel just allocated.  We memset this full region in case these are
COW pages (shouldn't ever be COW pages on the Intel ia32, but perhaps on
other arch's.)  We get the list by walking the kernel page tables, as
you can see in the code snippet.

Later on this list of pages will be mapped into user space by the DRM's
mmap routine.  It uses vma_nopage to accomplish this.  vma_nopage
functions should return struct page *, we do this by returning
pagelist[page_offset_in_pages_from_start_of_vma].  This should work on
fine on any arch's kernel.  We haven't gotten to this point in the code
yet, so its not the issue we are addressing.

Another thing that happens later is that we need the bus address of each
of these pages to program the card to do scatter gather dma from this
region.  We do virt_to_bus(pagelist[i]->virtual) to accomplish this
translation.  I assume this is what we would have to do on PPC to
program a device wanting u32 bus pointers on the PPC.  We aren't even
reaching that code yet, so its not the issue here.

I know on the ia32 a pgd/pmd can actually point to 4MB pages rather then
a real pte.  Does the PowerPC have anything like this?  I would doubt
that I would encounter anything like this from a vmalloc'ed area of
memory (since vmalloc is arch independent and it would call alloc_page
for each individual pte.)  Am I correct in this assumption?

Just FYI, the code I posted works fine on the ia32 platform (only tested
with the i386 classic 2-level page tables.)

Another thing we might be running into here is that vmalloc does not
guarantee a virtually contiguous area of memory (or so I am told.)  I've
NEVER seen this in practice on an ia32 platform.  Does this happen only
happen on other platforms, or perhaps happen more often on other platforms?

-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Jeff Hartmann wrote:

> Later on this list of pages will be mapped into user space by the DRM's
> mmap routine.  It uses vma_nopage to accomplish this.  vma_nopage
> functions should return struct page *, we do this by returning
> pagelist[page_offset_in_pages_from_start_of_vma].  This should work on
> fine on any arch's kernel.  We haven't gotten to this point in the code
> yet, so its not the issue we are addressing.

Do you make the page private already?

> Another thing that happens later is that we need the bus address of each
> of these pages to program the card to do scatter gather dma from this
> region.  We do virt_to_bus(pagelist[i]->virtual) to accomplish this
> translation.

Better use page_address() to get the virtual address and then use
pci_map_single() or pci_map_sg() to get a dma address.

> I know on the ia32 a pgd/pmd can actually point to 4MB pages rather then
> a real pte.  Does the PowerPC have anything like this?  I would doubt
> that I would encounter anything like this from a vmalloc'ed area of
> memory (since vmalloc is arch independent and it would call alloc_page
> for each individual pte.)  Am I correct in this assumption?

ppc doesn't have page table, it either uses a hash table or the tlb
entries have to be filled by software. Anyway, pgd/pmd/pte are used to
keep the management compatible to Linux and are like ia32 also two levels.

> Another thing we might be running into here is that vmalloc does not
> guarantee a virtually contiguous area of memory (or so I am told.)  I've
> NEVER seen this in practice on an ia32 platform.  Does this happen only
> happen on other platforms, or perhaps happen more often on other platforms?

Someone was confused :). vmalloc can't guarantee that it allocates a
physical contiguous area.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Frank Rowand]

Roman Zippel wrote:
>
> Hi,
>
> On Mon, 22 Jan 2001, Jeff Hartmann wrote:
>

>
> ppc doesn't have page table, it either uses a hash table or the tlb
> entries have to be filled by software. Anyway, pgd/pmd/pte are used to
> keep the management compatible to Linux and are like ia32 also two levels.

The 4xx implementation does have a page table (and doesn't have a hash table).

-Frank
--
Frank Rowand <frank_rowand@mvista.com>
MontaVista Software, Inc

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Jeff Hartmann wrote:

> Okay let me try and explain things a little better.

Got it.  That is what I was guessing, only surprised you are holding
page structs, but that makes sense.

> ......  Currently there is no kernel function to do
> this explicitly

I'm working on that.  The PowerPC port cheated by using BATs and
trivial macros, but this doesn't work on some of the newer processors
and more complex applications.  Other architectures did the same, and
I am surprised there aren't generic kernel functions to track down this
information.  In fact, these functions are already present for 4xx and
8xx processors, so don't write anything new.

> Another thing that happens later is that we need the bus address of each
> of these pages to program the card to do scatter gather dma from this
> region.

That's where this is going to fall apart on PowerPC.

> .....  We do virt_to_bus(pagelist[i]->virtual) to accomplish this
> translation.

I have to write some code (or actually remove some #ifdefs) before
this will work for you.

> I know on the ia32 a pgd/pmd can actually point to 4MB pages rather then
> a real pte.  Does the PowerPC have anything like this?

Not yet.  It's on the way....

> ....  I would doubt
> that I would encounter anything like this from a vmalloc'ed area of
> memory (since vmalloc is arch independent and it would call alloc_page
> for each individual pte.)  Am I correct in this assumption?

Yes.

> Just FYI, the code I posted works fine on the ia32 platform (only tested
> with the i386 classic 2-level page tables.)

What you are doing so far should work too on PowerPC.

> Another thing we might be running into here is that vmalloc does not
> guarantee a virtually contiguous area of memory (or so I am told.)

Ummm...of course it is virtually contiguous.  How could it be
different?  You request a size, and it returns a base virtual address.
If there were holes in it, how would you know?

> .....  I've
> NEVER seen this in practice on an ia32 platform.

It can't happen on any platform (or I don't understand something about
the comment, which could very well be the case today).



	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Dan Malek wrote:

> Jeff Hartmann wrote:
>
>> Okay let me try and explain things a little better.
>
>
> Got it.  That is what I was guessing, only surprised you are holding
> page structs, but that makes sense.
>
>> ......  Currently there is no kernel function to do
>> this explicitly
>
>
> I'm working on that.  The PowerPC port cheated by using BATs and
> trivial macros, but this doesn't work on some of the newer processors
> and more complex applications.  Other architectures did the same, and
> I am surprised there aren't generic kernel functions to track down this
> information.  In fact, these functions are already present for 4xx and
> 8xx processors, so don't write anything new.
>
>> Another thing that happens later is that we need the bus address of each
>> of these pages to program the card to do scatter gather dma from this
>> region.
>
>
> That's where this is going to fall apart on PowerPC.
>
>> .....  We do virt_to_bus(pagelist[i]->virtual) to accomplish this
>> translation.
>
>
> I have to write some code (or actually remove some #ifdefs) before
> this will work for you.
>
>> I know on the ia32 a pgd/pmd can actually point to 4MB pages rather then
>> a real pte.  Does the PowerPC have anything like this?
>
>
> Not yet.  It's on the way....
>
>> ....  I would doubt
>> that I would encounter anything like this from a vmalloc'ed area of
>> memory (since vmalloc is arch independent and it would call alloc_page
>> for each individual pte.)  Am I correct in this assumption?
>
>
> Yes.
>
>> Just FYI, the code I posted works fine on the ia32 platform (only tested
>> with the i386 classic 2-level page tables.)
>
>
> What you are doing so far should work too on PowerPC.
>
>> Another thing we might be running into here is that vmalloc does not
>> guarantee a virtually contiguous area of memory (or so I am told.)
>
>
> Ummm...of course it is virtually contiguous.  How could it be
> different?  You request a size, and it returns a base virtual address.
> If there were holes in it, how would you know?

Look at vread in vmalloc.c, I think it would handle holes in a
vmalloc'ed area (From a brief reading of the code.)  I've seen postings
about this on linux-kernel.  I don't see a vwrite implementation, but I
would assume you would have to do something similar for writes.

>
>
>> .....  I've
>> NEVER seen this in practice on an ia32 platform.
>
>
> It can't happen on any platform (or I don't understand something about
> the comment, which could very well be the case today).

I think it can, I've seen numerous people talk about it on
linux-kernel.  I've also been told that my /dev/agpgart isn't 'safe'
because it assumes vmalloc'ed memory is always virtually contiguous.  I
think this only happens when there isn't enough virtual address space in
the kernel and its fragmented (probably only happens on machines with
lots of memory.)

-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Jeff Hartmann wrote:

> > Ummm...of course it is virtually contiguous.  How could it be
> > different?  You request a size, and it returns a base virtual address.
> > If there were holes in it, how would you know?
>
> Look at vread in vmalloc.c, I think it would handle holes in a
> vmalloc'ed area (From a brief reading of the code.)  I've seen postings
> about this on linux-kernel.  I don't see a vwrite implementation, but I
> would assume you would have to do something similar for writes.

Between two vmalloced areas is of course a hole to catch overflow errors.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Mike Beede]

Jeff Hartmann wrote:
>
> > Dan Malek wrote:
> > Ummm...of course it is virtually contiguous.  How could it be
> > different?  You request a size, and it returns a base virtual address.
> > If there were holes in it, how would you know?
>
> Look at vread in vmalloc.c, I think it would handle holes in a
> vmalloc'ed area (From a brief reading of the code.)  I've seen postings
> about this on linux-kernel.  I don't see a vwrite implementation, but I
> would assume you would have to do something similar for writes.

Separate calls to vmalloc can return noncontiguous regions, but the
memory
from a single call is contiguous.  Unless you mean *physically*
contiguous,
which isn't guaranteed at all. (I don't think you do--just thought I'd
be
clear).  After all, what good would it do to allocate multiple regions
and
return a pointer to the first?

	Mike
--
Mike Beede
Cisco Systems

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Dan Malek wrote:

> > ......  Currently there is no kernel function to do
> > this explicitly
>
> I'm working on that.  The PowerPC port cheated by using BATs and
> trivial macros, but this doesn't work on some of the newer processors
> and more complex applications.  Other architectures did the same, and
> I am surprised there aren't generic kernel functions to track down this
> information.  In fact, these functions are already present for 4xx and
> 8xx processors, so don't write anything new.

AFAIK such tricks are used for mapping normal (low) memory and ioremapped
areas. For normal memory you can use phys_to_virt()/virt_to_phys() and for
ioremapped memory, you have to store the physical and virtual address
yourself. What am I missing?

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Roman Zippel wrote:

> AFAIK such tricks are used for mapping normal (low) memory and ioremapped
> areas. For normal memory you can use phys_to_virt()/virt_to_phys() and for
> ioremapped memory, you have to store the physical and virtual address
> yourself. What am I missing?

The PowerPC port, except for a couple of processors (4xx and 8xx again)
could never use virt_to_phys on any dynamically allocated (vmalloc()'ed)
space.  All of the drivers/devices that used these macros were using
ioremapped() or static buffers.  Perhaps the PCI mapping functions are
implementing some of this, and I need to take a look.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Takashi Oe]

On 1/22/01 6:24 PM, Dan Malek wrote:

> The PowerPC port, except for a couple of processors (4xx and 8xx again)
> could never use virt_to_phys on any dynamically allocated (vmalloc()'ed)
> space.  All of the drivers/devices that used these macros were using
> ioremapped() or static buffers.  Perhaps the PCI mapping functions are
> implementing some of this, and I need to take a look.

Is it really true that virt_to_phys on vmalloc'd memory is broken 6xx?  I
wonder why planb works at all....


Takashi Oe


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Takashi Oe wrote:

> Is it really true that virt_to_phys on vmalloc'd memory is broken 6xx?

Yep.  The virt_to_phys (except for APUS), only does address - KERNELBASE.
I posted a message about this a few days ago during my "mmu cleanup"
while merging new code.  I have discovered that architectures are
implementing private versions of functions/macros for things like
this that are all slightly different.  There is no sense to this,
as there should be generic Linux functions for many more memory
management functions (and cache management, and dma management,...).

> .....  I
> wonder why planb works at all....

Probably because no one stumbles across the memory it is trashing?
Currently, bad_thing_will_happen = vmalloc + virt_to_bus + dma.
It could be with the right memory size, modulo addressing, memory
controller configuration, timing of the vmalloc, it just may
accidently work.  If this is the case, I would be out buying
lottery tickets........


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Ralph Metzler]

Dan Malek writes:
 > Takashi Oe wrote:
 >
 > > Is it really true that virt_to_phys on vmalloc'd memory is broken 6xx?
 >
 > Yep.  The virt_to_phys (except for APUS), only does address - KERNELBASE.
 > I posted a message about this a few days ago during my "mmu cleanup"
 > while merging new code.  I have discovered that architectures are
 > implementing private versions of functions/macros for things like
 > this that are all slightly different.  There is no sense to this,
 > as there should be generic Linux functions for many more memory
 > management functions (and cache management, and dma management,...).
 >
 > > .....  I
 > > wonder why planb works at all....
 >
 > Probably because no one stumbles across the memory it is trashing?
 > Currently, bad_thing_will_happen = vmalloc + virt_to_bus + dma.
 > It could be with the right memory size, modulo addressing, memory
 > controller configuration, timing of the vmalloc, it just may
 > accidently work.  If this is the case, I would be out buying
 > lottery tickets........


A few years back I wasted a lot of hours going through the kernel mm
functions to come to that same conclusion. The conversion routines I
implemented for my Bt848 driver (bttv) already got copied in all kinds
of other frame grabber drivers. Other drivers implemented their
own versions of this.
In the meantime they got modified and improved by others. I
don't know how machine independent they are now. But I guess if the Bt848
driver works on your platform the routines in:

linux/drivers/media/video/bttv_driver.c

in the lines following:

/*******************************/
/* Memory management functions */
/*******************************/

should work for you.

There really should be standard kernel functions for all of this, or
are there now?


Ralph


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Takashi Oe]

On Mon, 22 Jan 2001, Dan Malek wrote:

> > .....  I
> > wonder why planb works at all....
>
> Probably because no one stumbles across the memory it is trashing?
> Currently, bad_thing_will_happen = vmalloc + virt_to_bus + dma.
> It could be with the right memory size, modulo addressing, memory
> controller configuration, timing of the vmalloc, it just may
> accidently work.  If this is the case, I would be out buying
> lottery tickets........

Ok, my apologies, planb doesn't use vmalloc at all these days.  So, there
is no problem of that kind.  (In the beginning, it had vmalloc with a lot
of problems as I recall now.)


Takashi Oe


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Dan Malek wrote:

> The PowerPC port, except for a couple of processors (4xx and 8xx again)
> could never use virt_to_phys on any dynamically allocated (vmalloc()'ed)
> space.  All of the drivers/devices that used these macros were using
> ioremapped() or static buffers.  Perhaps the PCI mapping functions are
> implementing some of this, and I need to take a look.

virt_to_phys is only guaranteed to work on normal kernel memory and on
nothing else. Drivers that use that function will not work on certain
architectures. With the new pci mapping you have the same problem as with
ioremapped memory, one has to store both pointers somewhere in the driver.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Frank Rowand]

Roman Zippel wrote:
>
> Hi,
>
> On Mon, 22 Jan 2001, Dan Malek wrote:
>
> > > ......  Currently there is no kernel function to do
> > > this explicitly
> >
> > I'm working on that.  The PowerPC port cheated by using BATs and
> > trivial macros, but this doesn't work on some of the newer processors
> > and more complex applications.  Other architectures did the same, and
> > I am surprised there aren't generic kernel functions to track down this
> > information.  In fact, these functions are already present for 4xx and
> > 8xx processors, so don't write anything new.
>
> AFAIK such tricks are used for mapping normal (low) memory and ioremapped
> areas. For normal memory you can use phys_to_virt()/virt_to_phys() and for
> ioremapped memory, you have to store the physical and virtual address
> yourself. What am I missing?
>
> bye, Roman

Take a look at Documentation/DMA-mapping.txt.

I've done something for the 405 that I'm especially wary of, but have been
hoping I can continue to get away with in the future.  I map all of physical
SDRAM to the kernel virtual address space:

  0xc000'0000 through (0xc000'0000 + size of memory - 1)

via large TLB entries (each entry typically covering a range of 8MB).

This means that I can't just change the TLB entry of a 4KB page if I want it
to be mapped uncached.  I instead allocate a new virtual address range and
map that single page as uncachable via the new virtual address.  Thus phys_to_virt()
incorrectly returns the cacheable virtual address instead of the uncacheable virtual
address.

Currently, my biggest user of this method is pci_alloc_consistent().  This use is
because the 405 is not IO cache coherent.  In this case, the "physical" address
is really a bus address, which pci_alloc_consistent calls "dma_handle".
DMA-mapping.txt says drivers should not use bus_to_virt(), which is a close
relative of phys_to_virt().

-Frank
--
Frank Rowand <frank_rowand@mvista.com>
MontaVista Software, Inc

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Frank Rowand]

Frank Rowand wrote:
>


> hoping I can continue to get away with in the future.  I map all of physical
> SDRAM to the kernel virtual address space:
>
>   0xc000'0000 through (0xc000'0000 + size of memory - 1)
>
> via large TLB entries (each entry typically covering a range of 8MB).

oops, trivial mis-speak.  Each large entry typically covering a range of 16MB.

-Frank
--
Frank Rowand <frank_rowand@mvista.com>
MontaVista Software, Inc

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Frank Rowand wrote:

> Currently, my biggest user of this method is pci_alloc_consistent().  This use is
> because the 405 is not IO cache coherent.  In this case, the "physical" address
> is really a bus address, which pci_alloc_consistent calls "dma_handle".
> DMA-mapping.txt says drivers should not use bus_to_virt(), which is a close
> relative of phys_to_virt().

That's true, if you look into include/asm-sparc64/io.h, you see that
bus_to_virt()/virt_to_bus() are not defined anymore.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Dan Malek wrote:

> ...I don't know who wrote:

(That was I too :). )

> > > > There is no pte table and so there is nothing mapped at that address,
>
> No, there is a pte table there, you just didn't get to printing
> anything from it......

Nope, there is no pte table, otherwise pte had been a valid pointer into
that table, so it's just the offset for that table.

> Err...ahhhh...I don't know if I would go looking in that file for
> examples.  I would prefer to understand the problem we are trying to
> solve, and perhaps write some functions to call if necessary.  Scattering
> code from functions in this file into other places may not be a good
> thing.

Sure, one missing information is, how he got that address.
Anyway, looking up virtual kernel memory that way is possible, with
ioremapped memory it already is dangerous, but for that you also don't get
any page struct.

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Roman Zippel wrote:

> Hi,
>
> On Mon, 22 Jan 2001, Dan Malek wrote:
>
>> ...I don't know who wrote:
>
>
> (That was I too :). )
>
>>>>> There is no pte table and so there is nothing mapped at that address,
>>>>
>> No, there is a pte table there, you just didn't get to printing
>> anything from it......
>
>
> Nope, there is no pte table, otherwise pte had been a valid pointer into
> that table, so it's just the offset for that table.
>
>> Err...ahhhh...I don't know if I would go looking in that file for
>> examples.  I would prefer to understand the problem we are trying to
>> solve, and perhaps write some functions to call if necessary.  Scattering
>> code from functions in this file into other places may not be a good
>> thing.
>
>
> Sure, one missing information is, how he got that address.

We take the value returned from vmalloc_32 and walk the kernel page
tables from that address.

>
> Anyway, looking up virtual kernel memory that way is possible, with
> ioremapped memory it already is dangerous, but for that you also don't get
> any page struct.
>
> bye, Roman
>
Doing this for ioremapped memory would be EVIL.  If your using an
ioremapped area of memory, you should know the addresses it refers too.
I suppose you could get them by walking the page tables, but that's just
icky.  The only reason we are walking the page tables is there is no
kernel call which gives us a list of pages and a virtually contiguous
address.

-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Michel Dänzer wrote:

>                 entry->pagelist[j]= pte_page( *pte );


Bzzzzt....you lose :-).

The pte_page returns the kernel's page_struct for a physical
memory page, not the physical address which you are after......
You ignored the compiler warning here, didn't you :-).

Just do this:

	entry->pagelist[j] = (unsigned long)(pte_val(*pte)) & PAGE_MASK;



	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Dan Malek wrote:

> Michel Dänzer wrote:
>
>>                 entry->pagelist[j]= pte_page( *pte );
>
>
>
> Bzzzzt....you lose :-).
>
> The pte_page returns the kernel's page_struct for a physical
> memory page, not the physical address which you are after......
> You ignored the compiler warning here, didn't you :-).
>
> Just do this:
>
> 	entry->pagelist[j] = (unsigned long)(pte_val(*pte)) & PAGE_MASK;
>
>
>
> 	-- Dan

entry->pagelist is of type struct page *.

-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Gareth Hughes]

Dan Malek wrote:
>
> Michel Dänzer wrote:
>
> >                 entry->pagelist[j]= pte_page( *pte );
>
> Bzzzzt....you lose :-).
>
> The pte_page returns the kernel's page_struct for a physical
> memory page, not the physical address which you are after......
> You ignored the compiler warning here, didn't you :-).
>
> Just do this:
>
>         entry->pagelist[j] = (unsigned long)(pte_val(*pte)) & PAGE_MASK;

Umm, entry->pagelist should contain a list of "struct page *"s, not
physical addresses.  I believe your code does not do this...

-- Gareth

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Dan Malek wrote:
>
> Michel Dänzer wrote:
>
> >                 entry->pagelist[j]= pte_page( *pte );
>
> Bzzzzt....you lose :-).
>
> The pte_page returns the kernel's page_struct for a physical
> memory page, not the physical address which you are after......
> You ignored the compiler warning here, didn't you :-).

No, that's not the problem, pagelist is struct page ** :-/ (and pagelist is
allocated)

BTW I tried to replace this construct with entry->pagelist[j]= virt_to_page(i)
and it ran through this loop, but died even worse later (pagelist[j]->virtual
!= i). Why doesn't that work?


Seems I have a lot to learn...

Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Michel Dänzer wrote:

> No, that's not the problem, pagelist is struct page ** :-/ (and pagelist is
> allocated)

Then, what the heck were you trying to print out with that
piece of code, and who further down the food chain uses
the page struct?


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Jeff Hartmann]

Dan Malek wrote:

> Michel Dänzer wrote:
>
>> No, that's not the problem, pagelist is struct page ** :-/ (and pagelist is
>> allocated)
>
>
> Then, what the heck were you trying to print out with that
> piece of code, and who further down the food chain uses
> the page struct?
>
>
> 	-- Dan

Some VM code later on uses the page struct to return the correct value
from vma_nopage.  This code supports userland mappings, not just kernel
mappings.  We could of course just use phys_to_page/virt_to_page in the
vma_nopage code, but struct page works just fine.

-Jeff


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michel Dänzer]

Dan Malek wrote:
>
> Michel Dänzer wrote:
>
> > No, that's not the problem, pagelist is struct page ** :-/ (and pagelist
> > is allocated)
>
> Then, what the heck were you trying to print out with that
> piece of code,

The printk's are only for debugging purposes ;)

> and who further down the food chain uses the page struct?

The DRM.


I think Roman's suggestion is very promising; There's something very
interesting in arch/ppc/mm/fault.c .


Michel


--
Earthling Michel Dänzer (MrCooper)    \   Debian GNU/Linux (powerpc) developer
CS student, Free Software enthusiast   \        XFree86 and DRI project member

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

OK, since I was stupid and couldn't see the forest for the trees,
let's take some giant steps backward and try again.....

Michel Dänzer wrote:

> I've narrowed down the problem by modifying the code like this:
>
>         for ( i = entry->handle, j = 0 ; j < pages ; i += PAGE_SIZE, j++ ) {
>                 printk("i: %08lx\n", i);
>                 pgd = pgd_offset_k( i );
>                 printk("pgd: %08lx\n", pgd);
>                 pmd = pmd_offset( pgd, i );
>                 printk("pmd: %08lx\n", pmd);
>                 pte = pte_offset( pmd, i );
>                 printk("pte: %08lx\n", pte);


I do this:

		pgd = pgd_offset_k(i);
		if (pgd) {
			pmd = pmd_offset(pgd, i);
			if (pmd && pmd_present(*pmd)) {
				pte = pte_offset(pmd, i);
				if (pte && pte_present(*pte)) {
					/* Do your stuff */
				}
			}
		}

> The kernel output is as follows:
>
> [drm] drm_sg_alloc
> i: ca292000
> pgd: c014dca0
> pmd: c014dca0
> pte: 00000a48

Using my code above, the 'if (pmd && pmd_present(*pmd))' would have
been false (the pmd_present would have failed).  Now, the confusing
part is if entry->handle is the result of a vmalloc this shouldn't
happen......So, Roman was right, no pte table.......

What kernel and system are you using?  Where in the course of
system operation (driver init, application open(), etc.) are you
running this code?

I apologize for my confusion.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Roman Zippel]

Hi,

On Mon, 22 Jan 2001, Dan Malek wrote:

> I do this:
>
> 		pgd = pgd_offset_k(i);
> 		if (pgd) {
> 			pmd = pmd_offset(pgd, i);
> 			if (pmd && pmd_present(*pmd)) {
> 				pte = pte_offset(pmd, i);
> 				if (pte && pte_present(*pte)) {
> 					/* Do your stuff */
> 				}
> 			}
> 		}

There is no need to test the pointers and you forgot a pgd_present(),
otherwise you might get a problem on architectures with 3-level mmus.
Anyway to get to the page struct, this should do it:

        pgd = pgd_offset_k(addr);
        if (!pgd_present(*pgd))
                return NULL;
        pmd = pmd_offset(pgd, addr);
        if (!pmd_present(*pmd))
                return NULL;
        pte = pte_offset(pmd, addr);
        if (!pte_present(*pte))
                return NULL;
        return pte_page(*pte);

bye, Roman


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Paul Mackerras]

Michel =?iso-8859-1?Q?D=81=E4nzer?= writes:

> I've narrowed down the problem by modifying the code like this:
>
>         for ( i = entry->handle, j = 0 ; j < pages ; i += PAGE_SIZE, j++ ) {
>                 printk("i: %08lx\n", i);
>                 pgd = pgd_offset_k( i );
>                 printk("pgd: %08lx\n", pgd);
>                 pmd = pmd_offset( pgd, i );
>                 printk("pmd: %08lx\n", pmd);
>                 pte = pte_offset( pmd, i );
>                 printk("pte: %08lx\n", pte);
[snip]
> The kernel output is as follows:
>
> [drm] drm_sg_alloc
> i: ca292000
> pgd: c014dca0
> pmd: c014dca0
> pte: 00000a48

Which means that *pgd == 0, which is very odd indeed, unless of course
you have run off the end of the vmalloc'd area (I assume you haven't).
What is the value of the symbol swapper_pg_dir in this kernel?
Have you tried accessing the vmalloc'd area?  Where did you get the
kernel source?

I hope to get to have a good look at the drm code myself shortly.

Paul.

--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare.  Support for the revolution.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Dan Malek]

Paul Mackerras wrote:

> Which means that *pgd == 0, which is very odd indeed, unless of course
> you have run off the end of the vmalloc'd area (I assume you haven't).

Right, I made the same assumption, this was the first time through
the loop.

> What is the value of the symbol swapper_pg_dir in this kernel?

It looks like swapper_pg_dir because of it's low memory address.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Michael Schmitz]

> > Additionally, on PPCs, you have another technique, which is to use
> > prom_print or xmon_printf() (the latest one support full printf semantics
> > while the first one only supports a simple string).
>
> xmon might really be helpful, but the crash is after the X server blanks the
> display, so I wonder if xmon output would be visible?

IIRC xmon output did show up on the serial console (similar X related
debugging last year on the Lombard).

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: [Dri-devel] PPC Lockup (ati-pcigart-branch)

[Wolfgang Denk]

In message <3A686DFE.9030308@valinux.com> Jeff Hartmann wrote:
>
> I dunno if the kgdb patch works on powerpc, but you might try it.  You
> could single step through this code since your not at a point where
> interrupts are disabled.
>
> Unfortunately kdb (built-in kernel debugger) does not work on the
> powerpc.  It is REALLY useful when your debugging code when interrupts

OTOH on many PPC systems you can just hook up a BDM / JTAG  debugger,
which  provides  a  tool  which  is even more powerful (since you can
freeze the whole CPU, including all internal timers etc.).

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
Only in our dreams we are free. The rest of the time we neeed  wages.
                                    - Terry Pratchett, _Wyrd Sisters_

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/