amd64-agp vs. swsusp

amd64-agp vs. swsusp

[Michal Schmidt]

Hello,

Does resuming from swsuspend work for anyone with amd64-agp loaded?

On my system when I suspend with amd64-agp loaded, I get a spontaneous 
reboot on resume. It reboots immediately after reading the saved image 
from disk.
This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

Regards,
Michal

Re: amd64-agp vs. swsusp

[Andreas Steinmetz]

Michal Schmidt wrote:
> Hello,
> 
> Does resuming from swsuspend work for anyone with amd64-agp loaded?
> 
> On my system when I suspend with amd64-agp loaded, I get a spontaneous
> reboot on resume. It reboots immediately after reading the saved image
> from disk.
> This is 100% reproducible.
> 
> Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

AMD Athlon(tm) 64 Processor 3000+, Acer Aspire

Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64
unknown unknown GNU/Linux

CONFIG_AGP=y
CONFIG_AGP_AMD64=y

swsusp works for me. Could it be mm, agp as a module or some speciality
of your hardware?
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: amd64-agp vs. swsusp

[Michal Schmidt]

Andreas Steinmetz wrote:
> Michal Schmidt wrote:
>>Does resuming from swsuspend work for anyone with amd64-agp loaded?
>>
>>On my system when I suspend with amd64-agp loaded, I get a spontaneous
>>reboot on resume. It reboots immediately after reading the saved image
>>from disk.
>>This is 100% reproducible.
>>
>>Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.
> 
> 
> AMD Athlon(tm) 64 Processor 3000+, Acer Aspire
> 
> Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64
> unknown unknown GNU/Linux
> 
> CONFIG_AGP=y
> CONFIG_AGP_AMD64=y
> 
> swsusp works for me. Could it be mm, agp as a module or some speciality
                                        ^^^^^^^^^^^^^^^
                                 That seems to be the problem!
> of your hardware?

I have rebuilt agpgart and amd64-agp into the kernel and now it has 
resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Michal

Re: amd64-agp vs. swsusp

[Rafael J. Wysocki]

Hi,

On Tuesday, 19 of July 2005 23:26, Michal Schmidt wrote:
> Andreas Steinmetz wrote:
> > Michal Schmidt wrote:
> >>Does resuming from swsuspend work for anyone with amd64-agp loaded?
> >>
> >>On my system when I suspend with amd64-agp loaded, I get a spontaneous
> >>reboot on resume. It reboots immediately after reading the saved image
> >>from disk.
> >>This is 100% reproducible.
> >>
> >>Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.
> > 
> > 
> > AMD Athlon(tm) 64 Processor 3000+, Acer Aspire
> > 
> > Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64
> > unknown unknown GNU/Linux
> > 
> > CONFIG_AGP=y
> > CONFIG_AGP_AMD64=y
> > 
> > swsusp works for me. Could it be mm, agp as a module or some speciality
>                                         ^^^^^^^^^^^^^^^
>                                  That seems to be the problem!
> > of your hardware?
> 
> I have rebuilt agpgart and amd64-agp into the kernel and now it has 
> resumed successfully for the first time. Thank you for the hint!
> 
> But I still wonder, why that makes a difference.

Before resume the module is not present.  When it gets loaded from the
image it probably runs with the assumption that the hardware was initialized
which is not correct.

Greets,
Rafael


-- 
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"

Re: amd64-agp vs. swsusp

[Michal Schmidt]

Rafael J. Wysocki wrote:
> On Tuesday, 19 of July 2005 23:26, Michal Schmidt wrote:
>>I have rebuilt agpgart and amd64-agp into the kernel and now it has 
>>resumed successfully for the first time. Thank you for the hint!
>>
>>But I still wonder, why that makes a difference.
> 
> 
> Before resume the module is not present.  When it gets loaded from the
> image it probably runs with the assumption that the hardware was initialized
> which is not correct.

It seems that the module doesn't even get a chance to run after resume. 
I've put some printks and udelays into kernel/power/swsusp.c and other 
places and I've found that the spontaneous reset occurs already in 
swsusp_arch_resume(), ie. before the drivers get their resume methods 
called. This is what I have in swsusp_suspend() now:
	...
         save_processor_state();
         if ((error = swsusp_arch_suspend()))
                 printk(KERN_ERR "Error %d suspending\n", error);
         /* Restore control flow magically appears here */
         restore_processor_state();
         printk(KERN_INFO "processor state restored!\n");/*I added this*/
         BUG_ON (nr_copy_pages_check != nr_copy_pages);
         restore_highmem();
         device_power_up();
	...

I'm recording the screen during resuming with a digital camera to see if 
the added printk is displayed before the reset and I am now sure that 
the reset occurs before that. The last thing I see is:

Stopping tasks: --|
Freeing memory... done (0 pages freed)
swsusp: Need to copy 8121 pages

Then on the next frame of the recorded MPEG, the display is already 
beginning to dim as the computer is resetting.

I also tried putting a printk before restore_processor_state(), but I'm 
not sure if it is safe to use printk there.
So I tried putting a loop of 5000 x udelay(1000) there to see if the 
reset would be delayed by 5s. It was not delayed, so I think that the 
reset occurs before restore_processor_state().

Michal

Re: amd64-agp vs. swsusp

[Rafael J. Wysocki]

On Thursday, 21 of July 2005 00:07, Michal Schmidt wrote:
> Rafael J. Wysocki wrote:
> > On Tuesday, 19 of July 2005 23:26, Michal Schmidt wrote:
> >>I have rebuilt agpgart and amd64-agp into the kernel and now it has 
> >>resumed successfully for the first time. Thank you for the hint!
> >>
> >>But I still wonder, why that makes a difference.
> > 
> > 
> > Before resume the module is not present.  When it gets loaded from the
> > image it probably runs with the assumption that the hardware was initialized
> > which is not correct.
> 
> It seems that the module doesn't even get a chance to run after resume. 
> I've put some printks and udelays into kernel/power/swsusp.c and other 
> places and I've found that the spontaneous reset occurs already in 
> swsusp_arch_resume(), ie. before the drivers get their resume methods 
> called. This is what I have in swsusp_suspend() now:
> 	...
>          save_processor_state();
>          if ((error = swsusp_arch_suspend()))
>                  printk(KERN_ERR "Error %d suspending\n", error);
>          /* Restore control flow magically appears here */
>          restore_processor_state();
>          printk(KERN_INFO "processor state restored!\n");/*I added this*/
>          BUG_ON (nr_copy_pages_check != nr_copy_pages);
>          restore_highmem();
>          device_power_up();
> 	...
> 
> I'm recording the screen during resuming with a digital camera to see if 
> the added printk is displayed before the reset and I am now sure that 
> the reset occurs before that. The last thing I see is:
> 
> Stopping tasks: --|
> Freeing memory... done (0 pages freed)
> swsusp: Need to copy 8121 pages

Please note that printk() only places the string in a buffer and it does not
get actually printed before it can be displayed which is probably after your
screen is set up (including the AGP resume, I'd guess). :-)

You may be able to get more from eg. the serial console, but this requires
some tampering with the serial code, AFAIR.

> Then on the next frame of the recorded MPEG, the display is already 
> beginning to dim as the computer is resetting.
> 
> I also tried putting a printk before restore_processor_state(), but I'm 
> not sure if it is safe to use printk there.

Yes, it is, but you may be unable to see the message if the box reboots before
it can be displayed.

Greets,
Rafael


-- 
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"

Re: amd64-agp vs. swsusp

[Michal Schmidt]

Rafael J. Wysocki wrote:
> On Thursday, 21 of July 2005 00:07, Michal Schmidt wrote:
>>I also tried putting a printk before restore_processor_state(), but I'm 
>>not sure if it is safe to use printk there.
> 
> 
> Yes, it is, but you may be unable to see the message if the box reboots before
> it can be displayed.

OK, but then I also tried putting a 5s long busy wait there and the 
reset was not delayed. Therefore, the reset must be occurring before 
restore_processor_state().
Or is there a reason why
	for(i=0; i<5000; i++)
		udelay(1000);
wouldn't work as expected?

Michal

Re: amd64-agp vs. swsusp

[Pavel Machek]

Hi!

> >>I have rebuilt agpgart and amd64-agp into the kernel and now it has 
> >>resumed successfully for the first time. Thank you for the hint!
> >>
> >>But I still wonder, why that makes a difference.
> >
> >
> >Before resume the module is not present.  When it gets loaded from the
> >image it probably runs with the assumption that the hardware was 
> >initialized
> >which is not correct.
> 
> It seems that the module doesn't even get a chance to run after resume. 
> I've put some printks and udelays into kernel/power/swsusp.c and other 
> places and I've found that the spontaneous reset occurs already in 
> swsusp_arch_resume(), ie. before the drivers get their resume methods 
> called. This is what I have in swsusp_suspend() now:
> 	...
>         save_processor_state();
>         if ((error = swsusp_arch_suspend()))
>                 printk(KERN_ERR "Error %d suspending\n", error);
>         /* Restore control flow magically appears here */
>         restore_processor_state();
>         printk(KERN_INFO "processor state restored!\n");/*I added this*/
>         BUG_ON (nr_copy_pages_check != nr_copy_pages);
>         restore_highmem();
>         device_power_up();
> 	...
> 
> I'm recording the screen during resuming with a digital camera to see if 
> the added printk is displayed before the reset and I am now sure that 
> the reset occurs before that. The last thing I see is:
> 
> Stopping tasks: --|
> Freeing memory... done (0 pages freed)
> swsusp: Need to copy 8121 pages
> 
> Then on the next frame of the recorded MPEG, the display is already 
> beginning to dim as the computer is resetting.
> 
> I also tried putting a printk before restore_processor_state(), but I'm 
> not sure if it is safe to use printk there.
> So I tried putting a loop of 5000 x udelay(1000) there to see if the 
> reset would be delayed by 5s. It was not delayed, so I think that the 
> reset occurs before restore_processor_state().

Long time ago there were i386 problems because we assumed that kernel
is mapped in one big mapping and agp broke that assumption. Copying
pages backwards "fixed" it (and then we done proper fix). It should
not be, but it seems similar to this problem....

								Pavel
-- 
Boycott Kodak -- for their patent abuse against Java.

Re: amd64-agp vs. swsusp

[Rafael J. Wysocki]

On Thursday, 21 of July 2005 03:25, Michal Schmidt wrote:
> Rafael J. Wysocki wrote:
> > On Thursday, 21 of July 2005 00:07, Michal Schmidt wrote:
> >>I also tried putting a printk before restore_processor_state(), but I'm 
> >>not sure if it is safe to use printk there.
> > 
> > 
> > Yes, it is, but you may be unable to see the message if the box reboots before
> > it can be displayed.
> 
> OK, but then I also tried putting a 5s long busy wait there and the 
> reset was not delayed. Therefore, the reset must be occurring before 
> restore_processor_state().
> Or is there a reason why
> 	for(i=0; i<5000; i++)
> 		udelay(1000);
> wouldn't work as expected?

No, it should work, AFAIK.  Then it looks like something gets copied into a wrong
place or in a wrong order while restoring the image.

Greets,
Rafael


-- 
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"

Re: amd64-agp vs. swsusp

[Michal Schmidt]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

Pavel Machek wrote:
> Long time ago there were i386 problems because we assumed that kernel
> is mapped in one big mapping and agp broke that assumption. Copying
> pages backwards "fixed" it (and then we done proper fix). It should
> not be, but it seems similar to this problem....

Do you mean this patch of yours?:
http://www.ussg.iu.edu/hypermail/linux/kernel/0404.3/0640.html

I'm trying to do something similar for x86_64. See the attached patch.
Unfortunately, it doesn't help. The behaviour seems unchanged (resume 
still works iff amd64-agp wasn't loaded before suspend).

Michal

[-- Attachment #2: swsusp-amd64-pgdir.diff --]
[-- Type: text/x-patch, Size: 1497 bytes --]

diff -Nurp -X dontdiff.new linux-mm/arch/x86_64/kernel/suspend_asm.S linux-mm.mich/arch/x86_64/kernel/suspend_asm.S
--- linux-mm/arch/x86_64/kernel/suspend_asm.S	2005-06-30 01:00:53.000000000 +0200
+++ linux-mm.mich/arch/x86_64/kernel/suspend_asm.S	2005-07-21 11:53:17.000000000 +0200
@@ -41,7 +41,7 @@ ENTRY(swsusp_arch_suspend)
 
 ENTRY(swsusp_arch_resume)
 	/* set up cr3 */	
-	leaq	init_level4_pgt(%rip),%rax
+	leaq	swsusp_level4_pgt(%rip),%rax
 	subq	$__START_KERNEL_map,%rax
 	movq	%rax,%cr3
 
diff -Nurp -X dontdiff.new linux-mm/arch/x86_64/mm/init.c linux-mm.mich/arch/x86_64/mm/init.c
--- linux-mm/arch/x86_64/mm/init.c	2005-07-18 19:48:12.000000000 +0200
+++ linux-mm.mich/arch/x86_64/mm/init.c	2005-07-21 11:21:36.000000000 +0200
@@ -310,10 +310,32 @@ void __init init_memory_mapping(unsigned
 
 extern struct x8664_pda cpu_pda[NR_CPUS];
 
+#ifdef CONFIG_SOFTWARE_SUSPEND
+/*
+ * Swap suspend & friends need this for resume because things like the intel-agp
+ * driver might have split up a kernel 4MB mapping.
+ */
+char __nosavedata swsusp_level4_pgt[PAGE_SIZE]
+	__attribute__ ((aligned (PAGE_SIZE)));
+
+static inline void save_pg_dir(void)
+{
+	memcpy(swsusp_level4_pgt, init_level4_pgt, PAGE_SIZE);
+}
+#else
+static inline void save_pg_dir(void)
+{
+}
+#endif
+
 /* Assumes all CPUs still execute in init_mm */
 void zap_low_mappings(void)
 {
-	pgd_t *pgd = pgd_offset_k(0UL);
+	pgd_t *pgd;
+
+	save_pg_dir();
+
+	pgd = pgd_offset_k(0UL);
 	pgd_clear(pgd);
 	flush_tlb_all();
 }

Re: amd64-agp vs. swsusp

[Pavel Machek]

Hi!

> >Long time ago there were i386 problems because we assumed that kernel
> >is mapped in one big mapping and agp broke that assumption. Copying
> >pages backwards "fixed" it (and then we done proper fix). It should
> >not be, but it seems similar to this problem....
> 
> Do you mean this patch of yours?:
> http://www.ussg.iu.edu/hypermail/linux/kernel/0404.3/0640.html

Yes.

> I'm trying to do something similar for x86_64. See the attached patch.
> Unfortunately, it doesn't help. The behaviour seems unchanged (resume 
> still works iff amd64-agp wasn't loaded before suspend).

Are you sure problem is on level4_pgt? We probably use constant
level4_pgt but split pages at some deeper level. You may want try
saving 3rd-level table, instead.
								Pavel
-- 
Boycott Kodak -- for their patent abuse against Java.

Re: amd64-agp vs. swsusp

[Michal Schmidt]

Pavel Machek wrote:
>>I'm trying to do something similar for x86_64. See the attached patch.
>>Unfortunately, it doesn't help. The behaviour seems unchanged (resume 
>>still works iff amd64-agp wasn't loaded before suspend).
> 
> 
> Are you sure problem is on level4_pgt? We probably use constant
> level4_pgt but split pages at some deeper level. You may want try
> saving 3rd-level table, instead.

I'm not sure about that at all. That was just my attempt of cargocult 
programming :-)
OK, I'll try saving the 3rd-level table. It'll take me some time to 
figure out how to do that, however :-)

Michal

Re: amd64-agp vs. swsusp

[Rafael J. Wysocki]

On Thursday, 21 of July 2005 17:24, Michal Schmidt wrote:
> Pavel Machek wrote:
> >>I'm trying to do something similar for x86_64. See the attached patch.
> >>Unfortunately, it doesn't help. The behaviour seems unchanged (resume 
> >>still works iff amd64-agp wasn't loaded before suspend).
> > 
> > 
> > Are you sure problem is on level4_pgt? We probably use constant
> > level4_pgt but split pages at some deeper level. You may want try
> > saving 3rd-level table, instead.
> 
> I'm not sure about that at all. That was just my attempt of cargocult 
> programming :-)
> OK, I'll try saving the 3rd-level table. It'll take me some time to 
> figure out how to do that, however :-)

I think the amd64-agp is the problem here.  There are some memory mappings
that seem to require the hardware to be initialized before they can be used
safely (at least as far as I understand it).

Greets,
Rafael


-- 
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"

Re: amd64-agp vs. swsusp

[Andrew Morton]

Michal Schmidt <xschmi00@stud.feec.vutbr.cz> wrote:
>
> Hello,
> 
> Does resuming from swsuspend work for anyone with amd64-agp loaded?

It would seem not ;)

> On my system when I suspend with amd64-agp loaded, I get a spontaneous 
> reboot on resume. It reboots immediately after reading the saved image 
> from disk.
> This is 100% reproducible.
> 
> Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

If this is reproducible in the soon-to-be-released 2.6.13-rc6 then could
you please raise an entry at bugzilla.kernel.org and we'll plug away at it
as the suspend stuff continues to get sorted out, thanks.

Re: amd64-agp vs. swsusp

[Pavel Machek]

Hi!

> > Does resuming from swsuspend work for anyone with amd64-agp loaded?
> 
> It would seem not ;)
> 
> > On my system when I suspend with amd64-agp loaded, I get a spontaneous 
> > reboot on resume. It reboots immediately after reading the saved image 
> > from disk.
> > This is 100% reproducible.
> > 
> > Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.
> 
> If this is reproducible in the soon-to-be-released 2.6.13-rc6 then could
> you please raise an entry at bugzilla.kernel.org and we'll plug away at it
> as the suspend stuff continues to get sorted out, thanks.

I assume it is in -rc6, too; it is long-standing bug and I am not
aware of any attempts at fixing it. Please file bug report, assign to
me.

OTOH it is not regression; AFAIK it never worked. Please do not let it
block 2.6.13.

								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

Re: amd64-agp vs. swsusp

[Cal Peake]

On Thu, 4 Aug 2005, Andrew Morton wrote:

> Michal Schmidt <xschmi00@stud.feec.vutbr.cz> wrote:
> >
> > Does resuming from swsuspend work for anyone with amd64-agp loaded?
> 
> It would seem not ;)

Must have missed the OP. Yes I can resume fine from swsusp with amd64-agp.

System is an Averatec 3270 (Mobile Sempron(K8)).

Not that this helps at all other than confirming it is possible ;)

-cp

-- 
"There are three kinds of lies: lies, damned lies, and statistics."
                     -- Benjamin Disraeli

Re: amd64-agp vs. swsusp

[Andreas Steinmetz]

Cal Peake wrote:
> On Thu, 4 Aug 2005, Andrew Morton wrote:
> 
> 
>>Michal Schmidt <xschmi00@stud.feec.vutbr.cz> wrote:
>>
>>>Does resuming from swsuspend work for anyone with amd64-agp loaded?
>>
>>It would seem not ;)
> 
> 
> Must have missed the OP. Yes I can resume fine from swsusp with amd64-agp.
> 
> System is an Averatec 3270 (Mobile Sempron(K8)).
> 
> Not that this helps at all other than confirming it is possible ;)
> 
> -cp
> 

AFAIK it works when agp is built into the kernel. You will get problems
when it is built as a module. In the module case you may want to try if
loading the module before resuming helps.

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: amd64-agp vs. swsusp

[Cal Peake]

On Fri, 5 Aug 2005, Andreas Steinmetz wrote:

> AFAIK it works when agp is built into the kernel. You will get problems
> when it is built as a module. In the module case you may want to try if
> loading the module before resuming helps.

Strange. I've had it built as a module from day one and never had a 
problem resuming.

-cp

-- 
"There are three kinds of lies: lies, damned lies, and statistics."
                     -- Benjamin Disraeli

Re: amd64-agp vs. swsusp

[Andreas Steinmetz]

Cal Peake wrote:
> On Fri, 5 Aug 2005, Andreas Steinmetz wrote:
> 
> 
>>AFAIK it works when agp is built into the kernel. You will get problems
>>when it is built as a module. In the module case you may want to try if
>>loading the module before resuming helps.
> 
> 
> Strange. I've had it built as a module from day one and never had a 
> problem resuming.

I guess it depends on what the BIOS does.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

Re: amd64-agp vs. swsusp

[Michal Schmidt]

Pavel Machek wrote:
> I assume it is in -rc6, too; it is long-standing bug and I am not
> aware of any attempts at fixing it. Please file bug report, assign to
> me.

I've filed it as Bug 5018.
Michal