All of lore.kernel.org
 help / color / mirror / Atom feed
* Help? Red Hat fails, Suse/Debian both work fine
@ 2006-02-28 21:45 Magenheimer, Dan (HP Labs Fort Collins)
  2006-02-28 21:51 ` Keir Fraser
  2006-03-03 11:02 ` [PATCH] Re: [Xen-ia64-devel] " Kouya SHIMURA
  0 siblings, 2 replies; 11+ messages in thread
From: Magenheimer, Dan (HP Labs Fort Collins) @ 2006-02-28 21:45 UTC (permalink / raw)
  To: xen-devel; +Cc: yo.fujita, Akio Takebe, xen-ia64-devel

Hi all --

We are seeing a strange problem where a recent cset causes
Red Hat to fail domU boot on ia64 complaining of a hotplug
problem but doesn't cause any problem for Suse or Debian.
It is likely some subtle difference (or bug), perhaps in mmap?
Suggestions/advice from anyone more familiar with distro
differences (on ia64) would be appreciated.

Changeset is xen-unstable 8783 ("Use /dev/kmem to map dom0
xenstore page instead of abusing the foreign mapping interface.",
Feb 8, committed by Christian).  Backing out this changeset
or using the small patch below causes the problem to go away,
so we have a workaround, but a root cause would be nice
to know and fix.

Full thread of discussion can be found here:
http://lists.xensource.com/archives/html/xen-ia64-devel/2006-02/msg00104
.html

Thanks,
Dan

> -----Original Message-----
> From: Akio Takebe [mailto:takebe_akio@jp.fujitsu.com] 
> Sent: Tuesday, February 28, 2006 2:47 AM
> To: Magenheimer, Dan (HP Labs Fort Collins); yo.fujita; 
> xen-ia64-devel@lists.xensource.com
> Cc: Akio Takebe
> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [2/3rd week]
> 
> Hi, Dan
> 
> I'm still debuging, but it is very difficult...
> Much advice is welcome. :-)
> 
> Now I can boot domU by using the following patch.
> 
> diff -r 6c43118bdba8 tools/xenstore/xenstored_domain.c
> --- a/tools/xenstore/xenstored_domain.c Fri Feb 24 15:41:08 2006 -0700
> +++ b/tools/xenstore/xenstored_domain.c Tue Feb 28 18:20:16 2006 +0900
> @@ -467,6 +467,7 @@ static int dom0_init(void)
>         int rc, fd;
>         evtchn_port_t port; 
>         unsigned long kva; 
> +       unsigned long mfn; 
>         char str[20]; 
>         struct domain *dom0; 
>  
> @@ -500,9 +501,16 @@ static int dom0_init(void)
>         if (fd == -1)
>                 return -1;
>  
> -       dom0->interface = mmap(NULL, getpagesize(), 
> PROT_READ|PROT_WRITE,
> -                              MAP_SHARED, fd, kva);
> -       if (dom0->interface == MAP_FAILED)
> +       mfn=((0x0fffffffffffffff & kva) >>14);
> +/*
> +        dom0->interface = mmap(NULL, getpagesize(), 
> PROT_READ|PROT_WRITE,
> +                               MAP_SHARED, fd, kva);
> +*/
> +       dom0->interface = xc_map_foreign_range(
> +               *xc_handle, 0,
> +               getpagesize(), PROT_READ|PROT_WRITE, mfn);
> +       if (!dom0->interface) 
> +//     if (dom0->interface == MAP_FAILED)
>                 goto outfd;
>  
>         close(fd);
> 
> Best Regards,
> 
> Akio Takebe
> 
> >Hi Akio --
> >
> >Any more progress on this issue?  If you are stuck,
> >maybe we should post the problem to xen-devel to
> >see if we can get help from a Red Hat person (since
> >the problem doesn't occur on Suse or Debian).
> >
> >Thanks,
> >Dan 
> >
> >> -----Original Message-----
> >> From: Akio Takebe [mailto:takebe_akio@jp.fujitsu.com] 
> >> Sent: Thursday, February 23, 2006 8:45 PM
> >> To: Magenheimer, Dan (HP Labs Fort Collins); yo.fujita; 
> >> xen-ia64-devel@lists.xensource.com
> >> Cc: Akio Takebe
> >> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [2/3rd week]
> >> 
> >> Hi, Dan and Alex
> >> 
> >> I think this issue is only on ia64.
> >> I seem that kmem_map@drivers/char/mem.c is used on ia64, 
> >> but mem_map@drivers/xen/char/mem.c is used on x86.
> >> So I think pfn or kva aren't set correctly.
> >> We tried to boot domU with revesing cset xen-ia64-ustable.8790 
> >> and it was good work.
> >> 
> >> I'm still debugging it. :-<
> >> 
> >> Best Regards,
> >> 
> >> Akio Takebe
> >> 
> >> >Confirmed cset xen-unstable 8783 fails while 8782 succeeds.
> >> >
> >> >Perhaps there's something different about mmap on RH
> >> >vs Suse and Debian?  Perhaps only on ia64?
> >> >
> >> 
> >> 
> >> 
> 
> 
> 

^ permalink raw reply	[flat|nested] 11+ messages in thread
* RE: [PATCH] Re: [Xen-ia64-devel] Help? Red Hat fails, Suse/Debian both work fine
@ 2006-03-03 12:08 Tian, Kevin
  0 siblings, 0 replies; 11+ messages in thread
From: Tian, Kevin @ 2006-03-03 12:08 UTC (permalink / raw)
  To: Kouya SHIMURA, Magenheimer, Dan (HP Labs Fort Collins)
  Cc: xen-devel, xen-ia64-devel

>From: Kouya SHIMURA
>Sent: 2006年3月3日 19:02
>Hi all
>
>Finally I found out the unstable problem about domU boot on ia64.
>Please see the comment in patch.
>
>This patch is one solution.
>Another solution is to ask the memory attributes to
>hypervisor but it looks excessive modification.
>
>Signed-off-by Kouya SHIMURA <kouya@jp.fujitsu.com>
>
>Best Regards,
>Kouya

Hi, Kouya,

Good finding for the cause, however several minor comments about your 
patch:
	- Be conservative when you want to modify common linux files, if 
without strong reasons.

	- Your change has potential issue. Your change actually makes all 
illegal addresses which is out of EFI memmap can get WB attribute, only
for accommodating another special page (store page) out of domU's 
knowledge. I don't think it's right way to go.

	- See how x86 works. All special pages including store page, console 
page, start info, etc. are constructed within domU's memory, and domU 
is aware of that. For XEN/IA64, you may want to steal last several pages 
from configured memory of domU. There's an ia64 specific field named 
sys_pgnr (in xc_linux_build), which was targeted for this issue before 
however incomplete yet. Then you can pass to Xen and add one reserved 
EFI memmap entry to cover sys_pgnr pages with attribute WB at 
dom_fw_init. Since it's reserved, domU won't use it. However 
efi_mem_attributes can give ideal result you want. This workaround has 
no impact to xenlinux code, which may be quick solution. Of course, we 
may consider reconstruct more to make it cleaner in the future. ;-)

Thanks,
Kevin

^ permalink raw reply	[flat|nested] 11+ messages in thread
* RE: [PATCH] Re: [Xen-ia64-devel] Help? Red Hat fails, Suse/Debian both work fine
@ 2006-03-03 12:26 Tian, Kevin
  2006-03-06  1:06 ` Akio Takebe
  0 siblings, 1 reply; 11+ messages in thread
From: Tian, Kevin @ 2006-03-03 12:26 UTC (permalink / raw)
  To: Tian, Kevin, Kouya SHIMURA,
	Magenheimer, Dan (HP Labs Fort Collins)
  Cc: xen-devel, xen-ia64-devel

After thinking more, I still think the cause that drivers/xen/char/mem.c is 
not compiled into xen0 kernel image (for xen/ia64 only). See 
uncached_access() within that file which is simple and different as original 
linux one:

static inline int uncached_access(struct file *file)
{
        if (file->f_flags & O_SYNC)
                return 1;
        /* Xen sets correct MTRR type on non-RAM for us. */
        return 0;
}

By this way, xenstore page of domU will be always presented as WB to 
dom0, and then memory attribute will become correct.

Akio told me that he seemed to make an incomplete experiment to include 
xen style kmem. Could you confirm whether following quick change can 
work for you: (Sorry for not patch style, due to no access to my box now:-)

1.	Modify drivers/xen/Makefile by adding:
		obj-y   += char/

2.	Then modify drivers/char/mem.c by define ARCH_HAS_DEV_MEM at 
the very start. (Just for quick experiment to ensure correct code included. 
If it works, please move the definition to appropriate header file)

Thanks,
Kevin
>-----Original Message-----
>From: xen-devel-bounces@lists.xensource.com
>[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tian, Kevin
>Sent: 2006年3月3日 20:08
>To: Kouya SHIMURA; Magenheimer, Dan (HP Labs Fort Collins)
>Cc: xen-devel; xen-ia64-devel@lists.xensource.com
>Subject: RE: [Xen-devel] [PATCH] Re: [Xen-ia64-devel] Help? Red Hat
>fails,Suse/Debian both work fine
>
>>From: Kouya SHIMURA
>>Sent: 2006年3月3日 19:02
>>Hi all
>>
>>Finally I found out the unstable problem about domU boot on ia64.
>>Please see the comment in patch.
>>
>>This patch is one solution.
>>Another solution is to ask the memory attributes to
>>hypervisor but it looks excessive modification.
>>
>>Signed-off-by Kouya SHIMURA <kouya@jp.fujitsu.com>
>>
>>Best Regards,
>>Kouya
>
>Hi, Kouya,
>
>Good finding for the cause, however several minor comments about your
>patch:
>	- Be conservative when you want to modify common linux files, if
>without strong reasons.
>
>	- Your change has potential issue. Your change actually makes all
>illegal addresses which is out of EFI memmap can get WB attribute, only
>for accommodating another special page (store page) out of domU's
>knowledge. I don't think it's right way to go.
>
>	- See how x86 works. All special pages including store page, console
>page, start info, etc. are constructed within domU's memory, and domU
>is aware of that. For XEN/IA64, you may want to steal last several pages
>from configured memory of domU. There's an ia64 specific field named
>sys_pgnr (in xc_linux_build), which was targeted for this issue before
>however incomplete yet. Then you can pass to Xen and add one reserved
>EFI memmap entry to cover sys_pgnr pages with attribute WB at
>dom_fw_init. Since it's reserved, domU won't use it. However
>efi_mem_attributes can give ideal result you want. This workaround has
>no impact to xenlinux code, which may be quick solution. Of course, we
>may consider reconstruct more to make it cleaner in the future. ;-)
>
>Thanks,
>Kevin
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2006-03-06  2:35 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-02-28 21:45 Help? Red Hat fails, Suse/Debian both work fine Magenheimer, Dan (HP Labs Fort Collins)
2006-02-28 21:51 ` Keir Fraser
2006-03-03 11:02 ` [PATCH] Re: [Xen-ia64-devel] " Kouya SHIMURA
2006-03-03 13:00   ` Keir Fraser
2006-03-03 16:56   ` Keir Fraser
2006-03-06  0:18     ` Kouya SHIMURA
2006-03-06  2:29       ` Kouya SHIMURA
2006-03-06  2:35         ` Alex Williamson
  -- strict thread matches above, loose matches on Subject: below --
2006-03-03 12:08 Tian, Kevin
2006-03-03 12:26 Tian, Kevin
2006-03-06  1:06 ` Akio Takebe

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.