Re: Re: Linux Stubdom Problem

[Jiageng Yu <yujiageng734@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 8773 bytes --]

2011/9/15 Stefano Stabellini <stefano.stabellini@eu.citrix.com>:
> On Wed, 14 Sep 2011, Jiageng Yu wrote:
>> Hi Stefano,
>>
>>      I just have a prototype of vram mapping and test it now. The
>> implementation of linux-stubdom kernel part is as follows.
>> xen_remap_domain_mfn_range2 function maps foreign dom's physical
>> address into linux kernel space. It is similar to
>> xen_remap_domain_mfn_range. But xen_remap_domain_mfn_range is used to
>> map foreign pages into linux user space.
>>
>>     But the page info seems wrong after executing xen_remap_domain_mfn_range2.
>>
>>     struct page *page=pfn_to_page(vmalloc_to_pfn(info->fb));
>>
>>     The page->_count = 0xc2c2c2c2. It is very strange.
>>
>>     Did I do the right thing?
>>
>
> use page_address instead of pfn_to_page to find the struct page
>
>
>>     Greeting.
>>
>> Jiageng Yu.
>>
>>
>> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
>> index 204e3ba..72a7808 100644
>> --- a/arch/x86/xen/mmu.c
>> +++ b/arch/x86/xen/mmu.c
>> @@ -2693,6 +2693,73 @@ out:
>>  }
>>  EXPORT_SYMBOL_GPL(xen_remap_domain_mfn_range);
>>
>> +int xen_remap_domain_mfn_range2(unsigned long addr,unsigned long gpfn,
>> +                int nr, unsigned domid)
>> +{
>> +    struct remap_data rmd;
>> +    struct mmu_update mmu_update[REMAP_BATCH_SIZE];
>> +    int level,i,batch,nr_page = nr;
>> +    unsigned long range;
>> +    int err = 0;
>> +    unsigned long vaddr,base_addr = addr;
>> +    pte_t pte,*ptep;
>> +
>> +    rmd.mfn = gpfn;
>> +    rmd.prot = __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |
>> _PAGE_IOMAP);
>> +
>> +    while(nr_page) {
>> +        batch = min(REMAP_BATCH_SIZE, nr);
>> +        range = (unsigned long)batch << PAGE_SHIFT;
>> +
>> +        rmd.mmu_update = mmu_update;
>> +
>> +        for(i=0; i < batch; i++){
>> +            pte = pte_mkspecial(pfn_pte(rmd.mfn++, rmd.prot));
>> +            vaddr = base_addr + i*PAGE_SIZE;
>> +            ptep = lookup_address(vaddr, &level);
>
> you need to check if ptep is valid here and the level is PG_LEVEL_4K
>
>> +            rmd.mmu_update->ptr = arbitrary_virt_to_machine(ptep).maddr |
>> +                                        MMU_NORMAL_PT_UPDATE;
>
> you can use pte_mfn(*ptep) instead of arbitrary_virt_to_machine
>
>
>> +            rmd.mmu_update->val = pte_val_ma(pte);
>> +            rmd.mmu_update++;
>> +        }
>> +
>> +        err = -EFAULT;
>> +        if(HYPERVISOR_mmu_update(mmu_update, batch, NULL, domid) < 0)
>> +            goto out;
>> +
>> +        nr_page -= batch;
>> +        base_addr += range;
>> +    }
>> +
>> +    err = 0;
>> +
>> +    base_addr = addr;
>> +    for(i=0; i < nr; i++){
>> +        vaddr = base_addr + i*PAGE_SIZE;
>> +        set_phys_to_machine(vmalloc_to_pfn(vaddr),
>> +                arbitrary_virt_to_machine(vaddr).maddr >> PAGE_SHIFT);
>> +    }
>
> The second argument (mfn) to set_phys_to_machine is wrong:
> arbitrary_virt_to_machine ends up calling virt_to_machine if
> virt_addr_valid. You need to manually call pte_mfn:
>
> /* the ptep content has been updated by Xen so we can lookup the foreign
>  * mfn from the pte now */
> pte = lookup_address(vaddr, &level);
> BUG_ON(pte == NULL);
> offset = vaddr & ~PAGE_MASK;
> mfn = XMADDR(((phys_addr_t)pte_mfn(*pte) << PAGE_SHIFT) + offset);
>
>
>> +
>> +out:
>> +     flush_tlb_all();
>> +     return err;
>> +}
>> +EXPORT_SYMBOL_GPL(xen_remap_domain_mfn_range2);
>
> the name should be changed to xen_remap_foreign_gpfn_range
>
>
>>  #ifdef CONFIG_XEN_PVHVM
>>  static void xen_hvm_exit_mmap(struct mm_struct *mm)
>>  {
>> diff --git a/drivers/video/xen-fbfront.c b/drivers/video/xen-fbfront.c
>> index dc72563..82da2ee 100644
>> --- a/drivers/video/xen-fbfront.c
>> +++ b/drivers/video/xen-fbfront.c
>> @@ -25,8 +25,12 @@
>>  #include <linux/module.h>
>>  #include <linux/vmalloc.h>
>>  #include <linux/mm.h>
>> +#include <linux/sched.h>
>> +#include <asm/pgtable.h>
>> +#include <asm/page.h>
>>
>>  #include <asm/xen/hypervisor.h>
>> +#include <asm/xen/page.h>
>>
>>  #include <xen/xen.h>
>>  #include <xen/events.h>
>> @@ -34,6 +38,7 @@
>>  #include <xen/interface/io/fbif.h>
>>  #include <xen/interface/io/protocols.h>
>>  #include <xen/xenbus.h>
>> +#include <xen/xen-ops.h>
>>
>>  struct xenfb_info {
>>       unsigned char           *fb;
>> @@ -62,6 +67,12 @@ module_param_array(video, int, NULL, 0);
>>  MODULE_PARM_DESC(video,
>>       "Video memory size in MB, width, height in pixels (default 2,800,600)");
>>
>> +static unsigned long foreign_vaddr = 0;
>> +module_param(foreign_vaddr, ulong, S_IRUGO);
>> +
>> +static unsigned long foreign_domid = 0;
>> +module_param(foreign_domid, ulong, S_IRUGO);
>> +
>>  static void xenfb_make_preferred_console(void);
>>  static int xenfb_remove(struct xenbus_device *);
>>  static void xenfb_init_shared_page(struct xenfb_info *, struct fb_info *);
>> @@ -398,7 +408,17 @@ static int __devinit xenfb_probe(struct xenbus_device *dev,
>>       if (info->fb == NULL)
>>               goto error_nomem;
>>       memset(info->fb, 0, fb_size);
>> -
>> +    if((foreign_vaddr != 0) && (foreign_domid != 0)){
>> +        ret = xen_remap_domain_mfn_range2((unsigned long)(info->fb),
>> +                                    foreign_vaddr >> PAGE_SHIFT,
>> +                                    fb_size >> PAGE_SHIFT, foreign_domid);
>
> you should rename foreign_vaddr to foreign_gpfn and pass the gpfn value
> that is the ram_addr (page shifted) passed to xen_ram_alloc in qemu.
>
>> +        if(ret < 0){
>> +            printk("Can not remap vram of hvm guest.\n");
>> +            goto error;
>> +        }
>> +    }
>>       info->nr_pages = (fb_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
>>
>>       info->mfns = vmalloc(sizeof(unsigned long) * info->nr_pages);
>> diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h
>> index 4349e89..1554531 100644
>> --- a/include/xen/xen-ops.h
>> +++ b/include/xen/xen-ops.h
>> @@ -20,6 +20,10 @@ int xen_remap_domain_mfn_range(struct vm_area_struct *vma,
>>                              unsigned long mfn, int nr,
>>                              pgprot_t prot, unsigned domid);
>>
>> +int xen_remap_domain_mfn_range2(unsigned long addr,unsigned long mfn,
>> +                int nr, unsigned domid);
>>  extern unsigned long *xen_contiguous_bitmap;
>>  int xen_create_contiguous_region(unsigned long vstart, unsigned int order,
>>                               unsigned int address_bits);
>>
>


Hi Stefano,

	I have some progress in linux based stubdom project. As shown in the
attached video, I have started the emulated vga device in the linux
based stubdom.

	In a short conclusion, for the linux based stubdom, there are two
major problems about vga device emulation. The first is the vram
mapping, which we discussed a lot previously and handled it. Another
is the vga BIOS mapping (address 0xc0000-0xc8fff of hvm guest).

	I found the vga BIOS mapping problem in remap_area_mfn_pte_fn()
function. The pte_mkspecial() will return invalid value when I try to
map 0xc0000-0xc8fff into linux based stubdom.

pte_mkspecial()
	->pte_set_flags()
		->native_pte_val()
		->native_make_pte()

	According to my test, the root cause of vga BIOS mapping problem is
native_xxx functions. We could avoid the problem by invoking functions
defined in paravirt.h instead. The patch is as follows. But I think it
is not a good way to handle the problem. Maybe you can give me some
suggestions.

	I also found the hard disk didn’t work well. I will investigate it these days.


--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -2639,12 +2640,16 @@ static int remap_area_mfn_pte_fn(pte_t *ptep,
pgtable_t token,
 				 unsigned long addr, void *data)
 {
 	struct remap_data *rmd = data;
-	pte_t pte = pte_mkspecial(pfn_pte(rmd->mfn++, rmd->prot));
+    if((rmd->mfn & 0xfffffff0) == 0xc0){
+	    pte_t pte = pfn_pte(rmd->mfn++, rmd->prot);
+	    rmd->mmu_update->val = pte_val(pte);
+    }else{
+	    pte_t pte = pte_mkspecial(pfn_pte(rmd->mfn++, rmd->prot));
+	    rmd->mmu_update->val = pte_val_ma(pte);
+    }

 	rmd->mmu_update->ptr = arbitrary_virt_to_machine(ptep).maddr;
-	rmd->mmu_update->val = pte_val_ma(pte);
 	rmd->mmu_update++;
-
 	return 0;
 }

[-- Attachment #2: out.ogv --]
[-- Type: video/ogg, Size: 3783903 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel