Sharing grant_table pages between 2 miniOS

Sharing grant_table pages between 2 miniOS

[Lamia M. Youseff]

Dear All,
I am doing a small experiment with grant table to share 4 pages of 
memory between 2 miniOS, but have not been successful. For some reason, 
my GNTTABOP_map_grant_ref returns -1, and the 2 miniOS does not see the 
same machine address. 

I have attached here the code for the two threads i created for DomA and 
DomB respectively, as well as their output. Please, suggest what i 
should change here.
I will appreciate too if you can point me to a pointer that explains 
grant tables with some example code.
Thank you for your reply in advance,
-- Lamia Youseff

  
/******************************************************************************
 * kernel.c -  mini-Os-grnt-garntee (Dom A. domid = 30)
 *******************************************************************************/ 


void UCSB_grantee_thread(void *p)
{

  grant_ref_t gref[1]; // each grant_ref_t should provide me with 4k of 
shared memory
  void **map;
  char mystring[30]="Welcome to Shared Memory.\n";
  unsigned long mfn;

  *map = (void *) alloc_pages(2);
  memcpy(*map, mystring, 30);
 
  // get the machine physical address of the domain
  mfn = virt_to_mfn(*map);
 
  // now, share the page with remote domain (say dom 31)
  gref[0] = gnttab_grant_access(   31, //domid
                   mfn, // the page to be shared
                   0 ); // i.e. 0 == rw. 1 == ro.

  printk("My machine address is %u\n", virt_to_mach(*map));
  printk("My grant_ref_t is %d.\n",  (unsigned int) gref[0]);
 
  // now sleep till the other domain access the page
  for (;;){
    sleep(1000);
  }
  return;
}


/****************************************************************************** 

 * kernel.c -- mini-Os-grnt-garnted (i.e. DomB, domid = 31)
 * 
****************************************************************************/
void UCSB_granted_thread(void *p)
{
  unsigned long vm_area;
  int error_val;
 
  gnttab_map_grant_ref_t aop[1];
  grant_handle_t ghandle[1];
  int i;
 
  grant_ref_t gref[1];
  gref[0] = 8;
 
  vm_area = (unsigned long) alloc_pages(4);
 
  aop[0].host_addr      = vm_area;
  aop[0].dom              =  30;  // domAid;
  aop[0].ref                =  gref[0];
  aop[0].flags             =  ( GNTMAP_host_map | GNTMAP_readonly );
 
  error_val = HYPERVISOR_grant_table_op( GNTTABOP_map_grant_ref, aop, 1);
 
  printk("My machine address is %u\n", virt_to_mach(vm_area));
  printk("Error in granting_op in (HYPERVISOR_grant_table_o). Error 
%d....%d\n", aop[0].status, error_val);
  printk("Error ==> %s\n", gnttabop_error(aop[0].status));

  if (aop[0].status != GNTST_okay){
    printk("Error in the return status for GNTTAB_entry %d\n", i);
    return;
  }
  
  if ( unlikely(aop[0].handle < 0) ){
    printk("Error in aop[0].handle\n");
    return;
  }
  else ghandle[0]= aop[0].handle;
 
  printk("My string is %s\n", (char *) vm_area);
 
  return ;
}

/**************************************************
*  Output from DomA
**************************************************/
Started domain Mini-OS-grantee
Xen Minimal OS!
start_info:   0001f000
  nr_pages:   8192  shared_inf: 001af000
  pt_base:    00022000  mod_start:  0x0
  mod_len:    0
  flags:      0x0
  stack:      000140e0-000160e0
MM: Init
  _text:        00000000
  _etext:       0000b19a
  _edata:       0000c7a0
  stack start:  000140e0
  _end:         00016534
  start_pfn:    2a
  max_pfn:      2000
Mapping memory range 0x400000 - 0x2000000
MM: Initialise page allocator for 38000(38000)-2000000(2000000)
MM: done
Demand map pfns start at 2200 (02200000).
Initialised demand area.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 02200000.
Initialising scheduler
Thread "Idle": pointer: 0x3a00c, stack: 0x3c000
Initialising xenbus
Thread "xenstore": pointer: 0x3a03c, stack: 0x3e000
Dummy main: start_info=000160e0
Thread "xenbus_tester": pointer: 0x3a06c, stack: 0x40000
Thread "UCSB_grantee_thread": pointer: 0x3a09c, stack: 0x42000
Xenbus tests disabled, because of a Xend bug.
Thread "xenbus_tester" exited.
My machine address is 979255296
My grant_ref_t is 8.


/**************************************************
*  Output from DomB
**************************************************/
Started domain Mini-OS-granted
Xen Minimal OS!
start_info:   0001f000
  nr_pages:   8192  shared_inf: 001a9000
  pt_base:    00022000  mod_start:  0x0
  mod_len:    0
  flags:      0x0
  cmd_line:  
  stack:      00014180-00016180
MM: Init
  _text:        00000000
  _etext:       0000b1ce
  _edata:       0000c840
  stack start:  00014180
  _end:         000165d4
  start_pfn:    2a
  max_pfn:      2000
Mapping memory range 0x400000 - 0x2000000
MM: Initialise page allocator for 38000(38000)-2000000(2000000)
MM: done
Demand map pfns start at 2200 (02200000).
Initialised demand area.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 02200000.
Initialising scheduler
Thread "Idle": pointer: 0x3a00c, stack: 0x3c000
Initialising xenbus
Thread "xenstore": pointer: 0x3a03c, stack: 0x3e000
Dummy main: start_info=00016180
Thread "xenbus_tester": pointer: 0x3a06c, stack: 0x40000
Thread "UCSB_granted_thread": pointer: 0x3a09c, stack: 0x42000
Xenbus tests disabled, because of a Xend bug.
Thread "xenbus_tester" exited.
My machine address is 200548352
Error in granting_op in (HYPERVISOR_grant_table_o). Error 0....-1
Error ==> okay
My string is
My string is
Thread "UCSB_granted_thread" exited.

[$$]# xm list
Name                                      ID   Mem VCPUs      State   
Time(s)
Domain-0                                   0   256     1     r-----   1120.0
Mini-OS-granted                           31    32     1     -b----      0.0
Mini-OS-grantee                           30    32     1     -b----      0.0

Re: Sharing grant_table pages between 2 miniOS

[Kieran Mansley]

On Mon, 2007-05-14 at 20:34 -0700, Lamia M. Youseff wrote:
> Dear All,
> I am doing a small experiment with grant table to share 4 pages of 
> memory between 2 miniOS, but have not been successful. For some reason, 
> my GNTTABOP_map_grant_ref returns -1

Is it possible that the grant_operation_permitted() call during the
map_grant_ref operation is failing?  The code in question is this (from
include/xen/iocap.h):

/*
 * Until TLB flushing issues are sorted out we consider it unsafe for
 * domains with no hardware-access privileges to perform grant
map/transfer
 * operations.
 */
#define grant_operation_permitted(d)                    \
    (!rangeset_is_empty((d)->iomem_caps))

If this is the case, giving the granting domain some iomem capability
should allow you to do what you want.  I'm not sure what the TLB
flushing issues are, or what the plan is to get a proper solution to
this.

Kieran