Re: workaround for dom0 crash due to QEMU using O_DIRECT

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Jul 04, 2013 at 07:19:40PM +0100, Stefano Stabellini wrote:
> Hi Alex,
> speaking with Ian about the dom0 kernel crash caused by using O_DIRECT
> in QEMU, we came up with a simple workaround that should turn the crash
> into a data corruption problem (same as native).

<chuckles> You should for fun also do 0xEE on the 'trade_page' whenever
we update the PTE. That way we can detect the corruption as by default
the trade_page would be 00.
> 
> The idea is that when we balloon out pages, we replace the original page
> with a mapping of a scrub page, so that if the network stack wants to
> access an old grant that doesn't exist anymore, it should find a valid
> page mapped there (the scrub page).

This will I think make it hard to do increase_reservation
as it consults the P2M to make sure the PFN is indeed 'free'.

But then this is a debug patch..

> 
> Could you please try the appended patch for Linux with QEMU that uses
> O_DIRECT to open a file on NFS?
> 
> Thanks!
> 
> - Stefano
> 
> ---
> 
> 
> diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
> index 930fb68..0663fda 100644
> --- a/drivers/xen/balloon.c
> +++ b/drivers/xen/balloon.c
> @@ -88,6 +88,7 @@ EXPORT_SYMBOL_GPL(balloon_stats);
>  
>  /* We increase/decrease in batches which fit in a page */
>  static xen_pfn_t frame_list[PAGE_SIZE / sizeof(unsigned long)];
> +static struct page* trade_page;
>  
>  #ifdef CONFIG_HIGHMEM
>  #define inc_totalhigh_pages() (totalhigh_pages++)
> @@ -423,7 +424,7 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>  		if (xen_pv_domain() && !PageHighMem(page)) {
>  			ret = HYPERVISOR_update_va_mapping(
>  				(unsigned long)__va(pfn << PAGE_SHIFT),
> -				__pte_ma(0), 0);
> +				pfn_pte(page_to_pfn(trade_page), PAGE_KERNEL), 0);
>  			BUG_ON(ret);
>  		}
>  #endif
> @@ -436,7 +437,7 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
>  	/* No more mappings: invalidate P2M and add to balloon. */
>  	for (i = 0; i < nr_pages; i++) {
>  		pfn = mfn_to_pfn(frame_list[i]);
> -		__set_phys_to_machine(pfn, INVALID_P2M_ENTRY);
> +		__set_phys_to_machine(pfn, pfn_to_mfn(page_to_pfn(trade_page)));
>  		balloon_append(pfn_to_page(pfn));
>  	}
>  
> @@ -591,6 +592,10 @@ static int __init balloon_init(void)
>  	if (!xen_domain())
>  		return -ENODEV;
>  
> +	trade_page = alloc_page(GFP_KERNEL);
> +	if (trade_page == NULL)
> +		return -ENOMEM;
> +
>  	pr_info("xen/balloon: Initialising balloon driver.\n");
>  
>  	balloon_stats.current_pages = xen_pv_domain()