From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dagaen Golomb Subject: Re: Problem Reading from XenStore in DomU Date: Sun, 15 May 2016 23:30:13 -0400 Message-ID: References: <242b1768-e958-0752-a6c2-8ac1a68468aa@cardoe.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5101263219200606120==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Meng Xu Cc: Doug Goldstein , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org --===============5101263219200606120== Content-Type: multipart/alternative; boundary=94eb2c088ffc0c6f340532ed3eff --94eb2c088ffc0c6f340532ed3eff Content-Type: text/plain; charset=UTF-8 > >>>>> Hi All, > >>>>> > >>>>> I'm having an interesting issue. I am working on a project that > >>>>> requires me to share memory between dom0 and domUs. I have this > >>>>> successfully working using the grant table and the XenStore to > >>>>> communicate grefs. > >>>>> > >>>>> My issue is this. I have one domU running Ubuntu 12.04 with a default > >>>>> 3.8.x kernel that has no issue reading or writing from the XenStore. > >>>>> My work also requires some kernel modifications, and we have made > >>>>> these changes in the 4.1.0 kernel. In particular, we've only added a > >>>>> simple hypercall. This modified kernel is what dom0 is running, on top > >>>>> of Xen 4.7 rc1. > >>>> > >>>> Without reading the rest of the thread but seeing the kernel versions. > >>>> Can you check how you're communicating to xenstore? Is it via > >>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3.14 will give you > >>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6 and newer should > >>>> prefer /dev/xen/xenbus. Same thing can happen with privcmd but making > >>>> that default didn't land until Xen 4.7. Since you're on the right > >>>> versions I expect you're using /dev/xen/xenbus but you never know. > >>> > >>> How do I know which is being used? /dev/xen/xenbus is there and so is > >>> process/xen/xenbus. Could this be a problem with header version > >>> mismatches or something similar? I'm using the xen/xenstore.h header > >>> file for all of my xenstore interactions. I'm running Xen 4.7 so it > >>> should be in /dev/, and the old kernel is before 3.14 but the new one > >>> is after, but I would presume the standard headers are updated to > >>> account for this. Is there an easy way to check for this? Also, would > >>> the same issue cause writes to fails? Because writes from the same > >>> domain work fine, and appear to other domains using xenstore-ls. > >>> > >>> Regards, > >>> Dagaen Golomb > >>> > >> > >> Use strace on the process and see what gets opened. > > > > Ah, of course. It seems both the working and non-working domains are > > using /proc/... > > Then according to Doug, "Anything after 3.14 will give you > > deadlocks if you try to use /proc/xen/xenbus.". Maybe the non-working domU uses kernel version after 3.14. It does, being the custom kernel on version 4.1.0. But Dom0 uses this same exact kernel and reads/writes just fine! The only solution if this is indeed the problem appears to be changing the kernel source we build on or some hacky method such as symlinking /proc/.. to /dev/.., there has to be an elegant real solution to this... Regards, Dagaen Golomb --94eb2c088ffc0c6f340532ed3eff Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

> >>>>> Hi All,
> >>>>>
> >>>>> I'm having an interesting issue. I am working= on a project that
> >>>>> requires me to share memory between dom0 and domU= s. I have this
> >>>>> successfully working using the grant table and th= e XenStore to
> >>>>> communicate grefs.
> >>>>>
> >>>>> My issue is this. I have one domU running Ubuntu = 12.04 with a default
> >>>>> 3.8.x kernel that has no issue reading or writing= from the XenStore.
> >>>>> My work also requires some kernel modifications, = and we have made
> >>>>> these changes in the 4.1.0 kernel. In particular,= we've only added a
> >>>>> simple hypercall. This modified kernel is what do= m0 is running, on top
> >>>>> of Xen 4.7 rc1.
> >>>>
> >>>> Without reading the rest of the thread but seeing the= kernel versions.
> >>>> Can you check how you're communicating to xenstor= e? Is it via
> >>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3= .14 will give you
> >>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6= and newer should
> >>>> prefer /dev/xen/xenbus. Same thing can happen with pr= ivcmd but making
> >>>> that default didn't land until Xen 4.7. Since you= 're on the right
> >>>> versions I expect you're using /dev/xen/xenbus bu= t you never know.
> >>>
> >>> How do I know which is being used? /dev/xen/xenbus is the= re and so is
> >>> process/xen/xenbus. Could this be a problem with header v= ersion
> >>> mismatches or something similar? I'm using the xen/xe= nstore.h header
> >>> file for all of my xenstore interactions. I'm running= Xen 4.7 so it
> >>> should be in /dev/, and the old kernel is before 3.14 but= the new one
> >>> is after, but I would presume the standard headers are up= dated to
> >>> account for this. Is there an easy way to check for this?= Also, would
> >>> the same issue cause writes to fails? Because writes from= the same
> >>> domain work fine, and appear to other domains using xenst= ore-ls.
> >>>
> >>> Regards,
> >>> Dagaen Golomb
> >>>
> >>
> >> Use strace on the process and see what gets opened.
> >
> > Ah, of course. It seems both the working and non-working domains = are
> > using /proc/...
>
> Then according to Doug, "Anything after 3.14 will give you
> > deadlocks if you try to use /proc/xen/xenbus.". Maybe the no= n-working domU uses kernel version after 3.14.

It does, being the custom kernel on version 4.1.0. But Dom0 = uses this same exact kernel and reads/writes just fine! The only solution i= f this is indeed the problem appears to be changing the kernel source we bu= ild on or some hacky method such as symlinking /proc/.. to /dev/.., there h= as to be an elegant real solution to this...

Regards,
Dagaen Golomb

--94eb2c088ffc0c6f340532ed3eff-- --===============5101263219200606120== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9y Zy94ZW4tZGV2ZWwK --===============5101263219200606120==--