From mboxrd@z Thu Jan 1 00:00:00 1970 From: Doug Goldstein Subject: Re: Hotplugged devices in Xen 4.5 and domain reboot Date: Tue, 1 Dec 2015 15:03:41 -0600 Message-ID: <565E0B2D.1040006@cardoe.com> References: <20151201140208.GA25722@citrix.com> <20151201152910.GW21588@citrix.com> <565DCF60.3020001@globallogic.com> <1448989503.15768.152.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1128417396763034299==" Return-path: In-Reply-To: <1448989503.15768.152.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell , Iurii Mykhalskyi , Wei Liu Cc: xen-devel@lists.xen.org, Pavlo Suikov , Ian Jackson , Andrii Anisov , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============1128417396763034299== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="OI1u9QlmNMeg5OV5VHroWOPfogqFgR5VS" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OI1u9QlmNMeg5OV5VHroWOPfogqFgR5VS Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 12/1/15 11:05 AM, Ian Campbell wrote: > On Tue, 2015-12-01 at 18:48 +0200, Iurii Mykhalskyi wrote: >> Thanks to all for a replays, please see my answers below: >> >> On 12/01/2015 05:29 PM, Wei Liu wrote: >>> On Tue, Dec 01, 2015 at 04:58:55PM +0200, Iurii Mykhalskyi wrote: >>>> Our real usb mass-storage device are located at driver domain (DomD)= =2E >>>> So we >>>> setup second block-device backend there. >>>> >>>> To hotplug usb mass-storage from DomD we use follow command: >>>> >>>> xl block-attach domU_id phy:/bla-bla,xvda10,w,backend=3D"DomD" >>>> >>> What happens if you run this in Dom0? I guess DomD doesn't respond to= >>> the request? >> Yes, there is no responded from domD, because actual storage devic= e >> are located there, and toolstack stuck on real device existence check.= >=20 > This check is a toolstack bug which should be fixed. We've squashed som= e of > them at various points, but I'd not be surprised if others remain. >=20 >>>> There was no support of attaching block-device in runtime from domai= n >>>> other >>>> to Domain-0, so we have made some hacks to allow call block-attach >>>> command >>>> from non-dom0 privileged domain. >>> So this is a special use case. This is the first time I know people >>> actually run xl block-attach in driver domain. >> Yes, this is special case and we this by our solution design. >>>> One of patches was - don't update >>>> /var/lib/xen/userdata-d.$DOMID-$UUID.libxl-json during execution of >>>> this >>>> command (because this log located on dom0 rootfs and we don't have >>>> any >>>> access to it from DomD). So, there is no different in configs before= >>>> and >>>> after hotplug. >>>> >>> The state of $DOMID is recorded in libxl-json file. No wonder you los= e >>> all state. >>> >>> But even if you write those states, they are going to be inside drive= r >>> domain. There is no way at the moment to synthesise the state inside= >>> Dom0 and DomD into one. There is also difficulty in how you can split= >>> the synthesised and dispatch the states to multiple entities again wh= en >>> rebuilding a domain. >>> >>> So I think having multiple entities managing state of one single doma= in >>> is bad. I think the proper way of making it work is to make hotplug >>> device from domain other than Dom0 work. >>> >>> There is a daemon "xl devd" in driver domain. We might be able to tea= ch >>> it to response to Dom0 toostack request. I'm a bit surprised if it >>> doesn't do that already. Did you forget to start that daemon? >> We can't run devd in driver domain, because it failed on connect t= o >> xenstored socket (/var/run/xenstored/socket - we have xenstored runnin= g >> only in dom0). =20 >=20 > devd _should_ be able to talk to xenstored over the kernel provided > interface to the shared ring rather than the local socket. >=20 > It is certainly not expected that devd be colocated in the same domain = as > happens to be running xenstored. >=20 > If this is not working then there is another bug. This works, but might have problems in Xen 4.5. If you're using running on Linux 3.14 or newer then you will have a problem. You need to backport commit 9c89dc95201ffed5fead17b35754bf9440fdbdc0 if you're using the C based xenstore and 7d418eab3b6dbdeec84bf73af301dca54368547a if you're using the Ocaml based xenstore. You can test this locally by supplying "--disable-socket" to xenstore when it starts up. >=20 >>> In general a driver domain would not be expected to have sufficient >>> privilege over e.g. a guest domain's /local/domain/domU/devices to >>> create >>> the f.e. dirs. >=20 >> In our solution we have to create 2 full privileged domains - Dom0= and DomD, so we need 2 toolstack domains. >> Any special privileges hack wasn't done - we need just to setup add= itional permissions for DomD. >=20 > I'm afraid you simply cannot have 2 toolstack domains. The toolstack is= a > singleton entity in a Xen system. >=20 > If you want to run toolstack operations from a non-toolstack domain the= n > you will need to arrange for some (likely out-of-band) mechanism for su= ch > domains to ask the single toolstack domain to do something on their beh= alf. >=20 > Ian. >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >=20 --=20 Doug Goldstein --OI1u9QlmNMeg5OV5VHroWOPfogqFgR5VS Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0 iQJ8BAEBCgBmBQJWXgsuXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNTM5MEQ2RTNFMTkyNzlCNzVDMzIwOTVB MkJDMDNEQzg3RUQxQkQ0AAoJEKK8A9yH7RvUizcP/04luwAbJna29v01meCtQQRs qf4HKH6I7s0qjr6xiaPrW9GS0r6UFNcgoRWR2a4OfkAXE//LXsOt35p4EQ/zRBMn nSsDcTwRqigIVdqUY1GbNnjcZg4p1PU7ia+6mEo/O0tcbp/kS8spai/HpH7kXPiP w61M38yj+hTqDb8gEvHVJYqUtEZ+cENMTHLLvenxZSYJNSd2YfZqY4WQkIdq8Fe1 zmCQKXgGWnHFjk0t+VyTo/TBiIwb86CCc75Abjxcfu8aD2rNyfrlOAwdTyOgXlrE 6QIC2LBBIYZ0Mg9iZnzTgOcoRR/TvrwD8KkKS2morGD8RpqUTviGkZXi3Wfky8i1 CiGW8nuKstnzdmXec5vRObgjywnIHJ1f4YiVDAm2DLysAOawRfDI7vaMKXJDFJcA 8TxNLy60657V/S1cu/agjfreRdJKbNw4fgtSLsDOC/STM0Ew5dc6vK65CJ+OsyX6 o2f7TN+LM9L90FSsdICWA+JMVaKJIR83m12JVzQnvTFt0pHVygepzgyDEniKIq2I iMPWaT0K6ijOJR8/O9pPVKBsYaanUUjUAJsJOGfTzuV2PQzfm+vxDj01IuctAED9 TG4tUxIwCuhlJZtxiIbTQW4uYmRPvzsOSZbU04ukawtR9JAbGlx1xUqHv1v6ivdR 0AYyRl4FNZWpHteEsCX3 =T7CE -----END PGP SIGNATURE----- --OI1u9QlmNMeg5OV5VHroWOPfogqFgR5VS-- --===============1128417396763034299== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1128417396763034299==--