From mboxrd@z Thu Jan 1 00:00:00 1970 From: Doug Goldstein Subject: Re: Critique of the Xen Security Process Date: Mon, 9 Nov 2015 15:48:42 -0600 Message-ID: <564114BA.8060206@cardoe.com> References: <20151106172228.GA2335@work-mutt> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3105566846972184587==" Return-path: In-Reply-To: <20151106172228.GA2335@work-mutt> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============3105566846972184587== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="xdv1MghcoCTepnxFx3ntJjwh0661hMsNG" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xdv1MghcoCTepnxFx3ntJjwh0661hMsNG Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/6/15 11:22 AM, Joanna Rutkowska wrote: > Hello, >=20 > Recently Xen has released the XSA-148 advisory [1] addressing a fatal b= ug in the > hypervisor. The bug has been lurking there for the last 7 years! We, th= e Qubes > OS Project, have commented on this in our Security Bulletin #22 [2]. An= d far > from enthusiastic commentary that was (FWIW, it was me who wrote this Q= SB, as > evidenced in the commits log, in case some from the Xen community would= like to > direct their rage towards a particular human being ;) Ian Jackson then = wrote a > response on the Xen blog [3]. I was then asked to share some more thoug= hts about > how I thought Xen could actually improve its security process [4]. So, = I share > some these below: >=20 > 1. First of all, I wish Xen was somehow more defensively coded. To prov= ide some > examples: >=20 > a. In XSA-109 [5] there was a problem with the hypervisor dereferencing= a NULL > pointer. The problem was fixed by the Xen Security Team by applying a p= atch > which (hopefully) made sure the execution path that lead to this NULL p= ointer > dereferencing code was never taken. Back then I suggested (on the Xen > pre-disclosure list) to make this patch more explicit though: >=20 >> On Wed, Jan 21, 2015 at 02:31:51PM +0100, Joanna Rutkowska wrote: >> (...) >>> >>> Wouldn't it be prudent to also check if: >>> >>> (v->arch.paging.mode>{write_guest_entry,cmpxchg_guest_entry} !=3D NUL= L) >>> >>> ... in the two affected functions, just before derefing these functio= n >>> pointers? >>> >>> Going even a step further: how about replacing all >>> function-pointer-based calls with macros that first validates the >>> pointer before derefing it? At least when the system doesn't have SME= P? >>> >=20 > ...to which I got a reply from one of the Xen Security Team engineers t= hat the > above might perhaps be justified in debug builds only, followed by a st= andard: > "feel free to contribute a patch". >=20 > b. The XSA-123 [6] was another critical security bug in Xen, this time = resulting > from one of the hypervisor developer's fetish to use an absolutely conf= using > construct in order to save a few modest bytes in a structure which migh= t have > been allocated by the system maybe a few tens of times at best. Even mo= re > worrying was the way how Xen Security Team decided to fix the bug: agai= n by > modifying some condition in the code further up the execution path, wit= h the > hopes that this time they would ensure this puzzling construct would al= ways be > used properly. We wrote more about this in our QSB #18 [7]. >=20 > c. Finally, the way how Xen fixed the recent XSA-148 looks also very re= active, > IMHO. With a bug of this calibre, I would expect Xen to carefully revie= w and > augment all its PV memory virtualization code with additional checks (A= SSERTs), > ensuring certain invariants are always satisfied. Such as e.g. that non= e of the > pages containing PDEs or PTEs are becoming writeable by the VM. >=20 > I can't help but have a feeling that some of the Xen developers seem to= be > overconfident in their belief they can fully understand all the possibl= e > execution paths in their code. Well, the XSAs quoted above are an indis= putable > prove that this is not quite always the case. Realizing that, each deve= loper by > themselves, might be a great step towards a more secure hypervisor... >=20 > 2. Another security-related aspect of the Xen project is how it totally= ignores > problems related to the build process security. Those who don't believe= me > should grep the sources for wget, which is now disguised as "FETCHER" s= hell > variable... (so grep for "FETCHER" string) >=20 > I feel embarrassed that I need to explain, at the end of 2015, why the = build > process of any serious software project should not blindly download uns= igned > components (sources) from the Internet, especially if it is about to ex= ecute > Makefiles from these components a moment later... Come on, guys! I'll echo this sentiment as well. Most distro packagers will dislike this and need to work around some of this behavior in their respective distros. Project Raisin is aiming to help with this but it doesn't seem to have a lot of community effort behind it and it too attempts to install dependencies on my machine and wants to be run with sudo. >=20 > (Of course we have been forced to get around this gapping security whol= e in > Qubes OS [8] ourselves, sadly with a method that is not well suited for= > upstreaming). >=20 > 3. Another thing is, of course: stop adding features to the core hyperv= isor. We > really need Xen to finally mature, stabilize, and for its development p= rocess to > be slowing down over time (just the bug fixes). We need a long-term-sup= ported > hypervisor, which doesn't change with subsonic speed. This would allow = this core > code to be widely audited by many experts. If some users want features,= these > should perhaps be maintained as additional modules (no, I don't mean dy= namically > loaded modules, just compile-time included), preferably in separate rep= os. >=20 As far as the compile time support goes I'm aiming for this as well. I've been working on Kconfig support and hope to get that pushed soon and the idea being that less mature features can be kept off until they're ready to be default on. The downstream distro/product makers can then fiddle with these flags to get the hypervisor that fits their needs.= > Perhaps also to move all the non-hypervisor code, such as all the tools= tacks, > stubdom, etc, into separate repos also. For hygiene, if for nothing els= e. =46rom a real high level things should be broken into at least 4 repos: 1) hypervisor 2) libxl 3) userland tools (xl and support scripts) 4) stubdom >=20 > Admittedly, some of the features are a result of hardware evolution, su= ch as > e.g. UEFI support. But many are not. Again, maintaining these as option= al code > (in separate repos) would be a great step into getting the hypervisor m= aturing, > finally. >=20 > I have already written about it years ago [9], as a matter of fact. >=20 > 4. Finally, I've been really surprised by the line of reasoning Ian exp= ressed in > the above-mentioned blog post. TL;DR: "we're still doing pretty great, = compared > to other projects, because: 1) we have smaller number of publicly discl= osed > bugs, and 2) we actually publicly disclose these bugs which we are awar= e of". >=20 > The attitude presented in the blog post is so wrong, that I'm not even = sure > where to start commenting on this... >=20 > With a single bug like the XSA-148 which, let me repeat that one more t= ime: had > been present in the hypervisor for the last 7 years, so with a bug like= this it > really doesn't matter how many (i.e. what number) of critical bugs does= the > competition have. Because only one bug of this calibre is enough for th= e > attacker to never really bother to find another one. The mere fact that= > competing hypervisors might got 12 bugs during the same period, really = doesn't > make Xen look any better, sorry. >=20 > Also, there is really nothing to be proud that you disclose the bugs. I= t would > be a problem if you didn't. >=20 > Hope the above comments might help improve the Xen security. Perhaps so= me would > perceive them as arrogant or rude. Too bad. Remember the actual attacke= rs will > not be arrogant or rude -- they will just come and exploit bugs, silent= ly. > Admittedly this might not hurt some of the developers ego, not in the s= hort time > at least. >=20 > Can we, the Qubes OS project, or myself personally, help with implement= ing the > above suggestions? Sadly, no. While some of us do contribute occasional= patches > to Xen (specifically Marek Marczykowski-G=C3=B3recki), we really work f= or a different > project and have different tasks and responsibilities. This is where I'll disagree with you. I too work for a different project and have different tasks and responsibilities but as a consumer of Xen we do have the responsibility to try to upstream things where applicable. Could create patches locally to disable features that I don't want/need on? Of course and it probably would take me less time but in the long run upstreaming a generic framework will have an immense pay off for not just me and my organization but everyone else. You cannot rail at a project for not doing the work which serves your end goals without doing some of that work yourself. >=20 > Regards, > joanna. >=20 > [1] http://xenbits.xen.org/xsa/advisory-148.html > [2] https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-022-2= 015.txt > [3] https://blog.xenproject.org/2015/10/30/security-vs-features/ > [4] https://twitter.com/xen_org/status/660151720463482880 > [5] http://xenbits.xen.org/xsa/advisory-109.html > [6] http://xenbits.xen.org/xsa/advisory-123.html > [7] https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-018-2= 015.txt > [8] https://github.com/QubesOS/qubes-vmm-xen/commit/dcd6c0a4f2c6226a9b7= 06e62469d420579c86975 > [9] http://lists.xen.org/archives/html/xen-devel/2013-09/msg01815.html >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >=20 --=20 Doug Goldstein --xdv1MghcoCTepnxFx3ntJjwh0661hMsNG Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0 iQJ8BAEBCgBmBQJWQRS7XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNTM5MEQ2RTNFMTkyNzlCNzVDMzIwOTVB MkJDMDNEQzg3RUQxQkQ0AAoJEKK8A9yH7RvUKZ4P/A6O9gECbkGX+OP+ZN9cMQqU dFZ+Odh+TNJhmRxY/bdoXIyKjIHjUiII8Wrb73VEeXrlcFpE9wZbWOuWG1W/BOQT yM7eSR/i6XW0T1aNLHAF7c21PKoWAT1uBJ4I+C5a0in9kpFIHDv8W8wIIvpFCo0g O8IGTPQBXu1Rv4VZMklE88Gt4LQCFoRj26Yasj3UuIl+M3OdSWDu2v2ro+M4LYK1 CPoz2LPRYzIQRuEH45nkGaAFRuUmpt+8FKGDBE95yj0dJg36DgiYP3DkUcQCvG91 yWkWQxg7KwG3pYFbaBcjcNYVQeuPdEnq9JPW1VebAl9GoIBDoeFYwby7UanueMju faf996Y2D5ckkD1NH+XzoZPn8y4/N4xI5O6pEHUoTeO52AY9ZSHr6SyF03XKXTAX x3v0z1C6dLfH9KRIDJkJ616GwRgoCBm1Bc02j5kcch7CEfXe0OR6og74CfbtKutK 5mDbwSwk+PIjHC5BBvtPpCocoBqDxHwZGFfu4c8MssiDFAb9O1r6dD+Rnbubccf0 /KgZGCNipZRQLnv99tsL1ZBPkcN4B+upJEpSLxqjSIFxHZJhYXu/NLVvoNNFB+nT GNL3vAngbAuOag9sFTlVi4uQeKIOTHXO07AQyPqvhg2sYIKIYTDtK9qeVJCEaJYL E5lBCn3sIIZhBtdtN8n6 =ajjk -----END PGP SIGNATURE----- --xdv1MghcoCTepnxFx3ntJjwh0661hMsNG-- --===============3105566846972184587== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============3105566846972184587==--