Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket 'fd' open from outside for unix socket

[Wei Xu <wexu@redhat.com>]

On 2016年06月14日 22:23, Aaron Conole wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
>> On Tue, Jun 14, 2016 at 04:03:43PM +0800, Wei Xu wrote:
>>> On 2016年06月09日 05:48, Aaron Conole wrote:
>>>> Flavio Leitner <fbl@redhat.com> writes:
>>>>
>>>>> Adding Aaron who is fixing exactly that on the OVS side.
>>>>>
>>>>> Aaron, please see the last question in the bottom of this email.
>>>>>
>>>>> On Wed, Jun 08, 2016 at 06:07:29AM -0400, Amnon Ilan wrote:
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Michal Privoznik" <mprivozn@redhat.com>
>>>>>>> To: "Daniel P. Berrange" <berrange@redhat.com>
>>>>>>> Cc: qemu-devel@nongnu.org, "amit shah" <amit.shah@redhat.com>,
>>>>>>> jasowang@redhat.com, "Wei Xu" <wexu@redhat.com>,
>>>>>>> armbru@redhat.com
>>>>>>> Sent: Thursday, June 2, 2016 2:38:53 PM
>>>>>>> Subject: Re: [Qemu-devel] [RFC Patch 0/3] Accept passed in socket
>>>>>>> 'fd' open from outside for unix socket
>>>>>>>
>>>>>>> On 02.06.2016 10:29, Daniel P. Berrange wrote:
>>>>>>>> On Thu, Jun 02, 2016 at 09:41:56AM +0200, Michal Privoznik wrote:
>>>>>>>>> On 01.06.2016 18:16, Wei Xu wrote:
>>>>>>>>>> On 2016年06月01日 00:44, Daniel P. Berrange wrote:
>>>>>>>>>>> On Wed, Jun 01, 2016 at 12:30:44AM +0800, wexu@redhat.com wrote:
>>>>>>>>>>>> From: Wei Xu <wexu@redhat.com>
>>>>>>>>>>>>
>>>>>>>>>>>> Recently I'm working on a fd passing issue,
>>>>>>>>>>>> selinux forbids qemu to
>>>>>>>>>>>> create a unix socket for a chardev when managing
>>>>>>>>>>>> VMs with libvirt,
>>>>>>>>>>>> because qemu don't have sufficient permissions in
>>>>>>>>>>>> this case, and
>>>>>>>>>>>> proposal from libvirt team is opening the 'fd' in
>>>>>>>>>>>> libvirt and merely
>>>>>>>>>>>> passing it to qemu.
>>>>>>>>>>>
>>>>>>>>>>> This sounds like a bug in libvirt, or selinux, or a mistaken
>>>>>>>>>>> configuration
>>>>>>>>>>> of the guest. It is entirely possible for QEMU to
>>>>>>>>>>> create a unix socket
>>>>>>>>>>> - not
>>>>>>>>>>> least because that is exactly what QEMU uses for its QMP monitor
>>>>>>>>>>> backend.
>>>>>>>>>>> Looking at your example command line, I think the
>>>>>>>>>>> issue is simply that
>>>>>>>>>>> you
>>>>>>>>>>> should be putting the sockets in a different location. ie at
>>>>>>>>>>> /var/lib/libvirt/qemu/$guest-vhost-user1.sock where QEMU has
>>>>>>>>>>> permission to
>>>>>>>>>>> create sockets already.
>>>>>>>>>> ah.. adjusting permission or file location can solve
>>>>>>>>>> this problem, i'm
>>>>>>>>>> guessing maybe this is a more security concern, the
>>>>>>>>>> socket is used as a
>>>>>>>>>> network interface for a vm, similar as the qcow image
>>>>>>>>>> file, thus should
>>>>>>>>>> prevent it to be arbitrarily accessed.
>>>>>>>>>>
>>>>>>>>>> Michael, do you have any comment on this?
>>>>>>>>>
>>>>>>>>> I haven't seen the patches. But in libvirt we allow users to create a
>>>>>>>>> vhostuser interface and even specify where the socket should be placed:
>>>>>>>>>
>>>>>>>>>       <interface type='vhostuser'>
>>>>>>>>>         <mac address='52:54:00:ee:96:6c'/>
>>>>>>>>>         <source type='unix' path='/tmp/vhost1.sock' mode='server'/>
>>>>>>>>>         <model type='virtio'/>
>>>>>>>>>       </interface>
>>>>>>>>>
>>>>>>>>> The following cmd line is generated by libvirt then:
>>>>>>>>>
>>>>>>>>> -chardev socket,id=charnet1,path=/tmp/vhost1.sock,server \
>>>>>>>>> -netdev type=vhost-user,id=hostnet1,chardev=charnet1 \
>>>>>>>>> -device
>>>>>>>>> virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:ee:96:6c,bus=pci.0,\
>>>>>>>>>
>>>>>>>>> Now, if we accept only /var/run/openvwitch path in
>>>>>>>>> /interface/source/@path (or whatever path to OVS is), we don't need this
>>>>>>>>> and have users manually label the dir (unless already labeled). But
>>>>>>>>> since we accept just any path in there, we should make sure that qemu is
>>>>>>>>> then able to create the socket. One possible fix would be to allow qemu
>>>>>>>>> create sockets just anywhere in the system. This, however, brings huge
>>>>>>>>> security risks and it's not acceptable IMO. The other option would be
>>>>>>>>> that libvirt would create the socket, and pass its FD to qemu (since
>>>>>>>>> libvirt already is allowed to create sockets anywhere).
>>>>>>>>
>>>>>>>> There are plenty of other places where we allow arbitrary paths in the
>>>>>>>> XML, but which have restrictions imposed by the security drivers. Not
>>>>>>>> least the <channel> devices which have the exact same scenario as this
>>>>>>>> network device, and require use of /var/lib/libvirt/qemu
>>>>>>>> as the directory
>>>>>>>> for the sockets. We certainly do not want to allow QEMU to
>>>>>>>> create sockets
>>>>>>>> anywhere.
>>>>>>>>
>>>>>>>> I don't think we want to grant QEMU svirtt permission to create sockets
>>>>>>>> in the /var/run/openvswitch directory either really.IMHO,
>>>>>>>> users of vhost
>>>>>>>> user should really be using /var/lib/libvirt/qemu, as is used for all
>>>>>>>> other UNIX sockets we create wrt other devices.
>>>>>>>
>>>>>>> Okay. I can live with that; but in that case we should document it
>>>>>>> somewhere, that we guarantee only paths under /var/lib/libvirt/ to be
>>>>>>> accessible and for the rest we do our best but maybe require sys admin
>>>>>>> intervention (e.g. to label the whole tree for a non-standard location).
>>>>>>
>>>>>> Does OVS has some limit for it's sockets to be only in
>>>>>> /var/run/openvswitch ?
>>>>
>>>> As of a recent commit, it can only be in /var/run/openvswitch or a
>>>> subdirectory therein (found in the openvswitch database).
>>> Aaron, thanks for your reply.
>>>
>>> Just a question about the usage of openvswitch, in this user case when
>>> launching a vhostuser/dpdk via libvirt, qemu works as server mode for socket
>>> under /var/run/openvswitch, but per my previous test, ovs/dpdk always works
>>> as server mode, which means ovs will creates the socket and listening for
>>> connection, thus qemu works as client mode, does ovs/dpdk support working in
>>> client mode? which means it's qemu's duty to create the socket? and ovs will
>>> connect to it on demanding?
>>
>> Oh, I was assuming that QEMU would be working in server mode - no wonder
>> we have somewhat different views :-)
>>
>> If OVS is running the UNIX socket server, and QEMU is purely the client,
>> then that means the solution would be slightly different. In particular
>> libvirt would *not* do any SELinux relabelling. Instead you would have
>> to get an addition to the SELinux policy, to allow svirt_t type to connect
>> to the SELinux type associated with the openvswitch socket.
>
> I agree, this is a good MAC solution.
OK.
>
>> For file permissions, if the OVS socket is owned by a particular UNIX
>> group, you could potentially add the 'qemu' user account to that group
>> to grant access.
>
> I actually would propose making a new vhost group, and adding the qemu
> user account and openvswitch user accounts to that group.  That way reduces
> pollution into other aspects of each process' function.
Michal,
Can we fix it with this as a replacement?

One of my concerns about the patch is looks there are a few other places 
in system which need 'fd' like sockets, therefore each needs a new 'fd' 
parameter in command line, this maybe not a clean and concise way, i 
think qemu_open() proposed by Eric is better to generally solve these 
kind of issues, maybe we can consider enhancing it along that direction 
in the future, what's your opinion?
>
> -Aaron
>
>>
>> Regards,
>> Daniel
>