From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40415) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WDxEt-0003wX-Nq for qemu-devel@nongnu.org; Thu, 13 Feb 2014 09:26:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WDxEn-0004LC-PL for qemu-devel@nongnu.org; Thu, 13 Feb 2014 09:26:07 -0500 Received: from mail-pd0-f181.google.com ([209.85.192.181]:36765) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WDxEn-0004L7-GM for qemu-devel@nongnu.org; Thu, 13 Feb 2014 09:26:01 -0500 Received: by mail-pd0-f181.google.com with SMTP id y10so10545723pdj.12 for ; Thu, 13 Feb 2014 06:26:00 -0800 (PST) Message-ID: <52FCD5F3.1020709@ozlabs.ru> Date: Fri, 14 Feb 2014 01:25:55 +1100 From: Alexey Kardashevskiy MIME-Version: 1.0 References: <52FC75A0.5020902@ozlabs.ru> <52FC9FC9.9000103@ozlabs.ru> <52FCCBC8.1070003@ozlabs.ru> <52FCD16D.5080007@ozlabs.ru> In-Reply-To: Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] tap networking - how? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Max Filippov Cc: "qemu-devel@nongnu.org" On 02/14/2014 01:17 AM, Max Filippov wrote: > On Thu, Feb 13, 2014 at 6:06 PM, Alexey Kardashevskiy wrote: >> On 02/14/2014 01:02 AM, Max Filippov wrote: >>> On Thu, Feb 13, 2014 at 5:42 PM, Alexey Kardashevskiy wrote: >>>> On 02/13/2014 11:23 PM, Max Filippov wrote: >>>>> On Thu, Feb 13, 2014 at 2:34 PM, Alexey Kardashevskiy wrote: >>>>>> On 02/13/2014 07:40 PM, Max Filippov wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On Thu, Feb 13, 2014 at 11:34 AM, Alexey Kardashevskiy wrote: >>>>>>>> Hi! >>>>>>>> >>>>>>>> I am debugging spapr-vlan and hit the following issue. >>>>>>>> >>>>>>>> When I run QEMU as below, the kernel's DHCP client does not continue till I >>>>>>>> hit any key in console. If I replace spapr-vlan with >>>>>>>> e1000/rtl8139/virtio-net, everything is just fine. If I use "user" network >>>>>>>> - everything is fine too. So the problem is with combination of spapr-vlan >>>>>>>> + tap. >>>>>>>> >>>>>>>> The issue looks like - the guest kernel boots and then prints: >>>>>>>> Sending DHCP requests .. >>>>>>>> and it keeps printing dots till I press key or timeout expires. tcpdump >>>>>>>> (running on the tap interface) shows one DHCP request and one DHCP response. >>>>>>>> >>>>>>>> What normally happens is that QEMU calls os_host_main_loop_wait() which >>>>>>>> calls qemu_poll_ns() and it is sitting there till eventfd signals. >>>>>>>> This eventfd is registered via qemu_init_main_loop() -> aio_context_new() >>>>>>>> -> aio_set_event_notifier() but I cannot find where it gets passed to the >>>>>>>> kernel (otherwise why would we need eventfd?). When eventfd signals, QEMU >>>>>>>> calls qemu_iohandler_poll() which checks if TAP device has something to >>>>>>>> read and eventually calls tap_send(). >>>>>>>> >>>>>>>> However in my bad example QEMU does not exit qemu_poll_ns() on eventfd, >>>>>>>> only on stdin event. >>>>>>>> >>>>>>>> I can see AIO eventfd created and event_notifier_test_and_clear() is called >>>>>>>> on it before the kernel starts using spapr-vlan. >>>>>>>> >>>>>>>> So. h_send_logical_lan() is called to sent a DHCP request packet. Now I >>>>>>>> expect eventfd to signal but this does not happen. Have I missed some reset >>>>>>>> or notification request or "bottom half" (virtio-net uses them but >>>>>>>> e1000/rtl8139 do not)? >>>>>>> >>>>>>> Sounds pretty much like the problem I had recently with opencores >>>>>>> 10/100 MAC: https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00073.html >>>>>>> >>>>>>> Does the following help?: >>>>>> >>>>>> Yes, it does, thanks a lot! >>>>>> >>>>>> While we are here and you seem to understand this stuff - >>>>>> how is tap expected to work to deliver a packet from the external network >>>>>> to the guest? I mean what event should be triggered in what order? My brain >>>>>> is melting :( I just cannot see how receiving a packet on "tap" in the host >>>>>> kernel can make os_host_main_loop_wait() exit in QEMU so it could call >>>>>> qemu_iohandler_poll() and do the job. Thanks! >>>>> >>>>> I'm not very experienced in this area of QEMU, so the following may be not >>>>> 100% accurate. >>>>> Tap file descriptor is registered among other file descriptors in an array >>>>> that os_host_main_loop_wait use to poll for events. So normally packet >>>>> arrives to the host, fd becomes readable, poll function completes and >>>>> registered handler (see tap_update_fd_handler) is called. The handler reads >>>>> packets and calls the attached NIC's NetClientInfo::receive callback through >>>>> network queuing infrastructure. But once NIC doesn't process a packet or its >>>>> NetClientInfo::can_receive returns false it stops polling for new packets >>>>> by updating handlers associated with its fd. So NIC needs to inform the >>>>> networking core when it can receive more packets by calling >>>>> qemu_flush_queued_packets, which will also complete polling and deliver >>>>> already queued packets. >>>> >>>> >>>> I am more interested in details :) >>>> os_host_main_loop_wait() calls glib_pollfds_fill() which puts actual fds >>>> into gpollfds GArray thing. Before the tap device started, its fd is not >>>> there but after the patch you proposed, tap's fd gets to the list. >>>> The actual fds are put into array by g_main_context_query() (if I read gdb >>>> output correctly). So there must be some callback somewhere which tells >>>> this g_main_context_query() what to poll for. I put a million breakpoints >>>> to know what is called but to no avail. >>> >>> I see that qemu_iohandler_fill puts fds into this array. And it only puts those >>> that have write handler or read handler and can read at the moment. >> >> >> os_host_main_loop_wait() - when things work, it waits on the tap device >> too. Without your patch, it does not wait on the tap device fd (i.e. this >> fd is not put to the array of fds by glib_pollfds_fill()). Where does this >> difference happen - this is my question... > > It is triggered by the guest adding new descriptor to the NIC RX ring. > Added qemu_flush_queued_packets completes poll that doesn't have > TAP fd in the array, and (assuming there were no packets queued) > the next main_loop_wait -> qemu_iohandler_fill puts the TAP fd into > that array: > > if (ioh->fd_read && > (!ioh->fd_read_poll || > ioh->fd_read_poll(ioh->opaque) != 0)) { > events |= G_IO_IN | G_IO_HUP | G_IO_ERR; > } > > because now NIC's can_receive (called here through ioh->fd_read_poll) > returns true. > Oh. Right. The mosaic became a picture :) Thanks! -- Alexey