Re: pvops netconsole

[Brad Plant <bplant@iinet.net.au>]

[-- Attachment #1.1.1: Type: text/plain, Size: 4001 bytes --]

On Sat, 13 Feb 2010 19:56:51 -0800
Jeremy Fitzhardinge <jeremy@goop.org> wrote:

> > 1. When the first message is sent over the net console, the WARN_ONCE() at net/core/netpoll.c:327 triggers. I'm not sure what to do about this.
> 
> What's the backtrace in the warning?

------------[ cut here ]------------
WARNING: at net/core/netpoll.c:329 netpoll_send_skb+0x145/0x1db()
netpoll_send_skb(): eth1 enabled interrupts in poll (xennet_start_xmit+0x0/0x6bb)
Pid: 1, comm: swapper Not tainted 2.6.32.7 #36
Call Trace:
 [<ffffffff81039100>] warn_slowpath_common+0x72/0x8a
 [<ffffffff81039165>] warn_slowpath_fmt+0x3c/0x3e
 [<ffffffff8100dd7f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8124e2a0>] ? xennet_start_xmit+0x0/0x6bb
 [<ffffffff81294e48>] netpoll_send_skb+0x145/0x1db
 [<ffffffff813604d4>] ? _spin_unlock_irqrestore+0x19/0x1c
 [<ffffffff812950dc>] netpoll_send_udp+0x1fe/0x20e
 [<ffffffff8124004d>] ? firmware_loading_store+0x120/0x1b5
 [<ffffffff8124f9d6>] write_msg+0x89/0xca
 [<ffffffff8124004d>] ? firmware_loading_store+0x120/0x1b5
 [<ffffffff81039281>] __call_console_drivers+0x67/0x79
 [<ffffffff810392ec>] _call_console_drivers+0x59/0x5d
 [<ffffffff81039613>] release_console_sem+0x120/0x1c7
 [<ffffffff81039e73>] register_console+0x22e/0x2ae
 [<ffffffff81503380>] init_netconsole+0x15d/0x1c8
 [<ffffffff81049033>] ? schedule_delayed_work+0x16/0x1b
 [<ffffffff81503223>] ? init_netconsole+0x0/0x1c8
 [<ffffffff8100a05f>] do_one_initcall+0x59/0x154
 [<ffffffff814e56a9>] kernel_init+0x152/0x1a8
 [<ffffffff810112aa>] child_rip+0xa/0x20
 [<ffffffff810104a1>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff81010c5d>] ? retint_restore_args+0x5/0x6
 [<ffffffff810112a0>] ? child_rip+0x0/0x20
---[ end trace 0f898deb8e1e2914 ]---

 
> I had a quick look at e1000's and it calls the interrupt handler; the 
> netfront equivalent would be to call xennet_interrupt() with interrupts 
> disabled.  I'm not sure virtio_net is a good model because it doesn't 
> seem to have any interrupt-related stuff in it; I guess that happens 
> elsewhere in virtio.

Actually, I tried something like this first because I first looked at pcnet32.c and e100.c:

disable_irq(dev->irq);
xennet_interrupt(dev->irq, dev);
enable_irq(dev->irq);

The same WARN_ONCE() was triggered.


> > 2. When either netconsole is setup or the first message is sent, messages already logged to the main console get logged again. Maybe this will go away when the above is fixed?
> 
> Probably not.  It sounds like a feature.

Ok. This one was the least of my worries anyway.


> > 3. Initially netconsole would not initialise because the interface didn't yet have a mac address as setup_netfront hadn't yet been called. I changed the module_init() in netconsole.c to late_initcall() if !CONFIG_MODULES&&  CONFIG_XEN. Is this allowed and sane?
>
> Probably not.  In practice every kernel will have modules and Xen 
> enabled, and it will change netconsole for all users.  I guess we 
> probably need to find some way to init the MAC address earlier, or 
> trigger netconsole init once the mac address is known.

I figured it was dodgy. How would you init the MAC address earlier though? I've observed the xen devices being initialised at slightly different times with respect to other devices which suggests to me that they (and possibly all devices) are being initialised in separate threads. It would mean setting the MAC address before device initialisation yes?

Not sure how correct this is either, but another solution might be to set the MAC address to say 00:00:00:00:00:01 when the netdev structure is first created. This lets netconsole initialise since the MAC address isn't all zeros or all 0xFF. By the time any net console logs are transmitted, setup_netfront will have already been called and the correct MAC address set so no packets will actually be sent with the above MAC address. I have attached a patch that does this.

Cheers,

Brad

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: xen-netconsole2.patch --]
[-- Type: text/x-patch, Size: 2528 bytes --]

diff -urpNX linux-2.6.32.7/Documentation/dontdiff linux-2.6.32.7/arch/x86/boot/compressed/piggy.S linux-2.6.32.7-xen-netconsole/arch/x86/boot/compressed/piggy.S
--- linux-2.6.32.7/arch/x86/boot/compressed/piggy.S	2010-02-14 09:21:54.000000000 +1100
+++ linux-2.6.32.7-xen-netconsole/arch/x86/boot/compressed/piggy.S	2010-02-14 15:46:52.000000000 +1100
@@ -1,6 +1,6 @@
 .section ".rodata.compressed","a",@progbits
 .globl z_input_len
-z_input_len = 2406261
+z_input_len = 2406344
 .globl z_output_len
 z_output_len = 11827328
 .globl z_extract_offset
diff -urpNX linux-2.6.32.7/Documentation/dontdiff linux-2.6.32.7/arch/x86/boot/zoffset.h linux-2.6.32.7-xen-netconsole/arch/x86/boot/zoffset.h
--- linux-2.6.32.7/arch/x86/boot/zoffset.h	2010-02-14 09:21:54.000000000 +1100
+++ linux-2.6.32.7-xen-netconsole/arch/x86/boot/zoffset.h	2010-02-14 15:46:52.000000000 +1100
@@ -3,5 +3,5 @@
 #define ZO_startup_32 0x0000000000000000
 #define ZO_z_extract_offset 0x0000000000905000
 #define ZO_z_extract_offset_negative 0xffffffffff6fb000
-#define ZO_z_input_len 0x000000000024b775
+#define ZO_z_input_len 0x000000000024b7c8
 #define ZO_z_output_len 0x0000000000b47880
diff -urpNX linux-2.6.32.7/Documentation/dontdiff linux-2.6.32.7/drivers/net/xen-netfront.c linux-2.6.32.7-xen-netconsole/drivers/net/xen-netfront.c
--- linux-2.6.32.7/drivers/net/xen-netfront.c	2009-12-03 14:51:21.000000000 +1100
+++ linux-2.6.32.7-xen-netconsole/drivers/net/xen-netfront.c	2010-02-14 15:51:51.000000000 +1100
@@ -989,6 +989,15 @@ err:
 	return work_done;
 }
 
+#ifdef CONFIG_NET_POLL_CONTROLLER
+static void xennet_netpoll(struct net_device *dev)
+{
+	struct netfront_info *np = netdev_priv(dev);
+
+	napi_schedule(&np->napi);
+}
+#endif
+
 static int xennet_change_mtu(struct net_device *dev, int mtu)
 {
 	int max = xennet_can_sg(dev) ? 65535 - ETH_HLEN : ETH_DATA_LEN;
@@ -1113,6 +1122,9 @@ static const struct net_device_ops xenne
 	.ndo_change_mtu	     = xennet_change_mtu,
 	.ndo_set_mac_address = eth_mac_addr,
 	.ndo_validate_addr   = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+	.ndo_poll_controller = xennet_netpoll,
+#endif
 };
 
 static struct net_device * __devinit xennet_create_dev(struct xenbus_device *dev)
@@ -1181,6 +1193,9 @@ static struct net_device * __devinit xen
 
 	np->netdev = netdev;
 
+	/* Set MAC to temporary (non-zero) value so that netconsole initialises */
+	netdev->dev_addr[ETH_ALEN - 1] = 1;
+
 	netif_carrier_off(netdev);
 
 	return netdev;

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel