multiqueue

multiqueue

[Ivo van Doorn]

Hi Johannes,

Below is a warning I get with your latest QOS patch series,
rt61pci registers 4 queues, but apparently the WARN_ON()
in ieee80211_wake_queue is being triggered.

void ieee80211_wake_queue(struct ieee80211_hw *hw, int queue)
{
	struct ieee80211_local *local = hw_to_local(hw);

	if (netif_is_multiqueue(local->mdev)) {
		netif_wake_subqueue(local->mdev, queue);
	} else {
		WARN_ON(queue != 0);
		netif_wake_queue(local->mdev);
	}
}

CONFIG_MAC80211_QOS is enabled in the kernel configuration,
and debugfs shows that rt2x00 is serving frames over multiple
TX queues, so the multiqueue is working...

Ivo

May  4 00:27:20 localhost ------------[ cut here ]------------
May  4 00:27:20 localhost WARNING: at net/mac80211/util.c:334 ieee80211_wake_queue+0x51/0x5b [mac80211]()
May  4 00:27:20 localhost Modules linked in: snd_seq_midi snd_emu10k1_synth snd_emux_synth snd_seq_virmidi snd_seq_midi_event snd_seq_midi_emul snd_seq snd_pcm_oss snd_mixer_oss snd_intel8x0 rt61pci rt2400pci rt2x00pci rt2x00lib snd_emu10k1 snd_rawmidi snd_ac97_codec mac80211 ac97_bus cfg80211 eeprom_93cx6 snd_pcm snd_seq_device snd_timer snd_page_alloc snd_util_mem snd_hwdep snd scsi_wait_scan
May  4 00:27:20 localhost Pid: 0, comm: swapper Not tainted 2.6.25-wl-06940-gd91c42d #1
May  4 00:27:20 localhost
May  4 00:27:20 localhost Call Trace:
May  4 00:27:20 localhost <IRQ>  [<ffffffff802314d1>] warn_on_slowpath+0x51/0x63
May  4 00:27:20 localhost [<ffffffff804994eb>] skb_queue_tail+0x17/0x3f
May  4 00:27:20 localhost [<ffffffff8804f79c>] :mac80211:ieee80211_tx_status_irqsafe+0x10b/0x16b
May  4 00:27:20 localhost [<ffffffff880d4cc5>] :rt2x00lib:rt2x00lib_txdone+0xd5/0x10b
May  4 00:27:20 localhost [<ffffffff880d6867>] :rt2x00lib:rt2x00queue_index_inc+0x1a/0xa3
May  4 00:27:20 localhost [<ffffffff8806573a>] :mac80211:ieee80211_wake_queue+0x51/0x5b
May  4 00:27:20 localhost [<ffffffff880ef5b8>] :rt61pci:rt61pci_interrupt+0x2dd/0x340
May  4 00:27:20 localhost [<ffffffff80256164>] handle_IRQ_event+0x32/0x64
May  4 00:27:20 localhost [<ffffffff802577a6>] handle_fasteoi_irq+0x78/0xdd
May  4 00:27:20 localhost [<ffffffff8020e69d>] do_IRQ+0xa6/0x175
May  4 00:27:20 localhost [<ffffffff8020b691>] ret_from_intr+0x0/0xa
May  4 00:27:20 localhost [<ffffffff8020a21a>] default_idle+0x31/0x55
May  4 00:27:20 localhost [<ffffffff80236a98>] __do_softirq+0x51/0xd4
May  4 00:27:20 localhost [<ffffffff8020c30c>] call_softirq+0x1c/0x30
May  4 00:27:20 localhost [<ffffffff8020e417>] do_softirq+0x30/0x6b
May  4 00:27:20 localhost [<ffffffff8021bf26>] smp_apic_timer_interrupt+0x6e/0xa8
May  4 00:27:20 localhost [<ffffffff8020bdb6>] apic_timer_interrupt+0x66/0x70
May  4 00:27:20 localhost <EOI>  [<ffffffff8020a21a>] default_idle+0x31/0x55
May  4 00:27:20 localhost [<ffffffff8020a215>] default_idle+0x2c/0x55
May  4 00:27:20 localhost [<ffffffff8020a1e9>] default_idle+0x0/0x55
May  4 00:27:20 localhost [<ffffffff8020a286>] cpu_idle+0x48/0x8b
May  4 00:27:20 localhost
May  4 00:27:20 localhost ---[ end trace b92f992a457f49f1 ]---

Re: multiqueue

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

Hi Ivo,

> Below is a warning I get with your latest QOS patch series,
> rt61pci registers 4 queues, but apparently the WARN_ON()
> in ieee80211_wake_queue is being triggered.
> 
> void ieee80211_wake_queue(struct ieee80211_hw *hw, int queue)
> {
> 	struct ieee80211_local *local = hw_to_local(hw);
> 
> 	if (netif_is_multiqueue(local->mdev)) {
> 		netif_wake_subqueue(local->mdev, queue);
> 	} else {
> 		WARN_ON(queue != 0);
> 		netif_wake_queue(local->mdev);
> 	}
> }
> 
> CONFIG_MAC80211_QOS is enabled in the kernel configuration,
> and debugfs shows that rt2x00 is serving frames over multiple
> TX queues, so the multiqueue is working...

Strange. Can you verify that the master device has the
NETIF_F_MULTI_QUEUE flag set? Since that config symbol got enabled you
must have multiqueue code compiled in. Or maybe I put the setting of the
MULTI_QUEUE flag into a wrong patch by accident? Check
net/mac80211/main.c please

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: multiqueue

[Ivo van Doorn]

On Sunday 04 May 2008, Johannes Berg wrote:
> Hi Ivo,
> 
> > Below is a warning I get with your latest QOS patch series,
> > rt61pci registers 4 queues, but apparently the WARN_ON()
> > in ieee80211_wake_queue is being triggered.
> > 
> > void ieee80211_wake_queue(struct ieee80211_hw *hw, int queue)
> > {
> > 	struct ieee80211_local *local = hw_to_local(hw);
> > 
> > 	if (netif_is_multiqueue(local->mdev)) {
> > 		netif_wake_subqueue(local->mdev, queue);
> > 	} else {
> > 		WARN_ON(queue != 0);
> > 		netif_wake_queue(local->mdev);
> > 	}
> > }
> > 
> > CONFIG_MAC80211_QOS is enabled in the kernel configuration,
> > and debugfs shows that rt2x00 is serving frames over multiple
> > TX queues, so the multiqueue is working...
> 
> Strange. Can you verify that the master device has the
> NETIF_F_MULTI_QUEUE flag set? Since that config symbol got enabled you
> must have multiqueue code compiled in. Or maybe I put the setting of the
> MULTI_QUEUE flag into a wrong patch by accident? Check
> net/mac80211/main.c please

My config options are:
CONFIG_NET_SCHED=y
CONFIG_NETDEVICES_MULTIQUEUE=y
CONFIG_CFG80211=m
CONFIG_NL80211=y
CONFIG_WIRELESS_EXT=y
CONFIG_MAC80211=m
CONFIG_MAC80211_QOS=y

snip from net/mac80211/main.c:

#ifdef CONFIG_MAC80211_QOS
	if (hw->queues > IEEE80211_MAX_QUEUES)
		hw->queues = IEEE80211_MAX_QUEUES;
	if (hw->ampdu_queues > IEEE80211_MAX_AMPDU_QUEUES)
		hw->ampdu_queues = IEEE80211_MAX_AMPDU_QUEUES;
	if (hw->queues < 4)
		hw->ampdu_queues = 0;
#else
	hw->queues = 1;
	hw->ampdu_queues = 0;
#endif

	/* for now, mdev needs sub_if_data :/ */
	mdev = alloc_netdev_mq(sizeof(struct ieee80211_sub_if_data),
			       "wmaster%d", ether_setup,
			       hw->queues + hw->ampdu_queues);
	if (!mdev)
		goto fail_mdev_alloc;
	if (hw->queues > 1)
		mdev->flags |= NETIF_F_MULTI_QUEUE;

So theoretically this flag should be set.
However /sys/class/net/wmaster1/features contains the value 0x0

Ivo

Re: multiqueue

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]


> 	if (!mdev)
> 		goto fail_mdev_alloc;
> 	if (hw->queues > 1)
> 		mdev->flags |= NETIF_F_MULTI_QUEUE;
> 
> So theoretically this flag should be set.
> However /sys/class/net/wmaster1/features contains the value 0x0

Yes, umm, guess why. That line should be mdev->features |= ...

Not really my week, sorry.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: multiqueue

[Ivo van Doorn]

On Sunday 04 May 2008, Johannes Berg wrote:
> 
> > 	if (!mdev)
> > 		goto fail_mdev_alloc;
> > 	if (hw->queues > 1)
> > 		mdev->flags |= NETIF_F_MULTI_QUEUE;
> > 
> > So theoretically this flag should be set.
> > However /sys/class/net/wmaster1/features contains the value 0x0
> 
> Yes, umm, guess why. That line should be mdev->features |= ...

That did the trick. :)

Thanks,

Ivo

multiqueue

[Eric Van Hensbergen]

Was looking at the virtio-fs multiqueue description -- on the face of
things having per processor virtio queues could be a benefit for us as
well at larger scales -- although I imagine there are certain
crossover points.

        -eric