Re: remus error

[Ashish Bijlani <ashish.bijlani@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4443 bytes --]

Couldn't get it to work. Could you please look at the modules I'm
using. Posting the relevant files.

May be you can fix these and post upstream as well.

Thanks,
Ashish

On Thu, Mar 4, 2010 at 2:59 AM, Ashish Bijlani <ashish.bijlani@gmail.com> wrote:
> Yes I was successfully able to run " tc qdisc add dev vif3.0 root queue"
>
> "tc qdisc show" prints this:
>
> qdisc pfifo_fast 0: dev peth0 root bands 3 priomap  1 2 2 2 1 2 0 0 1
> 1 1 1 1 1 1 1
> qdisc pfifo_fast 0: dev imq0 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1
> 1 1 1 1 1 1
> qdisc queue 8001: dev vif3.0 root
>
> all the modules are loaded
>
> Module                  Size  Used by
> sch_queue               3384  1
> ebtable_filter          2696  1
> ebtables               16008  1 ebtable_filter
> ebt_imq                 1640  1
> imq                     6192  0
> xt_IMQ                  2056  0
>
> I think I may have made a mistake while porting rtnetlink to new
> netlink API. I'm looking at that. Hopefully should be able to fix
> that. I'll update soon.
>
> Thanks,
> Ashish
>
> On Thu, Mar 4, 2010 at 2:46 AM, Brendan Cully <brendan@cs.ubc.ca> wrote:
>> On Thursday, 04 March 2010 at 02:16, Ashish Bijlani wrote:
>>> "tc qdisc" works fine I guess:
>>>
>>> qdisc pfifo_fast 0: dev peth0 root bands 3 priomap  1 2 2 2 1 2 0 0 1
>>> 1 1 1 1 1 1 1
>>> qdisc pfifo_fast 0: dev imq0 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1
>>> 1 1 1 1 1 1
>>> qdisc pfifo_fast 0: dev vif3.0 root bands 3 priomap  1 2 2 2 1 2 0 0 1
>>> 1 1 1 1 1 1 1
>>>
>>> what could be missing?
>>
>> Hmm, and sch_queue and ebt_imq are loaded? Can you run
>> $ tc qdisc add dev vif3.0 root queue
>> ?
>>
>> Do you have ebt: filter table support built? How about the netfilter
>> netlink interface?
>>
>> I haven't actually tried recent pvops myself. With luck I'll get some
>> time tomorrow to do so.
>>
>>> Thanks,
>>> Ashish
>>>
>>> On Thu, Mar 4, 2010 at 2:09 AM, Brendan Cully <brendan@cs.ubc.ca> wrote:
>>> > Sounds like your kernel is missing netlink support. I don't remember
>>> > the exact config option required, but if 'tc qdisc' (from the iproute
>>> > package) works, that probably means you've enabled the right
>>> > option. As I think I mentioned before, turning on something like Token
>>> > Bucket Filter ought to drag in the necessary dependencies.
>>> >
>>> > On Thursday, 04 March 2010 at 01:50, Ashish Bijlani wrote:
>>> >> I'd to modify remus sch_queue ebt_imq kmod to work with pvops (2.6.31.6)
>>> >>
>>> >> Now, I'm getting a different error with "remus -i 100 PVM 192.168.1.5"
>>> >>
>>> >> PROF: suspending at 1267684430.539464
>>> >> installing buffer on imq0
>>> >> RTNETLINK answers: No such file or directory
>>> >> ERROR Internal error: Suspend request failed
>>> >> ERROR Internal error: Domain appears not to have suspended
>>> >> Save exit rc=1
>>> >> Traceback (most recent call last):
>>> >>   File "/usr/bin/remus", line 359, in <module>
>>> >>     run(cfg)
>>> >>   File "/usr/bin/remus", line 340, in run
>>> >>     for buf in bufs:
>>> >>   File "/usr/bin/remus", line 277, in postsuspend
>>> >>     buf.postsuspend()
>>> >>   File "/usr/bin/remus", line 159, in postsuspend
>>> >>     self._setup()
>>> >>   File "/usr/bin/remus", line 185, in _setup
>>> >>     self.rth.talk(req.pack())
>>> >>   File "/usr/local/lib/python2.6/dist-packages/xen/remus/netlink.py",
>>> >> line 314, in talk
>>> >>     self._rth.talk(req)
>>> >> IOError: error sending message
>>> >>
>>> >>
>>> >> "remus --no-net PVM 192.168.1.5" works and imq0 exists, I don't know
>>> >> what is missing...any inputs?
>>> >>
>>> >> Thanks,
>>> >> Ashish
>>> >>
>>> >> On Sun, Feb 28, 2010 at 12:47 AM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>>> >> > On 02/27/2010 07:54 PM, Ashish Bijlani wrote:
>>> >> >>
>>> >> >> I fail to use remus with xen-unstable (4.0.0-rc4)+ Linux-2.6-pvops:
>>> >> >>
>>> >> >> # sudo remus -i 100 PVM 192.168.1.3
>>> >> >> modprobe sch_queue
>>> >> >> FATAL: Module sch_queue not found.
>>> >> >>
>>> >> >
>>> >> > As far as I know, there hasn't been enough work done to make remus work with
>>> >> > pvops kernels yet.
>>> >> >
>>> >> >    J
>>> >> >
>>> >>
>>> >
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>
>

[-- Attachment #2: sch_queue.c --]
[-- Type: text/x-csrc, Size: 5803 bytes --]

/*
 * sch_queue.c Queue traffic until an explicit release command
 *
 *             This program is free software; you can redistribute it and/or
 *             modify it under the terms of the GNU General Public License
 *             as published by the Free Software Foundation; either version
 *             2 of the License, or (at your option) any later version.
 *
 * The operation of the buffer is as follows:
 * When a checkpoint begins, a barrier is inserted into the
 *   network queue by a netlink request (it operates by storing
 *   a pointer to the next packet which arrives and blocking dequeue
 *   when that packet is at the head of the queue).
 * When a checkpoint completes (the backup acknowledges receipt),
 *   currently-queued packets are released.
 * So it supports two operations, barrier and release.
 */

#include <linux/autoconf.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <net/pkt_sched.h>
#include <xen/page.h>
#include <xen/features.h>
#include <linux/uaccess.h>

/* xenbus directory */
#define FIFO_BUF    (10*1024*1024)

#define TCQ_CHECKPOINT 0
#define TCQ_DEQUEUE    1

struct queue_sched_data {
  /* this packet is the first packet which should not be delivered.
   * If it is NULL, queue_enqueue will set it to the next packet it sees. */
  struct sk_buff *stop;
};

struct tc_queue_qopt {
  /* 0: reset stop packet pointer
   * 1: dequeue to stop pointer */
  int action;
};

/* borrowed from drivers/xen/netback/loopback.c */
#ifdef CONFIG_X86
static int is_foreign(unsigned long pfn)
{
  /* NB. Play it safe for auto-translation mode. */
  return (xen_feature(XENFEAT_auto_translated_physmap) ||
         (get_phys_to_machine(pfn) & FOREIGN_FRAME_BIT));
}
#else
/* How to detect a foreign mapping? Play it safe. */
#define is_foreign(pfn)	(1)
#endif

static int skb_remove_foreign_references(struct sk_buff *skb)
{
  struct page *page;
  unsigned long pfn;
  int i, off;
  char *vaddr;

  BUG_ON(skb_shinfo(skb)->frag_list);

  for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
    pfn = page_to_pfn(skb_shinfo(skb)->frags[i].page);
    if (!is_foreign(pfn))
      continue;
    /*
      printk("foreign ref found\n");
    */
    page = alloc_page(GFP_ATOMIC | __GFP_NOWARN);
    if (unlikely(!page))
      return 0;

    pagefault_disable();
    vaddr = page_address(skb_shinfo(skb)->frags[i].page);

    off = skb_shinfo(skb)->frags[i].page_offset;
    memcpy(page_address(page) + off, vaddr + off,
          skb_shinfo(skb)->frags[i].size);
    pagefault_enable();

    put_page(skb_shinfo(skb)->frags[i].page);
    skb_shinfo(skb)->frags[i].page = page;
  }

  return 1;
}

static int queue_enqueue(struct sk_buff *skb, struct Qdisc* sch)
{
  struct queue_sched_data *q = qdisc_priv(sch);

  if (likely(sch->qstats.backlog + skb->len <= FIFO_BUF))
  {
    if (!q->stop)
      q->stop = skb;

    if (!skb_remove_foreign_references(skb)) {
      printk("error removing foreign ref\n");
      return qdisc_reshape_fail(skb, sch);
    }

    return qdisc_enqueue_tail(skb, sch);
  }
  printk("queue reported full: %d,%d\n", sch->qstats.backlog, skb->len);

  return qdisc_reshape_fail(skb, sch);
}

/* dequeue doesn't actually dequeue until the release command is
 * received. */
static inline struct sk_buff *queue_dequeue(struct Qdisc* sch)
{
  struct queue_sched_data *q = qdisc_priv(sch);
  struct sk_buff* peek;
  /*
  struct timeval tv;

  if (!q->stop) {
    do_gettimeofday(&tv);
    printk("packet dequeued at %lu.%06lu\n", tv.tv_sec, tv.tv_usec);
  }
  */

  if (sch->flags & TCQ_F_THROTTLED)
    return NULL;

  peek = (struct sk_buff *)((sch->q).next);

  /* this pointer comparison may be shady */
  if (peek == q->stop) {
    /*
    do_gettimeofday(&tv);
    printk("stop packet at %lu.%06lu\n", tv.tv_sec, tv.tv_usec);
    */

    /* this is the tail of the last round. Release it and block the queue */
    sch->flags |= TCQ_F_THROTTLED;
    return NULL;
  }

  return qdisc_dequeue_head(sch);
}

static int queue_init(struct Qdisc *sch, struct  nlattr* opt)
{
  sch->flags |= TCQ_F_THROTTLED;

  return 0;
}

/* receives two messages:
 *   0: checkpoint queue (set stop to next packet)
 *   1: dequeue until stop */
static int queue_change(struct Qdisc* sch, struct  nlattr* opt)
{
  struct queue_sched_data *q = qdisc_priv(sch);
  struct tc_queue_qopt* msg;
  /*
  struct timeval tv;
  */

  if (!opt || nla_len(opt) < sizeof(*msg))
    return -EINVAL;

  msg = nla_data(opt);

  if (msg->action == TCQ_CHECKPOINT) {
    /* reset stop */
    q->stop = NULL;
  } else if (msg->action == TCQ_DEQUEUE) {
    /* dequeue */
    sch->flags &= ~TCQ_F_THROTTLED;
    netif_schedule_queue(sch->dev_queue);
    /*
    do_gettimeofday(&tv);
    printk("queue release at %lu.%06lu (%d bytes)\n", tv.tv_sec, tv.tv_usec,
          sch->qstats.backlog);
    */
  } else {
    return -EINVAL;
  }

  return 0;
}

struct sk_buff* queue_peek(struct Qdisc* sch)
{
  if (sch->flags & TCQ_F_THROTTLED)
    return NULL;

  return (struct sk_buff *)((sch->q).next);
}

struct Qdisc_ops queue_qdisc_ops = {
  .id          =       "queue",
  .priv_size   =       sizeof(struct queue_sched_data),
  .enqueue     =       queue_enqueue,
  .dequeue     =       queue_dequeue,
  .peek     =       queue_peek,
  .init                =       queue_init,
  .change       =      queue_change,
  .owner       =       THIS_MODULE,
};

static int __init queue_module_init(void)
{
  printk("loading queue\n");
  return register_qdisc(&queue_qdisc_ops);
}

static void __exit queue_module_exit(void)
{
  printk("queue unloaded\n");
  unregister_qdisc(&queue_qdisc_ops);
}
module_init(queue_module_init)
module_exit(queue_module_exit)
MODULE_LICENSE("GPL");

[-- Attachment #3: ebt_imq.c --]
[-- Type: text/x-csrc, Size: 1064 bytes --]

#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/netfilter.h>
#include <linux/netfilter/x_tables.h>
#include <linux/netfilter_bridge/ebtables.h>
#include "ebt_imq.h"

static unsigned int
ebt_target_imq(struct sk_buff *pskb, const struct xt_target_param *par)
{
  const struct ebt_imq_info *info = par->targinfo;

  if (!skb_make_writable(pskb, 0))
     return EBT_DROP;

  pskb->imq_flags = info->todev | IMQ_F_ENQUEUE;

  return EBT_CONTINUE;
}

static bool ebt_target_imq_check(const struct xt_tgchk_param *par)
{
  return true;
}

static struct xt_target imq_target =
{
  .name                = "imq",
  .revision       = 0,
  .family         = NFPROTO_BRIDGE,
  .target       = ebt_target_imq,
  .checkentry       = ebt_target_imq_check,
  .targetsize     = XT_ALIGN(sizeof(struct ebt_imq_info)),
  .me          = THIS_MODULE,
};

static int __init init(void)
{
  return xt_register_target(&imq_target);
}

static void __exit fini(void)
{
  xt_unregister_target(&imq_target);
}


module_init(init);
module_exit(fini);
MODULE_LICENSE("GPL");

[-- Attachment #4: Makefile --]
[-- Type: application/octet-stream, Size: 838 bytes --]

XEN_ROOT=../../..
include $(XEN_ROOT)/tools/Rules.mk

# Should make makefiles export linux build directory!
# This is a fragile hack to tide us over
ifeq ($(KERNELS),linux-2.6-xen)
LINUX_VER=2.6.18-xen
endif
ifeq ($(KERNELS),linux-2.6-xen0)
LINUX_VER=2.6.18-xen0
endif
ifeq ($(KERNELS),linux-2.6-pvops)
LINUX_VER=2.6-pvops
endif

KERNELDIR ?= $(XEN_ROOT)/build-linux-$(LINUX_VER)_$(XEN_TARGET_ARCH)

.PHONY: all
all:
	if test -d $(KERNELDIR) && grep -q ^CONFIG_IMQ= $(KERNELDIR)/.config 2>/dev/null; then $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` modules; fi

.PHONY: install
install: all
	if test -d $(KERNELDIR) && grep -q ^CONFIG_IMQ= $(KERNELDIR)/.config 2>/dev/null; then $(MAKE) -C $(KERNELDIR) SUBDIRS=`pwd` INSTALL_MOD_PATH=$(DESTDIR) modules_install; fi

clean::
	-rm -rf *.o *.ko *.mod.c *.mod.o Module.symvers .*.cmd .tmp_versions

[-- Attachment #5: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel