Resetting network device / skb leak

Resetting network device / skb leak

[Jacob Gorm Hansen]

hi,

I seem to have a problem with the Linux xen-network driver leaking skbs.
This is probably my own fault, upon resumption after migration I call
this:

void network_resume(void)
{
    struct net_device* dev = __dev_get_by_name("eth0");
    MOD_DEC_USE_COUNT;

    network_open(dev);
}

-- which works fine. However, in network_open() the call to
NETOP_RESET_RINGS seems to clear all the pointers to the old skbs, and
network_alloc_rx_buffers() allocs a bunch of new ones. It suppose that
means I have just leaked a whole bunch of skbs. I know the standard
suspend/resume code performs almost a full ifup/ifdown (which I cannot
do in my case) but I don't see any skb-freeing code in that case either?

I have tried dev_kfree_skb()'ing the contents of the np->rx_skbs[1:]
array before calling network_open, but doing so just crashes Linux :-(

Any ideas?

Jacob



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Resetting network device / skb leak

[Keir Fraser]

> hi,
> 
> I seem to have a problem with the Linux xen-network driver leaking skbs.
> This is probably my own fault, upon resumption after migration I call
> this:
> 
> void network_resume(void)
> {
>     struct net_device* dev = __dev_get_by_name("eth0");
>     MOD_DEC_USE_COUNT;
> 
>     network_open(dev);
> }
> 
> -- which works fine. However, in network_open() the call to
> NETOP_RESET_RINGS seems to clear all the pointers to the old skbs, and
> network_alloc_rx_buffers() allocs a bunch of new ones. It suppose that
> means I have just leaked a whole bunch of skbs. I know the standard
> suspend/resume code performs almost a full ifup/ifdown (which I cannot
> do in my case) but I don't see any skb-freeing code in that case either?
> 
> I have tried dev_kfree_skb()'ing the contents of the np->rx_skbs[1:]
> array before calling network_open, but doing so just crashes Linux :-(
> 
> Any ideas?
> 
> Jacob

The correct suspend behaviour is caused by executing dev->stop().

Why are you not able to run the existing ifdown/ifup code? It doesn't
execute notifier lists so shouldn't affect routing tables or anything
like that.

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Resetting network device / skb leak

[Jacob Gorm Hansen]

On Thu, 2004-04-22 at 08:19, Keir Fraser wrote: 
> The correct suspend behaviour is caused by executing dev->stop().
> 
> Why are you not able to run the existing ifdown/ifup code? It doesn't
> execute notifier lists so shouldn't affect routing tables or anything
> like that.

Mainly because I need the interface to remain active across the point
where I am checkpointing the system -- the interface is used in the
migration which is still ongoing at that time (I firewall off unrelated
connections to not mess things up externally).

Basically I need to revive the (already open but not
Xen-connected/initialised) interface when I wake up, which the
previously quoted code does fine, but apparently this causes a leak of
skbs. What I am looking for is a safe way of freeing or reusing the old
ones before reinitialising the interface.

Jacob



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Resetting network device / skb leak

[Keir Fraser]

> On Thu, 2004-04-22 at 08:19, Keir Fraser wrote: 
> > The correct suspend behaviour is caused by executing dev->stop().
> > 
> > Why are you not able to run the existing ifdown/ifup code? It doesn't
> > execute notifier lists so shouldn't affect routing tables or anything
> > like that.
> 
> Mainly because I need the interface to remain active across the point
> where I am checkpointing the system -- the interface is used in the
> migration which is still ongoing at that time (I firewall off unrelated
> connections to not mess things up externally).
> 
> Basically I need to revive the (already open but not
> Xen-connected/initialised) interface when I wake up, which the
> previously quoted code does fine, but apparently this causes a leak of
> skbs. What I am looking for is a safe way of freeing or reusing the old
> ones before reinitialising the interface.
> 
> Jacob
> 

So you even do the final parts of state migration within the domain
that is migrating? That sounds hard to get right. :-)

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Resetting network device / skb leak

[Jacob Gorm Hansen]

On Thu, 2004-04-22 at 12:28, Keir Fraser wrote:

> So you even do the final parts of state migration within the domain
> that is migrating? That sounds hard to get right. :-)

Otherwise it wouldn't be as much fun ;-) I _think_ this leak is the last
part that is missing (apart from block device migration which I am not
currently addressing, and some hard-coded limitations on the size of the
migrating guest OS), I suppose I should release a patch soon if anyone
is interested.

Jacob



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click