Re: kernel 2.6.25-rc7 highly unstable on high load

[Eric Dumazet <dada1@cosmosbay.com>]

Eric Dumazet a écrit :
> Denys Fedoryshchenko a écrit :
>> I did this tuning. Not sure what does mean rt_cache_entries, but 
>> seems they are still increasing.
>>
>> Kup /config #  rtstat -c1000 -i60
>> rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
>> rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
>> rt_cache|
>>  entries|  in_hit|in_slow_|in_slow_|in_no_ro|  
>> in_brd|in_marti|in_marti| 
>> out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|
>> out_hlis|
>>         |        |     tot|      mc|     ute|        |  an_dst|  
>> an_src|        |    _tot|     _mc|        |      ed|    miss| 
>> verflow| _search|t_search|
>>   274234| 1590009|  271953|       0|    1809|       6|       0|       
>> 0|   24551|    7365|       0|  279991|  279867|      24|       0| 
>> 5438630|   89656|
>>   280535|   25463|   19956|       0|     187|       0|       0|       
>> 0|     485|     263|       0|   20407|   20404|       3|       0|   
>> 12796|     242|
>>   288072|   25367|   19830|       0|     163|       0|       0|       
>> 0|     493|     256|       0|   20248|   20245|       3|       0|   
>> 17146|     346|
>>   294943|   25689|   19807|       0|     154|       0|       0|       
>> 0|     438|     250|       0|   20211|   20208|       3|       0|   
>> 21554|     432|
>>   301118|   25287|   19645|       0|     170|       0|       0|       
>> 0|     466|     236|       0|   20052|   20048|       3|       0|   
>> 25713|     528|
>>   307651|   25663|   20264|       0|     180|       0|       0|       
>> 0|     472|     234|       0|   20678|   20675|       3|       0|   
>> 30588|     607|
>>   313767|   25810|   20014|       0|     175|       0|       0|       
>> 0|     459|     237|       0|   20425|   20422|       3|       0|   
>> 33701|     699|
>>
>>
>>   
> Then try to lower gc_elasticity, to 3, or even 2
>
> echo 3 >/proc/sys/net/ipv4/route/gc_elasticity
>
>
One more idea before leaving :)

If rt_cache_entries still increasing, we might have a dst leak somewhere :
gc tries to evict entries that have a non null refcount -> they are put 
in dst_garbage.list

Following patch will show us how dst_garbage behaves.
 ( printk(KERN_DEBUG "dst_total: %d delayed: %d work_perf: %d" ...)


diff --git a/net/core/dst.c b/net/core/dst.c
index 7deef48..e634e5f 100644
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -31,6 +31,8 @@
  * 3) This list is guarded by a mutex,
  *    so that the gc_task and dst_dev_event() can be synchronized.
  */
+#undef RT_CACHE_DEBUG
+#define RT_CACHE_DEBUG 2
 #if RT_CACHE_DEBUG >= 2
 static atomic_t                         dst_total = ATOMIC_INIT(0);
 #endif