e100 latency, cpu cycle saver and e1000...

e100 latency, cpu cycle saver and e1000...

[Paul Rolland]

Hello,

A few days ago, there was a thread about e100 latency related to
CPU Cycle Saver...

Suggestion was to disabled it to return back to some "standard"
latency.

At the moment, I'm experiencing some strange behavior, not with
an e100 but with an e1000 on a 2.4.20 kernel...

Here are some traces, using Corey Minyard test application :

[root@IP3 tmp]# ./test IP1 32000 10000 370
Average: 201us, Max: 631us, Min: 200us
[root@IP3 tmp]# ./test IP2 32000 10000 370
Average: 422us, Max: 47581us, Min: 209us

All three machines are using 2.4.20 and e1000 drivers, they are
the same hardware.

But, definitely, IP2 is exhibiting much higher max latency...

Increasing the packet size up to 1200 bytes doesn't change the 
global behavior : IP2 is always much "latent" than IP1...

The problem is that it seems there is no CPU Cycle Saver on e1000
NIC. Is there some equivalent ? Could someone give a guess on
what's going on ?

Regards,
Paul

Re: e100 latency, cpu cycle saver and e1000...

[Corey Minyard]

My little program was crude at best.

You have to take into account what the machines are doing.  If you
notice, the "Min" latency is about the same, so it's not a CPU cycle
saver.  The long "Max" latencies mean you probably have something on IP2
that is blocking the execution of the IP stack (for, say up to 4.5ms). 
Are all the machine completely quiesced except for the test program? 
Have you tried switching the network connections to see if it is in the
network hardware?

-Corey

Paul Rolland wrote:

>Hello,
>
>A few days ago, there was a thread about e100 latency related to
>CPU Cycle Saver...
>
>Suggestion was to disabled it to return back to some "standard"
>latency.
>
>At the moment, I'm experiencing some strange behavior, not with
>an e100 but with an e1000 on a 2.4.20 kernel...
>
>Here are some traces, using Corey Minyard test application :
>
>[root@IP3 tmp]# ./test IP1 32000 10000 370
>Average: 201us, Max: 631us, Min: 200us
>[root@IP3 tmp]# ./test IP2 32000 10000 370
>Average: 422us, Max: 47581us, Min: 209us
>
>All three machines are using 2.4.20 and e1000 drivers, they are
>the same hardware.
>
>But, definitely, IP2 is exhibiting much higher max latency...
>
>Increasing the packet size up to 1200 bytes doesn't change the 
>global behavior : IP2 is always much "latent" than IP1...
>
>The problem is that it seems there is no CPU Cycle Saver on e1000
>NIC. Is there some equivalent ? Could someone give a guess on
>what's going on ?
>
>Regards,
>Paul
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>  
>

Re: e100 latency, cpu cycle saver and e1000...

[Paul Rolland]

Hello,

> My little program was crude at best.
Yes, I know, but it does exist !

> You have to take into account what the machines are doing.  
> If you notice, the "Min" latency is about the same, so it's 
> not a CPU cycle saver.  The long "Max" latencies mean you 
> probably have something on IP2 that is blocking the execution 
> of the IP stack (for, say up to 4.5ms). 
> Are all the machine completely quiesced except for the test program? 
> Have you tried switching the network connections to see if it 
> is in the network hardware?

Correct, machines are not idle... but 
 - they are doing globally the same work,
 - this behavior is something I can reproduce test after test, since
   I've started this morning...

I started using that because IP1 was exhibiting high latency yesterday
'til I rebooted it, and since it is working quite fine...
Of course, I can reboot also IP2, but I'd like to understand why
and how to avoid it later...

Regards,
Paul

Re: e100 latency, cpu cycle saver and e1000...

[Zwane Mwaikambo]

On Wed, 21 May 2003, Paul Rolland wrote:

> Correct, machines are not idle... but 
>  - they are doing globally the same work,
>  - this behavior is something I can reproduce test after test, since
>    I've started this morning...
> 
> I started using that because IP1 was exhibiting high latency yesterday
> 'til I rebooted it, and since it is working quite fine...
> Of course, I can reboot also IP2, but I'd like to understand why
> and how to avoid it later...

One thing you can do to reduce packet handling latency (at the cost of 
CPU) with both the e1000 is drop down the RX Delay Interrupt timers, ditto 
for the Tx Delay. The hardware delays in increments of 1.024ms

	Zwane
-- 
function.linuxpower.ca

Re: e100 latency, cpu cycle saver and e1000...

[Paul Rolland]

Hello,

> One thing you can do to reduce packet handling latency (at 
> the cost of 
> CPU) with both the e1000 is drop down the RX Delay Interrupt 
> timers, ditto 
> for the Tx Delay. The hardware delays in increments of 1.024ms
> 
That's right, though the driver is not exhibiting a R/W /proc interface,
which probably means I'll have to reboot the server.

However, what I'm trying to figure out is : why did this latency 
change suddenly, if it can't be related to a CPU Cycle Saver, as
the code doesn't seem to contain something that would allow a dynamic
change of such a parameter on a running machine...

Of course, the problem may be completely unrelated to the NIC,
my initial question being : is there a CPU Cycle Saver on the e1000
as there is one on the e100... ;-)

Regards,
Paul

Re: e100 latency, cpu cycle saver and e1000...

[Zwane Mwaikambo]

On Thu, 22 May 2003, Paul Rolland wrote:

> Of course, the problem may be completely unrelated to the NIC,
> my initial question being : is there a CPU Cycle Saver on the e1000
> as there is one on the e100... ;-)

I suppose you could call the hardware RX/TX interrupt mitigation a 'cpu 
cycle saver'

	Zwane 
-- 
function.linuxpower.ca