e1000e 82574L transmit latency?

e1000e 82574L transmit latency?

[Stanislav Meduna]

Hi,

anyone has experience / knows of bugs (not the ASPM one) causing
high (at least 70 ms) latency on Intel 82574L using the e1000e driver?

We have an instance of a connected device needing to be serviced
each 100 ms reporting a timeout while our internal measurements
show that the packets were given to the kernel max 30 ms apart.

These are raw packets, there is no other traffic on that interface,
no kernel messages, no errors in the /sys/class/net/eth1/statistics/*
and it happened after 5 days of uptime :/

I am using kernel 3.4 with RT PREEMPT - I note that the driver version
there is 1.9.5 while in 3.9 it is 2.2.14. Anyone knows if the network
drivers are back-portable without much hassle that far
or whether this is a complicated task?

e1000-devel: please Cc: me when replying, I am not subscribed
to the list.

Thanks
-- 
                                      Stano

Re: e1000e 82574L transmit latency?

[Nebojša Ćosić]

> Hi,
> 
> anyone has experience / knows of bugs (not the ASPM one) causing
> high (at least 70 ms) latency on Intel 82574L using the e1000e driver?
> 
> We have an instance of a connected device needing to be serviced
> each 100 ms reporting a timeout while our internal measurements
> show that the packets were given to the kernel max 30 ms apart.
> 
> These are raw packets, there is no other traffic on that interface,
> no kernel messages, no errors in the /sys/class/net/eth1/statistics/*
> and it happened after 5 days of uptime :/
> 
> I am using kernel 3.4 with RT PREEMPT - I note that the driver version
> there is 1.9.5 while in 3.9 it is 2.2.14. Anyone knows if the network
> drivers are back-portable without much hassle that far
> or whether this is a complicated task?
> 
> e1000-devel: please Cc: me when replying, I am not subscribed
> to the list.
> 
> Thanks

If you are having several processes/threads with different priorities
sending on same eth, combined with high cpu load on RT processes, than
it sounds a lot like a problem I had: Check
http://article.gmane.org/gmane.linux.rt.user/10163 for details

-- 
Nebojša
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: e1000e 82574L transmit latency?

[Stanislav Meduna]

On 28.05.2013 15:01, Nebojša Ćosić wrote:

> If you are having several processes/threads with different priorities
> sending on same eth, combined with high cpu load on RT processes, than
> it sounds a lot like a problem I had: Check
> http://article.gmane.org/gmane.linux.rt.user/10163 for details

Hm, interesting. However, there is just one thread sending on the
interface and all threads with higher priority generally only run for
brief intervals and do not do anything with ethernet. There are
lower-prio threads doing full TCP/IP but they communivate on another
interface (also a e100e one).

Thanks
-- 
                                    Stano

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: e1000e 82574L transmit latency?

[Nebojša Ćosić]

Дана Tue, 28 May 2013 15:49:45 +0200
Stanislav Meduna <stano@meduna.org> написа:

> On 28.05.2013 15:01, Nebojša Ćosić wrote:
> 
> > If you are having several processes/threads with different priorities
> > sending on same eth, combined with high cpu load on RT processes, than
> > it sounds a lot like a problem I had: Check
> > http://article.gmane.org/gmane.linux.rt.user/10163 for details
> 
> Hm, interesting. However, there is just one thread sending on the
> interface and all threads with higher priority generally only run for
> brief intervals and do not do anything with ethernet. There are
> lower-prio threads doing full TCP/IP but they communivate on another
> interface (also a e100e one).
> 
> Thanks
Well, I think it still applies. Lock which is responsible for this
problem is driver wise...
Lower prio thread can take driver lock, and thus delay higher thread
from accessing driver. It is enough that there are jobs with higher
prio than lowest prio tcp/ip sending thread, even if they have nothing
to do with network.

-- 
Nebojša
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: e1000e 82574L transmit latency?

[Stanislav Meduna]

On 28.05.2013 15:01, Nebojša Ćosić wrote:

> Check
> http://article.gmane.org/gmane.linux.rt.user/10163 for details

Anyway, could you publish the patch? What do you mean
by "effectively disabling queuing" - that the sending threads
will wait until the packet is out or that to-be-queued
packets are thrown away?

Thanks
-- 
                                    Stano

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: e1000e 82574L transmit latency?

[Nebojša Ćosić]

Дана Tue, 28 May 2013 16:05:33 +0200
Stanislav Meduna <stano@meduna.org> написа:

> On 28.05.2013 15:01, Nebojša Ćosić wrote:
> 
> > Check
> > http://article.gmane.org/gmane.linux.rt.user/10163 for details
> 
> Anyway, could you publish the patch? What do you mean
> by "effectively disabling queuing" - that the sending threads
> will wait until the packet is out or that to-be-queued
> packets are thrown away?
> 
> Thanks
Sending threads are going to sleep on mutex, instead of just passing by
and leaving packets in tx queue for later sending.

Patch is very simple - just comment out spin_* and HARD_TX_* calls in
sch_direct_xmit()


-- 
Nebojša
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html