From: Alexander Dahl <ada@thorsis.com>
To: netdev@vger.kernel.org
Cc: Grant Edwards <grant.b.edwards@gmail.com>,
linux-mtd@lists.infradead.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: net: macb: fail when there's no PHY
Date: Fri, 04 Dec 2020 09:28:53 +0100 [thread overview]
Message-ID: <3542036.FvJvBFsO4O@ada> (raw)
In-Reply-To: <rqbobm$5qk$1@ciao.gmane.io>
Hello Grant,
sorry if I just hijack your conversation, but I'm curious, because we are
using the same SoC. Adding linux-arm-kernel to Cc for the general performance
issues and linux-mtd for the ECC questions. O:-)
Am Donnerstag, 3. Dezember 2020, 23:20:38 CET schrieb Grant Edwards:
> On 2020-12-03, Andrew Lunn <andrew@lunn.ch> wrote:
> >> I don't think there's any way I could justify using a kernel that
> >> doesn't have long-term support.
> >
> > 5.10 is LTS. Well, it will be, once it is actually released!
>
> Convincing people to ship an unreleased kernel would be a whole
> 'nother bucket of worms.
+1
Judging just from the release dates of the last LTS kernels, I would have
guessed v5.11 will get LTS. But there has been no announcement yet and I
suppose there will be none before release? For ordinary users it's just like
staring in a crystal ball, so we aim at v5.4 for our more recent hardware
platforms. ¯\_(ツ)_/¯
>
> It's all moot now. The decision was just made to shelve the 5.4 kernel
> "upgrade" and stick with 2.6.33 for now.
>
> >> [It looks like we're going to have to abandon the effort to use
> >> 5.4. The performance is so bad compared to 2.6.33.7 that our product
> >> just plain won't work. We've already had remove features to the get
> >> 5.4 kernel down to a usable size, but we were prepared to live with
> >> that.]
> >
> > Ah. Small caches?
>
> Yep. It's An old Atmel ARM926T part (at91sam9g20) with 32KB I-cache
> and 32KB D-cache.
>
> A simple user-space multi-threaded TCP echo server benchmark showed a
> 30-50% (depending on number of parallel connections) drop in max
> throughput. Our typical applications also show a 15-25% increase in
> CPU usage for an equivalent workload. Another problem is high
> latencies with 5.4. A thread that is supposed to wake up every 1ms
> works reliably on 2.6.33, but is a long ways from working on 5.4.
We use the exact same SoC with kernel 4.9.220-rt143 (PREEMPT RT) currently,
after being stuck on 2.6.36.4 for quite a while. I did not notice significant
performance issues, but I have to admit, we never did extensive benchmarks on
network or CPU performance, because the workload also changed for that target.
However what gave us a lot less dropped packages was using the internal SRAM
as DMA buffer for RX packages received by macb. That did not make it in
mainline however, I did not put enough effort in polishing that patch back
when we migrated from 2.6 to 4.x. If you're curious, it's on GitHub:
https://github.com/LeSpocky/linux/tree/net-macb-sram-rx
> I asked on the arm kernel mailing list if this was typical/expected,
> but the post never made it past the moderator.
>
> > The OpenWRT guys make valid complaints that the code
> > hot path of the network stack is getting too big to fit in the cache
> > of small systems. So there is a lot of cache misses and performance is
> > not good. If i remember correctly, just having netfilter in the build
> > is bad, even if it is not used.
>
> We've already disabled absoltely everything we can and still have a
> working system. With the same features enabled, the 5.4 kernel was
> about 75% larger than a 2.6.33 kernel, so we had to trim quite a bit
> of meat to get it to boot on existing units.
Same here. v4.9 kernel image still fits, v4.14 is already too big for some
devices we delivered in the early days.
> We also can't get on-die ECC support for Micron NAND flash working
> with 5.4. Even it did work, we'd still have to add the ability to
> fall-back to SW ECC on read operations for the sake of backwards
> compatibility on units that were initially shipped without on-die ECC
> support enabled.
IIRC the SoC itself has issues with its ECC engine? See mainline
at91sam9g20ek_common.dtsi for example which sets nand-ecc-mode to "soft".
The SAM9G20 Errata chapter in the complete datasheet from 2015 (Atmel-6384F)
says two times in Section 44.1.3: "Perform the ECC computation by software."
Greets
Alex
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
WARNING: multiple messages have this Message-ID (diff)
From: Alexander Dahl <ada@thorsis.com>
To: netdev@vger.kernel.org
Cc: Grant Edwards <grant.b.edwards@gmail.com>,
linux-mtd@lists.infradead.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: net: macb: fail when there's no PHY
Date: Fri, 04 Dec 2020 09:28:53 +0100 [thread overview]
Message-ID: <3542036.FvJvBFsO4O@ada> (raw)
In-Reply-To: <rqbobm$5qk$1@ciao.gmane.io>
Hello Grant,
sorry if I just hijack your conversation, but I'm curious, because we are
using the same SoC. Adding linux-arm-kernel to Cc for the general performance
issues and linux-mtd for the ECC questions. O:-)
Am Donnerstag, 3. Dezember 2020, 23:20:38 CET schrieb Grant Edwards:
> On 2020-12-03, Andrew Lunn <andrew@lunn.ch> wrote:
> >> I don't think there's any way I could justify using a kernel that
> >> doesn't have long-term support.
> >
> > 5.10 is LTS. Well, it will be, once it is actually released!
>
> Convincing people to ship an unreleased kernel would be a whole
> 'nother bucket of worms.
+1
Judging just from the release dates of the last LTS kernels, I would have
guessed v5.11 will get LTS. But there has been no announcement yet and I
suppose there will be none before release? For ordinary users it's just like
staring in a crystal ball, so we aim at v5.4 for our more recent hardware
platforms. ¯\_(ツ)_/¯
>
> It's all moot now. The decision was just made to shelve the 5.4 kernel
> "upgrade" and stick with 2.6.33 for now.
>
> >> [It looks like we're going to have to abandon the effort to use
> >> 5.4. The performance is so bad compared to 2.6.33.7 that our product
> >> just plain won't work. We've already had remove features to the get
> >> 5.4 kernel down to a usable size, but we were prepared to live with
> >> that.]
> >
> > Ah. Small caches?
>
> Yep. It's An old Atmel ARM926T part (at91sam9g20) with 32KB I-cache
> and 32KB D-cache.
>
> A simple user-space multi-threaded TCP echo server benchmark showed a
> 30-50% (depending on number of parallel connections) drop in max
> throughput. Our typical applications also show a 15-25% increase in
> CPU usage for an equivalent workload. Another problem is high
> latencies with 5.4. A thread that is supposed to wake up every 1ms
> works reliably on 2.6.33, but is a long ways from working on 5.4.
We use the exact same SoC with kernel 4.9.220-rt143 (PREEMPT RT) currently,
after being stuck on 2.6.36.4 for quite a while. I did not notice significant
performance issues, but I have to admit, we never did extensive benchmarks on
network or CPU performance, because the workload also changed for that target.
However what gave us a lot less dropped packages was using the internal SRAM
as DMA buffer for RX packages received by macb. That did not make it in
mainline however, I did not put enough effort in polishing that patch back
when we migrated from 2.6 to 4.x. If you're curious, it's on GitHub:
https://github.com/LeSpocky/linux/tree/net-macb-sram-rx
> I asked on the arm kernel mailing list if this was typical/expected,
> but the post never made it past the moderator.
>
> > The OpenWRT guys make valid complaints that the code
> > hot path of the network stack is getting too big to fit in the cache
> > of small systems. So there is a lot of cache misses and performance is
> > not good. If i remember correctly, just having netfilter in the build
> > is bad, even if it is not used.
>
> We've already disabled absoltely everything we can and still have a
> working system. With the same features enabled, the 5.4 kernel was
> about 75% larger than a 2.6.33 kernel, so we had to trim quite a bit
> of meat to get it to boot on existing units.
Same here. v4.9 kernel image still fits, v4.14 is already too big for some
devices we delivered in the early days.
> We also can't get on-die ECC support for Micron NAND flash working
> with 5.4. Even it did work, we'd still have to add the ability to
> fall-back to SW ECC on read operations for the sake of backwards
> compatibility on units that were initially shipped without on-die ECC
> support enabled.
IIRC the SoC itself has issues with its ECC engine? See mainline
at91sam9g20ek_common.dtsi for example which sets nand-ecc-mode to "soft".
The SAM9G20 Errata chapter in the complete datasheet from 2015 (Atmel-6384F)
says two times in Section 44.1.3: "Perform the ECC computation by software."
Greets
Alex
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Alexander Dahl <ada@thorsis.com>
To: netdev@vger.kernel.org
Cc: Grant Edwards <grant.b.edwards@gmail.com>,
linux-arm-kernel@lists.infradead.org,
linux-mtd@lists.infradead.org
Subject: Re: net: macb: fail when there's no PHY
Date: Fri, 04 Dec 2020 09:28:53 +0100 [thread overview]
Message-ID: <3542036.FvJvBFsO4O@ada> (raw)
In-Reply-To: <rqbobm$5qk$1@ciao.gmane.io>
Hello Grant,
sorry if I just hijack your conversation, but I'm curious, because we are
using the same SoC. Adding linux-arm-kernel to Cc for the general performance
issues and linux-mtd for the ECC questions. O:-)
Am Donnerstag, 3. Dezember 2020, 23:20:38 CET schrieb Grant Edwards:
> On 2020-12-03, Andrew Lunn <andrew@lunn.ch> wrote:
> >> I don't think there's any way I could justify using a kernel that
> >> doesn't have long-term support.
> >
> > 5.10 is LTS. Well, it will be, once it is actually released!
>
> Convincing people to ship an unreleased kernel would be a whole
> 'nother bucket of worms.
+1
Judging just from the release dates of the last LTS kernels, I would have
guessed v5.11 will get LTS. But there has been no announcement yet and I
suppose there will be none before release? For ordinary users it's just like
staring in a crystal ball, so we aim at v5.4 for our more recent hardware
platforms. ¯\_(ツ)_/¯
>
> It's all moot now. The decision was just made to shelve the 5.4 kernel
> "upgrade" and stick with 2.6.33 for now.
>
> >> [It looks like we're going to have to abandon the effort to use
> >> 5.4. The performance is so bad compared to 2.6.33.7 that our product
> >> just plain won't work. We've already had remove features to the get
> >> 5.4 kernel down to a usable size, but we were prepared to live with
> >> that.]
> >
> > Ah. Small caches?
>
> Yep. It's An old Atmel ARM926T part (at91sam9g20) with 32KB I-cache
> and 32KB D-cache.
>
> A simple user-space multi-threaded TCP echo server benchmark showed a
> 30-50% (depending on number of parallel connections) drop in max
> throughput. Our typical applications also show a 15-25% increase in
> CPU usage for an equivalent workload. Another problem is high
> latencies with 5.4. A thread that is supposed to wake up every 1ms
> works reliably on 2.6.33, but is a long ways from working on 5.4.
We use the exact same SoC with kernel 4.9.220-rt143 (PREEMPT RT) currently,
after being stuck on 2.6.36.4 for quite a while. I did not notice significant
performance issues, but I have to admit, we never did extensive benchmarks on
network or CPU performance, because the workload also changed for that target.
However what gave us a lot less dropped packages was using the internal SRAM
as DMA buffer for RX packages received by macb. That did not make it in
mainline however, I did not put enough effort in polishing that patch back
when we migrated from 2.6 to 4.x. If you're curious, it's on GitHub:
https://github.com/LeSpocky/linux/tree/net-macb-sram-rx
> I asked on the arm kernel mailing list if this was typical/expected,
> but the post never made it past the moderator.
>
> > The OpenWRT guys make valid complaints that the code
> > hot path of the network stack is getting too big to fit in the cache
> > of small systems. So there is a lot of cache misses and performance is
> > not good. If i remember correctly, just having netfilter in the build
> > is bad, even if it is not used.
>
> We've already disabled absoltely everything we can and still have a
> working system. With the same features enabled, the 5.4 kernel was
> about 75% larger than a 2.6.33 kernel, so we had to trim quite a bit
> of meat to get it to boot on existing units.
Same here. v4.9 kernel image still fits, v4.14 is already too big for some
devices we delivered in the early days.
> We also can't get on-die ECC support for Micron NAND flash working
> with 5.4. Even it did work, we'd still have to add the ability to
> fall-back to SW ECC on read operations for the sake of backwards
> compatibility on units that were initially shipped without on-die ECC
> support enabled.
IIRC the SoC itself has issues with its ECC engine? See mainline
at91sam9g20ek_common.dtsi for example which sets nand-ecc-mode to "soft".
The SAM9G20 Errata chapter in the complete datasheet from 2015 (Atmel-6384F)
says two times in Section 44.1.3: "Perform the ECC computation by software."
Greets
Alex
next prev parent reply other threads:[~2020-12-04 8:30 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-21 19:59 net: macb: fail when there's no PHY Grant Edwards
2017-09-21 20:05 ` Florian Fainelli
2017-09-21 20:36 ` Grant Edwards
2017-09-21 21:35 ` Brandon Streiff
2017-09-29 7:05 ` Harini Katakam
[not found] ` <CAK=1mW6Gti0QpUjirB6PfMCiQvnDjkbb56pVKkQmpCSkRU6wtA@mail.gmail.com>
2020-12-02 18:10 ` Florian Fainelli
2020-12-02 18:24 ` Grant Edwards
2020-12-02 18:35 ` Andrew Lunn
2020-12-02 19:16 ` Grant Edwards
2020-12-02 21:11 ` Andrew Lunn
2020-12-02 21:23 ` Grant Edwards
2020-12-03 2:39 ` Andrew Lunn
2020-12-03 3:03 ` Grant Edwards
2020-12-03 3:42 ` Florian Fainelli
2020-12-03 3:54 ` Grant Edwards
2020-12-03 4:07 ` Andrew Lunn
2020-12-03 15:07 ` Grant Edwards
2020-12-03 21:17 ` Andrew Lunn
2020-12-03 21:39 ` Grant Edwards
2020-12-03 21:49 ` Andrew Lunn
2020-12-03 22:20 ` Grant Edwards
2020-12-04 8:28 ` Alexander Dahl [this message]
2020-12-04 8:28 ` Alexander Dahl
2020-12-04 8:28 ` Alexander Dahl
2020-12-04 13:42 ` Grant Edwards
2020-12-04 17:36 ` Andrew Lunn
2020-12-04 17:36 ` Andrew Lunn
2020-12-04 17:36 ` Andrew Lunn
2020-12-04 16:47 ` Florian Fainelli
2020-12-05 2:52 ` Grant Edwards
2020-12-05 3:06 ` Florian Fainelli
2020-12-03 4:00 ` Andrew Lunn
2020-12-02 18:10 ` Grant Edwards
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3542036.FvJvBFsO4O@ada \
--to=ada@thorsis.com \
--cc=grant.b.edwards@gmail.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-mtd@lists.infradead.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.