Re: [TEST] mptcp-connect

Re: [TEST] mptcp-connect

[Matthieu Baerts]

Hi Jakub,

(+cc MPTCP list)

On 07/01/2025 22:18, Jakub Kicinski wrote:
> Unfortunately mptcp_connect.sh has started flaking again :(
> Looks like it started around Dec 30th, so one of the recent PRs

Thank you for the heads-up!

After the last PR, I was focussing on monitoring simult_flows.sh -- the
recent behaviour changes could have affected it -- and forgot to look at
the others, sorry about that...

It looks like our CI doesn't have this issue, but the builds are less
frequent [1]. I'm going to investigate that ASAP.

[1] https://ci-results.mptcp.dev/flakes.html?branch=export-net

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [TEST] mptcp-connect

[Matthieu Baerts]

Hi Jakub,

On 08/01/2025 11:07, Matthieu Baerts wrote:
> Hi Jakub,
> 
> (+cc MPTCP list)
> 
> On 07/01/2025 22:18, Jakub Kicinski wrote:
>> Unfortunately mptcp_connect.sh has started flaking again :(
>> Looks like it started around Dec 30th, so one of the recent PRs
> 
> Thank you for the heads-up!
> 
> After the last PR, I was focussing on monitoring simult_flows.sh -- the
> recent behaviour changes could have affected it -- and forgot to look at
> the others, sorry about that...
> 
> It looks like our CI doesn't have this issue, but the builds are less
> frequent [1]. I'm going to investigate that ASAP.

It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!

Out of curiosity, is the netdev CI not too overloaded? To reproduce this
issue on my side, the host had to be quite busy: not dying with
stress-ng using all resources, but still competing with many other
processes, e.g. a kernel compilation running in parallel. I'm not
complaining here, because this situation helped finding this important
issue, but just curious about what to expect, especially for more
"sensitive" tests :)

On the MPTCP CI, some unessarry KConfig are disabled and ccache is used
to reduce the build time. Also, RETPOLINE is disabled (+ vng --append
mitigations=off) to save some CPU cycles during the tests.

[1]
https://lore.kernel.org/netdev/20250113-net-mptcp-connect-st-flakes-v1-0-0d986ee7b1b6@kernel.org/T/

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [TEST] mptcp-connect

[Jakub Kicinski]

On Mon, 13 Jan 2025 17:08:02 +0100 Matthieu Baerts wrote:
> It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!
> 
> Out of curiosity, is the netdev CI not too overloaded? To reproduce this
> issue on my side, the host had to be quite busy: not dying with
> stress-ng using all resources, but still competing with many other
> processes, e.g. a kernel compilation running in parallel. I'm not
> complaining here, because this situation helped finding this important
> issue, but just curious about what to expect, especially for more
> "sensitive" tests :)

The tests my overlap with other workers still building their kernel.
ccache is enabled.

> On the MPTCP CI, some unessarry KConfig are disabled and ccache is used
> to reduce the build time. Also, RETPOLINE is disabled (+ vng --append
> mitigations=off) to save some CPU cycles during the tests.

Could help. Let's put it on the list of things to do once we migrate
the CI to a public set of hosts :)

Re: [TEST] mptcp-connect

[Matthieu Baerts]

Hi Jakub,

On 13/01/2025 20:44, Jakub Kicinski wrote:
> On Mon, 13 Jan 2025 17:08:02 +0100 Matthieu Baerts wrote:
>> It was not easy to reproduce it, but Paolo managed to find a fix for it [1]!
>>
>> Out of curiosity, is the netdev CI not too overloaded? To reproduce this
>> issue on my side, the host had to be quite busy: not dying with
>> stress-ng using all resources, but still competing with many other
>> processes, e.g. a kernel compilation running in parallel. I'm not
>> complaining here, because this situation helped finding this important
>> issue, but just curious about what to expect, especially for more
>> "sensitive" tests :)
> 
> The tests my overlap with other workers still building their kernel.
> ccache is enabled.

Good to know, thank you.

>> On the MPTCP CI, some unessarry KConfig are disabled and ccache is used
>> to reduce the build time. Also, RETPOLINE is disabled (+ vng --append
>> mitigations=off) to save some CPU cycles during the tests.
> 
> Could help. Let's put it on the list of things to do once we migrate
> the CI to a public set of hosts :)

Good idea :)

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.