No care given to GDB scripts..

No care given to GDB scripts..

[Florian Fainelli]

Hello,

It is quite clear that there are zero cares being given to making sure 
that GDB scripts continue to work after making changes to core kernel 
code, and why would you, because you probably did not know those 
existed, but they do and they are used, and useful.

A recent example that was fixed by Kuan-Ying is this:

and now of course, "lx-interupts" also stopped working altogether after 
this change:

https://lore.kernel.org/r/20230519134902.1495562-4-sdonthineni@nvidia.com

and who knows what else I could not test that is also broken.

We really need to find a better way to stop breaking GDB scripts, they 
break way too often to be even remotely usable, and this is really sad.

It is also quite clear that we do not have enough continuous integration 
and regression testing to ensure those breakages are caught ahead of time...

</rant>
-- 
Florian

Re: No care given to GDB scripts..

[Andrew Morton]

On Wed, 29 Nov 2023 14:48:02 -0800 Florian Fainelli <f.fainelli@gmail.com> wrote:

> Hello,
> 
> It is quite clear that there are zero cares being given to making sure 
> that GDB scripts continue to work after making changes to core kernel 
> code, and why would you, because you probably did not know those 
> existed, but they do and they are used, and useful.
> 
> A recent example that was fixed by Kuan-Ying is this:
> 
> and now of course, "lx-interupts" also stopped working altogether after 
> this change:
> 
> https://lore.kernel.org/r/20230519134902.1495562-4-sdonthineni@nvidia.com
> 
> and who knows what else I could not test that is also broken.
> 
> We really need to find a better way to stop breaking GDB scripts, they 
> break way too often to be even remotely usable, and this is really sad.
> 
> It is also quite clear that we do not have enough continuous integration 
> and regression testing to ensure those breakages are caught ahead of time...
> 

This isn't terribly surprising - the gdb scripts are a pretty remote
corner and are peculiarly sensitive to getting damaged by routine
kernel development.

Is there any way of scripting the scripts so we can have some sort of
automated testing down under tools/testing/selftests/?

Re: No care given to GDB scripts..

[Florian Fainelli]

On 11/29/23 15:06, Andrew Morton wrote:
> On Wed, 29 Nov 2023 14:48:02 -0800 Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>> Hello,
>>
>> It is quite clear that there are zero cares being given to making sure
>> that GDB scripts continue to work after making changes to core kernel
>> code, and why would you, because you probably did not know those
>> existed, but they do and they are used, and useful.
>>
>> A recent example that was fixed by Kuan-Ying is this:
>>
>> and now of course, "lx-interupts" also stopped working altogether after
>> this change:
>>
>> https://lore.kernel.org/r/20230519134902.1495562-4-sdonthineni@nvidia.com
>>
>> and who knows what else I could not test that is also broken.
>>
>> We really need to find a better way to stop breaking GDB scripts, they
>> break way too often to be even remotely usable, and this is really sad.
>>
>> It is also quite clear that we do not have enough continuous integration
>> and regression testing to ensure those breakages are caught ahead of time...
>>
> 
> This isn't terribly surprising - the gdb scripts are a pretty remote
> corner and are peculiarly sensitive to getting damaged by routine
> kernel development.
> 
> Is there any way of scripting the scripts so we can have some sort of
> automated testing down under tools/testing/selftests/?

That might be a bit difficult to do as this would mean that we can self 
debug and introspect using gdb the live kernel. Testing using QEMU is 
definitively doable however. Of course, I just found another script that 
broke (device.py)!
-- 
Florian

Re: No care given to GDB scripts..

[Andrew Morton]

On Wed, 29 Nov 2023 16:23:55 -0800 Florian Fainelli <f.fainelli@gmail.com> wrote:

> > Is there any way of scripting the scripts so we can have some sort of
> > automated testing down under tools/testing/selftests/?
> 
> That might be a bit difficult to do as this would mean that we can self 
> debug and introspect using gdb the live kernel. Testing using QEMU is 
> definitively doable however. Of course, I just found another script that 
> broke (device.py)!

Oh.  That sounds quite the exercise in the context of tools/selftests.

One can of course just fire up gdb against the vmlinux elf file and
play around, but I assume this won't permit a useful amount of the
scripts to be exercised.

Suppose someone has set up qemu or whatever and has attached gdb to it.
Could we provide that person with a script or a canned set of commands
which exercise the gdb scripts to a useful extent?  So rather than
attempting automated testing under the seltests umbrella, we provide
less skilled individuals with the means to easily and quickly check for
regressions.  I'd expect that such a toolset would be particularly
helpful for regression testing the scripts across all the supported
architectures?

Re: No care given to GDB scripts..

[Florian Fainelli]

On 11/29/2023 5:02 PM, Andrew Morton wrote:
> On Wed, 29 Nov 2023 16:23:55 -0800 Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>>> Is there any way of scripting the scripts so we can have some sort of
>>> automated testing down under tools/testing/selftests/?
>>
>> That might be a bit difficult to do as this would mean that we can self
>> debug and introspect using gdb the live kernel. Testing using QEMU is
>> definitively doable however. Of course, I just found another script that
>> broke (device.py)!
> 
> Oh.  That sounds quite the exercise in the context of tools/selftests.
> 
> One can of course just fire up gdb against the vmlinux elf file and
> play around, but I assume this won't permit a useful amount of the
> scripts to be exercised.
> 
> Suppose someone has set up qemu or whatever and has attached gdb to it.
> Could we provide that person with a script or a canned set of commands
> which exercise the gdb scripts to a useful extent?  So rather than
> attempting automated testing under the seltests umbrella, we provide
> less skilled individuals with the means to easily and quickly check for
> regressions.  I'd expect that such a toolset would be particularly
> helpful for regression testing the scripts across all the supported
> architectures?

Yes it would, let me play with what GDB and QEMU can do and see if this 
can be somewhat automated.

Thanks!
-- 
Florian