[PATCH v2 0/5] devtool: ide: Improve VSCode support

[PATCH v2 0/5] devtool: ide: Improve VSCode support

[Enguerrand de Ribaucourt]

These patches improve the VSCode support in devtool ide-sdk from Adrian
Freihofer.

I added a generic C++ configuration for the VScode extension while
awaiting for autotools support.

A refactoring is proposed for the meson class. Without absolute compiler
paths, the linter inside VSCode will not be able to find the
cross-compiler. Let me know if you have any concerns about this
refactoring.

The other bug fixes are relatively minor and make the tool work for some
edge cases.

v2:
 - Added signed-off-by mentions
 - Removed duplicate topdir exception fix
 - Added a condition for the generic C++ configuration (matching autotools)
 - Kept the source mapping for debug source as it was required to not open the
   local files instead of the read-only files in the editor on some
   configurations

[PATCH v2 1/5] devtool: ide_sdk: Use bitbake's python3 for generated scripts

[Enguerrand de Ribaucourt]

The generated scripts use the sys.path configuration found inside
bitbake. It can be a different python version than the one used on the
host through the IDE.

For instance, when running the generated script
deploy_target_cmake-example-core2-64 from an eSDK generated on another
machine, I got the following exception:
    AssertionError: SRE module mismatch

We need to match the sys.executable to the sys.path.

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
---
 scripts/lib/devtool/ide_sdk.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 3986dc1436a..14679744807 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -750,7 +750,7 @@ class RecipeModified:
         does not need to start a bitbake server. All information from tinfoil
         is hard-coded in the generated script.
         """
-        cmd_lines = ['#!/usr/bin/env python3']
+        cmd_lines = ['#!%s' % str(sys.executable)]
         cmd_lines.append('import sys')
         cmd_lines.append('devtool_sys_path = %s' % str(sys.path))
         cmd_lines.append('devtool_sys_path.reverse()')
-- 
2.34.1

Re: [PATCH v2 1/5] devtool: ide_sdk: Use bitbake's python3 for generated scripts

[adrian.freihofer]

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
> The generated scripts use the sys.path configuration found inside
> bitbake. It can be a different python version than the one used on
> the
> host through the IDE.
> 
> For instance, when running the generated script
> deploy_target_cmake-example-core2-64 from an eSDK generated on
> another
> machine, I got the following exception:
>     AssertionError: SRE module mismatch
> 
> We need to match the sys.executable to the sys.path.
> 
> Signed-off-by: Enguerrand de Ribaucourt
> <enguerrand.de-ribaucourt@savoirfairelinux.com>
> ---
>  scripts/lib/devtool/ide_sdk.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/lib/devtool/ide_sdk.py
> b/scripts/lib/devtool/ide_sdk.py
> index 3986dc1436a..14679744807 100755
> --- a/scripts/lib/devtool/ide_sdk.py
> +++ b/scripts/lib/devtool/ide_sdk.py
> @@ -750,7 +750,7 @@ class RecipeModified:
>          does not need to start a bitbake server. All information
> from tinfoil
>          is hard-coded in the generated script.
>          """
> -        cmd_lines = ['#!/usr/bin/env python3']
> +        cmd_lines = ['#!%s' % str(sys.executable)]
>          cmd_lines.append('import sys')
>          cmd_lines.append('devtool_sys_path = %s' % str(sys.path))
>          cmd_lines.append('devtool_sys_path.reverse()')

LGTM.

Adrian

[PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

When launching the debug configuration, the source files from the debug
rootfs were openened in the editor instead of the local workspace files.
We add an exception to properly map them to the file being developed and
compiled by the IDE integration. This also more closely matches what the
user would expect compared to native development.

This is also true for the devtool fallback mode.

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
---
 scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
 scripts/lib/devtool/ide_sdk.py              | 1 +
 2 files changed, 2 insertions(+)

diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index b2193130d2e..c063b7d0590 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
         if gdb_cross_config.image_recipe.rootfs_dbg:
             launch_config['additionalSOLibSearchPath'] = modified_recipe.solib_search_path_str(
                 gdb_cross_config.image_recipe)
+            src_file_map[os.path.join("/usr/src/debug", modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
             src_file_map["/usr/src/debug"] = os.path.join(
                 gdb_cross_config.image_recipe.rootfs_dbg, "usr", "src", "debug")
         else:
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 14679744807..f292edbe25c 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -356,6 +356,7 @@ class RecipeModified:
             'PACKAGE_DEBUG_SPLIT_STYLE')
         self.path = recipe_d.getVar('PATH')
         self.pn = recipe_d.getVar('PN')
+        self.pv = recipe_d.getVar('PV')
         self.recipe_sysroot = os.path.realpath(
             recipe_d.getVar('RECIPE_SYSROOT'))
         self.recipe_sysroot_native = os.path.realpath(
-- 
2.34.1

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[adrian.freihofer]

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
> When launching the debug configuration, the source files from the
> debug
> rootfs were openened in the editor instead of the local workspace
> files.
> We add an exception to properly map them to the file being developed
> and
> compiled by the IDE integration. This also more closely matches what
> the
> user would expect compared to native development.
> 
> This is also true for the devtool fallback mode.

This looks still wrong to me. If files from the rootfs are openend for
a recipe which is in the workspace, the SDK is broken and we need to
understand and fix that. This patch does not solve that, but will make
it much harder to find the right solution for this issue.

Adrian

> 
> Signed-off-by: Enguerrand de Ribaucourt
> <enguerrand.de-ribaucourt@savoirfairelinux.com>
> ---
>  scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
>  scripts/lib/devtool/ide_sdk.py              | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
> b/scripts/lib/devtool/ide_plugins/ide_code.py
> index b2193130d2e..c063b7d0590 100644
> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
> @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
>          if gdb_cross_config.image_recipe.rootfs_dbg:
>              launch_config['additionalSOLibSearchPath'] =
> modified_recipe.solib_search_path_str(
>                  gdb_cross_config.image_recipe)
> +            src_file_map[os.path.join("/usr/src/debug",
> modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
>              src_file_map["/usr/src/debug"] = os.path.join(
>                  gdb_cross_config.image_recipe.rootfs_dbg, "usr",
> "src", "debug")
>          else:
> diff --git a/scripts/lib/devtool/ide_sdk.py
> b/scripts/lib/devtool/ide_sdk.py
> index 14679744807..f292edbe25c 100755
> --- a/scripts/lib/devtool/ide_sdk.py
> +++ b/scripts/lib/devtool/ide_sdk.py
> @@ -356,6 +356,7 @@ class RecipeModified:
>              'PACKAGE_DEBUG_SPLIT_STYLE')
>          self.path = recipe_d.getVar('PATH')
>          self.pn = recipe_d.getVar('PN')
> +        self.pv = recipe_d.getVar('PV')
>          self.recipe_sysroot = os.path.realpath(
>              recipe_d.getVar('RECIPE_SYSROOT'))
>          self.recipe_sysroot_native = os.path.realpath(

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

On 20/02/2024 10:01, adrian.freihofer@gmail.com wrote:
> On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
>> When launching the debug configuration, the source files from the
>> debug
>> rootfs were openened in the editor instead of the local workspace
>> files.
>> We add an exception to properly map them to the file being developed
>> and
>> compiled by the IDE integration. This also more closely matches what
>> the
>> user would expect compared to native development.
>>
>> This is also true for the devtool fallback mode.
> 
> This looks still wrong to me. If files from the rootfs are openend for
> a recipe which is in the workspace, the SDK is broken and we need to
> understand and fix that. This patch does not solve that, but will make
> it much harder to find the right solution for this issue.
Hi Adrian,

This is affecting essentially the devtool fallback mode. Since it's 
compiled through devtool, it's using the bitbake packaging information.

Steps to reproduce:
  - devtool modify powertop (autotools recipe, currently fallback)
  - devtool ide-sdk powertop [args...]
  - open the powertop sources workspace
  - run the debug launch configuration
  - initial breakpoint opens the rootfs-dbg main.cpp instead of the 
current workspace

This probably will get fixed for autotools with your specific class 
support, but will remain for the fallback mode.

Do we want to keep this source map, but only for the fallback mode? Like 
you say, it's kinda "cheating" the debug information, but since we are 
in fallback mode, the devtool packaging is supposed to be using the 
sources from the workspace, so it's more like "hinting" at the proper 
source.
> 
> Adrian
> 
>>
>> Signed-off-by: Enguerrand de Ribaucourt
>> <enguerrand.de-ribaucourt@savoirfairelinux.com>
>> ---
>>   scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
>>   scripts/lib/devtool/ide_sdk.py              | 1 +
>>   2 files changed, 2 insertions(+)
>>
>> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
>> b/scripts/lib/devtool/ide_plugins/ide_code.py
>> index b2193130d2e..c063b7d0590 100644
>> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
>> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
>> @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
>>           if gdb_cross_config.image_recipe.rootfs_dbg:
>>               launch_config['additionalSOLibSearchPath'] =
>> modified_recipe.solib_search_path_str(
>>                   gdb_cross_config.image_recipe)
>> +            src_file_map[os.path.join("/usr/src/debug",
>> modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
>>               src_file_map["/usr/src/debug"] = os.path.join(
>>                   gdb_cross_config.image_recipe.rootfs_dbg, "usr",
>> "src", "debug")
>>           else:
>> diff --git a/scripts/lib/devtool/ide_sdk.py
>> b/scripts/lib/devtool/ide_sdk.py
>> index 14679744807..f292edbe25c 100755
>> --- a/scripts/lib/devtool/ide_sdk.py
>> +++ b/scripts/lib/devtool/ide_sdk.py
>> @@ -356,6 +356,7 @@ class RecipeModified:
>>               'PACKAGE_DEBUG_SPLIT_STYLE')
>>           self.path = recipe_d.getVar('PATH')
>>           self.pn = recipe_d.getVar('PN')
>> +        self.pv = recipe_d.getVar('PV')
>>           self.recipe_sysroot = os.path.realpath(
>>               recipe_d.getVar('RECIPE_SYSROOT'))
>>           self.recipe_sysroot_native = os.path.realpath(
> 
Enguerrand de Ribaucourt

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

On 21/02/2024 15:12, Enguerrand de Ribaucourt wrote:
> 
> 
> On 20/02/2024 10:01, adrian.freihofer@gmail.com wrote:
>> On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
>>> When launching the debug configuration, the source files from the
>>> debug
>>> rootfs were openened in the editor instead of the local workspace
>>> files.
>>> We add an exception to properly map them to the file being developed
>>> and
>>> compiled by the IDE integration. This also more closely matches what
>>> the
>>> user would expect compared to native development.
>>>
>>> This is also true for the devtool fallback mode.
>>
>> This looks still wrong to me. If files from the rootfs are openend for
>> a recipe which is in the workspace, the SDK is broken and we need to
>> understand and fix that. This patch does not solve that, but will make
>> it much harder to find the right solution for this issue.
> Hi Adrian,
> 
> This is affecting essentially the devtool fallback mode. Since it's 
> compiled through devtool, it's using the bitbake packaging information.
> 
> Steps to reproduce:
>   - devtool modify powertop (autotools recipe, currently fallback)
>   - devtool ide-sdk powertop [args...]
>   - open the powertop sources workspace
>   - run the debug launch configuration
>   - initial breakpoint opens the rootfs-dbg main.cpp instead of the 
> current workspace
> 
> This probably will get fixed for autotools with your specific class 
> support, but will remain for the fallback mode.
> 
> Do we want to keep this source map, but only for the fallback mode? Like 
> you say, it's kinda "cheating" the debug information, but since we are 
> in fallback mode, the devtool packaging is supposed to be using the 
> sources from the workspace, so it's more like "hinting" at the proper 
> source.
It also affects more meson projects like lighttpd. The "hint" would 
still be valid given how it's built and deployed now. If you are still 
working on an alternative tinfoil deploy method, maybe it's not affected.
>>
>> Adrian
>>
>>>
>>> Signed-off-by: Enguerrand de Ribaucourt
>>> <enguerrand.de-ribaucourt@savoirfairelinux.com>
>>> ---
>>>   scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
>>>   scripts/lib/devtool/ide_sdk.py              | 1 +
>>>   2 files changed, 2 insertions(+)
>>>
>>> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
>>> b/scripts/lib/devtool/ide_plugins/ide_code.py
>>> index b2193130d2e..c063b7d0590 100644
>>> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
>>> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
>>> @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
>>>           if gdb_cross_config.image_recipe.rootfs_dbg:
>>>               launch_config['additionalSOLibSearchPath'] =
>>> modified_recipe.solib_search_path_str(
>>>                   gdb_cross_config.image_recipe)
>>> +            src_file_map[os.path.join("/usr/src/debug",
>>> modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
>>>               src_file_map["/usr/src/debug"] = os.path.join(
>>>                   gdb_cross_config.image_recipe.rootfs_dbg, "usr",
>>> "src", "debug")
>>>           else:
>>> diff --git a/scripts/lib/devtool/ide_sdk.py
>>> b/scripts/lib/devtool/ide_sdk.py
>>> index 14679744807..f292edbe25c 100755
>>> --- a/scripts/lib/devtool/ide_sdk.py
>>> +++ b/scripts/lib/devtool/ide_sdk.py
>>> @@ -356,6 +356,7 @@ class RecipeModified:
>>>               'PACKAGE_DEBUG_SPLIT_STYLE')
>>>           self.path = recipe_d.getVar('PATH')
>>>           self.pn = recipe_d.getVar('PN')
>>> +        self.pv = recipe_d.getVar('PV')
>>>           self.recipe_sysroot = os.path.realpath(
>>>               recipe_d.getVar('RECIPE_SYSROOT'))
>>>           self.recipe_sysroot_native = os.path.realpath(
>>
> Enguerrand de Ribaucourt
> 
Enguerrand de Ribaucourt

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[adrian.freihofer]

On Wed, 2024-02-21 at 15:56 +0100, Enguerrand de Ribaucourt wrote:
> 
> 
> On 21/02/2024 15:12, Enguerrand de Ribaucourt wrote:
> > 
> > 
> > On 20/02/2024 10:01, adrian.freihofer@gmail.com wrote:
> > > On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt
> > > wrote:
> > > > When launching the debug configuration, the source files from
> > > > the
> > > > debug
> > > > rootfs were openened in the editor instead of the local
> > > > workspace
> > > > files.
> > > > We add an exception to properly map them to the file being
> > > > developed
> > > > and
> > > > compiled by the IDE integration. This also more closely matches
> > > > what
> > > > the
> > > > user would expect compared to native development.
> > > > 
> > > > This is also true for the devtool fallback mode.
> > > 
> > > This looks still wrong to me. If files from the rootfs are
> > > openend for
> > > a recipe which is in the workspace, the SDK is broken and we need
> > > to
> > > understand and fix that. This patch does not solve that, but will
> > > make
> > > it much harder to find the right solution for this issue.
> > Hi Adrian,
> > 
> > This is affecting essentially the devtool fallback mode. Since it's
> > compiled through devtool, it's using the bitbake packaging
> > information.
> > 
> > Steps to reproduce:
> >   - devtool modify powertop (autotools recipe, currently fallback)
> >   - devtool ide-sdk powertop [args...]
> >   - open the powertop sources workspace
> >   - run the debug launch configuration
> >   - initial breakpoint opens the rootfs-dbg main.cpp instead of the
> > current workspace
> > 
> > This probably will get fixed for autotools with your specific class
> > support, but will remain for the fallback mode.
> > 
> > Do we want to keep this source map, but only for the fallback mode?
> > Like 
> > you say, it's kinda "cheating" the debug information, but since we
> > are 
> > in fallback mode, the devtool packaging is supposed to be using the
> > sources from the workspace, so it's more like "hinting" at the
> > proper 
> > source.
> It also affects more meson projects like lighttpd. The "hint" would 
> still be valid given how it's built and deployed now. If you are
> still 
> working on an alternative tinfoil deploy method, maybe it's not
> affected.

It looks like the devtool deploy-target step is missing. If you debug
packaged files this would behave like this. It's important to over
write the packaged files with the not packaged file from the image
folder. This is what the devtool deploy-target command does. It's
expected that this just happens, probably there is a bug in the
dependencies. Could you please try to run it explicit? Something like
this:

- devtool modify powertop (autotools recipe, currently fallback)
- devtool ide-sdk powertop [args...]

- devtool build powertop
- devtool deploy-target powertop

- open the powertop sources workspace
- run the debug launch configuration

I guess this would work as expected. The question is then: Why is the
deploy not executed?

Adrian


> > > 
> > > Adrian
> > > 
> > > > 
> > > > Signed-off-by: Enguerrand de Ribaucourt
> > > > <enguerrand.de-ribaucourt@savoirfairelinux.com>
> > > > ---
> > > >   scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
> > > >   scripts/lib/devtool/ide_sdk.py              | 1 +
> > > >   2 files changed, 2 insertions(+)
> > > > 
> > > > diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > b/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > index b2193130d2e..c063b7d0590 100644
> > > > --- a/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
> > > >           if gdb_cross_config.image_recipe.rootfs_dbg:
> > > >               launch_config['additionalSOLibSearchPath'] =
> > > > modified_recipe.solib_search_path_str(
> > > >                   gdb_cross_config.image_recipe)
> > > > +            src_file_map[os.path.join("/usr/src/debug",
> > > > modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
> > > >               src_file_map["/usr/src/debug"] = os.path.join(
> > > >                   gdb_cross_config.image_recipe.rootfs_dbg,
> > > > "usr",
> > > > "src", "debug")
> > > >           else:
> > > > diff --git a/scripts/lib/devtool/ide_sdk.py
> > > > b/scripts/lib/devtool/ide_sdk.py
> > > > index 14679744807..f292edbe25c 100755
> > > > --- a/scripts/lib/devtool/ide_sdk.py
> > > > +++ b/scripts/lib/devtool/ide_sdk.py
> > > > @@ -356,6 +356,7 @@ class RecipeModified:
> > > >               'PACKAGE_DEBUG_SPLIT_STYLE')
> > > >           self.path = recipe_d.getVar('PATH')
> > > >           self.pn = recipe_d.getVar('PN')
> > > > +        self.pv = recipe_d.getVar('PV')
> > > >           self.recipe_sysroot = os.path.realpath(
> > > >               recipe_d.getVar('RECIPE_SYSROOT'))
> > > >           self.recipe_sysroot_native = os.path.realpath(
> > > 
> > Enguerrand de Ribaucourt
> > 
> Enguerrand de Ribaucourt

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

On 22/02/2024 14:00, adrian.freihofer@gmail.com wrote:
> On Wed, 2024-02-21 at 15:56 +0100, Enguerrand de Ribaucourt wrote:
>>
>>
>> On 21/02/2024 15:12, Enguerrand de Ribaucourt wrote:
>>>
>>>
>>> On 20/02/2024 10:01, adrian.freihofer@gmail.com wrote:
>>>> On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt
>>>> wrote:
>>>>> When launching the debug configuration, the source files from
>>>>> the
>>>>> debug
>>>>> rootfs were openened in the editor instead of the local
>>>>> workspace
>>>>> files.
>>>>> We add an exception to properly map them to the file being
>>>>> developed
>>>>> and
>>>>> compiled by the IDE integration. This also more closely matches
>>>>> what
>>>>> the
>>>>> user would expect compared to native development.
>>>>>
>>>>> This is also true for the devtool fallback mode.
>>>>
>>>> This looks still wrong to me. If files from the rootfs are
>>>> openend for
>>>> a recipe which is in the workspace, the SDK is broken and we need
>>>> to
>>>> understand and fix that. This patch does not solve that, but will
>>>> make
>>>> it much harder to find the right solution for this issue.
>>> Hi Adrian,
>>>
>>> This is affecting essentially the devtool fallback mode. Since it's
>>> compiled through devtool, it's using the bitbake packaging
>>> information.
>>>
>>> Steps to reproduce:
>>>    - devtool modify powertop (autotools recipe, currently fallback)
>>>    - devtool ide-sdk powertop [args...]
>>>    - open the powertop sources workspace
>>>    - run the debug launch configuration
>>>    - initial breakpoint opens the rootfs-dbg main.cpp instead of the
>>> current workspace
>>>
>>> This probably will get fixed for autotools with your specific class
>>> support, but will remain for the fallback mode.
>>>
>>> Do we want to keep this source map, but only for the fallback mode?
>>> Like
>>> you say, it's kinda "cheating" the debug information, but since we
>>> are
>>> in fallback mode, the devtool packaging is supposed to be using the
>>> sources from the workspace, so it's more like "hinting" at the
>>> proper
>>> source.
>> It also affects more meson projects like lighttpd. The "hint" would
>> still be valid given how it's built and deployed now. If you are
>> still
>> working on an alternative tinfoil deploy method, maybe it's not
>> affected.
> 
> It looks like the devtool deploy-target step is missing. If you debug
> packaged files this would behave like this. It's important to over
> write the packaged files with the not packaged file from the image
> folder. This is what the devtool deploy-target command does. It's
> expected that this just happens, probably there is a bug in the
> dependencies. Could you please try to run it explicit? Something like
> this:
> 
> - devtool modify powertop (autotools recipe, currently fallback)
> - devtool ide-sdk powertop [args...]
> 
> - devtool build powertop
> - devtool deploy-target powertop
> 
> - open the powertop sources workspace
> - run the debug launch configuration
> 
> I guess this would work as expected. The question is then: Why is the
> deploy not executed?
The deploy-target is actually executed automatically through the launch 
configuration. I also get the same result if I deploy manually. I 
believe that the problem is rather how "devtool build" compiles and 
packages the binary.
> 
> Adrian
> 
> 
>>>>
>>>> Adrian
>>>>
>>>>>
>>>>> Signed-off-by: Enguerrand de Ribaucourt
>>>>> <enguerrand.de-ribaucourt@savoirfairelinux.com>
>>>>> ---
>>>>>    scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
>>>>>    scripts/lib/devtool/ide_sdk.py              | 1 +
>>>>>    2 files changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
>>>>> b/scripts/lib/devtool/ide_plugins/ide_code.py
>>>>> index b2193130d2e..c063b7d0590 100644
>>>>> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
>>>>> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
>>>>> @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
>>>>>            if gdb_cross_config.image_recipe.rootfs_dbg:
>>>>>                launch_config['additionalSOLibSearchPath'] =
>>>>> modified_recipe.solib_search_path_str(
>>>>>                    gdb_cross_config.image_recipe)
>>>>> +            src_file_map[os.path.join("/usr/src/debug",
>>>>> modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
>>>>>                src_file_map["/usr/src/debug"] = os.path.join(
>>>>>                    gdb_cross_config.image_recipe.rootfs_dbg,
>>>>> "usr",
>>>>> "src", "debug")
>>>>>            else:
>>>>> diff --git a/scripts/lib/devtool/ide_sdk.py
>>>>> b/scripts/lib/devtool/ide_sdk.py
>>>>> index 14679744807..f292edbe25c 100755
>>>>> --- a/scripts/lib/devtool/ide_sdk.py
>>>>> +++ b/scripts/lib/devtool/ide_sdk.py
>>>>> @@ -356,6 +356,7 @@ class RecipeModified:
>>>>>                'PACKAGE_DEBUG_SPLIT_STYLE')
>>>>>            self.path = recipe_d.getVar('PATH')
>>>>>            self.pn = recipe_d.getVar('PN')
>>>>> +        self.pv = recipe_d.getVar('PV')
>>>>>            self.recipe_sysroot = os.path.realpath(
>>>>>                recipe_d.getVar('RECIPE_SYSROOT'))
>>>>>            self.recipe_sysroot_native = os.path.realpath(
>>>>
>>> Enguerrand de Ribaucourt
>>>
>> Enguerrand de Ribaucourt
> 

Enguerrand de Ribaucourt

Re: [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[adrian.freihofer]

On Thu, 2024-02-22 at 16:51 +0100, Enguerrand de Ribaucourt wrote:
> 
> 
> On 22/02/2024 14:00, adrian.freihofer@gmail.com wrote:
> > On Wed, 2024-02-21 at 15:56 +0100, Enguerrand de Ribaucourt wrote:
> > > 
> > > 
> > > On 21/02/2024 15:12, Enguerrand de Ribaucourt wrote:
> > > > 
> > > > 
> > > > On 20/02/2024 10:01, adrian.freihofer@gmail.com wrote:
> > > > > On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt
> > > > > wrote:
> > > > > > When launching the debug configuration, the source files
> > > > > > from
> > > > > > the
> > > > > > debug
> > > > > > rootfs were openened in the editor instead of the local
> > > > > > workspace
> > > > > > files.
> > > > > > We add an exception to properly map them to the file being
> > > > > > developed
> > > > > > and
> > > > > > compiled by the IDE integration. This also more closely
> > > > > > matches
> > > > > > what
> > > > > > the
> > > > > > user would expect compared to native development.
> > > > > > 
> > > > > > This is also true for the devtool fallback mode.
> > > > > 
> > > > > This looks still wrong to me. If files from the rootfs are
> > > > > openend for
> > > > > a recipe which is in the workspace, the SDK is broken and we
> > > > > need
> > > > > to
> > > > > understand and fix that. This patch does not solve that, but
> > > > > will
> > > > > make
> > > > > it much harder to find the right solution for this issue.
> > > > Hi Adrian,
> > > > 
> > > > This is affecting essentially the devtool fallback mode. Since
> > > > it's
> > > > compiled through devtool, it's using the bitbake packaging
> > > > information.
> > > > 
> > > > Steps to reproduce:
> > > >    - devtool modify powertop (autotools recipe, currently
> > > > fallback)
> > > >    - devtool ide-sdk powertop [args...]
> > > >    - open the powertop sources workspace
> > > >    - run the debug launch configuration
> > > >    - initial breakpoint opens the rootfs-dbg main.cpp instead
> > > > of the
> > > > current workspace
> > > > 
> > > > This probably will get fixed for autotools with your specific
> > > > class
> > > > support, but will remain for the fallback mode.
> > > > 
> > > > Do we want to keep this source map, but only for the fallback
> > > > mode?
> > > > Like
> > > > you say, it's kinda "cheating" the debug information, but since
> > > > we
> > > > are
> > > > in fallback mode, the devtool packaging is supposed to be using
> > > > the
> > > > sources from the workspace, so it's more like "hinting" at the
> > > > proper
> > > > source.
> > > It also affects more meson projects like lighttpd. The "hint"
> > > would
> > > still be valid given how it's built and deployed now. If you are
> > > still
> > > working on an alternative tinfoil deploy method, maybe it's not
> > > affected.
> > 
> > It looks like the devtool deploy-target step is missing. If you
> > debug
> > packaged files this would behave like this. It's important to over
> > write the packaged files with the not packaged file from the image
> > folder. This is what the devtool deploy-target command does. It's
> > expected that this just happens, probably there is a bug in the
> > dependencies. Could you please try to run it explicit? Something
> > like
> > this:
> > 
> > - devtool modify powertop (autotools recipe, currently fallback)
> > - devtool ide-sdk powertop [args...]
> > 
> > - devtool build powertop
> > - devtool deploy-target powertop
> > 
> > - open the powertop sources workspace
> > - run the debug launch configuration
> > 
> > I guess this would work as expected. The question is then: Why is
> > the
> > deploy not executed?
> The deploy-target is actually executed automatically through the
> launch 
> configuration. I also get the same result if I deploy manually. I 
> believe that the problem is rather how "devtool build" compiles and 
> packages the binary.

You are right. We need this additional source mapping. I found some
time to look into the details. I propose to go with these 4 patches
(including your patch as well):
https://lists.openembedded.org/g/openembedded-core/message/196146

Adrian

> > 
> > Adrian
> > 
> > 
> > > > > 
> > > > > Adrian
> > > > > 
> > > > > > 
> > > > > > Signed-off-by: Enguerrand de Ribaucourt
> > > > > > <enguerrand.de-ribaucourt@savoirfairelinux.com>
> > > > > > ---
> > > > > >    scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
> > > > > >    scripts/lib/devtool/ide_sdk.py              | 1 +
> > > > > >    2 files changed, 2 insertions(+)
> > > > > > 
> > > > > > diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > > > b/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > > > index b2193130d2e..c063b7d0590 100644
> > > > > > --- a/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > > > +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
> > > > > > @@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
> > > > > >            if gdb_cross_config.image_recipe.rootfs_dbg:
> > > > > >                launch_config['additionalSOLibSearchPath'] =
> > > > > > modified_recipe.solib_search_path_str(
> > > > > >                    gdb_cross_config.image_recipe)
> > > > > > +            src_file_map[os.path.join("/usr/src/debug",
> > > > > > modified_recipe.pn, modified_recipe.pv)] =
> > > > > > "${workspaceFolder}"
> > > > > >                src_file_map["/usr/src/debug"] =
> > > > > > os.path.join(
> > > > > >                   
> > > > > > gdb_cross_config.image_recipe.rootfs_dbg,
> > > > > > "usr",
> > > > > > "src", "debug")
> > > > > >            else:
> > > > > > diff --git a/scripts/lib/devtool/ide_sdk.py
> > > > > > b/scripts/lib/devtool/ide_sdk.py
> > > > > > index 14679744807..f292edbe25c 100755
> > > > > > --- a/scripts/lib/devtool/ide_sdk.py
> > > > > > +++ b/scripts/lib/devtool/ide_sdk.py
> > > > > > @@ -356,6 +356,7 @@ class RecipeModified:
> > > > > >                'PACKAGE_DEBUG_SPLIT_STYLE')
> > > > > >            self.path = recipe_d.getVar('PATH')
> > > > > >            self.pn = recipe_d.getVar('PN')
> > > > > > +        self.pv = recipe_d.getVar('PV')
> > > > > >            self.recipe_sysroot = os.path.realpath(
> > > > > >                recipe_d.getVar('RECIPE_SYSROOT'))
> > > > > >            self.recipe_sysroot_native = os.path.realpath(
> > > > > 
> > > > Enguerrand de Ribaucourt
> > > > 
> > > Enguerrand de Ribaucourt
> > 
> 
> Enguerrand de Ribaucourt

[PATCH v2 3/5] devtool: ide: vscode: Configure read-only files

[Enguerrand de Ribaucourt]

When debugging or browsing files, the user may fall into external
sources from other packages in the sysroot or dbg-rootfs. Modifying them
will only lead to confusion since they will be overwritten by Yocto. The
user should open them in a separate devtool modify session if they want
to make changes. Meanwhile, we should prevent write access to them.

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
---
 scripts/lib/devtool/ide_plugins/ide_code.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index c063b7d0590..7b683c74086 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -125,7 +125,7 @@ class IdeVSCode(IdeBase):
         settings_dict["cmake.configureOnOpen"] = True
         settings_dict["cmake.sourceDirectory"] = modified_recipe.real_srctree
 
-    def vscode_settings(self, modified_recipe):
+    def vscode_settings(self, modified_recipe, image_recipe):
         files_excludes = {
             "**/.git/**": True,
             "**/oe-logs/**": True,
@@ -138,9 +138,16 @@ class IdeVSCode(IdeBase):
             "**/oe-workdir/**",
             "**/source-date-epoch/**"
         ]
+        files_readonly = {
+            modified_recipe.recipe_sysroot + '/**': True,
+            modified_recipe.recipe_sysroot_native + '/**': True,
+        }
+        if image_recipe.rootfs_dbg is not None:
+            files_readonly[image_recipe.rootfs_dbg + '/**'] = True
         settings_dict = {
             "files.watcherExclude": files_excludes,
             "files.exclude": files_excludes,
+            "files.readonlyInclude": files_readonly,
             "python.analysis.exclude": python_exclude
         }
         self.__vscode_settings_cmake(settings_dict, modified_recipe)
@@ -425,7 +432,7 @@ class IdeVSCode(IdeBase):
             self.vscode_tasks_fallback(args, modified_recipe)
 
     def setup_modified_recipe(self, args, image_recipe, modified_recipe):
-        self.vscode_settings(modified_recipe)
+        self.vscode_settings(modified_recipe, image_recipe)
         self.vscode_extensions(modified_recipe)
         self.vscode_c_cpp_properties(modified_recipe)
         if args.target:
-- 
2.34.1

Re: [PATCH v2 3/5] devtool: ide: vscode: Configure read-only files

[adrian.freihofer]

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
> When debugging or browsing files, the user may fall into external
> sources from other packages in the sysroot or dbg-rootfs. Modifying
> them
> will only lead to confusion since they will be overwritten by Yocto.
> The
> user should open them in a separate devtool modify session if they
> want
> to make changes. Meanwhile, we should prevent write access to them.
> 
> Signed-off-by: Enguerrand de Ribaucourt
> <enguerrand.de-ribaucourt@savoirfairelinux.com>
> ---
>  scripts/lib/devtool/ide_plugins/ide_code.py | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
> b/scripts/lib/devtool/ide_plugins/ide_code.py
> index c063b7d0590..7b683c74086 100644
> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
> @@ -125,7 +125,7 @@ class IdeVSCode(IdeBase):
>          settings_dict["cmake.configureOnOpen"] = True
>          settings_dict["cmake.sourceDirectory"] =
> modified_recipe.real_srctree
>  
> -    def vscode_settings(self, modified_recipe):
> +    def vscode_settings(self, modified_recipe, image_recipe):
>          files_excludes = {
>              "**/.git/**": True,
>              "**/oe-logs/**": True,
> @@ -138,9 +138,16 @@ class IdeVSCode(IdeBase):
>              "**/oe-workdir/**",
>              "**/source-date-epoch/**"
>          ]
> +        files_readonly = {
> +            modified_recipe.recipe_sysroot + '/**': True,
> +            modified_recipe.recipe_sysroot_native + '/**': True,
> +        }
> +        if image_recipe.rootfs_dbg is not None:
> +            files_readonly[image_recipe.rootfs_dbg + '/**'] = True
>          settings_dict = {
>              "files.watcherExclude": files_excludes,
>              "files.exclude": files_excludes,
> +            "files.readonlyInclude": files_readonly,
>              "python.analysis.exclude": python_exclude
>          }
>          self.__vscode_settings_cmake(settings_dict, modified_recipe)
> @@ -425,7 +432,7 @@ class IdeVSCode(IdeBase):
>              self.vscode_tasks_fallback(args, modified_recipe)
>  
>      def setup_modified_recipe(self, args, image_recipe,
> modified_recipe):
> -        self.vscode_settings(modified_recipe)
> +        self.vscode_settings(modified_recipe, image_recipe)
>          self.vscode_extensions(modified_recipe)
>          self.vscode_c_cpp_properties(modified_recipe)
>          if args.target:

LGTM.
Adrian

[PATCH v2 4/5] meson: use absolute cross-compiler paths

[Enguerrand de Ribaucourt]

Among the files generated by meson is compile_commands.json. It is not
used by bitbake during the build. However, if the devtool workspace is
opened inside an IDE, that IDE can use compile_commands.json to
configure linting and code completion. This is notably relied on by the
new devtool ide-sdk command.

The problem is that the IDE using compile_commands.json does not know
the $PATH set-up by bitbake, so it won't find the compiler. This results
in linting errors, like missing headers. We can fix this by expliciting
the absolute compiler paths in meson.cross.

The compile_commands.json specification expressly states:
"All paths specified in the command or file fields must be either
absolute or relative to this directory."
Link: https://clang.llvm.org/docs/JSONCompilationDatabase.html

An alternative way to implement this is to directly change CXX inside
bitbake.conf to make all recipes use absolute compiler paths.Since this
would affect all recipes, so I would like to have the maintainers'
opinion on this. It could make sense to use absolute compiler paths for
all toolchain binaries, we already do so for the sysroot
TOOLCHAIN_OPTIONS.

Discussions have been opened with meson/ninja maintainers to implement
this at their level:
 - https://github.com/ninja-build/ninja/issues/2383
 - https://github.com/mesonbuild/meson/issues/12834
These tools have even less information on the environment so it makes
sense for Yocto to provide the absolute paths.

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
---
 meta/classes-recipe/meson-routines.bbclass | 6 ++++++
 meta/classes-recipe/meson.bbclass          | 7 +++++--
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/meson-routines.bbclass b/meta/classes-recipe/meson-routines.bbclass
index a944a8fff1c..9925465ed8f 100644
--- a/meta/classes-recipe/meson-routines.bbclass
+++ b/meta/classes-recipe/meson-routines.bbclass
@@ -10,6 +10,12 @@ def meson_array(var, d):
     items = d.getVar(var).split()
     return repr(items[0] if len(items) == 1 else items)
 
+def meson_array_abspath(var, d):
+    import shutil
+    items = d.getVar(var).split()
+    items[0] = shutil.which(items[0]) or items[0]
+    return repr(items[0] if len(items) == 1 else items)
+
 # Map our ARCH values to what Meson expects:
 # http://mesonbuild.com/Reference-tables.html#cpu-families
 def meson_cpu_family(var, d):
diff --git a/meta/classes-recipe/meson.bbclass b/meta/classes-recipe/meson.bbclass
index 03fa2c06eb4..31675cf42d1 100644
--- a/meta/classes-recipe/meson.bbclass
+++ b/meta/classes-recipe/meson.bbclass
@@ -64,10 +64,13 @@ addtask write_config before do_configure
 do_write_config[vardeps] += "CC CXX AR NM STRIP READELF OBJCOPY CFLAGS CXXFLAGS LDFLAGS RUSTC RUSTFLAGS EXEWRAPPER_ENABLED"
 do_write_config() {
     # This needs to be Py to split the args into single-element lists
+    # The generated compile_commands.json file can be used by external IDEs
+    # which do not know the $PATH set-up by bitbake. They need the absolute
+    # compiler paths.
     cat >${WORKDIR}/meson.cross <<EOF
 [binaries]
-c = ${@meson_array('CC', d)}
-cpp = ${@meson_array('CXX', d)}
+c = ${@meson_array_abspath('CC', d)}
+cpp = ${@meson_array_abspath('CXX', d)}
 cython = 'cython3'
 ar = ${@meson_array('AR', d)}
 nm = ${@meson_array('NM', d)}
-- 
2.34.1

Re: [OE-core] [PATCH v2 4/5] meson: use absolute cross-compiler paths

[Richard Purdie]

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
> Among the files generated by meson is compile_commands.json. It is not
> used by bitbake during the build. However, if the devtool workspace is
> opened inside an IDE, that IDE can use compile_commands.json to
> configure linting and code completion. This is notably relied on by the
> new devtool ide-sdk command.
> 
> The problem is that the IDE using compile_commands.json does not know
> the $PATH set-up by bitbake, so it won't find the compiler. This results
> in linting errors, like missing headers. We can fix this by expliciting
> the absolute compiler paths in meson.cross.
> 
> The compile_commands.json specification expressly states:
> "All paths specified in the command or file fields must be either
> absolute or relative to this directory."
> Link: https://clang.llvm.org/docs/JSONCompilationDatabase.html
> 
> An alternative way to implement this is to directly change CXX inside
> bitbake.conf to make all recipes use absolute compiler paths.Since this
> would affect all recipes, so I would like to have the maintainers'
> opinion on this. It could make sense to use absolute compiler paths for
> all toolchain binaries, we already do so for the sysroot
> TOOLCHAIN_OPTIONS.

I'd prefer not to change CXX. 

We have a lot of problems with overflows of command line lengths
already and adding extra paths to such variables will make the problem
much worse. It also does make logs harder to read with long paths added
to them.

I suspect the readability issues along are enough to make this a change
for the worse overall.

Cheers,

Richard

[PATCH v2 5/5] devtool: code: Provide a generic C++ configuration

[Enguerrand de Ribaucourt]

By default, the cpptools VSCode extension will use the host's headers
and flags for linting. This results in a lot of include errors and
misleading definitions. Even though this generic configuration doesn't
include all the depenendencies, it is a proper fallback for recipe
classes we do not accurately cover, with at least the right sysroot.

Additionally, ide-sdk automatically detects and provides a launch.json
configuration for autotools recipes so we should recommend the C++
extensions for them.

Recipes which use C/C++ without a build system are not covered by this
patch.

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
---
 scripts/lib/devtool/ide_plugins/__init__.py |  3 +++
 scripts/lib/devtool/ide_plugins/ide_code.py | 27 +++++++++++++--------
 scripts/lib/devtool/ide_sdk.py              |  5 ++++
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py
index 3371b242640..59e90663908 100644
--- a/scripts/lib/devtool/ide_plugins/__init__.py
+++ b/scripts/lib/devtool/ide_plugins/__init__.py
@@ -21,6 +21,7 @@ class BuildTool(Enum):
     UNDEFINED = auto()
     CMAKE = auto()
     MESON = auto()
+    AUTOTOOLS = auto()
 
     @property
     def is_c_ccp(self):
@@ -28,6 +29,8 @@ class BuildTool(Enum):
             return True
         if self is BuildTool.MESON:
             return True
+        if self is BuildTool.AUTOTOOLS:
+            return True
         return False
 
 
diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index 7b683c74086..0942fde8196 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -157,31 +157,35 @@ class IdeVSCode(IdeBase):
         IdeBase.update_json_file(
             self.dot_code_dir(modified_recipe), settings_file, settings_dict)
 
-    def __vscode_extensions_cmake(self, modified_recipe, recommendations):
-        if modified_recipe.build_tool is not BuildTool.CMAKE:
+    def __vscode_extensions_generic(self, modified_recipe, recommendations):
+        if not modified_recipe.build_tool.is_c_ccp:
             return
         recommendations += [
-            "twxs.cmake",
-            "ms-vscode.cmake-tools",
             "ms-vscode.cpptools",
             "ms-vscode.cpptools-extension-pack",
             "ms-vscode.cpptools-themes"
         ]
 
+    def __vscode_extensions_cmake(self, modified_recipe, recommendations):
+        if modified_recipe.build_tool is not BuildTool.CMAKE:
+            return
+        recommendations += [
+            "twxs.cmake",
+            "ms-vscode.cmake-tools"
+        ]
+
     def __vscode_extensions_meson(self, modified_recipe, recommendations):
         if modified_recipe.build_tool is not BuildTool.MESON:
             return
         recommendations += [
-            'mesonbuild.mesonbuild',
-            "ms-vscode.cpptools",
-            "ms-vscode.cpptools-extension-pack",
-            "ms-vscode.cpptools-themes"
+            'mesonbuild.mesonbuild'
         ]
 
     def vscode_extensions(self, modified_recipe):
         recommendations = []
         self.__vscode_extensions_cmake(modified_recipe, recommendations)
         self.__vscode_extensions_meson(modified_recipe, recommendations)
+        self.__vscode_extensions_generic(modified_recipe, recommendations)
         extensions_file = 'extensions.json'
         IdeBase.update_json_file(
             self.dot_code_dir(modified_recipe), extensions_file, {"recommendations": recommendations})
@@ -194,8 +198,11 @@ class IdeVSCode(IdeBase):
             properties_dict["configurationProvider"] = "ms-vscode.cmake-tools"
         elif modified_recipe.build_tool is BuildTool.MESON:
             properties_dict["configurationProvider"] = "mesonbuild.mesonbuild"
-        else:  # no C/C++ build
-            return
+        elif modified_recipe.build_tool.is_c_ccp:
+            # Provide a generic linting configuration
+            # We provide a C++ configuration with the proper sysroot
+            properties_dict["compilerPath"] = os.path.join(modified_recipe.staging_bindir_toolchain, modified_recipe.cxx.split()[0])
+            properties_dict["compilerArgs"] = modified_recipe.cxx.split()[1:]
 
         properties_dicts = {
             "configurations": [
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index f292edbe25c..6313daa8700 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -345,6 +345,7 @@ class RecipeModified:
         self.base_libdir = recipe_d.getVar('base_libdir')
         self.bblayers = recipe_d.getVar('BBLAYERS').split()
         self.bpn = recipe_d.getVar('BPN')
+        self.cxx = recipe_d.getVar('CXX')
         self.d = recipe_d.getVar('D')
         self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD')
         self.fakerootenv = recipe_d.getVar('FAKEROOTENV')
@@ -361,6 +362,8 @@ class RecipeModified:
             recipe_d.getVar('RECIPE_SYSROOT'))
         self.recipe_sysroot_native = os.path.realpath(
             recipe_d.getVar('RECIPE_SYSROOT_NATIVE'))
+        self.staging_bindir_toolchain = os.path.realpath(
+            recipe_d.getVar('STAGING_BINDIR_TOOLCHAIN'))
         self.staging_incdir = os.path.realpath(
             recipe_d.getVar('STAGING_INCDIR'))
         self.strip_cmd = recipe_d.getVar('STRIP')
@@ -380,6 +383,8 @@ class RecipeModified:
             self.extra_oemeson = recipe_d.getVar('EXTRA_OEMESON')
             self.meson_cross_file = recipe_d.getVar('MESON_CROSS_FILE')
             self.build_tool = BuildTool.MESON
+        elif bb.data.inherits_class('autotools', recipe_d):
+            self.build_tool = BuildTool.AUTOTOOLS
 
         # Recipe ID is the identifier for IDE config sections
         self.recipe_id = self.bpn + "-" + self.package_arch
-- 
2.34.1

Re: [PATCH v2 5/5] devtool: code: Provide a generic C++ configuration

[adrian.freihofer]

Hi Enguerrand

I would really like to stay on the path with tests. This patch changes
code which is covered by tests without tests. Lets develop the
autotools support as a separate patch series. 

There are three parts needed:
- Extend the cpp-example with autotools support (already on my branch)
- What your patch does
- Extend the oe-selftest suite to cover also the autotools support.

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
> By default, the cpptools VSCode extension will use the host's headers
> and flags for linting. This results in a lot of include errors and
> misleading definitions. Even though this generic configuration
> doesn't
> include all the depenendencies, it is a proper fallback for recipe
> classes we do not accurately cover, with at least the right sysroot.
> 
> Additionally, ide-sdk automatically detects and provides a
> launch.json
> configuration for autotools recipes so we should recommend the C++
> extensions for them.
> 
> Recipes which use C/C++ without a build system are not covered by
> this
> patch.
> 
> Signed-off-by: Enguerrand de Ribaucourt
> <enguerrand.de-ribaucourt@savoirfairelinux.com>
> ---
>  scripts/lib/devtool/ide_plugins/__init__.py |  3 +++
>  scripts/lib/devtool/ide_plugins/ide_code.py | 27 +++++++++++++------
> --
>  scripts/lib/devtool/ide_sdk.py              |  5 ++++
>  3 files changed, 25 insertions(+), 10 deletions(-)
> 
> diff --git a/scripts/lib/devtool/ide_plugins/__init__.py
> b/scripts/lib/devtool/ide_plugins/__init__.py
> index 3371b242640..59e90663908 100644
> --- a/scripts/lib/devtool/ide_plugins/__init__.py
> +++ b/scripts/lib/devtool/ide_plugins/__init__.py
> @@ -21,6 +21,7 @@ class BuildTool(Enum):
>      UNDEFINED = auto()
>      CMAKE = auto()
>      MESON = auto()
> +    AUTOTOOLS = auto()
>  
>      @property
>      def is_c_ccp(self):
> @@ -28,6 +29,8 @@ class BuildTool(Enum):
>              return True
>          if self is BuildTool.MESON:
>              return True
> +        if self is BuildTool.AUTOTOOLS:
> +            return True
>          return False
>  
>  
> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
> b/scripts/lib/devtool/ide_plugins/ide_code.py
> index 7b683c74086..0942fde8196 100644
> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
> @@ -157,31 +157,35 @@ class IdeVSCode(IdeBase):
>          IdeBase.update_json_file(
>              self.dot_code_dir(modified_recipe), settings_file,
> settings_dict)
>  
> -    def __vscode_extensions_cmake(self, modified_recipe,
> recommendations):
> -        if modified_recipe.build_tool is not BuildTool.CMAKE:
> +    def __vscode_extensions_generic(self, modified_recipe,
> recommendations):
> +        if not modified_recipe.build_tool.is_c_ccp:
>              return
>          recommendations += [
> -            "twxs.cmake",
> -            "ms-vscode.cmake-tools",
>              "ms-vscode.cpptools",
>              "ms-vscode.cpptools-extension-pack",
>              "ms-vscode.cpptools-themes"
>          ]
>  
> +    def __vscode_extensions_cmake(self, modified_recipe,
> recommendations):
> +        if modified_recipe.build_tool is not BuildTool.CMAKE:
> +            return
> +        recommendations += [
> +            "twxs.cmake",
> +            "ms-vscode.cmake-tools"
> +        ]
> +
>      def __vscode_extensions_meson(self, modified_recipe,
> recommendations):
>          if modified_recipe.build_tool is not BuildTool.MESON:
>              return
>          recommendations += [
> -            'mesonbuild.mesonbuild',
> -            "ms-vscode.cpptools",
> -            "ms-vscode.cpptools-extension-pack",
> -            "ms-vscode.cpptools-themes"
> +            'mesonbuild.mesonbuild'
>          ]
>  
>      def vscode_extensions(self, modified_recipe):
>          recommendations = []
>          self.__vscode_extensions_cmake(modified_recipe,
> recommendations)
>          self.__vscode_extensions_meson(modified_recipe,
> recommendations)
> +        self.__vscode_extensions_generic(modified_recipe,
> recommendations)
>          extensions_file = 'extensions.json'
>          IdeBase.update_json_file(
>              self.dot_code_dir(modified_recipe), extensions_file,
> {"recommendations": recommendations})
> @@ -194,8 +198,11 @@ class IdeVSCode(IdeBase):
>              properties_dict["configurationProvider"] = "ms-
> vscode.cmake-tools"
>          elif modified_recipe.build_tool is BuildTool.MESON:
>              properties_dict["configurationProvider"] =
> "mesonbuild.mesonbuild"
> -        else:  # no C/C++ build
> -            return

else return is still required.

Adrian


> +        elif modified_recipe.build_tool.is_c_ccp:
> +            # Provide a generic linting configuration
> +            # We provide a C++ configuration with the proper sysroot
> +            properties_dict["compilerPath"] =
> os.path.join(modified_recipe.staging_bindir_toolchain,
> modified_recipe.cxx.split()[0])
> +            properties_dict["compilerArgs"] =
> modified_recipe.cxx.split()[1:]
>  
>          properties_dicts = {
>              "configurations": [
> diff --git a/scripts/lib/devtool/ide_sdk.py
> b/scripts/lib/devtool/ide_sdk.py
> index f292edbe25c..6313daa8700 100755
> --- a/scripts/lib/devtool/ide_sdk.py
> +++ b/scripts/lib/devtool/ide_sdk.py
> @@ -345,6 +345,7 @@ class RecipeModified:
>          self.base_libdir = recipe_d.getVar('base_libdir')
>          self.bblayers = recipe_d.getVar('BBLAYERS').split()
>          self.bpn = recipe_d.getVar('BPN')
> +        self.cxx = recipe_d.getVar('CXX')
>          self.d = recipe_d.getVar('D')
>          self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD')
>          self.fakerootenv = recipe_d.getVar('FAKEROOTENV')
> @@ -361,6 +362,8 @@ class RecipeModified:
>              recipe_d.getVar('RECIPE_SYSROOT'))
>          self.recipe_sysroot_native = os.path.realpath(
>              recipe_d.getVar('RECIPE_SYSROOT_NATIVE'))
> +        self.staging_bindir_toolchain = os.path.realpath(
> +            recipe_d.getVar('STAGING_BINDIR_TOOLCHAIN'))
>          self.staging_incdir = os.path.realpath(
>              recipe_d.getVar('STAGING_INCDIR'))
>          self.strip_cmd = recipe_d.getVar('STRIP')
> @@ -380,6 +383,8 @@ class RecipeModified:
>              self.extra_oemeson = recipe_d.getVar('EXTRA_OEMESON')
>              self.meson_cross_file =
> recipe_d.getVar('MESON_CROSS_FILE')
>              self.build_tool = BuildTool.MESON
> +        elif bb.data.inherits_class('autotools', recipe_d):
> +            self.build_tool = BuildTool.AUTOTOOLS
>  
>          # Recipe ID is the identifier for IDE config sections
>          self.recipe_id = self.bpn + "-" + self.package_arch

Re: [PATCH v2 5/5] devtool: code: Provide a generic C++ configuration

[Enguerrand de Ribaucourt]

Hi Adrian,

Thanks for your review.

Yes it doesn't makes a lot of sense to keep this patch compared to the 
specific support you are working on. It was especially useful for 
recipes using plain oe_runmake or CC/CXX outside of a class. But we 
can't distinguish these recipes from others classes like python/bash 
that would not make use of this C++ configuration.

I'll drop this patch and try to figure out if I can also fix the 
debugging configuration without a source map (PATH v2 3/5). It would 
rather indicate that the wrong binary is being deployed for some workspaces.

On 20/02/2024 10:20, adrian.freihofer@gmail.com wrote:
> Hi Enguerrand
> 
> I would really like to stay on the path with tests. This patch changes
> code which is covered by tests without tests. Lets develop the
> autotools support as a separate patch series.
> 
> There are three parts needed:
> - Extend the cpp-example with autotools support (already on my branch)
> - What your patch does
> - Extend the oe-selftest suite to cover also the autotools support.
> 
> On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:
>> By default, the cpptools VSCode extension will use the host's headers
>> and flags for linting. This results in a lot of include errors and
>> misleading definitions. Even though this generic configuration
>> doesn't
>> include all the depenendencies, it is a proper fallback for recipe
>> classes we do not accurately cover, with at least the right sysroot.
>>
>> Additionally, ide-sdk automatically detects and provides a
>> launch.json
>> configuration for autotools recipes so we should recommend the C++
>> extensions for them.
>>
>> Recipes which use C/C++ without a build system are not covered by
>> this
>> patch.
>>
>> Signed-off-by: Enguerrand de Ribaucourt
>> <enguerrand.de-ribaucourt@savoirfairelinux.com>
>> ---
>>   scripts/lib/devtool/ide_plugins/__init__.py |  3 +++
>>   scripts/lib/devtool/ide_plugins/ide_code.py | 27 +++++++++++++------
>> --
>>   scripts/lib/devtool/ide_sdk.py              |  5 ++++
>>   3 files changed, 25 insertions(+), 10 deletions(-)
>>
>> diff --git a/scripts/lib/devtool/ide_plugins/__init__.py
>> b/scripts/lib/devtool/ide_plugins/__init__.py
>> index 3371b242640..59e90663908 100644
>> --- a/scripts/lib/devtool/ide_plugins/__init__.py
>> +++ b/scripts/lib/devtool/ide_plugins/__init__.py
>> @@ -21,6 +21,7 @@ class BuildTool(Enum):
>>       UNDEFINED = auto()
>>       CMAKE = auto()
>>       MESON = auto()
>> +    AUTOTOOLS = auto()
>>   
>>       @property
>>       def is_c_ccp(self):
>> @@ -28,6 +29,8 @@ class BuildTool(Enum):
>>               return True
>>           if self is BuildTool.MESON:
>>               return True
>> +        if self is BuildTool.AUTOTOOLS:
>> +            return True
>>           return False
>>   
>>   
>> diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
>> b/scripts/lib/devtool/ide_plugins/ide_code.py
>> index 7b683c74086..0942fde8196 100644
>> --- a/scripts/lib/devtool/ide_plugins/ide_code.py
>> +++ b/scripts/lib/devtool/ide_plugins/ide_code.py
>> @@ -157,31 +157,35 @@ class IdeVSCode(IdeBase):
>>           IdeBase.update_json_file(
>>               self.dot_code_dir(modified_recipe), settings_file,
>> settings_dict)
>>   
>> -    def __vscode_extensions_cmake(self, modified_recipe,
>> recommendations):
>> -        if modified_recipe.build_tool is not BuildTool.CMAKE:
>> +    def __vscode_extensions_generic(self, modified_recipe,
>> recommendations):
>> +        if not modified_recipe.build_tool.is_c_ccp:
>>               return
>>           recommendations += [
>> -            "twxs.cmake",
>> -            "ms-vscode.cmake-tools",
>>               "ms-vscode.cpptools",
>>               "ms-vscode.cpptools-extension-pack",
>>               "ms-vscode.cpptools-themes"
>>           ]
>>   
>> +    def __vscode_extensions_cmake(self, modified_recipe,
>> recommendations):
>> +        if modified_recipe.build_tool is not BuildTool.CMAKE:
>> +            return
>> +        recommendations += [
>> +            "twxs.cmake",
>> +            "ms-vscode.cmake-tools"
>> +        ]
>> +
>>       def __vscode_extensions_meson(self, modified_recipe,
>> recommendations):
>>           if modified_recipe.build_tool is not BuildTool.MESON:
>>               return
>>           recommendations += [
>> -            'mesonbuild.mesonbuild',
>> -            "ms-vscode.cpptools",
>> -            "ms-vscode.cpptools-extension-pack",
>> -            "ms-vscode.cpptools-themes"
>> +            'mesonbuild.mesonbuild'
>>           ]
>>   
>>       def vscode_extensions(self, modified_recipe):
>>           recommendations = []
>>           self.__vscode_extensions_cmake(modified_recipe,
>> recommendations)
>>           self.__vscode_extensions_meson(modified_recipe,
>> recommendations)
>> +        self.__vscode_extensions_generic(modified_recipe,
>> recommendations)
>>           extensions_file = 'extensions.json'
>>           IdeBase.update_json_file(
>>               self.dot_code_dir(modified_recipe), extensions_file,
>> {"recommendations": recommendations})
>> @@ -194,8 +198,11 @@ class IdeVSCode(IdeBase):
>>               properties_dict["configurationProvider"] = "ms-
>> vscode.cmake-tools"
>>           elif modified_recipe.build_tool is BuildTool.MESON:
>>               properties_dict["configurationProvider"] =
>> "mesonbuild.mesonbuild"
>> -        else:  # no C/C++ build
>> -            return
> 
> else return is still required.
> 
> Adrian
> 
> 
>> +        elif modified_recipe.build_tool.is_c_ccp:
>> +            # Provide a generic linting configuration
>> +            # We provide a C++ configuration with the proper sysroot
>> +            properties_dict["compilerPath"] =
>> os.path.join(modified_recipe.staging_bindir_toolchain,
>> modified_recipe.cxx.split()[0])
>> +            properties_dict["compilerArgs"] =
>> modified_recipe.cxx.split()[1:]
>>   
>>           properties_dicts = {
>>               "configurations": [
>> diff --git a/scripts/lib/devtool/ide_sdk.py
>> b/scripts/lib/devtool/ide_sdk.py
>> index f292edbe25c..6313daa8700 100755
>> --- a/scripts/lib/devtool/ide_sdk.py
>> +++ b/scripts/lib/devtool/ide_sdk.py
>> @@ -345,6 +345,7 @@ class RecipeModified:
>>           self.base_libdir = recipe_d.getVar('base_libdir')
>>           self.bblayers = recipe_d.getVar('BBLAYERS').split()
>>           self.bpn = recipe_d.getVar('BPN')
>> +        self.cxx = recipe_d.getVar('CXX')
>>           self.d = recipe_d.getVar('D')
>>           self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD')
>>           self.fakerootenv = recipe_d.getVar('FAKEROOTENV')
>> @@ -361,6 +362,8 @@ class RecipeModified:
>>               recipe_d.getVar('RECIPE_SYSROOT'))
>>           self.recipe_sysroot_native = os.path.realpath(
>>               recipe_d.getVar('RECIPE_SYSROOT_NATIVE'))
>> +        self.staging_bindir_toolchain = os.path.realpath(
>> +            recipe_d.getVar('STAGING_BINDIR_TOOLCHAIN'))
>>           self.staging_incdir = os.path.realpath(
>>               recipe_d.getVar('STAGING_INCDIR'))
>>           self.strip_cmd = recipe_d.getVar('STRIP')
>> @@ -380,6 +383,8 @@ class RecipeModified:
>>               self.extra_oemeson = recipe_d.getVar('EXTRA_OEMESON')
>>               self.meson_cross_file =
>> recipe_d.getVar('MESON_CROSS_FILE')
>>               self.build_tool = BuildTool.MESON
>> +        elif bb.data.inherits_class('autotools', recipe_d):
>> +            self.build_tool = BuildTool.AUTOTOOLS
>>   
>>           # Recipe ID is the identifier for IDE config sections
>>           self.recipe_id = self.bpn + "-" + self.package_arch
> 

-- 
Savoir-faire Linux
Enguerrand de Ribaucourt
Consultant en logiciel libre / Ingénieur systèmes embarqués | Rennes, Fr
Bureau
(+33) 9 72 46 89 80 (606#)
Site web <https://www.savoirfairelinux.com/> | Blog
<https://blog.savoirfairelinux.com/fr-ca/> | Jami <https://jami.net/>

Messages de confidentialité : Ce courriel (de même que les fichiers
joints) est strictement réservé à l'usage de la personne ou de l'entité
à qui il est adressé et peut contenir de l'information privilégiée et
confidentielle. Toute divulgation, distribution ou copie de ce courriel
est strictement prohibée. Si vous avez reçu ce courriel par erreur,
veuillez nous en aviser sur-le-champ, détruire toutes les copies et le
supprimer de votre système informatique.