[Capstone] Feedback request for new architecture updater

[Capstone] Feedback request for new architecture updater

[Rot127]

I currently work on a new updater for architecture modules in Capstone.

The current update process has the problem that it is a lot of manual work.
Syncing the information from LLVM to Capstone requires a one to 
translate a lot of C++ to C code and fix a ton of build errors.
This lead to architecture modules not getting updated. So there are many 
new instructions in Capstone which are not supported.

The new updater will take away as much manual work as possible.
It does this by translating most C++ to C via a little translation tool 
and uses refactored LLVM-TableGen backends which emit directly the code 
Capstone needs.
Currently it is in a early stage but it is ready to receive some feedback.

If anyone finds the time to take a look at it and write down their 
thoughts, suggestions or improvement ideas, I would be very grateful.

Capstone draft PR: https://github.com/capstone-engine/capstone/pull/1949

Re: [Capstone] Feedback request for new architecture updater

[Thomas Huth]

On 12/01/2023 17.24, Rot127 wrote:
> I currently work on a new updater for architecture modules in Capstone.
> 
> The current update process has the problem that it is a lot of manual work.
> Syncing the information from LLVM to Capstone requires a one to translate a 
> lot of C++ to C code and fix a ton of build errors.
> This lead to architecture modules not getting updated. So there are many new 
> instructions in Capstone which are not supported.
> 
> The new updater will take away as much manual work as possible.
> It does this by translating most C++ to C via a little translation tool and 
> uses refactored LLVM-TableGen backends which emit directly the code Capstone 
> needs.
> Currently it is in a early stage but it is ready to receive some feedback.
> 
> If anyone finds the time to take a look at it and write down their thoughts, 
> suggestions or improvement ideas, I would be very grateful.
> 
> Capstone draft PR: https://github.com/capstone-engine/capstone/pull/1949

  Hi!

Thanks for the heads up! I see your pull request addresses the ARM part of 
Capstone, so I CC:-ed qemu-arm@nongnu.org in case anybody there wants to 
give it a try.

 From my personal point of view, what I really would love to see is an 
update for sparc and mips in the next version of Capstone, so we could 
finally get of the old disas/sparc.c and disas/mips.c files in QEMU (which 
were still disassembling more instructions than Capstone 4.0, so we kept 
them ... but AFAIK the upstream LLVM code implemented many missing 
instructions in the past years, so if Capstone gets refreshed, it should be 
good enough to replace the old code now).

Do you have any plans to work on Sparc and Mips any time soon? If not, do 
you have some short (shell) instructions how the update process 
theoretically works - maybe someone else can help?

  Thanks,
   Thomas

Re: [Capstone] Feedback request for new architecture updater

[Rot127]

[-- Attachment #1.1.1: Type: text/plain, Size: 2731 bytes --]

Hi!

 > Do you have any plans to work on Sparc and Mips any time soon?

The plan is to update ARM and PPC first, because they seemed to be the 
most complicated ones.
Afterwards other architectures (Mips and Sparc as well).
I can't say when exactly ARM will be ready (I hope at latest end of 
February, but I can't promise it).
Updating the other architectures is then way simpler, because the 
refactored LLVM backends won't need much work.

 > If not, do you have some short (shell) instructions how the update 
process theoretically works

In the PR description I mention a README where you can start. It should 
explain the overall process.
If not, please let me know so I can improve the documentation.

On 1/13/23 04:27, Thomas Huth wrote:
> On 12/01/2023 17.24, Rot127 wrote:
>> I currently work on a new updater for architecture modules in Capstone.
>>
>> The current update process has the problem that it is a lot of manual 
>> work.
>> Syncing the information from LLVM to Capstone requires a one to 
>> translate a lot of C++ to C code and fix a ton of build errors.
>> This lead to architecture modules not getting updated. So there are 
>> many new instructions in Capstone which are not supported.
>>
>> The new updater will take away as much manual work as possible.
>> It does this by translating most C++ to C via a little translation 
>> tool and uses refactored LLVM-TableGen backends which emit directly 
>> the code Capstone needs.
>> Currently it is in a early stage but it is ready to receive some 
>> feedback.
>>
>> If anyone finds the time to take a look at it and write down their 
>> thoughts, suggestions or improvement ideas, I would be very grateful.
>>
>> Capstone draft PR: https://github.com/capstone-engine/capstone/pull/1949
>
>  Hi!
>
> Thanks for the heads up! I see your pull request addresses the ARM 
> part of Capstone, so I CC:-ed qemu-arm@nongnu.org in case anybody 
> there wants to give it a try.
>
> From my personal point of view, what I really would love to see is an 
> update for sparc and mips in the next version of Capstone, so we could 
> finally get of the old disas/sparc.c and disas/mips.c files in QEMU 
> (which were still disassembling more instructions than Capstone 4.0, 
> so we kept them ... but AFAIK the upstream LLVM code implemented many 
> missing instructions in the past years, so if Capstone gets refreshed, 
> it should be good enough to replace the old code now).
>
> Do you have any plans to work on Sparc and Mips any time soon? If not, 
> do you have some short (shell) instructions how the update process 
> theoretically works - maybe someone else can help?
>
>  Thanks,
>   Thomas
>

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2147 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 659 bytes --]