[PATCH v2 0/4] diff: create pattern for javascript language

["Derick W. de M. Frias" <derick.william.moraes@gmail.com>]

I'm sending a new version of the diff driver. I tried to expand
recognition of functions to more possible cases, improved tests in
t4018 for function matching of the pattern and tried to add test 
t4034 for the matching of words.

I added a lot more test for t4018, so much that I got caught up if I 
was exaggerating. I was not trying to exhaust all possibilities, I just think
that javascript has a lot of different valid syntaxes for declaring
functions and I was trying to make a somewhat complete coverage of that.
Let me know if perhaps I'm going too far along this direction.

I also tried to incorporate your feedbacks, thanks! 

> Thank you for your contribution. We have had a number of submissions for
> Javascript or Typescript drivers in the past, but none of them were
> followed through to be integrated.

> Typescript:

> https://lore.kernel.org/git/20240404163827.5855-1-utsavp0213@gmail.com/
> https://lore.kernel.org/git/pull.1746.git.git.1721061218993.gitgitgadget@gmail.com/

> Javascript:

> https://lore.kernel.org/git/20240301074048.188835-1-sergiusnyah@gmail.com/
> https://lore.kernel.org/git/20220403132508.28196-1-a97410985new@gmail.com/

> Please review these submission and the responses that they received.
> Perhaps you can find inspiration for improvement from them.

It has beeing great fun :)! I took a deep dive into the suggested
submissions. For clarity purposes, I'll be addressing them for the
following index, and sometimes for the contributor's name: 

[1] Utsav Parmar: https://lore.kernel.org/git/20240404163827.5855-1-utsavp0213@gmail.com/
[2] Mattew Hughes: https://lore.kernel.org/git/pull.1746.git.git.1721061218993.gitgitgadget@gmail.com/
[3] Sergius Nyah: https://lore.kernel.org/git/20240301074048.188835-1-sergiusnyah@gmail.com/
[4] Xing Zhi Jiang: https://lore.kernel.org/git/20220403132508.28196-1-a97410985new@gmail.com/

From contributions [1], [2] and [3] I did not found many things that are
worth noting. 

Previous to sending the v1, I had already searched through the lore and
stumbled across Xing Zhi Jiang's[4] contribution. He had a lot of things
worked out, including well recognized nested functions, that is
something still not supported here in v2. Also the discutions around it
brought up interesting points that I took into consideration:

- ESModules syntax for exporting functions (export function FUNCNAME...)
- 'exports.' and 'module.exports.' syntax from commonJS.
- What should and what should not be supported.

I called 'vanillaJS' javascript as it is defined by ECMAScript's specs,
or what you will find if you are working with JS without things like
React, Vue, jQuery, Axios. I noticed that I used this term in the last
submission but had not explained what I was talking about. There I said
I wanted to be "vanillaJS" first, as I believe ECMAs specs should be the 
focal point of a tool like this one.

This is important, because some things that live outside 'vanillaJS', like
'commonJS' for example, which is a thing in Node, add syntax to valid
function declarations in their own context. Exhausting those syntaxes might
be impossible, and I belive it's not the way to go here. Also, it would be
huge work and I don't believe that I have JS knowledge extensive enough to 
do so.

As it was pointed out inside [4] discutions, it would be nice if a
heavilly spread syntax like 'export.' and 'module.export.' from
'commonJS' were supported, even thought it is not ECMA, because it would
benefit users in more diverse situations, which I agree. That is why I
tried adding support to recognize them.

Xing tried to add in v3 patterns for supporting jQuery. As I don't have
experience working, it is not added here. Like in the above paragraph, I
think it would greatly benefit a lot of developers if jQuery was
supported, but that  raised the discussion:

What this pattern should aim to give support to outside vanillaJS,
if it should? And if yes, where do we draw the line to define what is
widespread enough to be supported? I want to know your toughts on it.

> You do not have to include single-character punctuation; these are
> recognized as words automatically.

> Personally I prefer |===| over |={3}| (and similar for the others)
> because it is easier to understand.

In userdiff.c, I cleaned from the pattern the single character punctuation
identifiers and I improved the syntax of some other identifiers (Like ={3} --> ===).
For scaped punctuations, I kept the syntax with {} as I believe it reads
more clearly (Like \\.{3} instead of \\.\\.\\.}.

> I didn't see any test cases dealing with nested functions, which might
> add some value?

I agree it would add value. As I said, [4] had it working and for the moment 
but I still could not understand the strategy and what part of the regex dealt
with it. In his v1, Xing talks about 3 negations rules he added
to the matching pattern. I think that this is the part responsible for
solving the problem, but I don't understand how. I would appreciate if
someone took a little time to maybe figure it out.

> We avoid incomplete last lines in this code base if it is not mandated
> for some reason.

I put a blank line at the end of all files in this submission.
 
> Here and in the later test cases, the word "ChangeMe" is missing. How
> did these ever pass the tests?

Here Johannes was reffering to the t4018 test cases which were missing
"ChangeMe", it was a mistake from my part. This time I added "ChangeMe"
to the t4018 tests.

Derick W. de M. Frias (4):
  userdiff: add javascript diff driver
  t4034: add tests for javascript word literals
  t4018: add tests for recognizing javascript function syntax
  t4018: add tests for javascript 'exports.', 'module.exports' and
    'export' type functions

 Documentation/gitattributes.adoc              |  2 +
 .../javascript-anonymous-assigned-function    |  4 ++
 t/t4018/javascript-arrow-assigned-function    |  4 ++
 t/t4018/javascript-arrow-assigned-function-2  |  5 ++
 t/t4018/javascript-arrow-assigned-function-3  |  1 +
 t/t4018/javascript-assigned-function          |  4 ++
 t/t4018/javascript-async-assigned-function    |  4 ++
 t/t4018/javascript-async-function             |  4 +-
 t/t4018/javascript-class-get-function         |  5 ++
 t/t4018/javascript-class-set-function         |  5 ++
 t/t4018/javascript-class-static-function      |  5 ++
 t/t4018/javascript-class-unprefixed-function  |  6 ++
 ...ascript-dotexpors-async-anonymous-function |  3 +
 .../javascript-dotexports-anonymous-function  |  3 +
 t/t4018/javascript-dotexports-arrow-function  |  4 ++
 .../javascript-dotexports-arrow-function-2    |  4 ++
 .../javascript-dotexports-arrow-function-3    |  1 +
 .../javascript-dotexports-assigned-function   |  1 +
 ...javascript-dotexports-async-arrow-function |  3 +
 ...vascript-dotexports-async-arrow-function-2 |  4 ++
 ...vascript-dotexports-async-arrow-function-3 |  1 +
 t/t4018/javascript-dotexports-async-function  |  4 ++
 ...exports-async-generator-anonymous-function |  5 ++
 ...ports-async-generator-anonymous-function-2 |  5 ++
 ...script-dotexports-async-generator-function |  5 ++
 ...ript-dotexports-async-generator-function-2 |  5 ++
 t/t4018/javascript-dotexports-function        |  4 ++
 ...pt-dotexports-generator-anonymous-function |  5 ++
 ...-dotexports-generator-anonymous-function-2 |  5 ++
 .../javascript-dotexports-generator-function  |  5 ++
 ...javascript-dotexports-generator-function-2 |  5 ++
 t/t4018/javascript-export-arrow-function      |  4 ++
 ...t-export-async-anonymous-assigned-function |  4 ++
 .../javascript-export-async-arrow-function    |  4 ++
 t/t4018/javascript-export-async-function      |  4 ++
 ...sync-generator-anonymous-assigned-function |  4 ++
 ...nc-generator-anonymous-assigned-function-2 |  4 ++
 ...t-export-async-generator-assigned-function |  5 ++
 ...export-async-generator-assigned-function-2 |  5 ++
 ...javascript-export-async-generator-function |  5 ++
 ...vascript-export-async-generator-function-2 |  5 ++
 t/t4018/javascript-export-function            |  4 ++
 ...ascript-export-generator-assigned-function |  6 ++
 ...cript-export-generator-assigned-function-2 |  6 ++
 t/t4018/javascript-export-generator-function  |  5 ++
 .../javascript-export-generator-function-2    |  5 ++
 t/t4018/javascript-function                   |  4 +-
 .../javascript-generator-assigned-function    |  5 ++
 .../javascript-generator-assigned-function-2  |  5 ++
 t/t4018/javascript-generator-function         |  4 +-
 t/t4018/javascript-generator-function-2       |  4 +-
 t/t4018/javascript-method-unprefixed-function |  6 ++
 ...cript-module-dotexports-anonymous-function |  3 +
 ...avascript-module-dotexports-arrow-function |  4 ++
 ...ascript-module-dotexports-arrow-function-2 |  4 ++
 ...ascript-module-dotexports-arrow-function-3 |  1 +
 ...script-module-dotexports-assigned-function |  1 +
 ...ipt-module-dotexports-async-arrow-function |  3 +
 ...t-module-dotexports-async-arrow-function-2 |  4 ++
 ...t-module-dotexports-async-arrow-function-3 |  1 +
 ...avascript-module-dotexports-async-function |  4 ++
 ...exports-async-generator-anonymous-function |  5 ++
 ...ports-async-generator-anonymous-function-2 |  5 ++
 ...module-dotexports-async-generator-function |  5 ++
 ...dule-dotexports-async-generator-function-2 |  5 ++
 t/t4018/javascript-module-dotexports-function |  4 ++
 ...le-dotexports-generator-anonymous-function |  5 ++
 ...-dotexports-generator-anonymous-function-2 |  5 ++
 ...cript-module-dotexports-generator-function |  5 ++
 ...ipt-module-dotexports-generator-function-2 |  5 ++
 t/t4018/javascript-skip-function-calls        |  7 +++
 t/t4018/javascript-skip-reserved-words        | 38 ++++++++++++
 t/t4034-diff-words.sh                         |  1 +
 t/t4034/javascript/expect                     | 26 ++++++--
 t/t4034/javascript/post                       | 16 +++--
 t/t4034/javascript/pre                        | 16 +++--
 userdiff.c                                    | 62 +++++++++++--------
 77 files changed, 408 insertions(+), 46 deletions(-)
 create mode 100644 t/t4018/javascript-anonymous-assigned-function
 create mode 100644 t/t4018/javascript-arrow-assigned-function
 create mode 100644 t/t4018/javascript-arrow-assigned-function-2
 create mode 100644 t/t4018/javascript-arrow-assigned-function-3
 create mode 100644 t/t4018/javascript-assigned-function
 create mode 100644 t/t4018/javascript-async-assigned-function
 create mode 100644 t/t4018/javascript-class-get-function
 create mode 100644 t/t4018/javascript-class-set-function
 create mode 100644 t/t4018/javascript-class-static-function
 create mode 100644 t/t4018/javascript-class-unprefixed-function
 create mode 100644 t/t4018/javascript-dotexpors-async-anonymous-function
 create mode 100644 t/t4018/javascript-dotexports-anonymous-function
 create mode 100644 t/t4018/javascript-dotexports-arrow-function
 create mode 100644 t/t4018/javascript-dotexports-arrow-function-2
 create mode 100644 t/t4018/javascript-dotexports-arrow-function-3
 create mode 100644 t/t4018/javascript-dotexports-assigned-function
 create mode 100644 t/t4018/javascript-dotexports-async-arrow-function
 create mode 100644 t/t4018/javascript-dotexports-async-arrow-function-2
 create mode 100644 t/t4018/javascript-dotexports-async-arrow-function-3
 create mode 100644 t/t4018/javascript-dotexports-async-function
 create mode 100644 t/t4018/javascript-dotexports-async-generator-anonymous-function
 create mode 100644 t/t4018/javascript-dotexports-async-generator-anonymous-function-2
 create mode 100644 t/t4018/javascript-dotexports-async-generator-function
 create mode 100644 t/t4018/javascript-dotexports-async-generator-function-2
 create mode 100644 t/t4018/javascript-dotexports-function
 create mode 100644 t/t4018/javascript-dotexports-generator-anonymous-function
 create mode 100644 t/t4018/javascript-dotexports-generator-anonymous-function-2
 create mode 100644 t/t4018/javascript-dotexports-generator-function
 create mode 100644 t/t4018/javascript-dotexports-generator-function-2
 create mode 100644 t/t4018/javascript-export-arrow-function
 create mode 100644 t/t4018/javascript-export-async-anonymous-assigned-function
 create mode 100644 t/t4018/javascript-export-async-arrow-function
 create mode 100644 t/t4018/javascript-export-async-function
 create mode 100644 t/t4018/javascript-export-async-generator-anonymous-assigned-function
 create mode 100644 t/t4018/javascript-export-async-generator-anonymous-assigned-function-2
 create mode 100644 t/t4018/javascript-export-async-generator-assigned-function
 create mode 100644 t/t4018/javascript-export-async-generator-assigned-function-2
 create mode 100644 t/t4018/javascript-export-async-generator-function
 create mode 100644 t/t4018/javascript-export-async-generator-function-2
 create mode 100644 t/t4018/javascript-export-function
 create mode 100644 t/t4018/javascript-export-generator-assigned-function
 create mode 100644 t/t4018/javascript-export-generator-assigned-function-2
 create mode 100644 t/t4018/javascript-export-generator-function
 create mode 100644 t/t4018/javascript-export-generator-function-2
 create mode 100644 t/t4018/javascript-generator-assigned-function
 create mode 100644 t/t4018/javascript-generator-assigned-function-2
 create mode 100644 t/t4018/javascript-method-unprefixed-function
 create mode 100644 t/t4018/javascript-module-dotexports-anonymous-function
 create mode 100644 t/t4018/javascript-module-dotexports-arrow-function
 create mode 100644 t/t4018/javascript-module-dotexports-arrow-function-2
 create mode 100644 t/t4018/javascript-module-dotexports-arrow-function-3
 create mode 100644 t/t4018/javascript-module-dotexports-assigned-function
 create mode 100644 t/t4018/javascript-module-dotexports-async-arrow-function
 create mode 100644 t/t4018/javascript-module-dotexports-async-arrow-function-2
 create mode 100644 t/t4018/javascript-module-dotexports-async-arrow-function-3
 create mode 100644 t/t4018/javascript-module-dotexports-async-function
 create mode 100644 t/t4018/javascript-module-dotexports-async-generator-anonymous-function
 create mode 100644 t/t4018/javascript-module-dotexports-async-generator-anonymous-function-2
 create mode 100644 t/t4018/javascript-module-dotexports-async-generator-function
 create mode 100644 t/t4018/javascript-module-dotexports-async-generator-function-2
 create mode 100644 t/t4018/javascript-module-dotexports-function
 create mode 100644 t/t4018/javascript-module-dotexports-generator-anonymous-function
 create mode 100644 t/t4018/javascript-module-dotexports-generator-anonymous-function-2
 create mode 100644 t/t4018/javascript-module-dotexports-generator-function
 create mode 100644 t/t4018/javascript-module-dotexports-generator-function-2
 create mode 100644 t/t4018/javascript-skip-function-calls
 create mode 100644 t/t4018/javascript-skip-reserved-words

-- 
2.50.0.rc0.62.g658f0ae201.dirty