POSIX API for EVL

POSIX API for EVL

[Pierre FICHEUX]

Hi all,

I've read some discussions (may 2024) about the lack of POSIX skin in
EVL.  Supporting several skins just like for Xenomai 3 is not
really useful but POSIX is an industrial standard and so for people
willing to use Xenomai instead of PREEMPT_RT.

In the EVL documentation
(https://evlproject.org/core/user-api/thread/) I can read :

The main kernel’s thread is the basic execution unit in EVL.
The most common kind of EVL threads is a regular POSIX thread started
by pthread_create(3) which has attached itself to the EVL core by a
call to evl_attach_self().

Don't you think a kind of POSIX "wrapper" could be useful or is that a
silly question ? Did I miss anything ?

I'm about to start a study for a big industrial company and the lack
of POSIX support is a real issue.

Regards

-- 

Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
                             http://www.smile.fr
                             https://smile.eu/fr/offres/embarque-iot
I would love to change the world, but they won't give me the source code

Re: POSIX API for EVL

[Pierre FICHEUX]

Maybe we could talk about it during the next Xenomai community call ?


Le ven. 20 sept. 2024 à 07:57, Pierre FICHEUX
<pierre.ficheux@smile.fr> a écrit :
>
> Hi all,
>
> I've read some discussions (may 2024) about the lack of POSIX skin in
> EVL.  Supporting several skins just like for Xenomai 3 is not
> really useful but POSIX is an industrial standard and so for people
> willing to use Xenomai instead of PREEMPT_RT.
>
> In the EVL documentation
> (https://evlproject.org/core/user-api/thread/) I can read :
>
> The main kernel’s thread is the basic execution unit in EVL.
> The most common kind of EVL threads is a regular POSIX thread started
> by pthread_create(3) which has attached itself to the EVL core by a
> call to evl_attach_self().
>
> Don't you think a kind of POSIX "wrapper" could be useful or is that a
> silly question ? Did I miss anything ?
>
> I'm about to start a study for a big industrial company and the lack
> of POSIX support is a real issue.
>
> Regards
>
> --
>
> Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
>                              http://www.smile.fr
>                              https://smile.eu/fr/offres/embarque-iot
> I would love to change the world, but they won't give me the source code



--

Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
                             http://www.smile.fr
                             https://smile.eu/fr/offres/embarque-iot
I would love to change the world, but they won't give me the source code

Re: POSIX API for EVL

[Florian Bezdeka]

Hi Pierre,

On Fri, 2024-09-20 at 09:41 +0200, Pierre FICHEUX wrote:
> Maybe we could talk about it during the next Xenomai community call ?

Everyone is invited to attend to the next community call and ask
questions or bring up topics that should be discussed. So yes, that
would be an option.

Up to now most attendees in the community call have/had Xenomai 3
background. As Xenomai 4 is mainly driven by Philippe, we might have to
ask him if he could join as well. Now in CC.

According to my calendar the next call is scheduled for Wednesday, 16th
of October, 9:00 AM in the central European timezone. Dail-in
instructions can be found on the list and are normally re-shared a
couple of minutes before the event.

Btw: The "new" Xenomai mailing list is at xenomai@lists.linux.dev.

HTH,
Florian

> 
> 
> Le ven. 20 sept. 2024 à 07:57, Pierre FICHEUX
> <pierre.ficheux@smile.fr> a écrit :
> > 
> > Hi all,
> > 
> > I've read some discussions (may 2024) about the lack of POSIX skin in
> > EVL.  Supporting several skins just like for Xenomai 3 is not
> > really useful but POSIX is an industrial standard and so for people
> > willing to use Xenomai instead of PREEMPT_RT.
> > 
> > In the EVL documentation
> > (https://evlproject.org/core/user-api/thread/) I can read :
> > 
> > The main kernel’s thread is the basic execution unit in EVL.
> > The most common kind of EVL threads is a regular POSIX thread started
> > by pthread_create(3) which has attached itself to the EVL core by a
> > call to evl_attach_self().
> > 
> > Don't you think a kind of POSIX "wrapper" could be useful or is that a
> > silly question ? Did I miss anything ?
> > 
> > I'm about to start a study for a big industrial company and the lack
> > of POSIX support is a real issue.
> > 
> > Regards
> > 
> > --
> > 
> > Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
> >                              http://www.smile.fr
> >                              https://smile.eu/fr/offres/embarque-iot
> > I would love to change the world, but they won't give me the source code
> 
> 
> 
> --
> 
> Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
>                              http://www.smile.fr
>                              https://smile.eu/fr/offres/embarque-iot
> I would love to change the world, but they won't give me the source code
> 

Re: POSIX API for EVL

[Philippe Gerum]

Pierre FICHEUX <pierre.ficheux@smile.fr> writes:

> Hi all,
>
> I've read some discussions (may 2024) about the lack of POSIX skin in
> EVL.  Supporting several skins just like for Xenomai 3 is not
> really useful but POSIX is an industrial standard and so for people
> willing to use Xenomai instead of PREEMPT_RT.
>
> In the EVL documentation
> (https://evlproject.org/core/user-api/thread/) I can read :
>
> The main kernel’s thread is the basic execution unit in EVL.
> The most common kind of EVL threads is a regular POSIX thread started
> by pthread_create(3) which has attached itself to the EVL core by a
> call to evl_attach_self().
>
> Don't you think a kind of POSIX "wrapper" could be useful or is that a
> silly question ? Did I miss anything ?
>
> I'm about to start a study for a big industrial company and the lack
> of POSIX support is a real issue.
>

Years ago, there has been a proposal for a Common Xenomai Platform
effort (aka CXP). Its purpose was to establish bridges between the
xenomai3 and xenomai4 systems, user space (e.g. POSIX interface for evl)
and kernel space (e.g. RTDM emulation over the evl kernel API)
included. AFAIR, this proposal did not get any traction, no user
feedback showing interest on this list, so it had to be ditched as a
consequence. The only way to revive this effort is contributing detailed
ideas, a plan and some code.

-- 
Philippe.

Re: POSIX API for EVL

[Pierre FICHEUX]

Hi Philippe,

Yes, I do remember.

https://source.denx.de/Xenomai/xenomai/-/wikis/Common_Xenomai_Platform

The lack of interest is a bit strange as the industry is mostly based on POSIX.

regards


Le mar. 8 oct. 2024 à 20:22, Philippe Gerum <rpm@xenomai.org> a écrit :
>
> Pierre FICHEUX <pierre.ficheux@smile.fr> writes:
>
> > Hi all,
> >
> > I've read some discussions (may 2024) about the lack of POSIX skin in
> > EVL.  Supporting several skins just like for Xenomai 3 is not
> > really useful but POSIX is an industrial standard and so for people
> > willing to use Xenomai instead of PREEMPT_RT.
> >
> > In the EVL documentation
> > (https://evlproject.org/core/user-api/thread/) I can read :
> >
> > The main kernel’s thread is the basic execution unit in EVL.
> > The most common kind of EVL threads is a regular POSIX thread started
> > by pthread_create(3) which has attached itself to the EVL core by a
> > call to evl_attach_self().
> >
> > Don't you think a kind of POSIX "wrapper" could be useful or is that a
> > silly question ? Did I miss anything ?
> >
> > I'm about to start a study for a big industrial company and the lack
> > of POSIX support is a real issue.
> >
>
> Years ago, there has been a proposal for a Common Xenomai Platform
> effort (aka CXP). Its purpose was to establish bridges between the
> xenomai3 and xenomai4 systems, user space (e.g. POSIX interface for evl)
> and kernel space (e.g. RTDM emulation over the evl kernel API)
> included. AFAIR, this proposal did not get any traction, no user
> feedback showing interest on this list, so it had to be ditched as a
> consequence. The only way to revive this effort is contributing detailed
> ideas, a plan and some code.
>
> --
> Philippe.



-- 

Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
                             http://www.smile.fr
                             https://smile.eu/fr/offres/embarque-iot
I would love to change the world, but they won't give me the source code

Re: POSIX API for EVL

[Jan Kiszka]

On 09.10.24 09:53, Pierre FICHEUX wrote:
> Hi Philippe,
> 
> Yes, I do remember.
> 
> https://source.denx.de/Xenomai/xenomai/-/wikis/Common_Xenomai_Platform
> 
> The lack of interest is a bit strange as the industry is mostly based on POSIX.
> 

I would say the challenge of moving forward is right now that users
already on Xenomai 3 are happy enough to stay so far, plus some have the
blocker of missing POSIX with EVL. So, there is now an upfront invest
needed to enable POSIX over EVL in order to evaluate if the own
application works even better there. This is at least the case for our
internal users.

But my plan remains to enhance portability from Xenomai 3 to 4 so that
more comparisons are possible, and we may eventually even consolidate
over a single code base again.

Jan

> regards
> 
> 
> Le mar. 8 oct. 2024 à 20:22, Philippe Gerum <rpm@xenomai.org> a écrit :
>>
>> Pierre FICHEUX <pierre.ficheux@smile.fr> writes:
>>
>>> Hi all,
>>>
>>> I've read some discussions (may 2024) about the lack of POSIX skin in
>>> EVL.  Supporting several skins just like for Xenomai 3 is not
>>> really useful but POSIX is an industrial standard and so for people
>>> willing to use Xenomai instead of PREEMPT_RT.
>>>
>>> In the EVL documentation
>>> (https://evlproject.org/core/user-api/thread/) I can read :
>>>
>>> The main kernel’s thread is the basic execution unit in EVL.
>>> The most common kind of EVL threads is a regular POSIX thread started
>>> by pthread_create(3) which has attached itself to the EVL core by a
>>> call to evl_attach_self().
>>>
>>> Don't you think a kind of POSIX "wrapper" could be useful or is that a
>>> silly question ? Did I miss anything ?
>>>
>>> I'm about to start a study for a big industrial company and the lack
>>> of POSIX support is a real issue.
>>>
>>
>> Years ago, there has been a proposal for a Common Xenomai Platform
>> effort (aka CXP). Its purpose was to establish bridges between the
>> xenomai3 and xenomai4 systems, user space (e.g. POSIX interface for evl)
>> and kernel space (e.g. RTDM emulation over the evl kernel API)
>> included. AFAIR, this proposal did not get any traction, no user
>> feedback showing interest on this list, so it had to be ditched as a
>> consequence. The only way to revive this effort is contributing detailed
>> ideas, a plan and some code.
>>
>> --
>> Philippe.
> 
> 
> 

-- 
Siemens AG, Technology
Linux Expert Center

Re: POSIX API for EVL

[Jan Kiszka]

On 09.10.24 13:20, Pierre FICHEUX wrote:
> Hi Jan,
> 
> I don't understand: how  can you  "enhance portability" without POSIX ?

POSIX would be a precondition for us to get any traction with our users.
IOW, when we have the time to look into that, we will do so - but
nothing is scheduled so far.

Jan

> 
> regards
> 
> Le mer. 9 oct. 2024 à 11:59, Jan Kiszka <jan.kiszka@siemens.com
> <mailto:jan.kiszka@siemens.com>> a écrit :
> 
>     On 09.10.24 09:53, Pierre FICHEUX wrote:
>     > Hi Philippe,
>     >
>     > Yes, I do remember.
>     >
>     > https://source.denx.de/Xenomai/xenomai/-/wikis/
>     Common_Xenomai_Platform <https://source.denx.de/Xenomai/xenomai/-/
>     wikis/Common_Xenomai_Platform>
>     >
>     > The lack of interest is a bit strange as the industry is mostly
>     based on POSIX.
>     >
> 
>     I would say the challenge of moving forward is right now that users
>     already on Xenomai 3 are happy enough to stay so far, plus some have the
>     blocker of missing POSIX with EVL. So, there is now an upfront invest
>     needed to enable POSIX over EVL in order to evaluate if the own
>     application works even better there. This is at least the case for our
>     internal users.
> 
>     But my plan remains to enhance portability from Xenomai 3 to 4 so that
>     more comparisons are possible, and we may eventually even consolidate
>     over a single code base again.
> 
>     Jan
> 
>     > regards
>     >
>     >
>     > Le mar. 8 oct. 2024 à 20:22, Philippe Gerum <rpm@xenomai.org
>     <mailto:rpm@xenomai.org>> a écrit :
>     >>
>     >> Pierre FICHEUX <pierre.ficheux@smile.fr
>     <mailto:pierre.ficheux@smile.fr>> writes:
>     >>
>     >>> Hi all,
>     >>>
>     >>> I've read some discussions (may 2024) about the lack of POSIX
>     skin in
>     >>> EVL.  Supporting several skins just like for Xenomai 3 is not
>     >>> really useful but POSIX is an industrial standard and so for people
>     >>> willing to use Xenomai instead of PREEMPT_RT.
>     >>>
>     >>> In the EVL documentation
>     >>> (https://evlproject.org/core/user-api/thread/ <https://
>     evlproject.org/core/user-api/thread/>) I can read :
>     >>>
>     >>> The main kernel’s thread is the basic execution unit in EVL.
>     >>> The most common kind of EVL threads is a regular POSIX thread
>     started
>     >>> by pthread_create(3) which has attached itself to the EVL core by a
>     >>> call to evl_attach_self().
>     >>>
>     >>> Don't you think a kind of POSIX "wrapper" could be useful or is
>     that a
>     >>> silly question ? Did I miss anything ?
>     >>>
>     >>> I'm about to start a study for a big industrial company and the lack
>     >>> of POSIX support is a real issue.
>     >>>
>     >>
>     >> Years ago, there has been a proposal for a Common Xenomai Platform
>     >> effort (aka CXP). Its purpose was to establish bridges between the
>     >> xenomai3 and xenomai4 systems, user space (e.g. POSIX interface
>     for evl)
>     >> and kernel space (e.g. RTDM emulation over the evl kernel API)
>     >> included. AFAIR, this proposal did not get any traction, no user
>     >> feedback showing interest on this list, so it had to be ditched as a
>     >> consequence. The only way to revive this effort is contributing
>     detailed
>     >> ideas, a plan and some code.
>     >>
>     >> --
>     >> Philippe.
>     >
>     >
>     >
> 
>     -- 
>     Siemens AG, Technology
>     Linux Expert Center
> 
> 
> 
> -- 
> 
> Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr <mailto:pierre.ficheux@smile.fr>
>                              http://www.smile.fr <http://www.smile.fr/>
>                              https://smile.eu/fr/offres/embarque-iot <https://smile.eu/fr/offres/embarque-iot>
> I would love to change the world, but they won't give me the source code
> 


-- 
Siemens AG, Technology
Linux Expert Center

Re: POSIX API for EVL

[Philippe Gerum]

Jan Kiszka <jan.kiszka@siemens.com> writes:

> On 09.10.24 09:53, Pierre FICHEUX wrote:
>> Hi Philippe,
>> 
>> Yes, I do remember.
>> 
>> https://source.denx.de/Xenomai/xenomai/-/wikis/Common_Xenomai_Platform
>> 
>> The lack of interest is a bit strange as the industry is mostly based on POSIX.
>> 
>
> I would say the challenge of moving forward is right now that users
> already on Xenomai 3 are happy enough to stay so far, plus some have the
> blocker of missing POSIX with EVL. So, there is now an upfront invest
> needed to enable POSIX over EVL in order to evaluate if the own
> application works even better there. This is at least the case for our
> internal users.
>

I agree, this makes sense. This said, I may emphasize a couple of major
differences between x3 and x4:

- SMP scalability. Because of its outdated serialization scheme, x3
  performs properly (latency-wise) on up to 4-6? cores max. running
  real-time threads concurrently. x4 does not have the big-lock
  scalability issue x3 has, so virtually any number of cores can be used
  for concurrent real-time activity. Granted, not every rt app needs a
  truckload of CPUs to run concurrently, however the fine-grained
  locking scheme x4 implements like the rest of the kernel does also
  greatly improves concurrency within any given real-time CPU
  (typically, this shows when running the context switcher stress test
  alongside a latency-sensitive application on mid-range or low-end hw).

- Complexity, maintenance burden. Only looking at the Cobalt vs EVL core
  code footprints, including the netstack but no drivers, excluding user
  APIS, we currently have:

{rpm@pyro} cloc kernel/evl include/uapi/evl include/evl
     147 text files.
     139 unique files.                              
       8 files ignored.

github.com/AlDanial/cloc v 2.02  T=0.11 s (1268.6 files/s, 281809.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                               48           3955           4078          16404
C/C++ Header                    83           1260            932           4190
...
-------------------------------------------------------------------------------

VS

{rpm@pyro} cloc kernel/cobalt include/cobalt/kernel kernel/drivers/net/stack
     355 text files.
     329 unique files.                              
      26 files ignored.

github.com/AlDanial/cloc v 2.02  T=0.22 s (1479.4 files/s, 341314.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                              109           8781          11091          33066
C/C++ Header                   198           3762           6128          12771
...
-------------------------------------------------------------------------------

- Integration. EVL reuses the standard VFS abstraction to implement the
  everything-is-a-file scheme for accessing its features/elements, which
  x3 cannot. As a result, a real-time capable driver is a regular Linux
  character driver in essence, plus there is no redundant rt-specific
  file descriptor namespace. IOW, the regular Linux file resource
  management applies to any element created by the EVL core (reference
  tracking, release, dups and so on).

- EVL is Dovetail-native. For instance, valgrind support is readily
  there and for-free, because the core relies on the prctl() redirection
  to receive real-time syscalls, which x3 won't be able to use until its
  way of marshalling syscall args is entirely reworked. The real-time
  network stack can provide UDP and raw ethernet socket support by
  sharing few but important portions of the regular network stack when
  running out-of-band, which explains why the x4 netstack exhibits
  roughly a third of the x3 footprints.

So indeed, x4 may not be a requirement to some/many, but still, there
are some advantages there.

> But my plan remains to enhance portability from Xenomai 3 to 4 so that
> more comparisons are possible, and we may eventually even consolidate
> over a single code base again.
>

Ok, I'd be there to help if/when this happens.

-- 
Philippe.

Re: POSIX API for EVL

[Pierre FICHEUX]

I have a contract with a (big) customer to study several flavours of
Linux RT   for their future projects.

As it's for the future, I'm quite sure v4 should be better...as they
want to fire VxWorks !

A+



Le mer. 9 oct. 2024 à 13:40, Philippe Gerum <rpm@xenomai.org> a écrit :
>
> Jan Kiszka <jan.kiszka@siemens.com> writes:
>
> > On 09.10.24 09:53, Pierre FICHEUX wrote:
> >> Hi Philippe,
> >>
> >> Yes, I do remember.
> >>
> >> https://source.denx.de/Xenomai/xenomai/-/wikis/Common_Xenomai_Platform
> >>
> >> The lack of interest is a bit strange as the industry is mostly based on POSIX.
> >>
> >
> > I would say the challenge of moving forward is right now that users
> > already on Xenomai 3 are happy enough to stay so far, plus some have the
> > blocker of missing POSIX with EVL. So, there is now an upfront invest
> > needed to enable POSIX over EVL in order to evaluate if the own
> > application works even better there. This is at least the case for our
> > internal users.
> >
>
> I agree, this makes sense. This said, I may emphasize a couple of major
> differences between x3 and x4:
>
> - SMP scalability. Because of its outdated serialization scheme, x3
>   performs properly (latency-wise) on up to 4-6? cores max. running
>   real-time threads concurrently. x4 does not have the big-lock
>   scalability issue x3 has, so virtually any number of cores can be used
>   for concurrent real-time activity. Granted, not every rt app needs a
>   truckload of CPUs to run concurrently, however the fine-grained
>   locking scheme x4 implements like the rest of the kernel does also
>   greatly improves concurrency within any given real-time CPU
>   (typically, this shows when running the context switcher stress test
>   alongside a latency-sensitive application on mid-range or low-end hw).
>
> - Complexity, maintenance burden. Only looking at the Cobalt vs EVL core
>   code footprints, including the netstack but no drivers, excluding user
>   APIS, we currently have:
>
> {rpm@pyro} cloc kernel/evl include/uapi/evl include/evl
>      147 text files.
>      139 unique files.
>        8 files ignored.
>
> github.com/AlDanial/cloc v 2.02  T=0.11 s (1268.6 files/s, 281809.5 lines/s)
> -------------------------------------------------------------------------------
> Language                     files          blank        comment           code
> -------------------------------------------------------------------------------
> C                               48           3955           4078          16404
> C/C++ Header                    83           1260            932           4190
> ...
> -------------------------------------------------------------------------------
>
> VS
>
> {rpm@pyro} cloc kernel/cobalt include/cobalt/kernel kernel/drivers/net/stack
>      355 text files.
>      329 unique files.
>       26 files ignored.
>
> github.com/AlDanial/cloc v 2.02  T=0.22 s (1479.4 files/s, 341314.9 lines/s)
> -------------------------------------------------------------------------------
> Language                     files          blank        comment           code
> -------------------------------------------------------------------------------
> C                              109           8781          11091          33066
> C/C++ Header                   198           3762           6128          12771
> ...
> -------------------------------------------------------------------------------
>
> - Integration. EVL reuses the standard VFS abstraction to implement the
>   everything-is-a-file scheme for accessing its features/elements, which
>   x3 cannot. As a result, a real-time capable driver is a regular Linux
>   character driver in essence, plus there is no redundant rt-specific
>   file descriptor namespace. IOW, the regular Linux file resource
>   management applies to any element created by the EVL core (reference
>   tracking, release, dups and so on).
>
> - EVL is Dovetail-native. For instance, valgrind support is readily
>   there and for-free, because the core relies on the prctl() redirection
>   to receive real-time syscalls, which x3 won't be able to use until its
>   way of marshalling syscall args is entirely reworked. The real-time
>   network stack can provide UDP and raw ethernet socket support by
>   sharing few but important portions of the regular network stack when
>   running out-of-band, which explains why the x4 netstack exhibits
>   roughly a third of the x3 footprints.
>
> So indeed, x4 may not be a requirement to some/many, but still, there
> are some advantages there.
>
> > But my plan remains to enhance portability from Xenomai 3 to 4 so that
> > more comparisons are possible, and we may eventually even consolidate
> > over a single code base again.
> >
>
> Ok, I'd be there to help if/when this happens.
>
> --
> Philippe.



-- 

Pierre FICHEUX -/- CTO Smile ECS, France -\- pierre.ficheux@smile.fr
                             http://www.smile.fr
                             https://smile.eu/fr/offres/embarque-iot
I would love to change the world, but they won't give me the source code

Re: POSIX API for EVL

[Sam Chiu]

Hi. I’m also interested in the POSIX wrapper for the EVL API. A few months ago, I worked on a related project and developed a simple demo program. You can find it here: https://github.com/BUPT-OS/libevl/tree/posix. However, it’s still just a toy project, only including thread creation and locking functionality, and it hasn’t undergone rigorous testing. I’m glad to see that someone finds the features useful, and I’ll continue working on this project moving forward.

Re: POSIX API for EVL

[Schaffner, Tobias]

Hi Sam,

On Thu, 2024-10-10 at 14:00 +0000, Sam Chiu wrote:
> Hi. I’m also interested in the POSIX wrapper for the EVL API. A few
> months ago, I worked on a related project and developed a simple demo
> program. You can find it here:
> https://github.com/BUPT-OS/libevl/tree/posix. However, it’s still
> just a toy project, only including thread creation and locking
> functionality, and it hasn’t undergone rigorous testing. I’m glad to
> see that someone finds the features useful, and I’ll continue working
> on this project moving forward.

thanks for the heads up. Great that you are using EVL in RROS and that
you are working on a POSIX layer. I would recommend to send the
relevant patches as an RFC to this mailing list. This way you will get
more feedback by the maintainers of this project. This could be
valuable input for your next steps and increase the chance that it can
be upstreamed later. You can do this by using git-format-patch [1] and
git-send-email [2]. Try to find a guide on how to send patches to the
kernel mailing list. Most of the rules also apply here.

Feel free to send me an email if you need further assistance with the
workflow.

Best,
Tobias

[1] https://git-scm.com/docs/git-format-patch
[2] https://git-scm.com/docs/git-send-email