[RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

When we, the Git team at Google, first embarked on the libification
journey, we didn’t have a specific consumer to build a library for but
instead were interested in the various potential benefits of
libification for many use cases such as VFSes and submodules. Without a
specific consumer, it has been difficult to evaluate the scope of what
is necessary or not even for the first library, git-std-lib. Attempting
to solve problems such as error handling, symbol collisions, and
internal/external interfaces, in addition to separating out a library
turns out to be both too complex of a task to both develop and review
all at once. While we strive to eventually build an ideal library, we
have also realized in order to make meaningful and consistent progress,
we have to solve these problems iteratively in smaller pieces. That is
why over the last month, we have been working with the jj project [1] to
understand their current usage of libgit2-rs [2] and gitoxide [3] and
future library functionality they would be interested in. In doing so,
we have built cgit-rs, a Rust wrapper around libgit.a that allows Rust
code to call various basic Git functions.

[1] https://github.com/martinvonz/jj
[2] https://github.com/rust-lang/git2-rs
[3] https://github.com/Byron/gitoxide

This series provides a small Rust wrapper library around parts of
libgit.a, and a proof-of-concept Rust executable that uses the library
to interface with Git. Additionally, we have tested building JJ with our
library and used it to replace some of the libgit2-rs uses.

This exercise has clarified a lot of things for us, and we believe that
developing this wrapper further provides benefits both for downstream
consumers and the Git project itself:
* cgit-rs provides wrappers for Rust consumers of libraries (eg. jj)
* cgit-rs suggests focus areas for libification
  * shows us what potential challenges we face with library consumers
* Git libification improves git interfaces
* Libification improves cgit-rs FFI.

We are putting error handling on hold for now since it is too complex
and we intend other CLIs to be our first customers, in which case
printing out errors is not the worst.

While the wrapper itself lives in contrib/, there are a couple of
patches that touch git.git code. These patches are necessary for the
wrapper, but not for git.git itself, which may seem unnecessary to
merge. However, I would argue that other languages (not just limited to
Rust) have issues calling functions that require a pointer to
non-generic objects and essentially require a redefinition in their own
language.

We're sending this series as RFC because there is remaining work
we'd like to do, but we'd like to get early feedback on this approach,
and particularly to ask for advice on a few topics:

* alternative methods of exposing only a subset of symbols in our
  library

* bikeshedding on the name (yes, really). There is an active, unrelated
  CGit project [4] that we only recently became aware of. We originally
  took the name "cgit" because at $DAYJOB we sometimes refer to git.git
  as "cgit" to distinguish it from jgit [5].

* gauging the level of interest in calling Git code from Rust

[4] https://git.zx2c4.com/cgit
[5] https://www.eclipse.org/jgit

Remaining work includes:

* finding a better solution to the common-main split. We should probably
  have a separate initialization function including all of main() up to
  the call to cmd_main(), which can then be exposed in cgit-rs.

* adding unit and integration tests

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default

* automating the process of exporting additional functions via cgit-rs
  (possibly with a wrapper script around bindgen [6])

[6] https://github.com/rust-lang/rust-bindgen

Finally, a quick discussion about symbol collisions: if functions are
not prepended with “libgit_” or something similar, it leaves us open to
collision issues in the future – so this probably would’ve happened with
libification in general to begin with. Therefore it seems necessary to
have to wrap all the symbols we are looking to expose. While this seem
non-ideal, we couldn’t come up with a better method. Our next best
alternative is to simply expose all symbols by default, but this leads
to symbol collisions when library users link both cgit-rs and
libgit2-rs.


Calvin Wan (2):
  contrib/cgit-rs: add repo initialization and config access
  contrib/cgit-rs: add a subset of configset wrappers

Josh Steadmon (4):
  common-main: split common_exit() into a new file
  repository: add initialize_repo wrapper without pointer
  contrib/cgit-rs: introduce Rust wrapper for libgit.a
  config: add git_configset_alloc

 .gitignore                             |  1 +
 Makefile                               | 14 ++++
 common-exit.c                          | 26 +++++++
 common-main.c                          | 24 -------
 config.c                               |  5 ++
 config.h                               |  5 ++
 contrib/cgit-rs/Cargo.lock             | 99 ++++++++++++++++++++++++++
 contrib/cgit-rs/Cargo.toml             | 17 +++++
 contrib/cgit-rs/README.md              | 15 ++++
 contrib/cgit-rs/build.rs               | 33 +++++++++
 contrib/cgit-rs/public_symbol_export.c | 72 +++++++++++++++++++
 contrib/cgit-rs/public_symbol_export.h | 26 +++++++
 contrib/cgit-rs/src/lib.rs             | 81 +++++++++++++++++++++
 contrib/cgit-rs/src/main.rs            | 44 ++++++++++++
 repository.c                           |  9 +++
 repository.h                           |  1 +
 16 files changed, 448 insertions(+), 24 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 contrib/cgit-rs/Cargo.lock
 create mode 100644 contrib/cgit-rs/Cargo.toml
 create mode 100644 contrib/cgit-rs/README.md
 create mode 100644 contrib/cgit-rs/build.rs
 create mode 100644 contrib/cgit-rs/public_symbol_export.c
 create mode 100644 contrib/cgit-rs/public_symbol_export.h
 create mode 100644 contrib/cgit-rs/src/lib.rs
 create mode 100644 contrib/cgit-rs/src/main.rs


base-commit: 557ae147e6cdc9db121269b058c757ac5092f9c9
-- 
2.46.0.rc2.264.g509ed76dc8-goog

[RFC PATCH 1/6] common-main: split common_exit() into a new file

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  1 +
 common-exit.c | 26 ++++++++++++++++++++++++++
 common-main.c | 24 ------------------------
 3 files changed, 27 insertions(+), 24 deletions(-)
 create mode 100644 common-exit.c

diff --git a/Makefile b/Makefile
index 3eab701b10..1cac51a4f7 100644
--- a/Makefile
+++ b/Makefile
@@ -979,6 +979,7 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..af4dea049e 100644
--- a/common-main.c
+++ b/common-main.c
@@ -66,27 +66,3 @@ int main(int argc, const char **argv)
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [RFC PATCH 1/6] common-main: split common_exit() into a new file

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> diff --git a/common-exit.c b/common-exit.c
> new file mode 100644
> index 0000000000..1aaa538be3
> --- /dev/null
> +++ b/common-exit.c
> @@ -0,0 +1,26 @@
> +#include "git-compat-util.h"
> +#include "trace2.h"
> +
> +static void check_bug_if_BUG(void)
> +{
> +	if (!bug_called_must_BUG)
> +		return;
> +	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
> +}

Nice that this can stay file-scope static.

> +/* We wrap exit() to call common_exit() in git-compat-util.h */
> +int common_exit(const char *file, int line, int code)
> +{
> +	/*
> +	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
> +	 * to e.g. turn -1 into 255. On a POSIX system this is
> +	 * redundant, see exit(3) and wait(2), but as it doesn't harm
> +	 * anything there we don't need to guard this with an "ifdef".
> +	 */
> +	code &= 0xff;
> +
> +	check_bug_if_BUG();
> +	trace2_cmd_exit_fl(file, line, code);
> +
> +	return code;
> +}

I wonder if at least the part that primes the trace2 system needs to
also be split out of the file that defines our own main(), though.

Are libgit.a users now responsible for calling things like
trace2_initialize_clock(), trace2_initialize(), etc., themselves?  I
am wondering if these calls are encapsulated into a simple helper
function, say common_startup(), then the story we need to tell the
libgit.a users may become simpler, i.e. you call common_startup, do
your things, and then you somehow cause common_exit() to be called.

Re: [RFC PATCH 1/6] common-main: split common_exit() into a new file

[Josh Steadmon]

On 2024.08.07 14:21, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > diff --git a/common-exit.c b/common-exit.c
> > new file mode 100644
> > index 0000000000..1aaa538be3
> > --- /dev/null
> > +++ b/common-exit.c
> > @@ -0,0 +1,26 @@
> > +#include "git-compat-util.h"
> > +#include "trace2.h"
> > +
> > +static void check_bug_if_BUG(void)
> > +{
> > +	if (!bug_called_must_BUG)
> > +		return;
> > +	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
> > +}
> 
> Nice that this can stay file-scope static.
> 
> > +/* We wrap exit() to call common_exit() in git-compat-util.h */
> > +int common_exit(const char *file, int line, int code)
> > +{
> > +	/*
> > +	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
> > +	 * to e.g. turn -1 into 255. On a POSIX system this is
> > +	 * redundant, see exit(3) and wait(2), but as it doesn't harm
> > +	 * anything there we don't need to guard this with an "ifdef".
> > +	 */
> > +	code &= 0xff;
> > +
> > +	check_bug_if_BUG();
> > +	trace2_cmd_exit_fl(file, line, code);
> > +
> > +	return code;
> > +}
> 
> I wonder if at least the part that primes the trace2 system needs to
> also be split out of the file that defines our own main(), though.
> 
> Are libgit.a users now responsible for calling things like
> trace2_initialize_clock(), trace2_initialize(), etc., themselves?  I
> am wondering if these calls are encapsulated into a simple helper
> function, say common_startup(), then the story we need to tell the
> libgit.a users may become simpler, i.e. you call common_startup, do
> your things, and then you somehow cause common_exit() to be called.

Yeah, I actually already have a fixup patch to move the initialization
into a separate object file as well. Will include in V2.

[RFC PATCH 2/6] repository: add initialize_repo wrapper without pointer

[Josh Steadmon]

Non-C external consumers of libgit.a have to redefine the `repository`
object in their own language if they want to call
initialize_repository() to ensure memory for the object is allocated
correctly. This is not ideal for external consumers that have no need
for the entire `the_repository` object but need to call other functions
from an initialized repository. Therefore, add a friendly
initialize_repository() wrapper without a `the_repository` pointer.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 repository.c | 9 +++++++++
 repository.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/repository.c b/repository.c
index 9825a30899..15cb9ee735 100644
--- a/repository.c
+++ b/repository.c
@@ -81,6 +81,15 @@ void initialize_repository(struct repository *repo)
 		set_default_hash_algo(repo);
 }
 
+/*
+ * For non-C external consumers of libgit.a that do not need access
+ * to the entire `the_repository` object.
+ */
+void initialize_the_repository(void)
+{
+	initialize_repository(the_repository);
+}
+
 static void expand_base_dir(char **out, const char *in,
 			    const char *base_dir, const char *def_in)
 {
diff --git a/repository.h b/repository.h
index af6ea0a62c..65c9158866 100644
--- a/repository.h
+++ b/repository.h
@@ -227,6 +227,7 @@ void repo_set_compat_hash_algo(struct repository *repo, int compat_algo);
 void repo_set_ref_storage_format(struct repository *repo,
 				 enum ref_storage_format format);
 void initialize_repository(struct repository *repo);
+void initialize_the_repository(void);
 RESULT_MUST_BE_USED
 int repo_init(struct repository *r, const char *gitdir, const char *worktree);
 
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [RFC PATCH 2/6] repository: add initialize_repo wrapper without pointer

[Mike Hommey]

On Wed, Aug 07, 2024 at 11:21:27AM -0700, Josh Steadmon wrote:
> Non-C external consumers of libgit.a have to redefine the `repository`
> object in their own language if they want to call
> initialize_repository() to ensure memory for the object is allocated
> correctly. This is not ideal for external consumers that have no need
> for the entire `the_repository` object but need to call other functions
> from an initialized repository. Therefore, add a friendly
> initialize_repository() wrapper without a `the_repository` pointer.

Technically speaking, you don't really need this.

You can define `repository` as an opaque type in Rust:
```
#[allow(non_camel_case_types)]
#[repr(C)]
pub struct repository([u8; 0]);
```

And define `the_repository` as an extern symbol:
```
extern "C" {
    pub static mut the_repository: *mut repository;
}
```

Mike

Re: [RFC PATCH 2/6] repository: add initialize_repo wrapper without pointer

[Josh Steadmon]

On 2024.08.08 07:52, Mike Hommey wrote:
> On Wed, Aug 07, 2024 at 11:21:27AM -0700, Josh Steadmon wrote:
> > Non-C external consumers of libgit.a have to redefine the `repository`
> > object in their own language if they want to call
> > initialize_repository() to ensure memory for the object is allocated
> > correctly. This is not ideal for external consumers that have no need
> > for the entire `the_repository` object but need to call other functions
> > from an initialized repository. Therefore, add a friendly
> > initialize_repository() wrapper without a `the_repository` pointer.
> 
> Technically speaking, you don't really need this.
> 
> You can define `repository` as an opaque type in Rust:
> ```
> #[allow(non_camel_case_types)]
> #[repr(C)]
> pub struct repository([u8; 0]);
> ```
> 
> And define `the_repository` as an extern symbol:
> ```
> extern "C" {
>     pub static mut the_repository: *mut repository;
> }
> ```
> 
> Mike

I've actually already done a refactor for V2 that will avoid using this
patch entirely, but thank you for the pointer. We do something similar
to opaquely wrap configset pointers in a later patch (we use an empty
enum there, I'm not sure whether that approach or a zero-size array is
preferred).

Re: [RFC PATCH 2/6] repository: add initialize_repo wrapper without pointer

[Mike Hommey]

On Wed, Aug 07, 2024 at 04:23:05PM -0700, Josh Steadmon wrote:
> On 2024.08.08 07:52, Mike Hommey wrote:
> > On Wed, Aug 07, 2024 at 11:21:27AM -0700, Josh Steadmon wrote:
> > > Non-C external consumers of libgit.a have to redefine the `repository`
> > > object in their own language if they want to call
> > > initialize_repository() to ensure memory for the object is allocated
> > > correctly. This is not ideal for external consumers that have no need
> > > for the entire `the_repository` object but need to call other functions
> > > from an initialized repository. Therefore, add a friendly
> > > initialize_repository() wrapper without a `the_repository` pointer.
> > 
> > Technically speaking, you don't really need this.
> > 
> > You can define `repository` as an opaque type in Rust:
> > ```
> > #[allow(non_camel_case_types)]
> > #[repr(C)]
> > pub struct repository([u8; 0]);
> > ```
> > 
> > And define `the_repository` as an extern symbol:
> > ```
> > extern "C" {
> >     pub static mut the_repository: *mut repository;
> > }
> > ```
> > 
> > Mike
> 
> I've actually already done a refactor for V2 that will avoid using this
> patch entirely, but thank you for the pointer. We do something similar
> to opaquely wrap configset pointers in a later patch (we use an empty
> enum there, I'm not sure whether that approach or a zero-size array is
> preferred).

An empty enum is a never type, I wouldn't recommend using it as an opaque
wrapper. It will likely lead to the compiler doing bad things.
https://rust-lang.github.io/never-type-initiative/RFC.html

`#[repr(C)]` and `[u8; 0]` are recommended by the nomicon.
https://doc.rust-lang.org/nomicon/ffi.html#representing-opaque-structs

(the PhantomPinned wasn't there last time I saw that page)

Mike

Re: [RFC PATCH 2/6] repository: add initialize_repo wrapper without pointer

[Josh Steadmon]

On 2024.08.08 08:29, Mike Hommey wrote:
> On Wed, Aug 07, 2024 at 04:23:05PM -0700, Josh Steadmon wrote:
> > On 2024.08.08 07:52, Mike Hommey wrote:
> > > On Wed, Aug 07, 2024 at 11:21:27AM -0700, Josh Steadmon wrote:
> > > > Non-C external consumers of libgit.a have to redefine the `repository`
> > > > object in their own language if they want to call
> > > > initialize_repository() to ensure memory for the object is allocated
> > > > correctly. This is not ideal for external consumers that have no need
> > > > for the entire `the_repository` object but need to call other functions
> > > > from an initialized repository. Therefore, add a friendly
> > > > initialize_repository() wrapper without a `the_repository` pointer.
> > > 
> > > Technically speaking, you don't really need this.
> > > 
> > > You can define `repository` as an opaque type in Rust:
> > > ```
> > > #[allow(non_camel_case_types)]
> > > #[repr(C)]
> > > pub struct repository([u8; 0]);
> > > ```
> > > 
> > > And define `the_repository` as an extern symbol:
> > > ```
> > > extern "C" {
> > >     pub static mut the_repository: *mut repository;
> > > }
> > > ```
> > > 
> > > Mike
> > 
> > I've actually already done a refactor for V2 that will avoid using this
> > patch entirely, but thank you for the pointer. We do something similar
> > to opaquely wrap configset pointers in a later patch (we use an empty
> > enum there, I'm not sure whether that approach or a zero-size array is
> > preferred).
> 
> An empty enum is a never type, I wouldn't recommend using it as an opaque
> wrapper. It will likely lead to the compiler doing bad things.
> https://rust-lang.github.io/never-type-initiative/RFC.html
> 
> `#[repr(C)]` and `[u8; 0]` are recommended by the nomicon.
> https://doc.rust-lang.org/nomicon/ffi.html#representing-opaque-structs
> 
> (the PhantomPinned wasn't there last time I saw that page)
> 
> Mike

Fixed in V2 (for ConfigSet). Thanks!

[RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce cgit-rs, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. A proof-of-concept library consumer is provided in
contrib/cgit-rs/src/main.rs. This executable can be run with `cargo run`

Symbols in cgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also contains
libgit.a and other dependent libraries, with -fvisibility=hidden to hide
all symbols within those libraries that haven't been exposed with a
visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                             |  1 +
 Makefile                               | 13 ++++++++++
 contrib/cgit-rs/Cargo.lock             | 16 +++++++++++++
 contrib/cgit-rs/Cargo.toml             | 16 +++++++++++++
 contrib/cgit-rs/README.md              | 15 ++++++++++++
 contrib/cgit-rs/build.rs               | 33 ++++++++++++++++++++++++++
 contrib/cgit-rs/public_symbol_export.c | 20 ++++++++++++++++
 contrib/cgit-rs/public_symbol_export.h |  8 +++++++
 contrib/cgit-rs/src/lib.rs             |  7 ++++++
 contrib/cgit-rs/src/main.rs            | 10 ++++++++
 10 files changed, 139 insertions(+)
 create mode 100644 contrib/cgit-rs/Cargo.lock
 create mode 100644 contrib/cgit-rs/Cargo.toml
 create mode 100644 contrib/cgit-rs/README.md
 create mode 100644 contrib/cgit-rs/build.rs
 create mode 100644 contrib/cgit-rs/public_symbol_export.c
 create mode 100644 contrib/cgit-rs/public_symbol_export.h
 create mode 100644 contrib/cgit-rs/src/lib.rs
 create mode 100644 contrib/cgit-rs/src/main.rs

diff --git a/.gitignore b/.gitignore
index 8caf3700c2..7031012330 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,3 +248,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/cgit-rs/target
diff --git a/Makefile b/Makefile
index 1cac51a4f7..fcd06af123 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2712,6 +2714,7 @@ OBJECTS += $(XDIFF_OBJS)
 OBJECTS += $(FUZZ_OBJS)
 OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
+OBJECTS += contrib/cgit-rs/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3719,6 +3722,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/cgit-rs/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3864,3 +3868,12 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
 build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/cgit-rs/partial_symbol_export.o: contrib/cgit-rs/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/cgit-rs/hidden_symbol_export.o: contrib/cgit-rs/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/cgit-rs/libcgit.a: contrib/cgit-rs/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/cgit-rs/Cargo.lock b/contrib/cgit-rs/Cargo.lock
new file mode 100644
index 0000000000..f50c593995
--- /dev/null
+++ b/contrib/cgit-rs/Cargo.lock
@@ -0,0 +1,16 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "cgit"
+version = "0.1.0"
+dependencies = [
+ "libc",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.155"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
new file mode 100644
index 0000000000..7b55e6f3e1
--- /dev/null
+++ b/contrib/cgit-rs/Cargo.toml
@@ -0,0 +1,16 @@
+[package]
+name = "cgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "git"
+
+[[bin]]
+name = "cgit-test"
+path = "src/main.rs"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libc = "0.2.155"
diff --git a/contrib/cgit-rs/README.md b/contrib/cgit-rs/README.md
new file mode 100644
index 0000000000..7a59602c30
--- /dev/null
+++ b/contrib/cgit-rs/README.md
@@ -0,0 +1,15 @@
+# cgit-info
+
+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
+library.
+
+## Building
+
+`cargo build` automatically builds and picks up on changes made to both
+the Rust wrapper and git.git code so there is no need to run `make`
+beforehand.
+
+## Running
+
+Assuming you don't make any changes to the Git source, you can just work from
+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
diff --git a/contrib/cgit-rs/build.rs b/contrib/cgit-rs/build.rs
new file mode 100644
index 0000000000..0c1ec06737
--- /dev/null
+++ b/contrib/cgit-rs/build.rs
@@ -0,0 +1,33 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = std::process::Command::new("make")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args(&[
+            "CC=clang",
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/cgit-rs/libcgit.a"
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+                "Make failed:\n  stdout = {}\n  stderr = {}\n",
+                String::from_utf8(make_output.stdout).unwrap(),
+                String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libcgit.a"), dst.join("libcgit.a"))?;
+    println!("cargo::rustc-link-search=native={}", dst.into_os_string().into_string().unwrap());
+    println!("cargo::rustc-link-lib=cgit");
+    println!("cargo::rustc-link-lib=z");
+    println!("cargo::rerun-if-changed={}", git_root.into_os_string().into_string().unwrap());
+
+    Ok(())
+}
diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
new file mode 100644
index 0000000000..3d1cd6cc4f
--- /dev/null
+++ b/contrib/cgit-rs/public_symbol_export.c
@@ -0,0 +1,20 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoid conflicts with other libraries such as libgit2.
+
+#include "contrib/cgit-rs/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/cgit-rs/public_symbol_export.h b/contrib/cgit-rs/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/cgit-rs/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
new file mode 100644
index 0000000000..dc46e7ff42
--- /dev/null
+++ b/contrib/cgit-rs/src/lib.rs
@@ -0,0 +1,7 @@
+use libc::c_char;
+
+extern "C" {
+    // From version.c
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
diff --git a/contrib/cgit-rs/src/main.rs b/contrib/cgit-rs/src/main.rs
new file mode 100644
index 0000000000..1794e3f43e
--- /dev/null
+++ b/contrib/cgit-rs/src/main.rs
@@ -0,0 +1,10 @@
+use std::ffi::CStr;
+
+fn main() {
+    println!("Let's print some strings provided by Git");
+    let c_buf = unsafe { cgit::libgit_user_agent() };
+    let c_str = unsafe { CStr::from_ptr(c_buf) };
+    println!("git_user_agent() = {:?}", c_str);
+    println!("git_user_agent_sanitized() = {:?}",
+        unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
+}
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2646 bytes --]

On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to call
> functions in libgit.a. This initial patch defines build rules and an
> interface that exposes user agent string getter functions as a proof of
> concept. A proof-of-concept library consumer is provided in
> contrib/cgit-rs/src/main.rs. This executable can be run with `cargo run`
> 
> Symbols in cgit can collide with symbols from other libraries such as
> libgit2. We avoid this by first exposing library symbols in
> public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
> avoid collisions and set to visible using a visibility pragma. In
> build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also contains
> libgit.a and other dependent libraries, with -fvisibility=hidden to hide
> all symbols within those libraries that haven't been exposed with a
> visibility pragma.

I think this is a good idea.  It's optional and it allows us to add
functionality as we go along.  Platforms that don't have Rust can just
omit building it.

> +[dependencies]
> +libc = "0.2.155"

I don't love that we're using libc here.  It would be better to use
rustix because that provides safe APIs that are compatible with POSIX,
but I think for now we need this because rustix doesn't offer memory
management like free(3).  I'd really prefer that we didn't have to do
memory management in Rust, but maybe that can come in with a future
series.

libc also comes with the downside that it calls the actual libc
functions, so you have to write things like stat64 on Linux if you want
a 64-bit stat, but that doesn't exist on some of the BSDs, so you have
to write something else and compile it conditionally, and all of that
makes the portability of it worse than with C.

In any event, I have the intention to send a patch to replace libc with
rustix in the future if this series lands.

> diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
> new file mode 100644
> index 0000000000..3d1cd6cc4f
> --- /dev/null
> +++ b/contrib/cgit-rs/public_symbol_export.c
> @@ -0,0 +1,20 @@
> +// Shim to publicly export Git symbols. These must be renamed so that the
> +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> +// avoid conflicts with other libraries such as libgit2.
> +
> +#include "contrib/cgit-rs/public_symbol_export.h"
> +#include "version.h"
> +
> +#pragma GCC visibility push(default)

I assume this also works in clang?
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

RE: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[rsbecker]

On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
>On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
>> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to call
>> functions in libgit.a. This initial patch defines build rules and an
>> interface that exposes user agent string getter functions as a proof
>> of concept. A proof-of-concept library consumer is provided in
>> contrib/cgit-rs/src/main.rs. This executable can be run with `cargo
>> run`
>>
>> Symbols in cgit can collide with symbols from other libraries such as
>> libgit2. We avoid this by first exposing library symbols in
>> public_symbol_export.[ch]. These symbols are prepended with "libgit_"
>> to avoid collisions and set to visible using a visibility pragma. In
>> build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also contains
>> libgit.a and other dependent libraries, with -fvisibility=hidden to
>> hide all symbols within those libraries that haven't been exposed with
>> a visibility pragma.
>
>I think this is a good idea.  It's optional and it allows us to add functionality as we go
>along.  Platforms that don't have Rust can just omit building it.
>
>> +[dependencies]
>> +libc = "0.2.155"
>
>I don't love that we're using libc here.  It would be better to use rustix because that
>provides safe APIs that are compatible with POSIX, but I think for now we need this
>because rustix doesn't offer memory management like free(3).  I'd really prefer that
>we didn't have to do memory management in Rust, but maybe that can come in
>with a future series.

This is a good point. Libc is not portable, but because I can't build with RUST anyway,
I hope that libc is restricted to this facility if used. It should not be included in the
git C build. It is probably moot for me anyway for this series, but I have to mention it
in case anyone else gets the idea to include it as a dependency for git C.

>libc also comes with the downside that it calls the actual libc functions, so you have
>to write things like stat64 on Linux if you want a 64-bit stat, but that doesn't exist
>on some of the BSDs, so you have to write something else and compile it
>conditionally, and all of that makes the portability of it worse than with C.
>
>In any event, I have the intention to send a patch to replace libc with rustix in the
>future if this series lands.
>
>> diff --git a/contrib/cgit-rs/public_symbol_export.c
>> b/contrib/cgit-rs/public_symbol_export.c
>> new file mode 100644
>> index 0000000000..3d1cd6cc4f
>> --- /dev/null
>> +++ b/contrib/cgit-rs/public_symbol_export.c
>> @@ -0,0 +1,20 @@
>> +// Shim to publicly export Git symbols. These must be renamed so that
>> +the // original symbols can be hidden. Renaming these with a
>> +"libgit_" prefix also // avoid conflicts with other libraries such as libgit2.
>> +
>> +#include "contrib/cgit-rs/public_symbol_export.h"
>> +#include "version.h"
>> +
>> +#pragma GCC visibility push(default)
>
>I assume this also works in clang?
>--
>brian m. carlson (they/them or he/him)
>Toronto, Ontario, CA

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.07 17:40, rsbecker@nexbridge.com wrote:
> On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
> >On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
> >> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to call
> >> functions in libgit.a. This initial patch defines build rules and an
> >> interface that exposes user agent string getter functions as a proof
> >> of concept. A proof-of-concept library consumer is provided in
> >> contrib/cgit-rs/src/main.rs. This executable can be run with `cargo
> >> run`
> >>
> >> Symbols in cgit can collide with symbols from other libraries such as
> >> libgit2. We avoid this by first exposing library symbols in
> >> public_symbol_export.[ch]. These symbols are prepended with "libgit_"
> >> to avoid collisions and set to visible using a visibility pragma. In
> >> build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also contains
> >> libgit.a and other dependent libraries, with -fvisibility=hidden to
> >> hide all symbols within those libraries that haven't been exposed with
> >> a visibility pragma.
> >
> >I think this is a good idea.  It's optional and it allows us to add functionality as we go
> >along.  Platforms that don't have Rust can just omit building it.
> >
> >> +[dependencies]
> >> +libc = "0.2.155"
> >
> >I don't love that we're using libc here.  It would be better to use rustix because that
> >provides safe APIs that are compatible with POSIX, but I think for now we need this
> >because rustix doesn't offer memory management like free(3).  I'd really prefer that
> >we didn't have to do memory management in Rust, but maybe that can come in
> >with a future series.
> 
> This is a good point. Libc is not portable, but because I can't build with RUST anyway,
> I hope that libc is restricted to this facility if used. It should not be included in the
> git C build. It is probably moot for me anyway for this series, but I have to mention it
> in case anyone else gets the idea to include it as a dependency for git C.

I know you don't have access to Rust, but would you be able to test the
symbol visibility steps with `make contrib/cgit-rs/libcgit.a`?

RE: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[rsbecker]

On Wednesday, August 7, 2024 7:08 PM, Josh Steadmon wrote:
>On 2024.08.07 17:40, rsbecker@nexbridge.com wrote:
>> On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
>> >On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
>> >> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to
>> >> call functions in libgit.a. This initial patch defines build rules
>> >> and an interface that exposes user agent string getter functions as
>> >> a proof of concept. A proof-of-concept library consumer is provided
>> >> in contrib/cgit-rs/src/main.rs. This executable can be run with
>> >> `cargo run`
>> >>
>> >> Symbols in cgit can collide with symbols from other libraries such
>> >> as libgit2. We avoid this by first exposing library symbols in
>> >> public_symbol_export.[ch]. These symbols are prepended with "libgit_"
>> >> to avoid collisions and set to visible using a visibility pragma.
>> >> In build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also
>> >> contains libgit.a and other dependent libraries, with
>> >> -fvisibility=hidden to hide all symbols within those libraries that
>> >> haven't been exposed with a visibility pragma.
>> >
>> >I think this is a good idea.  It's optional and it allows us to add
>> >functionality as we go along.  Platforms that don't have Rust can just
omit
>building it.
>> >
>> >> +[dependencies]
>> >> +libc = "0.2.155"
>> >
>> >I don't love that we're using libc here.  It would be better to use
>> >rustix because that provides safe APIs that are compatible with
>> >POSIX, but I think for now we need this because rustix doesn't offer
>> >memory management like free(3).  I'd really prefer that we didn't
>> >have to do memory management in Rust, but maybe that can come in with a
>future series.
>>
>> This is a good point. Libc is not portable, but because I can't build
>> with RUST anyway, I hope that libc is restricted to this facility if
>> used. It should not be included in the git C build. It is probably
>> moot for me anyway for this series, but I have to mention it in case
anyone else
>gets the idea to include it as a dependency for git C.
>
>I know you don't have access to Rust, but would you be able to test the
symbol
>visibility steps with `make contrib/cgit-rs/libcgit.a`?

Of course. Branch? URI?

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.07 19:51, rsbecker@nexbridge.com wrote:
> On Wednesday, August 7, 2024 7:08 PM, Josh Steadmon wrote:
> >On 2024.08.07 17:40, rsbecker@nexbridge.com wrote:
> >> On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
> >> >On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
> >> >> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to
> >> >> call functions in libgit.a. This initial patch defines build rules
> >> >> and an interface that exposes user agent string getter functions as
> >> >> a proof of concept. A proof-of-concept library consumer is provided
> >> >> in contrib/cgit-rs/src/main.rs. This executable can be run with
> >> >> `cargo run`
> >> >>
> >> >> Symbols in cgit can collide with symbols from other libraries such
> >> >> as libgit2. We avoid this by first exposing library symbols in
> >> >> public_symbol_export.[ch]. These symbols are prepended with "libgit_"
> >> >> to avoid collisions and set to visible using a visibility pragma.
> >> >> In build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also
> >> >> contains libgit.a and other dependent libraries, with
> >> >> -fvisibility=hidden to hide all symbols within those libraries that
> >> >> haven't been exposed with a visibility pragma.
> >> >
> >> >I think this is a good idea.  It's optional and it allows us to add
> >> >functionality as we go along.  Platforms that don't have Rust can just
> omit
> >building it.
> >> >
> >> >> +[dependencies]
> >> >> +libc = "0.2.155"
> >> >
> >> >I don't love that we're using libc here.  It would be better to use
> >> >rustix because that provides safe APIs that are compatible with
> >> >POSIX, but I think for now we need this because rustix doesn't offer
> >> >memory management like free(3).  I'd really prefer that we didn't
> >> >have to do memory management in Rust, but maybe that can come in with a
> >future series.
> >>
> >> This is a good point. Libc is not portable, but because I can't build
> >> with RUST anyway, I hope that libc is restricted to this facility if
> >> used. It should not be included in the git C build. It is probably
> >> moot for me anyway for this series, but I have to mention it in case
> anyone else
> >gets the idea to include it as a dependency for git C.
> >
> >I know you don't have access to Rust, but would you be able to test the
> symbol
> >visibility steps with `make contrib/cgit-rs/libcgit.a`?
> 
> Of course. Branch? URI?

https://github.com/steadmon/git/tree/cgit-dev

RE: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[rsbecker]

On Thursday, August 8, 2024 1:13 PM, Josh Steadmon wrote:
>On 2024.08.07 19:51, rsbecker@nexbridge.com wrote:
>> On Wednesday, August 7, 2024 7:08 PM, Josh Steadmon wrote:
>> >On 2024.08.07 17:40, rsbecker@nexbridge.com wrote:
>> >> On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
>> >> >On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
>> >> >> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to
>> >> >> call functions in libgit.a. This initial patch defines build
>> >> >> rules and an interface that exposes user agent string getter
>> >> >> functions as a proof of concept. A proof-of-concept library
>> >> >> consumer is provided in contrib/cgit-rs/src/main.rs. This
>> >> >> executable can be run with `cargo run`
>> >> >>
>> >> >> Symbols in cgit can collide with symbols from other libraries
>> >> >> such as libgit2. We avoid this by first exposing library symbols
>> >> >> in public_symbol_export.[ch]. These symbols are prepended with
"libgit_"
>> >> >> to avoid collisions and set to visible using a visibility pragma.
>> >> >> In build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also
>> >> >> contains libgit.a and other dependent libraries, with
>> >> >> -fvisibility=hidden to hide all symbols within those libraries
>> >> >> that haven't been exposed with a visibility pragma.
>> >> >
>> >> >I think this is a good idea.  It's optional and it allows us to
>> >> >add functionality as we go along.  Platforms that don't have Rust
>> >> >can just
>> omit
>> >building it.
>> >> >
>> >> >> +[dependencies]
>> >> >> +libc = "0.2.155"
>> >> >
>> >> >I don't love that we're using libc here.  It would be better to
>> >> >use rustix because that provides safe APIs that are compatible
>> >> >with POSIX, but I think for now we need this because rustix
>> >> >doesn't offer memory management like free(3).  I'd really prefer
>> >> >that we didn't have to do memory management in Rust, but maybe
>> >> >that can come in with a
>> >future series.
>> >>
>> >> This is a good point. Libc is not portable, but because I can't
>> >> build with RUST anyway, I hope that libc is restricted to this
>> >> facility if used. It should not be included in the git C build. It
>> >> is probably moot for me anyway for this series, but I have to
>> >> mention it in case
>> anyone else
>> >gets the idea to include it as a dependency for git C.
>> >
>> >I know you don't have access to Rust, but would you be able to test
>> >the
>> symbol
>> >visibility steps with `make contrib/cgit-rs/libcgit.a`?
>>
>> Of course. Branch? URI?
>
>https://github.com/steadmon/git/tree/cgit-dev

I got to:

ld -r contrib/cgit-rs/public_symbol_export.o libgit.a reftable/libreftable.a
xdiff/lib.a -o contrib/cgit-rs/partial_symbol_export.o
/usr/bin/c89: illegal option -- r

The -r option is not supported on NonStop. I think we had discussed this.

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Junio C Hamano]

<rsbecker@nexbridge.com> writes:

> I got to:
>
> ld -r contrib/cgit-rs/public_symbol_export.o libgit.a reftable/libreftable.a
> xdiff/lib.a -o contrib/cgit-rs/partial_symbol_export.o
> /usr/bin/c89: illegal option -- r
>
> The -r option is not supported on NonStop. I think we had discussed this.

Does it happen to call the feature under different name?  'r' stands
for relocatable output, but some linkers may call the feature as
incremental linking and use '-i'.

RE: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[rsbecker]

On Thursday, August 8, 2024 3:58 PM, Junio C Hamano wrote:
><rsbecker@nexbridge.com> writes:
>
>> I got to:
>>
>> ld -r contrib/cgit-rs/public_symbol_export.o libgit.a
>> reftable/libreftable.a xdiff/lib.a -o
>> contrib/cgit-rs/partial_symbol_export.o
>> /usr/bin/c89: illegal option -- r
>>
>> The -r option is not supported on NonStop. I think we had discussed this.
>
>Does it happen to call the feature under different name?  'r' stands for
relocatable
>output, but some linkers may call the feature as incremental linking and
use '-i'.

Well... depends on some factors. ld should not be run directly. Other links
are done
via the CC setting in config.mak.uname. That would invoke c99 for us for the
linker
as with the rest of git C, and that would make thinks somewhat easier as the
linker
is either xld or eld depending on the hardware.

The -Wr option to c99 creates a load file in this situation, which might or
might not
work. I need to look at it after built. I don't think it is what we want.
What I think
we need is -Wxld=unres_symbols ignore which builds an object that can be
linked to other objects even with undefined link symbols - although maybe
not.
I need to be able to select a different option on ia64, but that may not
matter
much. I still worry that we are depending on non-portable linker semantics
that
may not give the result we ultimately want.

Would not the build option be best selected in config.mak.uname?

Maybe GITRS_LINKER= or GITRS_CFLAGS=, etc.

RE: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[rsbecker]

On Wednesday, August 7, 2024 7:08 PM, Josh Steadmon wrote:
>On 2024.08.07 17:40, rsbecker@nexbridge.com wrote:
>> On Wednesday, August 7, 2024 5:21 PM, brian m. carlson wrote:
>> >On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
>> >> Introduce cgit-rs, a Rust wrapper crate that allows Rust code to
>> >> call functions in libgit.a. This initial patch defines build rules
>> >> and an interface that exposes user agent string getter functions as
>> >> a proof of concept. A proof-of-concept library consumer is provided
>> >> in contrib/cgit-rs/src/main.rs. This executable can be run with
>> >> `cargo run`
>> >>
>> >> Symbols in cgit can collide with symbols from other libraries such
>> >> as libgit2. We avoid this by first exposing library symbols in
>> >> public_symbol_export.[ch]. These symbols are prepended with "libgit_"
>> >> to avoid collisions and set to visible using a visibility pragma.
>> >> In build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also
>> >> contains libgit.a and other dependent libraries, with
>> >> -fvisibility=hidden to hide all symbols within those libraries that
>> >> haven't been exposed with a visibility pragma.
>> >
>> >I think this is a good idea.  It's optional and it allows us to add
>> >functionality as we go along.  Platforms that don't have Rust can just
omit
>building it.
>> >
>> >> +[dependencies]
>> >> +libc = "0.2.155"
>> >
>> >I don't love that we're using libc here.  It would be better to use
>> >rustix because that provides safe APIs that are compatible with
>> >POSIX, but I think for now we need this because rustix doesn't offer
>> >memory management like free(3).  I'd really prefer that we didn't
>> >have to do memory management in Rust, but maybe that can come in with a
>future series.
>>
>> This is a good point. Libc is not portable, but because I can't build
>> with RUST anyway, I hope that libc is restricted to this facility if
>> used. It should not be included in the git C build. It is probably
>> moot for me anyway for this series, but I have to mention it in case
anyone else
>gets the idea to include it as a dependency for git C.
>
>I know you don't have access to Rust, but would you be able to test the
symbol
>visibility steps with `make contrib/cgit-rs/libcgit.a`?

This target is no longer valid. Is there another target I can try?

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.07 21:21, brian m. carlson wrote:
> On 2024-08-07 at 18:21:28, Josh Steadmon wrote:
> > Introduce cgit-rs, a Rust wrapper crate that allows Rust code to call
> > functions in libgit.a. This initial patch defines build rules and an
> > interface that exposes user agent string getter functions as a proof of
> > concept. A proof-of-concept library consumer is provided in
> > contrib/cgit-rs/src/main.rs. This executable can be run with `cargo run`
> > 
> > Symbols in cgit can collide with symbols from other libraries such as
> > libgit2. We avoid this by first exposing library symbols in
> > public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
> > avoid collisions and set to visible using a visibility pragma. In
> > build.rs, Rust builds contrib/cgit-rs/libcgit.a, which also contains
> > libgit.a and other dependent libraries, with -fvisibility=hidden to hide
> > all symbols within those libraries that haven't been exposed with a
> > visibility pragma.
> 
> I think this is a good idea.  It's optional and it allows us to add
> functionality as we go along.  Platforms that don't have Rust can just
> omit building it.
> 
> > +[dependencies]
> > +libc = "0.2.155"
> 
> I don't love that we're using libc here.  It would be better to use
> rustix because that provides safe APIs that are compatible with POSIX,
> but I think for now we need this because rustix doesn't offer memory
> management like free(3).  I'd really prefer that we didn't have to do
> memory management in Rust, but maybe that can come in with a future
> series.

Yeah, needing to free() is the only thing we striclty need from libc
right now. Please correct me if I'm wrong, but IIUC then any memory that
is allocated on the C side and then passed to Rust needs one of:
1) freed by libc::free() on the Rust side,
2) passed back to the C side to be freed there, or
3) leaked

Am I correct in assuming that your opinion is that writing additional
*_free() functions on the C side is worth it to avoid libc? If so, then
I'm fine with including that in V2.

> libc also comes with the downside that it calls the actual libc
> functions, so you have to write things like stat64 on Linux if you want
> a 64-bit stat, but that doesn't exist on some of the BSDs, so you have
> to write something else and compile it conditionally, and all of that
> makes the portability of it worse than with C.
> 
> In any event, I have the intention to send a patch to replace libc with
> rustix in the future if this series lands.

Even if we can't fully eliminate libc in V2, I'll keep this in mind and
avoid adding additional usage of libc as much as possible.


> > diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
> > new file mode 100644
> > index 0000000000..3d1cd6cc4f
> > --- /dev/null
> > +++ b/contrib/cgit-rs/public_symbol_export.c
> > @@ -0,0 +1,20 @@
> > +// Shim to publicly export Git symbols. These must be renamed so that the
> > +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> > +// avoid conflicts with other libraries such as libgit2.
> > +
> > +#include "contrib/cgit-rs/public_symbol_export.h"
> > +#include "version.h"
> > +
> > +#pragma GCC visibility push(default)
> 
> I assume this also works in clang?

Yep.


> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

On 2024-08-07 at 23:05:00, Josh Steadmon wrote:
> Yeah, needing to free() is the only thing we striclty need from libc
> right now. Please correct me if I'm wrong, but IIUC then any memory that
> is allocated on the C side and then passed to Rust needs one of:
> 1) freed by libc::free() on the Rust side,
> 2) passed back to the C side to be freed there, or
> 3) leaked
> 
> Am I correct in assuming that your opinion is that writing additional
> *_free() functions on the C side is worth it to avoid libc? If so, then
> I'm fine with including that in V2.

I think if we're going to be writing a general purpose API for
libification, we probably should provide free functions.  Normally, that
will be a call to free(3), but in some cases we may need more complex
logic, and by providing those, we're making the API more consistent and
easy to use.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> I think if we're going to be writing a general purpose API for
> libification, we probably should provide free functions.  Normally, that
> will be a call to free(3), but in some cases we may need more complex
> logic, and by providing those, we're making the API more consistent and
> easy to use.

Do you mean that we should have variants of free() that are specific
to each data structure?  E.g., Patrick taught fetch_task_release()
to release the task structure itself, in addition to the resources
it holds, while renaming fetch_task_release() to fetch_task_free(),
with ff25992c (submodule: fix leaking fetch tasks, 2024-08-07), so
if cgit-sys wants to expose fetch_task object to the outside world,
the consumers would call fetch_task_free() on it?

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.07 19:21, Junio C Hamano wrote:
> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> 
> > I think if we're going to be writing a general purpose API for
> > libification, we probably should provide free functions.  Normally, that
> > will be a call to free(3), but in some cases we may need more complex
> > logic, and by providing those, we're making the API more consistent and
> > easy to use.
> 
> Do you mean that we should have variants of free() that are specific
> to each data structure?  E.g., Patrick taught fetch_task_release()
> to release the task structure itself, in addition to the resources
> it holds, while renaming fetch_task_release() to fetch_task_free(),
> with ff25992c (submodule: fix leaking fetch tasks, 2024-08-07), so
> if cgit-sys wants to expose fetch_task object to the outside world,
> the consumers would call fetch_task_free() on it?

Yes, although hopefully the higher-level API we build on top of cgit-sys
will be able to hide this from the Rust consumers.

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.07 23:55, brian m. carlson wrote:
> On 2024-08-07 at 23:05:00, Josh Steadmon wrote:
> > Yeah, needing to free() is the only thing we striclty need from libc
> > right now. Please correct me if I'm wrong, but IIUC then any memory that
> > is allocated on the C side and then passed to Rust needs one of:
> > 1) freed by libc::free() on the Rust side,
> > 2) passed back to the C side to be freed there, or
> > 3) leaked
> > 
> > Am I correct in assuming that your opinion is that writing additional
> > *_free() functions on the C side is worth it to avoid libc? If so, then
> > I'm fine with including that in V2.
> 
> I think if we're going to be writing a general purpose API for
> libification, we probably should provide free functions.  Normally, that
> will be a call to free(3)

[snip]

So in this case, does that mean we'd replace our call to `libc::free()`
with just `free()`, and then add a declaration for `free` in our
`extern "C"` section of cgit-sys? It seems to work on my machine, but is
that actually the more portable option compared to using libc::free? Or
have I misunderstood something?

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Kyle Lippincott]

On Thu, Aug 8, 2024 at 11:22 AM Josh Steadmon <steadmon@google.com> wrote:
>
> On 2024.08.07 23:55, brian m. carlson wrote:
> > On 2024-08-07 at 23:05:00, Josh Steadmon wrote:
> > > Yeah, needing to free() is the only thing we striclty need from libc
> > > right now. Please correct me if I'm wrong, but IIUC then any memory that
> > > is allocated on the C side and then passed to Rust needs one of:
> > > 1) freed by libc::free() on the Rust side,
> > > 2) passed back to the C side to be freed there, or
> > > 3) leaked
> > >
> > > Am I correct in assuming that your opinion is that writing additional
> > > *_free() functions on the C side is worth it to avoid libc? If so, then
> > > I'm fine with including that in V2.
> >
> > I think if we're going to be writing a general purpose API for
> > libification, we probably should provide free functions.  Normally, that
> > will be a call to free(3)
>
> [snip]
>
> So in this case, does that mean we'd replace our call to `libc::free()`
> with just `free()`, and then add a declaration for `free` in our
> `extern "C"` section of cgit-sys? It seems to work on my machine, but is
> that actually the more portable option compared to using libc::free? Or
> have I misunderstood something?

I think both having a generic 'free' function, or requiring your API
consumer to have a compatible 'free' function is undesirable. If the
API hands you something that you must return/free, there should be a
function for that specifically. So I would expect if the API has a
`libgit_foo_get(foo** f)` function, there'd be a paired
`libgit_foo_release(foo* f)` (ignoring whatever squabbles we want to
have about the names). Requiring `libgit_foo_get(foo** f)` to be
paired with `libc::free(f)` limits us to always using libc malloc;
pairing it with `libgit_free((void*)f)` means we can't refcount it /
ignore it if it's a part of a parent object, etc.

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.08 13:18, Kyle Lippincott wrote:
> On Thu, Aug 8, 2024 at 11:22 AM Josh Steadmon <steadmon@google.com> wrote:
> >
> > On 2024.08.07 23:55, brian m. carlson wrote:
> > > On 2024-08-07 at 23:05:00, Josh Steadmon wrote:
> > > > Yeah, needing to free() is the only thing we striclty need from libc
> > > > right now. Please correct me if I'm wrong, but IIUC then any memory that
> > > > is allocated on the C side and then passed to Rust needs one of:
> > > > 1) freed by libc::free() on the Rust side,
> > > > 2) passed back to the C side to be freed there, or
> > > > 3) leaked
> > > >
> > > > Am I correct in assuming that your opinion is that writing additional
> > > > *_free() functions on the C side is worth it to avoid libc? If so, then
> > > > I'm fine with including that in V2.
> > >
> > > I think if we're going to be writing a general purpose API for
> > > libification, we probably should provide free functions.  Normally, that
> > > will be a call to free(3)
> >
> > [snip]
> >
> > So in this case, does that mean we'd replace our call to `libc::free()`
> > with just `free()`, and then add a declaration for `free` in our
> > `extern "C"` section of cgit-sys? It seems to work on my machine, but is
> > that actually the more portable option compared to using libc::free? Or
> > have I misunderstood something?
> 
> I think both having a generic 'free' function, or requiring your API
> consumer to have a compatible 'free' function is undesirable. If the
> API hands you something that you must return/free, there should be a
> function for that specifically. So I would expect if the API has a
> `libgit_foo_get(foo** f)` function, there'd be a paired
> `libgit_foo_release(foo* f)` (ignoring whatever squabbles we want to
> have about the names). Requiring `libgit_foo_get(foo** f)` to be
> paired with `libc::free(f)` limits us to always using libc malloc;
> pairing it with `libgit_free((void*)f)` means we can't refcount it /
> ignore it if it's a part of a parent object, etc.

Sorry, the diff context got removed, the the particular case I was
talking about here is where we get back a strdup()ed string from some of
the config API calls. I guess we could add a c_str_free() somewhere on
the Git side and call that, but it seems like a bit of overkill.

I do agree with you about providing _free() [or in our case,
_clear_and_free()] functions for more complicated data types though.

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Mike Hommey]

On Wed, Aug 07, 2024 at 11:21:28AM -0700, Josh Steadmon wrote:
> +contrib/cgit-rs/hidden_symbol_export.o: contrib/cgit-rs/partial_symbol_export.o
> +	$(OBJCOPY) --localize-hidden $^ $@

This is ELF-specific and won't work on Mac or Windows.

> +    let make_output = std::process::Command::new("make")
> +        .env_remove("PROFILE")
> +        .current_dir(git_root.clone())
> +        .args(&[
> +            "CC=clang",

You should probably not set CC at all here.

> +            "CFLAGS=-fvisibility=hidden",

This won't work for Windows targets.

> +            "contrib/cgit-rs/libcgit.a"
> +        ])
> +        .output()
> +        .expect("Make failed to run");
> +    if !make_output.status.success() {
> +        panic!(
> +                "Make failed:\n  stdout = {}\n  stderr = {}\n",
> +                String::from_utf8(make_output.stdout).unwrap(),
> +                String::from_utf8(make_output.stderr).unwrap()
> +        );
> +    }
> +    std::fs::copy(crate_root.join("libcgit.a"), dst.join("libcgit.a"))?;
> +    println!("cargo::rustc-link-search=native={}", dst.into_os_string().into_string().unwrap());

You might as well use `dst.display()`.

> +    println!("cargo::rustc-link-lib=cgit");
> +    println!("cargo::rustc-link-lib=z");
> +    println!("cargo::rerun-if-changed={}", git_root.into_os_string().into_string().unwrap());

Likewise.

Mike

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.08 07:47, Mike Hommey wrote:
> On Wed, Aug 07, 2024 at 11:21:28AM -0700, Josh Steadmon wrote:
> > +contrib/cgit-rs/hidden_symbol_export.o: contrib/cgit-rs/partial_symbol_export.o
> > +	$(OBJCOPY) --localize-hidden $^ $@
> 
> This is ELF-specific and won't work on Mac or Windows.
> 
> > +    let make_output = std::process::Command::new("make")
> > +        .env_remove("PROFILE")
> > +        .current_dir(git_root.clone())
> > +        .args(&[
> > +            "CC=clang",
> 
> You should probably not set CC at all here.

Ack, fixed in V2.

> > +            "CFLAGS=-fvisibility=hidden",
> 
> This won't work for Windows targets.

Understood. I'll have to do some studying on the symbol visibility
options.


> > +            "contrib/cgit-rs/libcgit.a"
> > +        ])
> > +        .output()
> > +        .expect("Make failed to run");
> > +    if !make_output.status.success() {
> > +        panic!(
> > +                "Make failed:\n  stdout = {}\n  stderr = {}\n",
> > +                String::from_utf8(make_output.stdout).unwrap(),
> > +                String::from_utf8(make_output.stderr).unwrap()
> > +        );
> > +    }
> > +    std::fs::copy(crate_root.join("libcgit.a"), dst.join("libcgit.a"))?;
> > +    println!("cargo::rustc-link-search=native={}", dst.into_os_string().into_string().unwrap());
> 
> You might as well use `dst.display()`.

Wouldn't that fail silently in the event that the path is non-UTF-8? I
think I'd prefer to explicitly fail in that case, even if it seems
unlikely.

> > +    println!("cargo::rustc-link-lib=cgit");
> > +    println!("cargo::rustc-link-lib=z");
> > +    println!("cargo::rerun-if-changed={}", git_root.into_os_string().into_string().unwrap());
> 
> Likewise.
> 
> Mike

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Mike Hommey]

On Wed, Aug 07, 2024 at 04:29:54PM -0700, Josh Steadmon wrote:
> > You might as well use `dst.display()`.
> 
> Wouldn't that fail silently in the event that the path is non-UTF-8? I
> think I'd prefer to explicitly fail in that case, even if it seems
> unlikely.

That's the theory, unfortunately, reality is that even the most central
Rust crates don't care:
https://github.com/rust-lang/cc-rs/blob/main/src/lib.rs#L1357-L1360

Even better, last time I tried, cargo or rustc (I don't remember which
one it was) would blatantly fail to work if the path is not UTF-8 in the
first place.

Mike

Re: [RFC PATCH 3/6] contrib/cgit-rs: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.08.08 09:17, Mike Hommey wrote:
> On Wed, Aug 07, 2024 at 04:29:54PM -0700, Josh Steadmon wrote:
> > > You might as well use `dst.display()`.
> > 
> > Wouldn't that fail silently in the event that the path is non-UTF-8? I
> > think I'd prefer to explicitly fail in that case, even if it seems
> > unlikely.
> 
> That's the theory, unfortunately, reality is that even the most central
> Rust crates don't care:
> https://github.com/rust-lang/cc-rs/blob/main/src/lib.rs#L1357-L1360
> 
> Even better, last time I tried, cargo or rustc (I don't remember which
> one it was) would blatantly fail to work if the path is not UTF-8 in the
> first place.
> 
> Mike

Thanks for the pointers. Fixed in V2.

[RFC PATCH 4/6] contrib/cgit-rs: add repo initialization and config access

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/cgit-rs/public_symbol_export.c | 27 ++++++++++++++++++++++++++
 contrib/cgit-rs/public_symbol_export.h |  8 ++++++++
 contrib/cgit-rs/src/lib.rs             | 13 ++++++++++++-
 contrib/cgit-rs/src/main.rs            | 23 +++++++++++++++++++++-
 4 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
index 3d1cd6cc4f..ab3401ac40 100644
--- a/contrib/cgit-rs/public_symbol_export.c
+++ b/contrib/cgit-rs/public_symbol_export.c
@@ -2,11 +2,35 @@
 // original symbols can be hidden. Renaming these with a "libgit_" prefix also
 // avoid conflicts with other libraries such as libgit2.
 
+#include "git-compat-util.h"
 #include "contrib/cgit-rs/public_symbol_export.h"
+#include "attr.h"
+#include "config.h"
+#include "setup.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+const char *libgit_setup_git_directory(void)
+{
+	return setup_git_directory();
+}
+
+int libgit_config_get_int(const char *key, int *dest)
+{
+	return git_config_get_int(key, dest);
+}
+
+void libgit_initialize_the_repository(void)
+{
+	initialize_the_repository();
+}
+
+int libgit_parse_maybe_bool(const char *val)
+{
+	return git_parse_maybe_bool(val);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
@@ -17,4 +41,7 @@ const char *libgit_user_agent_sanitized(void)
 	return git_user_agent_sanitized();
 }
 
+const char *libgit_attr__true = git_attr__true;
+const char *libgit_attr__false = git_attr__false;
+
 #pragma GCC visibility pop
diff --git a/contrib/cgit-rs/public_symbol_export.h b/contrib/cgit-rs/public_symbol_export.h
index a3372f93fa..97e8e871ba 100644
--- a/contrib/cgit-rs/public_symbol_export.h
+++ b/contrib/cgit-rs/public_symbol_export.h
@@ -1,6 +1,14 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+const char *libgit_setup_git_directory(void);
+
+int libgit_config_get_int(const char *key, int *dest);
+
+void libgit_initialize_the_repository(void);
+
+int libgit_parse_maybe_bool(const char *val);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
index dc46e7ff42..df350e758f 100644
--- a/contrib/cgit-rs/src/lib.rs
+++ b/contrib/cgit-rs/src/lib.rs
@@ -1,6 +1,17 @@
-use libc::c_char;
+use libc::{c_char, c_int};
 
 extern "C" {
+    pub fn libgit_setup_git_directory() -> *const c_char;
+
+    // From config.c
+    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) ->c_int;
+
+    // From repository.c
+    pub fn libgit_initialize_the_repository();
+
+    // From parse.c
+    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
+
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
diff --git a/contrib/cgit-rs/src/main.rs b/contrib/cgit-rs/src/main.rs
index 1794e3f43e..c5f8644fca 100644
--- a/contrib/cgit-rs/src/main.rs
+++ b/contrib/cgit-rs/src/main.rs
@@ -1,4 +1,4 @@
-use std::ffi::CStr;
+use std::ffi::{CStr, CString};
 
 fn main() {
     println!("Let's print some strings provided by Git");
@@ -7,4 +7,25 @@ fn main() {
     println!("git_user_agent() = {:?}", c_str);
     println!("git_user_agent_sanitized() = {:?}",
         unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
+
+    println!("\nNow try passing args");
+    let test_arg = CString::new("test_arg").unwrap();
+    println!("git_parse_maybe_bool(...) = {:?}",
+        unsafe { cgit::libgit_parse_maybe_bool(test_arg.as_ptr()) });
+
+    println!("\nCan we get an int out of our config??");
+    unsafe {
+        cgit::libgit_initialize_the_repository();
+        cgit::libgit_setup_git_directory();
+        let mut val: libc::c_int = 0;
+        let key = CString::new("trace2.eventNesting").unwrap();
+        cgit::libgit_config_get_int(
+            key.as_ptr(),
+            &mut val as *mut i32
+        );
+        println!(
+            "git_config_get_int(\"trace2.eventNesting\") -> {:?}",
+            val
+        );
+    };
 }
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [RFC PATCH 4/6] contrib/cgit-rs: add repo initialization and config access

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2876 bytes --]

On 2024-08-07 at 18:21:29, Josh Steadmon wrote:
> diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
> index dc46e7ff42..df350e758f 100644
> --- a/contrib/cgit-rs/src/lib.rs
> +++ b/contrib/cgit-rs/src/lib.rs
> @@ -1,6 +1,17 @@
> -use libc::c_char;
> +use libc::{c_char, c_int};
>  
>  extern "C" {
> +    pub fn libgit_setup_git_directory() -> *const c_char;
> +
> +    // From config.c
> +    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) ->c_int;
> +
> +    // From repository.c
> +    pub fn libgit_initialize_the_repository();
> +
> +    // From parse.c
> +    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
> +
>      // From version.c
>      pub fn libgit_user_agent() -> *const c_char;
>      pub fn libgit_user_agent_sanitized() -> *const c_char;

I think it would be helpful if we didn't expose the raw C API in Rust.
Nobody is going to want to write code that uses that in Rust.

If we _do_ expose that, it should be in a separate cgit-sys crate, which
is the customary naming, that exposes only that and then we can offer
nicer wrappers around it.

> diff --git a/contrib/cgit-rs/src/main.rs b/contrib/cgit-rs/src/main.rs
> index 1794e3f43e..c5f8644fca 100644
> --- a/contrib/cgit-rs/src/main.rs
> +++ b/contrib/cgit-rs/src/main.rs
> @@ -1,4 +1,4 @@
> -use std::ffi::CStr;
> +use std::ffi::{CStr, CString};
>  
>  fn main() {
>      println!("Let's print some strings provided by Git");
> @@ -7,4 +7,25 @@ fn main() {
>      println!("git_user_agent() = {:?}", c_str);
>      println!("git_user_agent_sanitized() = {:?}",
>          unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
> +
> +    println!("\nNow try passing args");
> +    let test_arg = CString::new("test_arg").unwrap();
> +    println!("git_parse_maybe_bool(...) = {:?}",
> +        unsafe { cgit::libgit_parse_maybe_bool(test_arg.as_ptr()) });
> +
> +    println!("\nCan we get an int out of our config??");
> +    unsafe {
> +        cgit::libgit_initialize_the_repository();
> +        cgit::libgit_setup_git_directory();
> +        let mut val: libc::c_int = 0;
> +        let key = CString::new("trace2.eventNesting").unwrap();
> +        cgit::libgit_config_get_int(
> +            key.as_ptr(),
> +            &mut val as *mut i32
> +        );
> +        println!(
> +            "git_config_get_int(\"trace2.eventNesting\") -> {:?}",
> +            val
> +        );
> +    };
>  }

It seems like this code wants to be a set of tests and maybe it would
be helpful to write it as a few instead.  For example, we can assume
that our user-agent starts with `git/` assuming it hasn't been
overridden, so maybe we could write that as a test, or at least that we
got a valid C string.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 4/6] contrib/cgit-rs: add repo initialization and config access

[Josh Steadmon]

On 2024.08.07 21:26, brian m. carlson wrote:
> On 2024-08-07 at 18:21:29, Josh Steadmon wrote:
> > diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
> > index dc46e7ff42..df350e758f 100644
> > --- a/contrib/cgit-rs/src/lib.rs
> > +++ b/contrib/cgit-rs/src/lib.rs
> > @@ -1,6 +1,17 @@
> > -use libc::c_char;
> > +use libc::{c_char, c_int};
> >  
> >  extern "C" {
> > +    pub fn libgit_setup_git_directory() -> *const c_char;
> > +
> > +    // From config.c
> > +    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) ->c_int;
> > +
> > +    // From repository.c
> > +    pub fn libgit_initialize_the_repository();
> > +
> > +    // From parse.c
> > +    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
> > +
> >      // From version.c
> >      pub fn libgit_user_agent() -> *const c_char;
> >      pub fn libgit_user_agent_sanitized() -> *const c_char;
> 
> I think it would be helpful if we didn't expose the raw C API in Rust.
> Nobody is going to want to write code that uses that in Rust.
> 
> If we _do_ expose that, it should be in a separate cgit-sys crate, which
> is the customary naming, that exposes only that and then we can offer
> nicer wrappers around it.

Yeah, that's a known NEEDSWORK that I forgot to mention in the cover
letter. I'm also going to get in touch soon with the gitoxide &
libgit2-rs folks to see if there's any joint work that we can do in
terms of defining nicer Rust APIs.

Semi-relatedly, I was wondering if you might be able to answer a cargo /
crates.io question: for a crate such as cgit-rs, which is not located in
the root of its VCS repo, will cargo balk at downloading the full
git.git worktree? Our build.rs expects the full Git source to be
available at "../.." relative to the crate root. We've currently only
tested consuming cgit-rs in JJ via a local path rather than through
crates.io.

libgit2-rs avoids this issue by including the C source of libgit2 as a
submodule, but I'd prefer for cgit-rs to live in the git.git repo if at
all possible.

> > diff --git a/contrib/cgit-rs/src/main.rs b/contrib/cgit-rs/src/main.rs
> > index 1794e3f43e..c5f8644fca 100644
> > --- a/contrib/cgit-rs/src/main.rs
> > +++ b/contrib/cgit-rs/src/main.rs
> > @@ -1,4 +1,4 @@
> > -use std::ffi::CStr;
> > +use std::ffi::{CStr, CString};
> >  
> >  fn main() {
> >      println!("Let's print some strings provided by Git");
> > @@ -7,4 +7,25 @@ fn main() {
> >      println!("git_user_agent() = {:?}", c_str);
> >      println!("git_user_agent_sanitized() = {:?}",
> >          unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
> > +
> > +    println!("\nNow try passing args");
> > +    let test_arg = CString::new("test_arg").unwrap();
> > +    println!("git_parse_maybe_bool(...) = {:?}",
> > +        unsafe { cgit::libgit_parse_maybe_bool(test_arg.as_ptr()) });
> > +
> > +    println!("\nCan we get an int out of our config??");
> > +    unsafe {
> > +        cgit::libgit_initialize_the_repository();
> > +        cgit::libgit_setup_git_directory();
> > +        let mut val: libc::c_int = 0;
> > +        let key = CString::new("trace2.eventNesting").unwrap();
> > +        cgit::libgit_config_get_int(
> > +            key.as_ptr(),
> > +            &mut val as *mut i32
> > +        );
> > +        println!(
> > +            "git_config_get_int(\"trace2.eventNesting\") -> {:?}",
> > +            val
> > +        );
> > +    };
> >  }
> 
> It seems like this code wants to be a set of tests and maybe it would
> be helpful to write it as a few instead.  For example, we can assume
> that our user-agent starts with `git/` assuming it hasn't been
> overridden, so maybe we could write that as a test, or at least that we
> got a valid C string.

Agreed.

> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

Re: [RFC PATCH 4/6] contrib/cgit-rs: add repo initialization and config access

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

On 2024-08-07 at 23:14:43, Josh Steadmon wrote:
> Semi-relatedly, I was wondering if you might be able to answer a cargo /
> crates.io question: for a crate such as cgit-rs, which is not located in
> the root of its VCS repo, will cargo balk at downloading the full
> git.git worktree? Our build.rs expects the full Git source to be
> available at "../.." relative to the crate root. We've currently only
> tested consuming cgit-rs in JJ via a local path rather than through
> crates.io.

The documentation[0] says this:

  Cargo fetches the git repository at that location and traverses the
  file tree to find Cargo.toml file for the requested crate anywhere
  inside the git repository. For example, regex-lite and regex-syntax
  are members of rust-lang/regex repo and can be referred to by the
  repo’s root URL (https://github.com/rust-lang/regex.git) regardless of
  where in the file tree they reside.

If you want to test, you should be able to push it to a repository
somewhere, even if that's a local path, and specify like so to test:

----
[dependencies]
cgit-rs = { version = "0.1.0", git = "file:///path/to/git.git", branch = "cgit-rs" }
----

and see if it finds it.

[0] https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

[RFC PATCH 5/6] config: add git_configset_alloc

[Josh Steadmon]

Add git_configset_alloc so that non-C external consumers can use
configset functions without redefining config_set.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 config.c | 5 +++++
 config.h | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/config.c b/config.c
index 6421894614..fcfe7fcf46 100644
--- a/config.c
+++ b/config.c
@@ -2324,6 +2324,11 @@ static int config_set_element_cmp(const void *cmp_data UNUSED,
 	return strcmp(e1->key, e2->key);
 }
 
+struct config_set *git_configset_alloc(void)
+{
+	return xmalloc(sizeof(struct config_set));
+}
+
 void git_configset_init(struct config_set *set)
 {
 	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
diff --git a/config.h b/config.h
index 54b47dec9e..abb4415b80 100644
--- a/config.h
+++ b/config.h
@@ -472,6 +472,11 @@ struct config_set {
 	struct configset_list list;
 };
 
+/**
+ * Alloc a config_set
+ */
+struct config_set *git_configset_alloc(void);
+
 /**
  * Initializes the config_set `cs`.
  */
-- 
2.46.0.rc2.264.g509ed76dc8-goog

[RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/cgit-rs/Cargo.lock             | 83 ++++++++++++++++++++++++++
 contrib/cgit-rs/Cargo.toml             |  1 +
 contrib/cgit-rs/public_symbol_export.c | 25 ++++++++
 contrib/cgit-rs/public_symbol_export.h | 10 ++++
 contrib/cgit-rs/src/lib.rs             | 65 +++++++++++++++++++-
 contrib/cgit-rs/src/main.rs            | 13 ++++
 6 files changed, 196 insertions(+), 1 deletion(-)

diff --git a/contrib/cgit-rs/Cargo.lock b/contrib/cgit-rs/Cargo.lock
index f50c593995..1d40ac5faa 100644
--- a/contrib/cgit-rs/Cargo.lock
+++ b/contrib/cgit-rs/Cargo.lock
@@ -6,11 +6,94 @@ version = 3
 name = "cgit"
 version = "0.1.0"
 dependencies = [
+ "home",
  "libc",
 ]
 
+[[package]]
+name = "home"
+version = "0.5.9"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e3d1354bf6b7235cb4a0576c2619fd4ed18183f689b12b006a0ee7329eeff9a5"
+dependencies = [
+ "windows-sys",
+]
+
 [[package]]
 name = "libc"
 version = "0.2.155"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
+
+[[package]]
+name = "windows-sys"
+version = "0.52.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
+dependencies = [
+ "windows-targets",
+]
+
+[[package]]
+name = "windows-targets"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
+dependencies = [
+ "windows_aarch64_gnullvm",
+ "windows_aarch64_msvc",
+ "windows_i686_gnu",
+ "windows_i686_gnullvm",
+ "windows_i686_msvc",
+ "windows_x86_64_gnu",
+ "windows_x86_64_gnullvm",
+ "windows_x86_64_msvc",
+]
+
+[[package]]
+name = "windows_aarch64_gnullvm"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
+
+[[package]]
+name = "windows_aarch64_msvc"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
+
+[[package]]
+name = "windows_i686_gnu"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
+
+[[package]]
+name = "windows_i686_gnullvm"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
+
+[[package]]
+name = "windows_i686_msvc"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
+
+[[package]]
+name = "windows_x86_64_gnu"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
+
+[[package]]
+name = "windows_x86_64_gnullvm"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
+
+[[package]]
+name = "windows_x86_64_msvc"
+version = "0.52.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
index 7b55e6f3e1..5768fce9e5 100644
--- a/contrib/cgit-rs/Cargo.toml
+++ b/contrib/cgit-rs/Cargo.toml
@@ -14,3 +14,4 @@ path = "src/lib.rs"
 
 [dependencies]
 libc = "0.2.155"
+home = "0.5.9"
diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
index ab3401ac40..9641afca89 100644
--- a/contrib/cgit-rs/public_symbol_export.c
+++ b/contrib/cgit-rs/public_symbol_export.c
@@ -31,6 +31,31 @@ int libgit_parse_maybe_bool(const char *val)
 	return git_parse_maybe_bool(val);
 }
 
+struct config_set *libgit_configset_alloc(void)
+{
+	return git_configset_alloc();
+}
+
+void libgit_configset_init(struct config_set *cs)
+{
+	git_configset_init(cs);
+}
+
+int libgit_configset_add_file(struct config_set *cs, const char *filename)
+{
+	return git_configset_add_file(cs, filename);
+}
+
+int libgit_configset_get_int(struct config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int(cs, key, dest);
+}
+
+int libgit_configset_get_string(struct config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string(cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/cgit-rs/public_symbol_export.h b/contrib/cgit-rs/public_symbol_export.h
index 97e8e871ba..f22937202a 100644
--- a/contrib/cgit-rs/public_symbol_export.h
+++ b/contrib/cgit-rs/public_symbol_export.h
@@ -9,6 +9,16 @@ void libgit_initialize_the_repository(void);
 
 int libgit_parse_maybe_bool(const char *val);
 
+struct config_set *libgit_configset_alloc(void);
+
+void libgit_configset_init(struct config_set *cs);
+
+int libgit_configset_add_file(struct config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
index df350e758f..acfd3659e8 100644
--- a/contrib/cgit-rs/src/lib.rs
+++ b/contrib/cgit-rs/src/lib.rs
@@ -1,4 +1,57 @@
-use libc::{c_char, c_int};
+use std::ffi::{CStr, CString};
+
+use libc::{c_char, c_int, c_void};
+
+pub enum GitConfigSet {}
+
+pub struct ConfigSet(*mut GitConfigSet);
+impl ConfigSet {
+
+    pub fn new() -> Self {
+        unsafe {
+            // TODO: we need to handle freeing this when the ConfigSet is dropped
+            // git_configset_clear(ptr) and then libc::free(ptr)
+            let ptr = libgit_configset_alloc();
+            libgit_configset_init(ptr);
+            ConfigSet(ptr)
+        }
+    }
+
+    pub fn add_files(&mut self, files: &[&str]) {
+        for file in files {
+            let rs = CString::new(*file).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val)
+    }
+
+    pub fn get_str(&mut self, key: &str) -> Option<CString> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0 {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str = CString::from_vec_with_nul(borrowed_str.to_bytes_with_nul().to_vec());
+            libc::free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str.unwrap())
+        }
+    }
+}
 
 extern "C" {
     pub fn libgit_setup_git_directory() -> *const c_char;
@@ -15,4 +68,14 @@ extern "C" {
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    fn libgit_configset_alloc() -> *mut GitConfigSet;
+
+    fn libgit_configset_init(cs: *mut GitConfigSet);
+
+    fn libgit_configset_add_file(cs: *mut GitConfigSet, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(cs: *mut GitConfigSet, key: *const c_char, int: *mut c_int) -> c_int;
+    pub fn libgit_configset_get_string(cs: *mut GitConfigSet, key: *const c_char, dest: *mut *mut c_char) -> c_int;
+
 }
diff --git a/contrib/cgit-rs/src/main.rs b/contrib/cgit-rs/src/main.rs
index c5f8644fca..07c8a71a13 100644
--- a/contrib/cgit-rs/src/main.rs
+++ b/contrib/cgit-rs/src/main.rs
@@ -1,4 +1,5 @@
 use std::ffi::{CStr, CString};
+use home::home_dir;
 
 fn main() {
     println!("Let's print some strings provided by Git");
@@ -28,4 +29,16 @@ fn main() {
             val
         );
     };
+
+    println!("\nTry out our configset wrappers");
+    let mut cs = cgit::ConfigSet::new();
+    let mut path = home_dir().expect("cannot get home directory path");
+    path.push(".gitconfig");
+    let path:String = path.into_os_string().into_string().unwrap();
+    cs.add_files(&["/etc/gitconfig", ".gitconfig", &path]);
+    /*
+     * Returns Some(x) if defined in local config, otherwise None
+     */
+    println!("get_configset_get_int = {:?}", cs.get_int("trace2.eventNesting"));
+    println!("cs.get_str(\"garbage\") = {:?}", cs.get_str("this_string_does_not_exist"));
 }
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2865 bytes --]

On 2024-08-07 at 18:21:31, Josh Steadmon wrote:
> diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
> index 7b55e6f3e1..5768fce9e5 100644
> --- a/contrib/cgit-rs/Cargo.toml
> +++ b/contrib/cgit-rs/Cargo.toml
> @@ -14,3 +14,4 @@ path = "src/lib.rs"
>  
>  [dependencies]
>  libc = "0.2.155"
> +home = "0.5.9"

Okay, here's where we get to my previous mention of supported platforms.
This depends on Rust 1.70, and Debian stable has only 1.63.  Trying
`cargo build --release` on that version returns this:

  Downloaded home v0.5.9
  Downloaded libc v0.2.155
  Downloaded 2 crates (752.3 KB) in 0.17s
error: package `home v0.5.9` cannot be built because it requires rustc 1.70.0 or newer, while the currently active rustc version is 1.63.0

My recommended approach here is to support the version in Debian stable,
plus the version in Debian oldstable for a year after the new stable
comes out, which is what I do.  That gives people a year to upgrade if
they want to use our code.  We _don't_ want to follow the
latest-stable-Rust approach because it isn't appropriate that software
has a six-week lifespan of support and that isn't going to work for
software like Git that people often compile locally on older versions.

We also need to be conscious that while Rust upstream provides some
binaries for some platforms, many platforms rely on the distro packages
because Rust upstream doesn't ship binaries for their target.  Thus,
simply using rustup is not viable for many targets, which is another
reason that latest-stable-Rust won't fly.

Debian stable is the version that most projects who have defined
lifespans track, so it's also what we should track.  According to my
recommended approach, that would be 1.63.

If the Rust project agrees to provide LTS versions, then we can switch
to those.

In any event, whatever we decide is necessarily going to involve us very
carefully planning our dependencies since some crates depend on the
latest version whenever it comes out and we're not going to want to do
that.

I'd also note that we don't actually want the home crate.  The
README says this:

  The definition of home_dir provided by the standard library is
  incorrect because it considers the HOME environment variable on
  Windows. This causes surprising situations where a Rust program will
  behave differently depending on whether it is run under a Unix
  emulation environment like Cygwin or MinGW. Neither Cargo nor rustup
  use the standard library's definition - they use the definition here.

Except that in Git, we _do_ want to honour `HOME` at all times.  Thus,
the use of the `home` crate instead of the standard library provides
exactly the wrong behaviour, and we should remove this dependency.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> My recommended approach here is to support the version in Debian stable,
> plus the version in Debian oldstable for a year after the new stable
> comes out, which is what I do.  That gives people a year to upgrade if
> they want to use our code.  We _don't_ want to follow the
> latest-stable-Rust approach because it isn't appropriate that software
> has a six-week lifespan of support and that isn't going to work for
> software like Git that people often compile locally on older versions.

This is sensible.  Conservative enough to avoid leaving folks on
other major distros behind, but still not too old to be a burden.

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Josh Steadmon]

On 2024.08.07 21:40, brian m. carlson wrote:
> On 2024-08-07 at 18:21:31, Josh Steadmon wrote:
> > diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
> > index 7b55e6f3e1..5768fce9e5 100644
> > --- a/contrib/cgit-rs/Cargo.toml
> > +++ b/contrib/cgit-rs/Cargo.toml
> > @@ -14,3 +14,4 @@ path = "src/lib.rs"
> >  
> >  [dependencies]
> >  libc = "0.2.155"
> > +home = "0.5.9"
> 
> Okay, here's where we get to my previous mention of supported platforms.
> This depends on Rust 1.70, and Debian stable has only 1.63.  Trying
> `cargo build --release` on that version returns this:
> 
>   Downloaded home v0.5.9
>   Downloaded libc v0.2.155
>   Downloaded 2 crates (752.3 KB) in 0.17s
> error: package `home v0.5.9` cannot be built because it requires rustc 1.70.0 or newer, while the currently active rustc version is 1.63.0
> 
> My recommended approach here is to support the version in Debian stable,
> plus the version in Debian oldstable for a year after the new stable
> comes out, which is what I do.  That gives people a year to upgrade if
> they want to use our code.  We _don't_ want to follow the
> latest-stable-Rust approach because it isn't appropriate that software
> has a six-week lifespan of support and that isn't going to work for
> software like Git that people often compile locally on older versions.
> 
> We also need to be conscious that while Rust upstream provides some
> binaries for some platforms, many platforms rely on the distro packages
> because Rust upstream doesn't ship binaries for their target.  Thus,
> simply using rustup is not viable for many targets, which is another
> reason that latest-stable-Rust won't fly.
> 
> Debian stable is the version that most projects who have defined
> lifespans track, so it's also what we should track.  According to my
> recommended approach, that would be 1.63.
> 
> If the Rust project agrees to provide LTS versions, then we can switch
> to those.
> 
> In any event, whatever we decide is necessarily going to involve us very
> carefully planning our dependencies since some crates depend on the
> latest version whenever it comes out and we're not going to want to do
> that.
> 
> I'd also note that we don't actually want the home crate.  The
> README says this:
> 
>   The definition of home_dir provided by the standard library is
>   incorrect because it considers the HOME environment variable on
>   Windows. This causes surprising situations where a Rust program will
>   behave differently depending on whether it is run under a Unix
>   emulation environment like Cygwin or MinGW. Neither Cargo nor rustup
>   use the standard library's definition - they use the definition here.
> 
> Except that in Git, we _do_ want to honour `HOME` at all times.  Thus,
> the use of the `home` crate instead of the standard library provides
> exactly the wrong behaviour, and we should remove this dependency.
> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

I've replaced home::home_dir with std::env::home_dir, however the latter
is deprecated, so we may want to reimplement this entirely.

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Calvin Wan]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:
> On 2024-08-07 at 18:21:31, Josh Steadmon wrote:
> > diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
> > index 7b55e6f3e1..5768fce9e5 100644
> > --- a/contrib/cgit-rs/Cargo.toml
> > +++ b/contrib/cgit-rs/Cargo.toml
> > @@ -14,3 +14,4 @@ path = "src/lib.rs"
> >  
> >  [dependencies]
> >  libc = "0.2.155"
> > +home = "0.5.9"
> 
> Okay, here's where we get to my previous mention of supported platforms.
> This depends on Rust 1.70, and Debian stable has only 1.63.  Trying
> `cargo build --release` on that version returns this:
> 
>   Downloaded home v0.5.9
>   Downloaded libc v0.2.155
>   Downloaded 2 crates (752.3 KB) in 0.17s
> error: package `home v0.5.9` cannot be built because it requires rustc 1.70.0 or newer, while the currently active rustc version is 1.63.0
> 
> My recommended approach here is to support the version in Debian stable,
> plus the version in Debian oldstable for a year after the new stable
> comes out, which is what I do.  That gives people a year to upgrade if
> they want to use our code.  We _don't_ want to follow the
> latest-stable-Rust approach because it isn't appropriate that software
> has a six-week lifespan of support and that isn't going to work for
> software like Git that people often compile locally on older versions.
> 
> We also need to be conscious that while Rust upstream provides some
> binaries for some platforms, many platforms rely on the distro packages
> because Rust upstream doesn't ship binaries for their target.  Thus,
> simply using rustup is not viable for many targets, which is another
> reason that latest-stable-Rust won't fly.
> 
> Debian stable is the version that most projects who have defined
> lifespans track, so it's also what we should track.  According to my
> recommended approach, that would be 1.63.

After getting rid of the `home` crate, the only other issue we ran into
downgrading the version to 1.63 was with `std::ffi{c_char, c_int}`.
Those were only made available in 1.64 and they are obviously quite
necessary for us to be able to call C functions. Do you know of any
alternatives we can use? I also don't think reinventing the wheel with
our own implementation makes sense in this case, and even if Debian were
to upgrade stable to a higher version today, we would still need to
support oldstable for another year.

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On 2024-09-04 at 17:30:53, Calvin Wan wrote:
> After getting rid of the `home` crate, the only other issue we ran into
> downgrading the version to 1.63 was with `std::ffi{c_char, c_int}`.
> Those were only made available in 1.64 and they are obviously quite
> necessary for us to be able to call C functions. Do you know of any
> alternatives we can use? I also don't think reinventing the wheel with
> our own implementation makes sense in this case, and even if Debian were
> to upgrade stable to a higher version today, we would still need to
> support oldstable for another year.

I think we can do this with libc, which you're importing at the moment.
You can do something like this:

src/types.rs:
----
pub use libc::{c_char, c_int};
----

and then do `use crate::types::{c_char, c_int}` wherever you want them.

Then, when we upgrade to 1.64 or newer, we can simply replace the
contents of `src/types.rs` with this:

----
pub use std::ffi::{c_char, c_int};
----

and that's the only thing we'll need to change.

If we switch to using rustix instead, then it will look like this:

----
pub use rustix::ffi::c_char;

pub type c_int = i32;
----

While the C standard requires that `int` only has 16 bits, Git doesn't
target any platforms where `int` is anything but 32 bits and it won't
work with any other configuration without serious changes.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Calvin Wan]

On Wed, Sep 4, 2024 at 10:49 AM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> On 2024-09-04 at 17:30:53, Calvin Wan wrote:
> > After getting rid of the `home` crate, the only other issue we ran into
> > downgrading the version to 1.63 was with `std::ffi{c_char, c_int}`.
> > Those were only made available in 1.64 and they are obviously quite
> > necessary for us to be able to call C functions. Do you know of any
> > alternatives we can use? I also don't think reinventing the wheel with
> > our own implementation makes sense in this case, and even if Debian were
> > to upgrade stable to a higher version today, we would still need to
> > support oldstable for another year.
>
> I think we can do this with libc, which you're importing at the moment.
> You can do something like this:
>
> src/types.rs:
> ----
> pub use libc::{c_char, c_int};
> ----
>
> and then do `use crate::types::{c_char, c_int}` wherever you want them.

While this gets rid of `std::ffi{c_char, c_int}` dependency,
`std::ffi{CStr, CString}` are also 1.64 unfortunately, and I don't see
the equivalent of it in libc. I'll reroll v3 without this change for
now. Hopefully Debian stable just upgrades to 1.64 or higher and we
don't have to worry about these ffi conversions :)

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]

On 2024-09-06 at 19:37:18, Calvin Wan wrote:
> While this gets rid of `std::ffi{c_char, c_int}` dependency,
> `std::ffi{CStr, CString}` are also 1.64 unfortunately, and I don't see
> the equivalent of it in libc. I'll reroll v3 without this change for
> now. Hopefully Debian stable just upgrades to 1.64 or higher and we
> don't have to worry about these ffi conversions :)

I think that's actually a mistake, since some arguments like
`from_bytes_with_nul` are labeled as 1.10.0.  In any event, the
following program compiles fine for me on 1.63, so I'm not worried about
their usage:

----
use std::ffi::CStr;

fn main() {
    let c = CStr::from_bytes_with_nul(b"abc\0");
    let c = c.to_owned();
    println!("Hello, world!\n");
}
----
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Junio C Hamano]

Calvin Wan <calvinwan@google.com> writes:

> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
>> ...
>> Debian stable is the version that most projects who have defined
>> lifespans track, so it's also what we should track.  According to my
>> recommended approach, that would be 1.63.
>
> ... I also don't think reinventing the wheel with
> our own implementation makes sense in this case,

I do agree that we would want to avoid that.

> and even if Debian were
> to upgrade stable to a higher version today, we would still need to
> support oldstable for another year.

I doubt that part, though.  As long as the rust binding stays an
optional code, as long as we are supported by the "current" system,
we would still have enough audience to matter.

What's the primary objective of this effort, by the way?  

Is it "we need to access the guts of Git implementation from Rust"?
Or does it merely serve as an example application to have Rust
bindings, a good goal to have to give us an incentive to clean up
the subsystem interactions in our code?  

If it is the former, we cannot reasonably achieve that goal until
some form of standardized foreign function interface becomes
available to wide audience.  If it is the latter, on the other hand,
it does not have to be Rust---if the version of Rust that is
distirbuted to the mainstream users is not yet ready to be used in
such a way, we could pick another goal (like, "Can we clean-up the
interface cgit uses to call into us, so that the parts used by them
look more like a proper library?").

Thanks.

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2062 bytes --]

On 2024-09-04 at 18:33:17, Junio C Hamano wrote:
> Is it "we need to access the guts of Git implementation from Rust"?
> Or does it merely serve as an example application to have Rust
> bindings, a good goal to have to give us an incentive to clean up
> the subsystem interactions in our code?
> 
> If it is the former, we cannot reasonably achieve that goal until
> some form of standardized foreign function interface becomes
> available to wide audience.  If it is the latter, on the other hand,
> it does not have to be Rust---if the version of Rust that is
> distirbuted to the mainstream users is not yet ready to be used in
> such a way, we could pick another goal (like, "Can we clean-up the
> interface cgit uses to call into us, so that the parts used by them
> look more like a proper library?").

Traditionally, in Rust, you don't use the C-style types because that
leads to portability problems.  Look at how using "unsigned long" as
"equivalent in size to void *" has gotten our C code to have sharp edges
on Windows, where that isn't true. The approach one typically uses is to
use things like int32_t, which is i32 in Rust, and size_t, which is
usize in Rust.  This leads to much more predictable behaviour on both
sides of the FFI.

The C-style types have long been available in libc and other crates that
are designed to work with C FFI, and as a practical matter you do need
to use that crate somewhere in your stack (or reimplement it) to call
functions in libc and the other core system libraries, so you're not
really lacking those types.

They're also available in the `std::os::raw` module as of Rust 1.1; it's
just that as of Rust 1.64, they're in `std::ffi` and `core::ffi` as
well, mostly to help embedded systems (which don't have `std`, and thus,
don't have `std::os::raw`).  Using `std::os::raw` or `libc` should be
fine for Git, since we're not targeting operating system kernels,
bootloaders, or firmware such as UEFI (I hope).
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> The C-style types have long been available in libc and other crates that
> are designed to work with C FFI, and as a practical matter you do need
> to use that crate somewhere in your stack (or reimplement it) to call
> functions in libc and the other core system libraries, so you're not
> really lacking those types.

Glad to know that the feature we need was already there.  So, it is
just the interface was in flux when they worked on the version
Debian happens to ship right now, I guess.

> They're also available in the `std::os::raw` module as of Rust 1.1; it's
> just that as of Rust 1.64, they're in `std::ffi` and `core::ffi` as
> well, mostly to help embedded systems (which don't have `std`, and thus,
> don't have `std::os::raw`).  Using `std::os::raw` or `libc` should be
> fine for Git, since we're not targeting operating system kernels,
> bootloaders, or firmware such as UEFI (I hope).

Yup, as long as we have a clean migration path (like the one you
showed Calvin in your other message in this thread), and use of a
slightly older way will not immediately be deprecated in a newer
version of Rust, then we should be fine going forward.

Thanks.

Re: [RFC PATCH 6/6] contrib/cgit-rs: add a subset of configset wrappers

[Josh Steadmon]

On 2024.09.04 11:33, Junio C Hamano wrote:
> Calvin Wan <calvinwan@google.com> writes:
> 
> > "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> >> ...
> >> Debian stable is the version that most projects who have defined
> >> lifespans track, so it's also what we should track.  According to my
> >> recommended approach, that would be 1.63.
> >
> > ... I also don't think reinventing the wheel with
> > our own implementation makes sense in this case,
> 
> I do agree that we would want to avoid that.
> 
> > and even if Debian were
> > to upgrade stable to a higher version today, we would still need to
> > support oldstable for another year.
> 
> I doubt that part, though.  As long as the rust binding stays an
> optional code, as long as we are supported by the "current" system,
> we would still have enough audience to matter.
> 
> What's the primary objective of this effort, by the way?  
> 
> Is it "we need to access the guts of Git implementation from Rust"?
> Or does it merely serve as an example application to have Rust
> bindings, a good goal to have to give us an incentive to clean up
> the subsystem interactions in our code?  

For us at $DAYJOB, it's "we need to access the guts of Git from Rust".

> If it is the former, we cannot reasonably achieve that goal until
> some form of standardized foreign function interface becomes
> available to wide audience.  If it is the latter, on the other hand,
> it does not have to be Rust---if the version of Rust that is
> distirbuted to the mainstream users is not yet ready to be used in
> such a way, we could pick another goal (like, "Can we clean-up the
> interface cgit uses to call into us, so that the parts used by them
> look more like a proper library?").
> 
> Thanks.

I think brian already addressed your point about having a standard FFI,
but if there's anything that still needs clarification please let me
know.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2398 bytes --]

On 2024-08-07 at 18:21:25, Josh Steadmon wrote:
> We are putting error handling on hold for now since it is too complex
> and we intend other CLIs to be our first customers, in which case
> printing out errors is not the worst.

While I think that's okay for now, that really needs to be one of the
first priorities of things we fix.  I have some ideas about an approach
which we can take, which I'll send out in the next few days in a
separate thread.

> While the wrapper itself lives in contrib/, there are a couple of
> patches that touch git.git code. These patches are necessary for the
> wrapper, but not for git.git itself, which may seem unnecessary to
> merge. However, I would argue that other languages (not just limited to
> Rust) have issues calling functions that require a pointer to
> non-generic objects and essentially require a redefinition in their own
> language.

I don't see a problem with this.  It seems very self contained.

> We're sending this series as RFC because there is remaining work
> we'd like to do, but we'd like to get early feedback on this approach,
> and particularly to ask for advice on a few topics:
> 
> * alternative methods of exposing only a subset of symbols in our
>   library

We should probably support symbol versioning in the libgit_ code on
supported platforms if we're going to expose it.  Otherwise, we're
setting ourselves and distributors up for a world of pain when we change
the ABI, such as when we add error handling.

> * bikeshedding on the name (yes, really). There is an active, unrelated
>   CGit project [4] that we only recently became aware of. We originally
>   took the name "cgit" because at $DAYJOB we sometimes refer to git.git
>   as "cgit" to distinguish it from jgit [5].
> 
> * gauging the level of interest in calling Git code from Rust

I left some comments in the series.  I think this is a nice first step
as a proof of concept, and I'm very pleased to see it.

If your goal is to simply expose C APIs, I suggest calling it something
like cgit-sys, since the -sys suffix is customary for wrappers of C
libraries.  If your goal is to provide actual wrappers that people will
use in real Rust code, then I think that we should _not_ expose the C
API, since most Rust users will not want to use that.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.07 22:03, brian m. carlson wrote:
> On 2024-08-07 at 18:21:25, Josh Steadmon wrote:
> > We are putting error handling on hold for now since it is too complex
> > and we intend other CLIs to be our first customers, in which case
> > printing out errors is not the worst.
> 
> While I think that's okay for now, that really needs to be one of the
> first priorities of things we fix.  I have some ideas about an approach
> which we can take, which I'll send out in the next few days in a
> separate thread.

I'll be very interested in seeing this, please CC me.


> > While the wrapper itself lives in contrib/, there are a couple of
> > patches that touch git.git code. These patches are necessary for the
> > wrapper, but not for git.git itself, which may seem unnecessary to
> > merge. However, I would argue that other languages (not just limited to
> > Rust) have issues calling functions that require a pointer to
> > non-generic objects and essentially require a redefinition in their own
> > language.
> 
> I don't see a problem with this.  It seems very self contained.
> 
> > We're sending this series as RFC because there is remaining work
> > we'd like to do, but we'd like to get early feedback on this approach,
> > and particularly to ask for advice on a few topics:
> > 
> > * alternative methods of exposing only a subset of symbols in our
> >   library
> 
> We should probably support symbol versioning in the libgit_ code on
> supported platforms if we're going to expose it.  Otherwise, we're
> setting ourselves and distributors up for a world of pain when we change
> the ABI, such as when we add error handling.

Symbol visibility and versioning are both areas I'm not very familiar
with. I'll do some homework but might need additional help later on, so
please keep an eye out for beginner mistakes I might make in later
versions.


> > * bikeshedding on the name (yes, really). There is an active, unrelated
> >   CGit project [4] that we only recently became aware of. We originally
> >   took the name "cgit" because at $DAYJOB we sometimes refer to git.git
> >   as "cgit" to distinguish it from jgit [5].
> > 
> > * gauging the level of interest in calling Git code from Rust
> 
> I left some comments in the series.  I think this is a nice first step
> as a proof of concept, and I'm very pleased to see it.
> 
> If your goal is to simply expose C APIs, I suggest calling it something
> like cgit-sys, since the -sys suffix is customary for wrappers of C
> libraries.  If your goal is to provide actual wrappers that people will
> use in real Rust code, then I think that we should _not_ expose the C
> API, since most Rust users will not want to use that.
> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]

On 2024-08-07 at 22:03:53, brian m. carlson wrote:
> I left some comments in the series.  I think this is a nice first step
> as a proof of concept, and I'm very pleased to see it.

I noticed a couple of other things.  First, the code has not been run
through rustfmt.  I think it would be helpful for us to do that since it
makes it easier to not argue about style and it can be easily enforced
in CI.  It will also reduce diff noise, which I expect Junio will
appreciate.

Second, cargo clippy complains about some of the code.  It's again
helpful if we can fix those warnings or, if they're not appropriate, to
disable them with an appropriate `allow` pragma.  (In this case, I think
they're both spot on, but I have seen some cases where I've disabled a
warning.)  This is something we may also want to test in CI in the
future, and downstream users of our crate will appreciate not getting
warnings when using clippy themselves, so we should be kind to them.

I noticed these because my editor complains about the latter and I have
now intuited enough of rustfmt's output that I can tell sometimes when
things aren't formatted with it.

For those members of the list who are less familiar with Rust, rustfmt
is the standard code formatter (and formatting verifier) and clippy is a
lint tool recommending best practices.  Both are shipped with Rust and
using both is customary for Rust projects.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.08 01:33, brian m. carlson wrote:
> On 2024-08-07 at 22:03:53, brian m. carlson wrote:
> > I left some comments in the series.  I think this is a nice first step
> > as a proof of concept, and I'm very pleased to see it.
> 
> I noticed a couple of other things.  First, the code has not been run
> through rustfmt.  I think it would be helpful for us to do that since it
> makes it easier to not argue about style and it can be easily enforced
> in CI.  It will also reduce diff noise, which I expect Junio will
> appreciate.
> 
> Second, cargo clippy complains about some of the code.  It's again
> helpful if we can fix those warnings or, if they're not appropriate, to
> disable them with an appropriate `allow` pragma.  (In this case, I think
> they're both spot on, but I have seen some cases where I've disabled a
> warning.)  This is something we may also want to test in CI in the
> future, and downstream users of our crate will appreciate not getting
> warnings when using clippy themselves, so we should be kind to them.
> 
> I noticed these because my editor complains about the latter and I have
> now intuited enough of rustfmt's output that I can tell sometimes when
> things aren't formatted with it.
> 
> For those members of the list who are less familiar with Rust, rustfmt
> is the standard code formatter (and formatting verifier) and clippy is a
> lint tool recommending best practices.  Both are shipped with Rust and
> using both is customary for Rust projects.
> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

Applied Clippy and rustfmt fixes throughout V2, thanks for the catch.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Jason A. Donenfeld]

Hi,

Cgit maintainer here...

On Wed, Aug 07, 2024 at 11:21:25AM -0700, Josh Steadmon wrote:
> * bikeshedding on the name (yes, really). There is an active, unrelated
>   CGit project [4] that we only recently became aware of. We originally
>   took the name "cgit" because at $DAYJOB we sometimes refer to git.git
>   as "cgit" to distinguish it from jgit [5].

Indeed, please find something else. Cgit is a real project, used by
many, such as git.kernel.org, and it'll turn into a real hassle for both
of us if there's ambiguity and confusion.

What about libgit-rs? Or even libgit3, where the rustness of it is
simply an implementation detail, so the name won't feel dated 15 years
from now when everything is written in Rust anyway and -rs is so 2020s?

Jason

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-08 13:51, Jason A. Donenfeld wrote:
> Cgit maintainer here...
> 
> On Wed, Aug 07, 2024 at 11:21:25AM -0700, Josh Steadmon wrote:
>> * bikeshedding on the name (yes, really). There is an active, 
>> unrelated
>>   CGit project [4] that we only recently became aware of. We 
>> originally
>>   took the name "cgit" because at $DAYJOB we sometimes refer to 
>> git.git
>>   as "cgit" to distinguish it from jgit [5].
> 
> Indeed, please find something else. Cgit is a real project, used by
> many, such as git.kernel.org, and it'll turn into a real hassle for 
> both
> of us if there's ambiguity and confusion.

Totally agreed, naming it cgit-rs makes pretty much no sense.

> What about libgit-rs? Or even libgit3, where the rustness of it is
> simply an implementation detail, so the name won't feel dated 15 years
> from now when everything is written in Rust anyway and -rs is so 2020s?

Well, there are still very active commercial projects written in COBOL
or Clipper, for example, so I wouldn't go as far as _everything_ being
written in Rust at some point.

RE: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[rsbecker]

On Thursday, August 8, 2024 9:59 AM, Dragan Simic wrote:
>On 2024-08-08 13:51, Jason A. Donenfeld wrote:
>> Cgit maintainer here...
>>
>> On Wed, Aug 07, 2024 at 11:21:25AM -0700, Josh Steadmon wrote:
>>> * bikeshedding on the name (yes, really). There is an active,
>>> unrelated
>>>   CGit project [4] that we only recently became aware of. We
>>> originally
>>>   took the name "cgit" because at $DAYJOB we sometimes refer to
>>> git.git
>>>   as "cgit" to distinguish it from jgit [5].
>>
>> Indeed, please find something else. Cgit is a real project, used by
>> many, such as git.kernel.org, and it'll turn into a real hassle for
>> both of us if there's ambiguity and confusion.
>
>Totally agreed, naming it cgit-rs makes pretty much no sense.
>
>> What about libgit-rs? Or even libgit3, where the rustness of it is
>> simply an implementation detail, so the name won't feel dated 15 years
>> from now when everything is written in Rust anyway and -rs is so 2020s?
>
>Well, there are still very active commercial projects written in COBOL or Clipper, for
>example, so I wouldn't go as far as _everything_ being written in Rust at some point.

There are hundreds of millions of lines of code in the NonStop (TAL, COBOL, C, Java,
in that order) community that is actively maintained to this day, with no Rust
involvement (not for lack of trying to get Rust ported). Without these projects, most
of your credit and debit cards would probably not work. 😉 Many of these repositories
are more than 500Mb, with a couple I have seen exceeding 2Gb (just for source code).

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-08 17:38, rsbecker@nexbridge.com wrote:
> On Thursday, August 8, 2024 9:59 AM, Dragan Simic wrote:
>> On 2024-08-08 13:51, Jason A. Donenfeld wrote:
>>> Cgit maintainer here...
>>> 
>>> On Wed, Aug 07, 2024 at 11:21:25AM -0700, Josh Steadmon wrote:
>>>> * bikeshedding on the name (yes, really). There is an active,
>>>> unrelated
>>>>   CGit project [4] that we only recently became aware of. We
>>>> originally
>>>>   took the name "cgit" because at $DAYJOB we sometimes refer to
>>>> git.git
>>>>   as "cgit" to distinguish it from jgit [5].
>>> 
>>> Indeed, please find something else. Cgit is a real project, used by
>>> many, such as git.kernel.org, and it'll turn into a real hassle for
>>> both of us if there's ambiguity and confusion.
>> 
>> Totally agreed, naming it cgit-rs makes pretty much no sense.
>> 
>>> What about libgit-rs? Or even libgit3, where the rustness of it is
>>> simply an implementation detail, so the name won't feel dated 15 
>>> years
>>> from now when everything is written in Rust anyway and -rs is so 
>>> 2020s?
>> 
>> Well, there are still very active commercial projects written in COBOL 
>> or Clipper, for
>> example, so I wouldn't go as far as _everything_ being written in Rust 
>> at some point.
> 
> There are hundreds of millions of lines of code in the NonStop (TAL,
> COBOL, C, Java,
> in that order) community that is actively maintained to this day, with 
> no Rust
> involvement (not for lack of trying to get Rust ported). Without these
> projects, most
> of your credit and debit cards would probably not work. 😉 Many of
> these repositories
> are more than 500Mb, with a couple I have seen exceeding 2Gb (just for
> source code).

Those are good examples.  Also, I believe that the entire financial 
system of
a country from South America runs on software written in Clipper, and 
they keep
producing modern Clipper compilers that run on Windows.  Let me remind 
you that
Cliper was used back in the days of DOS and 5.25-inch floppies. :)

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

"Jason A. Donenfeld" <Jason@zx2c4.com> writes:

> Cgit maintainer here...
>
> On Wed, Aug 07, 2024 at 11:21:25AM -0700, Josh Steadmon wrote:
>> * bikeshedding on the name (yes, really). There is an active, unrelated
>>   CGit project [4] that we only recently became aware of. We originally
>>   took the name "cgit" because at $DAYJOB we sometimes refer to git.git
>>   as "cgit" to distinguish it from jgit [5].
>
> Indeed, please find something else. Cgit is a real project, used by
> many, such as git.kernel.org, and it'll turn into a real hassle for both
> of us if there's ambiguity and confusion.
>
> What about libgit-rs? Or even libgit3, where the rustness of it is
> simply an implementation detail, so the name won't feel dated 15 years
> from now when everything is written in Rust anyway and -rs is so 2020s?

libgit-rs sounds like a descriptive and good name.

I am not sure about the wisdom of _not_ saying anything about Rust.
In 2035 maybe nobody is writing in Rust but in not-yet-invented
alternative that is even better.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> We're sending this series as RFC because there is remaining work
> we'd like to do, but we'd like to get early feedback on this approach,
> and particularly to ask for advice on a few topics:

I am not sure how much this is reusable, after seeing comments that
"cgit-rs" may not be the best name for this thing and pathnames may
have to change, but I needed the following merge-fix to get this
into "seen" and have the result pass "make", due to interactions
with the ps/config-wo-the-repository topic.

 contrib/cgit-rs/public_symbol_export.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
index 9641afca89..2732f5898e 100644
--- a/contrib/cgit-rs/public_symbol_export.c
+++ b/contrib/cgit-rs/public_symbol_export.c
@@ -9,6 +9,8 @@
 #include "setup.h"
 #include "version.h"
 
+extern struct repository *the_repository;
+
 #pragma GCC visibility push(default)
 
 const char *libgit_setup_git_directory(void)
@@ -18,7 +20,7 @@ const char *libgit_setup_git_directory(void)
 
 int libgit_config_get_int(const char *key, int *dest)
 {
-	return git_config_get_int(key, dest);
+	return repo_config_get_int(the_repository, key, dest);
 }
 
 void libgit_initialize_the_repository(void)
-- 
2.46.0-326-g1e046905fc

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Josh Steadmon <steadmon@google.com> writes:
>
>> We're sending this series as RFC because there is remaining work
>> we'd like to do, but we'd like to get early feedback on this approach,
>> and particularly to ask for advice on a few topics:
>
> I am not sure how much this is reusable, after seeing comments that
> "cgit-rs" may not be the best name for this thing and pathnames may
> have to change, but I needed the following merge-fix to get this
> into "seen" and have the result pass "make", due to interactions
> with the ps/config-wo-the-repository topic.

In case you are wondering "so... is there anything actionable for
*US*???", there isn't, exactly.  But you'd need to holler if the
"merge-fix" you saw in the message not correctly addressing the
semantic clash between these two topics.

Thanks.

>
>  contrib/cgit-rs/public_symbol_export.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/cgit-rs/public_symbol_export.c b/contrib/cgit-rs/public_symbol_export.c
> index 9641afca89..2732f5898e 100644
> --- a/contrib/cgit-rs/public_symbol_export.c
> +++ b/contrib/cgit-rs/public_symbol_export.c
> @@ -9,6 +9,8 @@
>  #include "setup.h"
>  #include "version.h"
>  
> +extern struct repository *the_repository;
> +
>  #pragma GCC visibility push(default)
>  
>  const char *libgit_setup_git_directory(void)
> @@ -18,7 +20,7 @@ const char *libgit_setup_git_directory(void)
>  
>  int libgit_config_get_int(const char *key, int *dest)
>  {
> -	return git_config_get_int(key, dest);
> +	return repo_config_get_int(the_repository, key, dest);
>  }
>  
>  void libgit_initialize_the_repository(void)

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.09 12:29, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Josh Steadmon <steadmon@google.com> writes:
> >
> >> We're sending this series as RFC because there is remaining work
> >> we'd like to do, but we'd like to get early feedback on this approach,
> >> and particularly to ask for advice on a few topics:
> >
> > I am not sure how much this is reusable, after seeing comments that
> > "cgit-rs" may not be the best name for this thing and pathnames may
> > have to change, but I needed the following merge-fix to get this
> > into "seen" and have the result pass "make", due to interactions
> > with the ps/config-wo-the-repository topic.
> 
> In case you are wondering "so... is there anything actionable for
> *US*???", there isn't, exactly.  But you'd need to holler if the
> "merge-fix" you saw in the message not correctly addressing the
> semantic clash between these two topics.
> 
> Thanks.

Looks fine to me, applied in V2.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Josh Steadmon <steadmon@google.com> writes:
>
>> We're sending this series as RFC because there is remaining work
>> we'd like to do, but we'd like to get early feedback on this approach,
>> and particularly to ask for advice on a few topics:
>
> I am not sure how much this is reusable, after seeing comments that
> "cgit-rs" may not be the best name for this thing and pathnames may
> have to change, but I needed the following merge-fix to get this
> into "seen" and have the result pass "make", due to interactions
> with the ps/config-wo-the-repository topic.
>
>  contrib/cgit-rs/public_symbol_export.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

There is another thing.

Listing this file in $(OBJECTS) means that you should be able to
pass "make sparse" to build contrib/cgit-rs/public_symbol_export.sp
but it seems to fail rather miserably.  I am tempted to suggest in
the meantime to futz with $(SP_OBJ) to filter it out in the top
level Makefile.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>>  contrib/cgit-rs/public_symbol_export.c | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> There is another thing.
>
> Listing this file in $(OBJECTS) means that you should be able to
> pass "make sparse" to build contrib/cgit-rs/public_symbol_export.sp
> but it seems to fail rather miserably.  I am tempted to suggest in
> the meantime to futz with $(SP_OBJ) to filter it out in the top
> level Makefile.

FWIW, this one _is_ actionable on your end ;-)  Make sure "make sparse"
does not barf due to the addition of libgit-rust.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.09 13:54, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Josh Steadmon <steadmon@google.com> writes:
> >
> >> We're sending this series as RFC because there is remaining work
> >> we'd like to do, but we'd like to get early feedback on this approach,
> >> and particularly to ask for advice on a few topics:
> >
> > I am not sure how much this is reusable, after seeing comments that
> > "cgit-rs" may not be the best name for this thing and pathnames may
> > have to change, but I needed the following merge-fix to get this
> > into "seen" and have the result pass "make", due to interactions
> > with the ps/config-wo-the-repository topic.
> >
> >  contrib/cgit-rs/public_symbol_export.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> There is another thing.
> 
> Listing this file in $(OBJECTS) means that you should be able to
> pass "make sparse" to build contrib/cgit-rs/public_symbol_export.sp
> but it seems to fail rather miserably.  I am tempted to suggest in
> the meantime to futz with $(SP_OBJ) to filter it out in the top
> level Makefile.

I believe that I fixed `make sparse` (at least in GitHub CI, it fails
for seemingly unrelated reasons on my desktop) by removing some
unnecessarily exposed symbols in public_symbol_export.c. If it still
fails for you in V2, please let me know.

Re: [RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.08.09 13:54, Junio C Hamano wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>> 
>> > Josh Steadmon <steadmon@google.com> writes:
>> >
>> >> We're sending this series as RFC because there is remaining work
>> >> we'd like to do, but we'd like to get early feedback on this approach,
>> >> and particularly to ask for advice on a few topics:
>> >
>> > I am not sure how much this is reusable, after seeing comments that
>> > "cgit-rs" may not be the best name for this thing and pathnames may
>> > have to change, but I needed the following merge-fix to get this
>> > into "seen" and have the result pass "make", due to interactions
>> > with the ps/config-wo-the-repository topic.
>> >
>> >  contrib/cgit-rs/public_symbol_export.c | 4 +++-
>> >  1 file changed, 3 insertions(+), 1 deletion(-)
>> 
>> There is another thing.
>> 
>> Listing this file in $(OBJECTS) means that you should be able to
>> pass "make sparse" to build contrib/cgit-rs/public_symbol_export.sp
>> but it seems to fail rather miserably.  I am tempted to suggest in
>> the meantime to futz with $(SP_OBJ) to filter it out in the top
>> level Makefile.
>
> I believe that I fixed `make sparse` (at least in GitHub CI, it fails
> for seemingly unrelated reasons on my desktop) by removing some
> unnecessarily exposed symbols in public_symbol_export.c. If it still
> fails for you in V2, please let me know.

Thanks.  Please let me know when you send v2 out ;-)

[PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

This series provides two small Rust wrapper libraries around parts of
Git: "cgit-sys", which exposes a few functions from libgit.a, and
"cgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/martinvonz/jj

Please find V1 cover letter here:
https://lore.kernel.org/git/cover.1723054623.git.steadmon@google.com/

Changes in V2:
* Split out the exposed C APIs into a cgit-sys module, and build nicer
  Rust interfaces on top of that in cgit-rs

* Replace the proof-of-concept cgit-info binary with unit tests

* In addition to splitting common_exit() into a new file, also move the
  initialization code in main() to a new init_git() function in its own
  file, and include this in libgit.a. This allows us to drop V1 patch #2
  [which added a wrapper around initialize_repo()]

* Remove libc dependency (by declaring an extern "C" free function, not
  sure if this is the best approach)

* Add git_configset_clear_and_free to also support the above

* Use "std::env::home_dir" instead of the "home" crate due to desired
  behavior on Windows. "std::env::home_dir" is deprecated, but is only
  used in unit tests which will need to be rewritten anyway to provide a
  testdata config.

* Use recommended empty array + PhantomData approach instead of empty
  enums for wrapping opaque C structs/pointers

* Don't force CC=clang in build.rs

* Simplify conversion of PathBufs to Strings in build.rs

* Don't unnecessarily expose git_attr__true and git_attr__false in
  public_symbol_export.c. That fixes `make sparse`.

* Clippy and rustfmt cleanups throughout

* Whitespace fix

Known NEEDSWORK:
* Support older Rust versions

* Investigate alternative methods of managing symbol visibility.

* Figure out symbol versioning

* Bikeshed on the name

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default

* Automate the process of exporting additional functions in cgit-sys
  (possibly with a wrapper script around bindgen)

* Provide testdata configs for unit tests

* Rethink the Rust-y ConfigSet API, particularly for Path vs. &str


Calvin Wan (2):
  contrib/cgit-rs: add repo initialization and config access
  contrib/cgit-rs: add a subset of configset wrappers

Josh Steadmon (3):
  common-main: split init and exit code into new files
  contrib/cgit-rs: introduce Rust wrapper for libgit.a
  config: add git_configset_alloc() and git_configset_clear_and_free()

 .gitignore                                    |   2 +
 Makefile                                      |  15 +++
 common-exit.c                                 |  26 ++++
 common-init.c                                 |  63 ++++++++++
 common-init.h                                 |   6 +
 common-main.c                                 |  83 +------------
 config.c                                      |  11 ++
 config.h                                      |  10 ++
 contrib/cgit-rs/Cargo.lock                    |  14 +++
 contrib/cgit-rs/Cargo.toml                    |  10 ++
 contrib/cgit-rs/cgit-sys/Cargo.lock           |   7 ++
 contrib/cgit-rs/cgit-sys/Cargo.toml           |   9 ++
 contrib/cgit-rs/cgit-sys/README.md            |  15 +++
 contrib/cgit-rs/cgit-sys/build.rs             |  32 +++++
 .../cgit-rs/cgit-sys/public_symbol_export.c   |  76 ++++++++++++
 .../cgit-rs/cgit-sys/public_symbol_export.h   |  28 +++++
 contrib/cgit-rs/cgit-sys/src/lib.rs           | 112 ++++++++++++++++++
 contrib/cgit-rs/src/lib.rs                    |  82 +++++++++++++
 18 files changed, 520 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/cgit-rs/Cargo.lock
 create mode 100644 contrib/cgit-rs/Cargo.toml
 create mode 100644 contrib/cgit-rs/cgit-sys/Cargo.lock
 create mode 100644 contrib/cgit-rs/cgit-sys/Cargo.toml
 create mode 100644 contrib/cgit-rs/cgit-sys/README.md
 create mode 100644 contrib/cgit-rs/cgit-sys/build.rs
 create mode 100644 contrib/cgit-rs/cgit-sys/public_symbol_export.c
 create mode 100644 contrib/cgit-rs/cgit-sys/public_symbol_export.h
 create mode 100644 contrib/cgit-rs/cgit-sys/src/lib.rs
 create mode 100644 contrib/cgit-rs/src/lib.rs

Range-diff against v1:
1:  78c2aa2ef9 ! 1:  800b37d16b common-main: split common_exit() into a new file
    @@ Metadata
     Author: Josh Steadmon <steadmon@google.com>
     
      ## Commit message ##
    -    common-main: split common_exit() into a new file
    +    common-main: split init and exit code into new files
     
         Currently, object files in libgit.a reference common_exit(), which is
         contained in common-main.o. However, common-main.o also includes main(),
    @@ Commit message
         conflicts. Now we are trying to make libgit.a more self-contained, so
         hopefully we can revisit this approach.
     
    +    Additionally, move the initialization code out of main() into a new
    +    init_git() function in its own file. Include this in libgit.a as well,
    +    so that external users can share our setup code without calling our
    +    main().
    +
         [1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
         [2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/
     
    -    Signed-off-by: Josh Steadmon <steadmon@google.com>
    -
     
      ## Makefile ##
    @@ Makefile: LIB_OBJS += combine-diff.o
      LIB_OBJS += commit-reach.o
      LIB_OBJS += commit.o
     +LIB_OBJS += common-exit.o
    ++LIB_OBJS += common-init.o
      LIB_OBJS += compat/nonblock.o
      LIB_OBJS += compat/obstack.o
      LIB_OBJS += compat/terminal.o
    @@ common-exit.c (new)
     +	return code;
     +}
     
    + ## common-init.c (new) ##
    +@@
    ++#define USE_THE_REPOSITORY_VARIABLE
    ++
    ++#include "git-compat-util.h"
    ++#include "common-init.h"
    ++#include "exec-cmd.h"
    ++#include "gettext.h"
    ++#include "attr.h"
    ++#include "repository.h"
    ++#include "setup.h"
    ++#include "strbuf.h"
    ++#include "trace2.h"
    ++
    ++/*
    ++ * Many parts of Git have subprograms communicate via pipe, expect the
    ++ * upstream of a pipe to die with SIGPIPE when the downstream of a
    ++ * pipe does not need to read all that is written.  Some third-party
    ++ * programs that ignore or block SIGPIPE for their own reason forget
    ++ * to restore SIGPIPE handling to the default before spawning Git and
    ++ * break this carefully orchestrated machinery.
    ++ *
    ++ * Restore the way SIGPIPE is handled to default, which is what we
    ++ * expect.
    ++ */
    ++static void restore_sigpipe_to_default(void)
    ++{
    ++	sigset_t unblock;
    ++
    ++	sigemptyset(&unblock);
    ++	sigaddset(&unblock, SIGPIPE);
    ++	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
    ++	signal(SIGPIPE, SIG_DFL);
    ++}
    ++
    ++void init_git(const char **argv)
    ++{
    ++	struct strbuf tmp = STRBUF_INIT;
    ++
    ++	trace2_initialize_clock();
    ++
    ++	/*
    ++	 * Always open file descriptors 0/1/2 to avoid clobbering files
    ++	 * in die().  It also avoids messing up when the pipes are dup'ed
    ++	 * onto stdin/stdout/stderr in the child processes we spawn.
    ++	 */
    ++	sanitize_stdfds();
    ++	restore_sigpipe_to_default();
    ++
    ++	git_resolve_executable_dir(argv[0]);
    ++
    ++	setlocale(LC_CTYPE, "");
    ++	git_setup_gettext();
    ++
    ++	initialize_repository(the_repository);
    ++
    ++	attr_start();
    ++
    ++	trace2_initialize();
    ++	trace2_cmd_start(argv);
    ++	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
    ++
    ++	if (!strbuf_getcwd(&tmp))
    ++		tmp_original_cwd = strbuf_detach(&tmp, NULL);
    ++}
    +
    + ## common-init.h (new) ##
    +@@
    ++#ifndef COMMON_INIT_H
    ++#define COMMON_INIT_H
    ++
    ++void init_git(const char **argv);
    ++
    ++#endif /* COMMON_INIT_H */
    +
      ## common-main.c ##
    -@@ common-main.c: int main(int argc, const char **argv)
    +@@
    +-#define USE_THE_REPOSITORY_VARIABLE
    +-
    + #include "git-compat-util.h"
    +-#include "exec-cmd.h"
    +-#include "gettext.h"
    +-#include "attr.h"
    +-#include "repository.h"
    +-#include "setup.h"
    +-#include "strbuf.h"
    +-#include "trace2.h"
    +-
    +-/*
    +- * Many parts of Git have subprograms communicate via pipe, expect the
    +- * upstream of a pipe to die with SIGPIPE when the downstream of a
    +- * pipe does not need to read all that is written.  Some third-party
    +- * programs that ignore or block SIGPIPE for their own reason forget
    +- * to restore SIGPIPE handling to the default before spawning Git and
    +- * break this carefully orchestrated machinery.
    +- *
    +- * Restore the way SIGPIPE is handled to default, which is what we
    +- * expect.
    +- */
    +-static void restore_sigpipe_to_default(void)
    +-{
    +-	sigset_t unblock;
    +-
    +-	sigemptyset(&unblock);
    +-	sigaddset(&unblock, SIGPIPE);
    +-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
    +-	signal(SIGPIPE, SIG_DFL);
    +-}
    ++#include "common-init.h"
    + 
    + int main(int argc, const char **argv)
    + {
    + 	int result;
    +-	struct strbuf tmp = STRBUF_INIT;
    +-
    +-	trace2_initialize_clock();
    +-
    +-	/*
    +-	 * Always open file descriptors 0/1/2 to avoid clobbering files
    +-	 * in die().  It also avoids messing up when the pipes are dup'ed
    +-	 * onto stdin/stdout/stderr in the child processes we spawn.
    +-	 */
    +-	sanitize_stdfds();
    +-	restore_sigpipe_to_default();
    +-
    +-	git_resolve_executable_dir(argv[0]);
    +-
    +-	setlocale(LC_CTYPE, "");
    +-	git_setup_gettext();
    +-
    +-	initialize_repository(the_repository);
    +-
    +-	attr_start();
    +-
    +-	trace2_initialize();
    +-	trace2_cmd_start(argv);
    +-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
    +-
    +-	if (!strbuf_getcwd(&tmp))
    +-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
    + 
    ++	init_git(argv);
    + 	result = cmd_main(argc, argv);
    + 
      	/* Not exit(3), but a wrapper calling our common_exit() */
      	exit(result);
      }
2:  5f2e816cf6 < -:  ---------- repository: add initialize_repo wrapper without pointer
3:  9a846c17c8 ! 2:  3589d2d6a2 contrib/cgit-rs: introduce Rust wrapper for libgit.a
    @@ Commit message
         Co-authored-by: Josh Steadmon <steadmon@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Kyle Lippincott <spectral@google.com>
    -    Signed-off-by: Josh Steadmon <steadmon@google.com>
    -
     
      ## .gitignore ##
    @@ .gitignore: Release/
      /git.VC.db
      *.dSYM
      /contrib/buildsystems/out
    -+/contrib/cgit-rs/target
    ++/contrib/cgit-rs/cgit-sys/target
     
      ## Makefile ##
     @@ Makefile: CURL_CONFIG = curl-config
    @@ Makefile: OBJECTS += $(XDIFF_OBJS)
      OBJECTS += $(FUZZ_OBJS)
      OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
      OBJECTS += $(UNIT_TEST_OBJS)
    -+OBJECTS += contrib/cgit-rs/public_symbol_export.o
    ++OBJECTS += contrib/cgit-rs/cgit-sys/public_symbol_export.o
      
      ifndef NO_CURL
      	OBJECTS += http.o http-walker.o remote-curl.o
    @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
    -+	$(RM) -r contrib/cgit-rs/target
    ++	$(RM) -r contrib/cgit-rs/cgit-sys/target
      ifndef NO_PERL
      	$(RM) -r perl/build/
      endif
    @@ Makefile: $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
      unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
      	$(MAKE) -C t/ unit-tests
     +
    -+contrib/cgit-rs/partial_symbol_export.o: contrib/cgit-rs/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    ++contrib/cgit-rs/cgit-sys/partial_symbol_export.o: contrib/cgit-rs/cgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
     +	$(LD) -r $^ -o $@
     +
    -+contrib/cgit-rs/hidden_symbol_export.o: contrib/cgit-rs/partial_symbol_export.o
    ++contrib/cgit-rs/cgit-sys/hidden_symbol_export.o: contrib/cgit-rs/cgit-sys/partial_symbol_export.o
     +	$(OBJCOPY) --localize-hidden $^ $@
     +
    -+contrib/cgit-rs/libcgit.a: contrib/cgit-rs/hidden_symbol_export.o
    ++contrib/cgit-rs/cgit-sys/libcgit.a: contrib/cgit-rs/cgit-sys/hidden_symbol_export.o
     +	$(AR) $(ARFLAGS) $@ $^
     
    - ## contrib/cgit-rs/Cargo.lock (new) ##
    + ## contrib/cgit-rs/cgit-sys/Cargo.lock (new) ##
     @@
     +# This file is automatically @generated by Cargo.
     +# It is not intended for manual editing.
     +version = 3
     +
     +[[package]]
    -+name = "cgit"
    ++name = "cgit-sys"
     +version = "0.1.0"
    -+dependencies = [
    -+ "libc",
    -+]
    -+
    -+[[package]]
    -+name = "libc"
    -+version = "0.2.155"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
     
    - ## contrib/cgit-rs/Cargo.toml (new) ##
    + ## contrib/cgit-rs/cgit-sys/Cargo.toml (new) ##
     @@
     +[package]
    -+name = "cgit"
    ++name = "cgit-sys"
     +version = "0.1.0"
     +edition = "2021"
     +build = "build.rs"
     +links = "git"
     +
    -+[[bin]]
    -+name = "cgit-test"
    -+path = "src/main.rs"
    -+
     +[lib]
     +path = "src/lib.rs"
    -+
    -+[dependencies]
    -+libc = "0.2.155"
     
    - ## contrib/cgit-rs/README.md (new) ##
    + ## contrib/cgit-rs/cgit-sys/README.md (new) ##
     @@
     +# cgit-info
     +
    @@ contrib/cgit-rs/README.md (new)
     +Assuming you don't make any changes to the Git source, you can just work from
     +`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
     
    - ## contrib/cgit-rs/build.rs (new) ##
    + ## contrib/cgit-rs/cgit-sys/build.rs (new) ##
     @@
     +use std::env;
     +use std::path::PathBuf;
     +
     +pub fn main() -> std::io::Result<()> {
     +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
    -+    let git_root = crate_root.join("../..");
    ++    let git_root = crate_root.join("../../..");
     +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
     +
     +    let make_output = std::process::Command::new("make")
     +        .env_remove("PROFILE")
     +        .current_dir(git_root.clone())
    -+        .args(&[
    -+            "CC=clang",
    ++        .args([
     +            "CFLAGS=-fvisibility=hidden",
    -+            "contrib/cgit-rs/libcgit.a"
    ++            "contrib/cgit-rs/cgit-sys/libcgit.a",
     +        ])
     +        .output()
     +        .expect("Make failed to run");
     +    if !make_output.status.success() {
     +        panic!(
    -+                "Make failed:\n  stdout = {}\n  stderr = {}\n",
    -+                String::from_utf8(make_output.stdout).unwrap(),
    -+                String::from_utf8(make_output.stderr).unwrap()
    ++            "Make failed:\n  stdout = {}\n  stderr = {}\n",
    ++            String::from_utf8(make_output.stdout).unwrap(),
    ++            String::from_utf8(make_output.stderr).unwrap()
     +        );
     +    }
     +    std::fs::copy(crate_root.join("libcgit.a"), dst.join("libcgit.a"))?;
    -+    println!("cargo::rustc-link-search=native={}", dst.into_os_string().into_string().unwrap());
    ++    println!("cargo::rustc-link-search=native={}", dst.display());
     +    println!("cargo::rustc-link-lib=cgit");
     +    println!("cargo::rustc-link-lib=z");
    -+    println!("cargo::rerun-if-changed={}", git_root.into_os_string().into_string().unwrap());
    ++    println!("cargo::rerun-if-changed={}", git_root.display());
     +
     +    Ok(())
     +}
     
    - ## contrib/cgit-rs/public_symbol_export.c (new) ##
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.c (new) ##
     @@
     +// Shim to publicly export Git symbols. These must be renamed so that the
     +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
     +// avoid conflicts with other libraries such as libgit2.
     +
    -+#include "contrib/cgit-rs/public_symbol_export.h"
    ++#include "contrib/cgit-rs/cgit-sys/public_symbol_export.h"
     +#include "version.h"
     +
     +#pragma GCC visibility push(default)
    @@ contrib/cgit-rs/public_symbol_export.c (new)
     +
     +#pragma GCC visibility pop
     
    - ## contrib/cgit-rs/public_symbol_export.h (new) ##
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.h (new) ##
     @@
     +#ifndef PUBLIC_SYMBOL_EXPORT_H
     +#define PUBLIC_SYMBOL_EXPORT_H
    @@ contrib/cgit-rs/public_symbol_export.h (new)
     +
     +#endif /* PUBLIC_SYMBOL_EXPORT_H */
     
    - ## contrib/cgit-rs/src/lib.rs (new) ##
    + ## contrib/cgit-rs/cgit-sys/src/lib.rs (new) ##
     @@
    -+use libc::c_char;
    ++use std::ffi::c_char;
     +
     +extern "C" {
     +    // From version.c
     +    pub fn libgit_user_agent() -> *const c_char;
     +    pub fn libgit_user_agent_sanitized() -> *const c_char;
     +}
    -
    - ## contrib/cgit-rs/src/main.rs (new) ##
    -@@
    -+use std::ffi::CStr;
    -+
    -+fn main() {
    -+    println!("Let's print some strings provided by Git");
    -+    let c_buf = unsafe { cgit::libgit_user_agent() };
    -+    let c_str = unsafe { CStr::from_ptr(c_buf) };
    -+    println!("git_user_agent() = {:?}", c_str);
    -+    println!("git_user_agent_sanitized() = {:?}",
    -+        unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
    ++
    ++#[cfg(test)]
    ++mod tests {
    ++    use std::ffi::CStr;
    ++
    ++    use super::*;
    ++
    ++    #[test]
    ++    fn user_agent_starts_with_git() {
    ++        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
    ++        let agent = c_str
    ++            .to_str()
    ++            .expect("User agent contains invalid UTF-8 data");
    ++        assert!(
    ++            agent.starts_with("git/"),
    ++            r#"Expected user agent to start with "git/", got: {}"#,
    ++            agent
    ++        );
    ++    }
    ++
    ++    #[test]
    ++    fn sanitized_user_agent_starts_with_git() {
    ++        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
    ++        let agent = c_str
    ++            .to_str()
    ++            .expect("Sanitized user agent contains invalid UTF-8 data");
    ++        assert!(
    ++            agent.starts_with("git/"),
    ++            r#"Expected user agent to start with "git/", got: {}"#,
    ++            agent
    ++        );
    ++    }
     +}
4:  b84a8210a0 ! 3:  527780f816 contrib/cgit-rs: add repo initialization and config access
    @@ Commit message
         contrib/cgit-rs: add repo initialization and config access
     
         Co-authored-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
    -    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
    - ## contrib/cgit-rs/public_symbol_export.c ##
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.c ##
     @@
      // original symbols can be hidden. Renaming these with a "libgit_" prefix also
      // avoid conflicts with other libraries such as libgit2.
      
     +#include "git-compat-util.h"
    - #include "contrib/cgit-rs/public_symbol_export.h"
    -+#include "attr.h"
    + #include "contrib/cgit-rs/cgit-sys/public_symbol_export.h"
    ++#include "common-init.h"
     +#include "config.h"
     +#include "setup.h"
      #include "version.h"
      
    ++extern struct repository *the_repository;
    ++
      #pragma GCC visibility push(default)
      
     +const char *libgit_setup_git_directory(void)
    @@ contrib/cgit-rs/public_symbol_export.c
     +
     +int libgit_config_get_int(const char *key, int *dest)
     +{
    -+	return git_config_get_int(key, dest);
    ++	return repo_config_get_int(the_repository, key, dest);
     +}
     +
    -+void libgit_initialize_the_repository(void)
    ++void libgit_init_git(const char **argv)
     +{
    -+	initialize_the_repository();
    ++	init_git(argv);
     +}
     +
     +int libgit_parse_maybe_bool(const char *val)
    @@ contrib/cgit-rs/public_symbol_export.c
      const char *libgit_user_agent(void)
      {
      	return git_user_agent();
    -@@ contrib/cgit-rs/public_symbol_export.c: const char *libgit_user_agent_sanitized(void)
    - 	return git_user_agent_sanitized();
    - }
    - 
    -+const char *libgit_attr__true = git_attr__true;
    -+const char *libgit_attr__false = git_attr__false;
    -+
    - #pragma GCC visibility pop
     
    - ## contrib/cgit-rs/public_symbol_export.h ##
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.h ##
     @@
      #ifndef PUBLIC_SYMBOL_EXPORT_H
      #define PUBLIC_SYMBOL_EXPORT_H
    @@ contrib/cgit-rs/public_symbol_export.h
     +
     +int libgit_config_get_int(const char *key, int *dest);
     +
    -+void libgit_initialize_the_repository(void);
    ++void libgit_init_git(const char **argv);
     +
     +int libgit_parse_maybe_bool(const char *val);
     +
    @@ contrib/cgit-rs/public_symbol_export.h
      
      const char *libgit_user_agent_sanitized(void);
     
    - ## contrib/cgit-rs/src/lib.rs ##
    + ## contrib/cgit-rs/cgit-sys/src/lib.rs ##
     @@
    --use libc::c_char;
    -+use libc::{c_char, c_int};
    +-use std::ffi::c_char;
    ++use std::ffi::{c_char, c_int};
      
      extern "C" {
     +    pub fn libgit_setup_git_directory() -> *const c_char;
     +
     +    // From config.c
    -+    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) ->c_int;
    ++    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) -> c_int;
     +
    -+    // From repository.c
    -+    pub fn libgit_initialize_the_repository();
    ++    // From common-init.c
    ++    pub fn libgit_init_git(argv: *const *const c_char);
     +
     +    // From parse.c
     +    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
    @@ contrib/cgit-rs/src/lib.rs
          // From version.c
          pub fn libgit_user_agent() -> *const c_char;
          pub fn libgit_user_agent_sanitized() -> *const c_char;
    -
    - ## contrib/cgit-rs/src/main.rs ##
    -@@
    --use std::ffi::CStr;
    -+use std::ffi::{CStr, CString};
    +@@ contrib/cgit-rs/cgit-sys/src/lib.rs: extern "C" {
      
    - fn main() {
    -     println!("Let's print some strings provided by Git");
    -@@ contrib/cgit-rs/src/main.rs: fn main() {
    -     println!("git_user_agent() = {:?}", c_str);
    -     println!("git_user_agent_sanitized() = {:?}",
    -         unsafe { CStr::from_ptr(cgit::libgit_user_agent_sanitized()) });
    -+
    -+    println!("\nNow try passing args");
    -+    let test_arg = CString::new("test_arg").unwrap();
    -+    println!("git_parse_maybe_bool(...) = {:?}",
    -+        unsafe { cgit::libgit_parse_maybe_bool(test_arg.as_ptr()) });
    -+
    -+    println!("\nCan we get an int out of our config??");
    -+    unsafe {
    -+        cgit::libgit_initialize_the_repository();
    -+        cgit::libgit_setup_git_directory();
    -+        let mut val: libc::c_int = 0;
    + #[cfg(test)]
    + mod tests {
    +-    use std::ffi::CStr;
    ++    use std::ffi::{CStr, CString};
    + 
    +     use super::*;
    + 
    +@@ contrib/cgit-rs/cgit-sys/src/lib.rs: mod tests {
    +             agent
    +         );
    +     }
    ++
    ++    #[test]
    ++    fn parse_bools_from_strings() {
    ++        let arg = CString::new("true").unwrap();
    ++        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
    ++
    ++        let arg = CString::new("yes").unwrap();
    ++        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
    ++
    ++        let arg = CString::new("false").unwrap();
    ++        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
    ++
    ++        let arg = CString::new("no").unwrap();
    ++        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
    ++
    ++        let arg = CString::new("maybe").unwrap();
    ++        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, -1);
    ++    }
    ++
    ++    #[test]
    ++    fn access_configs() {
    ++        // NEEDSWORK: we need to supply a testdata config
    ++        let fake_argv = [std::ptr::null::<c_char>()];
    ++        unsafe {
    ++            libgit_init_git(fake_argv.as_ptr());
    ++            libgit_setup_git_directory();
    ++        }
    ++        let mut val: c_int = 0;
     +        let key = CString::new("trace2.eventNesting").unwrap();
    -+        cgit::libgit_config_get_int(
    -+            key.as_ptr(),
    -+            &mut val as *mut i32
    -+        );
    -+        println!(
    -+            "git_config_get_int(\"trace2.eventNesting\") -> {:?}",
    -+            val
    -+        );
    -+    };
    ++        unsafe { libgit_config_get_int(key.as_ptr(), &mut val as *mut i32) };
    ++        assert_eq!(val, 5);
    ++    }
      }
5:  c8befb680e ! 4:  908ad0b82f config: add git_configset_alloc
    @@ Metadata
     Author: Josh Steadmon <steadmon@google.com>
     
      ## Commit message ##
    -    config: add git_configset_alloc
    +    config: add git_configset_alloc() and git_configset_clear_and_free()
     
    -    Add git_configset_alloc so that non-C external consumers can use
    -    configset functions without redefining config_set.
    +    Add git_configset_alloc() so that and git_configset_clear_and_free()
    +    functions so that callers can manage config_set structs on the heap.
    +    This also allows non-C external consumers to treat config_sets as opaque
    +    structs.
     
         Co-authored-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
    -    Signed-off-by: Josh Steadmon <steadmon@google.com>
    -
     
    +    Also add _clear_and_free
    +
    +
      ## config.c ##
     @@ config.c: static int config_set_element_cmp(const void *cmp_data UNUSED,
      	return strcmp(e1->key, e2->key);
    @@ config.c: static int config_set_element_cmp(const void *cmp_data UNUSED,
      void git_configset_init(struct config_set *set)
      {
      	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
    +@@ config.c: void git_configset_clear(struct config_set *set)
    + 	set->list.items = NULL;
    + }
    + 
    ++void git_configset_clear_and_free(struct config_set *set)
    ++{
    ++	git_configset_clear(set);
    ++	free(set);
    ++}
    ++
    + static int config_set_callback(const char *key, const char *value,
    + 			       const struct config_context *ctx,
    + 			       void *cb)
     
      ## config.h ##
     @@ config.h: struct config_set {
    @@ config.h: struct config_set {
      /**
       * Initializes the config_set `cs`.
       */
    +@@ config.h: int git_configset_get_string_multi(struct config_set *cs, const char *key,
    +  */
    + void git_configset_clear(struct config_set *cs);
    + 
    ++/**
    ++ * Clears and frees a heap-allocated `config_set` structure.
    ++ */
    ++void git_configset_clear_and_free(struct config_set *cs);
    ++
    + /*
    +  * These functions return 1 if not found, and 0 if found, leaving the found
    +  * value in the 'dest' pointer.
6:  1e981a6880 ! 5:  514c744ba4 contrib/cgit-rs: add a subset of configset wrappers
    @@ Metadata
      ## Commit message ##
         contrib/cgit-rs: add a subset of configset wrappers
     
         Signed-off-by: Calvin Wan <calvinwan@google.com>
    -    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
    - ## contrib/cgit-rs/Cargo.lock ##
    -@@ contrib/cgit-rs/Cargo.lock: version = 3
    - name = "cgit"
    - version = "0.1.0"
    - dependencies = [
    -+ "home",
    -  "libc",
    - ]
    - 
    -+[[package]]
    -+name = "home"
    -+version = "0.5.9"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "e3d1354bf6b7235cb4a0576c2619fd4ed18183f689b12b006a0ee7329eeff9a5"
    -+dependencies = [
    -+ "windows-sys",
    -+]
    -+
    - [[package]]
    - name = "libc"
    - version = "0.2.155"
    - source = "registry+https://github.com/rust-lang/crates.io-index"
    - checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
    -+
    -+[[package]]
    -+name = "windows-sys"
    -+version = "0.52.0"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
    -+dependencies = [
    -+ "windows-targets",
    -+]
    + ## .gitignore ##
    +@@ .gitignore: Release/
    + /git.VC.db
    + *.dSYM
    + /contrib/buildsystems/out
    ++/contrib/cgit-rs/target
    + /contrib/cgit-rs/cgit-sys/target
    +
    + ## Makefile ##
    +@@ Makefile: clean: profile-clean coverage-clean cocciclean
    + 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
    + 	$(MAKE) -C Documentation/ clean
    + 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
    +-	$(RM) -r contrib/cgit-rs/cgit-sys/target
    ++	$(RM) -r contrib/cgit-rs/target contrib/cgit-rs/cgit-sys/target
    + ifndef NO_PERL
    + 	$(RM) -r perl/build/
    + endif
    +
    + ## contrib/cgit-rs/Cargo.lock (new) ##
    +@@
    ++# This file is automatically @generated by Cargo.
    ++# It is not intended for manual editing.
    ++version = 3
     +
     +[[package]]
    -+name = "windows-targets"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
    ++name = "cgit"
    ++version = "0.1.0"
     +dependencies = [
    -+ "windows_aarch64_gnullvm",
    -+ "windows_aarch64_msvc",
    -+ "windows_i686_gnu",
    -+ "windows_i686_gnullvm",
    -+ "windows_i686_msvc",
    -+ "windows_x86_64_gnu",
    -+ "windows_x86_64_gnullvm",
    -+ "windows_x86_64_msvc",
    ++ "cgit-sys",
     +]
     +
     +[[package]]
    -+name = "windows_aarch64_gnullvm"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
    -+
    -+[[package]]
    -+name = "windows_aarch64_msvc"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
    -+
    -+[[package]]
    -+name = "windows_i686_gnu"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
    -+
    -+[[package]]
    -+name = "windows_i686_gnullvm"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
    -+
    -+[[package]]
    -+name = "windows_i686_msvc"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
    -+
    -+[[package]]
    -+name = "windows_x86_64_gnu"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
    ++name = "cgit-sys"
    ++version = "0.1.0"
    +
    + ## contrib/cgit-rs/Cargo.toml (new) ##
    +@@
    ++[package]
    ++name = "cgit"
    ++version = "0.1.0"
    ++edition = "2021"
     +
    -+[[package]]
    -+name = "windows_x86_64_gnullvm"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
    ++[lib]
    ++path = "src/lib.rs"
     +
    -+[[package]]
    -+name = "windows_x86_64_msvc"
    -+version = "0.52.6"
    -+source = "registry+https://github.com/rust-lang/crates.io-index"
    -+checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
    ++[dependencies]
    ++cgit-sys = { version = "0.1.0", path = "cgit-sys" }
     
    - ## contrib/cgit-rs/Cargo.toml ##
    -@@ contrib/cgit-rs/Cargo.toml: path = "src/lib.rs"
    - 
    - [dependencies]
    - libc = "0.2.155"
    -+home = "0.5.9"
    -
    - ## contrib/cgit-rs/public_symbol_export.c ##
    -@@ contrib/cgit-rs/public_symbol_export.c: int libgit_parse_maybe_bool(const char *val)
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.c ##
    +@@ contrib/cgit-rs/cgit-sys/public_symbol_export.c: int libgit_parse_maybe_bool(const char *val)
      	return git_parse_maybe_bool(val);
      }
      
    @@ contrib/cgit-rs/public_symbol_export.c: int libgit_parse_maybe_bool(const char *
     +	return git_configset_alloc();
     +}
     +
    ++void libgit_configset_clear_and_free(struct config_set *cs)
    ++{
    ++	git_configset_clear_and_free(cs);
    ++}
    ++
     +void libgit_configset_init(struct config_set *cs)
     +{
     +	git_configset_init(cs);
    @@ contrib/cgit-rs/public_symbol_export.c: int libgit_parse_maybe_bool(const char *
      {
      	return git_user_agent();
     
    - ## contrib/cgit-rs/public_symbol_export.h ##
    -@@ contrib/cgit-rs/public_symbol_export.h: void libgit_initialize_the_repository(void);
    + ## contrib/cgit-rs/cgit-sys/public_symbol_export.h ##
    +@@ contrib/cgit-rs/cgit-sys/public_symbol_export.h: void libgit_init_git(const char **argv);
      
      int libgit_parse_maybe_bool(const char *val);
      
     +struct config_set *libgit_configset_alloc(void);
     +
    ++void libgit_configset_clear_and_free(struct config_set *cs);
    ++
     +void libgit_configset_init(struct config_set *cs);
     +
     +int libgit_configset_add_file(struct config_set *cs, const char *filename);
    @@ contrib/cgit-rs/public_symbol_export.h: void libgit_initialize_the_repository(vo
      
      const char *libgit_user_agent_sanitized(void);
     
    - ## contrib/cgit-rs/src/lib.rs ##
    + ## contrib/cgit-rs/cgit-sys/src/lib.rs ##
     @@
    --use libc::{c_char, c_int};
    -+use std::ffi::{CStr, CString};
    +-use std::ffi::{c_char, c_int};
    ++use std::ffi::{c_char, c_int, c_void};
    ++
    ++#[allow(non_camel_case_types)]
    ++#[repr(C)]
    ++pub struct config_set {
    ++    _data: [u8; 0],
    ++    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
    ++}
    + 
    + extern "C" {
    ++    pub fn free(ptr: *mut c_void);
     +
    -+use libc::{c_char, c_int, c_void};
    +     pub fn libgit_setup_git_directory() -> *const c_char;
    + 
    +     // From config.c
    +@@ contrib/cgit-rs/cgit-sys/src/lib.rs: extern "C" {
    +     // From version.c
    +     pub fn libgit_user_agent() -> *const c_char;
    +     pub fn libgit_user_agent_sanitized() -> *const c_char;
     +
    -+pub enum GitConfigSet {}
    ++    pub fn libgit_configset_alloc() -> *mut config_set;
    ++    pub fn libgit_configset_clear_and_free(cs: *mut config_set);
     +
    -+pub struct ConfigSet(*mut GitConfigSet);
    -+impl ConfigSet {
    ++    pub fn libgit_configset_init(cs: *mut config_set);
    ++
    ++    pub fn libgit_configset_add_file(cs: *mut config_set, filename: *const c_char) -> c_int;
    ++
    ++    pub fn libgit_configset_get_int(
    ++        cs: *mut config_set,
    ++        key: *const c_char,
    ++        int: *mut c_int,
    ++    ) -> c_int;
     +
    ++    pub fn libgit_configset_get_string(
    ++        cs: *mut config_set,
    ++        key: *const c_char,
    ++        dest: *mut *mut c_char,
    ++    ) -> c_int;
    ++
    + }
    + 
    + #[cfg(test)]
    +
    + ## contrib/cgit-rs/src/lib.rs (new) ##
    +@@
    ++use std::ffi::{c_char, c_int, c_void, CStr, CString};
    ++
    ++use cgit_sys::*;
    ++
    ++pub struct ConfigSet(*mut config_set);
    ++impl ConfigSet {
     +    pub fn new() -> Self {
     +        unsafe {
    -+            // TODO: we need to handle freeing this when the ConfigSet is dropped
    -+            // git_configset_clear(ptr) and then libc::free(ptr)
     +            let ptr = libgit_configset_alloc();
     +            libgit_configset_init(ptr);
     +            ConfigSet(ptr)
     +        }
     +    }
     +
    ++    // NEEDSWORK: maybe replace &str with &Path
     +    pub fn add_files(&mut self, files: &[&str]) {
     +        for file in files {
     +            let rs = CString::new(*file).expect("Couldn't convert to CString");
    @@ contrib/cgit-rs/src/lib.rs
     +        let key = CString::new(key).expect("Couldn't convert to CString");
     +        let mut val: *mut c_char = std::ptr::null_mut();
     +        unsafe {
    -+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0 {
    ++            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
    ++            {
     +                return None;
     +            }
     +            let borrowed_str = CStr::from_ptr(val);
     +            let owned_str = CString::from_vec_with_nul(borrowed_str.to_bytes_with_nul().to_vec());
    -+            libc::free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
    ++            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
     +            Some(owned_str.unwrap())
     +        }
     +    }
     +}
    - 
    - extern "C" {
    -     pub fn libgit_setup_git_directory() -> *const c_char;
    -@@ contrib/cgit-rs/src/lib.rs: extern "C" {
    -     // From version.c
    -     pub fn libgit_user_agent() -> *const c_char;
    -     pub fn libgit_user_agent_sanitized() -> *const c_char;
    -+
    -+    fn libgit_configset_alloc() -> *mut GitConfigSet;
    -+
    -+    fn libgit_configset_init(cs: *mut GitConfigSet);
     +
    -+    fn libgit_configset_add_file(cs: *mut GitConfigSet, filename: *const c_char) -> c_int;
    ++impl Default for ConfigSet {
    ++    fn default() -> Self {
    ++        Self::new()
    ++    }
    ++}
     +
    -+    pub fn libgit_configset_get_int(cs: *mut GitConfigSet, key: *const c_char, int: *mut c_int) -> c_int;
    -+    pub fn libgit_configset_get_string(cs: *mut GitConfigSet, key: *const c_char, dest: *mut *mut c_char) -> c_int;
    ++impl Drop for ConfigSet {
    ++    fn drop(&mut self) {
    ++        unsafe {
    ++            libgit_configset_clear_and_free(self.0);
    ++        }
    ++    }
    ++}
     +
    - }
    -
    - ## contrib/cgit-rs/src/main.rs ##
    -@@
    - use std::ffi::{CStr, CString};
    -+use home::home_dir;
    - 
    - fn main() {
    -     println!("Let's print some strings provided by Git");
    -@@ contrib/cgit-rs/src/main.rs: fn main() {
    -             val
    -         );
    -     };
    -+
    -+    println!("\nTry out our configset wrappers");
    -+    let mut cs = cgit::ConfigSet::new();
    -+    let mut path = home_dir().expect("cannot get home directory path");
    -+    path.push(".gitconfig");
    -+    let path:String = path.into_os_string().into_string().unwrap();
    -+    cs.add_files(&["/etc/gitconfig", ".gitconfig", &path]);
    -+    /*
    -+     * Returns Some(x) if defined in local config, otherwise None
    -+     */
    -+    println!("get_configset_get_int = {:?}", cs.get_int("trace2.eventNesting")); 
    -+    println!("cs.get_str(\"garbage\") = {:?}", cs.get_str("this_string_does_not_exist"));
    - }
    ++#[cfg(test)]
    ++mod tests {
    ++    use super::*;
    ++
    ++    #[test]
    ++    fn load_configs_via_configset() {
    ++        // NEEDSWORK: we need to supply a testdata config
    ++        let mut cs = ConfigSet::new();
    ++        let mut path = std::env::home_dir().expect("cannot get home directory path");
    ++        path.push(".gitconfig");
    ++        let path: String = path.into_os_string().into_string().unwrap();
    ++        cs.add_files(&["/etc/gitconfig", ".gitconfig", &path]);
    ++        assert_eq!(cs.get_int("trace2.eventNesting"), Some(5));
    ++        assert_eq!(cs.get_str("no_such_config_item"), None);
    ++    }
    ++}

base-commit: 557ae147e6cdc9db121269b058c757ac5092f9c9
-- 
2.46.0.76.ge559c4bf1a-goog

[PATCH v2 1/5] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 5 files changed, 99 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 3eab701b10..7caeb3c872 100644
--- a/Makefile
+++ b/Makefile
@@ -979,6 +979,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.46.0.76.ge559c4bf1a-goog

[PATCH v2 2/5] cgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce cgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library providing a more Rust-friendly interface will be
added at `contrib/cgit-rs`.

Symbols in cgit-sys can collide with symbols from other libraries such
as libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/cgit-rs/cgit-sys/libcgit.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                    |  1 +
 Makefile                                      | 13 ++++++
 contrib/cgit-rs/cgit-sys/Cargo.lock           |  7 ++++
 contrib/cgit-rs/cgit-sys/Cargo.toml           |  9 +++++
 contrib/cgit-rs/cgit-sys/README.md            | 15 +++++++
 contrib/cgit-rs/cgit-sys/build.rs             | 32 +++++++++++++++
 .../cgit-rs/cgit-sys/public_symbol_export.c   | 20 ++++++++++
 .../cgit-rs/cgit-sys/public_symbol_export.h   |  8 ++++
 contrib/cgit-rs/cgit-sys/src/lib.rs           | 40 +++++++++++++++++++
 9 files changed, 145 insertions(+)
 create mode 100644 contrib/cgit-rs/cgit-sys/Cargo.lock
 create mode 100644 contrib/cgit-rs/cgit-sys/Cargo.toml
 create mode 100644 contrib/cgit-rs/cgit-sys/README.md
 create mode 100644 contrib/cgit-rs/cgit-sys/build.rs
 create mode 100644 contrib/cgit-rs/cgit-sys/public_symbol_export.c
 create mode 100644 contrib/cgit-rs/cgit-sys/public_symbol_export.h
 create mode 100644 contrib/cgit-rs/cgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index 8caf3700c2..567cc9888f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,3 +248,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/cgit-rs/cgit-sys/target
diff --git a/Makefile b/Makefile
index 7caeb3c872..db8af99f20 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2713,6 +2715,7 @@ OBJECTS += $(XDIFF_OBJS)
 OBJECTS += $(FUZZ_OBJS)
 OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
+OBJECTS += contrib/cgit-rs/cgit-sys/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3720,6 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/cgit-rs/cgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3865,3 +3869,12 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
 build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/cgit-rs/cgit-sys/partial_symbol_export.o: contrib/cgit-rs/cgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/cgit-rs/cgit-sys/hidden_symbol_export.o: contrib/cgit-rs/cgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/cgit-rs/cgit-sys/libcgit.a: contrib/cgit-rs/cgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/cgit-rs/cgit-sys/Cargo.lock b/contrib/cgit-rs/cgit-sys/Cargo.lock
new file mode 100644
index 0000000000..419a6d42f2
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/Cargo.lock
@@ -0,0 +1,7 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "cgit-sys"
+version = "0.1.0"
diff --git a/contrib/cgit-rs/cgit-sys/Cargo.toml b/contrib/cgit-rs/cgit-sys/Cargo.toml
new file mode 100644
index 0000000000..e840f04024
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "cgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "git"
+
+[lib]
+path = "src/lib.rs"
diff --git a/contrib/cgit-rs/cgit-sys/README.md b/contrib/cgit-rs/cgit-sys/README.md
new file mode 100644
index 0000000000..7a59602c30
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/README.md
@@ -0,0 +1,15 @@
+# cgit-info
+
+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
+library.
+
+## Building
+
+`cargo build` automatically builds and picks up on changes made to both
+the Rust wrapper and git.git code so there is no need to run `make`
+beforehand.
+
+## Running
+
+Assuming you don't make any changes to the Git source, you can just work from
+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
diff --git a/contrib/cgit-rs/cgit-sys/build.rs b/contrib/cgit-rs/cgit-sys/build.rs
new file mode 100644
index 0000000000..e29d703870
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/build.rs
@@ -0,0 +1,32 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = std::process::Command::new("make")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/cgit-rs/cgit-sys/libcgit.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libcgit.a"), dst.join("libcgit.a"))?;
+    println!("cargo::rustc-link-search=native={}", dst.display());
+    println!("cargo::rustc-link-lib=cgit");
+    println!("cargo::rustc-link-lib=z");
+    println!("cargo::rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.c b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..deef4488c0
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
@@ -0,0 +1,20 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoid conflicts with other libraries such as libgit2.
+
+#include "contrib/cgit-rs/cgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/cgit-rs/cgit-sys/src/lib.rs b/contrib/cgit-rs/cgit-sys/src/lib.rs
new file mode 100644
index 0000000000..bc33c6c498
--- /dev/null
+++ b/contrib/cgit-rs/cgit-sys/src/lib.rs
@@ -0,0 +1,40 @@
+use std::ffi::c_char;
+
+extern "C" {
+    // From version.c
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
-- 
2.46.0.76.ge559c4bf1a-goog

[PATCH v2 3/5] cgit-sys: add repo initialization and config access

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Wrap a few repo setup and config access functions in cgit-sys. These
were selected as proof-of-concept items to show that we can access local
config from Rust.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .../cgit-rs/cgit-sys/public_symbol_export.c   | 26 ++++++++++
 .../cgit-rs/cgit-sys/public_symbol_export.h   |  8 ++++
 contrib/cgit-rs/cgit-sys/src/lib.rs           | 47 ++++++++++++++++++-
 3 files changed, 79 insertions(+), 2 deletions(-)

diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.c b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
index deef4488c0..62a91f76d0 100644
--- a/contrib/cgit-rs/cgit-sys/public_symbol_export.c
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
@@ -2,11 +2,37 @@
 // original symbols can be hidden. Renaming these with a "libgit_" prefix also
 // avoid conflicts with other libraries such as libgit2.
 
+#include "git-compat-util.h"
 #include "contrib/cgit-rs/cgit-sys/public_symbol_export.h"
+#include "common-init.h"
+#include "config.h"
+#include "setup.h"
 #include "version.h"
 
+extern struct repository *the_repository;
+
 #pragma GCC visibility push(default)
 
+const char *libgit_setup_git_directory(void)
+{
+	return setup_git_directory();
+}
+
+int libgit_config_get_int(const char *key, int *dest)
+{
+	return repo_config_get_int(the_repository, key, dest);
+}
+
+void libgit_init_git(const char **argv)
+{
+	init_git(argv);
+}
+
+int libgit_parse_maybe_bool(const char *val)
+{
+	return git_parse_maybe_bool(val);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
index a3372f93fa..64332f30de 100644
--- a/contrib/cgit-rs/cgit-sys/public_symbol_export.h
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
@@ -1,6 +1,14 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+const char *libgit_setup_git_directory(void);
+
+int libgit_config_get_int(const char *key, int *dest);
+
+void libgit_init_git(const char **argv);
+
+int libgit_parse_maybe_bool(const char *val);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/cgit-rs/cgit-sys/src/lib.rs b/contrib/cgit-rs/cgit-sys/src/lib.rs
index bc33c6c498..8c3ccc2859 100644
--- a/contrib/cgit-rs/cgit-sys/src/lib.rs
+++ b/contrib/cgit-rs/cgit-sys/src/lib.rs
@@ -1,6 +1,17 @@
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 extern "C" {
+    pub fn libgit_setup_git_directory() -> *const c_char;
+
+    // From config.c
+    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) -> c_int;
+
+    // From common-init.c
+    pub fn libgit_init_git(argv: *const *const c_char);
+
+    // From parse.c
+    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
+
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
@@ -8,7 +19,7 @@ extern "C" {
 
 #[cfg(test)]
 mod tests {
-    use std::ffi::CStr;
+    use std::ffi::{CStr, CString};
 
     use super::*;
 
@@ -37,4 +48,36 @@ mod tests {
             agent
         );
     }
+
+    #[test]
+    fn parse_bools_from_strings() {
+        let arg = CString::new("true").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
+
+        let arg = CString::new("yes").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
+
+        let arg = CString::new("false").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
+
+        let arg = CString::new("no").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
+
+        let arg = CString::new("maybe").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, -1);
+    }
+
+    #[test]
+    fn access_configs() {
+        // NEEDSWORK: we need to supply a testdata config
+        let fake_argv = [std::ptr::null::<c_char>()];
+        unsafe {
+            libgit_init_git(fake_argv.as_ptr());
+            libgit_setup_git_directory();
+        }
+        let mut val: c_int = 0;
+        let key = CString::new("trace2.eventNesting").unwrap();
+        unsafe { libgit_config_get_int(key.as_ptr(), &mut val as *mut i32) };
+        assert_eq!(val, 5);
+    }
 }
-- 
2.46.0.76.ge559c4bf1a-goog

[PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Josh Steadmon]

Add git_configset_alloc() and git_configset_clear_and_free() functions
so that callers can manage config_set structs on the heap. This also
allows non-C external consumers to treat config_sets as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 config.c | 11 +++++++++++
 config.h | 10 ++++++++++
 2 files changed, 21 insertions(+)

diff --git a/config.c b/config.c
index 6421894614..444db8de79 100644
--- a/config.c
+++ b/config.c
@@ -2324,6 +2324,11 @@ static int config_set_element_cmp(const void *cmp_data UNUSED,
 	return strcmp(e1->key, e2->key);
 }
 
+struct config_set *git_configset_alloc(void)
+{
+	return xmalloc(sizeof(struct config_set));
+}
+
 void git_configset_init(struct config_set *set)
 {
 	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
@@ -2353,6 +2358,12 @@ void git_configset_clear(struct config_set *set)
 	set->list.items = NULL;
 }
 
+void git_configset_clear_and_free(struct config_set *set)
+{
+	git_configset_clear(set);
+	free(set);
+}
+
 static int config_set_callback(const char *key, const char *value,
 			       const struct config_context *ctx,
 			       void *cb)
diff --git a/config.h b/config.h
index 54b47dec9e..074c85a788 100644
--- a/config.h
+++ b/config.h
@@ -472,6 +472,11 @@ struct config_set {
 	struct configset_list list;
 };
 
+/**
+ * Alloc a config_set
+ */
+struct config_set *git_configset_alloc(void);
+
 /**
  * Initializes the config_set `cs`.
  */
@@ -520,6 +525,11 @@ int git_configset_get_string_multi(struct config_set *cs, const char *key,
  */
 void git_configset_clear(struct config_set *cs);
 
+/**
+ * Clears and frees a heap-allocated `config_set` structure.
+ */
+void git_configset_clear_and_free(struct config_set *cs);
+
 /*
  * These functions return 1 if not found, and 0 if found, leaving the found
  * value in the 'dest' pointer.
-- 
2.46.0.76.ge559c4bf1a-goog

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Phillip Wood]

Hi Josh

On 09/08/2024 23:41, Josh Steadmon wrote:
> Add git_configset_alloc() and git_configset_clear_and_free() functions
> so that callers can manage config_set structs on the heap. This also
> allows non-C external consumers to treat config_sets as opaque structs.

Do we really need to add this code to config.c rather than handling it 
in the wrapper layer in the next patch?

Looking ahead I wonder how useful it is to users of the library to 
separate out allocation from initialization. A function that allocates 
and initializes a configset would be more convenient and harder to 
misuse. Calling release functions *_free() rather than 
*_clear_and_free() would be more convenient as well. I also noticed that 
the data types are not namespaced when they are exported. So perhaps we 
could drop this patch and add the following to the next patch.

/*
  * Namespace data types as well as functions and ensure consistent
  * naming of data types and the functions that operate on them.
  */
struct libgit_configset {
	struct config_set set;
};

struct libgit_configset *libgit_configset_new() {
	struct libgit_configset *set = xmalloc(sizeof(*set));

	git_configset_init(&set->set);
	return set;
}

void libgit_configset_free(struct libgit_configset *set) {
	git_configset_clear(&set->set);
	free(set);
}

Best Wishes

Phillip

> Co-authored-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
>   config.c | 11 +++++++++++
>   config.h | 10 ++++++++++
>   2 files changed, 21 insertions(+)
> 
> diff --git a/config.c b/config.c
> index 6421894614..444db8de79 100644
> --- a/config.c
> +++ b/config.c
> @@ -2324,6 +2324,11 @@ static int config_set_element_cmp(const void *cmp_data UNUSED,
>   	return strcmp(e1->key, e2->key);
>   }
>   
> +struct config_set *git_configset_alloc(void)
> +{
> +	return xmalloc(sizeof(struct config_set));
> +}
> +
>   void git_configset_init(struct config_set *set)
>   {
>   	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
> @@ -2353,6 +2358,12 @@ void git_configset_clear(struct config_set *set)
>   	set->list.items = NULL;
>   }
>   
> +void git_configset_clear_and_free(struct config_set *set)
> +{
> +	git_configset_clear(set);
> +	free(set);
> +}
> +
>   static int config_set_callback(const char *key, const char *value,
>   			       const struct config_context *ctx,
>   			       void *cb)
> diff --git a/config.h b/config.h
> index 54b47dec9e..074c85a788 100644
> --- a/config.h
> +++ b/config.h
> @@ -472,6 +472,11 @@ struct config_set {
>   	struct configset_list list;
>   };
>   
> +/**
> + * Alloc a config_set
> + */
> +struct config_set *git_configset_alloc(void);
> +
>   /**
>    * Initializes the config_set `cs`.
>    */
> @@ -520,6 +525,11 @@ int git_configset_get_string_multi(struct config_set *cs, const char *key,
>    */
>   void git_configset_clear(struct config_set *cs);
>   
> +/**
> + * Clears and frees a heap-allocated `config_set` structure.
> + */
> +void git_configset_clear_and_free(struct config_set *cs);
> +
>   /*
>    * These functions return 1 if not found, and 0 if found, leaving the found
>    * value in the 'dest' pointer.

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Josh Steadmon]

On 2024.08.12 10:10, Phillip Wood wrote:
> Hi Josh
> 
> On 09/08/2024 23:41, Josh Steadmon wrote:
> > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > so that callers can manage config_set structs on the heap. This also
> > allows non-C external consumers to treat config_sets as opaque structs.
> 
> Do we really need to add this code to config.c rather than handling it in
> the wrapper layer in the next patch?
> 
> Looking ahead I wonder how useful it is to users of the library to separate
> out allocation from initialization. A function that allocates and
> initializes a configset would be more convenient and harder to misuse.
> Calling release functions *_free() rather than *_clear_and_free() would be
> more convenient as well. I also noticed that the data types are not
> namespaced when they are exported. So perhaps we could drop this patch and
> add the following to the next patch.
> 
> /*
>  * Namespace data types as well as functions and ensure consistent
>  * naming of data types and the functions that operate on them.
>  */
> struct libgit_configset {
> 	struct config_set set;
> };
> 
> struct libgit_configset *libgit_configset_new() {
> 	struct libgit_configset *set = xmalloc(sizeof(*set));
> 
> 	git_configset_init(&set->set);
> 	return set;
> }
> 
> void libgit_configset_free(struct libgit_configset *set) {
> 	git_configset_clear(&set->set);
> 	free(set);
> }
> 
> Best Wishes
> 
> Phillip

Hmmm I see your point, but I am also hoping to keep the symbol export
shim as small as possible, so that we can try to autogenerate it rather
than add entries by hand. However, if people feel strongly that we don't
want to add helper functions like *_alloc() or *_free() for types that don't
already have them upstream, perhaps we can just put them in a separate
rust-helpers.c file or something.

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Kyle Lippincott]

On Mon, Aug 12, 2024 at 2:39 PM Josh Steadmon <steadmon@google.com> wrote:
>
> On 2024.08.12 10:10, Phillip Wood wrote:
> > Hi Josh
> >
> > On 09/08/2024 23:41, Josh Steadmon wrote:
> > > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > > so that callers can manage config_set structs on the heap. This also
> > > allows non-C external consumers to treat config_sets as opaque structs.
> >
> > Do we really need to add this code to config.c rather than handling it in
> > the wrapper layer in the next patch?
> >
> > Looking ahead I wonder how useful it is to users of the library to separate
> > out allocation from initialization. A function that allocates and
> > initializes a configset would be more convenient and harder to misuse.
> > Calling release functions *_free() rather than *_clear_and_free() would be
> > more convenient as well. I also noticed that the data types are not
> > namespaced when they are exported. So perhaps we could drop this patch and
> > add the following to the next patch.
> >
> > /*
> >  * Namespace data types as well as functions and ensure consistent
> >  * naming of data types and the functions that operate on them.
> >  */
> > struct libgit_configset {
> >       struct config_set set;
> > };
> >
> > struct libgit_configset *libgit_configset_new() {
> >       struct libgit_configset *set = xmalloc(sizeof(*set));
> >
> >       git_configset_init(&set->set);
> >       return set;
> > }
> >
> > void libgit_configset_free(struct libgit_configset *set) {
> >       git_configset_clear(&set->set);
> >       free(set);
> > }
> >
> > Best Wishes
> >
> > Phillip
>
> Hmmm I see your point, but I am also hoping to keep the symbol export
> shim as small as possible, so that we can try to autogenerate it rather
> than add entries by hand. However, if people feel strongly that we don't
> want to add helper functions like *_alloc() or *_free() for types that don't
> already have them upstream, perhaps we can just put them in a separate
> rust-helpers.c file or something.

I'm thinking of this patch series as two closely related but
technically separable things: the creation of a .a/.so that can be
used outside of git, and the rust wrapper around that library. I think
these functions would be needed by all users of the library,
regardless of what language they're implemented in. i.e. they
shouldn't be thought of as 'rust helpers' and instead just the way
that the library is designed. _All_ functions that allocate memory
should have a paired "free" method, and that should be used
exclusively, regardless of host language.

So nit: I wouldn't call it rust-helpers.c ;)

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[phillip.wood123]

Hi Kyle

On 12/08/2024 22:55, Kyle Lippincott wrote:
> On Mon, Aug 12, 2024 at 2:39 PM Josh Steadmon <steadmon@google.com> wrote:
>> Hmmm I see your point, but I am also hoping to keep the symbol export
>> shim as small as possible, so that we can try to autogenerate it rather
>> than add entries by hand. However, if people feel strongly that we don't
>> want to add helper functions like *_alloc() or *_free() for types that don't
>> already have them upstream, perhaps we can just put them in a separate
>> rust-helpers.c file or something.
> 
> I'm thinking of this patch series as two closely related but
> technically separable things: the creation of a .a/.so that can be
> used outside of git, and the rust wrapper around that library. I think
> these functions would be needed by all users of the library,
> regardless of what language they're implemented in. i.e. they
> shouldn't be thought of as 'rust helpers' and instead just the way
> that the library is designed. _All_ functions that allocate memory
> should have a paired "free" method, and that should be used
> exclusively, regardless of host language.

Thanks for writing this, I agree we should be designing the library 
wrapper as a general purpose library, not as an implementation detail of 
the rust code.

Best Wishes

Phillip

> So nit: I wouldn't call it rust-helpers.c ;)

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Junio C Hamano]

phillip.wood123@gmail.com writes:

>> I'm thinking of this patch series as two closely related but
>> technically separable things: the creation of a .a/.so that can be
>> used outside of git, and the rust wrapper around that library. I think
>> these functions would be needed by all users of the library,
>> regardless of what language they're implemented in. i.e. they
>> shouldn't be thought of as 'rust helpers' and instead just the way
>> that the library is designed.
> ...
> Thanks for writing this, I agree we should be designing the library
> wrapper as a general purpose library, not as an implementation detail
> of the rust code.

Excellent.  Yes, make it wrapped for Rust may be a good motivator,
but cleaning up the interface so that it is more pleasant to wrap
for or directly call from any language should be an important goal
of this effort.

Thanks, all.

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[phillip.wood123]

Hi Josh

On 12/08/2024 22:39, Josh Steadmon wrote:
> On 2024.08.12 10:10, Phillip Wood wrote:
>> Hi Josh
>>
>> Do we really need to add this code to config.c rather than handling it in
>> the wrapper layer in the next patch?
>>
>> Looking ahead I wonder how useful it is to users of the library to separate
>> out allocation from initialization. A function that allocates and
>> initializes a configset would be more convenient and harder to misuse.
>> Calling release functions *_free() rather than *_clear_and_free() would be
>> more convenient as well. I also noticed that the data types are not
>> namespaced when they are exported. So perhaps we could drop this patch and
>> add the following to the next patch.
>>
>> /*
>>   * Namespace data types as well as functions and ensure consistent
>>   * naming of data types and the functions that operate on them.
>>   */
>> struct libgit_configset {
>> 	struct config_set set;
>> };
>>
>> struct libgit_configset *libgit_configset_new() {
>> 	struct libgit_configset *set = xmalloc(sizeof(*set));
>>
>> 	git_configset_init(&set->set);
>> 	return set;
>> }
>>
>> void libgit_configset_free(struct libgit_configset *set) {
>> 	git_configset_clear(&set->set);
>> 	free(set);
>> }
> 
> Hmmm I see your point, but I am also hoping to keep the symbol export
> shim as small as possible, so that we can try to autogenerate it rather
> than add entries by hand.

That's a nice aim - how do you plan to address namespacing data types 
with that approach? The autogenerator would need to know "struct 
config_set" wants to be wrapped as "struct libgit_configset" to ensure 
we end up with consistent naming for the data type and its functions. 
We'd also still want to make sure we end up with an ergonomic api which 
means one function to allocate and initialize each data type, not 
separate functions for allocation and initialization.

> However, if people feel strongly that we don't
> want to add helper functions like *_alloc() or *_free() for types that don't
> already have them upstream, perhaps we can just put them in a separate
> rust-helpers.c file or something.

If we're not using them upstream they're just clutter as far as git is 
concerned. Having said that if you get the autogeneration working well 
so the output is usable without a lot of manual tweaking I can see an 
argument for having these functions upstream.

Best Wishes

Phillip

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Josh Steadmon]

On 2024.08.13 10:48, phillip.wood123@gmail.com wrote:
> Hi Josh
> 
> On 12/08/2024 22:39, Josh Steadmon wrote:
> > On 2024.08.12 10:10, Phillip Wood wrote:
> > > Hi Josh
> > > 
> > > Do we really need to add this code to config.c rather than handling it in
> > > the wrapper layer in the next patch?
> > > 
> > > Looking ahead I wonder how useful it is to users of the library to separate
> > > out allocation from initialization. A function that allocates and
> > > initializes a configset would be more convenient and harder to misuse.
> > > Calling release functions *_free() rather than *_clear_and_free() would be
> > > more convenient as well. I also noticed that the data types are not
> > > namespaced when they are exported. So perhaps we could drop this patch and
> > > add the following to the next patch.
> > > 
> > > /*
> > >   * Namespace data types as well as functions and ensure consistent
> > >   * naming of data types and the functions that operate on them.
> > >   */
> > > struct libgit_configset {
> > > 	struct config_set set;
> > > };
> > > 
> > > struct libgit_configset *libgit_configset_new() {
> > > 	struct libgit_configset *set = xmalloc(sizeof(*set));
> > > 
> > > 	git_configset_init(&set->set);
> > > 	return set;
> > > }
> > > 
> > > void libgit_configset_free(struct libgit_configset *set) {
> > > 	git_configset_clear(&set->set);
> > > 	free(set);
> > > }
> > 
> > Hmmm I see your point, but I am also hoping to keep the symbol export
> > shim as small as possible, so that we can try to autogenerate it rather
> > than add entries by hand.
> 
> That's a nice aim - how do you plan to address namespacing data types with
> that approach? The autogenerator would need to know "struct config_set"
> wants to be wrapped as "struct libgit_configset" to ensure we end up with
> consistent naming for the data type and its functions. We'd also still want
> to make sure we end up with an ergonomic api which means one function to
> allocate and initialize each data type, not separate functions for
> allocation and initialization.
> 
> > However, if people feel strongly that we don't
> > want to add helper functions like *_alloc() or *_free() for types that don't
> > already have them upstream, perhaps we can just put them in a separate
> > rust-helpers.c file or something.
> 
> If we're not using them upstream they're just clutter as far as git is
> concerned. Having said that if you get the autogeneration working well so
> the output is usable without a lot of manual tweaking I can see an argument
> for having these functions upstream.
> 
> Best Wishes
> 
> Phillip

For V4 I'm going to drop the ambition to autogenerate the shim library,
and as such there's no longer a reason to keep helper functions out of
the shim library. So I've moved the _alloc and _free functions into the
shim.

For namespacing data types: for now I'm just letting the compiler infer
an opaque "struct libgit_config_set" and manually cast back to "struct
config_set" when we cross the boundary from the shim library back to
libgit.a. I'm not sure this is the right approach, but it avoids having
to wrap a single pointer in a separate struct.

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Patrick Steinhardt]

On Mon, Aug 12, 2024 at 10:10:48AM +0100, Phillip Wood wrote:
> Hi Josh
> 
> On 09/08/2024 23:41, Josh Steadmon wrote:
> > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > so that callers can manage config_set structs on the heap. This also
> > allows non-C external consumers to treat config_sets as opaque structs.
> 
> Do we really need to add this code to config.c rather than handling it in
> the wrapper layer in the next patch?
> 
> Looking ahead I wonder how useful it is to users of the library to separate
> out allocation from initialization. A function that allocates and
> initializes a configset would be more convenient and harder to misuse.
> Calling release functions *_free() rather than *_clear_and_free() would be
> more convenient as well. I also noticed that the data types are not
> namespaced when they are exported. So perhaps we could drop this patch and
> add the following to the next patch.

Agreed. This is also part of our coding guidelines as of 10f0723c8d
(Documentation: document idiomatic function names, 2024-07-30).

Patrick

Re: [PATCH v2 4/5] config: add git_configset_alloc() and git_configset_clear_and_free()

[Josh Steadmon]

On 2024.08.16 13:24, Patrick Steinhardt wrote:
> On Mon, Aug 12, 2024 at 10:10:48AM +0100, Phillip Wood wrote:
> > Hi Josh
> > 
> > On 09/08/2024 23:41, Josh Steadmon wrote:
> > > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > > so that callers can manage config_set structs on the heap. This also
> > > allows non-C external consumers to treat config_sets as opaque structs.
> > 
> > Do we really need to add this code to config.c rather than handling it in
> > the wrapper layer in the next patch?
> > 
> > Looking ahead I wonder how useful it is to users of the library to separate
> > out allocation from initialization. A function that allocates and
> > initializes a configset would be more convenient and harder to misuse.
> > Calling release functions *_free() rather than *_clear_and_free() would be
> > more convenient as well. I also noticed that the data types are not
> > namespaced when they are exported. So perhaps we could drop this patch and
> > add the following to the next patch.
> 
> Agreed. This is also part of our coding guidelines as of 10f0723c8d
> (Documentation: document idiomatic function names, 2024-07-30).
> 
> Patrick

Fixed in V4.

[PATCH v2 5/5] cgit: add higher-level cgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Wrap `struct config_set` and a few of its associated functions in
cgit-sys. Also introduce a higher-level "cgit" crate which provides a
more Rust-friendly interface to config_set structs.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                    |  1 +
 Makefile                                      |  2 +-
 contrib/cgit-rs/Cargo.lock                    | 14 ++++
 contrib/cgit-rs/Cargo.toml                    | 10 +++
 .../cgit-rs/cgit-sys/public_symbol_export.c   | 30 +++++++
 .../cgit-rs/cgit-sys/public_symbol_export.h   | 12 +++
 contrib/cgit-rs/cgit-sys/src/lib.rs           | 31 ++++++-
 contrib/cgit-rs/src/lib.rs                    | 82 +++++++++++++++++++
 8 files changed, 180 insertions(+), 2 deletions(-)
 create mode 100644 contrib/cgit-rs/Cargo.lock
 create mode 100644 contrib/cgit-rs/Cargo.toml
 create mode 100644 contrib/cgit-rs/src/lib.rs

diff --git a/.gitignore b/.gitignore
index 567cc9888f..7f2ee89b8a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,4 +248,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/cgit-rs/target
 /contrib/cgit-rs/cgit-sys/target
diff --git a/Makefile b/Makefile
index db8af99f20..3a71d10fbe 100644
--- a/Makefile
+++ b/Makefile
@@ -3723,7 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/cgit-rs/cgit-sys/target
+	$(RM) -r contrib/cgit-rs/target contrib/cgit-rs/cgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
diff --git a/contrib/cgit-rs/Cargo.lock b/contrib/cgit-rs/Cargo.lock
new file mode 100644
index 0000000000..991e775c34
--- /dev/null
+++ b/contrib/cgit-rs/Cargo.lock
@@ -0,0 +1,14 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "cgit"
+version = "0.1.0"
+dependencies = [
+ "cgit-sys",
+]
+
+[[package]]
+name = "cgit-sys"
+version = "0.1.0"
diff --git a/contrib/cgit-rs/Cargo.toml b/contrib/cgit-rs/Cargo.toml
new file mode 100644
index 0000000000..31f4d84522
--- /dev/null
+++ b/contrib/cgit-rs/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "cgit"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+cgit-sys = { version = "0.1.0", path = "cgit-sys" }
diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.c b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
index 62a91f76d0..adf8b21f11 100644
--- a/contrib/cgit-rs/cgit-sys/public_symbol_export.c
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.c
@@ -33,6 +33,36 @@ int libgit_parse_maybe_bool(const char *val)
 	return git_parse_maybe_bool(val);
 }
 
+struct config_set *libgit_configset_alloc(void)
+{
+	return git_configset_alloc();
+}
+
+void libgit_configset_clear_and_free(struct config_set *cs)
+{
+	git_configset_clear_and_free(cs);
+}
+
+void libgit_configset_init(struct config_set *cs)
+{
+	git_configset_init(cs);
+}
+
+int libgit_configset_add_file(struct config_set *cs, const char *filename)
+{
+	return git_configset_add_file(cs, filename);
+}
+
+int libgit_configset_get_int(struct config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int(cs, key, dest);
+}
+
+int libgit_configset_get_string(struct config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string(cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
index 64332f30de..882c7932e8 100644
--- a/contrib/cgit-rs/cgit-sys/public_symbol_export.h
+++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
@@ -9,6 +9,18 @@ void libgit_init_git(const char **argv);
 
 int libgit_parse_maybe_bool(const char *val);
 
+struct config_set *libgit_configset_alloc(void);
+
+void libgit_configset_clear_and_free(struct config_set *cs);
+
+void libgit_configset_init(struct config_set *cs);
+
+int libgit_configset_add_file(struct config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/cgit-rs/cgit-sys/src/lib.rs b/contrib/cgit-rs/cgit-sys/src/lib.rs
index 8c3ccc2859..d5072ea587 100644
--- a/contrib/cgit-rs/cgit-sys/src/lib.rs
+++ b/contrib/cgit-rs/cgit-sys/src/lib.rs
@@ -1,6 +1,15 @@
-use std::ffi::{c_char, c_int};
+use std::ffi::{c_char, c_int, c_void};
+
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_setup_git_directory() -> *const c_char;
 
     // From config.c
@@ -15,6 +24,26 @@ extern "C" {
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut config_set;
+    pub fn libgit_configset_clear_and_free(cs: *mut config_set);
+
+    pub fn libgit_configset_init(cs: *mut config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
diff --git a/contrib/cgit-rs/src/lib.rs b/contrib/cgit-rs/src/lib.rs
new file mode 100644
index 0000000000..74c83d161a
--- /dev/null
+++ b/contrib/cgit-rs/src/lib.rs
@@ -0,0 +1,82 @@
+use std::ffi::{c_char, c_int, c_void, CStr, CString};
+
+use cgit_sys::*;
+
+pub struct ConfigSet(*mut config_set);
+impl ConfigSet {
+    pub fn new() -> Self {
+        unsafe {
+            let ptr = libgit_configset_alloc();
+            libgit_configset_init(ptr);
+            ConfigSet(ptr)
+        }
+    }
+
+    // NEEDSWORK: maybe replace &str with &Path
+    pub fn add_files(&mut self, files: &[&str]) {
+        for file in files {
+            let rs = CString::new(*file).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val)
+    }
+
+    pub fn get_str(&mut self, key: &str) -> Option<CString> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str = CString::from_vec_with_nul(borrowed_str.to_bytes_with_nul().to_vec());
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str.unwrap())
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_clear_and_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        // NEEDSWORK: we need to supply a testdata config
+        let mut cs = ConfigSet::new();
+        let mut path = std::env::home_dir().expect("cannot get home directory path");
+        path.push(".gitconfig");
+        let path: String = path.into_os_string().into_string().unwrap();
+        cs.add_files(&["/etc/gitconfig", ".gitconfig", &path]);
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(5));
+        assert_eq!(cs.get_str("no_such_config_item"), None);
+    }
+}
-- 
2.46.0.76.ge559c4bf1a-goog

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Phillip Wood]

Hi Josh

On 09/08/2024 23:41, Josh Steadmon wrote:
> From: Calvin Wan <calvinwan@google.com>
> 
> Wrap `struct config_set` and a few of its associated functions in
> cgit-sys. Also introduce a higher-level "cgit" crate which provides a
> more Rust-friendly interface to config_set structs.

Having an ergonamic interface is a really good idea. As far as the 
naming goes I think the suggestion of "libgit-rs" is a good one.

> diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> index 64332f30de..882c7932e8 100644
> --- a/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> +++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> @@ -9,6 +9,18 @@ void libgit_init_git(const char **argv);
>   
>   int libgit_parse_maybe_bool(const char *val);

I'm suprised the compiler does not complain that 'struct config_set' is 
not declared in this header - I was expecting to see

	struct config_set;

before the function declarations. As I said in my comments on the last 
patch I think we'd be better to namespace our types as well as our 
functions in this library layer so this can be resued by other language 
bindings.

 > [...]
> +    pub fn get_str(&mut self, key: &str) -> Option<CString> {

If we're adding an ergonomic api then having return CString isn't ideal. 
I think the equivalent function in libgit2-rs has variants that return a 
String which is convinent if the caller is expecting utf8 values or 
Vec<u8> for non-utf8 values.

Best Wishes

Phillip

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Calvin Wan]

Phillip Wood <phillip.wood123@gmail.com> writes:
> Hi Josh
> 
> On 09/08/2024 23:41, Josh Steadmon wrote:
> > From: Calvin Wan <calvinwan@google.com>
> > 
> > Wrap `struct config_set` and a few of its associated functions in
> > cgit-sys. Also introduce a higher-level "cgit" crate which provides a
> > more Rust-friendly interface to config_set structs.
> 
> Having an ergonamic interface is a really good idea. As far as the 
> naming goes I think the suggestion of "libgit-rs" is a good one.

Agreed -- we plan on renaming it to "libgit-rs" in the next reroll.

> 
> > diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > index 64332f30de..882c7932e8 100644
> > --- a/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > +++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > @@ -9,6 +9,18 @@ void libgit_init_git(const char **argv);
> >   
> >   int libgit_parse_maybe_bool(const char *val);
> 
> I'm suprised the compiler does not complain that 'struct config_set' is 
> not declared in this header - I was expecting to see
> 
> 	struct config_set;

I'm surprised as well actually. Removing the forward declaration of
"struct config_set" in repository.h doesn't result in any complaints
from the compiler either. Will add it in the reroll, but am curious if
anyone has any ideas why the compiler isn't complaining.

> before the function declarations. As I said in my comments on the last 
> patch I think we'd be better to namespace our types as well as our 
> functions in this library layer so this can be resued by other language 
> bindings.

Are you suggesting something like "#define libgit_config_set
config_set"? I wouldn't be comfortable renaming config_set in git.git
just yet until config/config_set can be a standalone library by itself.

> 
>  > [...]
> > +    pub fn get_str(&mut self, key: &str) -> Option<CString> {
> 
> If we're adding an ergonomic api then having return CString isn't ideal. 
> I think the equivalent function in libgit2-rs has variants that return a 
> String which is convinent if the caller is expecting utf8 values or 
> Vec<u8> for non-utf8 values.

Having both get_cstr() and get_str() makes sense to me.

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Kyle Lippincott]

On Wed, Aug 21, 2024 at 11:46 AM Calvin Wan <calvinwan@google.com> wrote:
>
> Phillip Wood <phillip.wood123@gmail.com> writes:
> > Hi Josh
> >
> > On 09/08/2024 23:41, Josh Steadmon wrote:
> > > From: Calvin Wan <calvinwan@google.com>
> > >
> > > Wrap `struct config_set` and a few of its associated functions in
> > > cgit-sys. Also introduce a higher-level "cgit" crate which provides a
> > > more Rust-friendly interface to config_set structs.
> >
> > Having an ergonamic interface is a really good idea. As far as the
> > naming goes I think the suggestion of "libgit-rs" is a good one.
>
> Agreed -- we plan on renaming it to "libgit-rs" in the next reroll.
>
> >
> > > diff --git a/contrib/cgit-rs/cgit-sys/public_symbol_export.h b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > > index 64332f30de..882c7932e8 100644
> > > --- a/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > > +++ b/contrib/cgit-rs/cgit-sys/public_symbol_export.h
> > > @@ -9,6 +9,18 @@ void libgit_init_git(const char **argv);
> > >
> > >   int libgit_parse_maybe_bool(const char *val);
> >
> > I'm suprised the compiler does not complain that 'struct config_set' is
> > not declared in this header - I was expecting to see
> >
> >       struct config_set;
>
> I'm surprised as well actually. Removing the forward declaration of
> "struct config_set" in repository.h doesn't result in any complaints
> from the compiler either. Will add it in the reroll, but am curious if
> anyone has any ideas why the compiler isn't complaining.

C doesn't require structs be forward declared separately. You can
change the name to be anything you want, and as long as the forward
declaration of the function and the function definition agree, you're
fine. If they don't agree, well, let's hope you don't encounter that
(it's the same problem as if you have a forward declaration that's
*not* visible from the definition that disagrees with the definition:
`void some_func();` vs. `int some_func(int arg) { ... }` -- if the
forward declaration wasn't made in the same translation unit that
defines `some_func`, nothing detects this misuse in C).

For this reason, you should only ever use forward declarations that
are provided by "the code" that's being forward declared. i.e. if
you're trying to use a function from foo.c, please forward declare it
in foo.h, and only there. This way, assuming foo.c includes foo.h,
you'll detect mismatches.

[apologies if people got multiple copies of this, I sent with HTML
mode enabled the first time]

>
> > before the function declarations. As I said in my comments on the last
> > patch I think we'd be better to namespace our types as well as our
> > functions in this library layer so this can be resued by other language
> > bindings.
>
> Are you suggesting something like "#define libgit_config_set
> config_set"? I wouldn't be comfortable renaming config_set in git.git
> just yet until config/config_set can be a standalone library by itself.
>
> >
> >  > [...]
> > > +    pub fn get_str(&mut self, key: &str) -> Option<CString> {
> >
> > If we're adding an ergonomic api then having return CString isn't ideal.
> > I think the equivalent function in libgit2-rs has variants that return a
> > String which is convinent if the caller is expecting utf8 values or
> > Vec<u8> for non-utf8 values.
>
> Having both get_cstr() and get_str() makes sense to me.

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Phillip Wood]

On 21/08/2024 20:23, Kyle Lippincott wrote:
> On Wed, Aug 21, 2024 at 11:46 AM Calvin Wan <calvinwan@google.com> wrote:
>>
>> Phillip Wood <phillip.wood123@gmail.com> writes:
>>> 
>>> I'm suprised the compiler does not complain that 'struct config_set' is
>>> not declared in this header - I was expecting to see
>>>
>>>        struct config_set;
>>
>> I'm surprised as well actually. Removing the forward declaration of
>> "struct config_set" in repository.h doesn't result in any complaints
>> from the compiler either. Will add it in the reroll, but am curious if
>> anyone has any ideas why the compiler isn't complaining.
> 
> C doesn't require structs be forward declared separately. You can
> change the name to be anything you want, and as long as the forward
> declaration of the function and the function definition agree, you're
> fine.

Oh, TIL. The declaration

     struct foo *f(struct bar*);

declares "struct foo" and "struct bar" as well as function "f". The 
declaration of "struct bar" is scoped to the parameter list and so is 
not visible to the rest of the file. This is why we forward declare 
structs that are used in parameter lists and why the compiler complains 
if we don't. The declaration of "struct foo" does have file scope though 
which explains why the compiler does not complain about 
public_symbol_export.h because "struct config_set" is declared as a 
return value before it is used in any parameter lists.

Thanks

Phillip

> If they don't agree, well, let's hope you don't encounter that
> (it's the same problem as if you have a forward declaration that's
> *not* visible from the definition that disagrees with the definition:
> `void some_func();` vs. `int some_func(int arg) { ... }` -- if the
> forward declaration wasn't made in the same translation unit that
> defines `some_func`, nothing detects this misuse in C).
> 
> For this reason, you should only ever use forward declarations that
> are provided by "the code" that's being forward declared. i.e. if
> you're trying to use a function from foo.c, please forward declare it
> in foo.h, and only there. This way, assuming foo.c includes foo.h,
> you'll detect mismatches.
> 
> [apologies if people got multiple copies of this, I sent with HTML
> mode enabled the first time]
> 
>>
>>> before the function declarations. As I said in my comments on the last
>>> patch I think we'd be better to namespace our types as well as our
>>> functions in this library layer so this can be resued by other language
>>> bindings.
>>
>> Are you suggesting something like "#define libgit_config_set
>> config_set"? I wouldn't be comfortable renaming config_set in git.git
>> just yet until config/config_set can be a standalone library by itself.
>>
>>>
>>>   > [...]
>>>> +    pub fn get_str(&mut self, key: &str) -> Option<CString> {
>>>
>>> If we're adding an ergonomic api then having return CString isn't ideal.
>>> I think the equivalent function in libgit2-rs has variants that return a
>>> String which is convinent if the caller is expecting utf8 values or
>>> Vec<u8> for non-utf8 values.
>>
>> Having both get_cstr() and get_str() makes sense to me.
> 

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Phillip Wood]

Hi Calvin

On 21/08/2024 19:46, Calvin Wan wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
 >
>> before the function declarations. As I said in my comments on the last
>> patch I think we'd be better to namespace our types as well as our
>> functions in this library layer so this can be resued by other language
>> bindings.
> 
> Are you suggesting something like "#define libgit_config_set
> config_set"? I wouldn't be comfortable renaming config_set in git.git
> just yet until config/config_set can be a standalone library by itself.

I was suggesting[1] adding

     struct libgit_configset {
	    struct config_set set;
     };

to public_symbol_export.c and rewriting the wrappers in that file to use 
this struct e.g.

     int libgit_configset_get_int(struct libgit_configset *cs,
				 const char *key, int *dest)
     {
	    return git_configset_get_int(&cs.set, key, dest);
     }

In public_symbol_export.h we'd then have

     struct libgit_configset;

     int libgit_configset_get_int(struct libgit_configset *,
				 const char *, int *);

If we want the symbol exports to be useful outside of the rust bindings 
I think we need to namespace our types as well as our functions.

[1] 
https://lore.kernel.org/git/5720d5b9-a850-4024-a1fd-54acc6b15a74@gmail.com

>>> +    pub fn get_str(&mut self, key: &str) -> Option<CString> {
>>
>> If we're adding an ergonomic api then having return CString isn't ideal.
>> I think the equivalent function in libgit2-rs has variants that return a
>> String which is convinent if the caller is expecting utf8 values or
>> Vec<u8> for non-utf8 values.
> 
> Having both get_cstr() and get_str() makes sense to me.

Just to be clear get_cstr() would return Vec<u8>? I'm far from a rust 
expert but my understanding was that crates that wrap C libraries 
generally avoid using CString in their API.

Best Wishes

Phillip

Re: [PATCH v2 5/5] cgit: add higher-level cgit crate

[Josh Steadmon]

On 2024.08.22 10:12, Phillip Wood wrote:
> Hi Calvin
> 
> On 21/08/2024 19:46, Calvin Wan wrote:
> > Phillip Wood <phillip.wood123@gmail.com> writes:
> >
> > > before the function declarations. As I said in my comments on the last
> > > patch I think we'd be better to namespace our types as well as our
> > > functions in this library layer so this can be resued by other language
> > > bindings.
> > 
> > Are you suggesting something like "#define libgit_config_set
> > config_set"? I wouldn't be comfortable renaming config_set in git.git
> > just yet until config/config_set can be a standalone library by itself.
> 
> I was suggesting[1] adding
> 
>     struct libgit_configset {
> 	    struct config_set set;
>     };
> 
> to public_symbol_export.c and rewriting the wrappers in that file to use
> this struct e.g.
> 
>     int libgit_configset_get_int(struct libgit_configset *cs,
> 				 const char *key, int *dest)
>     {
> 	    return git_configset_get_int(&cs.set, key, dest);
>     }
> 
> In public_symbol_export.h we'd then have
> 
>     struct libgit_configset;
> 
>     int libgit_configset_get_int(struct libgit_configset *,
> 				 const char *, int *);
> 
> If we want the symbol exports to be useful outside of the rust bindings I
> think we need to namespace our types as well as our functions.
> 
> [1]
> https://lore.kernel.org/git/5720d5b9-a850-4024-a1fd-54acc6b15a74@gmail.com
> 
> > > > +    pub fn get_str(&mut self, key: &str) -> Option<CString> {
> > > 
> > > If we're adding an ergonomic api then having return CString isn't ideal.
> > > I think the equivalent function in libgit2-rs has variants that return a
> > > String which is convinent if the caller is expecting utf8 values or
> > > Vec<u8> for non-utf8 values.
> > 
> > Having both get_cstr() and get_str() makes sense to me.
> 
> Just to be clear get_cstr() would return Vec<u8>? I'm far from a rust expert
> but my understanding was that crates that wrap C libraries generally avoid
> using CString in their API.
> 
> Best Wishes
> 
> Phillip

Yeah let's just go with "get_string()" and return an Option<String>.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> This series provides two small Rust wrapper libraries around parts of
> Git: "cgit-sys", which exposes a few functions from libgit.a, and
> "cgit", which provides a more Rust-friendly interface to some of those
> functions. In addition to included unit tests, at $DAYJOB we have tested
> building JJ[1] with our library and used it to replace some of the
> libgit2-rs uses.

Thanks, queued.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Jason A. Donenfeld]

Still the same name for v2? Cmon.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-10 15:15, Jason A. Donenfeld wrote:
> Still the same name for v2? Cmon.

Yeah, I was also surprised to see that.  This _isn't_ cgit.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Eric Sunshine]

On Sun, Aug 11, 2024 at 1:27 PM Dragan Simic <dsimic@manjaro.org> wrote:
> On 2024-08-10 15:15, Jason A. Donenfeld wrote:
> > Still the same name for v2? Cmon.
>
> Yeah, I was also surprised to see that.  This _isn't_ cgit.

Josh addressed this point in the v2 cover letter by saying:

    Known NEEDSWORK:
    ...
    * Bikeshed on the name

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-12 01:03, Eric Sunshine wrote:
> On Sun, Aug 11, 2024 at 1:27 PM Dragan Simic <dsimic@manjaro.org> 
> wrote:
>> On 2024-08-10 15:15, Jason A. Donenfeld wrote:
>> > Still the same name for v2? Cmon.
>> 
>> Yeah, I was also surprised to see that.  This _isn't_ cgit.
> 
> Josh addressed this point in the v2 cover letter by saying:
> 
>     Known NEEDSWORK:
>     ...
>     * Bikeshed on the name

But didn't Junio already say that the v2 of this series will be merged
as-is? [1]  That's what actually made me surprised and confused.

[1] https://lore.kernel.org/git/xmqqfrrd9slb.fsf@gitster.g/

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Eric Sunshine]

On Sun, Aug 11, 2024 at 7:23 PM Dragan Simic <dsimic@manjaro.org> wrote:
> On 2024-08-12 01:03, Eric Sunshine wrote:
> > Josh addressed this point in the v2 cover letter by saying:
> >
> >     Known NEEDSWORK:
> >     ...
> >     * Bikeshed on the name
>
> But didn't Junio already say that the v2 of this series will be merged
> as-is? [1]  That's what actually made me surprised and confused.
>
> [1] https://lore.kernel.org/git/xmqqfrrd9slb.fsf@gitster.g/

That only means that Junio placed the topic in his "seen" branch,
however, no promises about ultimate acceptance are attached to topics
in that branch. Quoting from Junio's "What's Cooking" emails (such as
[*]):

    Commits prefixed with '-' are only in 'seen', and aren't
    considered "accepted" at all and may be annotated with an URL to a
    message that raises issues but they are no means exhaustive.  A
    topic without enough support may be discarded after a long period
    of no activity (of course they can be resubmit when new interests
    arise).

[*]: https://lore.kernel.org/git/xmqqo762frkz.fsf@gitster.g/

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-12 01:33, Eric Sunshine wrote:
> On Sun, Aug 11, 2024 at 7:23 PM Dragan Simic <dsimic@manjaro.org> 
> wrote:
>> On 2024-08-12 01:03, Eric Sunshine wrote:
>> > Josh addressed this point in the v2 cover letter by saying:
>> >
>> >     Known NEEDSWORK:
>> >     ...
>> >     * Bikeshed on the name
>> 
>> But didn't Junio already say that the v2 of this series will be merged
>> as-is? [1]  That's what actually made me surprised and confused.
>> 
>> [1] https://lore.kernel.org/git/xmqqfrrd9slb.fsf@gitster.g/
> 
> That only means that Junio placed the topic in his "seen" branch,
> however, no promises about ultimate acceptance are attached to topics
> in that branch. Quoting from Junio's "What's Cooking" emails (such as
> [*]):
> 
>     Commits prefixed with '-' are only in 'seen', and aren't
>     considered "accepted" at all and may be annotated with an URL to a
>     message that raises issues but they are no means exhaustive.  A
>     topic without enough support may be discarded after a long period
>     of no activity (of course they can be resubmit when new interests
>     arise).
> 
> [*]: https://lore.kernel.org/git/xmqqo762frkz.fsf@gitster.g/

Ah, good, thanks for the clarification.  It's all fine then, as far as
I'm concerned.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Sun, Aug 11, 2024 at 1:27 PM Dragan Simic <dsimic@manjaro.org> wrote:
>> On 2024-08-10 15:15, Jason A. Donenfeld wrote:
>> > Still the same name for v2? Cmon.
>>
>> Yeah, I was also surprised to see that.  This _isn't_ cgit.
>
> Josh addressed this point in the v2 cover letter by saying:
>
>     Known NEEDSWORK:
>     ...
>     * Bikeshed on the name

I do not quite consider it as as "addressed this point" to just slap
a NEEDSWORK label and doing nothing else, though.

The original iteration had this:

    * bikeshedding on the name (yes, really). There is an active, unrelated
      CGit project [4] that we only recently became aware of. We originally
      took the name "cgit" because at $DAYJOB we sometimes refer to git.git
      as "cgit" to distinguish it from jgit [5].

and then now they as well as reviewers all have seen the tentative
cgit name, saw the reaction it caused, and now know that not just
potentially confusing other project _exists_, but it does matter.

Reviewers already have spent some time on suggesting that "git" part
should not be "c"git, as well as "rs" part may better be "sys",
etc.?.  There should be _some_ response, even if it does not yet
propose a new name.

If it acknowledged that the time and knowledge reviewers gave the
topic were appreciated, e.g., "The proposers of this topic saw THIS
point and THAT point as a input that we WILL need to consider when
we decide on the name.  We acknowledge that the name "cgit-rs" is
not ideal and needs to be changed.  But we haven't reached any
concrete alternative name yet, so this round still uses the same
name", I'd call that "addressed this point", though.

But just a dismissing "Bikeshed on the name", as if they do not care
to be mistaken as saying "those who complain about the name are only
bikeshedding and not worth listening to"?

We should do better than that.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Eric Sunshine]

On Mon, Aug 12, 2024 at 4:15 AM Junio C Hamano <gitster@pobox.com> wrote:
> The original iteration had this:
>
>     * bikeshedding on the name (yes, really). There is an active, unrelated
>       CGit project [4] that we only recently became aware of. We originally
>       took the name "cgit" because at $DAYJOB we sometimes refer to git.git
>       as "cgit" to distinguish it from jgit [5].

A tangent: Speaking of external/other projects, I don't think we've
seen an explanation yet as to why this Rust wrapper is proposed as a
`contrib/` item of Git itself, as opposed to being a separate project.

I can only think of two possible reasons why they might want it in the
Git project itself...

(1) Easier access to the library portions of Git ("libgit") since that
portion of the code is not otherwise published as a standalone
library. However, a workable alternative would be for the Rust wrapper
to carry its own "vendored"[1] copy of Git. This would also ensure
more reliable builds since they wouldn't have to worry about the
"libgit" API changing from under them, and can adjust for "libgit" API
changes when they manually pull in a new vendored copy. Hence, I'm not
convinced that this is a valid reason to carry the Rust wrapper in
Git.

(2) Perhaps the intention is that this Rust wrapper work will allow
Rust to be used within Git itself[3]? If that's the case, then
`contrib/` seems the wrong resting place for this code.

On the other hand, as a standalone project, a big benefit is that the
Rust wrapper could have its own release cadence distinct from Git,
which would likely be very beneficial since it is such a young
(indeed, nascent) library; it is likely that the maintainers will want
to release early and often at this stage.

[1]: Other Rust projects carry vendored copies of projects upon which
they rely. For instance, the "native_tls" crate has a vendored copy of
OpenSSL[2].

[2]: https://docs.rs/native-tls/latest/native_tls/#cargo-features

[3]: https://lore.kernel.org/git/CABPp-BFWsWCGogqQ=haMsS4OhOdSwc3frcAxa6soQR5ORTceOA@mail.gmail.com/

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> A tangent: Speaking of external/other projects, I don't think we've
> seen an explanation yet as to why this Rust wrapper is proposed as a
> `contrib/` item of Git itself, as opposed to being a separate project.
>
> I can only think of two possible reasons why they might want it in the
> Git project itself...
>
> (1) Easier access to the library portions of Git ("libgit") since that
> portion of the code is not otherwise published as a standalone
> library.

This is not a good reason at all, if we look at what (the real) cgit
project does ;-), which is to bind us as their submodule.

> (2) Perhaps the intention is that this Rust wrapper work will allow
> Rust to be used within Git itself[3]? If that's the case, then
> `contrib/` seems the wrong resting place for this code.

The contrib/ hierarchy is a mixed bag, and we may want to clean it
up by Git 3.0 happens.

 - We used to put things that one-way depend on Git that are (1)
   useless by themselves, if Git did not exist, and (2) Git can do
   fine without them in the contrib/ hierarchy.

   The primary reason for doing so was because the Git was young and
   relatively unknown, and our community was small.  Even for an
   add-on that may be at most "nice to have" from Git's point of
   view, it made some sense to give these add-ons wider exposure by
   simply being bundled with Git.  We stopped doing that after Git
   ecosystem have matured enough and encouraged them to move either
   up (i.e. prove that Git cannot really do fine without it because
   the add-on is so useful, and become the first class part of Git)
   or out (i.e. it may be a fine add-on, but there are other similar
   add-ons that aim to achieve the same, similar, or overlapping
   goal---unlike the nascent era of Git, it should be able to become
   an independent project to compete fairly with others on its merit
   alone).

 - We have some stuff like completion and prompt scripts that have
   proven that the end-user experience cannot be "fine without", but
   haven't moved out of "contrib/", mostly by inercia.

 - We also have things that are meant to help Git developers in the
   hierarchy (Cocci rules come to mind).  This one could fall into
   that category, but I am not sure.

> On the other hand, as a standalone project, a big benefit is that the
> Rust wrapper could have its own release cadence distinct from Git,
> which would likely be very beneficial since it is such a young
> (indeed, nascent) library; it is likely that the maintainers will want
> to release early and often at this stage.

That's cgit approach.  If something is useless if Git did not exist,
but Git does not need it at all to function, that may be a sensible
thing to do.

If we plan to eventually be able to rewrite the logic for say
history traveral or specifying "set of commits" in such a way that
is more involved than just "those that can be reached from these tip
commits excluding the ones that can be reached from those bottom
commits" (in other words, "A..B C..D" that is not "^A B ^C D"), and
find it unpleasant to do in C and want to employ Rust, for example,
we'd certainly need it somewhere in _our_ tree.  Perhaps by the time
it happens we might have an extension/dll mechanism in such a way
that those who do not want (or cannot build) certain extensions can
opt out and these "optional" things would by convention live
somewhere other than the main source code live, but even then,
contrib/ sounds like a wrong place and we'd do so perhaps in extra/
or something.

> [1]: Other Rust projects carry vendored copies of projects upon which
> they rely. For instance, the "native_tls" crate has a vendored copy of
> OpenSSL[2].
>
> [2]: https://docs.rs/native-tls/latest/native_tls/#cargo-features
>
> [3]: https://lore.kernel.org/git/CABPp-BFWsWCGogqQ=haMsS4OhOdSwc3frcAxa6soQR5ORTceOA@mail.gmail.com/

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.12 05:03, Eric Sunshine wrote:
> On Mon, Aug 12, 2024 at 4:15 AM Junio C Hamano <gitster@pobox.com> wrote:
> > The original iteration had this:
> >
> >     * bikeshedding on the name (yes, really). There is an active, unrelated
> >       CGit project [4] that we only recently became aware of. We originally
> >       took the name "cgit" because at $DAYJOB we sometimes refer to git.git
> >       as "cgit" to distinguish it from jgit [5].
> 
> A tangent: Speaking of external/other projects, I don't think we've
> seen an explanation yet as to why this Rust wrapper is proposed as a
> `contrib/` item of Git itself, as opposed to being a separate project.
> 
> I can only think of two possible reasons why they might want it in the
> Git project itself...
> 
> (1) Easier access to the library portions of Git ("libgit") since that
> portion of the code is not otherwise published as a standalone
> library. However, a workable alternative would be for the Rust wrapper
> to carry its own "vendored"[1] copy of Git. This would also ensure
> more reliable builds since they wouldn't have to worry about the
> "libgit" API changing from under them, and can adjust for "libgit" API
> changes when they manually pull in a new vendored copy. Hence, I'm not
> convinced that this is a valid reason to carry the Rust wrapper in
> Git.
> 
> (2) Perhaps the intention is that this Rust wrapper work will allow
> Rust to be used within Git itself[3]? If that's the case, then
> `contrib/` seems the wrong resting place for this code.

Neither, actually. We hope that by keeping the crate definition in the
main Git repo and by allowing developers to optionally run them by
default as part of CI and `make test`, we can catch breaking bugs as
early as possible. We're also hoping that we'll get more attention from
interested developers by staying in the main repo rather than in a
separate project.

> On the other hand, as a standalone project, a big benefit is that the
> Rust wrapper could have its own release cadence distinct from Git,
> which would likely be very beneficial since it is such a young
> (indeed, nascent) library; it is likely that the maintainers will want
> to release early and often at this stage.

AFAIK, the crate releases don't have to be strictly tied to Git
releases. In theory we could choose to publish updates to the crate
whenever "master" or any other branch is updated; whether or not that is
a good idea is a separate discussion.


> [1]: Other Rust projects carry vendored copies of projects upon which
> they rely. For instance, the "native_tls" crate has a vendored copy of
> OpenSSL[2].
> 
> [2]: https://docs.rs/native-tls/latest/native_tls/#cargo-features
> 
> [3]: https://lore.kernel.org/git/CABPp-BFWsWCGogqQ=haMsS4OhOdSwc3frcAxa6soQR5ORTceOA@mail.gmail.com/

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Patrick Steinhardt]

On Mon, Aug 12, 2024 at 02:32:12PM -0700, Josh Steadmon wrote:
> On 2024.08.12 05:03, Eric Sunshine wrote:
> > On Mon, Aug 12, 2024 at 4:15 AM Junio C Hamano <gitster@pobox.com> wrote:
> > > The original iteration had this:
> > >
> > >     * bikeshedding on the name (yes, really). There is an active, unrelated
> > >       CGit project [4] that we only recently became aware of. We originally
> > >       took the name "cgit" because at $DAYJOB we sometimes refer to git.git
> > >       as "cgit" to distinguish it from jgit [5].
> > 
> > A tangent: Speaking of external/other projects, I don't think we've
> > seen an explanation yet as to why this Rust wrapper is proposed as a
> > `contrib/` item of Git itself, as opposed to being a separate project.
> > 
> > I can only think of two possible reasons why they might want it in the
> > Git project itself...
> > 
> > (1) Easier access to the library portions of Git ("libgit") since that
> > portion of the code is not otherwise published as a standalone
> > library. However, a workable alternative would be for the Rust wrapper
> > to carry its own "vendored"[1] copy of Git. This would also ensure
> > more reliable builds since they wouldn't have to worry about the
> > "libgit" API changing from under them, and can adjust for "libgit" API
> > changes when they manually pull in a new vendored copy. Hence, I'm not
> > convinced that this is a valid reason to carry the Rust wrapper in
> > Git.
> > 
> > (2) Perhaps the intention is that this Rust wrapper work will allow
> > Rust to be used within Git itself[3]? If that's the case, then
> > `contrib/` seems the wrong resting place for this code.
> 
> Neither, actually. We hope that by keeping the crate definition in the
> main Git repo and by allowing developers to optionally run them by
> default as part of CI and `make test`, we can catch breaking bugs as
> early as possible. We're also hoping that we'll get more attention from
> interested developers by staying in the main repo rather than in a
> separate project.

That to me raises an important question: who is the one fixing breakage?
Hypothetically speaking, if I continue with my refactoring spree to drop
`the_repository`, do I also have to fix the Rust bindings that I break
as a consequence?

Patrick

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1531 bytes --]

On 2024-08-16 at 11:39:24, Patrick Steinhardt wrote:
> That to me raises an important question: who is the one fixing breakage?
> Hypothetically speaking, if I continue with my refactoring spree to drop
> `the_repository`, do I also have to fix the Rust bindings that I break
> as a consequence?

If we're testing it in CI, then you are responsible for not breaking it,
even if you don't use it, just as I am responsible for not breaking
Windows, even though I don't use that platform.  That's typically been
the policy here and elsewhere.

That being said, there are two things I'd like to point out.  First,
your work to drop the implicit `the_repository` is very useful, because
Rust code implicitly requires thread safety as part of the language, and
you will be making the dependency on the object explicit so Rust can
guard against concurrent access.  I expect that you will end up breaking
very few endpoints with those series because we can't really expose
endpoints with implicit global thread-unsafe state (such as
`the_repository`) in Rust anyway except as `unsafe`.

Second, I and others will probably be happy to help if you have
questions or need assistance fixing bindings.  I've done a decent amount
of work in Rust, including mostly porting a service at $DAYJOB from C to
Rust (the one that serves Git requests, no less), so I'm familiar with a
lot of those kinds of problems, as I'm sure others are on the list as
well.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Eric Sunshine]

On Fri, Aug 16, 2024 at 5:38 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> On 2024-08-16 at 11:39:24, Patrick Steinhardt wrote:
> > That to me raises an important question: who is the one fixing breakage?
> > Hypothetically speaking, if I continue with my refactoring spree to drop
> > `the_repository`, do I also have to fix the Rust bindings that I break
> > as a consequence?

I share Patrick's concern, and it's one of the reasons I posed the
question in the first place about why these Rust bindings are being
proposed for Git's "contrib/" rather than as a standalone project.

> If we're testing it in CI, then you are responsible for not breaking it,
> even if you don't use it, just as I am responsible for not breaking
> Windows, even though I don't use that platform.  That's typically been
> the policy here and elsewhere.

This is an apples-and-oranges comparison, isn't it? Although the
majority of Git developers may be Unix folk, Git itself has a
reasonably large Windows user population, so breaking it on Windows
carries some potentially real consequences (i.e. hurting the immediate
user community). On the other hand, the Rust bindings under discussion
are (1) as yet effectively hypothetical, (2) will almost certainly be
relied upon by a tiny number of other projects, and (3) impact only
developers, not users, of Git-related tooling.

As such, it seems far too early to be placing the onus on *all* Git
developers to have to worry about the Rust bindings. If, at some point
in the future, the Rust bindings gain significance or become
indispensable to the Git project itself, then the onus might well be
warranted, but at this early stage, it seems unreasonable. A more
friendly approach to the overall Git developer community would be for
those interested in the Rust bindings to maintain them. (The case with
the CMake project files is similar. In the end, the people interested
in utilizing them took on the responsibility of maintaining them.)

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> (The case with the CMake project files is similar. In the end, the
> people interested in utilizing them took on the responsibility of
> maintaining them.)

That is a good analogy.  We would all be slowed down if you have to
adjust CMake stuff (or autoconf stuff for that matter) when the
build procedure changes you made only to Makefile (because you only
have a ready access and expertise on the "make" proper), but in
practice, niche things that nobody cares about (and remember, if no
stakeholder is there to volunteer, you cannot force others to care
about your niche) will bitrot enough to cause somebody to care and
get fixed.  Another possibility is that it proves that nobody cares,
and we just drop that part.  Either outcome is fine in the end.

I think the Rust thing will proceed the same way.  At least, I am
sensing that this time there are already motivated to lend help from
outside the group of folks that are proposing this change.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 3681 bytes --]

On 2024-08-12 at 09:03:57, Eric Sunshine wrote:
> On Mon, Aug 12, 2024 at 4:15 AM Junio C Hamano <gitster@pobox.com> wrote:
> > The original iteration had this:
> >
> >     * bikeshedding on the name (yes, really). There is an active, unrelated
> >       CGit project [4] that we only recently became aware of. We originally
> >       took the name "cgit" because at $DAYJOB we sometimes refer to git.git
> >       as "cgit" to distinguish it from jgit [5].
> 
> A tangent: Speaking of external/other projects, I don't think we've
> seen an explanation yet as to why this Rust wrapper is proposed as a
> `contrib/` item of Git itself, as opposed to being a separate project.
> 
> I can only think of two possible reasons why they might want it in the
> Git project itself...
> 
> (1) Easier access to the library portions of Git ("libgit") since that
> portion of the code is not otherwise published as a standalone
> library. However, a workable alternative would be for the Rust wrapper
> to carry its own "vendored"[1] copy of Git. This would also ensure
> more reliable builds since they wouldn't have to worry about the
> "libgit" API changing from under them, and can adjust for "libgit" API
> changes when they manually pull in a new vendored copy. Hence, I'm not
> convinced that this is a valid reason to carry the Rust wrapper in
> Git.

Please don't vendor packages like this.  This means that every time
there's a security vulnerability in Git, even if we ship a public libgit
and even if the security impact doesn't affect libgit, every security
scanning tool will demand that the crate be updated immediately.  This
wastes colossal amounts of work and it's hostile to distros as well, who
will have to repackage the source to remove the vendoring.

At work, I maintain a codebase using Nokogiri, which is a Ruby gem that
vendors libxml2 (which has an absolutely abysmal security record), and I
cannot tell you how much useless work I perform updating this gem
because of security scanners screaming about the vendored libxml2, even
though it's completely possible to use the system library.

> (2) Perhaps the intention is that this Rust wrapper work will allow
> Rust to be used within Git itself[3]? If that's the case, then
> `contrib/` seems the wrong resting place for this code.

I think contrib is the right place for now.  We may in the future want
to include some Rust code in the codebase in some context, and this is a
good way for us to experiment with it on an interim basis and discover
whether that's a viable approach for us.  Perhaps it is, and perhaps it
is not, but we'll have built the experience to know for certain.

I could imagine (and this is hypothetical and speculative) that we might
allow some sorts of performance optimizations and optional features in
Rust, but require core functionality be in C until Rust is better
supported on some platforms.  That might be a nice way to improve things
like multithreading, which Rust excels at, for example.

As an example, I'd like to write a random-access tree parser[0] in Rust
(such as for "dev:README.md"), since our current approach performs
terribly on large trees (it's linear instead of logarithmic).  However,
I'm not willing to write that code in C because it's a giant landmine of
memory unsafety and I think it's too likely to be buggy in C.

[0] I am aware that due to the structure of trees, this is not _always_
possible to do, but it is possible in _many_ cases, and optimistically
trying and falling back to the old approach may still be faster.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Dragan Simic]

On 2024-08-12 10:15, Junio C Hamano wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
>> On Sun, Aug 11, 2024 at 1:27 PM Dragan Simic <dsimic@manjaro.org> 
>> wrote:
>>> On 2024-08-10 15:15, Jason A. Donenfeld wrote:
>>> > Still the same name for v2? Cmon.
>>> 
>>> Yeah, I was also surprised to see that.  This _isn't_ cgit.
>> 
>> Josh addressed this point in the v2 cover letter by saying:
>> 
>>     Known NEEDSWORK:
>>     ...
>>     * Bikeshed on the name
> 
> I do not quite consider it as as "addressed this point" to just slap
> a NEEDSWORK label and doing nothing else, though.
> 
> The original iteration had this:
> 
>     * bikeshedding on the name (yes, really). There is an active, 
> unrelated
>       CGit project [4] that we only recently became aware of. We 
> originally
>       took the name "cgit" because at $DAYJOB we sometimes refer to 
> git.git
>       as "cgit" to distinguish it from jgit [5].
> 
> and then now they as well as reviewers all have seen the tentative
> cgit name, saw the reaction it caused, and now know that not just
> potentially confusing other project _exists_, but it does matter.
> 
> Reviewers already have spent some time on suggesting that "git" part
> should not be "c"git, as well as "rs" part may better be "sys",
> etc.?.  There should be _some_ response, even if it does not yet
> propose a new name.
> 
> If it acknowledged that the time and knowledge reviewers gave the
> topic were appreciated, e.g., "The proposers of this topic saw THIS
> point and THAT point as a input that we WILL need to consider when
> we decide on the name.  We acknowledge that the name "cgit-rs" is
> not ideal and needs to be changed.  But we haven't reached any
> concrete alternative name yet, so this round still uses the same
> name", I'd call that "addressed this point", though.
> 
> But just a dismissing "Bikeshed on the name", as if they do not care
> to be mistaken as saying "those who complain about the name are only
> bikeshedding and not worth listening to"?
> 
> We should do better than that.

I really appreciate and support your response, Junio.  Brushing it off
originally as "bikeshedding" did leave a sour taste, because it 
presented
the conflicting naming as a non-issue.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2024.08.12 01:15, Junio C Hamano wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
> > On Sun, Aug 11, 2024 at 1:27 PM Dragan Simic <dsimic@manjaro.org> wrote:
> >> On 2024-08-10 15:15, Jason A. Donenfeld wrote:
> >> > Still the same name for v2? Cmon.
> >>
> >> Yeah, I was also surprised to see that.  This _isn't_ cgit.
> >
> > Josh addressed this point in the v2 cover letter by saying:
> >
> >     Known NEEDSWORK:
> >     ...
> >     * Bikeshed on the name
> 
> I do not quite consider it as as "addressed this point" to just slap
> a NEEDSWORK label and doing nothing else, though.
> 
> The original iteration had this:
> 
>     * bikeshedding on the name (yes, really). There is an active, unrelated
>       CGit project [4] that we only recently became aware of. We originally
>       took the name "cgit" because at $DAYJOB we sometimes refer to git.git
>       as "cgit" to distinguish it from jgit [5].
> 
> and then now they as well as reviewers all have seen the tentative
> cgit name, saw the reaction it caused, and now know that not just
> potentially confusing other project _exists_, but it does matter.
> 
> Reviewers already have spent some time on suggesting that "git" part
> should not be "c"git, as well as "rs" part may better be "sys",
> etc.?.  There should be _some_ response, even if it does not yet
> propose a new name.
> 
> If it acknowledged that the time and knowledge reviewers gave the
> topic were appreciated, e.g., "The proposers of this topic saw THIS
> point and THAT point as a input that we WILL need to consider when
> we decide on the name.  We acknowledge that the name "cgit-rs" is
> not ideal and needs to be changed.  But we haven't reached any
> concrete alternative name yet, so this round still uses the same
> name", I'd call that "addressed this point", though.
> 
> But just a dismissing "Bikeshed on the name", as if they do not care
> to be mistaken as saying "those who complain about the name are only
> bikeshedding and not worth listening to"?
> 
> We should do better than that.

I am quite surprised that people felt this was dismissive. So to be
clear: yes, we need a new name before this lands in next. I thought that
leaving that as a known needs-work item was sufficient to call that out,
but I guess I was wrong.

As far as proposed names go, I am not super happy with either "libgit"
or "libgit3", because that seems confusing when we consider there are
also a separate libgit2 and libgit2-rs projects. I know that we're using
"libgit_" as a prefix for visible symbols at the moment, but I plan on
renaming those once we have settled on the actual library name.

I was avoiding addressing this topic in the hopes that folks might
suggest additional name options, but as it stands right now, I'm leaning
towards just "git-sys" for the low-level wrapper crate, "git" for the
Rust API, and "git-rs" for the directory in contrib/ which will contain
the crates.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> I was avoiding addressing this topic in the hopes that folks might
> suggest additional name options,...

I've seen libgit-rs (or was it libgit-rust?) suggested already and
found them reasonable.  If libgit2 folks do not use libgit_ prefix,
I do not see any reason for us to avoid it, either.

Thanks.

Re: [PATCH v2 0/5] Introduce cgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

>> But just a dismissing "Bikeshed on the name", as if they do not care
>> to be mistaken as saying "those who complain about the name are only
>> bikeshedding and not worth listening to"?
>> 
>> We should do better than that.
>
> I am quite surprised that people felt this was dismissive. So to be
> clear: yes, we need a new name before this lands in next. I thought that
> leaving that as a known needs-work item was sufficient to call that out,
> but I guess I was wrong.

Yes, I had a similar initial reaction, but I guess that comes
primarily from the fact that we both misjudged how "cgit" is already
deeply established name that refers to something other than this
project.

[PATCH v3 0/6] Introduce libgit-rs, a Rust wrapper around libgit.a

[Calvin Wan]

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/martinvonz/jj

Please find V1 cover letter here:
https://lore.kernel.org/git/cover.1723054623.git.steadmon@google.com/

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml

Known NEEDSWORK:

* Supports older Rust versions by removing any dependencies introduced
  after 1.63.0 (Debian stable)

* Investigate alternative methods of managing symbol visibility.

* Figure out symbol versioning

* Automate the process of exporting additional functions in libgit-sys
  (possibly with a wrapper script around bindgen)

Calvin Wan (3):
  libgit-sys: add repo initialization and config access
  libgit: add higher-level libgit crate
  Makefile: add option to build and test libgit-rs and libgit-rs-sys

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  config: add git_configset_alloc() and git_configset_clear_and_free()

 .gitignore                                    |   2 +
 Makefile                                      |  31 +++++
 common-exit.c                                 |  26 ++++
 common-init.c                                 |  63 +++++++++
 common-init.h                                 |   6 +
 common-main.c                                 |  83 +-----------
 config.c                                      |  11 ++
 config.h                                      |  10 ++
 contrib/libgit-rs/Cargo.lock                  |  62 +++++++++
 contrib/libgit-rs/Cargo.toml                  |  10 ++
 contrib/libgit-rs/libgit-sys/Cargo.lock       |  55 ++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       |  12 ++
 contrib/libgit-rs/libgit-sys/README.md        |  15 +++
 contrib/libgit-rs/libgit-sys/build.rs         |  31 +++++
 .../libgit-sys/public_symbol_export.c         |  76 +++++++++++
 .../libgit-sys/public_symbol_export.h         |  28 ++++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 124 ++++++++++++++++++
 contrib/libgit-rs/src/lib.rs                  |  85 ++++++++++++
 contrib/libgit-rs/testdata/config1            |   2 +
 contrib/libgit-rs/testdata/config2            |   2 +
 contrib/libgit-rs/testdata/config3            |   2 +
 t/Makefile                                    |  16 +++
 22 files changed, 671 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

-- 
2.46.0.469.g59c65b2a67-goog

[PATCH v3 1/6] common-main: split init and exit code into new files

[Calvin Wan]

From: Josh Steadmon <steadmon@google.com>

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Change-Id: I3d9b23ac28a50a4b79760987a3f37b0c6f4cbf9c
Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 5 files changed, 99 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 3eab701b10..7caeb3c872 100644
--- a/Makefile
+++ b/Makefile
@@ -979,6 +979,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.46.0.469.g59c65b2a67-goog

[PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Calvin Wan]

From: Josh Steadmon <steadmon@google.com>

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Change-Id: Ibb98a3907e69839231355c098b2488ef06d1ba45
---
 .gitignore                                    |  1 +
 Makefile                                      | 13 +++++
 contrib/libgit-rs/libgit-sys/Cargo.lock       | 55 +++++++++++++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       | 12 ++++
 contrib/libgit-rs/libgit-sys/README.md        | 15 +++++
 contrib/libgit-rs/libgit-sys/build.rs         | 31 +++++++++++
 .../libgit-sys/public_symbol_export.c         | 20 +++++++
 .../libgit-sys/public_symbol_export.h         |  8 +++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 42 ++++++++++++++
 9 files changed, 197 insertions(+)
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index 8caf3700c2..dfd72820fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,3 +248,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 7caeb3c872..0090514e55 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2713,6 +2715,7 @@ OBJECTS += $(XDIFF_OBJS)
 OBJECTS += $(FUZZ_OBJS)
 OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
+OBJECTS += contrib/libgit-rs/libgit-sys/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3720,6 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-rs/libgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3865,3 +3869,12 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
 build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.lock b/contrib/libgit-rs/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..af8676ab42
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.lock
@@ -0,0 +1,55 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "libz-sys",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.toml b/contrib/libgit-rs/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..905fd6774f
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.toml
@@ -0,0 +1,12 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
\ No newline at end of file
diff --git a/contrib/libgit-rs/libgit-sys/README.md b/contrib/libgit-rs/libgit-sys/README.md
new file mode 100644
index 0000000000..7a59602c30
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/README.md
@@ -0,0 +1,15 @@
+# cgit-info
+
+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
+library.
+
+## Building
+
+`cargo build` automatically builds and picks up on changes made to both
+the Rust wrapper and git.git code so there is no need to run `make`
+beforehand.
+
+## Running
+
+Assuming you don't make any changes to the Git source, you can just work from
+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
new file mode 100644
index 0000000000..c187ab6cd0
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/build.rs
@@ -0,0 +1,31 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = std::process::Command::new("make")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/libgit-rs/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo::rustc-link-search=native={}", dst.display());
+    println!("cargo::rustc-link-lib=gitpub");
+    println!("cargo::rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..39c27d9c1a
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -0,0 +1,20 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoid conflicts with other libraries such as libgit2.
+
+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..346b26083d
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -0,0 +1,42 @@
+use std::ffi::c_char;
+
+extern crate libz_sys;
+
+extern "C" {
+    // From version.c
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Eric Sunshine]

On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> From: Josh Steadmon <steadmon@google.com>

This is curious...

> Co-authored-by: Kyle Lippincott <spectral@google.com>
> Co-authored-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Kyle Lippincott <spectral@google.com>

... since there is no mention of Josh here.

> diff --git a/contrib/libgit-rs/libgit-sys/Cargo.toml b/contrib/libgit-rs/libgit-sys/Cargo.toml
> @@ -0,0 +1,12 @@
> +[package]
> +name = "libgit-sys"
> [...]
> +[dependencies]
> +libz-sys = "1.1.19"
> \ No newline at end of file

Let's give this file a proper line terminator.

> diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
> @@ -0,0 +1,31 @@
> +pub fn main() -> std::io::Result<()> {
> +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
> +    let git_root = crate_root.join("../../..");
> +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
> +
> +    let make_output = std::process::Command::new("make")

Providing a mechanism for people to override this hardcoded spelling
of "make" could be another item for your NEEDSWORK list; in
particular, I'm thinking about platforms on which GNU "make" is
installed as "gmake".

> diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> @@ -0,0 +1,20 @@
> +// Shim to publicly export Git symbols. These must be renamed so that the
> +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> +// avoid conflicts with other libraries such as libgit2.

s/avoid/avoids/

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Mike Hommey]

On Fri, Sep 06, 2024 at 06:39:17PM -0400, Eric Sunshine wrote:
> > diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
> > @@ -0,0 +1,31 @@
> > +pub fn main() -> std::io::Result<()> {
> > +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
> > +    let git_root = crate_root.join("../../..");
> > +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
> > +
> > +    let make_output = std::process::Command::new("make")
> 
> Providing a mechanism for people to override this hardcoded spelling
> of "make" could be another item for your NEEDSWORK list; in
> particular, I'm thinking about platforms on which GNU "make" is
> installed as "gmake".

And/or, use the `make_cmd` crate.

Mike

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> On Fri, Sep 06, 2024 at 06:39:17PM -0400, Eric Sunshine wrote:
>> > diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
>> > @@ -0,0 +1,31 @@
>> > +pub fn main() -> std::io::Result<()> {
>> > +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
>> > +    let git_root = crate_root.join("../../..");
>> > +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
>> > +
>> > +    let make_output = std::process::Command::new("make")
>> 
>> Providing a mechanism for people to override this hardcoded spelling
>> of "make" could be another item for your NEEDSWORK list; in
>> particular, I'm thinking about platforms on which GNU "make" is
>> installed as "gmake".
>
> And/or, use the `make_cmd` crate.

Thanks for helping.

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.09.07 08:04, Mike Hommey wrote:
> On Fri, Sep 06, 2024 at 06:39:17PM -0400, Eric Sunshine wrote:
> > > diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
> > > @@ -0,0 +1,31 @@
> > > +pub fn main() -> std::io::Result<()> {
> > > +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
> > > +    let git_root = crate_root.join("../../..");
> > > +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
> > > +
> > > +    let make_output = std::process::Command::new("make")
> > 
> > Providing a mechanism for people to override this hardcoded spelling
> > of "make" could be another item for your NEEDSWORK list; in
> > particular, I'm thinking about platforms on which GNU "make" is
> > installed as "gmake".
> 
> And/or, use the `make_cmd` crate.

Will do, thanks!

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Calvin Wan]

Thanks for the catches -- will fix in the next reroll

Re: [PATCH v3 2/6] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2024.09.06 18:39, Eric Sunshine wrote:
> On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> > From: Josh Steadmon <steadmon@google.com>
> 
> This is curious...
> 
> > Co-authored-by: Kyle Lippincott <spectral@google.com>
> > Co-authored-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Kyle Lippincott <spectral@google.com>
> 
> ... since there is no mention of Josh here.
> 
> > diff --git a/contrib/libgit-rs/libgit-sys/Cargo.toml b/contrib/libgit-rs/libgit-sys/Cargo.toml
> > @@ -0,0 +1,12 @@
> > +[package]
> > +name = "libgit-sys"
> > [...]
> > +[dependencies]
> > +libz-sys = "1.1.19"
> > \ No newline at end of file
> 
> Let's give this file a proper line terminator.

Fixed in V4.

> > diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
> > @@ -0,0 +1,31 @@
> > +pub fn main() -> std::io::Result<()> {
> > +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
> > +    let git_root = crate_root.join("../../..");
> > +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
> > +
> > +    let make_output = std::process::Command::new("make")
> 
> Providing a mechanism for people to override this hardcoded spelling
> of "make" could be another item for your NEEDSWORK list; in
> particular, I'm thinking about platforms on which GNU "make" is
> installed as "gmake".

Ack, will use make_cmd as Mike suggests.

> > diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> > @@ -0,0 +1,20 @@
> > +// Shim to publicly export Git symbols. These must be renamed so that the
> > +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> > +// avoid conflicts with other libraries such as libgit2.
> 
> s/avoid/avoids/

Fixed in V4.

Thanks for the review!

[PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Calvin Wan]

Wrap a few repo setup and config access functions in libgit-sys. These
were selected as proof-of-concept items to show that we can access local
config from Rust.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Change-Id: I6dd886af8c63e1f0f3251064cd8903aecdf768bb
---
 .../libgit-sys/public_symbol_export.c         | 26 +++++++++
 .../libgit-sys/public_symbol_export.h         |  8 +++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 57 ++++++++++++++++++-
 3 files changed, 89 insertions(+), 2 deletions(-)

diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 39c27d9c1a..65d1620d28 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -2,11 +2,37 @@
 // original symbols can be hidden. Renaming these with a "libgit_" prefix also
 // avoid conflicts with other libraries such as libgit2.
 
+#include "git-compat-util.h"
 #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+#include "common-init.h"
+#include "config.h"
+#include "setup.h"
 #include "version.h"
 
+extern struct repository *the_repository;
+
 #pragma GCC visibility push(default)
 
+const char *libgit_setup_git_directory(void)
+{
+	return setup_git_directory();
+}
+
+int libgit_config_get_int(const char *key, int *dest)
+{
+	return repo_config_get_int(the_repository, key, dest);
+}
+
+void libgit_init_git(const char **argv)
+{
+	init_git(argv);
+}
+
+int libgit_parse_maybe_bool(const char *val)
+{
+	return git_parse_maybe_bool(val);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
index a3372f93fa..64332f30de 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -1,6 +1,14 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+const char *libgit_setup_git_directory(void);
+
+int libgit_config_get_int(const char *key, int *dest);
+
+void libgit_init_git(const char **argv);
+
+int libgit_parse_maybe_bool(const char *val);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index 346b26083d..9b2e85dff8 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,8 +1,19 @@
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 extern crate libz_sys;
 
 extern "C" {
+    pub fn libgit_setup_git_directory() -> *const c_char;
+
+    // From config.c
+    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) -> c_int;
+
+    // From common-init.c
+    pub fn libgit_init_git(argv: *const *const c_char);
+
+    // From parse.c
+    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;
+
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
@@ -10,7 +21,7 @@ extern "C" {
 
 #[cfg(test)]
 mod tests {
-    use std::ffi::CStr;
+    use std::ffi::{CStr, CString};
 
     use super::*;
 
@@ -39,4 +50,46 @@ mod tests {
             agent
         );
     }
+
+    #[test]
+    fn parse_bools_from_strings() {
+        let arg = CString::new("true").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
+
+        let arg = CString::new("yes").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 1);
+
+        let arg = CString::new("false").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
+
+        let arg = CString::new("no").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, 0);
+
+        let arg = CString::new("maybe").unwrap();
+        assert_eq!(unsafe { libgit_parse_maybe_bool(arg.as_ptr()) }, -1);
+    }
+
+    #[test]
+    fn access_configs() {
+        /*
+         * Since we can't easily set a config for the local repo, test
+         * for the existence of a key rather than a specific value 
+         */
+        let fake_argv = [std::ptr::null::<c_char>()];
+        unsafe {
+            libgit_init_git(fake_argv.as_ptr());
+            libgit_setup_git_directory();
+        }
+        let mut dest: c_int = 0;
+        let key = CString::new("core.repositoryformatversion").unwrap();
+        unsafe { 
+            let val = libgit_config_get_int(key.as_ptr(), &mut dest as *mut i32);
+            assert_eq!(val, 0)
+        };
+        let missing_key = CString::new("foo.bar").unwrap();
+        unsafe { 
+            let val = libgit_config_get_int(missing_key.as_ptr(), &mut dest as *mut i32);
+            assert_eq!(val, 1)
+        };
+    }
 }
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Eric Sunshine]

On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> Wrap a few repo setup and config access functions in libgit-sys. These
> were selected as proof-of-concept items to show that we can access local
> config from Rust.
>
> Co-authored-by: Josh Steadmon <steadmon@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>

Josh's sign-off is missing?

> ---
> diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
> @@ -1,8 +1,19 @@
>  extern "C" {
> +    pub fn libgit_setup_git_directory() -> *const c_char;
> +
> +    // From config.c
> +    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) -> c_int;

Perhaps add a comment above libgit_setup_git_directory() stating its
origin, as you do for the other functions mentioned here?

    // From setup.c
    pub fn libgit_setup_git_directory() -> *const c_char;

(Nit: I would probably drop the word "From" from these comments, as it
doesn't seem to add value and ends up being noise. Even better, drop
the comments altogether since they don't really add value and can
easily become outdated if code is ever moved around.)

> +
> +    // From common-init.c
> +    pub fn libgit_init_git(argv: *const *const c_char);
> +
> +    // From parse.c
> +    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Josh Steadmon]

On 2024.09.06 18:53, Eric Sunshine wrote:
> On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> > Wrap a few repo setup and config access functions in libgit-sys. These
> > were selected as proof-of-concept items to show that we can access local
> > config from Rust.
> >
> > Co-authored-by: Josh Steadmon <steadmon@google.com>
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> 
> Josh's sign-off is missing?
> 
> > ---
> > diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
> > @@ -1,8 +1,19 @@
> >  extern "C" {
> > +    pub fn libgit_setup_git_directory() -> *const c_char;
> > +
> > +    // From config.c
> > +    pub fn libgit_config_get_int(key: *const c_char, dest: *mut c_int) -> c_int;
> 
> Perhaps add a comment above libgit_setup_git_directory() stating its
> origin, as you do for the other functions mentioned here?
> 
>     // From setup.c
>     pub fn libgit_setup_git_directory() -> *const c_char;
> 
> (Nit: I would probably drop the word "From" from these comments, as it
> doesn't seem to add value and ends up being noise. Even better, drop
> the comments altogether since they don't really add value and can
> easily become outdated if code is ever moved around.)

Removed in V4.

> > +
> > +    // From common-init.c
> > +    pub fn libgit_init_git(argv: *const *const c_char);
> > +
> > +    // From parse.c
> > +    pub fn libgit_parse_maybe_bool(val: *const c_char) -> c_int;

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Junio C Hamano]

Calvin Wan <calvinwan@google.com> writes:

> Wrap a few repo setup and config access functions in libgit-sys. These
> were selected as proof-of-concept items to show that we can access local
> config from Rust.
>
> Co-authored-by: Josh Steadmon <steadmon@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Change-Id: I6dd886af8c63e1f0f3251064cd8903aecdf768bb
> ---


Common to all other steps, I suspect that you meant to but forgot to
strip out the Change-Id: thing here?

Also there are a few whitespace breakages in this step.

Applying: libgit-sys: add repo initialization and config access
.git/rebase-apply/patch:129: trailing whitespace.
         * for the existence of a key rather than a specific value
.git/rebase-apply/patch:138: trailing whitespace.
        unsafe {
.git/rebase-apply/patch:143: trailing whitespace.
        unsafe {
warning: 3 lines add whitespace errors.


There is another one in a later step.

Applying: libgit: add higher-level libgit crate
.git/rebase-apply/patch:325: trailing whitespace.
        // ConfigSet retrieves correct value
warning: 1 line adds whitespace errors.

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Josh Steadmon]

On 2024.09.06 16:45, Junio C Hamano wrote:
> Calvin Wan <calvinwan@google.com> writes:
> 
> > Wrap a few repo setup and config access functions in libgit-sys. These
> > were selected as proof-of-concept items to show that we can access local
> > config from Rust.
> >
> > Co-authored-by: Josh Steadmon <steadmon@google.com>
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> > Change-Id: I6dd886af8c63e1f0f3251064cd8903aecdf768bb
> > ---
> 
> 
> Common to all other steps, I suspect that you meant to but forgot to
> strip out the Change-Id: thing here?
> 
> Also there are a few whitespace breakages in this step.
> 
> Applying: libgit-sys: add repo initialization and config access
> .git/rebase-apply/patch:129: trailing whitespace.
>          * for the existence of a key rather than a specific value
> .git/rebase-apply/patch:138: trailing whitespace.
>         unsafe {
> .git/rebase-apply/patch:143: trailing whitespace.
>         unsafe {
> warning: 3 lines add whitespace errors.
> 
> 
> There is another one in a later step.
> 
> Applying: libgit: add higher-level libgit crate
> .git/rebase-apply/patch:325: trailing whitespace.
>         // ConfigSet retrieves correct value
> warning: 1 line adds whitespace errors.

Fixed the whitespace errors in V4.

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Patrick Steinhardt]

On Fri, Sep 06, 2024 at 10:21:13PM +0000, Calvin Wan wrote:
> diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> index 39c27d9c1a..65d1620d28 100644
> --- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> +++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> @@ -2,11 +2,37 @@
>  // original symbols can be hidden. Renaming these with a "libgit_" prefix also
>  // avoid conflicts with other libraries such as libgit2.
>  
> +#include "git-compat-util.h"
>  #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
> +#include "common-init.h"
> +#include "config.h"
> +#include "setup.h"
>  #include "version.h"
>  
> +extern struct repository *the_repository;
> +
>  #pragma GCC visibility push(default)
>  
> +const char *libgit_setup_git_directory(void)
> +{
> +	return setup_git_directory();
> +}
> +
> +int libgit_config_get_int(const char *key, int *dest)
> +{
> +	return repo_config_get_int(the_repository, key, dest);
> +}
> +
> +void libgit_init_git(const char **argv)
> +{
> +	init_git(argv);
> +}
> +
> +int libgit_parse_maybe_bool(const char *val)
> +{
> +	return git_parse_maybe_bool(val);
> +}
> +

I don't quite get why we expose functionality that is inherently not
libified. Exposing the current state to library users certainly does not
feel right to me, doubly so because `the_repository` is deprecated and
will eventually go away. So we already know that we'll have to break the
API here once that has happened. I'd rather want to see that introducing
the library makes us double down on providing properly encapsulated
interfaces from hereon.

Also, we already have ways to initialize a repository and read their
config without relying on `the_repository`. So shouldn't we expose that
instead?

Patrick

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Josh Steadmon]

On 2024.09.10 08:42, Patrick Steinhardt wrote:
> On Fri, Sep 06, 2024 at 10:21:13PM +0000, Calvin Wan wrote:
> > diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> > index 39c27d9c1a..65d1620d28 100644
> > --- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> > +++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> > @@ -2,11 +2,37 @@
> >  // original symbols can be hidden. Renaming these with a "libgit_" prefix also
> >  // avoid conflicts with other libraries such as libgit2.
> >  
> > +#include "git-compat-util.h"
> >  #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
> > +#include "common-init.h"
> > +#include "config.h"
> > +#include "setup.h"
> >  #include "version.h"
> >  
> > +extern struct repository *the_repository;
> > +
> >  #pragma GCC visibility push(default)
> >  
> > +const char *libgit_setup_git_directory(void)
> > +{
> > +	return setup_git_directory();
> > +}
> > +
> > +int libgit_config_get_int(const char *key, int *dest)
> > +{
> > +	return repo_config_get_int(the_repository, key, dest);
> > +}
> > +
> > +void libgit_init_git(const char **argv)
> > +{
> > +	init_git(argv);
> > +}
> > +
> > +int libgit_parse_maybe_bool(const char *val)
> > +{
> > +	return git_parse_maybe_bool(val);
> > +}
> > +
> 
> I don't quite get why we expose functionality that is inherently not
> libified. Exposing the current state to library users certainly does not
> feel right to me, doubly so because `the_repository` is deprecated and
> will eventually go away. So we already know that we'll have to break the
> API here once that has happened. I'd rather want to see that introducing
> the library makes us double down on providing properly encapsulated
> interfaces from hereon.
> 
> Also, we already have ways to initialize a repository and read their
> config without relying on `the_repository`. So shouldn't we expose that
> instead?
> 
> Patrick

Specifically in this case, yeah, we should have started with the
libified version. This is an artifact due to the way we are figuring out
C/Rust interop as we go, and it was easier to convert it this way
without making the Rust side care about handling repository pointers.
But you're right, and I'll fix this soon for V4.

In general though, we're treating the initial version of libgit-rs as a
proof-of-concept that we can call from JJ into Git without blowing
things up. We might not always have the option of exposing
fully-libified code paths, and for our $DAYJOB purposes, we may have to
deal with non-ideal interfaces for the early versions. However, we do
want to use this as a way to motivate development of better, libified
APIs when we find real-world use cases for them.

We've said before (even before we started libgit-rs) that we're not
going to be able to make guarantees about early libified APIs, because
we're learning as we go along. I don't want to use that as an excuse to
cover up bad design, but I do want to be upfront that we can't commit to
fully libifying any given code path before we expose it through the shim
library or through libgit-rs.

Re: [PATCH v3 3/6] libgit-sys: add repo initialization and config access

[Josh Steadmon]

On 2024.10.07 14:21, Josh Steadmon wrote:
> On 2024.09.10 08:42, Patrick Steinhardt wrote:
> > I don't quite get why we expose functionality that is inherently not
> > libified. Exposing the current state to library users certainly does not
> > feel right to me, doubly so because `the_repository` is deprecated and
> > will eventually go away. So we already know that we'll have to break the
> > API here once that has happened. I'd rather want to see that introducing
> > the library makes us double down on providing properly encapsulated
> > interfaces from hereon.
> > 
> > Also, we already have ways to initialize a repository and read their
> > config without relying on `the_repository`. So shouldn't we expose that
> > instead?
> > 
> > Patrick
> 
> Specifically in this case, yeah, we should have started with the
> libified version. This is an artifact due to the way we are figuring out
> C/Rust interop as we go, and it was easier to convert it this way
> without making the Rust side care about handling repository pointers.
> But you're right, and I'll fix this soon for V4.
> 
> In general though, we're treating the initial version of libgit-rs as a
> proof-of-concept that we can call from JJ into Git without blowing
> things up. We might not always have the option of exposing
> fully-libified code paths, and for our $DAYJOB purposes, we may have to
> deal with non-ideal interfaces for the early versions. However, we do
> want to use this as a way to motivate development of better, libified
> APIs when we find real-world use cases for them.
> 
> We've said before (even before we started libgit-rs) that we're not
> going to be able to make guarantees about early libified APIs, because
> we're learning as we go along. I don't want to use that as an excuse to
> cover up bad design, but I do want to be upfront that we can't commit to
> fully libifying any given code path before we expose it through the shim
> library or through libgit-rs.

In fact, since the JJ proof-of-concept doesn't rely on repository
initialization or repository-level config access, I think we can just
drop this patch for now and worry about getting a better interface for
repo initialization later.

[PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Calvin Wan]

From: Josh Steadmon <steadmon@google.com>

Add git_configset_alloc() and git_configset_clear_and_free() functions
so that callers can manage config_set structs on the heap. This also
allows non-C external consumers to treat config_sets as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Change-Id: I21684c87691d978e3303e04324428aa3567dbd70
---
 config.c | 11 +++++++++++
 config.h | 10 ++++++++++
 2 files changed, 21 insertions(+)

diff --git a/config.c b/config.c
index 6421894614..444db8de79 100644
--- a/config.c
+++ b/config.c
@@ -2324,6 +2324,11 @@ static int config_set_element_cmp(const void *cmp_data UNUSED,
 	return strcmp(e1->key, e2->key);
 }
 
+struct config_set *git_configset_alloc(void)
+{
+	return xmalloc(sizeof(struct config_set));
+}
+
 void git_configset_init(struct config_set *set)
 {
 	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
@@ -2353,6 +2358,12 @@ void git_configset_clear(struct config_set *set)
 	set->list.items = NULL;
 }
 
+void git_configset_clear_and_free(struct config_set *set)
+{
+	git_configset_clear(set);
+	free(set);
+}
+
 static int config_set_callback(const char *key, const char *value,
 			       const struct config_context *ctx,
 			       void *cb)
diff --git a/config.h b/config.h
index 54b47dec9e..074c85a788 100644
--- a/config.h
+++ b/config.h
@@ -472,6 +472,11 @@ struct config_set {
 	struct configset_list list;
 };
 
+/**
+ * Alloc a config_set
+ */
+struct config_set *git_configset_alloc(void);
+
 /**
  * Initializes the config_set `cs`.
  */
@@ -520,6 +525,11 @@ int git_configset_get_string_multi(struct config_set *cs, const char *key,
  */
 void git_configset_clear(struct config_set *cs);
 
+/**
+ * Clears and frees a heap-allocated `config_set` structure.
+ */
+void git_configset_clear_and_free(struct config_set *cs);
+
 /*
  * These functions return 1 if not found, and 0 if found, leaving the found
  * value in the 'dest' pointer.
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Eric Sunshine]

On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> From: Josh Steadmon <steadmon@google.com>
>
> Add git_configset_alloc() and git_configset_clear_and_free() functions
> so that callers can manage config_set structs on the heap. This also
> allows non-C external consumers to treat config_sets as opaque structs.
>
> Co-authored-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>

Almost all these patches suffer from some sort of missing Josh
trailer, so I'll stop mentioning it now.

> diff --git a/config.h b/config.h
> @@ -472,6 +472,11 @@ struct config_set {
> +/**
> + * Alloc a config_set
> + */
> +struct config_set *git_configset_alloc(void);

Should this documentation string mention that git_configset_alloc()
does _not_ initialize the configset? Alternatively, should this
function also initialize it as a convenience (and mention so in the
documentation)?

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Patrick Steinhardt]

On Fri, Sep 06, 2024 at 07:24:22PM -0400, Eric Sunshine wrote:
> On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> > From: Josh Steadmon <steadmon@google.com>
> >
> > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > so that callers can manage config_set structs on the heap. This also
> > allows non-C external consumers to treat config_sets as opaque structs.
> >
> > Co-authored-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> 
> Almost all these patches suffer from some sort of missing Josh
> trailer, so I'll stop mentioning it now.
> 
> > diff --git a/config.h b/config.h
> > @@ -472,6 +472,11 @@ struct config_set {
> > +/**
> > + * Alloc a config_set
> > + */
> > +struct config_set *git_configset_alloc(void);
> 
> Should this documentation string mention that git_configset_alloc()
> does _not_ initialize the configset? Alternatively, should this
> function also initialize it as a convenience (and mention so in the
> documentation)?

I'd think so. I don't quite see the point in splitting this up into two
separate function calls. Is there ever a case where one wants to
allocate the configset, but not use it?

Patrick

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Phillip Wood]

On 10/09/2024 07:41, Patrick Steinhardt wrote:
> Is there ever a case where one wants to
> allocate the configset, but not use it?

That was my thought too - I suggested providing git_configset_new() that 
would allocate and initialize a config set in my response to the last 
round [1]. It is good to see that the struct in now namespaced in the 
next patch but separating out allocation and initialization makes the 
api harder to use than it needs to be. I'd also like to see 
git_configset_clear_and_free() become git_configset_free().

Best Wishes

Phillip

[1] 
https://lore.kernel.org/git/47b18fa4-f01b-4f42-8d04-9e145515ccc1@gmail.com

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> On 10/09/2024 07:41, Patrick Steinhardt wrote:
>> Is there ever a case where one wants to
>> allocate the configset, but not use it?
>
> That was my thought too - I suggested providing git_configset_new()
> that would allocate and initialize a config set in my response to the
> last round [1]. It is good to see that the struct in now namespaced in
> the next patch but separating out allocation and initialization makes
> the api harder to use than it needs to be. I'd also like to see
> git_configset_clear_and_free() become git_configset_free().

Makes sense.

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Calvin Wan]

On Tue, Sep 10, 2024 at 1:50 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>
> On 10/09/2024 07:41, Patrick Steinhardt wrote:
> > Is there ever a case where one wants to
> > allocate the configset, but not use it?
>
> That was my thought too - I suggested providing git_configset_new() that
> would allocate and initialize a config set in my response to the last
> round [1]. It is good to see that the struct in now namespaced in the
> next patch but separating out allocation and initialization makes the
> api harder to use than it needs to be. I'd also like to see
> git_configset_clear_and_free() become git_configset_free().
>
> Best Wishes
>
> Phillip
>
> [1]
> https://lore.kernel.org/git/47b18fa4-f01b-4f42-8d04-9e145515ccc1@gmail.com

Agreed, it doesn't make sense to have both alloc() and init() when
they can be combined and same with renaming
git_configset_clear_and_free() to git_configset_free().

Re: [PATCH v3 4/6] config: add git_configset_alloc() and git_configset_clear_and_free()

[Josh Steadmon]

On 2024.09.06 19:24, Eric Sunshine wrote:
> On Fri, Sep 6, 2024 at 6:21 PM Calvin Wan <calvinwan@google.com> wrote:
> > From: Josh Steadmon <steadmon@google.com>
> >
> > Add git_configset_alloc() and git_configset_clear_and_free() functions
> > so that callers can manage config_set structs on the heap. This also
> > allows non-C external consumers to treat config_sets as opaque structs.
> >
> > Co-authored-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> 
> Almost all these patches suffer from some sort of missing Josh
> trailer, so I'll stop mentioning it now.
> 
> > diff --git a/config.h b/config.h
> > @@ -472,6 +472,11 @@ struct config_set {
> > +/**
> > + * Alloc a config_set
> > + */
> > +struct config_set *git_configset_alloc(void);
> 
> Should this documentation string mention that git_configset_alloc()
> does _not_ initialize the configset? Alternatively, should this
> function also initialize it as a convenience (and mention so in the
> documentation)?

Yes, in V4 this will both alloc and init the configset (and the function
has been moved to the shim library).

[PATCH v3 5/6] libgit: add higher-level libgit crate

[Calvin Wan]

Wrap `struct config_set` and a few of its associated functions in
libgit-sys. Also introduce a higher-level "libgit" crate which provides
a more Rust-friendly interface to config_set structs.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Change-Id: If85e77809434f0ef05e2db35132ecf84621e10ef
---
 .gitignore                                    |  1 +
 Makefile                                      |  2 +-
 contrib/libgit-rs/Cargo.lock                  | 62 ++++++++++++++
 contrib/libgit-rs/Cargo.toml                  | 10 +++
 .../libgit-sys/public_symbol_export.c         | 30 +++++++
 .../libgit-sys/public_symbol_export.h         | 12 +++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 31 ++++++-
 contrib/libgit-rs/src/lib.rs                  | 85 +++++++++++++++++++
 contrib/libgit-rs/testdata/config1            |  2 +
 contrib/libgit-rs/testdata/config2            |  2 +
 contrib/libgit-rs/testdata/config3            |  2 +
 11 files changed, 237 insertions(+), 2 deletions(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index dfd72820fb..0a42f27117 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,4 +248,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 0090514e55..abeee01d9e 100644
--- a/Makefile
+++ b/Makefile
@@ -3723,7 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-rs/libgit-sys/target
+	$(RM) -r contrib/libgit-rs/target contrib/libgit-rs/libgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..187176f5df
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,62 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "libz-sys",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..b8914517e0
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "libgit-sys" }
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 65d1620d28..07d6bfdd84 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -33,6 +33,36 @@ int libgit_parse_maybe_bool(const char *val)
 	return git_parse_maybe_bool(val);
 }
 
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	return git_configset_alloc();
+}
+
+void libgit_configset_clear_and_free(struct libgit_config_set *cs)
+{
+	git_configset_clear_and_free(cs);
+}
+
+void libgit_configset_init(struct libgit_config_set *cs)
+{
+	git_configset_init(cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file(cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int(cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string(cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
index 64332f30de..3933698976 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -9,6 +9,18 @@ void libgit_init_git(const char **argv);
 
 int libgit_parse_maybe_bool(const char *val);
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_clear_and_free(struct libgit_config_set *cs);
+
+void libgit_configset_init(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index 9b2e85dff8..f3eb44a238 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,8 +1,17 @@
-use std::ffi::{c_char, c_int};
+use std::ffi::{c_char, c_int, c_void};
+
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
 
 extern crate libz_sys;
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_setup_git_directory() -> *const c_char;
 
     // From config.c
@@ -17,6 +26,26 @@ extern "C" {
     // From version.c
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_clear_and_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_init(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..43aae09656
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1,85 @@
+use std::ffi::{c_char, c_int, c_void, CStr, CString};
+use std::path::Path;
+
+use libgit_sys::*;
+
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    pub fn new() -> Self {
+        unsafe {
+            let ptr = libgit_configset_alloc();
+            libgit_configset_init(ptr);
+            ConfigSet(ptr)
+        }
+    }
+
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr: &str = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(&*pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val)
+    }
+
+    pub fn get_str(&mut self, key: &str) -> Option<CString> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str = CString::from_vec_with_nul(borrowed_str.to_bytes_with_nul().to_vec());
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str.unwrap())
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_clear_and_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[Path::new("testdata/config1"),
+                       Path::new("testdata/config2"),
+                       Path::new("testdata/config3")]);
+        // ConfigSet retrieves correct value 
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_str("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Junio C Hamano]

Calvin Wan <calvinwan@google.com> writes:

> diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> index 65d1620d28..07d6bfdd84 100644
> --- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> +++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
> @@ -33,6 +33,36 @@ int libgit_parse_maybe_bool(const char *val)
>  	return git_parse_maybe_bool(val);
>  }
>  
> +struct libgit_config_set *libgit_configset_alloc(void)
> +{
> +	return git_configset_alloc();
> +}

git_configset_alloc() returns "struct config_set *" while this thing
returns an incompatible pointer.  

Sent out an outdated version or something?  This wouldn't have
passed even a compile test, I suspect.



$ make contrib/libgit-rs/libgit-sys/public_symbol_export.o
    CC contrib/libgit-rs/libgit-sys/public_symbol_export.o
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_alloc':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:38:16: error: returning 'struct config_set *' from a function with incompatible return type 'struct libgit_config_set *' [-Werror=incompatible-pointer-types]
   38 |         return git_configset_alloc();
      |                ^~~~~~~~~~~~~~~~~~~~~
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_clear_and_free':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:43:38: error: passing argument 1 of 'git_configset_clear_and_free' from incompatible pointer type [-Werror=incompatible-pointer-types]
   43 |         git_configset_clear_and_free(cs);
      |                                      ^~
      |                                      |
      |                                      struct libgit_config_set *
In file included from contrib/libgit-rs/libgit-sys/public_symbol_export.c:8:
./config.h:543:54: note: expected 'struct config_set *' but argument is of type 'struct libgit_config_set *'
  543 | void git_configset_clear_and_free(struct config_set *cs);
      |                                   ~~~~~~~~~~~~~~~~~~~^~
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_init':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:48:28: error: passing argument 1 of 'git_configset_init' from incompatible pointer type [-Werror=incompatible-pointer-types]
   48 |         git_configset_init(cs);
      |                            ^~
      |                            |
      |                            struct libgit_config_set *
./config.h:495:44: note: expected 'struct config_set *' but argument is of type 'struct libgit_config_set *'
  495 | void git_configset_init(struct config_set *cs);
      |                         ~~~~~~~~~~~~~~~~~~~^~
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_add_file':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:53:39: error: passing argument 1 of 'git_configset_add_file' from incompatible pointer type [-Werror=incompatible-pointer-types]
   53 |         return git_configset_add_file(cs, filename);
      |                                       ^~
      |                                       |
      |                                       struct libgit_config_set *
./config.h:504:47: note: expected 'struct config_set *' but argument is of type 'struct libgit_config_set *'
  504 | int git_configset_add_file(struct config_set *cs, const char *filename);
      |                            ~~~~~~~~~~~~~~~~~~~^~
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_get_int':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:58:38: error: passing argument 1 of 'git_configset_get_int' from incompatible pointer type [-Werror=incompatible-pointer-types]
   58 |         return git_configset_get_int(cs, key, dest);
      |                                      ^~
      |                                      |
      |                                      struct libgit_config_set *
./config.h:568:46: note: expected 'struct config_set *' but argument is of type 'struct libgit_config_set *'
  568 | int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
      |                           ~~~~~~~~~~~~~~~~~~~^~
contrib/libgit-rs/libgit-sys/public_symbol_export.c: In function 'libgit_configset_get_string':
contrib/libgit-rs/libgit-sys/public_symbol_export.c:63:41: error: passing argument 1 of 'git_configset_get_string' from incompatible pointer type [-Werror=incompatible-pointer-types]
   63 |         return git_configset_get_string(cs, key, dest);
      |                                         ^~
      |                                         |
      |                                         struct libgit_config_set *
./config.h:567:49: note: expected 'struct config_set *' but argument is of type 'struct libgit_config_set *'
  567 | int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
      |                              ~~~~~~~~~~~~~~~~~~~^~
cc1: all warnings being treated as errors
gmake: *** [Makefile:2802: contrib/libgit-rs/libgit-sys/public_symbol_export.o] Error 1

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>> +struct libgit_config_set *libgit_configset_alloc(void)
>> +{
>> +	return git_configset_alloc();
>> +}
>
> git_configset_alloc() returns "struct config_set *" while this thing
> returns an incompatible pointer.  
>
> Sent out an outdated version or something?  This wouldn't have
> passed even a compile test, I suspect.

The "shim" layer should hide the details of interfacing to the git
proper from callers, as well as it should hide the callers' from the
git proper.  So if you really want to hide "struct config_set" from
your library callers, you may need to do something like the
attached, perhaps?  At least this does pass compilation test.

 .../libgit-rs/libgit-sys/public_symbol_export.h    | 11 +++++++++
 .../libgit-rs/libgit-sys/public_symbol_export.c    | 26 ++++++++++++----------
 2 files changed, 25 insertions(+), 12 deletions(-)

diff --git c/contrib/libgit-rs/libgit-sys/public_symbol_export.h w/contrib/libgit-rs/libgit-sys/public_symbol_export.h
index 3933698976..70701ca63e 100644
--- c/contrib/libgit-rs/libgit-sys/public_symbol_export.h
+++ w/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -1,6 +1,17 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+/* shim */
+struct libgit_config_set {
+	struct config_set cs;
+	/* 
+	 * the shim layer may want to add more members below, but then
+	 * it may need to wrap underlying config_set differently,
+	 * e.g., store a pointer of an allocated config_set in this
+	 * shim structure.
+	 */
+};
+
 const char *libgit_setup_git_directory(void);
 
 int libgit_config_get_int(const char *key, int *dest);

diff --git c/contrib/libgit-rs/libgit-sys/public_symbol_export.c w/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 07d6bfdd84..6f5eb3b249 100644
--- c/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ w/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -3,12 +3,13 @@
 // avoid conflicts with other libraries such as libgit2.
 
 #include "git-compat-util.h"
-#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
 #include "common-init.h"
 #include "config.h"
 #include "setup.h"
 #include "version.h"
 
+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+
 extern struct repository *the_repository;
 
 #pragma GCC visibility push(default)
@@ -35,32 +36,33 @@ int libgit_parse_maybe_bool(const char *val)
 
 struct libgit_config_set *libgit_configset_alloc(void)
 {
-	return git_configset_alloc();
+	void *cs = git_configset_alloc();
+	return (struct libgit_config_set *)cs;
 }
 
-void libgit_configset_clear_and_free(struct libgit_config_set *cs)
+void libgit_configset_clear_and_free(struct libgit_config_set *lgcs)
 {
-	git_configset_clear_and_free(cs);
+	git_configset_clear_and_free(&lgcs->cs);
 }
 
-void libgit_configset_init(struct libgit_config_set *cs)
+void libgit_configset_init(struct libgit_config_set *lgcs)
 {
-	git_configset_init(cs);
+	git_configset_init(&lgcs->cs);
 }
 
-int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+int libgit_configset_add_file(struct libgit_config_set *lgcs, const char *filename)
 {
-	return git_configset_add_file(cs, filename);
+	return git_configset_add_file(&lgcs->cs, filename);
 }
 
-int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+int libgit_configset_get_int(struct libgit_config_set *lgcs, const char *key, int *dest)
 {
-	return git_configset_get_int(cs, key, dest);
+	return git_configset_get_int(&lgcs->cs, key, dest);
 }
 
-int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+int libgit_configset_get_string(struct libgit_config_set *lgcs, const char *key, char **dest)
 {
-	return git_configset_get_string(cs, key, dest);
+	return git_configset_get_string(&lgcs->cs, key, dest);
 }
 
 const char *libgit_user_agent(void)

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Calvin Wan]

On Mon, Sep 9, 2024 at 1:47 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Junio C Hamano <gitster@pobox.com> writes:
>
> >> +struct libgit_config_set *libgit_configset_alloc(void)
> >> +{
> >> +    return git_configset_alloc();
> >> +}
> >
> > git_configset_alloc() returns "struct config_set *" while this thing
> > returns an incompatible pointer.
> >
> > Sent out an outdated version or something?  This wouldn't have
> > passed even a compile test, I suspect.
>
> The "shim" layer should hide the details of interfacing to the git
> proper from callers, as well as it should hide the callers' from the
> git proper.  So if you really want to hide "struct config_set" from
> your library callers, you may need to do something like the
> attached, perhaps?  At least this does pass compilation test.

You're correct that this is supposed to be the shim layer, where
callers aren't supposed to be able to directly interface with the
config_set struct. From Rust's perspective, all pointers that come
back from C code are effectively void* so the naming doesn't make a
functional difference for our use case (and consequently these
warnings did not show up from the build.rs script nor did the tests
fail). However, I agree that the public interface should pass the
compilation test and your approach does that -- will reroll with those
changes and I believe that we should also fix the build.rs so that
warnings also show up during cargo build.

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Junio C Hamano]

Calvin Wan <calvinwan@google.com> writes:

> However, I agree that the public interface should pass the
> compilation test and your approach does that -- will reroll with those
> changes and I believe that we should also fix the build.rs so that
> warnings also show up during cargo build.

Thanks.  I couldn't quite tell if *.c was supposed to be compilable
into *.o directly (if not, then Makefile needs fixing), and I am OK,
if the shim layer is only internally used, if the public.h defined
all pointers as "void *".

I wasn't happy to cast "struct config_set *" between "struct
libgit_config_set *" merely because one embeds the other without
adding any other member.  If they have to be bit-for-bit identical,
shouldn't we just use the real name of the struct everywhere?

The situation might become different if you address others' comments
like "why do we want allocation-only interface exposed?  shoudln't
it also initialize?" etc.

Thanks.

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2024.09.10 12:14, Junio C Hamano wrote:
> Calvin Wan <calvinwan@google.com> writes:
> 
> > However, I agree that the public interface should pass the
> > compilation test and your approach does that -- will reroll with those
> > changes and I believe that we should also fix the build.rs so that
> > warnings also show up during cargo build.
> 
> Thanks.  I couldn't quite tell if *.c was supposed to be compilable
> into *.o directly (if not, then Makefile needs fixing), and I am OK,
> if the shim layer is only internally used, if the public.h defined
> all pointers as "void *".
> 
> I wasn't happy to cast "struct config_set *" between "struct
> libgit_config_set *" merely because one embeds the other without
> adding any other member.  If they have to be bit-for-bit identical,
> shouldn't we just use the real name of the struct everywhere?

We want to namespace types as well as functions, as Phillip pointed out
in 47b18fa4-f01b-4f42-8d04-9e145515ccc1@gmail.com.

Is there a reason why we need the shim struct from your
xmqqcylcpnah.fsf@gitster.g and can't just cast directly like so:

 contrib/libgit-rs/libgit-sys/public_symbol_export.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 07d6bfdd84..c96fa15ab6 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -3,12 +3,13 @@
 // avoid conflicts with other libraries such as libgit2.

 #include "git-compat-util.h"
-#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
 #include "common-init.h"
 #include "config.h"
 #include "setup.h"
 #include "version.h"

+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+
 extern struct repository *the_repository;

 #pragma GCC visibility push(default)
@@ -35,32 +36,32 @@ int libgit_parse_maybe_bool(const char *val)

 struct libgit_config_set *libgit_configset_alloc(void)
 {
-       return git_configset_alloc();
+       return (struct libgit_config_set *) git_configset_alloc();
 }

 void libgit_configset_clear_and_free(struct libgit_config_set *cs)
 {
-       git_configset_clear_and_free(cs);
+       git_configset_clear_and_free((struct config_set *) cs);
 }

 void libgit_configset_init(struct libgit_config_set *cs)
 {
-       git_configset_init(cs);
+       git_configset_init((struct config_set *) cs);
 }

 int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
 {
-       return git_configset_add_file(cs, filename);
+       return git_configset_add_file((struct config_set *) cs, filename);
 }

 int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
 {
-       return git_configset_get_int(cs, key, dest);
+       return git_configset_get_int((struct config_set *) cs, key, dest);
 }

 int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
 {
-       return git_configset_get_string(cs, key, dest);
+       return git_configset_get_string((struct config_set *) cs, key, dest);
 }

 const char *libgit_user_agent(void)

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> We want to namespace types as well as functions, as Phillip pointed out
> in 47b18fa4-f01b-4f42-8d04-9e145515ccc1@gmail.com.
>
> Is there a reason why we need the shim struct from your
> xmqqcylcpnah.fsf@gitster.g and can't just cast directly like so:
> ...
>  int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
>  {
> -       return git_configset_get_string(cs, key, dest);
> +       return git_configset_get_string((struct config_set *) cs, key, dest);
>  }

Not at all.  I just didn't see your intentions in the patch if all
you wanted to do was merely to flip names, or wanted to leave the
possibility to allow the wrapped ones to optionally have different
shape (e.g. for bookkeeping purposes for either the host environment
or the shim layer).  If it is merely "we do not want to expose these
names but we want bit-for-bit identical data", then you do not need
extra logic at all---the casts would be suffficient[*].

PS. I am not feeling well today, so please expect delayed and/or
sparse responses.


[Footnote]

* Building objects that go to libgit.a, partially linking them to
  resolve internal references, and then rewriting the symbol table
  of the resulting relocatable object file to expose only the entry
  points and data you want to show to the rust world to whatever
  names you want, would be a less gross solution, I would imagine.
  You only then need to write a (fake) public_symbol_export.h file
  that will never has to be seen on the C side, but to be seen as
  the header to describe that C library to the rust side (and
  obviously you do not need public_symbol_export.c file only to keep
  these casts).

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2024.09.18 09:34, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > We want to namespace types as well as functions, as Phillip pointed out
> > in 47b18fa4-f01b-4f42-8d04-9e145515ccc1@gmail.com.
> >
> > Is there a reason why we need the shim struct from your
> > xmqqcylcpnah.fsf@gitster.g and can't just cast directly like so:
> > ...
> >  int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
> >  {
> > -       return git_configset_get_string(cs, key, dest);
> > +       return git_configset_get_string((struct config_set *) cs, key, dest);
> >  }
> 
> Not at all.  I just didn't see your intentions in the patch if all
> you wanted to do was merely to flip names, or wanted to leave the
> possibility to allow the wrapped ones to optionally have different
> shape (e.g. for bookkeeping purposes for either the host environment
> or the shim layer).  If it is merely "we do not want to expose these
> names but we want bit-for-bit identical data", then you do not need
> extra logic at all---the casts would be suffficient[*].
> 
> PS. I am not feeling well today, so please expect delayed and/or
> sparse responses.
> 
> 
> [Footnote]
> 
> * Building objects that go to libgit.a, partially linking them to
>   resolve internal references, and then rewriting the symbol table
>   of the resulting relocatable object file to expose only the entry
>   points and data you want to show to the rust world to whatever
>   names you want, would be a less gross solution, I would imagine.
>   You only then need to write a (fake) public_symbol_export.h file
>   that will never has to be seen on the C side, but to be seen as
>   the header to describe that C library to the rust side (and
>   obviously you do not need public_symbol_export.c file only to keep
>   these casts).

I don't think I'll have time to experiment with this approach for V4,
but I'll put it on our list of future work. For now I'm just keeping
public_symbol_export.{c,h} as-is and casting to avoid exposing the
unmangled struct name.

Re: [PATCH v3 5/6] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2024.09.10 12:04, Calvin Wan wrote:
> On Mon, Sep 9, 2024 at 1:47 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Junio C Hamano <gitster@pobox.com> writes:
> >
> > >> +struct libgit_config_set *libgit_configset_alloc(void)
> > >> +{
> > >> +    return git_configset_alloc();
> > >> +}
> > >
> > > git_configset_alloc() returns "struct config_set *" while this thing
> > > returns an incompatible pointer.
> > >
> > > Sent out an outdated version or something?  This wouldn't have
> > > passed even a compile test, I suspect.
> >
> > The "shim" layer should hide the details of interfacing to the git
> > proper from callers, as well as it should hide the callers' from the
> > git proper.  So if you really want to hide "struct config_set" from
> > your library callers, you may need to do something like the
> > attached, perhaps?  At least this does pass compilation test.
> 
> You're correct that this is supposed to be the shim layer, where
> callers aren't supposed to be able to directly interface with the
> config_set struct. From Rust's perspective, all pointers that come
> back from C code are effectively void* so the naming doesn't make a
> functional difference for our use case (and consequently these
> warnings did not show up from the build.rs script nor did the tests
> fail). However, I agree that the public interface should pass the
> compilation test and your approach does that -- will reroll with those
> changes and I believe that we should also fix the build.rs so that
> warnings also show up during cargo build.

Yeah, cargo will complain if DEVELOPER=1 is set. We should probably
enforce that in build.rs.

[PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Calvin Wan]

Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
to their respective Makefiles so they can be built and tested without
having to run cargo build/test.

Add environment variable, INCLUDE_LIBGIT_RS, that when set,
automatically builds and tests libgit-rs and libgit-rs-sys when `make
all` is ran.

Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 Makefile   | 16 ++++++++++++++++
 t/Makefile | 16 ++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/Makefile b/Makefile
index abeee01d9e..41ad458aef 100644
--- a/Makefile
+++ b/Makefile
@@ -3870,6 +3870,22 @@ build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
+.PHONY: libgitrs-sys
+libgitrs-sys:
+	$(QUIET)(\
+		cd contrib/libgit-rs/libgit-sys && \
+		cargo build \
+	)
+.PHONY: libgitrs
+libgitrs:
+	$(QUIET)(\
+		cd contrib/libgit-rs && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs
+endif
+
 contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
 	$(LD) -r $^ -o $@
 
diff --git a/t/Makefile b/t/Makefile
index b2eb9f770b..066cb5c2b4 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -169,3 +169,19 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgitrs-sys-test
+libgitrs-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs/libgit-sys && \
+		cargo test \
+	)
+.PHONY: libgitrs-test
+libgitrs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs-sys-test libgitrs-test
+endif
-- 
2.46.0.469.g59c65b2a67-goog

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Sean Allred]

Calvin Wan <calvinwan@google.com> writes:
> Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> to their respective Makefiles so they can be built and tested without
> having to run cargo build/test.

I feel like clippy should be run as part of these somehow, but I'm not
sure where.

> +libgitrs-sys:
> +	$(QUIET)(\
> +		cd contrib/libgit-rs/libgit-sys && \
> +		cargo build \
> +	)
> +.PHONY: libgitrs
> +libgitrs:
> +	$(QUIET)(\
> +		cd contrib/libgit-rs && \
> +		cargo build \
> +	)

We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
`-Wall` in the C world, no? These crates should build without warnings.

Very excited to see this work; thank you for putting the time into it
:-)

-- 
Sean Allred

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Junio C Hamano]

Sean Allred <allred.sean@gmail.com> writes:

> Calvin Wan <calvinwan@google.com> writes:
>> Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
>> to their respective Makefiles so they can be built and tested without
>> having to run cargo build/test.
>
> I feel like clippy should be run as part of these somehow, but I'm not
> sure where.
>
>> +libgitrs-sys:
>> +	$(QUIET)(\
>> +		cd contrib/libgit-rs/libgit-sys && \
>> +		cargo build \
>> +	)
>> +.PHONY: libgitrs
>> +libgitrs:
>> +	$(QUIET)(\
>> +		cd contrib/libgit-rs && \
>> +		cargo build \
>> +	)
>
> We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
> `-Wall` in the C world, no? These crates should build without warnings.
>
> Very excited to see this work; thank you for putting the time into it
> :-)

Thanks for helping.

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Calvin Wan]

On Sat, Sep 7, 2024 at 8:15 AM Sean Allred <allred.sean@gmail.com> wrote:
>
> Calvin Wan <calvinwan@google.com> writes:
> > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > to their respective Makefiles so they can be built and tested without
> > having to run cargo build/test.
>
> I feel like clippy should be run as part of these somehow, but I'm not
> sure where.
>
> > +libgitrs-sys:
> > +     $(QUIET)(\
> > +             cd contrib/libgit-rs/libgit-sys && \
> > +             cargo build \
> > +     )
> > +.PHONY: libgitrs
> > +libgitrs:
> > +     $(QUIET)(\
> > +             cd contrib/libgit-rs && \
> > +             cargo build \
> > +     )
>
> We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
> `-Wall` in the C world, no? These crates should build without warnings.
>
> Very excited to see this work; thank you for putting the time into it
> :-)
>
> --
> Sean Allred

Thanks for the suggestions! Will look into both for the next reroll.

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1373 bytes --]

On 2024-09-07 at 15:15:12, Sean Allred wrote:
> Calvin Wan <calvinwan@google.com> writes:
> > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > to their respective Makefiles so they can be built and tested without
> > having to run cargo build/test.
> 
> I feel like clippy should be run as part of these somehow, but I'm not
> sure where.

Yes, that seems like a good idea in CI.

> > +libgitrs-sys:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs/libgit-sys && \
> > +		cargo build \
> > +	)
> > +.PHONY: libgitrs
> > +libgitrs:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs && \
> > +		cargo build \
> > +	)
> 
> We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
> `-Wall` in the C world, no? These crates should build without warnings.

I believe -Dwarnings turns warnings into errors (at least it does in my
tests), which is equivalent to -Werror.  We don't want that because it
breaks compiling older code with newer versions of the compiler, which
makes it harder to bisect changes or compiler on the system compiler (or
sometimes, other architectures or OSes).

That would be fine for clippy, however, because that would only run in
CI, where we _would_ want to catch newer changes, but we want to
compile nonetheless.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.09.13 19:01, brian m. carlson wrote:
> On 2024-09-07 at 15:15:12, Sean Allred wrote:
> > Calvin Wan <calvinwan@google.com> writes:
> > > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > > to their respective Makefiles so they can be built and tested without
> > > having to run cargo build/test.
> > 
> > I feel like clippy should be run as part of these somehow, but I'm not
> > sure where.
> 
> Yes, that seems like a good idea in CI.
> 
> > > +libgitrs-sys:
> > > +	$(QUIET)(\
> > > +		cd contrib/libgit-rs/libgit-sys && \
> > > +		cargo build \
> > > +	)
> > > +.PHONY: libgitrs
> > > +libgitrs:
> > > +	$(QUIET)(\
> > > +		cd contrib/libgit-rs && \
> > > +		cargo build \
> > > +	)
> > 
> > We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
> > `-Wall` in the C world, no? These crates should build without warnings.
> 
> I believe -Dwarnings turns warnings into errors (at least it does in my
> tests), which is equivalent to -Werror.  We don't want that because it
> breaks compiling older code with newer versions of the compiler, which
> makes it harder to bisect changes or compiler on the system compiler (or
> sometimes, other architectures or OSes).
> 
> That would be fine for clippy, however, because that would only run in
> CI, where we _would_ want to catch newer changes, but we want to
> compile nonetheless.
> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

Sorry, I missed your message before replying on this point last week.
I'll remove this from our build.rs in V4.

Re: [PATCH v3 6/6] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.09.07 10:15, Sean Allred wrote:
> Calvin Wan <calvinwan@google.com> writes:
> > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > to their respective Makefiles so they can be built and tested without
> > having to run cargo build/test.
> 
> I feel like clippy should be run as part of these somehow, but I'm not
> sure where.
> 
> > +libgitrs-sys:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs/libgit-sys && \
> > +		cargo build \
> > +	)
> > +.PHONY: libgitrs
> > +libgitrs:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs && \
> > +		cargo build \
> > +	)
> 
> We should definitely be setting `RUSTFLAGS=-Dwarnings` as an analog to
> `-Wall` in the C world, no? These crates should build without warnings.
> 
> Very excited to see this work; thank you for putting the time into it
> :-)
> 
> -- 
> Sean Allred

Done in V4, thanks for the suggestion.

[PATCH v4 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/martinvonz/jj

Please find V1 cover letter here:
https://lore.kernel.org/git/cover.1723054623.git.steadmon@google.com/

Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml

Known NEEDSWORK:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning


Calvin Wan (2):
  libgit: add higher-level libgit crate
  Makefile: add option to build and test libgit-rs and libgit-rs-sys

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                    |  2 +
 Makefile                                      | 31 ++++++
 common-exit.c                                 | 26 +++++
 common-init.c                                 | 63 ++++++++++++
 common-init.h                                 |  6 ++
 common-main.c                                 | 83 +---------------
 contrib/libgit-rs/Cargo.lock                  | 77 +++++++++++++++
 contrib/libgit-rs/Cargo.toml                  | 15 +++
 contrib/libgit-rs/build.rs                    |  4 +
 contrib/libgit-rs/libgit-sys/Cargo.lock       | 69 ++++++++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       | 17 ++++
 contrib/libgit-rs/libgit-sys/README.md        | 15 +++
 contrib/libgit-rs/libgit-sys/build.rs         | 35 +++++++
 .../libgit-sys/public_symbol_export.c         | 50 ++++++++++
 .../libgit-sys/public_symbol_export.h         | 18 ++++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 79 +++++++++++++++
 contrib/libgit-rs/src/lib.rs                  | 95 +++++++++++++++++++
 contrib/libgit-rs/testdata/config1            |  2 +
 contrib/libgit-rs/testdata/config2            |  2 +
 contrib/libgit-rs/testdata/config3            |  2 +
 t/Makefile                                    | 16 ++++
 21 files changed, 626 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

Range-diff against v3:
1:  83afd7f4a2 ! 1:  1ae14207f6 common-main: split init and exit code into new files
    @@ Commit message
         [2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/
     
    -    Signed-off-by: Calvin Wan <calvinwan@google.com>
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
      ## Makefile ##
     @@ Makefile: LIB_OBJS += combine-diff.o
2:  354b498494 ! 2:  ed925d6a33 libgit-sys: introduce Rust wrapper for libgit.a
    @@ Commit message
         Co-authored-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Kyle Lippincott <spectral@google.com>
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
      ## .gitignore ##
    @@ contrib/libgit-rs/libgit-sys/Cargo.lock (new)
     +version = 3
     +
     +[[package]]
    ++name = "autocfg"
    ++version = "1.4.0"
    ++source = "registry+https://github.com/rust-lang/crates.io-index"
    ++checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
    ++
    ++[[package]]
     +name = "cc"
     +version = "1.1.15"
     +source = "registry+https://github.com/rust-lang/crates.io-index"
    @@ contrib/libgit-rs/libgit-sys/Cargo.lock (new)
     +name = "libgit-sys"
     +version = "0.1.0"
     +dependencies = [
    ++ "autocfg",
     + "libz-sys",
    ++ "make-cmd",
     +]
     +
     +[[package]]
    @@ contrib/libgit-rs/libgit-sys/Cargo.lock (new)
     +]
     +
     +[[package]]
    ++name = "make-cmd"
    ++version = "0.1.0"
    ++source = "registry+https://github.com/rust-lang/crates.io-index"
    ++checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
    ++
    ++[[package]]
     +name = "pkg-config"
     +version = "0.3.30"
     +source = "registry+https://github.com/rust-lang/crates.io-index"
    @@ contrib/libgit-rs/libgit-sys/Cargo.toml (new)
     +edition = "2021"
     +build = "build.rs"
     +links = "gitpub"
    ++rust-version = "1.63"
     +
     +[lib]
     +path = "src/lib.rs"
     +
     +[dependencies]
     +libz-sys = "1.1.19"
    - \ No newline at end of file
    ++
    ++[build-dependencies]
    ++autocfg = "1.4.0"
    ++make-cmd = "0.1.0"
     
      ## contrib/libgit-rs/libgit-sys/README.md (new) ##
     @@
    @@ contrib/libgit-rs/libgit-sys/build.rs (new)
     +use std::path::PathBuf;
     +
     +pub fn main() -> std::io::Result<()> {
    ++    let ac = autocfg::new();
    ++    ac.emit_has_path("std::ffi::c_char");
    ++
     +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
     +    let git_root = crate_root.join("../../..");
     +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
     +
    -+    let make_output = std::process::Command::new("make")
    ++    let make_output = make_cmd::gnu_make()
    ++        .env("DEVELOPER", "1")
     +        .env_remove("PROFILE")
     +        .current_dir(git_root.clone())
     +        .args([
    @@ contrib/libgit-rs/libgit-sys/build.rs (new)
     +        );
     +    }
     +    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
    -+    println!("cargo::rustc-link-search=native={}", dst.display());
    -+    println!("cargo::rustc-link-lib=gitpub");
    -+    println!("cargo::rerun-if-changed={}", git_root.display());
    ++    println!("cargo:rustc-link-search=native={}", dst.display());
    ++    println!("cargo:rustc-link-lib=gitpub");
    ++    println!("cargo:rerun-if-changed={}", git_root.display());
     +
     +    Ok(())
     +}
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.c (new)
     @@
     +// Shim to publicly export Git symbols. These must be renamed so that the
     +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
    -+// avoid conflicts with other libraries such as libgit2.
    ++// avoids conflicts with other libraries such as libgit2.
     +
    ++#include "git-compat-util.h"
     +#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
     +#include "version.h"
     +
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.h (new)
     
      ## contrib/libgit-rs/libgit-sys/src/lib.rs (new) ##
     @@
    ++#[cfg(has_std__ffi__c_char)]
     +use std::ffi::c_char;
     +
    ++#[cfg(not(has_std__ffi__c_char))]
    ++#[allow(non_camel_case_types)]
    ++pub type c_char = i8;
    ++
     +extern crate libz_sys;
     +
     +extern "C" {
    -+    // From version.c
     +    pub fn libgit_user_agent() -> *const c_char;
     +    pub fn libgit_user_agent_sanitized() -> *const c_char;
     +}
3:  2ebff081f8 < -:  ---------- libgit-sys: add repo initialization and config access
4:  6b4b7e6ced ! 3:  8eeab7b418 config: add git_configset_alloc() and git_configset_clear_and_free()
    @@ Metadata
     Author: Josh Steadmon <steadmon@google.com>
     
      ## Commit message ##
    -    config: add git_configset_alloc() and git_configset_clear_and_free()
    +    libgit-sys: also export some config_set functions
     
    -    Add git_configset_alloc() and git_configset_clear_and_free() functions
    -    so that callers can manage config_set structs on the heap. This also
    -    allows non-C external consumers to treat config_sets as opaque structs.
    +    In preparation for implementing a higher-level Rust API for accessing
    +    Git configs, export some of the upstream configset API via libgitpub and
    +    libgit-sys. Since this will be exercised as part of the higher-level API
    +    in the next commit, no tests have been added for libgit-sys.
    +
    +    While we're at it, add git_configset_alloc() and git_configset_free()
    +    functions in libgitpub so that callers can manage config_set structs on
    +    the heap. This also allows non-C external consumers to treat config_sets
    +    as opaque structs.
     
         Co-authored-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
    - ## config.c ##
    -@@ config.c: static int config_set_element_cmp(const void *cmp_data UNUSED,
    - 	return strcmp(e1->key, e2->key);
    - }
    + ## contrib/libgit-rs/libgit-sys/public_symbol_export.c ##
    +@@
    + // avoids conflicts with other libraries such as libgit2.
    + 
    + #include "git-compat-util.h"
    ++#include "config.h"
    + #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
    + #include "version.h"
      
    -+struct config_set *git_configset_alloc(void)
    + #pragma GCC visibility push(default)
    + 
    ++struct libgit_config_set *libgit_configset_alloc(void)
     +{
    -+	return xmalloc(sizeof(struct config_set));
    ++	struct config_set *cs = xmalloc(sizeof(struct config_set));
    ++	git_configset_init(cs);
    ++	return (struct libgit_config_set *) cs;
     +}
     +
    - void git_configset_init(struct config_set *set)
    - {
    - 	hashmap_init(&set->config_hash, config_set_element_cmp, NULL, 0);
    -@@ config.c: void git_configset_clear(struct config_set *set)
    - 	set->list.items = NULL;
    - }
    - 
    -+void git_configset_clear_and_free(struct config_set *set)
    ++void libgit_configset_free(struct libgit_config_set *cs)
    ++{
    ++	git_configset_clear((struct config_set *) cs);
    ++	free((struct config_set *) cs);
    ++}
    ++
    ++int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
     +{
    -+	git_configset_clear(set);
    -+	free(set);
    ++	return git_configset_add_file((struct config_set *) cs, filename);
     +}
     +
    - static int config_set_callback(const char *key, const char *value,
    - 			       const struct config_context *ctx,
    - 			       void *cb)
    ++int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
    ++{
    ++	return git_configset_get_int((struct config_set *) cs, key, dest);
    ++}
    ++
    ++int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
    ++{
    ++	return git_configset_get_string((struct config_set *) cs, key, dest);
    ++}
    ++
    + const char *libgit_user_agent(void)
    + {
    + 	return git_user_agent();
    +
    + ## contrib/libgit-rs/libgit-sys/public_symbol_export.h ##
    +@@
    + #ifndef PUBLIC_SYMBOL_EXPORT_H
    + #define PUBLIC_SYMBOL_EXPORT_H
    + 
    ++struct libgit_config_set *libgit_configset_alloc(void);
    ++
    ++void libgit_configset_free(struct libgit_config_set *cs);
    ++
    ++int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
    ++
    ++int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
    ++
    ++int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
    ++
    + const char *libgit_user_agent(void);
    + 
    + const char *libgit_user_agent_sanitized(void);
     
    - ## config.h ##
    -@@ config.h: struct config_set {
    - 	struct configset_list list;
    - };
    + ## contrib/libgit-rs/libgit-sys/src/lib.rs ##
    +@@
    + #[cfg(has_std__ffi__c_char)]
    +-use std::ffi::c_char;
    ++use std::ffi::{c_char, c_int};
    + 
    + #[cfg(not(has_std__ffi__c_char))]
    + #[allow(non_camel_case_types)]
    + pub type c_char = i8;
    + 
    ++#[cfg(not(has_std__ffi__c_char))]
    ++#[allow(non_camel_case_types)]
    ++pub type c_int = i32;
    ++
    + extern crate libz_sys;
      
    -+/**
    -+ * Alloc a config_set
    -+ */
    -+struct config_set *git_configset_alloc(void);
    -+
    - /**
    -  * Initializes the config_set `cs`.
    -  */
    -@@ config.h: int git_configset_get_string_multi(struct config_set *cs, const char *key,
    -  */
    - void git_configset_clear(struct config_set *cs);
    ++#[allow(non_camel_case_types)]
    ++#[repr(C)]
    ++pub struct libgit_config_set {
    ++    _data: [u8; 0],
    ++    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
    ++}
    ++
    + extern "C" {
    +     pub fn libgit_user_agent() -> *const c_char;
    +     pub fn libgit_user_agent_sanitized() -> *const c_char;
    ++
    ++    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
    ++    pub fn libgit_configset_free(cs: *mut libgit_config_set);
    ++
    ++    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
    ++
    ++    pub fn libgit_configset_get_int(
    ++        cs: *mut libgit_config_set,
    ++        key: *const c_char,
    ++        int: *mut c_int,
    ++    ) -> c_int;
    ++
    ++    pub fn libgit_configset_get_string(
    ++        cs: *mut libgit_config_set,
    ++        key: *const c_char,
    ++        dest: *mut *mut c_char,
    ++    ) -> c_int;
    ++
    + }
      
    -+/**
    -+ * Clears and frees a heap-allocated `config_set` structure.
    -+ */
    -+void git_configset_clear_and_free(struct config_set *cs);
    -+
    - /*
    -  * These functions return 1 if not found, and 0 if found, leaving the found
    -  * value in the 'dest' pointer.
    + #[cfg(test)]
5:  18fea79783 ! 4:  29599e9c7b libgit: add higher-level libgit crate
    @@ Commit message
         a more Rust-friendly interface to config_set structs.
     
         Co-authored-by: Josh Steadmon <steadmon@google.com>
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
     
    @@ contrib/libgit-rs/Cargo.lock (new)
     +version = 3
     +
     +[[package]]
    ++name = "autocfg"
    ++version = "1.4.0"
    ++source = "registry+https://github.com/rust-lang/crates.io-index"
    ++checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
    ++
    ++[[package]]
     +name = "cc"
     +version = "1.1.15"
     +source = "registry+https://github.com/rust-lang/crates.io-index"
    @@ contrib/libgit-rs/Cargo.lock (new)
     +name = "libgit"
     +version = "0.1.0"
     +dependencies = [
    ++ "autocfg",
     + "libgit-sys",
     +]
     +
    @@ contrib/libgit-rs/Cargo.lock (new)
     +name = "libgit-sys"
     +version = "0.1.0"
     +dependencies = [
    ++ "autocfg",
     + "libz-sys",
    ++ "make-cmd",
     +]
     +
     +[[package]]
    @@ contrib/libgit-rs/Cargo.lock (new)
     +]
     +
     +[[package]]
    ++name = "make-cmd"
    ++version = "0.1.0"
    ++source = "registry+https://github.com/rust-lang/crates.io-index"
    ++checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
    ++
    ++[[package]]
     +name = "pkg-config"
     +version = "0.3.30"
     +source = "registry+https://github.com/rust-lang/crates.io-index"
    @@ contrib/libgit-rs/Cargo.toml (new)
     +name = "libgit"
     +version = "0.1.0"
     +edition = "2021"
    ++build = "build.rs"
    ++rust-version = "1.63"
     +
     +[lib]
     +path = "src/lib.rs"
     +
     +[dependencies]
     +libgit-sys = { version = "0.1.0", path = "libgit-sys" }
    -
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.c ##
    -@@ contrib/libgit-rs/libgit-sys/public_symbol_export.c: int libgit_parse_maybe_bool(const char *val)
    - 	return git_parse_maybe_bool(val);
    - }
    - 
    -+struct libgit_config_set *libgit_configset_alloc(void)
    -+{
    -+	return git_configset_alloc();
    -+}
    -+
    -+void libgit_configset_clear_and_free(struct libgit_config_set *cs)
    -+{
    -+	git_configset_clear_and_free(cs);
    -+}
     +
    -+void libgit_configset_init(struct libgit_config_set *cs)
    -+{
    -+	git_configset_init(cs);
    -+}
    -+
    -+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
    -+{
    -+	return git_configset_add_file(cs, filename);
    -+}
    -+
    -+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
    -+{
    -+	return git_configset_get_int(cs, key, dest);
    -+}
    -+
    -+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
    -+{
    -+	return git_configset_get_string(cs, key, dest);
    -+}
    -+
    - const char *libgit_user_agent(void)
    - {
    - 	return git_user_agent();
    ++[build-dependencies]
    ++autocfg = "1.4.0"
     
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.h ##
    -@@ contrib/libgit-rs/libgit-sys/public_symbol_export.h: void libgit_init_git(const char **argv);
    - 
    - int libgit_parse_maybe_bool(const char *val);
    - 
    -+struct libgit_config_set *libgit_configset_alloc(void);
    -+
    -+void libgit_configset_clear_and_free(struct libgit_config_set *cs);
    -+
    -+void libgit_configset_init(struct libgit_config_set *cs);
    -+
    -+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
    -+
    -+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
    -+
    -+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
    -+
    - const char *libgit_user_agent(void);
    - 
    - const char *libgit_user_agent_sanitized(void);
    + ## contrib/libgit-rs/build.rs (new) ##
    +@@
    ++pub fn main() {
    ++    let ac = autocfg::new();
    ++    ac.emit_has_path("std::ffi::c_char");
    ++}
     
      ## contrib/libgit-rs/libgit-sys/src/lib.rs ##
     @@
    --use std::ffi::{c_char, c_int};
    -+use std::ffi::{c_char, c_int, c_void};
    ++use std::ffi::c_void;
     +
    -+#[allow(non_camel_case_types)]
    -+#[repr(C)]
    -+pub struct libgit_config_set {
    -+    _data: [u8; 0],
    -+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
    -+}
    + #[cfg(has_std__ffi__c_char)]
    + use std::ffi::{c_char, c_int};
      
    - extern crate libz_sys;
    +@@ contrib/libgit-rs/libgit-sys/src/lib.rs: pub struct libgit_config_set {
    + }
      
      extern "C" {
     +    pub fn free(ptr: *mut c_void);
     +
    -     pub fn libgit_setup_git_directory() -> *const c_char;
    - 
    -     // From config.c
    -@@ contrib/libgit-rs/libgit-sys/src/lib.rs: extern "C" {
    -     // From version.c
          pub fn libgit_user_agent() -> *const c_char;
          pub fn libgit_user_agent_sanitized() -> *const c_char;
    -+
    -+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
    -+    pub fn libgit_configset_clear_and_free(cs: *mut libgit_config_set);
    -+
    -+    pub fn libgit_configset_init(cs: *mut libgit_config_set);
    -+
    -+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
    -+
    -+    pub fn libgit_configset_get_int(
    -+        cs: *mut libgit_config_set,
    -+        key: *const c_char,
    -+        int: *mut c_int,
    -+    ) -> c_int;
    -+
    -+    pub fn libgit_configset_get_string(
    -+        cs: *mut libgit_config_set,
    -+        key: *const c_char,
    -+        dest: *mut *mut c_char,
    -+    ) -> c_int;
    -+
    - }
      
    - #[cfg(test)]
     
      ## contrib/libgit-rs/src/lib.rs (new) ##
     @@
    -+use std::ffi::{c_char, c_int, c_void, CStr, CString};
    ++use std::ffi::{c_void, CStr, CString};
     +use std::path::Path;
     +
    ++#[cfg(has_std__ffi__c_char)]
    ++use std::ffi::{c_char, c_int};
    ++
    ++#[cfg(not(has_std__ffi__c_char))]
    ++#[allow(non_camel_case_types)]
    ++pub type c_char = i8;
    ++
    ++#[cfg(not(has_std__ffi__c_char))]
    ++#[allow(non_camel_case_types)]
    ++pub type c_int = i32;
    ++
     +use libgit_sys::*;
     +
     +pub struct ConfigSet(*mut libgit_config_set);
     +impl ConfigSet {
     +    pub fn new() -> Self {
    -+        unsafe {
    -+            let ptr = libgit_configset_alloc();
    -+            libgit_configset_init(ptr);
    -+            ConfigSet(ptr)
    -+        }
    ++        unsafe { ConfigSet(libgit_configset_alloc()) }
     +    }
     +
     +    pub fn add_files(&mut self, files: &[&Path]) {
     +        for file in files {
    -+            let pstr: &str = file.to_str().expect("Invalid UTF-8");
    -+            let rs = CString::new(&*pstr).expect("Couldn't convert to CString");
    ++            let pstr = file.to_str().expect("Invalid UTF-8");
    ++            let rs = CString::new(pstr).expect("Couldn't convert to CString");
     +            unsafe {
     +                libgit_configset_add_file(self.0, rs.as_ptr());
     +            }
    @@ contrib/libgit-rs/src/lib.rs (new)
     +        Some(val)
     +    }
     +
    -+    pub fn get_str(&mut self, key: &str) -> Option<CString> {
    -+        let key = CString::new(key).expect("Couldn't convert to CString");
    ++    pub fn get_string(&mut self, key: &str) -> Option<String> {
    ++        let key = CString::new(key).expect("Couldn't convert key to CString");
     +        let mut val: *mut c_char = std::ptr::null_mut();
     +        unsafe {
     +            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
    @@ contrib/libgit-rs/src/lib.rs (new)
     +                return None;
     +            }
     +            let borrowed_str = CStr::from_ptr(val);
    -+            let owned_str = CString::from_vec_with_nul(borrowed_str.to_bytes_with_nul().to_vec());
    ++            let owned_str =
    ++                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
     +            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
    -+            Some(owned_str.unwrap())
    ++            Some(owned_str)
     +        }
     +    }
     +}
    @@ contrib/libgit-rs/src/lib.rs (new)
     +impl Drop for ConfigSet {
     +    fn drop(&mut self) {
     +        unsafe {
    -+            libgit_configset_clear_and_free(self.0);
    ++            libgit_configset_free(self.0);
     +        }
     +    }
     +}
    @@ contrib/libgit-rs/src/lib.rs (new)
     +    #[test]
     +    fn load_configs_via_configset() {
     +        let mut cs = ConfigSet::new();
    -+        cs.add_files(&[Path::new("testdata/config1"),
    -+                       Path::new("testdata/config2"),
    -+                       Path::new("testdata/config3")]);
    -+        // ConfigSet retrieves correct value 
    ++        cs.add_files(&[
    ++            Path::new("testdata/config1"),
    ++            Path::new("testdata/config2"),
    ++            Path::new("testdata/config3"),
    ++        ]);
    ++        // ConfigSet retrieves correct value
     +        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
     +        // ConfigSet respects last config value set
     +        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
     +        // ConfigSet returns None for missing key
    -+        assert_eq!(cs.get_str("foo.bar"), None);
    ++        assert_eq!(cs.get_string("foo.bar"), None);
     +    }
     +}
     
6:  67c11dd143 ! 5:  2ed503216f Makefile: add option to build and test libgit-rs and libgit-rs-sys
    @@ Commit message
         automatically builds and tests libgit-rs and libgit-rs-sys when `make
         all` is ran.
     
         Signed-off-by: Calvin Wan <calvinwan@google.com>
     
      ## Makefile ##

base-commit: 557ae147e6cdc9db121269b058c757ac5092f9c9
-- 
2.47.0.rc0.187.ge670bccf7e-goog

[PATCH v4 1/5] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 5 files changed, 99 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 3eab701b10..7caeb3c872 100644
--- a/Makefile
+++ b/Makefile
@@ -979,6 +979,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.47.0.rc0.187.ge670bccf7e-goog

[PATCH v4 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                    |  1 +
 Makefile                                      | 13 ++++
 contrib/libgit-rs/libgit-sys/Cargo.lock       | 69 +++++++++++++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       | 17 +++++
 contrib/libgit-rs/libgit-sys/README.md        | 15 ++++
 contrib/libgit-rs/libgit-sys/build.rs         | 35 ++++++++++
 .../libgit-sys/public_symbol_export.c         | 21 ++++++
 .../libgit-sys/public_symbol_export.h         |  8 +++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 46 +++++++++++++
 9 files changed, 225 insertions(+)
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index 8caf3700c2..dfd72820fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,3 +248,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 7caeb3c872..0090514e55 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2713,6 +2715,7 @@ OBJECTS += $(XDIFF_OBJS)
 OBJECTS += $(FUZZ_OBJS)
 OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
+OBJECTS += contrib/libgit-rs/libgit-sys/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3720,6 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-rs/libgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3865,3 +3869,12 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
 build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.lock b/contrib/libgit-rs/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.toml b/contrib/libgit-rs/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..7a230a4437
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-rs/libgit-sys/README.md b/contrib/libgit-rs/libgit-sys/README.md
new file mode 100644
index 0000000000..7a59602c30
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/README.md
@@ -0,0 +1,15 @@
+# cgit-info
+
+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
+library.
+
+## Building
+
+`cargo build` automatically builds and picks up on changes made to both
+the Rust wrapper and git.git code so there is no need to run `make`
+beforehand.
+
+## Running
+
+Assuming you don't make any changes to the Git source, you can just work from
+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
new file mode 100644
index 0000000000..8d74120191
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/libgit-rs/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..8e4b94d821
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -0,0 +1,21 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoids conflicts with other libraries such as libgit2.
+
+#include "git-compat-util.h"
+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
-- 
2.47.0.rc0.187.ge670bccf7e-goog

[PATCH v4 3/5] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .../libgit-sys/public_symbol_export.c         | 29 +++++++++++++++++
 .../libgit-sys/public_symbol_export.h         | 10 ++++++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 31 ++++++++++++++++++-
 3 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 8e4b94d821..29e8a5363e 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -3,11 +3,40 @@
 // avoids conflicts with other libraries such as libgit2.
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct config_set *cs = xmalloc(sizeof(struct config_set));
+	git_configset_init(cs);
+	return (struct libgit_config_set *) cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear((struct config_set *) cs);
+	free((struct config_set *) cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file((struct config_set *) cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int((struct config_set *) cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string((struct config_set *) cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.47.0.rc0.187.ge670bccf7e-goog

[PATCH v4 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Wrap `struct config_set` and a few of its associated functions in
libgit-sys. Also introduce a higher-level "libgit" crate which provides
a more Rust-friendly interface to config_set structs.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                              |  1 +
 Makefile                                |  2 +-
 contrib/libgit-rs/Cargo.lock            | 77 ++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml            | 15 ++++
 contrib/libgit-rs/build.rs              |  4 ++
 contrib/libgit-rs/libgit-sys/src/lib.rs |  4 ++
 contrib/libgit-rs/src/lib.rs            | 95 +++++++++++++++++++++++++
 contrib/libgit-rs/testdata/config1      |  2 +
 contrib/libgit-rs/testdata/config2      |  2 +
 contrib/libgit-rs/testdata/config3      |  2 +
 10 files changed, 203 insertions(+), 1 deletion(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index dfd72820fb..0a42f27117 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,4 +248,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 0090514e55..abeee01d9e 100644
--- a/Makefile
+++ b/Makefile
@@ -3723,7 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-rs/libgit-sys/target
+	$(RM) -r contrib/libgit-rs/target contrib/libgit-rs/libgit-sys/target
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..6b4387b9e7
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..d228091f3d
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1,95 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
+use libgit_sys::*;
+
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val)
+    }
+
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
-- 
2.47.0.rc0.187.ge670bccf7e-goog

Re: [PATCH v4 4/5] libgit: add higher-level libgit crate

[Eric Sunshine]

On Tue, Oct 8, 2024 at 7:19 PM Josh Steadmon <steadmon@google.com> wrote:
> Wrap `struct config_set` and a few of its associated functions in
> libgit-sys. Also introduce a higher-level "libgit" crate which provides
> a more Rust-friendly interface to config_set structs.
>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
> diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
> @@ -0,0 +1,95 @@
> +pub struct ConfigSet(*mut libgit_config_set);
> +impl ConfigSet {
> +    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
> +        let key = CString::new(key).expect("Couldn't convert to CString");
> +        let mut val: c_int = 0;
> +        unsafe {
> +            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
> +                return None;
> +            }
> +        }
> +        Some(val)
> +    }

Considering that v4 finally gets around to swapping out `CString` for
`String` in order to make this high-level crate more
Rust-programmer-friendly, I was more than a little surprised to see
that this function is still exposing `c_int` rather than, say, `i64`
or such.

Re: [PATCH v4 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2024.10.09 02:18, Eric Sunshine wrote:
> On Tue, Oct 8, 2024 at 7:19 PM Josh Steadmon <steadmon@google.com> wrote:
> > Wrap `struct config_set` and a few of its associated functions in
> > libgit-sys. Also introduce a higher-level "libgit" crate which provides
> > a more Rust-friendly interface to config_set structs.
> >
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Josh Steadmon <steadmon@google.com>
> > ---
> > diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
> > @@ -0,0 +1,95 @@
> > +pub struct ConfigSet(*mut libgit_config_set);
> > +impl ConfigSet {
> > +    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
> > +        let key = CString::new(key).expect("Couldn't convert to CString");
> > +        let mut val: c_int = 0;
> > +        unsafe {
> > +            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
> > +                return None;
> > +            }
> > +        }
> > +        Some(val)
> > +    }
> 
> Considering that v4 finally gets around to swapping out `CString` for
> `String` in order to make this high-level crate more
> Rust-programmer-friendly, I was more than a little surprised to see
> that this function is still exposing `c_int` rather than, say, `i64`
> or such.

Yes, definitely an oversight. I'll fix in V5.

Re: [PATCH v4 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2024.10.08 16:19, Josh Steadmon wrote:
> From: Calvin Wan <calvinwan@google.com>
> 
> Wrap `struct config_set` and a few of its associated functions in
> libgit-sys. Also introduce a higher-level "libgit" crate which provides
> a more Rust-friendly interface to config_set structs.
> 
> Co-authored-by: Josh Steadmon <steadmon@google.com>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
>  .gitignore                              |  1 +
>  Makefile                                |  2 +-
>  contrib/libgit-rs/Cargo.lock            | 77 ++++++++++++++++++++
>  contrib/libgit-rs/Cargo.toml            | 15 ++++
>  contrib/libgit-rs/build.rs              |  4 ++
>  contrib/libgit-rs/libgit-sys/src/lib.rs |  4 ++
>  contrib/libgit-rs/src/lib.rs            | 95 +++++++++++++++++++++++++
>  contrib/libgit-rs/testdata/config1      |  2 +
>  contrib/libgit-rs/testdata/config2      |  2 +
>  contrib/libgit-rs/testdata/config3      |  2 +
>  10 files changed, 203 insertions(+), 1 deletion(-)
>  create mode 100644 contrib/libgit-rs/Cargo.lock
>  create mode 100644 contrib/libgit-rs/Cargo.toml
>  create mode 100644 contrib/libgit-rs/build.rs
>  create mode 100644 contrib/libgit-rs/src/lib.rs
>  create mode 100644 contrib/libgit-rs/testdata/config1
>  create mode 100644 contrib/libgit-rs/testdata/config2
>  create mode 100644 contrib/libgit-rs/testdata/config3

Just realized that this commit message is not accurate anymore (and
could provide more useful info anyway). I've reworded it in V5.

[PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
to their respective Makefiles so they can be built and tested without
having to run cargo build/test.

Add environment variable, INCLUDE_LIBGIT_RS, that when set,
automatically builds and tests libgit-rs and libgit-rs-sys when `make
all` is ran.

Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile   | 16 ++++++++++++++++
 t/Makefile | 16 ++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/Makefile b/Makefile
index abeee01d9e..41ad458aef 100644
--- a/Makefile
+++ b/Makefile
@@ -3870,6 +3870,22 @@ build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
+.PHONY: libgitrs-sys
+libgitrs-sys:
+	$(QUIET)(\
+		cd contrib/libgit-rs/libgit-sys && \
+		cargo build \
+	)
+.PHONY: libgitrs
+libgitrs:
+	$(QUIET)(\
+		cd contrib/libgit-rs && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs
+endif
+
 contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
 	$(LD) -r $^ -o $@
 
diff --git a/t/Makefile b/t/Makefile
index b2eb9f770b..066cb5c2b4 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -169,3 +169,19 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgitrs-sys-test
+libgitrs-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs/libgit-sys && \
+		cargo test \
+	)
+.PHONY: libgitrs-test
+libgitrs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs-sys-test libgitrs-test
+endif
-- 
2.47.0.rc0.187.ge670bccf7e-goog

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> automatically builds and tests libgit-rs and libgit-rs-sys when `make
> all` is ran.

Is this unusual, or is it just like how other makefile macros like
say USE_NSEC (to cause the resulting Git to use subsecond mtimes)
are meant to be used to control the build?  IOW, shouldn't this be
documented near the top of the Makefile, e.g.

    diff --git i/Makefile w/Makefile
    index 41ad458aef..2b55fe9672 100644
    --- i/Makefile
    +++ w/Makefile
    @@ -392,6 +392,9 @@ include shared.mak
     # INSTALL_STRIP can be set to "-s" to strip binaries during installation,
     # if your $(INSTALL) command supports the option.
     #
    +# Define INCLUDE_LIBGIT_RS if you want your gostak to distim
    +# the doshes and ...
    +#
     # Define GENERATE_COMPILATION_DATABASE to "yes" to generate JSON compilation
     # database entries during compilation if your compiler supports it, using the
     # `-MJ` flag. The JSON entries will be placed in the `compile_commands/`

It might make sense to follow naming convention to call it NO_RUST
and flip its polarity.  Those who do not have or want libgit-rs and
friends can say NO_RUST but otherwise it gets built by default.  It
would give you a wider developer population coverage.

Thanks.

RE: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[rsbecker]

On October 8, 2024 7:46 PM, Junio C Hamano wrote:
>Josh Steadmon <steadmon@google.com> writes:
>
>> Add environment variable, INCLUDE_LIBGIT_RS, that when set,
>> automatically builds and tests libgit-rs and libgit-rs-sys when `make
>> all` is ran.
>
>Is this unusual, or is it just like how other makefile macros like say
USE_NSEC (to
>cause the resulting Git to use subsecond mtimes) are meant to be used to
control
>the build?  IOW, shouldn't this be documented near the top of the Makefile,
e.g.
>
>    diff --git i/Makefile w/Makefile
>    index 41ad458aef..2b55fe9672 100644
>    --- i/Makefile
>    +++ w/Makefile
>    @@ -392,6 +392,9 @@ include shared.mak
>     # INSTALL_STRIP can be set to "-s" to strip binaries during
installation,
>     # if your $(INSTALL) command supports the option.
>     #
>    +# Define INCLUDE_LIBGIT_RS if you want your gostak to distim
>    +# the doshes and ...
>    +#
>     # Define GENERATE_COMPILATION_DATABASE to "yes" to generate JSON
>compilation
>     # database entries during compilation if your compiler supports it,
using the
>     # `-MJ` flag. The JSON entries will be placed in the
`compile_commands/`
>
>It might make sense to follow naming convention to call it NO_RUST and flip
its
>polarity.  Those who do not have or want libgit-rs and friends can say
NO_RUST but
>otherwise it gets built by default.  It would give you a wider developer
population
>coverage.

Some of us who do not have Rust (yet) approve this message. I hope our
situation
will change on having Rust on NonStop.

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.10.08 16:45, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> > automatically builds and tests libgit-rs and libgit-rs-sys when `make
> > all` is ran.
> 
> Is this unusual, or is it just like how other makefile macros like
> say USE_NSEC (to cause the resulting Git to use subsecond mtimes)
> are meant to be used to control the build?  IOW, shouldn't this be
> documented near the top of the Makefile, e.g.
> 
>     diff --git i/Makefile w/Makefile
>     index 41ad458aef..2b55fe9672 100644
>     --- i/Makefile
>     +++ w/Makefile
>     @@ -392,6 +392,9 @@ include shared.mak
>      # INSTALL_STRIP can be set to "-s" to strip binaries during installation,
>      # if your $(INSTALL) command supports the option.
>      #
>     +# Define INCLUDE_LIBGIT_RS if you want your gostak to distim
>     +# the doshes and ...
>     +#
>      # Define GENERATE_COMPILATION_DATABASE to "yes" to generate JSON compilation
>      # database entries during compilation if your compiler supports it, using the
>      # `-MJ` flag. The JSON entries will be placed in the `compile_commands/`
> 
> It might make sense to follow naming convention to call it NO_RUST
> and flip its polarity.  Those who do not have or want libgit-rs and
> friends can say NO_RUST but otherwise it gets built by default.  It
> would give you a wider developer population coverage.
> 
> Thanks.

For now I'd be more comfortable keeping it off by default. I don't want
to force those not interested in Rust to work around our in-progress
projects. Once it's more stable and we have CI I'd feel better about
turning it on by default (and maybe moving it out of contrib/ at that
point?).

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> From: Calvin Wan <calvinwan@google.com>
>
> Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> to their respective Makefiles so they can be built and tested without
> having to run cargo build/test.
>
> Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> automatically builds and tests libgit-rs and libgit-rs-sys when `make
> all` is ran.
>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
>  Makefile   | 16 ++++++++++++++++
>  t/Makefile | 16 ++++++++++++++++
>  2 files changed, 32 insertions(+)

Interesting.

I tried

	$ make INCLUDE_LIBGIT_RS=YesPlease

which did not fail, and then did the same

	$ make INCLUDE_LIBGIT_RS=YesPlease

and was surprised to see that not only the libgit-sys part but
everything was recompiled and rebuilt.

> diff --git a/Makefile b/Makefile
> ...
> +.PHONY: libgitrs
> +libgitrs:
> +	$(QUIET)(\
> +		cd contrib/libgit-rs && \
> +		cargo build \
> +	)
> +ifdef INCLUDE_LIBGIT_RS
> +all:: libgitrs
> +endif
> +
>  contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
>  	$(LD) -r $^ -o $@

I can see libgitrs is a phony target designed to run every time it
gets triggered, and I would imagine "cargo build" itself would avoid
repeating unnecessary work, but I do not see this patch screwing up
with the dependencies for other object files.

Is it fair to say this is still a WIP?  Showing a WIP to others and
asking for help is OK, but it is fair to make sure that others know
what is expected of them.

Thanks.

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.10.08 17:01, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > From: Calvin Wan <calvinwan@google.com>
> >
> > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > to their respective Makefiles so they can be built and tested without
> > having to run cargo build/test.
> >
> > Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> > automatically builds and tests libgit-rs and libgit-rs-sys when `make
> > all` is ran.
> >
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Josh Steadmon <steadmon@google.com>
> > ---
> >  Makefile   | 16 ++++++++++++++++
> >  t/Makefile | 16 ++++++++++++++++
> >  2 files changed, 32 insertions(+)
> 
> Interesting.
> 
> I tried
> 
> 	$ make INCLUDE_LIBGIT_RS=YesPlease
> 
> which did not fail, and then did the same
> 
> 	$ make INCLUDE_LIBGIT_RS=YesPlease
> 
> and was surprised to see that not only the libgit-sys part but
> everything was recompiled and rebuilt.
> 
> > diff --git a/Makefile b/Makefile
> > ...
> > +.PHONY: libgitrs
> > +libgitrs:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs && \
> > +		cargo build \
> > +	)
> > +ifdef INCLUDE_LIBGIT_RS
> > +all:: libgitrs
> > +endif
> > +
> >  contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
> >  	$(LD) -r $^ -o $@
> 
> I can see libgitrs is a phony target designed to run every time it
> gets triggered, and I would imagine "cargo build" itself would avoid
> repeating unnecessary work, but I do not see this patch screwing up
> with the dependencies for other object files.
> 
> Is it fair to say this is still a WIP?  Showing a WIP to others and
> asking for help is OK, but it is fair to make sure that others know
> what is expected of them.

Hmm, I think this may be an unfortunate interaction between Git's `make
all`, followed by libgit-sys's `build.rs` calling make again (with
different CFLAGS, specifically '-fvisibility=hidden') to build
libgitpub.a. So then the second `make all` has to rebuild everything due
to changing the CFLAGS back, and then libgit-sys has to rebuild
libgitpub.a once again.

Unfortunately, I don't see a way around this problem, at least with our
current approach for building libgitpub.a. We have to pass
'-fvisibility=hidden' when compiling each source file, not just at link
time, so I think the object files created when building vanilla Git will
necessarily differ from those created when building libgit-rs.

I think that means we'll need to drop this patch for now, and revisit
if/when we change our libgitpub.a strategy.

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> Hmm, I think this may be an unfortunate interaction between Git's `make
> all`, followed by libgit-sys's `build.rs` calling make again (with
> different CFLAGS, specifically '-fvisibility=hidden') to build
> libgitpub.a. So then the second `make all` has to rebuild everything due
> to changing the CFLAGS back, and then libgit-sys has to rebuild
> libgitpub.a once again.

Ah, OK, if we need to compile in two different ways, then it is a
matter of giving a dedicated *.o build directory to each, and until
that happens, the object files for Git proper and libgit-sys would
try to stomp on each other.

I thought Patrick's build procedure update has out-of-tree build as
one of its goals, in which case we may be able to piggy-back on the
effort once it starts to stabilize.

Thanks.

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.10.09 17:52, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > Hmm, I think this may be an unfortunate interaction between Git's `make
> > all`, followed by libgit-sys's `build.rs` calling make again (with
> > different CFLAGS, specifically '-fvisibility=hidden') to build
> > libgitpub.a. So then the second `make all` has to rebuild everything due
> > to changing the CFLAGS back, and then libgit-sys has to rebuild
> > libgitpub.a once again.
> 
> Ah, OK, if we need to compile in two different ways, then it is a
> matter of giving a dedicated *.o build directory to each, and until
> that happens, the object files for Git proper and libgit-sys would
> try to stomp on each other.
> 
> I thought Patrick's build procedure update has out-of-tree build as
> one of its goals, in which case we may be able to piggy-back on the
> effort once it starts to stabilize.
> 
> Thanks.

Actually, including '-fvisibility=hidden' by default doesn't break the
main build, so I think we can just add this to BASIC_CFLAGS if
INCLUDE_LIBGIT_RS is set. I'll do that (and add documentation as
requested) and restore this patch in V5.

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> From: Calvin Wan <calvinwan@google.com>
>
> Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> to their respective Makefiles so they can be built and tested without
> having to run cargo build/test.
>
> Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> automatically builds and tests libgit-rs and libgit-rs-sys when `make
> all` is ran.
>
> Signed-off-by: Calvin Wan <calvinwan@google.com>
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
>  Makefile   | 16 ++++++++++++++++
>  t/Makefile | 16 ++++++++++++++++
>  2 files changed, 32 insertions(+)

After 

    $ make INCLUDE_LIBGIT_RS=YesPlease

running either

    $ make INCLUDE_LIBGIT_RS=YesPlease distclean
    $ make distclean

leaves

    $ git clean -n -x
    Would remove contrib/libgit-rs/libgit-sys/libgitpub.a

behind.  We'd need to add a bit more to the Makefile, it seems.



 Makefile | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git i/Makefile w/Makefile
index 41ad458aef..2acb5353d1 100644
--- i/Makefile
+++ w/Makefile
@@ -392,6 +392,9 @@ include shared.mak
 # INSTALL_STRIP can be set to "-s" to strip binaries during installation,
 # if your $(INSTALL) command supports the option.
 #
+# Define INCLUDE_LIBGIT_RS if you want your gostak to distim
+# the doshes.
+#
 # Define GENERATE_COMPILATION_DATABASE to "yes" to generate JSON compilation
 # database entries during compilation if your compiler supports it, using the
 # `-MJ` flag. The JSON entries will be placed in the `compile_commands/`
@@ -771,6 +774,9 @@ PROGRAM_OBJS += shell.o
 .PHONY: program-objs
 program-objs: $(PROGRAM_OBJS)
 
+# libgit-rs stuff
+LIBGITPUB_A = contrib/libgit-rs/libgit-sys/libgitpub.a
+
 # Binary suffix, set to .exe for Windows builds
 X =
 
@@ -3708,6 +3714,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) po/git.pot po/git-core.pot
 	$(RM) git.res
 	$(RM) $(OBJECTS)
+	$(RM) $(LIBGITPUB_A)
 	$(RM) headless-git.o
 	$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)
 	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)
@@ -3892,5 +3899,5 @@ contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-s
 contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
 	$(OBJCOPY) --localize-hidden $^ $@
 
-contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
+$(LIBGITPUB_A): contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
 	$(AR) $(ARFLAGS) $@ $^

Re: [PATCH v4 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

On 2024.10.08 17:10, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > From: Calvin Wan <calvinwan@google.com>
> >
> > Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
> > to their respective Makefiles so they can be built and tested without
> > having to run cargo build/test.
> >
> > Add environment variable, INCLUDE_LIBGIT_RS, that when set,
> > automatically builds and tests libgit-rs and libgit-rs-sys when `make
> > all` is ran.
> >
> > Signed-off-by: Calvin Wan <calvinwan@google.com>
> > Signed-off-by: Josh Steadmon <steadmon@google.com>
> > ---
> >  Makefile   | 16 ++++++++++++++++
> >  t/Makefile | 16 ++++++++++++++++
> >  2 files changed, 32 insertions(+)
> 
> After 
> 
>     $ make INCLUDE_LIBGIT_RS=YesPlease
> 
> running either
> 
>     $ make INCLUDE_LIBGIT_RS=YesPlease distclean
>     $ make distclean
> 
> leaves
> 
>     $ git clean -n -x
>     Would remove contrib/libgit-rs/libgit-sys/libgitpub.a
> 
> behind.  We'd need to add a bit more to the Makefile, it seems.
> 
> 
> 
>  Makefile | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git i/Makefile w/Makefile
> index 41ad458aef..2acb5353d1 100644
> --- i/Makefile
> +++ w/Makefile
> @@ -392,6 +392,9 @@ include shared.mak
>  # INSTALL_STRIP can be set to "-s" to strip binaries during installation,
>  # if your $(INSTALL) command supports the option.
>  #
> +# Define INCLUDE_LIBGIT_RS if you want your gostak to distim
> +# the doshes.
> +#
>  # Define GENERATE_COMPILATION_DATABASE to "yes" to generate JSON compilation
>  # database entries during compilation if your compiler supports it, using the
>  # `-MJ` flag. The JSON entries will be placed in the `compile_commands/`
> @@ -771,6 +774,9 @@ PROGRAM_OBJS += shell.o
>  .PHONY: program-objs
>  program-objs: $(PROGRAM_OBJS)
>  
> +# libgit-rs stuff
> +LIBGITPUB_A = contrib/libgit-rs/libgit-sys/libgitpub.a
> +
>  # Binary suffix, set to .exe for Windows builds
>  X =
>  
> @@ -3708,6 +3714,7 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) po/git.pot po/git-core.pot
>  	$(RM) git.res
>  	$(RM) $(OBJECTS)
> +	$(RM) $(LIBGITPUB_A)
>  	$(RM) headless-git.o
>  	$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)
>  	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)
> @@ -3892,5 +3899,5 @@ contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-s
>  contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
>  	$(OBJCOPY) --localize-hidden $^ $@
>  
> -contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
> +$(LIBGITPUB_A): contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
>  	$(AR) $(ARFLAGS) $@ $^

Done in V5.

[PATCH v5 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

A quick note: normally I'd wait longer for more feedback before sending
out a new reroll, but due to some firefighting at $DAYJOB, I will not
have time to focus on this series for the next several weeks, possibly
up to a month. I wanted to get V5 out before then, but please understand
that I will not be able to address new feedback for a while.

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/martinvonz/jj

Please find V1 cover letter here:
https://lore.kernel.org/git/cover.1723054623.git.steadmon@google.com/

Known NEEDSWORK:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning

Changes in V5:
* When building with INCLUDE_LIBGIT_RS defined, add
  "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
  visibility in libgitpub.a without causing `make all` to rebuild from
  scratch due to changing CFLAGS.

* Avoid using c_int in the higher-level Rust API.

* Remove libgitpub.a and intermediate files with `make clean`.


Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml


Calvin Wan (2):
  libgit: add higher-level libgit crate
  Makefile: add option to build and test libgit-rs and libgit-rs-sys

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                    |  2 +
 Makefile                                      | 44 +++++++++
 common-exit.c                                 | 26 +++++
 common-init.c                                 | 63 ++++++++++++
 common-init.h                                 |  6 ++
 common-main.c                                 | 83 +---------------
 contrib/libgit-rs/Cargo.lock                  | 77 +++++++++++++++
 contrib/libgit-rs/Cargo.toml                  | 15 +++
 contrib/libgit-rs/build.rs                    |  4 +
 contrib/libgit-rs/libgit-sys/Cargo.lock       | 69 ++++++++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       | 17 ++++
 contrib/libgit-rs/libgit-sys/README.md        | 15 +++
 contrib/libgit-rs/libgit-sys/build.rs         | 35 +++++++
 .../libgit-sys/public_symbol_export.c         | 50 ++++++++++
 .../libgit-sys/public_symbol_export.h         | 18 ++++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 79 +++++++++++++++
 contrib/libgit-rs/src/lib.rs                  | 95 +++++++++++++++++++
 contrib/libgit-rs/testdata/config1            |  2 +
 contrib/libgit-rs/testdata/config2            |  2 +
 contrib/libgit-rs/testdata/config3            |  2 +
 t/Makefile                                    | 16 ++++
 21 files changed, 639 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

Range-diff against v4:
4:  2ed503216f ! 1:  1ae14207f6 Makefile: add option to build and test libgit-rs and libgit-rs-sys
    @@
      ## Metadata ##
    -Author: Calvin Wan <calvinwan@google.com>
    +Author: Josh Steadmon <steadmon@google.com>
     
      ## Commit message ##
    -    Makefile: add option to build and test libgit-rs and libgit-rs-sys
    +    common-main: split init and exit code into new files
     
    -    Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
    -    to their respective Makefiles so they can be built and tested without
    -    having to run cargo build/test.
    +    Currently, object files in libgit.a reference common_exit(), which is
    +    contained in common-main.o. However, common-main.o also includes main(),
    +    which references cmd_main() in git.o, which in turn depends on all the
    +    builtin/*.o objects.
     
    -    Add environment variable, INCLUDE_LIBGIT_RS, that when set,
    -    automatically builds and tests libgit-rs and libgit-rs-sys when `make
    -    all` is ran.
    +    We would like to allow external users to link libgit.a without needing
    +    to include so many extra objects. Enable this by splitting common_exit()
    +    and check_bug_if_BUG() into a new file common-exit.c, and add
    +    common-exit.o to LIB_OBJS so that these are included in libgit.a.
     
    -    Signed-off-by: Calvin Wan <calvinwan@google.com>
    +    This split has previously been proposed ([1], [2]) to support fuzz tests
    +    and unit tests by avoiding conflicting definitions for main(). However,
    +    both of those issues were resolved by other methods of avoiding symbol
    +    conflicts. Now we are trying to make libgit.a more self-contained, so
    +    hopefully we can revisit this approach.
    +
    +    Additionally, move the initialization code out of main() into a new
    +    init_git() function in its own file. Include this in libgit.a as well,
    +    so that external users can share our setup code without calling our
    +    main().
    +
    +    [1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
    +    [2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/
    +
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
      ## Makefile ##
    -@@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS)
    - unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
    - 	$(MAKE) -C t/ unit-tests
    +@@ Makefile: LIB_OBJS += combine-diff.o
    + LIB_OBJS += commit-graph.o
    + LIB_OBJS += commit-reach.o
    + LIB_OBJS += commit.o
    ++LIB_OBJS += common-exit.o
    ++LIB_OBJS += common-init.o
    + LIB_OBJS += compat/nonblock.o
    + LIB_OBJS += compat/obstack.o
    + LIB_OBJS += compat/terminal.o
    +
    + ## common-exit.c (new) ##
    +@@
    ++#include "git-compat-util.h"
    ++#include "trace2.h"
    ++
    ++static void check_bug_if_BUG(void)
    ++{
    ++	if (!bug_called_must_BUG)
    ++		return;
    ++	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
    ++}
    ++
    ++/* We wrap exit() to call common_exit() in git-compat-util.h */
    ++int common_exit(const char *file, int line, int code)
    ++{
    ++	/*
    ++	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
    ++	 * to e.g. turn -1 into 255. On a POSIX system this is
    ++	 * redundant, see exit(3) and wait(2), but as it doesn't harm
    ++	 * anything there we don't need to guard this with an "ifdef".
    ++	 */
    ++	code &= 0xff;
    ++
    ++	check_bug_if_BUG();
    ++	trace2_cmd_exit_fl(file, line, code);
    ++
    ++	return code;
    ++}
    +
    + ## common-init.c (new) ##
    +@@
    ++#define USE_THE_REPOSITORY_VARIABLE
    ++
    ++#include "git-compat-util.h"
    ++#include "common-init.h"
    ++#include "exec-cmd.h"
    ++#include "gettext.h"
    ++#include "attr.h"
    ++#include "repository.h"
    ++#include "setup.h"
    ++#include "strbuf.h"
    ++#include "trace2.h"
    ++
    ++/*
    ++ * Many parts of Git have subprograms communicate via pipe, expect the
    ++ * upstream of a pipe to die with SIGPIPE when the downstream of a
    ++ * pipe does not need to read all that is written.  Some third-party
    ++ * programs that ignore or block SIGPIPE for their own reason forget
    ++ * to restore SIGPIPE handling to the default before spawning Git and
    ++ * break this carefully orchestrated machinery.
    ++ *
    ++ * Restore the way SIGPIPE is handled to default, which is what we
    ++ * expect.
    ++ */
    ++static void restore_sigpipe_to_default(void)
    ++{
    ++	sigset_t unblock;
    ++
    ++	sigemptyset(&unblock);
    ++	sigaddset(&unblock, SIGPIPE);
    ++	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
    ++	signal(SIGPIPE, SIG_DFL);
    ++}
    ++
    ++void init_git(const char **argv)
    ++{
    ++	struct strbuf tmp = STRBUF_INIT;
    ++
    ++	trace2_initialize_clock();
    ++
    ++	/*
    ++	 * Always open file descriptors 0/1/2 to avoid clobbering files
    ++	 * in die().  It also avoids messing up when the pipes are dup'ed
    ++	 * onto stdin/stdout/stderr in the child processes we spawn.
    ++	 */
    ++	sanitize_stdfds();
    ++	restore_sigpipe_to_default();
    ++
    ++	git_resolve_executable_dir(argv[0]);
    ++
    ++	setlocale(LC_CTYPE, "");
    ++	git_setup_gettext();
    ++
    ++	initialize_repository(the_repository);
    ++
    ++	attr_start();
    ++
    ++	trace2_initialize();
    ++	trace2_cmd_start(argv);
    ++	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
    ++
    ++	if (!strbuf_getcwd(&tmp))
    ++		tmp_original_cwd = strbuf_detach(&tmp, NULL);
    ++}
    +
    + ## common-init.h (new) ##
    +@@
    ++#ifndef COMMON_INIT_H
    ++#define COMMON_INIT_H
    ++
    ++void init_git(const char **argv);
    ++
    ++#endif /* COMMON_INIT_H */
    +
    + ## common-main.c ##
    +@@
    +-#define USE_THE_REPOSITORY_VARIABLE
    +-
    + #include "git-compat-util.h"
    +-#include "exec-cmd.h"
    +-#include "gettext.h"
    +-#include "attr.h"
    +-#include "repository.h"
    +-#include "setup.h"
    +-#include "strbuf.h"
    +-#include "trace2.h"
    +-
    +-/*
    +- * Many parts of Git have subprograms communicate via pipe, expect the
    +- * upstream of a pipe to die with SIGPIPE when the downstream of a
    +- * pipe does not need to read all that is written.  Some third-party
    +- * programs that ignore or block SIGPIPE for their own reason forget
    +- * to restore SIGPIPE handling to the default before spawning Git and
    +- * break this carefully orchestrated machinery.
    +- *
    +- * Restore the way SIGPIPE is handled to default, which is what we
    +- * expect.
    +- */
    +-static void restore_sigpipe_to_default(void)
    +-{
    +-	sigset_t unblock;
    +-
    +-	sigemptyset(&unblock);
    +-	sigaddset(&unblock, SIGPIPE);
    +-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
    +-	signal(SIGPIPE, SIG_DFL);
    +-}
    ++#include "common-init.h"
      
    -+.PHONY: libgitrs-sys
    -+libgitrs-sys:
    -+	$(QUIET)(\
    -+		cd contrib/libgit-rs/libgit-sys && \
    -+		cargo build \
    -+	)
    -+.PHONY: libgitrs
    -+libgitrs:
    -+	$(QUIET)(\
    -+		cd contrib/libgit-rs && \
    -+		cargo build \
    -+	)
    -+ifdef INCLUDE_LIBGIT_RS
    -+all:: libgitrs
    -+endif
    -+
    - contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    - 	$(LD) -r $^ -o $@
    + int main(int argc, const char **argv)
    + {
    + 	int result;
    +-	struct strbuf tmp = STRBUF_INIT;
    +-
    +-	trace2_initialize_clock();
    +-
    +-	/*
    +-	 * Always open file descriptors 0/1/2 to avoid clobbering files
    +-	 * in die().  It also avoids messing up when the pipes are dup'ed
    +-	 * onto stdin/stdout/stderr in the child processes we spawn.
    +-	 */
    +-	sanitize_stdfds();
    +-	restore_sigpipe_to_default();
    +-
    +-	git_resolve_executable_dir(argv[0]);
    +-
    +-	setlocale(LC_CTYPE, "");
    +-	git_setup_gettext();
    +-
    +-	initialize_repository(the_repository);
    +-
    +-	attr_start();
    +-
    +-	trace2_initialize();
    +-	trace2_cmd_start(argv);
    +-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
    +-
    +-	if (!strbuf_getcwd(&tmp))
    +-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
      
    -
    - ## t/Makefile ##
    -@@ t/Makefile: perf:
    ++	init_git(argv);
    + 	result = cmd_main(argc, argv);
      
    - .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
    - 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
    -+
    -+.PHONY: libgitrs-sys-test
    -+libgitrs-sys-test:
    -+	$(QUIET)(\
    -+		cd ../contrib/libgit-rs/libgit-sys && \
    -+		cargo test \
    -+	)
    -+.PHONY: libgitrs-test
    -+libgitrs-test:
    -+	$(QUIET)(\
    -+		cd ../contrib/libgit-rs && \
    -+		cargo test \
    -+	)
    -+ifdef INCLUDE_LIBGIT_RS
    -+all:: libgitrs-sys-test libgitrs-test
    -+endif
    + 	/* Not exit(3), but a wrapper calling our common_exit() */
    + 	exit(result);
    + }
    +-
    +-static void check_bug_if_BUG(void)
    +-{
    +-	if (!bug_called_must_BUG)
    +-		return;
    +-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
    +-}
    +-
    +-/* We wrap exit() to call common_exit() in git-compat-util.h */
    +-int common_exit(const char *file, int line, int code)
    +-{
    +-	/*
    +-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
    +-	 * to e.g. turn -1 into 255. On a POSIX system this is
    +-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
    +-	 * anything there we don't need to guard this with an "ifdef".
    +-	 */
    +-	code &= 0xff;
    +-
    +-	check_bug_if_BUG();
    +-	trace2_cmd_exit_fl(file, line, code);
    +-
    +-	return code;
    +-}
1:  ed925d6a33 ! 2:  1ed010c378 libgit-sys: introduce Rust wrapper for libgit.a
    @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
     +	$(RM) -r contrib/libgit-rs/libgit-sys/target
    ++	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
    ++	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
    ++	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
      ifndef NO_PERL
      	$(RM) -r perl/build/
      endif
2:  8eeab7b418 = 3:  00762b57d0 libgit-sys: also export some config_set functions
3:  29599e9c7b ! 4:  4e5360931b libgit: add higher-level libgit crate
    @@ Metadata
      ## Commit message ##
         libgit: add higher-level libgit crate
     
    -    Wrap `struct config_set` and a few of its associated functions in
    -    libgit-sys. Also introduce a higher-level "libgit" crate which provides
    -    a more Rust-friendly interface to config_set structs.
    +    The C functions exported by libgit-sys do not provide an idiomatic Rust
    +    interface. To make it easier to use these functions via Rust, add a
    +    higher-level "libgit" crate, that wraps the lower-level configset API
    +    with an interface that is more Rust-y.
    +
    +    This combination of $X and $X-sys crates is a common pattern for FFI in
    +    Rust, as documented in "The Cargo Book" [1].
    +
    +    [1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages
     
         Co-authored-by: Josh Steadmon <steadmon@google.com>
         Signed-off-by: Josh Steadmon <steadmon@google.com>
    @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
     -	$(RM) -r contrib/libgit-rs/libgit-sys/target
     +	$(RM) -r contrib/libgit-rs/target contrib/libgit-rs/libgit-sys/target
    - ifndef NO_PERL
    - 	$(RM) -r perl/build/
    - endif
    + 	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
    + 	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
    + 	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
     
      ## contrib/libgit-rs/Cargo.lock (new) ##
     @@
    @@ contrib/libgit-rs/src/lib.rs (new)
     +        }
     +    }
     +
    -+    pub fn get_int(&mut self, key: &str) -> Option<c_int> {
    ++    pub fn get_int(&mut self, key: &str) -> Option<i32> {
     +        let key = CString::new(key).expect("Couldn't convert to CString");
     +        let mut val: c_int = 0;
     +        unsafe {
    @@ contrib/libgit-rs/src/lib.rs (new)
     +            }
     +        }
     +
    -+        Some(val)
    ++        Some(val.into())
     +    }
     +
     +    pub fn get_string(&mut self, key: &str) -> Option<String> {
-:  ---------- > 5:  15ce989de8 Makefile: add option to build and test libgit-rs and libgit-rs-sys

base-commit: 557ae147e6cdc9db121269b058c757ac5092f9c9
-- 
2.47.0.rc1.288.g06298d1525-goog

[PATCH v5 1/5] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 5 files changed, 99 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 3eab701b10..7caeb3c872 100644
--- a/Makefile
+++ b/Makefile
@@ -979,6 +979,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.47.0.rc1.288.g06298d1525-goog

[PATCH v5 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                    |  1 +
 Makefile                                      | 16 +++++
 contrib/libgit-rs/libgit-sys/Cargo.lock       | 69 +++++++++++++++++++
 contrib/libgit-rs/libgit-sys/Cargo.toml       | 17 +++++
 contrib/libgit-rs/libgit-sys/README.md        | 15 ++++
 contrib/libgit-rs/libgit-sys/build.rs         | 35 ++++++++++
 .../libgit-sys/public_symbol_export.c         | 21 ++++++
 .../libgit-sys/public_symbol_export.h         |  8 +++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 46 +++++++++++++
 9 files changed, 228 insertions(+)
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-rs/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-rs/libgit-sys/README.md
 create mode 100644 contrib/libgit-rs/libgit-sys/build.rs
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-rs/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-rs/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index 8caf3700c2..dfd72820fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,3 +248,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 7caeb3c872..4f7467a8c2 100644
--- a/Makefile
+++ b/Makefile
@@ -653,6 +653,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2713,6 +2715,7 @@ OBJECTS += $(XDIFF_OBJS)
 OBJECTS += $(FUZZ_OBJS)
 OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
+OBJECTS += contrib/libgit-rs/libgit-sys/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3720,6 +3723,10 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-rs/libgit-sys/target
+	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
+	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
+	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3865,3 +3872,12 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
 build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.lock b/contrib/libgit-rs/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/libgit-sys/Cargo.toml b/contrib/libgit-rs/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..7a230a4437
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-rs/libgit-sys/README.md b/contrib/libgit-rs/libgit-sys/README.md
new file mode 100644
index 0000000000..7a59602c30
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/README.md
@@ -0,0 +1,15 @@
+# cgit-info
+
+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
+library.
+
+## Building
+
+`cargo build` automatically builds and picks up on changes made to both
+the Rust wrapper and git.git code so there is no need to run `make`
+beforehand.
+
+## Running
+
+Assuming you don't make any changes to the Git source, you can just work from
+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
new file mode 100644
index 0000000000..8d74120191
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/libgit-rs/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..8e4b94d821
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -0,0 +1,21 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoids conflicts with other libraries such as libgit2.
+
+#include "git-compat-util.h"
+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
-- 
2.47.0.rc1.288.g06298d1525-goog

[PATCH v5 3/5] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .../libgit-sys/public_symbol_export.c         | 29 +++++++++++++++++
 .../libgit-sys/public_symbol_export.h         | 10 ++++++
 contrib/libgit-rs/libgit-sys/src/lib.rs       | 31 ++++++++++++++++++-
 3 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.c b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
index 8e4b94d821..29e8a5363e 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.c
@@ -3,11 +3,40 @@
 // avoids conflicts with other libraries such as libgit2.
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct config_set *cs = xmalloc(sizeof(struct config_set));
+	git_configset_init(cs);
+	return (struct libgit_config_set *) cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear((struct config_set *) cs);
+	free((struct config_set *) cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file((struct config_set *) cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int((struct config_set *) cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string((struct config_set *) cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-rs/libgit-sys/public_symbol_export.h b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-rs/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-rs/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.47.0.rc1.288.g06298d1525-goog

[PATCH v5 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

The C functions exported by libgit-sys do not provide an idiomatic Rust
interface. To make it easier to use these functions via Rust, add a
higher-level "libgit" crate, that wraps the lower-level configset API
with an interface that is more Rust-y.

This combination of $X and $X-sys crates is a common pattern for FFI in
Rust, as documented in "The Cargo Book" [1].

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 .gitignore                              |  1 +
 Makefile                                |  2 +-
 contrib/libgit-rs/Cargo.lock            | 77 ++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml            | 15 ++++
 contrib/libgit-rs/build.rs              |  4 ++
 contrib/libgit-rs/libgit-sys/src/lib.rs |  4 ++
 contrib/libgit-rs/src/lib.rs            | 95 +++++++++++++++++++++++++
 contrib/libgit-rs/testdata/config1      |  2 +
 contrib/libgit-rs/testdata/config2      |  2 +
 contrib/libgit-rs/testdata/config3      |  2 +
 10 files changed, 203 insertions(+), 1 deletion(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index dfd72820fb..0a42f27117 100644
--- a/.gitignore
+++ b/.gitignore
@@ -248,4 +248,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-rs/libgit-sys/target
diff --git a/Makefile b/Makefile
index 4f7467a8c2..1790c737bd 100644
--- a/Makefile
+++ b/Makefile
@@ -3723,7 +3723,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-rs/libgit-sys/target
+	$(RM) -r contrib/libgit-rs/target contrib/libgit-rs/libgit-sys/target
 	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
 	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
 	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..6b4387b9e7
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/libgit-sys/src/lib.rs b/contrib/libgit-rs/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-rs/libgit-sys/src/lib.rs
+++ b/contrib/libgit-rs/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..27b6fd63f1
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1,95 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
+use libgit_sys::*;
+
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<i32> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val.into())
+    }
+
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
-- 
2.47.0.rc1.288.g06298d1525-goog

[PATCH v5 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
to their respective Makefiles so they can be built and tested without
having to run cargo build/test.

Add build variable, INCLUDE_LIBGIT_RS, that when set, automatically
builds and tests libgit-rs and libgit-rs-sys when `make all` is run.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile                              | 26 ++++++++++++++++++++++++++
 contrib/libgit-rs/libgit-sys/build.rs |  2 +-
 t/Makefile                            | 16 ++++++++++++++++
 3 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 1790c737bd..f191ee32bb 100644
--- a/Makefile
+++ b/Makefile
@@ -412,6 +412,9 @@ include shared.mak
 # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
 # programs in oss-fuzz/.
 #
+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
+# test the Rust crates in contrib/libgit-rs/ and contrib/libgit-rs/libgit-sys/.
+#
 # === Optional library: libintl ===
 #
 # Define NO_GETTEXT if you don't want Git output to be translated.
@@ -2181,6 +2184,13 @@ ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
+ifdef INCLUDE_LIBGIT_RS
+	# Enable symbol hiding in contrib/libgit-rs/libgit-sys/libgitpub.a
+	# without making us rebuild the whole tree every time we run a Rust
+	# build.
+	BASIC_CFLAGS += -fvisibility=hidden
+endif
+
 ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
@@ -3873,6 +3883,22 @@ build-unit-tests: $(UNIT_TEST_PROGS)
 unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
+.PHONY: libgitrs-sys
+libgitrs-sys:
+	$(QUIET)(\
+		cd contrib/libgit-rs/libgit-sys && \
+		cargo build \
+	)
+.PHONY: libgitrs
+libgitrs:
+	$(QUIET)(\
+		cd contrib/libgit-rs && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs
+endif
+
 contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
 	$(LD) -r $^ -o $@
 
diff --git a/contrib/libgit-rs/libgit-sys/build.rs b/contrib/libgit-rs/libgit-sys/build.rs
index 8d74120191..d6d6210ab2 100644
--- a/contrib/libgit-rs/libgit-sys/build.rs
+++ b/contrib/libgit-rs/libgit-sys/build.rs
@@ -14,7 +14,7 @@ pub fn main() -> std::io::Result<()> {
         .env_remove("PROFILE")
         .current_dir(git_root.clone())
         .args([
-            "CFLAGS=-fvisibility=hidden",
+            "INCLUDE_LIBGIT_RS=YesPlease",
             "contrib/libgit-rs/libgit-sys/libgitpub.a",
         ])
         .output()
diff --git a/t/Makefile b/t/Makefile
index b2eb9f770b..066cb5c2b4 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -169,3 +169,19 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgitrs-sys-test
+libgitrs-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs/libgit-sys && \
+		cargo test \
+	)
+.PHONY: libgitrs-test
+libgitrs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgitrs-sys-test libgitrs-test
+endif
-- 
2.47.0.rc1.288.g06298d1525-goog

Re: [PATCH v5 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> A quick note: normally I'd wait longer for more feedback before sending
> out a new reroll, but due to some firefighting at $DAYJOB, I will not
> have time to focus on this series for the next several weeks, possibly
> up to a month. I wanted to get V5 out before then, but please understand
> that I will not be able to address new feedback for a while.
>
> This series provides two small Rust wrapper libraries around parts of
> Git: "libgit-sys", which exposes a few functions from libgit.a, and
> "libgit", which provides a more Rust-friendly interface to some of those
> functions. In addition to included unit tests, at $DAYJOB we have tested
> building JJ[1] with our library and used it to replace some of the
> libgit2-rs uses.
>
> [1] https://github.com/martinvonz/jj
>
> Please find V1 cover letter here:
> https://lore.kernel.org/git/cover.1723054623.git.steadmon@google.com/

Haven't seen any traffic on this one since it was posted.

It happened during my absense, but that is just one potential
reviewer, and it is unnerving that nobody showed any interested in
the topic, unlike any of the previous iterations with active
discussion subthreads.

Anybody interested in reviewing and championing the topic, or should
we wait until the topic gets refined and resubmit?

Thanks.

[PATCH v6 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

Apologies for the long delay on V6; I am finally back after several
months of $DAYJOB firefighting, holidays, and sick leave. I should have
time to devote to this series again, but given the lack of feedback on
V5 I am hopeful that this will be the final iteration of this series.

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/martinvonz/jj

There is known NEEDSWORK, but I feel that they can be addressed in
follow-up changes, rather than in this series. If you feel otherwise,
please let me know:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning

Changes in V6:
* Rebased onto current master, since V5 was several months old.

* Move libgit-sys out of libgit-rs; while this sort of nesting is common
  in Rust crates with standalone repositories, it doesn't make as much
  sense when they're contained in the larger Git project's repo.

* Standardize the naming of some of the Makefile targets to always
  include a dash in the "-rs" or "-sys" suffixes.

* Clean up READMEs and crate descriptions in preparation for
  uploading to crates.io.

Changes in V5:
* When building with INCLUDE_LIBGIT_RS defined, add
  "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
  visibility in libgitpub.a without causing `make all` to rebuild from
  scratch due to changing CFLAGS.

* Avoid using c_int in the higher-level Rust API.

* Remove libgitpub.a and intermediate files with `make clean`.

Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml


Calvin Wan (2):
  libgit: add higher-level libgit crate
  Makefile: add option to build and test libgit-rs and libgit-rs-sys

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                |  2 +
 Makefile                                  | 44 +++++++++++
 common-exit.c                             | 26 +++++++
 common-init.c                             | 63 +++++++++++++++
 common-init.h                             |  6 ++
 common-main.c                             | 83 +-------------------
 contrib/libgit-rs/Cargo.lock              | 77 ++++++++++++++++++
 contrib/libgit-rs/Cargo.toml              | 15 ++++
 contrib/libgit-rs/README.md               | 13 ++++
 contrib/libgit-rs/build.rs                |  4 +
 contrib/libgit-rs/src/lib.rs              | 95 +++++++++++++++++++++++
 contrib/libgit-rs/testdata/config1        |  2 +
 contrib/libgit-rs/testdata/config2        |  2 +
 contrib/libgit-rs/testdata/config3        |  2 +
 contrib/libgit-sys/Cargo.lock             | 69 ++++++++++++++++
 contrib/libgit-sys/Cargo.toml             | 18 +++++
 contrib/libgit-sys/README.md              |  4 +
 contrib/libgit-sys/build.rs               | 35 +++++++++
 contrib/libgit-sys/public_symbol_export.c | 50 ++++++++++++
 contrib/libgit-sys/public_symbol_export.h | 18 +++++
 contrib/libgit-sys/src/lib.rs             | 79 +++++++++++++++++++
 t/Makefile                                | 16 ++++
 22 files changed, 642 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

Range-diff against v5:
1:  1ae14207f6 = 1:  ff6cd62397 common-main: split init and exit code into new files
2:  1ed010c378 ! 2:  5fc66cdb16 libgit-sys: introduce Rust wrapper for libgit.a
    @@ .gitignore: Release/
      /git.VC.db
      *.dSYM
      /contrib/buildsystems/out
    -+/contrib/libgit-rs/libgit-sys/target
    ++/contrib/libgit-sys/target
     
      ## Makefile ##
     @@ Makefile: CURL_CONFIG = curl-config
    @@ Makefile: CURL_CONFIG = curl-config
      
      export TCL_PATH TCLTK_PATH
      
    -@@ Makefile: OBJECTS += $(XDIFF_OBJS)
    - OBJECTS += $(FUZZ_OBJS)
    - OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
    +@@ Makefile: OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
      OBJECTS += $(UNIT_TEST_OBJS)
    -+OBJECTS += contrib/libgit-rs/libgit-sys/public_symbol_export.o
    + OBJECTS += $(CLAR_TEST_OBJS)
    + OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
    ++OBJECTS += contrib/libgit-sys/public_symbol_export.o
      
      ifndef NO_CURL
      	OBJECTS += http.o http-walker.o remote-curl.o
    @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
    -+	$(RM) -r contrib/libgit-rs/libgit-sys/target
    -+	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
    -+	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
    -+	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
    ++	$(RM) -r contrib/libgit-sys/target
    ++	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
    ++	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
    ++	$(RM) -r contrib/libgit-sys/libgitpub.a
      ifndef NO_PERL
      	$(RM) -r perl/build/
      endif
    -@@ Makefile: $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o \
    - build-unit-tests: $(UNIT_TEST_PROGS)
    - unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
    +@@ Makefile: $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
    + build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
    + unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
      	$(MAKE) -C t/ unit-tests
     +
    -+contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    ++contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
     +	$(LD) -r $^ -o $@
     +
    -+contrib/libgit-rs/libgit-sys/hidden_symbol_export.o: contrib/libgit-rs/libgit-sys/partial_symbol_export.o
    ++contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
     +	$(OBJCOPY) --localize-hidden $^ $@
     +
    -+contrib/libgit-rs/libgit-sys/libgitpub.a: contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
    ++contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
     +	$(AR) $(ARFLAGS) $@ $^
     
    - ## contrib/libgit-rs/libgit-sys/Cargo.lock (new) ##
    + ## contrib/libgit-sys/Cargo.lock (new) ##
     @@
     +# This file is automatically @generated by Cargo.
     +# It is not intended for manual editing.
    @@ contrib/libgit-rs/libgit-sys/Cargo.lock (new)
     +source = "registry+https://github.com/rust-lang/crates.io-index"
     +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
     
    - ## contrib/libgit-rs/libgit-sys/Cargo.toml (new) ##
    + ## contrib/libgit-sys/Cargo.toml (new) ##
     @@
     +[package]
     +name = "libgit-sys"
    @@ contrib/libgit-rs/libgit-sys/Cargo.toml (new)
     +build = "build.rs"
     +links = "gitpub"
     +rust-version = "1.63"
    ++description = "Native bindings to a portion of libgit"
     +
     +[lib]
     +path = "src/lib.rs"
    @@ contrib/libgit-rs/libgit-sys/Cargo.toml (new)
     +autocfg = "1.4.0"
     +make-cmd = "0.1.0"
     
    - ## contrib/libgit-rs/libgit-sys/README.md (new) ##
    + ## contrib/libgit-sys/README.md (new) ##
     @@
    -+# cgit-info
    ++# libgit-sys
     +
    -+A small hacky proof-of-concept showing how to provide a Rust FFI for the Git
    -+library.
    -+
    -+## Building
    -+
    -+`cargo build` automatically builds and picks up on changes made to both
    -+the Rust wrapper and git.git code so there is no need to run `make`
    -+beforehand.
    -+
    -+## Running
    -+
    -+Assuming you don't make any changes to the Git source, you can just work from
    -+`contrib/cgit-rs` and use `cargo build` or `cargo run` as usual.
    ++A small proof-of-concept crate showing how to provide a Rust FFI to Git
    ++internals.
     
    - ## contrib/libgit-rs/libgit-sys/build.rs (new) ##
    + ## contrib/libgit-sys/build.rs (new) ##
     @@
     +use std::env;
     +use std::path::PathBuf;
    @@ contrib/libgit-rs/libgit-sys/build.rs (new)
     +    ac.emit_has_path("std::ffi::c_char");
     +
     +    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
    -+    let git_root = crate_root.join("../../..");
    ++    let git_root = crate_root.join("../..");
     +    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
     +
     +    let make_output = make_cmd::gnu_make()
    @@ contrib/libgit-rs/libgit-sys/build.rs (new)
     +        .current_dir(git_root.clone())
     +        .args([
     +            "CFLAGS=-fvisibility=hidden",
    -+            "contrib/libgit-rs/libgit-sys/libgitpub.a",
    ++            "contrib/libgit-sys/libgitpub.a",
     +        ])
     +        .output()
     +        .expect("Make failed to run");
    @@ contrib/libgit-rs/libgit-sys/build.rs (new)
     +    Ok(())
     +}
     
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.c (new) ##
    + ## contrib/libgit-sys/public_symbol_export.c (new) ##
     @@
     +// Shim to publicly export Git symbols. These must be renamed so that the
     +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
     +// avoids conflicts with other libraries such as libgit2.
     +
     +#include "git-compat-util.h"
    -+#include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
    ++#include "contrib/libgit-sys/public_symbol_export.h"
     +#include "version.h"
     +
     +#pragma GCC visibility push(default)
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.c (new)
     +
     +#pragma GCC visibility pop
     
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.h (new) ##
    + ## contrib/libgit-sys/public_symbol_export.h (new) ##
     @@
     +#ifndef PUBLIC_SYMBOL_EXPORT_H
     +#define PUBLIC_SYMBOL_EXPORT_H
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.h (new)
     +
     +#endif /* PUBLIC_SYMBOL_EXPORT_H */
     
    - ## contrib/libgit-rs/libgit-sys/src/lib.rs (new) ##
    + ## contrib/libgit-sys/src/lib.rs (new) ##
     @@
     +#[cfg(has_std__ffi__c_char)]
     +use std::ffi::c_char;
3:  00762b57d0 ! 3:  03f39b6c3a libgit-sys: also export some config_set functions
    @@ Commit message
         Signed-off-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Josh Steadmon <steadmon@google.com>
     
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.c ##
    + ## contrib/libgit-sys/public_symbol_export.c ##
     @@
      // avoids conflicts with other libraries such as libgit2.
      
      #include "git-compat-util.h"
     +#include "config.h"
    - #include "contrib/libgit-rs/libgit-sys/public_symbol_export.h"
    + #include "contrib/libgit-sys/public_symbol_export.h"
      #include "version.h"
      
      #pragma GCC visibility push(default)
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.c
      {
      	return git_user_agent();
     
    - ## contrib/libgit-rs/libgit-sys/public_symbol_export.h ##
    + ## contrib/libgit-sys/public_symbol_export.h ##
     @@
      #ifndef PUBLIC_SYMBOL_EXPORT_H
      #define PUBLIC_SYMBOL_EXPORT_H
    @@ contrib/libgit-rs/libgit-sys/public_symbol_export.h
      
      const char *libgit_user_agent_sanitized(void);
     
    - ## contrib/libgit-rs/libgit-sys/src/lib.rs ##
    + ## contrib/libgit-sys/src/lib.rs ##
     @@
      #[cfg(has_std__ffi__c_char)]
     -use std::ffi::c_char;
4:  4e5360931b ! 4:  65166ea0c0 libgit: add higher-level libgit crate
    @@ .gitignore: Release/
      *.dSYM
      /contrib/buildsystems/out
     +/contrib/libgit-rs/target
    - /contrib/libgit-rs/libgit-sys/target
    + /contrib/libgit-sys/target
     
      ## Makefile ##
     @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
    --	$(RM) -r contrib/libgit-rs/libgit-sys/target
    -+	$(RM) -r contrib/libgit-rs/target contrib/libgit-rs/libgit-sys/target
    - 	$(RM) -r contrib/libgit-rs/libgit-sys/partial_symbol_export.o
    - 	$(RM) -r contrib/libgit-rs/libgit-sys/hidden_symbol_export.o
    - 	$(RM) -r contrib/libgit-rs/libgit-sys/libgitpub.a
    +-	$(RM) -r contrib/libgit-sys/target
    ++	$(RM) -r contrib/libgit-rs/target contrib/libgit-sys/target
    + 	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
    + 	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
    + 	$(RM) -r contrib/libgit-sys/libgitpub.a
     
      ## contrib/libgit-rs/Cargo.lock (new) ##
     @@
    @@ contrib/libgit-rs/Cargo.toml (new)
     +path = "src/lib.rs"
     +
     +[dependencies]
    -+libgit-sys = { version = "0.1.0", path = "libgit-sys" }
    ++libgit-sys = { version = "0.1.0", path = "../libgit-sys" }
     +
     +[build-dependencies]
     +autocfg = "1.4.0"
     
    + ## contrib/libgit-rs/README.md (new) ##
    +@@
    ++# libgit-rs
    ++
    ++Proof-of-concept Git bindings for Rust.
    ++
    ++```toml
    ++[dependencies]
    ++libgit = "0.1.0"
    ++```
    ++
    ++## Rust version requirements
    ++
    ++libgit-rs should support Rust versions at least as old as the version included
    ++in Debian stable (currently 1.63).
    +
      ## contrib/libgit-rs/build.rs (new) ##
     @@
     +pub fn main() {
    @@ contrib/libgit-rs/build.rs (new)
     +    ac.emit_has_path("std::ffi::c_char");
     +}
     
    - ## contrib/libgit-rs/libgit-sys/src/lib.rs ##
    -@@
    -+use std::ffi::c_void;
    -+
    - #[cfg(has_std__ffi__c_char)]
    - use std::ffi::{c_char, c_int};
    - 
    -@@ contrib/libgit-rs/libgit-sys/src/lib.rs: pub struct libgit_config_set {
    - }
    - 
    - extern "C" {
    -+    pub fn free(ptr: *mut c_void);
    -+
    -     pub fn libgit_user_agent() -> *const c_char;
    -     pub fn libgit_user_agent_sanitized() -> *const c_char;
    - 
    -
      ## contrib/libgit-rs/src/lib.rs (new) ##
     @@
     +use std::ffi::{c_void, CStr, CString};
    @@ contrib/libgit-rs/testdata/config3 (new)
     @@
     +[trace2]
     +	eventNesting = 3
    +
    + ## contrib/libgit-sys/src/lib.rs ##
    +@@
    ++use std::ffi::c_void;
    ++
    + #[cfg(has_std__ffi__c_char)]
    + use std::ffi::{c_char, c_int};
    + 
    +@@ contrib/libgit-sys/src/lib.rs: pub struct libgit_config_set {
    + }
    + 
    + extern "C" {
    ++    pub fn free(ptr: *mut c_void);
    ++
    +     pub fn libgit_user_agent() -> *const c_char;
    +     pub fn libgit_user_agent_sanitized() -> *const c_char;
    + 
5:  15ce989de8 ! 5:  84706f0db7 Makefile: add option to build and test libgit-rs and libgit-rs-sys
    @@ Makefile: ifdef FSMONITOR_OS_SETTINGS
      ifeq ($(TCLTK_PATH),)
      NO_TCLTK = NoThanks
      endif
    -@@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS)
    - unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X
    +@@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
    + unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
      	$(MAKE) -C t/ unit-tests
      
    -+.PHONY: libgitrs-sys
    -+libgitrs-sys:
    ++.PHONY: libgit-sys
    ++libgit-sys:
     +	$(QUIET)(\
    -+		cd contrib/libgit-rs/libgit-sys && \
    ++		cd contrib/libgit-sys && \
     +		cargo build \
     +	)
    -+.PHONY: libgitrs
    -+libgitrs:
    ++.PHONY: libgit-rs
    ++libgit-rs:
     +	$(QUIET)(\
     +		cd contrib/libgit-rs && \
     +		cargo build \
     +	)
     +ifdef INCLUDE_LIBGIT_RS
    -+all:: libgitrs
    ++all:: libgit-rs
     +endif
     +
    - contrib/libgit-rs/libgit-sys/partial_symbol_export.o: contrib/libgit-rs/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    + contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
      	$(LD) -r $^ -o $@
      
     
    - ## contrib/libgit-rs/libgit-sys/build.rs ##
    -@@ contrib/libgit-rs/libgit-sys/build.rs: pub fn main() -> std::io::Result<()> {
    + ## contrib/libgit-sys/build.rs ##
    +@@ contrib/libgit-sys/build.rs: pub fn main() -> std::io::Result<()> {
              .env_remove("PROFILE")
              .current_dir(git_root.clone())
              .args([
     -            "CFLAGS=-fvisibility=hidden",
     +            "INCLUDE_LIBGIT_RS=YesPlease",
    -             "contrib/libgit-rs/libgit-sys/libgitpub.a",
    +             "contrib/libgit-sys/libgitpub.a",
              ])
              .output()
     
    @@ t/Makefile: perf:
      .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
      	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
     +
    -+.PHONY: libgitrs-sys-test
    -+libgitrs-sys-test:
    ++.PHONY: libgit-sys-test
    ++libgit-sys-test:
     +	$(QUIET)(\
    -+		cd ../contrib/libgit-rs/libgit-sys && \
    ++		cd ../contrib/libgit-sys && \
     +		cargo test \
     +	)
    -+.PHONY: libgitrs-test
    -+libgitrs-test:
    ++.PHONY: libgit-rs-test
    ++libgit-rs-test:
     +	$(QUIET)(\
     +		cd ../contrib/libgit-rs && \
     +		cargo test \
     +	)
     +ifdef INCLUDE_LIBGIT_RS
    -+all:: libgitrs-sys-test libgitrs-test
    ++all:: libgit-sys-test libgit-rs-test
     +endif

base-commit: 757161efcca150a9a96b312d9e780a071e601a03
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v6 1/5] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 5 files changed, 99 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 97e8385b66..27e68ac039 100644
--- a/Makefile
+++ b/Makefile
@@ -981,6 +981,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
-- 
2.48.0.rc2.279.g1de40edade-goog

Re: [PATCH v6 1/5] common-main: split init and exit code into new files

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> This split has previously been proposed ([1], [2]) to support fuzz tests
> and unit tests by avoiding conflicting definitions for main(). However,
> both of those issues were resolved by other methods of avoiding symbol
> conflicts. Now we are trying to make libgit.a more self-contained, so
> hopefully we can revisit this approach.

Yay!

> Additionally, move the initialization code out of main() into a new
> init_git() function in its own file. Include this in libgit.a as well,
> so that external users can share our setup code without calling our
> main().

Sounds good.

> diff --git a/common-main.c b/common-main.c
> index 8e68ac9e42..6b7ab077b0 100644
> --- a/common-main.c
> +++ b/common-main.c
> @@ -1,92 +1,13 @@
> ...
> +#include "common-init.h"
>  
>  int main(int argc, const char **argv)
>  {
>  	int result;
>  
> +	init_git(argv);
>  	result = cmd_main(argc, argv);
>  
>  	/* Not exit(3), but a wrapper calling our common_exit() */
>  	exit(result);
>  }

Nice.  Very nice.  I wasn't too carefully validating what we lost
here is identical to what we added to init_git(), but hopefully
others would spot and stop us if that is not the case.

Thanks.

Re: [PATCH v6 1/5] common-main: split init and exit code into new files

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Josh Steadmon <steadmon@google.com> writes:
>
>> This split has previously been proposed ([1], [2]) to support fuzz tests
>> and unit tests by avoiding conflicting definitions for main(). However,
>> both of those issues were resolved by other methods of avoiding symbol
>> conflicts. Now we are trying to make libgit.a more self-contained, so
>> hopefully we can revisit this approach.
>
> Yay!
>
>> Additionally, move the initialization code out of main() into a new
>> init_git() function in its own file. Include this in libgit.a as well,
>> so that external users can share our setup code without calling our
>> main().
>
> Sounds good.
>
>> diff --git a/common-main.c b/common-main.c
>> index 8e68ac9e42..6b7ab077b0 100644
>> --- a/common-main.c
>> +++ b/common-main.c
>> @@ -1,92 +1,13 @@
>> ...
>> +#include "common-init.h"
>>  
>>  int main(int argc, const char **argv)
>>  {
>>  	int result;
>>  
>> +	init_git(argv);
>>  	result = cmd_main(argc, argv);
>>  
>>  	/* Not exit(3), but a wrapper calling our common_exit() */
>>  	exit(result);
>>  }
>
> Nice.  Very nice.  I wasn't too carefully validating what we lost
> here is identical to what we added to init_git(), but hopefully
> others would spot and stop us if that is not the case.
>
> Thanks.

Unfortunately, build based on meson does not seem to like the
init_git() thing.  Perhaps we need to add some missing files to
relevant lists in meson.build file or something silly like that?

https://github.com/git/git/actions/runs/12800227601/job/35687658673#step:8:961

Re: [PATCH v6 1/5] common-main: split init and exit code into new files

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Unfortunately, build based on meson does not seem to like the
> init_git() thing.  Perhaps we need to add some missing files to
> relevant lists in meson.build file or something silly like that?
>
> https://github.com/git/git/actions/runs/12800227601/job/35687658673#step:8:961

I needed the following to get "meson compile" pass in my local
environment.  I suspect that Mesonized CI jobs exercise a bit more
than just "meson compile", so there is no guarantee that the
following is enough, but at least hopefully it would nudge you (and
those who may be interested in helping to build a working Rust
bindings) in the right direction.

I think it should be squashed into the step these files are added,
i.e. [PATCH 1/5].

Thanks.

 meson.build | 2 ++
 1 file changed, 2 insertions(+)

diff --git c/meson.build w/meson.build
index 0064eb64f5..e5ba28b47f 100644
--- c/meson.build
+++ w/meson.build
@@ -245,6 +245,8 @@ libgit_sources = [
   'commit-graph.c',
   'commit-reach.c',
   'commit.c',
+  'common-exit.c',
+  'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
   'compat/terminal.c',

Re: [PATCH v6 1/5] common-main: split init and exit code into new files

[Patrick Steinhardt]

On Thu, Jan 16, 2025 at 01:02:33PM -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Unfortunately, build based on meson does not seem to like the
> > init_git() thing.  Perhaps we need to add some missing files to
> > relevant lists in meson.build file or something silly like that?
> >
> > https://github.com/git/git/actions/runs/12800227601/job/35687658673#step:8:961
> 
> I needed the following to get "meson compile" pass in my local
> environment.  I suspect that Mesonized CI jobs exercise a bit more
> than just "meson compile", so there is no guarantee that the
> following is enough, but at least hopefully it would nudge you (and
> those who may be interested in helping to build a working Rust
> bindings) in the right direction.
> 
> I think it should be squashed into the step these files are added,
> i.e. [PATCH 1/5].
> 
> Thanks.
> 
>  meson.build | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git c/meson.build w/meson.build
> index 0064eb64f5..e5ba28b47f 100644
> --- c/meson.build
> +++ w/meson.build
> @@ -245,6 +245,8 @@ libgit_sources = [
>    'commit-graph.c',
>    'commit-reach.c',
>    'commit.c',
> +  'common-exit.c',
> +  'common-init.c',
>    'compat/nonblock.c',
>    'compat/obstack.c',
>    'compat/terminal.c',

Yeah, I remember having the same hunk while Meson was still in-flight in
order to make it compatible with "seen". So this should be sufficient.

Patrick

Re: [PATCH v6 1/5] common-main: split init and exit code into new files

[Josh Steadmon]

On 2025.01.17 10:44, Patrick Steinhardt wrote:
> On Thu, Jan 16, 2025 at 01:02:33PM -0800, Junio C Hamano wrote:
> > Junio C Hamano <gitster@pobox.com> writes:
> > 
> > > Unfortunately, build based on meson does not seem to like the
> > > init_git() thing.  Perhaps we need to add some missing files to
> > > relevant lists in meson.build file or something silly like that?
> > >
> > > https://github.com/git/git/actions/runs/12800227601/job/35687658673#step:8:961
> > 
> > I needed the following to get "meson compile" pass in my local
> > environment.  I suspect that Mesonized CI jobs exercise a bit more
> > than just "meson compile", so there is no guarantee that the
> > following is enough, but at least hopefully it would nudge you (and
> > those who may be interested in helping to build a working Rust
> > bindings) in the right direction.
> > 
> > I think it should be squashed into the step these files are added,
> > i.e. [PATCH 1/5].
> > 
> > Thanks.
> > 
> >  meson.build | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git c/meson.build w/meson.build
> > index 0064eb64f5..e5ba28b47f 100644
> > --- c/meson.build
> > +++ w/meson.build
> > @@ -245,6 +245,8 @@ libgit_sources = [
> >    'commit-graph.c',
> >    'commit-reach.c',
> >    'commit.c',
> > +  'common-exit.c',
> > +  'common-init.c',
> >    'compat/nonblock.c',
> >    'compat/obstack.c',
> >    'compat/terminal.c',
> 
> Yeah, I remember having the same hunk while Meson was still in-flight in
> order to make it compatible with "seen". So this should be sufficient.
> 
> Patrick

ACK, sorry about that. I missed the Meson developments while I was
off-list, so it wasn't on my radar. I'll make sure to squash the fix in
to V7. Thanks both.

[PATCH v6 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                |  1 +
 Makefile                                  | 16 ++++++
 contrib/libgit-sys/Cargo.lock             | 69 +++++++++++++++++++++++
 contrib/libgit-sys/Cargo.toml             | 18 ++++++
 contrib/libgit-sys/README.md              |  4 ++
 contrib/libgit-sys/build.rs               | 35 ++++++++++++
 contrib/libgit-sys/public_symbol_export.c | 21 +++++++
 contrib/libgit-sys/public_symbol_export.h |  8 +++
 contrib/libgit-sys/src/lib.rs             | 46 +++++++++++++++
 9 files changed, 218 insertions(+)
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index e82aa19df0..31d7e64287 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,3 +250,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 27e68ac039..47e864a861 100644
--- a/Makefile
+++ b/Makefile
@@ -657,6 +657,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2731,6 +2733,7 @@ OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
 OBJECTS += $(UNIT_TEST_OBJS)
 OBJECTS += $(CLAR_TEST_OBJS)
 OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
+OBJECTS += contrib/libgit-sys/public_symbol_export.o
 
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
@@ -3726,6 +3729,10 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-sys/target
+	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
+	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
+	$(RM) -r contrib/libgit-sys/libgitpub.a
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3887,3 +3894,12 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
 build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-sys/Cargo.lock b/contrib/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-sys/Cargo.toml b/contrib/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..15b28c3585
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.toml
@@ -0,0 +1,18 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63"
+description = "Native bindings to a portion of libgit"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
new file mode 100644
index 0000000000..c061cfcaf5
--- /dev/null
+++ b/contrib/libgit-sys/README.md
@@ -0,0 +1,4 @@
+# libgit-sys
+
+A small proof-of-concept crate showing how to provide a Rust FFI to Git
+internals.
diff --git a/contrib/libgit-sys/build.rs b/contrib/libgit-sys/build.rs
new file mode 100644
index 0000000000..b6c65193bc
--- /dev/null
+++ b/contrib/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "CFLAGS=-fvisibility=hidden",
+            "contrib/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..7cd5007902
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -0,0 +1,21 @@
+// Shim to publicly export Git symbols. These must be renamed so that the
+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
+// avoids conflicts with other libraries such as libgit2.
+
+#include "git-compat-util.h"
+#include "contrib/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
-- 
2.48.0.rc2.279.g1de40edade-goog

Re: [PATCH v6 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> diff --git a/Makefile b/Makefile
> index 27e68ac039..47e864a861 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -657,6 +657,8 @@ CURL_CONFIG = curl-config
>  GCOV = gcov
>  STRIP = strip
>  SPATCH = spatch
> +LD = ld
> +OBJCOPY = objcopy

This assumes GNU binutils is available.  As long as our intention is
to start the Rust support as an optional feature, that is OK.
Hopefully the piece that requires $(OBJCOPY) is arranged to be
easily opted out.  Let's keep reading.

> @@ -2731,6 +2733,7 @@ OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
>  OBJECTS += $(UNIT_TEST_OBJS)
>  OBJECTS += $(CLAR_TEST_OBJS)
>  OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
> +OBJECTS += contrib/libgit-sys/public_symbol_export.o

This is compiled for everybody, even for those whose platform cannot
support Rust interface (or those who choose not to build it).  As
long as what is in the file is written portably, it is fine to have
stubs and entry points that their build will not use.

>  ifndef NO_CURL
>  	OBJECTS += http.o http-walker.o remote-curl.o
> @@ -3726,6 +3729,10 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
>  	$(MAKE) -C Documentation/ clean
>  	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
> +	$(RM) -r contrib/libgit-sys/target
> +	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
> +	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
> +	$(RM) -r contrib/libgit-sys/libgitpub.a

Which one of the above is a directory?  The latter three smells like
a regular file, so we shouldn't say "-r" there.

> @@ -3887,3 +3894,12 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
>  build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
>  unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
>  	$(MAKE) -C t/ unit-tests
> +
> +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
> +	$(LD) -r $^ -o $@

OK.  We build a "relocatable" object, which is unconditionally made
as part of $(OBJECTS) above.  Even without GNU binutils "ld", people
hopefully can convince their linker to do the equivalent.  I am not
sure if it is healthy to assume that such a linker also uses "-r"
for the feature, so we may have to make this rule more customizable,
or make partial_symbol_export.o only conditionally part of $(OBJECTS)
to allow them to opt out.

> +contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
> +	$(OBJCOPY) --localize-hidden $^ $@

Unlike the "public" thing, hidden_symbol_export.o was not made part
of $(OBJECTS), so this part is arranged to allow people without
$(OBJCOPY) to easily opt out of this part of the system, which is
good.

> +contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
> +	$(AR) $(ARFLAGS) $@ $^

Likewise, people can easily opt out of building "libgitpub.a", which
is good (these targets are triggered only from build.rs).

> diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
> new file mode 100644
> index 0000000000..c061cfcaf5
> --- /dev/null
> +++ b/contrib/libgit-sys/README.md
> @@ -0,0 +1,4 @@
> +# libgit-sys
> +
> +A small proof-of-concept crate showing how to provide a Rust FFI to Git
> +internals.

OK.

> diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> new file mode 100644
> index 0000000000..7cd5007902
> --- /dev/null
> +++ b/contrib/libgit-sys/public_symbol_export.c
> @@ -0,0 +1,21 @@
> +// Shim to publicly export Git symbols. These must be renamed so that the
> +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> +// avoids conflicts with other libraries such as libgit2.

Style.

> +#include "git-compat-util.h"
> +#include "contrib/libgit-sys/public_symbol_export.h"
> +#include "version.h"
> +
> +#pragma GCC visibility push(default)
> +
> +const char *libgit_user_agent(void)
> +{
> +	return git_user_agent();
> +}
> +
> +const char *libgit_user_agent_sanitized(void)
> +{
> +	return git_user_agent_sanitized();
> +}
> +
> +#pragma GCC visibility pop

I do not think we would mind not having Rust binding support on
platforms without GCC (and clang---I assume it would be aware of and
react to that #pragma GCC the same way?).  But do we allow this file
to be left uncompiled when the build wants to opt out of Rust
support?

> diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
> new file mode 100644
> index 0000000000..a3372f93fa
> --- /dev/null
> +++ b/contrib/libgit-sys/public_symbol_export.h
> @@ -0,0 +1,8 @@
> +#ifndef PUBLIC_SYMBOL_EXPORT_H
> +#define PUBLIC_SYMBOL_EXPORT_H
> +
> +const char *libgit_user_agent(void);
> +
> +const char *libgit_user_agent_sanitized(void);
> +
> +#endif /* PUBLIC_SYMBOL_EXPORT_H */

OK.

Re: [PATCH v6 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2025.01.15 15:13, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > diff --git a/Makefile b/Makefile
> > index 27e68ac039..47e864a861 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -657,6 +657,8 @@ CURL_CONFIG = curl-config
> >  GCOV = gcov
> >  STRIP = strip
> >  SPATCH = spatch
> > +LD = ld
> > +OBJCOPY = objcopy
> 
> This assumes GNU binutils is available.  As long as our intention is
> to start the Rust support as an optional feature, that is OK.
> Hopefully the piece that requires $(OBJCOPY) is arranged to be
> easily opted out.  Let's keep reading.
> 
> > @@ -2731,6 +2733,7 @@ OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
> >  OBJECTS += $(UNIT_TEST_OBJS)
> >  OBJECTS += $(CLAR_TEST_OBJS)
> >  OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
> > +OBJECTS += contrib/libgit-sys/public_symbol_export.o
> 
> This is compiled for everybody, even for those whose platform cannot
> support Rust interface (or those who choose not to build it).  As
> long as what is in the file is written portably, it is fine to have
> stubs and entry points that their build will not use.

Later on in the series we add an INCLUDE_LIBGIT_RS variable to control
builds and tests; I don't see any reason why we can't move that earlier
in the series, so I'll do so in V7 and then look at what we need to do
to make things more portable.

> >  ifndef NO_CURL
> >  	OBJECTS += http.o http-walker.o remote-curl.o
> > @@ -3726,6 +3729,10 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> >  	$(MAKE) -C Documentation/ clean
> >  	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
> > +	$(RM) -r contrib/libgit-sys/target
> > +	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
> > +	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
> > +	$(RM) -r contrib/libgit-sys/libgitpub.a
> 
> Which one of the above is a directory?  The latter three smells like
> a regular file, so we shouldn't say "-r" there.
> 
> > @@ -3887,3 +3894,12 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
> >  build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
> >  unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
> >  	$(MAKE) -C t/ unit-tests
> > +
> > +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
> > +	$(LD) -r $^ -o $@
> 
> OK.  We build a "relocatable" object, which is unconditionally made
> as part of $(OBJECTS) above.  Even without GNU binutils "ld", people
> hopefully can convince their linker to do the equivalent.  I am not
> sure if it is healthy to assume that such a linker also uses "-r"
> for the feature, so we may have to make this rule more customizable,
> or make partial_symbol_export.o only conditionally part of $(OBJECTS)
> to allow them to opt out.
> 
> > +contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
> > +	$(OBJCOPY) --localize-hidden $^ $@
> 
> Unlike the "public" thing, hidden_symbol_export.o was not made part
> of $(OBJECTS), so this part is arranged to allow people without
> $(OBJCOPY) to easily opt out of this part of the system, which is
> good.
> 
> > +contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
> > +	$(AR) $(ARFLAGS) $@ $^
> 
> Likewise, people can easily opt out of building "libgitpub.a", which
> is good (these targets are triggered only from build.rs).
> 
> > diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
> > new file mode 100644
> > index 0000000000..c061cfcaf5
> > --- /dev/null
> > +++ b/contrib/libgit-sys/README.md
> > @@ -0,0 +1,4 @@
> > +# libgit-sys
> > +
> > +A small proof-of-concept crate showing how to provide a Rust FFI to Git
> > +internals.
> 
> OK.
> 
> > diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> > new file mode 100644
> > index 0000000000..7cd5007902
> > --- /dev/null
> > +++ b/contrib/libgit-sys/public_symbol_export.c
> > @@ -0,0 +1,21 @@
> > +// Shim to publicly export Git symbols. These must be renamed so that the
> > +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> > +// avoids conflicts with other libraries such as libgit2.
> 
> Style.
> 
> > +#include "git-compat-util.h"
> > +#include "contrib/libgit-sys/public_symbol_export.h"
> > +#include "version.h"
> > +
> > +#pragma GCC visibility push(default)
> > +
> > +const char *libgit_user_agent(void)
> > +{
> > +	return git_user_agent();
> > +}
> > +
> > +const char *libgit_user_agent_sanitized(void)
> > +{
> > +	return git_user_agent_sanitized();
> > +}
> > +
> > +#pragma GCC visibility pop
> 
> I do not think we would mind not having Rust binding support on
> platforms without GCC (and clang---I assume it would be aware of and
> react to that #pragma GCC the same way?).  But do we allow this file
> to be left uncompiled when the build wants to opt out of Rust
> support?
> 
> > diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
> > new file mode 100644
> > index 0000000000..a3372f93fa
> > --- /dev/null
> > +++ b/contrib/libgit-sys/public_symbol_export.h
> > @@ -0,0 +1,8 @@
> > +#ifndef PUBLIC_SYMBOL_EXPORT_H
> > +#define PUBLIC_SYMBOL_EXPORT_H
> > +
> > +const char *libgit_user_agent(void);
> > +
> > +const char *libgit_user_agent_sanitized(void);
> > +
> > +#endif /* PUBLIC_SYMBOL_EXPORT_H */
> 
> OK.

Re: [PATCH v6 2/5] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2025.01.15 15:13, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> > new file mode 100644
> > index 0000000000..7cd5007902
> > --- /dev/null
> > +++ b/contrib/libgit-sys/public_symbol_export.c
> > @@ -0,0 +1,21 @@
> > +// Shim to publicly export Git symbols. These must be renamed so that the
> > +// original symbols can be hidden. Renaming these with a "libgit_" prefix also
> > +// avoids conflicts with other libraries such as libgit2.
> 
> Style.
> 
> > +#include "git-compat-util.h"
> > +#include "contrib/libgit-sys/public_symbol_export.h"
> > +#include "version.h"
> > +
> > +#pragma GCC visibility push(default)
> > +
> > +const char *libgit_user_agent(void)
> > +{
> > +	return git_user_agent();
> > +}
> > +
> > +const char *libgit_user_agent_sanitized(void)
> > +{
> > +	return git_user_agent_sanitized();
> > +}
> > +
> > +#pragma GCC visibility pop
> 
> I do not think we would mind not having Rust binding support on
> platforms without GCC (and clang---I assume it would be aware of and
> react to that #pragma GCC the same way?).

Yes, GCC and clang both handle this the same way.


> But do we allow this file
> to be left uncompiled when the build wants to opt out of Rust
> support?

Yes, in V7 this will only be built if INCLUDE_LIBGIT_RS is set.

[PATCH v6 3/5] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/libgit-sys/public_symbol_export.c | 29 +++++++++++++++++++++
 contrib/libgit-sys/public_symbol_export.h | 10 ++++++++
 contrib/libgit-sys/src/lib.rs             | 31 ++++++++++++++++++++++-
 3 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
index 7cd5007902..1799fc6fde 100644
--- a/contrib/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -3,11 +3,40 @@
 // avoids conflicts with other libraries such as libgit2.
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct config_set *cs = xmalloc(sizeof(struct config_set));
+	git_configset_init(cs);
+	return (struct libgit_config_set *) cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear((struct config_set *) cs);
+	free((struct config_set *) cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file((struct config_set *) cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int((struct config_set *) cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string((struct config_set *) cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v6 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

The C functions exported by libgit-sys do not provide an idiomatic Rust
interface. To make it easier to use these functions via Rust, add a
higher-level "libgit" crate, that wraps the lower-level configset API
with an interface that is more Rust-y.

This combination of $X and $X-sys crates is a common pattern for FFI in
Rust, as documented in "The Cargo Book" [1].

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                         |  1 +
 Makefile                           |  2 +-
 contrib/libgit-rs/Cargo.lock       | 77 ++++++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml       | 15 +++++
 contrib/libgit-rs/README.md        | 13 ++++
 contrib/libgit-rs/build.rs         |  4 ++
 contrib/libgit-rs/src/lib.rs       | 95 ++++++++++++++++++++++++++++++
 contrib/libgit-rs/testdata/config1 |  2 +
 contrib/libgit-rs/testdata/config2 |  2 +
 contrib/libgit-rs/testdata/config3 |  2 +
 contrib/libgit-sys/src/lib.rs      |  4 ++
 11 files changed, 216 insertions(+), 1 deletion(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index 31d7e64287..acdd8ce7c7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,4 +250,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 47e864a861..230d366457 100644
--- a/Makefile
+++ b/Makefile
@@ -3729,7 +3729,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-sys/target
+	$(RM) -r contrib/libgit-rs/target contrib/libgit-sys/target
 	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
 	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
 	$(RM) -r contrib/libgit-sys/libgitpub.a
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..634435cd6c
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "../libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/README.md b/contrib/libgit-rs/README.md
new file mode 100644
index 0000000000..ff945e1ce2
--- /dev/null
+++ b/contrib/libgit-rs/README.md
@@ -0,0 +1,13 @@
+# libgit-rs
+
+Proof-of-concept Git bindings for Rust.
+
+```toml
+[dependencies]
+libgit = "0.1.0"
+```
+
+## Rust version requirements
+
+libgit-rs should support Rust versions at least as old as the version included
+in Debian stable (currently 1.63).
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..27b6fd63f1
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1,95 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
+use libgit_sys::*;
+
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    pub fn get_int(&mut self, key: &str) -> Option<i32> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val.into())
+    }
+
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
-- 
2.48.0.rc2.279.g1de40edade-goog

Re: [PATCH v6 4/5] libgit: add higher-level libgit crate

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 6016 bytes --]

On 2025-01-15 at 20:05:43, Josh Steadmon wrote:
> From: Calvin Wan <calvinwan@google.com>
> diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
> new file mode 100644
> index 0000000000..a30c7c8d33
> --- /dev/null
> +++ b/contrib/libgit-rs/Cargo.lock
> @@ -0,0 +1,77 @@
> +# This file is automatically @generated by Cargo.
> +# It is not intended for manual editing.
> +version = 3
> +
> +[[package]]
> +name = "autocfg"
> +version = "1.4.0"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
> +
> +[[package]]
> +name = "cc"
> +version = "1.1.15"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
> +dependencies = [
> + "shlex",
> +]
> +
> +[[package]]
> +name = "libc"
> +version = "0.2.158"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
> +
> +[[package]]
> +name = "libgit"
> +version = "0.1.0"
> +dependencies = [
> + "autocfg",
> + "libgit-sys",
> +]
> +
> +[[package]]
> +name = "libgit-sys"
> +version = "0.1.0"
> +dependencies = [
> + "autocfg",
> + "libz-sys",
> + "make-cmd",
> +]
> +
> +[[package]]
> +name = "libz-sys"
> +version = "1.1.20"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
> +dependencies = [
> + "cc",
> + "libc",
> + "pkg-config",
> + "vcpkg",
> +]
> +
> +[[package]]
> +name = "make-cmd"
> +version = "0.1.0"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
> +
> +[[package]]
> +name = "pkg-config"
> +version = "0.3.30"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
> +
> +[[package]]
> +name = "shlex"
> +version = "1.3.0"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
> +
> +[[package]]
> +name = "vcpkg"
> +version = "0.2.15"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"

There are two possibilities here.  The first is to check in the
Cargo.lock, in which case all users will have to use these versions.
That produces a more stable and reliable approach, but it has some
downsides.

Say, for instance, that a cool new platform or architecture is added to
libc and we'd like to support it, but that version of libc requires a
newer version of Rust.  We then would have to hold off supporting that
new platform due to compatibility reasons.  But if we omitted the
Cargo.lock, users could install any version that meets their needs.

I believe Rust just got the ability to install only versions that honour
the rust-version directive in 1.84, whereas older versions will try to
use the latest version, even if that fails.  So I think it's okay for
now to use Cargo.lock, because that means that things will be better out
of the box for users on older Rust.  But we may want to drop it once
1.84 is older than our supported version.

> diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
> new file mode 100644
> index 0000000000..27b6fd63f1
> --- /dev/null
> +++ b/contrib/libgit-rs/src/lib.rs
> @@ -0,0 +1,95 @@
> +use std::ffi::{c_void, CStr, CString};
> +use std::path::Path;
> +
> +#[cfg(has_std__ffi__c_char)]
> +use std::ffi::{c_char, c_int};
> +
> +#[cfg(not(has_std__ffi__c_char))]
> +#[allow(non_camel_case_types)]
> +pub type c_char = i8;
> +
> +#[cfg(not(has_std__ffi__c_char))]
> +#[allow(non_camel_case_types)]
> +pub type c_int = i32;

By making these `pub`, you're exporting them.  We probably do not want
to do that, since they are not part of our API.

If we need them more generally in the code, let's put them in a module
called `ffi` or such that's `pub(crate)`, and then use them from there.

> +use libgit_sys::*;
> +
> +pub struct ConfigSet(*mut libgit_config_set);
> +impl ConfigSet {

I would suggest we place these in a module, such as `config`.  We should
expect to have a lot more things in our crate in the future and putting
a little thought into this now will make it easier for users in the
future.

I'd also, of course, suggest documentation comments, since if we ever
upload this to crates.io, people are overwhelmingly going to read only
the docs and not the source, and right now we've said nothing about how
this works or should work.

> +#[cfg(test)]
> +mod tests {
> +    use super::*;
> +
> +    #[test]
> +    fn load_configs_via_configset() {
> +        let mut cs = ConfigSet::new();
> +        cs.add_files(&[
> +            Path::new("testdata/config1"),
> +            Path::new("testdata/config2"),
> +            Path::new("testdata/config3"),
> +        ]);
> +        // ConfigSet retrieves correct value
> +        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
> +        // ConfigSet respects last config value set
> +        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
> +        // ConfigSet returns None for missing key
> +        assert_eq!(cs.get_string("foo.bar"), None);
> +    }
> +}

I am, of course, delighted to see tests.  This is a nice improvement in
our code that we can take advantage of, and we can test both the C code
and Rust code at the same time.  And if, in the future, we decide that
we'd like to implement a Rust-based version of this API to replace the C
one, we've already written tests.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: [PATCH v6 4/5] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2025.01.21 00:00, brian m. carlson wrote:
> On 2025-01-15 at 20:05:43, Josh Steadmon wrote:
> > From: Calvin Wan <calvinwan@google.com>
> > diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
> > new file mode 100644
> > index 0000000000..a30c7c8d33
> > --- /dev/null
> > +++ b/contrib/libgit-rs/Cargo.lock
> > @@ -0,0 +1,77 @@
> > +# This file is automatically @generated by Cargo.
> > +# It is not intended for manual editing.
> > +version = 3
> > +
> > +[[package]]
> > +name = "autocfg"
> > +version = "1.4.0"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
> > +
> > +[[package]]
> > +name = "cc"
> > +version = "1.1.15"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
> > +dependencies = [
> > + "shlex",
> > +]
> > +
> > +[[package]]
> > +name = "libc"
> > +version = "0.2.158"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
> > +
> > +[[package]]
> > +name = "libgit"
> > +version = "0.1.0"
> > +dependencies = [
> > + "autocfg",
> > + "libgit-sys",
> > +]
> > +
> > +[[package]]
> > +name = "libgit-sys"
> > +version = "0.1.0"
> > +dependencies = [
> > + "autocfg",
> > + "libz-sys",
> > + "make-cmd",
> > +]
> > +
> > +[[package]]
> > +name = "libz-sys"
> > +version = "1.1.20"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
> > +dependencies = [
> > + "cc",
> > + "libc",
> > + "pkg-config",
> > + "vcpkg",
> > +]
> > +
> > +[[package]]
> > +name = "make-cmd"
> > +version = "0.1.0"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
> > +
> > +[[package]]
> > +name = "pkg-config"
> > +version = "0.3.30"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
> > +
> > +[[package]]
> > +name = "shlex"
> > +version = "1.3.0"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
> > +
> > +[[package]]
> > +name = "vcpkg"
> > +version = "0.2.15"
> > +source = "registry+https://github.com/rust-lang/crates.io-index"
> > +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
> 
> There are two possibilities here.  The first is to check in the
> Cargo.lock, in which case all users will have to use these versions.
> That produces a more stable and reliable approach, but it has some
> downsides.
> 
> Say, for instance, that a cool new platform or architecture is added to
> libc and we'd like to support it, but that version of libc requires a
> newer version of Rust.  We then would have to hold off supporting that
> new platform due to compatibility reasons.  But if we omitted the
> Cargo.lock, users could install any version that meets their needs.
> 
> I believe Rust just got the ability to install only versions that honour
> the rust-version directive in 1.84, whereas older versions will try to
> use the latest version, even if that fails.  So I think it's okay for
> now to use Cargo.lock, because that means that things will be better out
> of the box for users on older Rust.  But we may want to drop it once
> 1.84 is older than our supported version.

OK, I'll add a TODO comment in Cargo.toml.


> > diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
> > new file mode 100644
> > index 0000000000..27b6fd63f1
> > --- /dev/null
> > +++ b/contrib/libgit-rs/src/lib.rs
> > @@ -0,0 +1,95 @@
> > +use std::ffi::{c_void, CStr, CString};
> > +use std::path::Path;
> > +
> > +#[cfg(has_std__ffi__c_char)]
> > +use std::ffi::{c_char, c_int};
> > +
> > +#[cfg(not(has_std__ffi__c_char))]
> > +#[allow(non_camel_case_types)]
> > +pub type c_char = i8;
> > +
> > +#[cfg(not(has_std__ffi__c_char))]
> > +#[allow(non_camel_case_types)]
> > +pub type c_int = i32;
> 
> By making these `pub`, you're exporting them.  We probably do not want
> to do that, since they are not part of our API.

Fixed for V7.


> If we need them more generally in the code, let's put them in a module
> called `ffi` or such that's `pub(crate)`, and then use them from there.

Hmm, I guess we'd need to do that for libgit-sys as well? Or maybe not,
since they're part of the API and thus we should just keep the current
`pub type ...` setup?

> > +use libgit_sys::*;
> > +
> > +pub struct ConfigSet(*mut libgit_config_set);
> > +impl ConfigSet {
> 
> I would suggest we place these in a module, such as `config`.  We should
> expect to have a lot more things in our crate in the future and putting
> a little thought into this now will make it easier for users in the
> future.

ACK, will do for V7.


> I'd also, of course, suggest documentation comments, since if we ever
> upload this to crates.io, people are overwhelmingly going to read only
> the docs and not the source, and right now we've said nothing about how
> this works or should work.

ACK.


> > +#[cfg(test)]
> > +mod tests {
> > +    use super::*;
> > +
> > +    #[test]
> > +    fn load_configs_via_configset() {
> > +        let mut cs = ConfigSet::new();
> > +        cs.add_files(&[
> > +            Path::new("testdata/config1"),
> > +            Path::new("testdata/config2"),
> > +            Path::new("testdata/config3"),
> > +        ]);
> > +        // ConfigSet retrieves correct value
> > +        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
> > +        // ConfigSet respects last config value set
> > +        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
> > +        // ConfigSet returns None for missing key
> > +        assert_eq!(cs.get_string("foo.bar"), None);
> > +    }
> > +}
> 
> I am, of course, delighted to see tests.  This is a nice improvement in
> our code that we can take advantage of, and we can test both the C code
> and Rust code at the same time.  And if, in the future, we decide that
> we'd like to implement a Rust-based version of this API to replace the C
> one, we've already written tests.

Thanks for the review and advice (both for this round and all the
previous ones), it's much appreciated.

> -- 
> brian m. carlson (they/them or he/him)
> Toronto, Ontario, CA

[PATCH v6 5/5] Makefile: add option to build and test libgit-rs and libgit-rs-sys

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
to their respective Makefiles so they can be built and tested without
having to run cargo build/test.

Add build variable, INCLUDE_LIBGIT_RS, that when set, automatically
builds and tests libgit-rs and libgit-rs-sys when `make all` is run.

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile                    | 26 ++++++++++++++++++++++++++
 contrib/libgit-sys/build.rs |  2 +-
 t/Makefile                  | 16 ++++++++++++++++
 3 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 230d366457..133090a145 100644
--- a/Makefile
+++ b/Makefile
@@ -416,6 +416,9 @@ include shared.mak
 # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
 # programs in oss-fuzz/.
 #
+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
+# test the Rust crates in contrib/libgit-rs/ and contrib/libgit-rs/libgit-sys/.
+#
 # === Optional library: libintl ===
 #
 # Define NO_GETTEXT if you don't want Git output to be translated.
@@ -2238,6 +2241,13 @@ ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
+ifdef INCLUDE_LIBGIT_RS
+	# Enable symbol hiding in contrib/libgit-rs/libgit-sys/libgitpub.a
+	# without making us rebuild the whole tree every time we run a Rust
+	# build.
+	BASIC_CFLAGS += -fvisibility=hidden
+endif
+
 ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
@@ -3895,6 +3905,22 @@ build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
+.PHONY: libgit-sys
+libgit-sys:
+	$(QUIET)(\
+		cd contrib/libgit-sys && \
+		cargo build \
+	)
+.PHONY: libgit-rs
+libgit-rs:
+	$(QUIET)(\
+		cd contrib/libgit-rs && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-rs
+endif
+
 contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
 	$(LD) -r $^ -o $@
 
diff --git a/contrib/libgit-sys/build.rs b/contrib/libgit-sys/build.rs
index b6c65193bc..3ffd80ad91 100644
--- a/contrib/libgit-sys/build.rs
+++ b/contrib/libgit-sys/build.rs
@@ -14,7 +14,7 @@ pub fn main() -> std::io::Result<()> {
         .env_remove("PROFILE")
         .current_dir(git_root.clone())
         .args([
-            "CFLAGS=-fvisibility=hidden",
+            "INCLUDE_LIBGIT_RS=YesPlease",
             "contrib/libgit-sys/libgitpub.a",
         ])
         .output()
diff --git a/t/Makefile b/t/Makefile
index daa5fcae86..45fe6089b2 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -177,3 +177,19 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgit-sys-test
+libgit-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-sys && \
+		cargo test \
+	)
+.PHONY: libgit-rs-test
+libgit-rs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys-test libgit-rs-test
+endif
-- 
2.48.0.rc2.279.g1de40edade-goog

Re: [PATCH v6 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> Apologies for the long delay on V6; I am finally back after several
> months of $DAYJOB firefighting, holidays, and sick leave. I should have
> time to devote to this series again, but given the lack of feedback on
> V5 I am hopeful that this will be the final iteration of this series.

Thanks and welcome back ;-)

Given the lack of feedback on the previous round, I hope we will see
enthused support on the topic.  Otherwise it is hard to tell if the
previous lack of feedback was merely lack of interest, or lack of
anything lacking in the series.

> There is known NEEDSWORK, but I feel that they can be addressed in
> follow-up changes, rather than in this series. If you feel otherwise,
> please let me know:
>
> * Investigate alternative methods of managing symbol visibility &
>   renaming.
>
> * Figure out symbol versioning

OK.  Let's see what people find out.

Re: [PATCH v6 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]

On 2025-01-15 at 20:05:39, Josh Steadmon wrote:
> Apologies for the long delay on V6; I am finally back after several
> months of $DAYJOB firefighting, holidays, and sick leave. I should have
> time to devote to this series again, but given the lack of feedback on
> V5 I am hopeful that this will be the final iteration of this series.
> 
> This series provides two small Rust wrapper libraries around parts of
> Git: "libgit-sys", which exposes a few functions from libgit.a, and
> "libgit", which provides a more Rust-friendly interface to some of those
> functions. In addition to included unit tests, at $DAYJOB we have tested
> building JJ[1] with our library and used it to replace some of the
> libgit2-rs uses.
> 
> [1] https://github.com/martinvonz/jj
> 
> There is known NEEDSWORK, but I feel that they can be addressed in
> follow-up changes, rather than in this series. If you feel otherwise,
> please let me know:
> 
> * Investigate alternative methods of managing symbol visibility &
>   renaming.
> 
> * Figure out symbol versioning

It looks like we're building a general Rust lib crate and a static
library here, so symbol versioning isn't an issue.  I expect in the
future we may want to provide a shared library, in which case we will
indeed want to do that, but I agree that can wait until later.

In any event, I overall think this series is a nice improvement, and I
am very enthusiastic about it.  (This is mostly for the benefit of
Junio, since I think the authors of this series already know that.)
Once it lands, I do plan to build on it somewhat.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: [PATCH v6 0/5] Introduce libgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> It looks like we're building a general Rust lib crate and a static
> library here, so symbol versioning isn't an issue.  I expect in the
> future we may want to provide a shared library, in which case we will
> indeed want to do that, but I agree that can wait until later.
>
> In any event, I overall think this series is a nice improvement, and I
> am very enthusiastic about it.  (This is mostly for the benefit of
> Junio, since I think the authors of this series already know that.)
> Once it lands, I do plan to build on it somewhat.

;-)  Thanks, I heard you.

[PATCH v7 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/jj-vcs/jj

There is known NEEDSWORK, but I feel that they can be addressed in
follow-up changes, rather than in this series. If you feel otherwise,
please let me know:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning

Changes in V7:
* Moved the ConfigSet implementation in libgit-rs to a `config` module.

* Added doc comments for ConfigSet and its methods.

* Fix meson builds by adding new object files to `libgit_sources`

* Moved INCLUDE_LIBGIT_RS Makefile changes earlier in the series, so
  that we can make it optional to compile some of the libgitpub sources.
  Squashed V6 patch 5/5 into this series' patch 4/4.

* Don't publicly export FFI types in libgit-rs.

* Removed extraneous `-r` argument to $(RM) in the clean rules.

* Added TODO reminder in Cargo.toml about removing Cargo.lock once we
  hit a certain minimum supported Rust version.

* Style cleanup in public_symbol_export.c

Changes in V6:
* Rebased onto current master, since V5 was several months old.

* Move libgit-sys out of libgit-rs; while this sort of nesting is common
  in Rust crates with standalone repositories, it doesn't make as much
  sense when they're contained in the larger Git project's repo.

* Standardize the naming of some of the Makefile targets to always
  include a dash in the "-rs" or "-sys" suffixes.

* Clean up READMEs and crate descriptions in preparation for
  uploading to crates.io.

Changes in V5:
* When building with INCLUDE_LIBGIT_RS defined, add
  "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
  visibility in libgitpub.a without causing `make all` to rebuild from
  scratch due to changing CFLAGS.

* Avoid using c_int in the higher-level Rust API.

* Remove libgitpub.a and intermediate files with `make clean`.

Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml


Calvin Wan (1):
  libgit: add higher-level libgit crate

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                |   2 +
 Makefile                                  |  45 +++++++++
 common-exit.c                             |  26 ++++++
 common-init.c                             |  63 +++++++++++++
 common-init.h                             |   6 ++
 common-main.c                             |  83 +----------------
 contrib/libgit-rs/Cargo.lock              |  77 ++++++++++++++++
 contrib/libgit-rs/Cargo.toml              |  17 ++++
 contrib/libgit-rs/README.md               |  13 +++
 contrib/libgit-rs/build.rs                |   4 +
 contrib/libgit-rs/src/config.rs           | 106 ++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs              |   1 +
 contrib/libgit-rs/testdata/config1        |   2 +
 contrib/libgit-rs/testdata/config2        |   2 +
 contrib/libgit-rs/testdata/config3        |   2 +
 contrib/libgit-sys/Cargo.lock             |  69 ++++++++++++++
 contrib/libgit-sys/Cargo.toml             |  19 ++++
 contrib/libgit-sys/README.md              |   4 +
 contrib/libgit-sys/build.rs               |  35 +++++++
 contrib/libgit-sys/public_symbol_export.c |  51 +++++++++++
 contrib/libgit-sys/public_symbol_export.h |  18 ++++
 contrib/libgit-sys/src/lib.rs             |  79 ++++++++++++++++
 meson.build                               |   2 +
 t/Makefile                                |  15 +++
 24 files changed, 660 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

Range-diff against v6:
1:  ff6cd62397 ! 1:  cd0cb9aa04 common-main: split init and exit code into new files
    @@ common-main.c
     -
     -	return code;
     -}
    +
    + ## meson.build ##
    +@@ meson.build: libgit_sources = [
    +   'commit-graph.c',
    +   'commit-reach.c',
    +   'commit.c',
    ++  'common-exit.c',
    ++  'common-init.c',
    +   'compat/nonblock.c',
    +   'compat/obstack.c',
    +   'compat/terminal.c',
2:  5fc66cdb16 ! 2:  f1502b8590 libgit-sys: introduce Rust wrapper for libgit.a
    @@ .gitignore: Release/
     +/contrib/libgit-sys/target
     
      ## Makefile ##
    +@@ Makefile: include shared.mak
    + # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
    + # programs in oss-fuzz/.
    + #
    ++# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
    ++# test the Rust crate in contrib/libgit-sys.
    ++#
    + # === Optional library: libintl ===
    + #
    + # Define NO_GETTEXT if you don't want Git output to be translated.
     @@ Makefile: CURL_CONFIG = curl-config
      GCOV = gcov
      STRIP = strip
    @@ Makefile: CURL_CONFIG = curl-config
      
      export TCL_PATH TCLTK_PATH
      
    -@@ Makefile: OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)
    - OBJECTS += $(UNIT_TEST_OBJS)
    +@@ Makefile: ifdef FSMONITOR_OS_SETTINGS
    + 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
    + endif
    + 
    ++ifdef INCLUDE_LIBGIT_RS
    ++	# Enable symbol hiding in contrib/libgit-sys/libgitpub.a without making
    ++	# us rebuild the whole tree every time we run a Rust build.
    ++	BASIC_CFLAGS += -fvisibility=hidden
    ++endif
    ++
    + ifeq ($(TCLTK_PATH),)
    + NO_TCLTK = NoThanks
    + endif
    +@@ Makefile: OBJECTS += $(UNIT_TEST_OBJS)
      OBJECTS += $(CLAR_TEST_OBJS)
      OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
    -+OBJECTS += contrib/libgit-sys/public_symbol_export.o
      
    ++ifdef INCLUDE_LIBGIT_RS
    ++	OBJECTS += contrib/libgit-sys/public_symbol_export.o
    ++endif
    ++
      ifndef NO_CURL
      	OBJECTS += http.o http-walker.o remote-curl.o
    + endif
     @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
     +	$(RM) -r contrib/libgit-sys/target
    -+	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
    -+	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
    -+	$(RM) -r contrib/libgit-sys/libgitpub.a
    ++	$(RM) contrib/libgit-sys/partial_symbol_export.o
    ++	$(RM) contrib/libgit-sys/hidden_symbol_export.o
    ++	$(RM) contrib/libgit-sys/libgitpub.a
      ifndef NO_PERL
      	$(RM) -r perl/build/
      endif
    @@ Makefile: $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GIT
      unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
      	$(MAKE) -C t/ unit-tests
     +
    ++.PHONY: libgit-sys
    ++libgit-sys:
    ++	$(QUIET)(\
    ++		cd contrib/libgit-sys && \
    ++		cargo build \
    ++	)
    ++ifdef INCLUDE_LIBGIT_RS
    ++all:: libgit-sys
    ++endif
    ++
     +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
     +	$(LD) -r $^ -o $@
     +
    @@ contrib/libgit-sys/Cargo.toml (new)
     +edition = "2021"
     +build = "build.rs"
     +links = "gitpub"
    -+rust-version = "1.63"
    ++rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
    ++                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
     +description = "Native bindings to a portion of libgit"
     +
     +[lib]
    @@ contrib/libgit-sys/build.rs (new)
     +        .env_remove("PROFILE")
     +        .current_dir(git_root.clone())
     +        .args([
    -+            "CFLAGS=-fvisibility=hidden",
    ++            "INCLUDE_LIBGIT_RS=YesPlease",
     +            "contrib/libgit-sys/libgitpub.a",
     +        ])
     +        .output()
    @@ contrib/libgit-sys/build.rs (new)
     
      ## contrib/libgit-sys/public_symbol_export.c (new) ##
     @@
    -+// Shim to publicly export Git symbols. These must be renamed so that the
    -+// original symbols can be hidden. Renaming these with a "libgit_" prefix also
    -+// avoids conflicts with other libraries such as libgit2.
    ++/* Shim to publicly export Git symbols. These must be renamed so that the
    ++ * original symbols can be hidden. Renaming these with a "libgit_" prefix also
    ++ * avoids conflicts with other libraries such as libgit2.
    ++ */
     +
     +#include "git-compat-util.h"
     +#include "contrib/libgit-sys/public_symbol_export.h"
    @@ contrib/libgit-sys/src/lib.rs (new)
     +        );
     +    }
     +}
    +
    + ## t/Makefile ##
    +@@ t/Makefile: perf:
    + 
    + .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
    + 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
    ++
    ++.PHONY: libgit-sys-test
    ++libgit-sys-test:
    ++	$(QUIET)(\
    ++		cd ../contrib/libgit-sys && \
    ++		cargo test \
    ++	)
    ++ifdef INCLUDE_LIBGIT_RS
    ++all:: libgit-sys-test
    ++endif
3:  03f39b6c3a < -:  ---------- libgit-sys: also export some config_set functions
5:  84706f0db7 ! 3:  d67d3648d1 Makefile: add option to build and test libgit-rs and libgit-rs-sys
    @@
      ## Metadata ##
    -Author: Calvin Wan <calvinwan@google.com>
    +Author: Josh Steadmon <steadmon@google.com>
     
      ## Commit message ##
    -    Makefile: add option to build and test libgit-rs and libgit-rs-sys
    +    libgit-sys: also export some config_set functions
     
    -    Add libgitrs, libgitrs-sys, libgitrs-test, and libgitrs-sys-test targets
    -    to their respective Makefiles so they can be built and tested without
    -    having to run cargo build/test.
    +    In preparation for implementing a higher-level Rust API for accessing
    +    Git configs, export some of the upstream configset API via libgitpub and
    +    libgit-sys. Since this will be exercised as part of the higher-level API
    +    in the next commit, no tests have been added for libgit-sys.
     
    -    Add build variable, INCLUDE_LIBGIT_RS, that when set, automatically
    -    builds and tests libgit-rs and libgit-rs-sys when `make all` is run.
    +    While we're at it, add git_configset_alloc() and git_configset_free()
    +    functions in libgitpub so that callers can manage config_set structs on
    +    the heap. This also allows non-C external consumers to treat config_sets
    +    as opaque structs.
     
    -    Co-authored-by: Josh Steadmon <steadmon@google.com>
    +    Co-authored-by: Calvin Wan <calvinwan@google.com>
         Signed-off-by: Calvin Wan <calvinwan@google.com>
    +    Signed-off-by: Josh Steadmon <steadmon@google.com>
     
    - ## Makefile ##
    -@@ Makefile: include shared.mak
    - # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
    - # programs in oss-fuzz/.
    - #
    -+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
    -+# test the Rust crates in contrib/libgit-rs/ and contrib/libgit-rs/libgit-sys/.
    -+#
    - # === Optional library: libintl ===
    - #
    - # Define NO_GETTEXT if you don't want Git output to be translated.
    -@@ Makefile: ifdef FSMONITOR_OS_SETTINGS
    - 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
    - endif
    + ## contrib/libgit-sys/public_symbol_export.c ##
    +@@
    +  */
      
    -+ifdef INCLUDE_LIBGIT_RS
    -+	# Enable symbol hiding in contrib/libgit-rs/libgit-sys/libgitpub.a
    -+	# without making us rebuild the whole tree every time we run a Rust
    -+	# build.
    -+	BASIC_CFLAGS += -fvisibility=hidden
    -+endif
    -+
    - ifeq ($(TCLTK_PATH),)
    - NO_TCLTK = NoThanks
    - endif
    -@@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
    - unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
    - 	$(MAKE) -C t/ unit-tests
    + #include "git-compat-util.h"
    ++#include "config.h"
    + #include "contrib/libgit-sys/public_symbol_export.h"
    + #include "version.h"
      
    -+.PHONY: libgit-sys
    -+libgit-sys:
    -+	$(QUIET)(\
    -+		cd contrib/libgit-sys && \
    -+		cargo build \
    -+	)
    -+.PHONY: libgit-rs
    -+libgit-rs:
    -+	$(QUIET)(\
    -+		cd contrib/libgit-rs && \
    -+		cargo build \
    -+	)
    -+ifdef INCLUDE_LIBGIT_RS
    -+all:: libgit-rs
    -+endif
    -+
    - contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    - 	$(LD) -r $^ -o $@
    + #pragma GCC visibility push(default)
      
    ++struct libgit_config_set *libgit_configset_alloc(void)
    ++{
    ++	struct config_set *cs = xmalloc(sizeof(struct config_set));
    ++	git_configset_init(cs);
    ++	return (struct libgit_config_set *) cs;
    ++}
    ++
    ++void libgit_configset_free(struct libgit_config_set *cs)
    ++{
    ++	git_configset_clear((struct config_set *) cs);
    ++	free((struct config_set *) cs);
    ++}
    ++
    ++int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
    ++{
    ++	return git_configset_add_file((struct config_set *) cs, filename);
    ++}
    ++
    ++int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
    ++{
    ++	return git_configset_get_int((struct config_set *) cs, key, dest);
    ++}
    ++
    ++int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
    ++{
    ++	return git_configset_get_string((struct config_set *) cs, key, dest);
    ++}
    ++
    + const char *libgit_user_agent(void)
    + {
    + 	return git_user_agent();
     
    - ## contrib/libgit-sys/build.rs ##
    -@@ contrib/libgit-sys/build.rs: pub fn main() -> std::io::Result<()> {
    -         .env_remove("PROFILE")
    -         .current_dir(git_root.clone())
    -         .args([
    --            "CFLAGS=-fvisibility=hidden",
    -+            "INCLUDE_LIBGIT_RS=YesPlease",
    -             "contrib/libgit-sys/libgitpub.a",
    -         ])
    -         .output()
    + ## contrib/libgit-sys/public_symbol_export.h ##
    +@@
    + #ifndef PUBLIC_SYMBOL_EXPORT_H
    + #define PUBLIC_SYMBOL_EXPORT_H
    + 
    ++struct libgit_config_set *libgit_configset_alloc(void);
    ++
    ++void libgit_configset_free(struct libgit_config_set *cs);
    ++
    ++int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
    ++
    ++int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
    ++
    ++int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
    ++
    + const char *libgit_user_agent(void);
    + 
    + const char *libgit_user_agent_sanitized(void);
     
    - ## t/Makefile ##
    -@@ t/Makefile: perf:
    + ## contrib/libgit-sys/src/lib.rs ##
    +@@
    + #[cfg(has_std__ffi__c_char)]
    +-use std::ffi::c_char;
    ++use std::ffi::{c_char, c_int};
    + 
    + #[cfg(not(has_std__ffi__c_char))]
    + #[allow(non_camel_case_types)]
    + pub type c_char = i8;
    + 
    ++#[cfg(not(has_std__ffi__c_char))]
    ++#[allow(non_camel_case_types)]
    ++pub type c_int = i32;
    ++
    + extern crate libz_sys;
    + 
    ++#[allow(non_camel_case_types)]
    ++#[repr(C)]
    ++pub struct libgit_config_set {
    ++    _data: [u8; 0],
    ++    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
    ++}
    ++
    + extern "C" {
    +     pub fn libgit_user_agent() -> *const c_char;
    +     pub fn libgit_user_agent_sanitized() -> *const c_char;
    ++
    ++    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
    ++    pub fn libgit_configset_free(cs: *mut libgit_config_set);
    ++
    ++    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
    ++
    ++    pub fn libgit_configset_get_int(
    ++        cs: *mut libgit_config_set,
    ++        key: *const c_char,
    ++        int: *mut c_int,
    ++    ) -> c_int;
    ++
    ++    pub fn libgit_configset_get_string(
    ++        cs: *mut libgit_config_set,
    ++        key: *const c_char,
    ++        dest: *mut *mut c_char,
    ++    ) -> c_int;
    ++
    + }
      
    - .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
    - 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
    -+
    -+.PHONY: libgit-sys-test
    -+libgit-sys-test:
    -+	$(QUIET)(\
    -+		cd ../contrib/libgit-sys && \
    -+		cargo test \
    -+	)
    -+.PHONY: libgit-rs-test
    -+libgit-rs-test:
    -+	$(QUIET)(\
    -+		cd ../contrib/libgit-rs && \
    -+		cargo test \
    -+	)
    -+ifdef INCLUDE_LIBGIT_RS
    -+all:: libgit-sys-test libgit-rs-test
    -+endif
    + #[cfg(test)]
4:  65166ea0c0 ! 4:  88425bb0b1 libgit: add higher-level libgit crate
    @@ .gitignore: Release/
      /contrib/libgit-sys/target
     
      ## Makefile ##
    +@@ Makefile: include shared.mak
    + # programs in oss-fuzz/.
    + #
    + # Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
    +-# test the Rust crate in contrib/libgit-sys.
    ++# test the Rust crates in contrib/libgit-sys and contrib/libgit-rs.
    + #
    + # === Optional library: libintl ===
    + #
     @@ Makefile: clean: profile-clean coverage-clean cocciclean
      	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
      	$(MAKE) -C Documentation/ clean
      	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
     -	$(RM) -r contrib/libgit-sys/target
    -+	$(RM) -r contrib/libgit-rs/target contrib/libgit-sys/target
    - 	$(RM) -r contrib/libgit-sys/partial_symbol_export.o
    - 	$(RM) -r contrib/libgit-sys/hidden_symbol_export.o
    - 	$(RM) -r contrib/libgit-sys/libgitpub.a
    ++	$(RM) -r contrib/libgit-sys/target contrib/libgit-rs/target
    + 	$(RM) contrib/libgit-sys/partial_symbol_export.o
    + 	$(RM) contrib/libgit-sys/hidden_symbol_export.o
    + 	$(RM) contrib/libgit-sys/libgitpub.a
    +@@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
    + unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
    + 	$(MAKE) -C t/ unit-tests
    + 
    +-.PHONY: libgit-sys
    ++.PHONY: libgit-sys libgit-rs
    + libgit-sys:
    + 	$(QUIET)(\
    + 		cd contrib/libgit-sys && \
    + 		cargo build \
    + 	)
    ++libgit-rs:
    ++	$(QUIET)(\
    ++		cd contrib/libgit-rs && \
    ++		cargo build \
    ++	)
    + ifdef INCLUDE_LIBGIT_RS
    +-all:: libgit-sys
    ++all:: libgit-sys libgit-rs
    + endif
    + 
    + contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
     
      ## contrib/libgit-rs/Cargo.lock (new) ##
     @@
    @@ contrib/libgit-rs/Cargo.toml (new)
     +version = "0.1.0"
     +edition = "2021"
     +build = "build.rs"
    -+rust-version = "1.63"
    ++rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
    ++                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
    ++
     +
     +[lib]
     +path = "src/lib.rs"
    @@ contrib/libgit-rs/build.rs (new)
     +    ac.emit_has_path("std::ffi::c_char");
     +}
     
    - ## contrib/libgit-rs/src/lib.rs (new) ##
    + ## contrib/libgit-rs/src/config.rs (new) ##
     @@
     +use std::ffi::{c_void, CStr, CString};
     +use std::path::Path;
    @@ contrib/libgit-rs/src/lib.rs (new)
     +
     +#[cfg(not(has_std__ffi__c_char))]
     +#[allow(non_camel_case_types)]
    -+pub type c_char = i8;
    ++type c_char = i8;
     +
     +#[cfg(not(has_std__ffi__c_char))]
     +#[allow(non_camel_case_types)]
    -+pub type c_int = i32;
    ++type c_int = i32;
     +
     +use libgit_sys::*;
     +
    ++/// A ConfigSet is an in-memory cache for config-like files such as `.gitmodules` or `.gitconfig`.
    ++/// It does not support all config directives; notably, it will not process `include` or
    ++/// `includeIf` directives (but it will store them so that callers can choose whether and how to
    ++/// handle them).
     +pub struct ConfigSet(*mut libgit_config_set);
     +impl ConfigSet {
    ++    /// Allocate a new ConfigSet
     +    pub fn new() -> Self {
     +        unsafe { ConfigSet(libgit_configset_alloc()) }
     +    }
     +
    ++    /// Load the given files into the ConfigSet; conflicting directives in later files will
    ++    /// override those given in earlier files.
     +    pub fn add_files(&mut self, files: &[&Path]) {
     +        for file in files {
     +            let pstr = file.to_str().expect("Invalid UTF-8");
    @@ contrib/libgit-rs/src/lib.rs (new)
     +        }
     +    }
     +
    ++    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
    ++    /// if the value cannot be parsed. Returns None if the key is not present.
     +    pub fn get_int(&mut self, key: &str) -> Option<i32> {
     +        let key = CString::new(key).expect("Couldn't convert to CString");
     +        let mut val: c_int = 0;
    @@ contrib/libgit-rs/src/lib.rs (new)
     +        Some(val.into())
     +    }
     +
    ++    /// Clones the value for the given key. Dies with a fatal error if the value cannot be
    ++    /// converted to a String. Returns None if the key is not present.
     +    pub fn get_string(&mut self, key: &str) -> Option<String> {
     +        let key = CString::new(key).expect("Couldn't convert key to CString");
     +        let mut val: *mut c_char = std::ptr::null_mut();
    @@ contrib/libgit-rs/src/lib.rs (new)
     +    }
     +}
     
    + ## contrib/libgit-rs/src/lib.rs (new) ##
    +@@
    ++pub mod config;
    +
      ## contrib/libgit-rs/testdata/config1 (new) ##
     @@
     +[trace2]
    @@ contrib/libgit-sys/src/lib.rs: pub struct libgit_config_set {
          pub fn libgit_user_agent() -> *const c_char;
          pub fn libgit_user_agent_sanitized() -> *const c_char;
      
    +
    + ## t/Makefile ##
    +@@ t/Makefile: perf:
    + .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
    + 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
    + 
    +-.PHONY: libgit-sys-test
    ++.PHONY: libgit-sys-test libgit-rs-test
    + libgit-sys-test:
    + 	$(QUIET)(\
    + 		cd ../contrib/libgit-sys && \
    + 		cargo test \
    + 	)
    ++libgit-rs-test:
    ++	$(QUIET)(\
    ++		cd ../contrib/libgit-rs && \
    ++		cargo test \
    ++	)
    + ifdef INCLUDE_LIBGIT_RS
    +-all:: libgit-sys-test
    ++all:: libgit-sys-test libgit-rs-test
    + endif

base-commit: 757161efcca150a9a96b312d9e780a071e601a03
-- 
2.48.1.262.g85cc9f2d1e-goog

[PATCH v7 1/4] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 meson.build   |  2 ++
 6 files changed, 101 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 97e8385b66..27e68ac039 100644
--- a/Makefile
+++ b/Makefile
@@ -981,6 +981,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
diff --git a/meson.build b/meson.build
index 0064eb64f5..e5ba28b47f 100644
--- a/meson.build
+++ b/meson.build
@@ -245,6 +245,8 @@ libgit_sources = [
   'commit-graph.c',
   'commit-reach.c',
   'commit.c',
+  'common-exit.c',
+  'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
   'compat/terminal.c',
-- 
2.48.1.262.g85cc9f2d1e-goog

[PATCH v7 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                |  1 +
 Makefile                                  | 38 +++++++++++++
 contrib/libgit-sys/Cargo.lock             | 69 +++++++++++++++++++++++
 contrib/libgit-sys/Cargo.toml             | 19 +++++++
 contrib/libgit-sys/README.md              |  4 ++
 contrib/libgit-sys/build.rs               | 35 ++++++++++++
 contrib/libgit-sys/public_symbol_export.c | 22 ++++++++
 contrib/libgit-sys/public_symbol_export.h |  8 +++
 contrib/libgit-sys/src/lib.rs             | 46 +++++++++++++++
 t/Makefile                                | 10 ++++
 10 files changed, 252 insertions(+)
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index e82aa19df0..31d7e64287 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,3 +250,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 27e68ac039..92989d3311 100644
--- a/Makefile
+++ b/Makefile
@@ -416,6 +416,9 @@ include shared.mak
 # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
 # programs in oss-fuzz/.
 #
+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
+# test the Rust crate in contrib/libgit-sys.
+#
 # === Optional library: libintl ===
 #
 # Define NO_GETTEXT if you don't want Git output to be translated.
@@ -657,6 +660,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2236,6 +2241,12 @@ ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
+ifdef INCLUDE_LIBGIT_RS
+	# Enable symbol hiding in contrib/libgit-sys/libgitpub.a without making
+	# us rebuild the whole tree every time we run a Rust build.
+	BASIC_CFLAGS += -fvisibility=hidden
+endif
+
 ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
@@ -2732,6 +2743,10 @@ OBJECTS += $(UNIT_TEST_OBJS)
 OBJECTS += $(CLAR_TEST_OBJS)
 OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
 
+ifdef INCLUDE_LIBGIT_RS
+	OBJECTS += contrib/libgit-sys/public_symbol_export.o
+endif
+
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
 endif
@@ -3726,6 +3741,10 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-sys/target
+	$(RM) contrib/libgit-sys/partial_symbol_export.o
+	$(RM) contrib/libgit-sys/hidden_symbol_export.o
+	$(RM) contrib/libgit-sys/libgitpub.a
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3887,3 +3906,22 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
 build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+.PHONY: libgit-sys
+libgit-sys:
+	$(QUIET)(\
+		cd contrib/libgit-sys && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys
+endif
+
+contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
+	$(LD) -r $^ -o $@
+
+contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-sys/Cargo.lock b/contrib/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-sys/Cargo.toml b/contrib/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..e0623022c3
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+description = "Native bindings to a portion of libgit"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
new file mode 100644
index 0000000000..c061cfcaf5
--- /dev/null
+++ b/contrib/libgit-sys/README.md
@@ -0,0 +1,4 @@
+# libgit-sys
+
+A small proof-of-concept crate showing how to provide a Rust FFI to Git
+internals.
diff --git a/contrib/libgit-sys/build.rs b/contrib/libgit-sys/build.rs
new file mode 100644
index 0000000000..3ffd80ad91
--- /dev/null
+++ b/contrib/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "INCLUDE_LIBGIT_RS=YesPlease",
+            "contrib/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..cd1602206e
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -0,0 +1,22 @@
+/* Shim to publicly export Git symbols. These must be renamed so that the
+ * original symbols can be hidden. Renaming these with a "libgit_" prefix also
+ * avoids conflicts with other libraries such as libgit2.
+ */
+
+#include "git-compat-util.h"
+#include "contrib/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
diff --git a/t/Makefile b/t/Makefile
index daa5fcae86..53ba01c21b 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -177,3 +177,13 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgit-sys-test
+libgit-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-sys && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys-test
+endif
-- 
2.48.1.262.g85cc9f2d1e-goog

Re: [PATCH v7 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Phillip Wood]

On 28/01/2025 00:19, Josh Steadmon wrote:
> Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
> functions in libgit.a. This initial patch defines build rules and an
> interface that exposes user agent string getter functions as a proof of
> concept. This library can be tested with `cargo test`. 

It's great to see some tests. This is looking good, I've left a couple 
of small comments below.

> +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
> +	$(LD) -r $^ -o $@

This is a very long line and the ones below are pretty long - perhaps we 
could put the list of sources in a variable?

> +contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
> +	$(OBJCOPY) --localize-hidden $^ $@
> +
> +contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
> +	$(AR) $(ARFLAGS) $@ $^
 > [...]
> diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> new file mode 100644
> index 0000000000..cd1602206e
> --- /dev/null
> +++ b/contrib/libgit-sys/public_symbol_export.c
> @@ -0,0 +1,22 @@
> +/* Shim to publicly export Git symbols. These must be renamed so that the

Style: Multiline comments start with an empty "/*" so this should be

/*
  * Shim ...

> + * original symbols can be hidden. Renaming these with a "libgit_" prefix also
> + * avoids conflicts with other libraries such as libgit2.
> + */

Best Wishes

Phillip

> +#include "git-compat-util.h"
> +#include "contrib/libgit-sys/public_symbol_export.h"
> +#include "version.h"
> +
> +#pragma GCC visibility push(default)
> +
> +const char *libgit_user_agent(void)
> +{
> +	return git_user_agent();
> +}
> +
> +const char *libgit_user_agent_sanitized(void)
> +{
> +	return git_user_agent_sanitized();
> +}
> +
> +#pragma GCC visibility pop
> diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
> new file mode 100644
> index 0000000000..a3372f93fa
> --- /dev/null
> +++ b/contrib/libgit-sys/public_symbol_export.h
> @@ -0,0 +1,8 @@
> +#ifndef PUBLIC_SYMBOL_EXPORT_H
> +#define PUBLIC_SYMBOL_EXPORT_H
> +
> +const char *libgit_user_agent(void);
> +
> +const char *libgit_user_agent_sanitized(void);
> +
> +#endif /* PUBLIC_SYMBOL_EXPORT_H */
> diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
> new file mode 100644
> index 0000000000..d4853f3074
> --- /dev/null
> +++ b/contrib/libgit-sys/src/lib.rs
> @@ -0,0 +1,46 @@
> +#[cfg(has_std__ffi__c_char)]
> +use std::ffi::c_char;
> +
> +#[cfg(not(has_std__ffi__c_char))]
> +#[allow(non_camel_case_types)]
> +pub type c_char = i8;
> +
> +extern crate libz_sys;
> +
> +extern "C" {
> +    pub fn libgit_user_agent() -> *const c_char;
> +    pub fn libgit_user_agent_sanitized() -> *const c_char;
> +}
> +
> +#[cfg(test)]
> +mod tests {
> +    use std::ffi::CStr;
> +
> +    use super::*;
> +
> +    #[test]
> +    fn user_agent_starts_with_git() {
> +        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
> +        let agent = c_str
> +            .to_str()
> +            .expect("User agent contains invalid UTF-8 data");
> +        assert!(
> +            agent.starts_with("git/"),
> +            r#"Expected user agent to start with "git/", got: {}"#,
> +            agent
> +        );
> +    }
> +
> +    #[test]
> +    fn sanitized_user_agent_starts_with_git() {
> +        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
> +        let agent = c_str
> +            .to_str()
> +            .expect("Sanitized user agent contains invalid UTF-8 data");
> +        assert!(
> +            agent.starts_with("git/"),
> +            r#"Expected user agent to start with "git/", got: {}"#,
> +            agent
> +        );
> +    }
> +}
> diff --git a/t/Makefile b/t/Makefile
> index daa5fcae86..53ba01c21b 100644
> --- a/t/Makefile
> +++ b/t/Makefile
> @@ -177,3 +177,13 @@ perf:
>   
>   .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
>   	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
> +
> +.PHONY: libgit-sys-test
> +libgit-sys-test:
> +	$(QUIET)(\
> +		cd ../contrib/libgit-sys && \
> +		cargo test \
> +	)
> +ifdef INCLUDE_LIBGIT_RS
> +all:: libgit-sys-test
> +endif

Re: [PATCH v7 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2025.01.28 15:08, Phillip Wood wrote:
> 
> 
> On 28/01/2025 00:19, Josh Steadmon wrote:
> > Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
> > functions in libgit.a. This initial patch defines build rules and an
> > interface that exposes user agent string getter functions as a proof of
> > concept. This library can be tested with `cargo test`.
> 
> It's great to see some tests. This is looking good, I've left a couple of
> small comments below.
> 
> > +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
> > +	$(LD) -r $^ -o $@
> 
> This is a very long line and the ones below are pretty long - perhaps we
> could put the list of sources in a variable?

Done for V8.


> > +contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
> > +	$(OBJCOPY) --localize-hidden $^ $@
> > +
> > +contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
> > +	$(AR) $(ARFLAGS) $@ $^
> > [...]
> > diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> > new file mode 100644
> > index 0000000000..cd1602206e
> > --- /dev/null
> > +++ b/contrib/libgit-sys/public_symbol_export.c
> > @@ -0,0 +1,22 @@
> > +/* Shim to publicly export Git symbols. These must be renamed so that the
> 
> Style: Multiline comments start with an empty "/*" so this should be
> 
> /*
>  * Shim ...

Whoops, fixed, thanks.

Re: [PATCH v7 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> On 2025.01.28 15:08, Phillip Wood wrote:
>> 
>> 
>> On 28/01/2025 00:19, Josh Steadmon wrote:
>> > Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
>> > functions in libgit.a. This initial patch defines build rules and an
>> > interface that exposes user agent string getter functions as a proof of
>> > concept. This library can be tested with `cargo test`.
>> 
>> It's great to see some tests. This is looking good, I've left a couple of
>> small comments below.
>> 
>> > +contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
>> > +	$(LD) -r $^ -o $@
>> 
>> This is a very long line and the ones below are pretty long - perhaps we
>> could put the list of sources in a variable?
>
> Done for V8.
>
>
>> > +contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
>> > +	$(OBJCOPY) --localize-hidden $^ $@
>> > +
>> > +contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
>> > +	$(AR) $(ARFLAGS) $@ $^
>> > [...]
>> > diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
>> > new file mode 100644
>> > index 0000000000..cd1602206e
>> > --- /dev/null
>> > +++ b/contrib/libgit-sys/public_symbol_export.c
>> > @@ -0,0 +1,22 @@
>> > +/* Shim to publicly export Git symbols. These must be renamed so that the
>> 
>> Style: Multiline comments start with an empty "/*" so this should be
>> 
>> /*
>>  * Shim ...
>
> Whoops, fixed, thanks.

The series is looking good and it seems that we'll be expecting a
hopefully small and final reroll?  I'll mark the topic in the
"What's cooking" draft as such.

Thanks, all.

[PATCH v7 3/4] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/libgit-sys/public_symbol_export.c | 29 +++++++++++++++++++++
 contrib/libgit-sys/public_symbol_export.h | 10 ++++++++
 contrib/libgit-sys/src/lib.rs             | 31 ++++++++++++++++++++++-
 3 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
index cd1602206e..a0297cb1a5 100644
--- a/contrib/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -4,11 +4,40 @@
  */
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct config_set *cs = xmalloc(sizeof(struct config_set));
+	git_configset_init(cs);
+	return (struct libgit_config_set *) cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear((struct config_set *) cs);
+	free((struct config_set *) cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file((struct config_set *) cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
+{
+	return git_configset_get_int((struct config_set *) cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
+{
+	return git_configset_get_string((struct config_set *) cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.48.1.262.g85cc9f2d1e-goog

Re: [PATCH v7 3/4] libgit-sys: also export some config_set functions

[Phillip Wood]

Hi Josh

On 28/01/2025 00:19, Josh Steadmon wrote:
> In preparation for implementing a higher-level Rust API for accessing
> Git configs, export some of the upstream configset API via libgitpub and
> libgit-sys. Since this will be exercised as part of the higher-level API
> in the next commit, no tests have been added for libgit-sys.
> 
> While we're at it, add git_configset_alloc() and git_configset_free()
> functions in libgitpub so that callers can manage config_set structs on
> the heap. This also allows non-C external consumers to treat config_sets
> as opaque structs.

This interface is looks nice, I've left a couple of comments below

> diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> index cd1602206e..a0297cb1a5 100644
> --- a/contrib/libgit-sys/public_symbol_export.c
> +++ b/contrib/libgit-sys/public_symbol_export.c
> @@ -4,11 +4,40 @@
>    */
>   
>   #include "git-compat-util.h"
> +#include "config.h"
>   #include "contrib/libgit-sys/public_symbol_export.h"
>   #include "version.h"
>   
>   #pragma GCC visibility push(default)

Personally I'd prefer it if we actually defined struct libgit_config_set 
here

struct libgit_config_set {
	struct config_set cs;
}

Then we could avoid all the casts below. For example

struct libgit_config_set *libgit_configset_alloc(void)
{
	struct libget_config_set *cs =
		xmalloc(sizeof(struct libgit_config_set));
	git_configset_init(&cs->cs);
	return cs;
}

> +struct libgit_config_set *libgit_configset_alloc(void)
> +{
> +	struct config_set *cs = xmalloc(sizeof(struct config_set));
> +	git_configset_init(cs);
> +	return (struct libgit_config_set *) cs;
> +}
> +
> +void libgit_configset_free(struct libgit_config_set *cs)
> +{
> +	git_configset_clear((struct config_set *) cs);
> +	free((struct config_set *) cs);
> +}
> +
> +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
> +{
> +	return git_configset_add_file((struct config_set *) cs, filename);
> +}
> +
> +int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)

Style: this and the one below could do with being wrapped at 80 characters

This whole series looks pretty good to me

Best Wishes

Phillip

> +{
> +	return git_configset_get_int((struct config_set *) cs, key, dest);
> +}
> +
> +int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
> +{
> +	return git_configset_get_string((struct config_set *) cs, key, dest);
> +}
> +
>   const char *libgit_user_agent(void)
>   {
>   	return git_user_agent();
> diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
> index a3372f93fa..701db92d53 100644
> --- a/contrib/libgit-sys/public_symbol_export.h
> +++ b/contrib/libgit-sys/public_symbol_export.h
> @@ -1,6 +1,16 @@
>   #ifndef PUBLIC_SYMBOL_EXPORT_H
>   #define PUBLIC_SYMBOL_EXPORT_H
>   
> +struct libgit_config_set *libgit_configset_alloc(void);
> +
> +void libgit_configset_free(struct libgit_config_set *cs);
> +
> +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
> +
> +int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
> +
> +int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
> +
>   const char *libgit_user_agent(void);
>   
>   const char *libgit_user_agent_sanitized(void);
> diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
> index d4853f3074..dadb4e5f40 100644
> --- a/contrib/libgit-sys/src/lib.rs
> +++ b/contrib/libgit-sys/src/lib.rs
> @@ -1,15 +1,44 @@
>   #[cfg(has_std__ffi__c_char)]
> -use std::ffi::c_char;
> +use std::ffi::{c_char, c_int};
>   
>   #[cfg(not(has_std__ffi__c_char))]
>   #[allow(non_camel_case_types)]
>   pub type c_char = i8;
>   
> +#[cfg(not(has_std__ffi__c_char))]
> +#[allow(non_camel_case_types)]
> +pub type c_int = i32;
> +
>   extern crate libz_sys;
>   
> +#[allow(non_camel_case_types)]
> +#[repr(C)]
> +pub struct libgit_config_set {
> +    _data: [u8; 0],
> +    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
> +}
> +
>   extern "C" {
>       pub fn libgit_user_agent() -> *const c_char;
>       pub fn libgit_user_agent_sanitized() -> *const c_char;
> +
> +    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
> +    pub fn libgit_configset_free(cs: *mut libgit_config_set);
> +
> +    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
> +
> +    pub fn libgit_configset_get_int(
> +        cs: *mut libgit_config_set,
> +        key: *const c_char,
> +        int: *mut c_int,
> +    ) -> c_int;
> +
> +    pub fn libgit_configset_get_string(
> +        cs: *mut libgit_config_set,
> +        key: *const c_char,
> +        dest: *mut *mut c_char,
> +    ) -> c_int;
> +
>   }
>   
>   #[cfg(test)]

Re: [PATCH v7 3/4] libgit-sys: also export some config_set functions

[Josh Steadmon]

On 2025.01.28 15:08, Phillip Wood wrote:
> Hi Josh
> 
> On 28/01/2025 00:19, Josh Steadmon wrote:
> > In preparation for implementing a higher-level Rust API for accessing
> > Git configs, export some of the upstream configset API via libgitpub and
> > libgit-sys. Since this will be exercised as part of the higher-level API
> > in the next commit, no tests have been added for libgit-sys.
> > 
> > While we're at it, add git_configset_alloc() and git_configset_free()
> > functions in libgitpub so that callers can manage config_set structs on
> > the heap. This also allows non-C external consumers to treat config_sets
> > as opaque structs.
> 
> This interface is looks nice, I've left a couple of comments below
> 
> > diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
> > index cd1602206e..a0297cb1a5 100644
> > --- a/contrib/libgit-sys/public_symbol_export.c
> > +++ b/contrib/libgit-sys/public_symbol_export.c
> > @@ -4,11 +4,40 @@
> >    */
> >   #include "git-compat-util.h"
> > +#include "config.h"
> >   #include "contrib/libgit-sys/public_symbol_export.h"
> >   #include "version.h"
> >   #pragma GCC visibility push(default)
> 
> Personally I'd prefer it if we actually defined struct libgit_config_set
> here
> 
> struct libgit_config_set {
> 	struct config_set cs;
> }
> 
> Then we could avoid all the casts below. For example
> 
> struct libgit_config_set *libgit_configset_alloc(void)
> {
> 	struct libget_config_set *cs =
> 		xmalloc(sizeof(struct libgit_config_set));
> 	git_configset_init(&cs->cs);
> 	return cs;
> }

Hmm yeah I remember this feedback from (checks Lore) back in V2. I think
you're right, we should have gone this way from the beginning. Done in
V8.

> > +struct libgit_config_set *libgit_configset_alloc(void)
> > +{
> > +	struct config_set *cs = xmalloc(sizeof(struct config_set));
> > +	git_configset_init(cs);
> > +	return (struct libgit_config_set *) cs;
> > +}
> > +
> > +void libgit_configset_free(struct libgit_config_set *cs)
> > +{
> > +	git_configset_clear((struct config_set *) cs);
> > +	free((struct config_set *) cs);
> > +}
> > +
> > +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
> > +{
> > +	return git_configset_add_file((struct config_set *) cs, filename);
> > +}
> > +
> > +int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
> 
> Style: this and the one below could do with being wrapped at 80 characters

Fixed.

> This whole series looks pretty good to me
> 
> Best Wishes
> 
> Phillip

[PATCH v7 4/4] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

The C functions exported by libgit-sys do not provide an idiomatic Rust
interface. To make it easier to use these functions via Rust, add a
higher-level "libgit" crate, that wraps the lower-level configset API
with an interface that is more Rust-y.

This combination of $X and $X-sys crates is a common pattern for FFI in
Rust, as documented in "The Cargo Book" [1].

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 .gitignore                         |   1 +
 Makefile                           |  13 ++--
 contrib/libgit-rs/Cargo.lock       |  77 +++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml       |  17 +++++
 contrib/libgit-rs/README.md        |  13 ++++
 contrib/libgit-rs/build.rs         |   4 ++
 contrib/libgit-rs/src/config.rs    | 106 +++++++++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs       |   1 +
 contrib/libgit-rs/testdata/config1 |   2 +
 contrib/libgit-rs/testdata/config2 |   2 +
 contrib/libgit-rs/testdata/config3 |   2 +
 contrib/libgit-sys/src/lib.rs      |   4 ++
 t/Makefile                         |   9 ++-
 13 files changed, 245 insertions(+), 6 deletions(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index 31d7e64287..acdd8ce7c7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,4 +250,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 92989d3311..37b63de079 100644
--- a/Makefile
+++ b/Makefile
@@ -417,7 +417,7 @@ include shared.mak
 # programs in oss-fuzz/.
 #
 # Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
-# test the Rust crate in contrib/libgit-sys.
+# test the Rust crates in contrib/libgit-sys and contrib/libgit-rs.
 #
 # === Optional library: libintl ===
 #
@@ -3741,7 +3741,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-sys/target
+	$(RM) -r contrib/libgit-sys/target contrib/libgit-rs/target
 	$(RM) contrib/libgit-sys/partial_symbol_export.o
 	$(RM) contrib/libgit-sys/hidden_symbol_export.o
 	$(RM) contrib/libgit-sys/libgitpub.a
@@ -3907,14 +3907,19 @@ build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
-.PHONY: libgit-sys
+.PHONY: libgit-sys libgit-rs
 libgit-sys:
 	$(QUIET)(\
 		cd contrib/libgit-sys && \
 		cargo build \
 	)
+libgit-rs:
+	$(QUIET)(\
+		cd contrib/libgit-rs && \
+		cargo build \
+	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys
+all:: libgit-sys libgit-rs
 endif
 
 contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..c3289e69db
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "../libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/README.md b/contrib/libgit-rs/README.md
new file mode 100644
index 0000000000..ff945e1ce2
--- /dev/null
+++ b/contrib/libgit-rs/README.md
@@ -0,0 +1,13 @@
+# libgit-rs
+
+Proof-of-concept Git bindings for Rust.
+
+```toml
+[dependencies]
+libgit = "0.1.0"
+```
+
+## Rust version requirements
+
+libgit-rs should support Rust versions at least as old as the version included
+in Debian stable (currently 1.63).
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/src/config.rs b/contrib/libgit-rs/src/config.rs
new file mode 100644
index 0000000000..6bf04845c8
--- /dev/null
+++ b/contrib/libgit-rs/src/config.rs
@@ -0,0 +1,106 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_int = i32;
+
+use libgit_sys::*;
+
+/// A ConfigSet is an in-memory cache for config-like files such as `.gitmodules` or `.gitconfig`.
+/// It does not support all config directives; notably, it will not process `include` or
+/// `includeIf` directives (but it will store them so that callers can choose whether and how to
+/// handle them).
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    /// Allocate a new ConfigSet
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    /// Load the given files into the ConfigSet; conflicting directives in later files will
+    /// override those given in earlier files.
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
+    /// if the value cannot be parsed. Returns None if the key is not present.
+    pub fn get_int(&mut self, key: &str) -> Option<i32> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val.into())
+    }
+
+    /// Clones the value for the given key. Dies with a fatal error if the value cannot be
+    /// converted to a String. Returns None if the key is not present.
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..ef68c36943
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1 @@
+pub mod config;
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
diff --git a/t/Makefile b/t/Makefile
index 53ba01c21b..2994eb5fa9 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -178,12 +178,17 @@ perf:
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
 
-.PHONY: libgit-sys-test
+.PHONY: libgit-sys-test libgit-rs-test
 libgit-sys-test:
 	$(QUIET)(\
 		cd ../contrib/libgit-sys && \
 		cargo test \
 	)
+libgit-rs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys-test
+all:: libgit-sys-test libgit-rs-test
 endif
-- 
2.48.1.262.g85cc9f2d1e-goog

Re: [PATCH v7 4/4] libgit: add higher-level libgit crate

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> -.PHONY: libgit-sys
> +.PHONY: libgit-sys libgit-rs
>  libgit-sys:
>  	$(QUIET)(\
>  		cd contrib/libgit-sys && \
>  		cargo build \
>  	)
> +libgit-rs:
> +	$(QUIET)(\
> +		cd contrib/libgit-rs && \
> +		cargo build \
> +	)
>  ifdef INCLUDE_LIBGIT_RS
> -all:: libgit-sys
> +all:: libgit-sys libgit-rs
>  endif

I somehow would have expected this part of the patch to do

	libgit-sys libgit-rs:
		$(QUIET)( \
			cd contrib/$@ && cargo build \
		)

but the above longhand is fine.

> +    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
> +    /// if the value cannot be parsed. Returns None if the key is not present.
> +    pub fn get_int(&mut self, key: &str) -> Option<i32> {
> +        let key = CString::new(key).expect("Couldn't convert to CString");
> +        let mut val: c_int = 0;
> +        unsafe {
> +            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
> +                return None;
> +            }
> +        }
> +
> +        Some(val.into())
> +    }

Nice.

I was wondering why libgit_configset_get_int() in [3/4] does not do
better than just wrapping the raw C interface, which returns the
error/success status separately from the value we parsed.  With this
higher-layer wrapper around it, the interface becomes a bit more
like a higher-level language.

Thanks, will queue.

Re: [PATCH v7 4/4] libgit: add higher-level libgit crate

[Josh Steadmon]

On 2025.01.27 20:56, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > -.PHONY: libgit-sys
> > +.PHONY: libgit-sys libgit-rs
> >  libgit-sys:
> >  	$(QUIET)(\
> >  		cd contrib/libgit-sys && \
> >  		cargo build \
> >  	)
> > +libgit-rs:
> > +	$(QUIET)(\
> > +		cd contrib/libgit-rs && \
> > +		cargo build \
> > +	)
> >  ifdef INCLUDE_LIBGIT_RS
> > -all:: libgit-sys
> > +all:: libgit-sys libgit-rs
> >  endif
> 
> I somehow would have expected this part of the patch to do
> 
> 	libgit-sys libgit-rs:
> 		$(QUIET)( \
> 			cd contrib/$@ && cargo build \
> 		)
> 
> but the above longhand is fine.

Fixed since I'm sending out a V8 for Phillip's feedback anyway. Thanks.

> > +    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
> > +    /// if the value cannot be parsed. Returns None if the key is not present.
> > +    pub fn get_int(&mut self, key: &str) -> Option<i32> {
> > +        let key = CString::new(key).expect("Couldn't convert to CString");
> > +        let mut val: c_int = 0;
> > +        unsafe {
> > +            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
> > +                return None;
> > +            }
> > +        }
> > +
> > +        Some(val.into())
> > +    }
> 
> Nice.
> 
> I was wondering why libgit_configset_get_int() in [3/4] does not do
> better than just wrapping the raw C interface, which returns the
> error/success status separately from the value we parsed.  With this
> higher-layer wrapper around it, the interface becomes a bit more
> like a higher-level language.
> 
> Thanks, will queue.

Re: [PATCH v7 4/4] libgit: add higher-level libgit crate

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

>> I somehow would have expected this part of the patch to do
>> 
>> 	libgit-sys libgit-rs:
>> 		$(QUIET)( \
>> 			cd contrib/$@ && cargo build \
>> 		)
>> 
>> but the above longhand is fine.
>
> Fixed since I'm sending out a V8 for Phillip's feedback anyway. Thanks.

It would be a "fix" only if we anticipate that we will add in the
future more Rust things to contrib/ and all of them would follow the
same pattern.  Otherwise, it does not really matter, and as I said,
what you posted is just fine.

Thanks.

[PATCH v8 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/jj-vcs/jj

There is known NEEDSWORK, but I feel that they can be addressed in
follow-up changes, rather than in this series. If you feel otherwise,
please let me know:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning

Changes in V8:
* Define a private libgit_config_set struct to avoid excessive casting
  in public_symbol_export.c.

* Style fixes: merge some Makefile rules, limit rule line length by
  defining intermediate variables, add initial empty comment line, add
  linebreaks in function definitions.

Changes in V7:
* Moved the ConfigSet implementation in libgit-rs to a `config` module.

* Added doc comments for ConfigSet and its methods.

* Fix meson builds by adding new object files to `libgit_sources`

* Moved INCLUDE_LIBGIT_RS Makefile changes earlier in the series, so
  that we can make it optional to compile some of the libgitpub sources.
  Squashed V6 patch 5/5 into this series' patch 4/4.

* Don't publicly export FFI types in libgit-rs.

* Removed extraneous `-r` argument to $(RM) in the clean rules.

* Added TODO reminder in Cargo.toml about removing Cargo.lock once we
  hit a certain minimum supported Rust version.

* Style cleanup in public_symbol_export.c

Changes in V6:
* Rebased onto current master, since V5 was several months old.

* Move libgit-sys out of libgit-rs; while this sort of nesting is common
  in Rust crates with standalone repositories, it doesn't make as much
  sense when they're contained in the larger Git project's repo.

* Standardize the naming of some of the Makefile targets to always
  include a dash in the "-rs" or "-sys" suffixes.

* Clean up READMEs and crate descriptions in preparation for
  uploading to crates.io.

Changes in V5:
* When building with INCLUDE_LIBGIT_RS defined, add
  "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
  visibility in libgitpub.a without causing `make all` to rebuild from
  scratch due to changing CFLAGS.

* Avoid using c_int in the higher-level Rust API.

* Remove libgitpub.a and intermediate files with `make clean`.

Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml


Calvin Wan (1):
  libgit: add higher-level libgit crate

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                |   2 +
 Makefile                                  |  49 ++++++++++
 common-exit.c                             |  26 ++++++
 common-init.c                             |  63 +++++++++++++
 common-init.h                             |   6 ++
 common-main.c                             |  83 +----------------
 contrib/libgit-rs/Cargo.lock              |  77 ++++++++++++++++
 contrib/libgit-rs/Cargo.toml              |  17 ++++
 contrib/libgit-rs/README.md               |  13 +++
 contrib/libgit-rs/build.rs                |   4 +
 contrib/libgit-rs/src/config.rs           | 106 ++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs              |   1 +
 contrib/libgit-rs/testdata/config1        |   2 +
 contrib/libgit-rs/testdata/config2        |   2 +
 contrib/libgit-rs/testdata/config3        |   2 +
 contrib/libgit-sys/Cargo.lock             |  69 ++++++++++++++
 contrib/libgit-sys/Cargo.toml             |  19 ++++
 contrib/libgit-sys/README.md              |   4 +
 contrib/libgit-sys/build.rs               |  35 +++++++
 contrib/libgit-sys/public_symbol_export.c |  59 ++++++++++++
 contrib/libgit-sys/public_symbol_export.h |  18 ++++
 contrib/libgit-sys/src/lib.rs             |  79 ++++++++++++++++
 meson.build                               |   2 +
 t/Makefile                                |  15 +++
 24 files changed, 672 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

Range-diff against v7:
1:  cd0cb9aa04 = 1:  cd0cb9aa04 common-main: split init and exit code into new files
2:  f1502b8590 ! 2:  3588a3c3fc libgit-sys: introduce Rust wrapper for libgit.a
    @@ Makefile: $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GIT
     +all:: libgit-sys
     +endif
     +
    -+contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    ++LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
    ++LIBGIT_PUB_OBJS += libgit.a
    ++LIBGIT_PUB_OBJS += reftable/libreftable.a
    ++LIBGIT_PUB_OBJS += xdiff/lib.a
    ++
    ++LIBGIT_PARTIAL_EXPORT = contrib/libgit-sys/partial_symbol_export.o
    ++
    ++LIBGIT_HIDDEN_EXPORT = contrib/libgit-sys/hidden_symbol_export.o
    ++
    ++$(LIBGIT_PARTIAL_EXPORT): $(LIBGIT_PUB_OBJS)
     +	$(LD) -r $^ -o $@
     +
    -+contrib/libgit-sys/hidden_symbol_export.o: contrib/libgit-sys/partial_symbol_export.o
    ++$(LIBGIT_HIDDEN_EXPORT): $(LIBGIT_PARTIAL_EXPORT)
     +	$(OBJCOPY) --localize-hidden $^ $@
     +
    -+contrib/libgit-sys/libgitpub.a: contrib/libgit-sys/hidden_symbol_export.o
    ++contrib/libgit-sys/libgitpub.a: $(LIBGIT_HIDDEN_EXPORT)
     +	$(AR) $(ARFLAGS) $@ $^
     
      ## contrib/libgit-sys/Cargo.lock (new) ##
    @@ contrib/libgit-sys/build.rs (new)
     
      ## contrib/libgit-sys/public_symbol_export.c (new) ##
     @@
    -+/* Shim to publicly export Git symbols. These must be renamed so that the
    ++/*
    ++ * Shim to publicly export Git symbols. These must be renamed so that the
     + * original symbols can be hidden. Renaming these with a "libgit_" prefix also
     + * avoids conflicts with other libraries such as libgit2.
     + */
3:  d67d3648d1 ! 3:  f4452fffe6 libgit-sys: also export some config_set functions
    @@ contrib/libgit-sys/public_symbol_export.c
      
      #pragma GCC visibility push(default)
      
    ++struct libgit_config_set {
    ++	struct config_set cs;
    ++};
    ++
     +struct libgit_config_set *libgit_configset_alloc(void)
     +{
    -+	struct config_set *cs = xmalloc(sizeof(struct config_set));
    -+	git_configset_init(cs);
    -+	return (struct libgit_config_set *) cs;
    ++	struct libgit_config_set *cs =
    ++			xmalloc(sizeof(struct libgit_config_set));
    ++	git_configset_init(&cs->cs);
    ++	return cs;
     +}
     +
     +void libgit_configset_free(struct libgit_config_set *cs)
     +{
    -+	git_configset_clear((struct config_set *) cs);
    -+	free((struct config_set *) cs);
    ++	git_configset_clear(&cs->cs);
    ++	free(&cs->cs);
     +}
     +
     +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
     +{
    -+	return git_configset_add_file((struct config_set *) cs, filename);
    ++	return git_configset_add_file(&cs->cs, filename);
     +}
     +
    -+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest)
    ++int libgit_configset_get_int(struct libgit_config_set *cs, const char *key,
    ++			     int *dest)
     +{
    -+	return git_configset_get_int((struct config_set *) cs, key, dest);
    ++	return git_configset_get_int(&cs->cs, key, dest);
     +}
     +
    -+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest)
    ++int libgit_configset_get_string(struct libgit_config_set *cs, const char *key,
    ++				char **dest)
     +{
    -+	return git_configset_get_string((struct config_set *) cs, key, dest);
    ++	return git_configset_get_string(&cs->cs, key, dest);
     +}
     +
      const char *libgit_user_agent(void)
4:  88425bb0b1 ! 4:  ada9fc0a13 libgit: add higher-level libgit crate
    @@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
      	$(MAKE) -C t/ unit-tests
      
     -.PHONY: libgit-sys
    +-libgit-sys:
     +.PHONY: libgit-sys libgit-rs
    - libgit-sys:
    ++libgit-sys libgit-rs:
      	$(QUIET)(\
    - 		cd contrib/libgit-sys && \
    +-		cd contrib/libgit-sys && \
    ++		cd contrib/$@ && \
      		cargo build \
      	)
    -+libgit-rs:
    -+	$(QUIET)(\
    -+		cd contrib/libgit-rs && \
    -+		cargo build \
    -+	)
      ifdef INCLUDE_LIBGIT_RS
     -all:: libgit-sys
     +all:: libgit-sys libgit-rs
      endif
      
    - contrib/libgit-sys/partial_symbol_export.o: contrib/libgit-sys/public_symbol_export.o libgit.a reftable/libreftable.a xdiff/lib.a
    + LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
     
      ## contrib/libgit-rs/Cargo.lock (new) ##
     @@

base-commit: 757161efcca150a9a96b312d9e780a071e601a03
-- 
2.48.1.262.g85cc9f2d1e-goog

[PATCH v8 1/4] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 meson.build   |  2 ++
 6 files changed, 101 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 97e8385b66..27e68ac039 100644
--- a/Makefile
+++ b/Makefile
@@ -981,6 +981,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
diff --git a/meson.build b/meson.build
index 0064eb64f5..e5ba28b47f 100644
--- a/meson.build
+++ b/meson.build
@@ -245,6 +245,8 @@ libgit_sources = [
   'commit-graph.c',
   'commit-reach.c',
   'commit.c',
+  'common-exit.c',
+  'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
   'compat/terminal.c',
-- 
2.48.1.262.g85cc9f2d1e-goog

[PATCH v8 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                |  1 +
 Makefile                                  | 47 +++++++++++++++
 contrib/libgit-sys/Cargo.lock             | 69 +++++++++++++++++++++++
 contrib/libgit-sys/Cargo.toml             | 19 +++++++
 contrib/libgit-sys/README.md              |  4 ++
 contrib/libgit-sys/build.rs               | 35 ++++++++++++
 contrib/libgit-sys/public_symbol_export.c | 23 ++++++++
 contrib/libgit-sys/public_symbol_export.h |  8 +++
 contrib/libgit-sys/src/lib.rs             | 46 +++++++++++++++
 t/Makefile                                | 10 ++++
 10 files changed, 262 insertions(+)
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index e82aa19df0..31d7e64287 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,3 +250,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 27e68ac039..f2c08df4cb 100644
--- a/Makefile
+++ b/Makefile
@@ -416,6 +416,9 @@ include shared.mak
 # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
 # programs in oss-fuzz/.
 #
+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
+# test the Rust crate in contrib/libgit-sys.
+#
 # === Optional library: libintl ===
 #
 # Define NO_GETTEXT if you don't want Git output to be translated.
@@ -657,6 +660,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2236,6 +2241,12 @@ ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
+ifdef INCLUDE_LIBGIT_RS
+	# Enable symbol hiding in contrib/libgit-sys/libgitpub.a without making
+	# us rebuild the whole tree every time we run a Rust build.
+	BASIC_CFLAGS += -fvisibility=hidden
+endif
+
 ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
@@ -2732,6 +2743,10 @@ OBJECTS += $(UNIT_TEST_OBJS)
 OBJECTS += $(CLAR_TEST_OBJS)
 OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
 
+ifdef INCLUDE_LIBGIT_RS
+	OBJECTS += contrib/libgit-sys/public_symbol_export.o
+endif
+
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
 endif
@@ -3726,6 +3741,10 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-sys/target
+	$(RM) contrib/libgit-sys/partial_symbol_export.o
+	$(RM) contrib/libgit-sys/hidden_symbol_export.o
+	$(RM) contrib/libgit-sys/libgitpub.a
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3887,3 +3906,31 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
 build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+.PHONY: libgit-sys
+libgit-sys:
+	$(QUIET)(\
+		cd contrib/libgit-sys && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys
+endif
+
+LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
+LIBGIT_PUB_OBJS += libgit.a
+LIBGIT_PUB_OBJS += reftable/libreftable.a
+LIBGIT_PUB_OBJS += xdiff/lib.a
+
+LIBGIT_PARTIAL_EXPORT = contrib/libgit-sys/partial_symbol_export.o
+
+LIBGIT_HIDDEN_EXPORT = contrib/libgit-sys/hidden_symbol_export.o
+
+$(LIBGIT_PARTIAL_EXPORT): $(LIBGIT_PUB_OBJS)
+	$(LD) -r $^ -o $@
+
+$(LIBGIT_HIDDEN_EXPORT): $(LIBGIT_PARTIAL_EXPORT)
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-sys/libgitpub.a: $(LIBGIT_HIDDEN_EXPORT)
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-sys/Cargo.lock b/contrib/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-sys/Cargo.toml b/contrib/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..e0623022c3
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+description = "Native bindings to a portion of libgit"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
new file mode 100644
index 0000000000..c061cfcaf5
--- /dev/null
+++ b/contrib/libgit-sys/README.md
@@ -0,0 +1,4 @@
+# libgit-sys
+
+A small proof-of-concept crate showing how to provide a Rust FFI to Git
+internals.
diff --git a/contrib/libgit-sys/build.rs b/contrib/libgit-sys/build.rs
new file mode 100644
index 0000000000..3ffd80ad91
--- /dev/null
+++ b/contrib/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "INCLUDE_LIBGIT_RS=YesPlease",
+            "contrib/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..4c153c6f0d
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -0,0 +1,23 @@
+/*
+ * Shim to publicly export Git symbols. These must be renamed so that the
+ * original symbols can be hidden. Renaming these with a "libgit_" prefix also
+ * avoids conflicts with other libraries such as libgit2.
+ */
+
+#include "git-compat-util.h"
+#include "contrib/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
diff --git a/t/Makefile b/t/Makefile
index daa5fcae86..53ba01c21b 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -177,3 +177,13 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgit-sys-test
+libgit-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-sys && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys-test
+endif
-- 
2.48.1.262.g85cc9f2d1e-goog

Re: [PATCH v8 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> diff --git a/Makefile b/Makefile
> index 27e68ac039..f2c08df4cb 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -416,6 +416,9 @@ include shared.mak
>  # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
>  # programs in oss-fuzz/.
>  #
> +# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
> +# test the Rust crate in contrib/libgit-sys.
> +#
>  # === Optional library: libintl ===
>  #
>  # Define NO_GETTEXT if you don't want Git output to be translated.
> @@ -657,6 +660,8 @@ CURL_CONFIG = curl-config
>  GCOV = gcov
>  STRIP = strip
>  SPATCH = spatch
> +LD = ld
> +OBJCOPY = objcopy
>  
>  export TCL_PATH TCLTK_PATH

Good.

> +LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
> +LIBGIT_PUB_OBJS += libgit.a
> +LIBGIT_PUB_OBJS += reftable/libreftable.a
> +LIBGIT_PUB_OBJS += xdiff/lib.a

This is a fairly minor point, but if we look at the places where
OBJECTS, PROGRAMS, and friends are prepared in the Makefile, they
all start with empty, i.e.

	LIBGIT_PUB_OBJS =

at a fairly early place in the file, and then appends real contents
to the symbol with += assignment.  It probably makes sense to follow
suit, although the result would be the same.

Other than that, nothing jumped at me as fishy.  Nicely done.

Thanks.  Will replace.

Re: [PATCH v8 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

On 2025.01.28 14:43, Junio C Hamano wrote:
> Josh Steadmon <steadmon@google.com> writes:
> 
> > diff --git a/Makefile b/Makefile
> > index 27e68ac039..f2c08df4cb 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -416,6 +416,9 @@ include shared.mak
> >  # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
> >  # programs in oss-fuzz/.
> >  #
> > +# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
> > +# test the Rust crate in contrib/libgit-sys.
> > +#
> >  # === Optional library: libintl ===
> >  #
> >  # Define NO_GETTEXT if you don't want Git output to be translated.
> > @@ -657,6 +660,8 @@ CURL_CONFIG = curl-config
> >  GCOV = gcov
> >  STRIP = strip
> >  SPATCH = spatch
> > +LD = ld
> > +OBJCOPY = objcopy
> >  
> >  export TCL_PATH TCLTK_PATH
> 
> Good.
> 
> > +LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
> > +LIBGIT_PUB_OBJS += libgit.a
> > +LIBGIT_PUB_OBJS += reftable/libreftable.a
> > +LIBGIT_PUB_OBJS += xdiff/lib.a
> 
> This is a fairly minor point, but if we look at the places where
> OBJECTS, PROGRAMS, and friends are prepared in the Makefile, they
> all start with empty, i.e.
> 
> 	LIBGIT_PUB_OBJS =
> 
> at a fairly early place in the file, and then appends real contents
> to the symbol with += assignment.  It probably makes sense to follow
> suit, although the result would be the same.
> 
> Other than that, nothing jumped at me as fishy.  Nicely done.
> 
> Thanks.  Will replace.

Ack, will send a V9 soon with this & a fix for Phillip's feedback.
Thanks.

[PATCH v8 3/4] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/libgit-sys/public_symbol_export.c | 36 +++++++++++++++++++++++
 contrib/libgit-sys/public_symbol_export.h | 10 +++++++
 contrib/libgit-sys/src/lib.rs             | 31 ++++++++++++++++++-
 3 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
index 4c153c6f0d..dbb7948104 100644
--- a/contrib/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -5,11 +5,47 @@
  */
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set {
+	struct config_set cs;
+};
+
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct libgit_config_set *cs =
+			xmalloc(sizeof(struct libgit_config_set));
+	git_configset_init(&cs->cs);
+	return cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear(&cs->cs);
+	free(&cs->cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file(&cs->cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key,
+			     int *dest)
+{
+	return git_configset_get_int(&cs->cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key,
+				char **dest)
+{
+	return git_configset_get_string(&cs->cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.48.1.262.g85cc9f2d1e-goog

[PATCH v8 4/4] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

The C functions exported by libgit-sys do not provide an idiomatic Rust
interface. To make it easier to use these functions via Rust, add a
higher-level "libgit" crate, that wraps the lower-level configset API
with an interface that is more Rust-y.

This combination of $X and $X-sys crates is a common pattern for FFI in
Rust, as documented in "The Cargo Book" [1].

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 .gitignore                         |   1 +
 Makefile                           |  12 ++--
 contrib/libgit-rs/Cargo.lock       |  77 +++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml       |  17 +++++
 contrib/libgit-rs/README.md        |  13 ++++
 contrib/libgit-rs/build.rs         |   4 ++
 contrib/libgit-rs/src/config.rs    | 106 +++++++++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs       |   1 +
 contrib/libgit-rs/testdata/config1 |   2 +
 contrib/libgit-rs/testdata/config2 |   2 +
 contrib/libgit-rs/testdata/config3 |   2 +
 contrib/libgit-sys/src/lib.rs      |   4 ++
 t/Makefile                         |   9 ++-
 13 files changed, 242 insertions(+), 8 deletions(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index 31d7e64287..acdd8ce7c7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,4 +250,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index f2c08df4cb..f883d310aa 100644
--- a/Makefile
+++ b/Makefile
@@ -417,7 +417,7 @@ include shared.mak
 # programs in oss-fuzz/.
 #
 # Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
-# test the Rust crate in contrib/libgit-sys.
+# test the Rust crates in contrib/libgit-sys and contrib/libgit-rs.
 #
 # === Optional library: libintl ===
 #
@@ -3741,7 +3741,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-sys/target
+	$(RM) -r contrib/libgit-sys/target contrib/libgit-rs/target
 	$(RM) contrib/libgit-sys/partial_symbol_export.o
 	$(RM) contrib/libgit-sys/hidden_symbol_export.o
 	$(RM) contrib/libgit-sys/libgitpub.a
@@ -3907,14 +3907,14 @@ build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
-.PHONY: libgit-sys
-libgit-sys:
+.PHONY: libgit-sys libgit-rs
+libgit-sys libgit-rs:
 	$(QUIET)(\
-		cd contrib/libgit-sys && \
+		cd contrib/$@ && \
 		cargo build \
 	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys
+all:: libgit-sys libgit-rs
 endif
 
 LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..c3289e69db
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "../libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/README.md b/contrib/libgit-rs/README.md
new file mode 100644
index 0000000000..ff945e1ce2
--- /dev/null
+++ b/contrib/libgit-rs/README.md
@@ -0,0 +1,13 @@
+# libgit-rs
+
+Proof-of-concept Git bindings for Rust.
+
+```toml
+[dependencies]
+libgit = "0.1.0"
+```
+
+## Rust version requirements
+
+libgit-rs should support Rust versions at least as old as the version included
+in Debian stable (currently 1.63).
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/src/config.rs b/contrib/libgit-rs/src/config.rs
new file mode 100644
index 0000000000..6bf04845c8
--- /dev/null
+++ b/contrib/libgit-rs/src/config.rs
@@ -0,0 +1,106 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_int = i32;
+
+use libgit_sys::*;
+
+/// A ConfigSet is an in-memory cache for config-like files such as `.gitmodules` or `.gitconfig`.
+/// It does not support all config directives; notably, it will not process `include` or
+/// `includeIf` directives (but it will store them so that callers can choose whether and how to
+/// handle them).
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    /// Allocate a new ConfigSet
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    /// Load the given files into the ConfigSet; conflicting directives in later files will
+    /// override those given in earlier files.
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
+    /// if the value cannot be parsed. Returns None if the key is not present.
+    pub fn get_int(&mut self, key: &str) -> Option<i32> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val.into())
+    }
+
+    /// Clones the value for the given key. Dies with a fatal error if the value cannot be
+    /// converted to a String. Returns None if the key is not present.
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..ef68c36943
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1 @@
+pub mod config;
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
diff --git a/t/Makefile b/t/Makefile
index 53ba01c21b..2994eb5fa9 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -178,12 +178,17 @@ perf:
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
 
-.PHONY: libgit-sys-test
+.PHONY: libgit-sys-test libgit-rs-test
 libgit-sys-test:
 	$(QUIET)(\
 		cd ../contrib/libgit-sys && \
 		cargo test \
 	)
+libgit-rs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys-test
+all:: libgit-sys-test libgit-rs-test
 endif
-- 
2.48.1.262.g85cc9f2d1e-goog

Re: [PATCH v8 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Phillip Wood]

Hi Josh

On 28/01/2025 22:01, Josh Steadmon wrote:

Thanks for re-rolling, the range-diff looks good to me apart from

>       +void libgit_configset_free(struct libgit_config_set *cs)
>       +{
>      -+	git_configset_clear((struct config_set *) cs);
>      -+	free((struct config_set *) cs);
>      ++	git_configset_clear(&cs->cs);
>      ++	free(&cs->cs);

Which I think should be "free(cs)". In practice it does not matter 
because we pass the same value to free() but it seems a bit odd to pass 
the address of the first member of the struct rather than the address of 
the struct itself.

I'm looking forward to seeing this merged soon

Best Wishes

Phillip

Re: [PATCH v8 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

On 2025.01.29 15:24, Phillip Wood wrote:
> Hi Josh
> 
> On 28/01/2025 22:01, Josh Steadmon wrote:
> 
> Thanks for re-rolling, the range-diff looks good to me apart from
> 
> >       +void libgit_configset_free(struct libgit_config_set *cs)
> >       +{
> >      -+	git_configset_clear((struct config_set *) cs);
> >      -+	free((struct config_set *) cs);
> >      ++	git_configset_clear(&cs->cs);
> >      ++	free(&cs->cs);
> 
> Which I think should be "free(cs)". In practice it does not matter because
> we pass the same value to free() but it seems a bit odd to pass the address
> of the first member of the struct rather than the address of the struct
> itself.

Yep sorry, got a bit careless with search-and-replace. Thanks for the
catch!

> I'm looking forward to seeing this merged soon
> 
> Best Wishes
> 
> Phillip
> 

[PATCH v9 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Josh Steadmon]

This series provides two small Rust wrapper libraries around parts of
Git: "libgit-sys", which exposes a few functions from libgit.a, and
"libgit", which provides a more Rust-friendly interface to some of those
functions. In addition to included unit tests, at $DAYJOB we have tested
building JJ[1] with our library and used it to replace some of the
libgit2-rs uses.

[1] https://github.com/jj-vcs/jj

There is known NEEDSWORK, but I feel that they can be addressed in
follow-up changes, rather than in this series. If you feel otherwise,
please let me know:

* Investigate alternative methods of managing symbol visibility &
  renaming.

* Figure out symbol versioning

Changes in V9:
* Properly initialize Makefile var LIBGIT_PUB_OBJS.

* Pass the correct pointer to free in libgit_configset_free().

Changes in V8:
* Define a private libgit_config_set struct to avoid excessive casting
  in public_symbol_export.c.

* Style fixes: merge some Makefile rules, limit rule line length by
  defining intermediate variables, add initial empty comment line, add
  linebreaks in function definitions.

Changes in V7:
* Moved the ConfigSet implementation in libgit-rs to a `config` module.

* Added doc comments for ConfigSet and its methods.

* Fix meson builds by adding new object files to `libgit_sources`

* Moved INCLUDE_LIBGIT_RS Makefile changes earlier in the series, so
  that we can make it optional to compile some of the libgitpub sources.
  Squashed V6 patch 5/5 into this series' patch 4/4.

* Don't publicly export FFI types in libgit-rs.

* Removed extraneous `-r` argument to $(RM) in the clean rules.

* Added TODO reminder in Cargo.toml about removing Cargo.lock once we
  hit a certain minimum supported Rust version.

* Style cleanup in public_symbol_export.c

Changes in V6:
* Rebased onto current master, since V5 was several months old.

* Move libgit-sys out of libgit-rs; while this sort of nesting is common
  in Rust crates with standalone repositories, it doesn't make as much
  sense when they're contained in the larger Git project's repo.

* Standardize the naming of some of the Makefile targets to always
  include a dash in the "-rs" or "-sys" suffixes.

* Clean up READMEs and crate descriptions in preparation for
  uploading to crates.io.

Changes in V5:
* When building with INCLUDE_LIBGIT_RS defined, add
  "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
  visibility in libgitpub.a without causing `make all` to rebuild from
  scratch due to changing CFLAGS.

* Avoid using c_int in the higher-level Rust API.

* Remove libgitpub.a and intermediate files with `make clean`.

Changes in V4:
* Drop V3 patch #3, which added wrappers around repository
  initialization and config access. These are not well-libified, and
  they are not necessary for JJ's proof-of-concept use case, so let's
  avoid exporting them for now.

* Set a minimum supported Rust version of 1.63. Autodetect whether our
  Rust version has c_int and c_char types; if not, define them
  ourselves.

* When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
  additional errors at build time.

* In build.rs, use the make_cmd crate to portable select the correct
  invocation of GNU Make.

* Follow naming standards for _alloc() and _free() functions.

* Use String instead of CString in higher-level API.

* Move libgit_configset_alloc() and libgit_configset_free() out of
  upstream Git, to the libgitpub shim library.

* In libgitpub, initialize libgit_config_set structs in the _alloc()
  function rather than with a separate _init() function.

* Remove unnecessary comments in libgit-sys showing where the wrapped
  functions were originally defined.

* Fix clippy lint: don't reborrow configfile path references.

* Various typo fixes and `cargo fmt` fixes.

Changes in V3:
* Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default (Patch 6)

* Provide testdata configs for unit tests

* ConfigSet API now uses &Path instead of &str -- more ergonomic for
  Rust users to pass in and errors out if the path string isn't UTF-8

* Fixed unresolved dependency on libz in Cargo.toml


Calvin Wan (1):
  libgit: add higher-level libgit crate

Josh Steadmon (3):
  common-main: split init and exit code into new files
  libgit-sys: introduce Rust wrapper for libgit.a
  libgit-sys: also export some config_set functions

 .gitignore                                |   2 +
 Makefile                                  |  50 ++++++++++
 common-exit.c                             |  26 ++++++
 common-init.c                             |  63 +++++++++++++
 common-init.h                             |   6 ++
 common-main.c                             |  83 +----------------
 contrib/libgit-rs/Cargo.lock              |  77 ++++++++++++++++
 contrib/libgit-rs/Cargo.toml              |  17 ++++
 contrib/libgit-rs/README.md               |  13 +++
 contrib/libgit-rs/build.rs                |   4 +
 contrib/libgit-rs/src/config.rs           | 106 ++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs              |   1 +
 contrib/libgit-rs/testdata/config1        |   2 +
 contrib/libgit-rs/testdata/config2        |   2 +
 contrib/libgit-rs/testdata/config3        |   2 +
 contrib/libgit-sys/Cargo.lock             |  69 ++++++++++++++
 contrib/libgit-sys/Cargo.toml             |  19 ++++
 contrib/libgit-sys/README.md              |   4 +
 contrib/libgit-sys/build.rs               |  35 +++++++
 contrib/libgit-sys/public_symbol_export.c |  59 ++++++++++++
 contrib/libgit-sys/public_symbol_export.h |  18 ++++
 contrib/libgit-sys/src/lib.rs             |  79 ++++++++++++++++
 meson.build                               |   2 +
 t/Makefile                                |  15 +++
 24 files changed, 673 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

Range-diff against v8:
1:  cd0cb9aa04 = 1:  cd0cb9aa04 common-main: split init and exit code into new files
2:  3588a3c3fc ! 2:  8793ff64a7 libgit-sys: introduce Rust wrapper for libgit.a
    @@ Makefile: $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GIT
     +all:: libgit-sys
     +endif
     +
    -+LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
    ++LIBGIT_PUB_OBJS =
    ++LIBGIT_PUB_OBJS += contrib/libgit-sys/public_symbol_export.o
     +LIBGIT_PUB_OBJS += libgit.a
     +LIBGIT_PUB_OBJS += reftable/libreftable.a
     +LIBGIT_PUB_OBJS += xdiff/lib.a
3:  f4452fffe6 ! 3:  ab32bd1d07 libgit-sys: also export some config_set functions
    @@ contrib/libgit-sys/public_symbol_export.c
     +void libgit_configset_free(struct libgit_config_set *cs)
     +{
     +	git_configset_clear(&cs->cs);
    -+	free(&cs->cs);
    ++	free(cs);
     +}
     +
     +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
4:  ada9fc0a13 ! 4:  1bf8c5392c libgit: add higher-level libgit crate
    @@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
     +all:: libgit-sys libgit-rs
      endif
      
    - LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
    + LIBGIT_PUB_OBJS =
     
      ## contrib/libgit-rs/Cargo.lock (new) ##
     @@

base-commit: 757161efcca150a9a96b312d9e780a071e601a03
-- 
2.48.1.362.g079036d154-goog

[PATCH v9 1/4] common-main: split init and exit code into new files

[Josh Steadmon]

Currently, object files in libgit.a reference common_exit(), which is
contained in common-main.o. However, common-main.o also includes main(),
which references cmd_main() in git.o, which in turn depends on all the
builtin/*.o objects.

We would like to allow external users to link libgit.a without needing
to include so many extra objects. Enable this by splitting common_exit()
and check_bug_if_BUG() into a new file common-exit.c, and add
common-exit.o to LIB_OBJS so that these are included in libgit.a.

This split has previously been proposed ([1], [2]) to support fuzz tests
and unit tests by avoiding conflicting definitions for main(). However,
both of those issues were resolved by other methods of avoiding symbol
conflicts. Now we are trying to make libgit.a more self-contained, so
hopefully we can revisit this approach.

Additionally, move the initialization code out of main() into a new
init_git() function in its own file. Include this in libgit.a as well,
so that external users can share our setup code without calling our
main().

[1] https://lore.kernel.org/git/Yp+wjCPhqieTku3X@google.com/
[2] https://lore.kernel.org/git/20230517-unit-tests-v2-v2-1-21b5b60f4b32@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 Makefile      |  2 ++
 common-exit.c | 26 ++++++++++++++++
 common-init.c | 63 ++++++++++++++++++++++++++++++++++++++
 common-init.h |  6 ++++
 common-main.c | 83 ++-------------------------------------------------
 meson.build   |  2 ++
 6 files changed, 101 insertions(+), 81 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 common-init.c
 create mode 100644 common-init.h

diff --git a/Makefile b/Makefile
index 97e8385b66..27e68ac039 100644
--- a/Makefile
+++ b/Makefile
@@ -981,6 +981,8 @@ LIB_OBJS += combine-diff.o
 LIB_OBJS += commit-graph.o
 LIB_OBJS += commit-reach.o
 LIB_OBJS += commit.o
+LIB_OBJS += common-exit.o
+LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
 LIB_OBJS += compat/terminal.o
diff --git a/common-exit.c b/common-exit.c
new file mode 100644
index 0000000000..1aaa538be3
--- /dev/null
+++ b/common-exit.c
@@ -0,0 +1,26 @@
+#include "git-compat-util.h"
+#include "trace2.h"
+
+static void check_bug_if_BUG(void)
+{
+	if (!bug_called_must_BUG)
+		return;
+	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
+}
+
+/* We wrap exit() to call common_exit() in git-compat-util.h */
+int common_exit(const char *file, int line, int code)
+{
+	/*
+	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
+	 * to e.g. turn -1 into 255. On a POSIX system this is
+	 * redundant, see exit(3) and wait(2), but as it doesn't harm
+	 * anything there we don't need to guard this with an "ifdef".
+	 */
+	code &= 0xff;
+
+	check_bug_if_BUG();
+	trace2_cmd_exit_fl(file, line, code);
+
+	return code;
+}
diff --git a/common-init.c b/common-init.c
new file mode 100644
index 0000000000..5cc73f058c
--- /dev/null
+++ b/common-init.c
@@ -0,0 +1,63 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "common-init.h"
+#include "exec-cmd.h"
+#include "gettext.h"
+#include "attr.h"
+#include "repository.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "trace2.h"
+
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of a pipe to die with SIGPIPE when the downstream of a
+ * pipe does not need to read all that is written.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
+
+void init_git(const char **argv)
+{
+	struct strbuf tmp = STRBUF_INIT;
+
+	trace2_initialize_clock();
+
+	/*
+	 * Always open file descriptors 0/1/2 to avoid clobbering files
+	 * in die().  It also avoids messing up when the pipes are dup'ed
+	 * onto stdin/stdout/stderr in the child processes we spawn.
+	 */
+	sanitize_stdfds();
+	restore_sigpipe_to_default();
+
+	git_resolve_executable_dir(argv[0]);
+
+	setlocale(LC_CTYPE, "");
+	git_setup_gettext();
+
+	initialize_repository(the_repository);
+
+	attr_start();
+
+	trace2_initialize();
+	trace2_cmd_start(argv);
+	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
+
+	if (!strbuf_getcwd(&tmp))
+		tmp_original_cwd = strbuf_detach(&tmp, NULL);
+}
diff --git a/common-init.h b/common-init.h
new file mode 100644
index 0000000000..3e6db20cae
--- /dev/null
+++ b/common-init.h
@@ -0,0 +1,6 @@
+#ifndef COMMON_INIT_H
+#define COMMON_INIT_H
+
+void init_git(const char **argv);
+
+#endif /* COMMON_INIT_H */
diff --git a/common-main.c b/common-main.c
index 8e68ac9e42..6b7ab077b0 100644
--- a/common-main.c
+++ b/common-main.c
@@ -1,92 +1,13 @@
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
-#include "exec-cmd.h"
-#include "gettext.h"
-#include "attr.h"
-#include "repository.h"
-#include "setup.h"
-#include "strbuf.h"
-#include "trace2.h"
-
-/*
- * Many parts of Git have subprograms communicate via pipe, expect the
- * upstream of a pipe to die with SIGPIPE when the downstream of a
- * pipe does not need to read all that is written.  Some third-party
- * programs that ignore or block SIGPIPE for their own reason forget
- * to restore SIGPIPE handling to the default before spawning Git and
- * break this carefully orchestrated machinery.
- *
- * Restore the way SIGPIPE is handled to default, which is what we
- * expect.
- */
-static void restore_sigpipe_to_default(void)
-{
-	sigset_t unblock;
-
-	sigemptyset(&unblock);
-	sigaddset(&unblock, SIGPIPE);
-	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
-	signal(SIGPIPE, SIG_DFL);
-}
+#include "common-init.h"
 
 int main(int argc, const char **argv)
 {
 	int result;
-	struct strbuf tmp = STRBUF_INIT;
-
-	trace2_initialize_clock();
-
-	/*
-	 * Always open file descriptors 0/1/2 to avoid clobbering files
-	 * in die().  It also avoids messing up when the pipes are dup'ed
-	 * onto stdin/stdout/stderr in the child processes we spawn.
-	 */
-	sanitize_stdfds();
-	restore_sigpipe_to_default();
-
-	git_resolve_executable_dir(argv[0]);
-
-	setlocale(LC_CTYPE, "");
-	git_setup_gettext();
-
-	initialize_repository(the_repository);
-
-	attr_start();
-
-	trace2_initialize();
-	trace2_cmd_start(argv);
-	trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP);
-
-	if (!strbuf_getcwd(&tmp))
-		tmp_original_cwd = strbuf_detach(&tmp, NULL);
 
+	init_git(argv);
 	result = cmd_main(argc, argv);
 
 	/* Not exit(3), but a wrapper calling our common_exit() */
 	exit(result);
 }
-
-static void check_bug_if_BUG(void)
-{
-	if (!bug_called_must_BUG)
-		return;
-	BUG("on exit(): had bug() call(s) in this process without explicit BUG_if_bug()");
-}
-
-/* We wrap exit() to call common_exit() in git-compat-util.h */
-int common_exit(const char *file, int line, int code)
-{
-	/*
-	 * For non-POSIX systems: Take the lowest 8 bits of the "code"
-	 * to e.g. turn -1 into 255. On a POSIX system this is
-	 * redundant, see exit(3) and wait(2), but as it doesn't harm
-	 * anything there we don't need to guard this with an "ifdef".
-	 */
-	code &= 0xff;
-
-	check_bug_if_BUG();
-	trace2_cmd_exit_fl(file, line, code);
-
-	return code;
-}
diff --git a/meson.build b/meson.build
index 0064eb64f5..e5ba28b47f 100644
--- a/meson.build
+++ b/meson.build
@@ -245,6 +245,8 @@ libgit_sources = [
   'commit-graph.c',
   'commit-reach.c',
   'commit.c',
+  'common-exit.c',
+  'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
   'compat/terminal.c',
-- 
2.48.1.362.g079036d154-goog

[PATCH v9 2/4] libgit-sys: introduce Rust wrapper for libgit.a

[Josh Steadmon]

Introduce libgit-sys, a Rust wrapper crate that allows Rust code to call
functions in libgit.a. This initial patch defines build rules and an
interface that exposes user agent string getter functions as a proof of
concept. This library can be tested with `cargo test`. In later commits,
a higher-level library containing a more Rust-friendly interface will be
added at `contrib/libgit-rs`.

Symbols in libgit can collide with symbols from other libraries such as
libgit2. We avoid this by first exposing library symbols in
public_symbol_export.[ch]. These symbols are prepended with "libgit_" to
avoid collisions and set to visible using a visibility pragma. In
build.rs, Rust builds contrib/libgit-rs/libgit-sys/libgitpub.a, which also
contains libgit.a and other dependent libraries, with
-fvisibility=hidden to hide all symbols within those libraries that
haven't been exposed with a visibility pragma.

Co-authored-by: Kyle Lippincott <spectral@google.com>
Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .gitignore                                |  1 +
 Makefile                                  | 48 ++++++++++++++++
 contrib/libgit-sys/Cargo.lock             | 69 +++++++++++++++++++++++
 contrib/libgit-sys/Cargo.toml             | 19 +++++++
 contrib/libgit-sys/README.md              |  4 ++
 contrib/libgit-sys/build.rs               | 35 ++++++++++++
 contrib/libgit-sys/public_symbol_export.c | 23 ++++++++
 contrib/libgit-sys/public_symbol_export.h |  8 +++
 contrib/libgit-sys/src/lib.rs             | 46 +++++++++++++++
 t/Makefile                                | 10 ++++
 10 files changed, 263 insertions(+)
 create mode 100644 contrib/libgit-sys/Cargo.lock
 create mode 100644 contrib/libgit-sys/Cargo.toml
 create mode 100644 contrib/libgit-sys/README.md
 create mode 100644 contrib/libgit-sys/build.rs
 create mode 100644 contrib/libgit-sys/public_symbol_export.c
 create mode 100644 contrib/libgit-sys/public_symbol_export.h
 create mode 100644 contrib/libgit-sys/src/lib.rs

diff --git a/.gitignore b/.gitignore
index e82aa19df0..31d7e64287 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,3 +250,4 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index 27e68ac039..a772bff186 100644
--- a/Makefile
+++ b/Makefile
@@ -416,6 +416,9 @@ include shared.mak
 # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test
 # programs in oss-fuzz/.
 #
+# Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
+# test the Rust crate in contrib/libgit-sys.
+#
 # === Optional library: libintl ===
 #
 # Define NO_GETTEXT if you don't want Git output to be translated.
@@ -657,6 +660,8 @@ CURL_CONFIG = curl-config
 GCOV = gcov
 STRIP = strip
 SPATCH = spatch
+LD = ld
+OBJCOPY = objcopy
 
 export TCL_PATH TCLTK_PATH
 
@@ -2236,6 +2241,12 @@ ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
+ifdef INCLUDE_LIBGIT_RS
+	# Enable symbol hiding in contrib/libgit-sys/libgitpub.a without making
+	# us rebuild the whole tree every time we run a Rust build.
+	BASIC_CFLAGS += -fvisibility=hidden
+endif
+
 ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
@@ -2732,6 +2743,10 @@ OBJECTS += $(UNIT_TEST_OBJS)
 OBJECTS += $(CLAR_TEST_OBJS)
 OBJECTS += $(patsubst %,$(UNIT_TEST_DIR)/%.o,$(UNIT_TEST_PROGRAMS))
 
+ifdef INCLUDE_LIBGIT_RS
+	OBJECTS += contrib/libgit-sys/public_symbol_export.o
+endif
+
 ifndef NO_CURL
 	OBJECTS += http.o http-walker.o remote-curl.o
 endif
@@ -3726,6 +3741,10 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
+	$(RM) -r contrib/libgit-sys/target
+	$(RM) contrib/libgit-sys/partial_symbol_export.o
+	$(RM) contrib/libgit-sys/hidden_symbol_export.o
+	$(RM) contrib/libgit-sys/libgitpub.a
 ifndef NO_PERL
 	$(RM) -r perl/build/
 endif
@@ -3887,3 +3906,32 @@ $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-
 build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
+
+.PHONY: libgit-sys
+libgit-sys:
+	$(QUIET)(\
+		cd contrib/libgit-sys && \
+		cargo build \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys
+endif
+
+LIBGIT_PUB_OBJS =
+LIBGIT_PUB_OBJS += contrib/libgit-sys/public_symbol_export.o
+LIBGIT_PUB_OBJS += libgit.a
+LIBGIT_PUB_OBJS += reftable/libreftable.a
+LIBGIT_PUB_OBJS += xdiff/lib.a
+
+LIBGIT_PARTIAL_EXPORT = contrib/libgit-sys/partial_symbol_export.o
+
+LIBGIT_HIDDEN_EXPORT = contrib/libgit-sys/hidden_symbol_export.o
+
+$(LIBGIT_PARTIAL_EXPORT): $(LIBGIT_PUB_OBJS)
+	$(LD) -r $^ -o $@
+
+$(LIBGIT_HIDDEN_EXPORT): $(LIBGIT_PARTIAL_EXPORT)
+	$(OBJCOPY) --localize-hidden $^ $@
+
+contrib/libgit-sys/libgitpub.a: $(LIBGIT_HIDDEN_EXPORT)
+	$(AR) $(ARFLAGS) $@ $^
diff --git a/contrib/libgit-sys/Cargo.lock b/contrib/libgit-sys/Cargo.lock
new file mode 100644
index 0000000000..427a4c66b7
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.lock
@@ -0,0 +1,69 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-sys/Cargo.toml b/contrib/libgit-sys/Cargo.toml
new file mode 100644
index 0000000000..e0623022c3
--- /dev/null
+++ b/contrib/libgit-sys/Cargo.toml
@@ -0,0 +1,19 @@
+[package]
+name = "libgit-sys"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+links = "gitpub"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+description = "Native bindings to a portion of libgit"
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libz-sys = "1.1.19"
+
+[build-dependencies]
+autocfg = "1.4.0"
+make-cmd = "0.1.0"
diff --git a/contrib/libgit-sys/README.md b/contrib/libgit-sys/README.md
new file mode 100644
index 0000000000..c061cfcaf5
--- /dev/null
+++ b/contrib/libgit-sys/README.md
@@ -0,0 +1,4 @@
+# libgit-sys
+
+A small proof-of-concept crate showing how to provide a Rust FFI to Git
+internals.
diff --git a/contrib/libgit-sys/build.rs b/contrib/libgit-sys/build.rs
new file mode 100644
index 0000000000..3ffd80ad91
--- /dev/null
+++ b/contrib/libgit-sys/build.rs
@@ -0,0 +1,35 @@
+use std::env;
+use std::path::PathBuf;
+
+pub fn main() -> std::io::Result<()> {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+
+    let crate_root = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+    let git_root = crate_root.join("../..");
+    let dst = PathBuf::from(env::var_os("OUT_DIR").unwrap());
+
+    let make_output = make_cmd::gnu_make()
+        .env("DEVELOPER", "1")
+        .env_remove("PROFILE")
+        .current_dir(git_root.clone())
+        .args([
+            "INCLUDE_LIBGIT_RS=YesPlease",
+            "contrib/libgit-sys/libgitpub.a",
+        ])
+        .output()
+        .expect("Make failed to run");
+    if !make_output.status.success() {
+        panic!(
+            "Make failed:\n  stdout = {}\n  stderr = {}\n",
+            String::from_utf8(make_output.stdout).unwrap(),
+            String::from_utf8(make_output.stderr).unwrap()
+        );
+    }
+    std::fs::copy(crate_root.join("libgitpub.a"), dst.join("libgitpub.a"))?;
+    println!("cargo:rustc-link-search=native={}", dst.display());
+    println!("cargo:rustc-link-lib=gitpub");
+    println!("cargo:rerun-if-changed={}", git_root.display());
+
+    Ok(())
+}
diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
new file mode 100644
index 0000000000..4c153c6f0d
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -0,0 +1,23 @@
+/*
+ * Shim to publicly export Git symbols. These must be renamed so that the
+ * original symbols can be hidden. Renaming these with a "libgit_" prefix also
+ * avoids conflicts with other libraries such as libgit2.
+ */
+
+#include "git-compat-util.h"
+#include "contrib/libgit-sys/public_symbol_export.h"
+#include "version.h"
+
+#pragma GCC visibility push(default)
+
+const char *libgit_user_agent(void)
+{
+	return git_user_agent();
+}
+
+const char *libgit_user_agent_sanitized(void)
+{
+	return git_user_agent_sanitized();
+}
+
+#pragma GCC visibility pop
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
new file mode 100644
index 0000000000..a3372f93fa
--- /dev/null
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -0,0 +1,8 @@
+#ifndef PUBLIC_SYMBOL_EXPORT_H
+#define PUBLIC_SYMBOL_EXPORT_H
+
+const char *libgit_user_agent(void);
+
+const char *libgit_user_agent_sanitized(void);
+
+#endif /* PUBLIC_SYMBOL_EXPORT_H */
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
new file mode 100644
index 0000000000..d4853f3074
--- /dev/null
+++ b/contrib/libgit-sys/src/lib.rs
@@ -0,0 +1,46 @@
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::c_char;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_char = i8;
+
+extern crate libz_sys;
+
+extern "C" {
+    pub fn libgit_user_agent() -> *const c_char;
+    pub fn libgit_user_agent_sanitized() -> *const c_char;
+}
+
+#[cfg(test)]
+mod tests {
+    use std::ffi::CStr;
+
+    use super::*;
+
+    #[test]
+    fn user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent()) };
+        let agent = c_str
+            .to_str()
+            .expect("User agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+
+    #[test]
+    fn sanitized_user_agent_starts_with_git() {
+        let c_str = unsafe { CStr::from_ptr(libgit_user_agent_sanitized()) };
+        let agent = c_str
+            .to_str()
+            .expect("Sanitized user agent contains invalid UTF-8 data");
+        assert!(
+            agent.starts_with("git/"),
+            r#"Expected user agent to start with "git/", got: {}"#,
+            agent
+        );
+    }
+}
diff --git a/t/Makefile b/t/Makefile
index daa5fcae86..53ba01c21b 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -177,3 +177,13 @@ perf:
 
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
+
+.PHONY: libgit-sys-test
+libgit-sys-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-sys && \
+		cargo test \
+	)
+ifdef INCLUDE_LIBGIT_RS
+all:: libgit-sys-test
+endif
-- 
2.48.1.362.g079036d154-goog

[PATCH v9 3/4] libgit-sys: also export some config_set functions

[Josh Steadmon]

In preparation for implementing a higher-level Rust API for accessing
Git configs, export some of the upstream configset API via libgitpub and
libgit-sys. Since this will be exercised as part of the higher-level API
in the next commit, no tests have been added for libgit-sys.

While we're at it, add git_configset_alloc() and git_configset_free()
functions in libgitpub so that callers can manage config_set structs on
the heap. This also allows non-C external consumers to treat config_sets
as opaque structs.

Co-authored-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 contrib/libgit-sys/public_symbol_export.c | 36 +++++++++++++++++++++++
 contrib/libgit-sys/public_symbol_export.h | 10 +++++++
 contrib/libgit-sys/src/lib.rs             | 31 ++++++++++++++++++-
 3 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/contrib/libgit-sys/public_symbol_export.c b/contrib/libgit-sys/public_symbol_export.c
index 4c153c6f0d..dfbb257115 100644
--- a/contrib/libgit-sys/public_symbol_export.c
+++ b/contrib/libgit-sys/public_symbol_export.c
@@ -5,11 +5,47 @@
  */
 
 #include "git-compat-util.h"
+#include "config.h"
 #include "contrib/libgit-sys/public_symbol_export.h"
 #include "version.h"
 
 #pragma GCC visibility push(default)
 
+struct libgit_config_set {
+	struct config_set cs;
+};
+
+struct libgit_config_set *libgit_configset_alloc(void)
+{
+	struct libgit_config_set *cs =
+			xmalloc(sizeof(struct libgit_config_set));
+	git_configset_init(&cs->cs);
+	return cs;
+}
+
+void libgit_configset_free(struct libgit_config_set *cs)
+{
+	git_configset_clear(&cs->cs);
+	free(cs);
+}
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
+{
+	return git_configset_add_file(&cs->cs, filename);
+}
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key,
+			     int *dest)
+{
+	return git_configset_get_int(&cs->cs, key, dest);
+}
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key,
+				char **dest)
+{
+	return git_configset_get_string(&cs->cs, key, dest);
+}
+
 const char *libgit_user_agent(void)
 {
 	return git_user_agent();
diff --git a/contrib/libgit-sys/public_symbol_export.h b/contrib/libgit-sys/public_symbol_export.h
index a3372f93fa..701db92d53 100644
--- a/contrib/libgit-sys/public_symbol_export.h
+++ b/contrib/libgit-sys/public_symbol_export.h
@@ -1,6 +1,16 @@
 #ifndef PUBLIC_SYMBOL_EXPORT_H
 #define PUBLIC_SYMBOL_EXPORT_H
 
+struct libgit_config_set *libgit_configset_alloc(void);
+
+void libgit_configset_free(struct libgit_config_set *cs);
+
+int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename);
+
+int libgit_configset_get_int(struct libgit_config_set *cs, const char *key, int *dest);
+
+int libgit_configset_get_string(struct libgit_config_set *cs, const char *key, char **dest);
+
 const char *libgit_user_agent(void);
 
 const char *libgit_user_agent_sanitized(void);
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index d4853f3074..dadb4e5f40 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,15 +1,44 @@
 #[cfg(has_std__ffi__c_char)]
-use std::ffi::c_char;
+use std::ffi::{c_char, c_int};
 
 #[cfg(not(has_std__ffi__c_char))]
 #[allow(non_camel_case_types)]
 pub type c_char = i8;
 
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+pub type c_int = i32;
+
 extern crate libz_sys;
 
+#[allow(non_camel_case_types)]
+#[repr(C)]
+pub struct libgit_config_set {
+    _data: [u8; 0],
+    _marker: core::marker::PhantomData<(*mut u8, core::marker::PhantomPinned)>,
+}
+
 extern "C" {
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
+
+    pub fn libgit_configset_alloc() -> *mut libgit_config_set;
+    pub fn libgit_configset_free(cs: *mut libgit_config_set);
+
+    pub fn libgit_configset_add_file(cs: *mut libgit_config_set, filename: *const c_char) -> c_int;
+
+    pub fn libgit_configset_get_int(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        int: *mut c_int,
+    ) -> c_int;
+
+    pub fn libgit_configset_get_string(
+        cs: *mut libgit_config_set,
+        key: *const c_char,
+        dest: *mut *mut c_char,
+    ) -> c_int;
+
 }
 
 #[cfg(test)]
-- 
2.48.1.362.g079036d154-goog

[PATCH v9 4/4] libgit: add higher-level libgit crate

[Josh Steadmon]

From: Calvin Wan <calvinwan@google.com>

The C functions exported by libgit-sys do not provide an idiomatic Rust
interface. To make it easier to use these functions via Rust, add a
higher-level "libgit" crate, that wraps the lower-level configset API
with an interface that is more Rust-y.

This combination of $X and $X-sys crates is a common pattern for FFI in
Rust, as documented in "The Cargo Book" [1].

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html#-sys-packages

Co-authored-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Calvin Wan <calvinwan@google.com>
---
 .gitignore                         |   1 +
 Makefile                           |  12 ++--
 contrib/libgit-rs/Cargo.lock       |  77 +++++++++++++++++++++
 contrib/libgit-rs/Cargo.toml       |  17 +++++
 contrib/libgit-rs/README.md        |  13 ++++
 contrib/libgit-rs/build.rs         |   4 ++
 contrib/libgit-rs/src/config.rs    | 106 +++++++++++++++++++++++++++++
 contrib/libgit-rs/src/lib.rs       |   1 +
 contrib/libgit-rs/testdata/config1 |   2 +
 contrib/libgit-rs/testdata/config2 |   2 +
 contrib/libgit-rs/testdata/config3 |   2 +
 contrib/libgit-sys/src/lib.rs      |   4 ++
 t/Makefile                         |   9 ++-
 13 files changed, 242 insertions(+), 8 deletions(-)
 create mode 100644 contrib/libgit-rs/Cargo.lock
 create mode 100644 contrib/libgit-rs/Cargo.toml
 create mode 100644 contrib/libgit-rs/README.md
 create mode 100644 contrib/libgit-rs/build.rs
 create mode 100644 contrib/libgit-rs/src/config.rs
 create mode 100644 contrib/libgit-rs/src/lib.rs
 create mode 100644 contrib/libgit-rs/testdata/config1
 create mode 100644 contrib/libgit-rs/testdata/config2
 create mode 100644 contrib/libgit-rs/testdata/config3

diff --git a/.gitignore b/.gitignore
index 31d7e64287..acdd8ce7c7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -250,4 +250,5 @@ Release/
 /git.VC.db
 *.dSYM
 /contrib/buildsystems/out
+/contrib/libgit-rs/target
 /contrib/libgit-sys/target
diff --git a/Makefile b/Makefile
index a772bff186..7681ec9a3d 100644
--- a/Makefile
+++ b/Makefile
@@ -417,7 +417,7 @@ include shared.mak
 # programs in oss-fuzz/.
 #
 # Define INCLUDE_LIBGIT_RS if you want `make all` and `make test` to build and
-# test the Rust crate in contrib/libgit-sys.
+# test the Rust crates in contrib/libgit-sys and contrib/libgit-rs.
 #
 # === Optional library: libintl ===
 #
@@ -3741,7 +3741,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
 	$(MAKE) -C Documentation/ clean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
-	$(RM) -r contrib/libgit-sys/target
+	$(RM) -r contrib/libgit-sys/target contrib/libgit-rs/target
 	$(RM) contrib/libgit-sys/partial_symbol_export.o
 	$(RM) contrib/libgit-sys/hidden_symbol_export.o
 	$(RM) contrib/libgit-sys/libgitpub.a
@@ -3907,14 +3907,14 @@ build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
 unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG) t/helper/test-tool$X
 	$(MAKE) -C t/ unit-tests
 
-.PHONY: libgit-sys
-libgit-sys:
+.PHONY: libgit-sys libgit-rs
+libgit-sys libgit-rs:
 	$(QUIET)(\
-		cd contrib/libgit-sys && \
+		cd contrib/$@ && \
 		cargo build \
 	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys
+all:: libgit-sys libgit-rs
 endif
 
 LIBGIT_PUB_OBJS =
diff --git a/contrib/libgit-rs/Cargo.lock b/contrib/libgit-rs/Cargo.lock
new file mode 100644
index 0000000000..a30c7c8d33
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.lock
@@ -0,0 +1,77 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "autocfg"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
+
+[[package]]
+name = "cc"
+version = "1.1.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57b6a275aa2903740dc87da01c62040406b8812552e97129a63ea8850a17c6e6"
+dependencies = [
+ "shlex",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.158"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d8adc4bb1803a324070e64a98ae98f38934d91957a99cfb3a43dcbc01bc56439"
+
+[[package]]
+name = "libgit"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libgit-sys",
+]
+
+[[package]]
+name = "libgit-sys"
+version = "0.1.0"
+dependencies = [
+ "autocfg",
+ "libz-sys",
+ "make-cmd",
+]
+
+[[package]]
+name = "libz-sys"
+version = "1.1.20"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
+dependencies = [
+ "cc",
+ "libc",
+ "pkg-config",
+ "vcpkg",
+]
+
+[[package]]
+name = "make-cmd"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8ca8afbe8af1785e09636acb5a41e08a765f5f0340568716c18a8700ba3c0d3"
+
+[[package]]
+name = "pkg-config"
+version = "0.3.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d231b230927b5e4ad203db57bbcbee2802f6bce620b1e4a9024a07d94e2907ec"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "vcpkg"
+version = "0.2.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
diff --git a/contrib/libgit-rs/Cargo.toml b/contrib/libgit-rs/Cargo.toml
new file mode 100644
index 0000000000..c3289e69db
--- /dev/null
+++ b/contrib/libgit-rs/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "libgit"
+version = "0.1.0"
+edition = "2021"
+build = "build.rs"
+rust-version = "1.63" # TODO: Once we hit 1.84 or newer, we may want to remove Cargo.lock from
+                      # version control. See https://lore.kernel.org/git/Z47jgK-oMjFRSslr@tapette.crustytoothpaste.net/
+
+
+[lib]
+path = "src/lib.rs"
+
+[dependencies]
+libgit-sys = { version = "0.1.0", path = "../libgit-sys" }
+
+[build-dependencies]
+autocfg = "1.4.0"
diff --git a/contrib/libgit-rs/README.md b/contrib/libgit-rs/README.md
new file mode 100644
index 0000000000..ff945e1ce2
--- /dev/null
+++ b/contrib/libgit-rs/README.md
@@ -0,0 +1,13 @@
+# libgit-rs
+
+Proof-of-concept Git bindings for Rust.
+
+```toml
+[dependencies]
+libgit = "0.1.0"
+```
+
+## Rust version requirements
+
+libgit-rs should support Rust versions at least as old as the version included
+in Debian stable (currently 1.63).
diff --git a/contrib/libgit-rs/build.rs b/contrib/libgit-rs/build.rs
new file mode 100644
index 0000000000..f8bd01a690
--- /dev/null
+++ b/contrib/libgit-rs/build.rs
@@ -0,0 +1,4 @@
+pub fn main() {
+    let ac = autocfg::new();
+    ac.emit_has_path("std::ffi::c_char");
+}
diff --git a/contrib/libgit-rs/src/config.rs b/contrib/libgit-rs/src/config.rs
new file mode 100644
index 0000000000..6bf04845c8
--- /dev/null
+++ b/contrib/libgit-rs/src/config.rs
@@ -0,0 +1,106 @@
+use std::ffi::{c_void, CStr, CString};
+use std::path::Path;
+
+#[cfg(has_std__ffi__c_char)]
+use std::ffi::{c_char, c_int};
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_char = i8;
+
+#[cfg(not(has_std__ffi__c_char))]
+#[allow(non_camel_case_types)]
+type c_int = i32;
+
+use libgit_sys::*;
+
+/// A ConfigSet is an in-memory cache for config-like files such as `.gitmodules` or `.gitconfig`.
+/// It does not support all config directives; notably, it will not process `include` or
+/// `includeIf` directives (but it will store them so that callers can choose whether and how to
+/// handle them).
+pub struct ConfigSet(*mut libgit_config_set);
+impl ConfigSet {
+    /// Allocate a new ConfigSet
+    pub fn new() -> Self {
+        unsafe { ConfigSet(libgit_configset_alloc()) }
+    }
+
+    /// Load the given files into the ConfigSet; conflicting directives in later files will
+    /// override those given in earlier files.
+    pub fn add_files(&mut self, files: &[&Path]) {
+        for file in files {
+            let pstr = file.to_str().expect("Invalid UTF-8");
+            let rs = CString::new(pstr).expect("Couldn't convert to CString");
+            unsafe {
+                libgit_configset_add_file(self.0, rs.as_ptr());
+            }
+        }
+    }
+
+    /// Load the value for the given key and attempt to parse it as an i32. Dies with a fatal error
+    /// if the value cannot be parsed. Returns None if the key is not present.
+    pub fn get_int(&mut self, key: &str) -> Option<i32> {
+        let key = CString::new(key).expect("Couldn't convert to CString");
+        let mut val: c_int = 0;
+        unsafe {
+            if libgit_configset_get_int(self.0, key.as_ptr(), &mut val as *mut c_int) != 0 {
+                return None;
+            }
+        }
+
+        Some(val.into())
+    }
+
+    /// Clones the value for the given key. Dies with a fatal error if the value cannot be
+    /// converted to a String. Returns None if the key is not present.
+    pub fn get_string(&mut self, key: &str) -> Option<String> {
+        let key = CString::new(key).expect("Couldn't convert key to CString");
+        let mut val: *mut c_char = std::ptr::null_mut();
+        unsafe {
+            if libgit_configset_get_string(self.0, key.as_ptr(), &mut val as *mut *mut c_char) != 0
+            {
+                return None;
+            }
+            let borrowed_str = CStr::from_ptr(val);
+            let owned_str =
+                String::from(borrowed_str.to_str().expect("Couldn't convert val to str"));
+            free(val as *mut c_void); // Free the xstrdup()ed pointer from the C side
+            Some(owned_str)
+        }
+    }
+}
+
+impl Default for ConfigSet {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for ConfigSet {
+    fn drop(&mut self) {
+        unsafe {
+            libgit_configset_free(self.0);
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn load_configs_via_configset() {
+        let mut cs = ConfigSet::new();
+        cs.add_files(&[
+            Path::new("testdata/config1"),
+            Path::new("testdata/config2"),
+            Path::new("testdata/config3"),
+        ]);
+        // ConfigSet retrieves correct value
+        assert_eq!(cs.get_int("trace2.eventTarget"), Some(1));
+        // ConfigSet respects last config value set
+        assert_eq!(cs.get_int("trace2.eventNesting"), Some(3));
+        // ConfigSet returns None for missing key
+        assert_eq!(cs.get_string("foo.bar"), None);
+    }
+}
diff --git a/contrib/libgit-rs/src/lib.rs b/contrib/libgit-rs/src/lib.rs
new file mode 100644
index 0000000000..ef68c36943
--- /dev/null
+++ b/contrib/libgit-rs/src/lib.rs
@@ -0,0 +1 @@
+pub mod config;
diff --git a/contrib/libgit-rs/testdata/config1 b/contrib/libgit-rs/testdata/config1
new file mode 100644
index 0000000000..4e9a9d25d1
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config1
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 1
diff --git a/contrib/libgit-rs/testdata/config2 b/contrib/libgit-rs/testdata/config2
new file mode 100644
index 0000000000..b8d1eca423
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config2
@@ -0,0 +1,2 @@
+[trace2]
+	eventTarget = 1
diff --git a/contrib/libgit-rs/testdata/config3 b/contrib/libgit-rs/testdata/config3
new file mode 100644
index 0000000000..ca7b9a7c38
--- /dev/null
+++ b/contrib/libgit-rs/testdata/config3
@@ -0,0 +1,2 @@
+[trace2]
+	eventNesting = 3
diff --git a/contrib/libgit-sys/src/lib.rs b/contrib/libgit-sys/src/lib.rs
index dadb4e5f40..4bfc650450 100644
--- a/contrib/libgit-sys/src/lib.rs
+++ b/contrib/libgit-sys/src/lib.rs
@@ -1,3 +1,5 @@
+use std::ffi::c_void;
+
 #[cfg(has_std__ffi__c_char)]
 use std::ffi::{c_char, c_int};
 
@@ -19,6 +21,8 @@ pub struct libgit_config_set {
 }
 
 extern "C" {
+    pub fn free(ptr: *mut c_void);
+
     pub fn libgit_user_agent() -> *const c_char;
     pub fn libgit_user_agent_sanitized() -> *const c_char;
 
diff --git a/t/Makefile b/t/Makefile
index 53ba01c21b..2994eb5fa9 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -178,12 +178,17 @@ perf:
 .PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
 	check-chainlint clean-chainlint test-chainlint $(UNIT_TESTS)
 
-.PHONY: libgit-sys-test
+.PHONY: libgit-sys-test libgit-rs-test
 libgit-sys-test:
 	$(QUIET)(\
 		cd ../contrib/libgit-sys && \
 		cargo test \
 	)
+libgit-rs-test:
+	$(QUIET)(\
+		cd ../contrib/libgit-rs && \
+		cargo test \
+	)
 ifdef INCLUDE_LIBGIT_RS
-all:: libgit-sys-test
+all:: libgit-sys-test libgit-rs-test
 endif
-- 
2.48.1.362.g079036d154-goog

Re: [PATCH v9 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[phillip.wood123]

Hi Josh

The range-diff looks good. It's really nice to see this being added to git.

Best Wishes

Phillip

On 29/01/2025 21:50, Josh Steadmon wrote:
> This series provides two small Rust wrapper libraries around parts of
> Git: "libgit-sys", which exposes a few functions from libgit.a, and
> "libgit", which provides a more Rust-friendly interface to some of those
> functions. In addition to included unit tests, at $DAYJOB we have tested
> building JJ[1] with our library and used it to replace some of the
> libgit2-rs uses.
> 
> [1] https://github.com/jj-vcs/jj
> 
> There is known NEEDSWORK, but I feel that they can be addressed in
> follow-up changes, rather than in this series. If you feel otherwise,
> please let me know:
> 
> * Investigate alternative methods of managing symbol visibility &
>    renaming.
> 
> * Figure out symbol versioning
> 
> Changes in V9:
> * Properly initialize Makefile var LIBGIT_PUB_OBJS.
> 
> * Pass the correct pointer to free in libgit_configset_free().
> 
> Changes in V8:
> * Define a private libgit_config_set struct to avoid excessive casting
>    in public_symbol_export.c.
> 
> * Style fixes: merge some Makefile rules, limit rule line length by
>    defining intermediate variables, add initial empty comment line, add
>    linebreaks in function definitions.
> 
> Changes in V7:
> * Moved the ConfigSet implementation in libgit-rs to a `config` module.
> 
> * Added doc comments for ConfigSet and its methods.
> 
> * Fix meson builds by adding new object files to `libgit_sources`
> 
> * Moved INCLUDE_LIBGIT_RS Makefile changes earlier in the series, so
>    that we can make it optional to compile some of the libgitpub sources.
>    Squashed V6 patch 5/5 into this series' patch 4/4.
> 
> * Don't publicly export FFI types in libgit-rs.
> 
> * Removed extraneous `-r` argument to $(RM) in the clean rules.
> 
> * Added TODO reminder in Cargo.toml about removing Cargo.lock once we
>    hit a certain minimum supported Rust version.
> 
> * Style cleanup in public_symbol_export.c
> 
> Changes in V6:
> * Rebased onto current master, since V5 was several months old.
> 
> * Move libgit-sys out of libgit-rs; while this sort of nesting is common
>    in Rust crates with standalone repositories, it doesn't make as much
>    sense when they're contained in the larger Git project's repo.
> 
> * Standardize the naming of some of the Makefile targets to always
>    include a dash in the "-rs" or "-sys" suffixes.
> 
> * Clean up READMEs and crate descriptions in preparation for
>    uploading to crates.io.
> 
> Changes in V5:
> * When building with INCLUDE_LIBGIT_RS defined, add
>    "-fvisibility=hidden" to CFLAGS. This allows us to manage symbol
>    visibility in libgitpub.a without causing `make all` to rebuild from
>    scratch due to changing CFLAGS.
> 
> * Avoid using c_int in the higher-level Rust API.
> 
> * Remove libgitpub.a and intermediate files with `make clean`.
> 
> Changes in V4:
> * Drop V3 patch #3, which added wrappers around repository
>    initialization and config access. These are not well-libified, and
>    they are not necessary for JJ's proof-of-concept use case, so let's
>    avoid exporting them for now.
> 
> * Set a minimum supported Rust version of 1.63. Autodetect whether our
>    Rust version has c_int and c_char types; if not, define them
>    ourselves.
> 
> * When building libgitpub.a via build.rs, set DEVELOPER=1 to catch
>    additional errors at build time.
> 
> * In build.rs, use the make_cmd crate to portable select the correct
>    invocation of GNU Make.
> 
> * Follow naming standards for _alloc() and _free() functions.
> 
> * Use String instead of CString in higher-level API.
> 
> * Move libgit_configset_alloc() and libgit_configset_free() out of
>    upstream Git, to the libgitpub shim library.
> 
> * In libgitpub, initialize libgit_config_set structs in the _alloc()
>    function rather than with a separate _init() function.
> 
> * Remove unnecessary comments in libgit-sys showing where the wrapped
>    functions were originally defined.
> 
> * Fix clippy lint: don't reborrow configfile path references.
> 
> * Various typo fixes and `cargo fmt` fixes.
> 
> Changes in V3:
> * Renamed cgit-rs to libgit-rs and cgit-sys to libgit-sys
> 
> * Makefile cleanup, particularly adding config.mak options that
>    developers can set to run Rust builds and tests by default (Patch 6)
> 
> * Provide testdata configs for unit tests
> 
> * ConfigSet API now uses &Path instead of &str -- more ergonomic for
>    Rust users to pass in and errors out if the path string isn't UTF-8
> 
> * Fixed unresolved dependency on libz in Cargo.toml
> 
> 
> Calvin Wan (1):
>    libgit: add higher-level libgit crate
> 
> Josh Steadmon (3):
>    common-main: split init and exit code into new files
>    libgit-sys: introduce Rust wrapper for libgit.a
>    libgit-sys: also export some config_set functions
> 
>   .gitignore                                |   2 +
>   Makefile                                  |  50 ++++++++++
>   common-exit.c                             |  26 ++++++
>   common-init.c                             |  63 +++++++++++++
>   common-init.h                             |   6 ++
>   common-main.c                             |  83 +----------------
>   contrib/libgit-rs/Cargo.lock              |  77 ++++++++++++++++
>   contrib/libgit-rs/Cargo.toml              |  17 ++++
>   contrib/libgit-rs/README.md               |  13 +++
>   contrib/libgit-rs/build.rs                |   4 +
>   contrib/libgit-rs/src/config.rs           | 106 ++++++++++++++++++++++
>   contrib/libgit-rs/src/lib.rs              |   1 +
>   contrib/libgit-rs/testdata/config1        |   2 +
>   contrib/libgit-rs/testdata/config2        |   2 +
>   contrib/libgit-rs/testdata/config3        |   2 +
>   contrib/libgit-sys/Cargo.lock             |  69 ++++++++++++++
>   contrib/libgit-sys/Cargo.toml             |  19 ++++
>   contrib/libgit-sys/README.md              |   4 +
>   contrib/libgit-sys/build.rs               |  35 +++++++
>   contrib/libgit-sys/public_symbol_export.c |  59 ++++++++++++
>   contrib/libgit-sys/public_symbol_export.h |  18 ++++
>   contrib/libgit-sys/src/lib.rs             |  79 ++++++++++++++++
>   meson.build                               |   2 +
>   t/Makefile                                |  15 +++
>   24 files changed, 673 insertions(+), 81 deletions(-)
>   create mode 100644 common-exit.c
>   create mode 100644 common-init.c
>   create mode 100644 common-init.h
>   create mode 100644 contrib/libgit-rs/Cargo.lock
>   create mode 100644 contrib/libgit-rs/Cargo.toml
>   create mode 100644 contrib/libgit-rs/README.md
>   create mode 100644 contrib/libgit-rs/build.rs
>   create mode 100644 contrib/libgit-rs/src/config.rs
>   create mode 100644 contrib/libgit-rs/src/lib.rs
>   create mode 100644 contrib/libgit-rs/testdata/config1
>   create mode 100644 contrib/libgit-rs/testdata/config2
>   create mode 100644 contrib/libgit-rs/testdata/config3
>   create mode 100644 contrib/libgit-sys/Cargo.lock
>   create mode 100644 contrib/libgit-sys/Cargo.toml
>   create mode 100644 contrib/libgit-sys/README.md
>   create mode 100644 contrib/libgit-sys/build.rs
>   create mode 100644 contrib/libgit-sys/public_symbol_export.c
>   create mode 100644 contrib/libgit-sys/public_symbol_export.h
>   create mode 100644 contrib/libgit-sys/src/lib.rs
> 
> Range-diff against v8:
> 1:  cd0cb9aa04 = 1:  cd0cb9aa04 common-main: split init and exit code into new files
> 2:  3588a3c3fc ! 2:  8793ff64a7 libgit-sys: introduce Rust wrapper for libgit.a
>      @@ Makefile: $(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GIT
>       +all:: libgit-sys
>       +endif
>       +
>      -+LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
>      ++LIBGIT_PUB_OBJS =
>      ++LIBGIT_PUB_OBJS += contrib/libgit-sys/public_symbol_export.o
>       +LIBGIT_PUB_OBJS += libgit.a
>       +LIBGIT_PUB_OBJS += reftable/libreftable.a
>       +LIBGIT_PUB_OBJS += xdiff/lib.a
> 3:  f4452fffe6 ! 3:  ab32bd1d07 libgit-sys: also export some config_set functions
>      @@ contrib/libgit-sys/public_symbol_export.c
>       +void libgit_configset_free(struct libgit_config_set *cs)
>       +{
>       +	git_configset_clear(&cs->cs);
>      -+	free(&cs->cs);
>      ++	free(cs);
>       +}
>       +
>       +int libgit_configset_add_file(struct libgit_config_set *cs, const char *filename)
> 4:  ada9fc0a13 ! 4:  1bf8c5392c libgit: add higher-level libgit crate
>      @@ Makefile: build-unit-tests: $(UNIT_TEST_PROGS) $(CLAR_TEST_PROG)
>       +all:: libgit-sys libgit-rs
>        endif
>        
>      - LIBGIT_PUB_OBJS = contrib/libgit-sys/public_symbol_export.o
>      + LIBGIT_PUB_OBJS =
>       
>        ## contrib/libgit-rs/Cargo.lock (new) ##
>       @@
> 
> base-commit: 757161efcca150a9a96b312d9e780a071e601a03

Re: [PATCH v9 0/4] Introduce libgit-rs, a Rust wrapper around libgit.a

[Junio C Hamano]

phillip.wood123@gmail.com writes:

> Hi Josh
>
> The range-diff looks good. It's really nice to see this being added to git.
>
> Best Wishes
>
> Phillip

Thanks.  Let's plan to mark the topic for 'next' unless there are
other comments.