Re: [PATCH v3 01/10] doc: define unambiguous type mappings across C and Rust

[Junio C Hamano <gitster@pobox.com>]

"Ezekiel Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +== Character types
> +
> +This is where C and Rust don't have a clean one-to-one mapping.
> +
> +C comparison problem: While the sign of `char` is implementation defined, it's
> +also signless (neither signed nor unsigned). When building with
> +`make DEVELOPER=1` it will complain about a "differ in signedness" when `char`
> +is compared with `uint8_t` or `int8_t`.
> +
> +Rust's `char` type is an unsigned 32-bit integer that is used to describe
> +Unicode code points. Even though a C `char` is the same width as `u8`, `char`
> +should be converted to u8 where it is describing bytes in memory. If a C
> +`char` is not describing bytes, then it should be converted to a more accurate
> +unambiguous type. The reason for mentioning Unicode here is because of how &str
> +is defined in Rust and how to create a &str from &[u8]. Rust assumes that &str
> +is a correctly encoded utf-8 string, i.e. text in memory. Where as a C `char`
> +makes no assumption about the bytes that it is representing.

Even though you write excuses for bringing up Unicode here, I am
afraid that most of the above is irrelevant tangent that makes the
point of this documentation muddier.  Anybody who is involved in
this effort would at least know that C's char is not about
representing Unicode codepoints (it is way too narrow for that),
while Rust's char type exactly is, and I do not see much point in
making such an apples-and-oranges comparison to spend extra words
here.

Another thing I found confusing is your mention of &[u8] vs &str.
Surely, Rust will have trouble if an array of u8 we FFI an array of
bytes we have on the C side, if the byte sequence were a broken
UTF-8.  But that would not be fixed if you only rewrote C code to
use `uint8_t[]` where it originally used `char[]`, would it?  If we
have on C-side char[] that has iso8859-1 in it, we still would want
to use uint8_t[] when we smuggle the result of passing it to iconv()
to translate that into UTF-8 into Rust.  Or we may pass such an
iso8859-1 encoded string directly as an uint8_t[] byte array to Rust
and let Rust side run an equivalent of iconv() to obtain char array.

The point is that "your byte sequence has to be valid UTF-8" does
not fit well in the narrative here.  If we want to move/interface
the handling of "encoding" header in commit objects with code
written in Rust, this starts to matter.

So even if it is technically correct, it is another irrelevant
tangent when we discuss why we want to use uint8_t on the C side to
help cbindgen/bindgen to map it to u8 on Rust side.

Wouldn't just directly going into

    If a piece of C code uses `char` to represent a byte, it makes
    it easier to interface with Rust to rewrite it to use uint8_t
    and let cbindgen/bindgen map it to u8 on the Rust side.

be clearer, would it?  We never deal with a single Unicode codepoint
or an array of them (we do deal with utf8 encoded array of bytes,
though) on the C side, and I do not think it is likely to change, so
there is nothing lost if we did not talk about how `char` in Rust
behaves at all.

And of course, not talking about `char` in Rust does not mean that
we need a rule like "if you want to interface with C, never use
`char` on the Rust side".  `char` may have its uses on Rust side,
just like `char` may have its uses on C side.

Also I do not quite get your precondition "If a C `char` is not
describing bytes".  What `char` in C on modern platforms would
describe something _other_ _than_ bytes?  Even the way things like
varint use `char` is exactly for accessing individual bytes.  Even
when it is used as a space-saver in a structure member whose value
would never exceed 100, i.e., a small integer, we would know and be
implicitly relying on the fact that the member is a byte-wide.

Thanks.