[PATCH v8 0/2] Add dma coherent allocator abstraction

[PATCH v8 0/2] Add dma coherent allocator abstraction

[Abdiel Janulgue]

Changes since v7:
- Remove cpu_buf() and cpu_buf_mut() as exporting a r/w interface via
  a slice is undefined behaviour due to slice's requirement that the
  underlying pointer should not be modified (Alice Ryhl, Robin Murphy).
- Reintroduce r/w helpers instead which includes proper safety
  invariants (Daniel Almeida).
- Link to v7: https://lore.kernel.org/lkml/20241210221603.3174929-1-abdiel.janulgue@gmail.com/

Changes since v6:
- Include the dma_attrs in the constructor, use alloc::Flags as inpiration

Changes since v5:
- Remove unnecessary lifetime annotation when returning the CPU buffer.

Changes since v4:
- Documentation and example fixes, use Markdown formatting (Miguel Ojeda).
- Discard read()/write() helpers to remove bound on Copy and fix overhead
  (Daniel Almeida).
- Improve error-handling in the constructor block (Andreas Hindborg).

Changes since v3:
- Reject ZST types by checking the type size in the constructor in
  addition to requiring FromBytes/AsBytes traits for the type (Alice Ryhl).

Changes since v2:
- Fixed missing header for generating the bindings.

Changes since v1:
- Fix missing info in commit log where EOVERFLOW is used.
- Restrict the dma coherent allocator to numeric types for now for valid
  behaviour (Daniel Almeida).
- Build slice dynamically.

Abdiel Janulgue (2):
  rust: error: Add EOVERFLOW
  rust: add dma coherent allocator abstraction.

 rust/bindings/bindings_helper.h |   1 +
 rust/kernel/dma.rs              | 271 ++++++++++++++++++++++++++++++++
 rust/kernel/error.rs            |   1 +
 rust/kernel/lib.rs              |   1 +
 4 files changed, 274 insertions(+)
 create mode 100644 rust/kernel/dma.rs


base-commit: 0c5928deada15a8d075516e6e0d9ee19011bb000
-- 
2.43.0

[PATCH v8 1/2] rust: error: Add EOVERFLOW

[Abdiel Janulgue]

Trivial addition for missing EOVERFLOW error. This is used by a
subsequent patch that might require returning EOVERFLOW as a result
of `checked_mul`.

Signed-off-by: Abdiel Janulgue <abdiel.janulgue@gmail.com>
---
 rust/kernel/error.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs
index 914e8dec1abd..eb06be2ca9c5 100644
--- a/rust/kernel/error.rs
+++ b/rust/kernel/error.rs
@@ -63,6 +63,7 @@ macro_rules! declare_err {
     declare_err!(EPIPE, "Broken pipe.");
     declare_err!(EDOM, "Math argument out of domain of func.");
     declare_err!(ERANGE, "Math result not representable.");
+    declare_err!(EOVERFLOW, "Value too large for defined data type.");
     declare_err!(ERESTARTSYS, "Restart the system call.");
     declare_err!(ERESTARTNOINTR, "System call was interrupted by a signal and will be restarted.");
     declare_err!(ERESTARTNOHAND, "Restart if no handler.");
-- 
2.43.0

[PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Abdiel Janulgue]

Add a simple dma coherent allocator rust abstraction. Based on
Andreas Hindborg's dma abstractions from the rnvme driver, which
was also based on earlier work by Wedson Almeida Filho.

Signed-off-by: Abdiel Janulgue <abdiel.janulgue@gmail.com>
---
 rust/bindings/bindings_helper.h |   1 +
 rust/kernel/dma.rs              | 271 ++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs              |   1 +
 3 files changed, 273 insertions(+)
 create mode 100644 rust/kernel/dma.rs

diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h
index 5c4dfe22f41a..49bf713b9bb6 100644
--- a/rust/bindings/bindings_helper.h
+++ b/rust/bindings/bindings_helper.h
@@ -11,6 +11,7 @@
 #include <linux/blk_types.h>
 #include <linux/blkdev.h>
 #include <linux/cred.h>
+#include <linux/dma-mapping.h>
 #include <linux/errname.h>
 #include <linux/ethtool.h>
 #include <linux/file.h>
diff --git a/rust/kernel/dma.rs b/rust/kernel/dma.rs
new file mode 100644
index 000000000000..c0661bcc9f72
--- /dev/null
+++ b/rust/kernel/dma.rs
@@ -0,0 +1,271 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Direct memory access (DMA).
+//!
+//! C header: [`include/linux/dma-mapping.h`](srctree/include/linux/dma-mapping.h)
+
+use crate::{
+    bindings, build_assert,
+    device::Device,
+    error::code::*,
+    error::Result,
+    transmute::{AsBytes, FromBytes},
+    types::ARef,
+};
+
+/// Possible attributes associated with a DMA mapping.
+///
+/// They can be combined with the operators `|`, `&`, and `!`.
+///
+/// Values can be used from the [`attrs`] module.
+#[derive(Clone, Copy, PartialEq)]
+pub struct Attrs(pub u32);
+
+impl Attrs {
+    /// Get the raw representation of this attribute.
+    pub(crate) fn as_raw(self) -> usize {
+        self.0.try_into().unwrap()
+    }
+
+    /// Check whether `flags` is contained in `self`.
+    pub fn contains(self, flags: Attrs) -> bool {
+        (self & flags) == flags
+    }
+}
+
+impl core::ops::BitOr for Attrs {
+    type Output = Self;
+    fn bitor(self, rhs: Self) -> Self::Output {
+        Self(self.0 | rhs.0)
+    }
+}
+
+impl core::ops::BitAnd for Attrs {
+    type Output = Self;
+    fn bitand(self, rhs: Self) -> Self::Output {
+        Self(self.0 & rhs.0)
+    }
+}
+
+impl core::ops::Not for Attrs {
+    type Output = Self;
+    fn not(self) -> Self::Output {
+        Self(!self.0)
+    }
+}
+
+/// DMA mapping attrributes.
+pub mod attrs {
+    use super::Attrs;
+
+    /// Specifies that reads and writes to the mapping may be weakly ordered, that is that reads
+    /// and writes may pass each other.
+    pub const DMA_ATTR_WEAK_ORDERING: Attrs = Attrs(bindings::DMA_ATTR_WEAK_ORDERING);
+
+    /// Specifies that writes to the mapping may be buffered to improve performance.
+    pub const DMA_ATTR_WRITE_COMBINE: Attrs = Attrs(bindings::DMA_ATTR_WRITE_COMBINE);
+
+    /// Lets the platform to avoid creating a kernel virtual mapping for the allocated buffer.
+    pub const DMA_ATTR_NO_KERNEL_MAPPING: Attrs = Attrs(bindings::DMA_ATTR_NO_KERNEL_MAPPING);
+
+    /// Allows platform code to skip synchronization of the CPU cache for the given buffer assuming
+    /// that it has been already transferred to 'device' domain.
+    pub const DMA_ATTR_SKIP_CPU_SYNC: Attrs = Attrs(bindings::DMA_ATTR_SKIP_CPU_SYNC);
+
+    /// Forces contiguous allocation of the buffer in physical memory.
+    pub const DMA_ATTR_FORCE_CONTIGUOUS: Attrs = Attrs(bindings::DMA_ATTR_FORCE_CONTIGUOUS);
+
+    /// This is a hint to the DMA-mapping subsystem that it's probably not worth the time to try
+    /// to allocate memory to in a way that gives better TLB efficiency.
+    pub const DMA_ATTR_ALLOC_SINGLE_PAGES: Attrs = Attrs(bindings::DMA_ATTR_ALLOC_SINGLE_PAGES);
+
+    /// This tells the DMA-mapping subsystem to suppress allocation failure reports (similarly to
+    /// __GFP_NOWARN).
+    pub const DMA_ATTR_NO_WARN: Attrs = Attrs(bindings::DMA_ATTR_NO_WARN);
+
+    /// Used to indicate that the buffer is fully accessible at an elevated privilege level (and
+    /// ideally inaccessible or at least read-only at lesser-privileged levels).
+    pub const DMA_ATTR_PRIVILEGED: Attrs = Attrs(bindings::DMA_ATTR_PRIVILEGED);
+}
+
+/// An abstraction of the `dma_alloc_coherent` API.
+///
+/// This is an abstraction around the `dma_alloc_coherent` API which is used to allocate and map
+/// large consistent DMA regions.
+///
+/// A [`CoherentAllocation`] instance contains a pointer to the allocated region (in the
+/// processor's virtual address space) and the device address which can be given to the device
+/// as the DMA address base of the region. The region is released once [`CoherentAllocation`]
+/// is dropped.
+///
+/// # Invariants
+///
+/// For the lifetime of an instance of [`CoherentAllocation`], the cpu address is a valid pointer
+/// to an allocated region of consistent memory and we hold a reference to the device.
+pub struct CoherentAllocation<T: AsBytes + FromBytes> {
+    dev: ARef<Device>,
+    dma_handle: bindings::dma_addr_t,
+    count: usize,
+    cpu_addr: *mut T,
+    dma_attrs: Attrs,
+}
+
+impl<T: AsBytes + FromBytes> CoherentAllocation<T> {
+    /// Allocates a region of `size_of::<T> * count` of consistent memory.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::device::Device;
+    /// use kernel::dma::{attrs::*, CoherentAllocation};
+    ///
+    /// # fn test(dev: &Device) -> Result {
+    /// let c: CoherentAllocation<u64> = CoherentAllocation::alloc_attrs(dev, 4, GFP_KERNEL,
+    ///                                                                  DMA_ATTR_NO_WARN)?;
+    /// # Ok::<(), Error>(()) }
+    /// ```
+    pub fn alloc_attrs(
+        dev: &Device,
+        count: usize,
+        gfp_flags: kernel::alloc::Flags,
+        dma_attrs: Attrs,
+    ) -> Result<CoherentAllocation<T>> {
+        build_assert!(
+            core::mem::size_of::<T>() > 0,
+            "It doesn't make sense for the allocated type to be a ZST"
+        );
+
+        let size = count
+            .checked_mul(core::mem::size_of::<T>())
+            .ok_or(EOVERFLOW)?;
+        let mut dma_handle = 0;
+        // SAFETY: device pointer is guaranteed as valid by invariant on `Device`.
+        // We ensure that we catch the failure on this function and throw an ENOMEM
+        let ret = unsafe {
+            bindings::dma_alloc_attrs(
+                dev.as_raw(),
+                size,
+                &mut dma_handle,
+                gfp_flags.as_raw(),
+                dma_attrs.as_raw(),
+            )
+        };
+        if ret.is_null() {
+            return Err(ENOMEM);
+        }
+        // INVARIANT: We just successfully allocated a coherent region which is accessible for
+        // `count` elements, hence the cpu address is valid. We also hold a refcounted reference
+        // to the device.
+        Ok(Self {
+            dev: dev.into(),
+            dma_handle,
+            count,
+            cpu_addr: ret as *mut T,
+            dma_attrs,
+        })
+    }
+
+    /// Performs the same functionality as `alloc_attrs`, except the `dma_attrs` is 0 by default.
+    pub fn alloc_coherent(
+        dev: &Device,
+        count: usize,
+        gfp_flags: kernel::alloc::Flags,
+    ) -> Result<CoherentAllocation<T>> {
+        CoherentAllocation::alloc_attrs(dev, count, gfp_flags, Attrs(0))
+    }
+
+    /// Returns the base address, dma handle, attributes and the size of the allocated region.
+    /// The caller takes ownership of the returned resources, i.e., will have the responsibility
+    /// in calling `bindings::dma_free_attrs`.
+    pub fn into_parts(self) -> (*mut T, bindings::dma_addr_t, usize, usize) {
+        let size = self.count * core::mem::size_of::<T>();
+        let ret = (
+            self.cpu_addr,
+            self.dma_handle,
+            self.dma_attrs.as_raw(),
+            size,
+        );
+        // Drop the device's reference count associated with this object. This is needed as no
+        // destructor will be called on this object once this function returns.
+        // SAFETY: the device pointer is still valid as of this point due to the type invariants
+        // on `CoherentAllocation`.
+        unsafe { bindings::put_device(self.dev.as_raw()) }
+        core::mem::forget(self);
+        ret
+    }
+
+    /// Returns the base address to the allocated region in the CPU's virtual address space.
+    pub fn start_ptr(&self) -> *const T {
+        self.cpu_addr
+    }
+
+    /// Returns the base address to the allocated region in the CPU's virtual address space as
+    /// a mutable pointer.
+    pub fn start_ptr_mut(&mut self) -> *mut T {
+        self.cpu_addr
+    }
+
+    /// Returns a DMA handle which may given to the device as the DMA address base of
+    /// the region.
+    pub fn dma_handle(&self) -> bindings::dma_addr_t {
+        self.dma_handle
+    }
+
+    /// Reads data from the region starting from `offset` as a slice.
+    /// `offset` and `count` are in units of `T`, not the number of bytes.
+    ///
+    /// Due to the safety requirements of slice, the data returned should be regarded by the
+    /// caller as a snapshot of the region when this function is called, as the region could
+    /// be modified by the device at anytime. For ringbuffer type of r/w access or use-cases
+    /// where the pointer to the live data is needed, `start_ptr()` or `start_ptr_mut()`
+    /// could be used instead.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that no hardware operations that involve the buffer are currently
+    /// taking place while the returned slice is live.
+    pub unsafe fn read(&self, offset: usize, count: usize) -> Result<&[T]> {
+        if offset + count >= self.count {
+            return Err(EINVAL);
+        }
+        // SAFETY: The pointer is valid due to type invariant on `CoherentAllocation`,
+        // we've just checked that the range and index is within bounds. The immutability of the
+        // of data is also guaranteed by the safety requirements of the function.
+        Ok(unsafe { core::slice::from_raw_parts(self.cpu_addr.wrapping_add(offset), count) })
+    }
+
+    /// Writes data to the region starting from `offset`. `offset` is in units of `T`, not the
+    /// number of bytes.
+    pub fn write(&self, src: &[T], offset: usize) -> Result {
+        if offset + src.len() >= self.count {
+            return Err(EINVAL);
+        }
+        // SAFETY: The pointer is valid due to type invariant on `CoherentAllocation`
+        // and we've just checked that the range and index is within bounds.
+        unsafe {
+            core::ptr::copy_nonoverlapping(
+                src.as_ptr(),
+                self.cpu_addr.wrapping_add(offset),
+                src.len(),
+            )
+        };
+        Ok(())
+    }
+}
+
+impl<T: AsBytes + FromBytes> Drop for CoherentAllocation<T> {
+    fn drop(&mut self) {
+        let size = self.count * core::mem::size_of::<T>();
+        // SAFETY: the device, cpu address, and the dma handle is valid due to the
+        // type invariants on `CoherentAllocation`.
+        unsafe {
+            bindings::dma_free_attrs(
+                self.dev.as_raw(),
+                size,
+                self.cpu_addr as _,
+                self.dma_handle,
+                self.dma_attrs.as_raw(),
+            )
+        }
+    }
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index e1065a7551a3..6e90ebf5a130 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -35,6 +35,7 @@
 mod build_assert;
 pub mod cred;
 pub mod device;
+pub mod dma;
 pub mod error;
 #[cfg(CONFIG_RUST_FW_LOADER_ABSTRACTIONS)]
 pub mod firmware;
-- 
2.43.0

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

No rust code in kernel/dma, please.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Miguel Ojeda]

On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
>
> No rust code in kernel/dma, please.

What do you suggest?

Thanks!

Cheers,
Miguel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Wed, Jan 08, 2025 at 04:16:18PM +0100, Miguel Ojeda wrote:
> On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
> >
> > No rust code in kernel/dma, please.
> 
> What do you suggest?

Keep the wrappers in your code instead of making life painful for
others.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Wed, Jan 08, 2025 at 04:18:58PM +0100, Christoph Hellwig wrote:
> On Wed, Jan 08, 2025 at 04:16:18PM +0100, Miguel Ojeda wrote:
> > On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > No rust code in kernel/dma, please.
> > 
> > What do you suggest?
> 
> Keep the wrappers in your code instead of making life painful for
> others.
> 

What does "your code" mean? Duplicated in every driver? Otherwise
rust/kernel/dma.rs seems reasonable, no?

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> What does "your code" mean? Duplicated in every driver?

Yes, interfaces to the DMA API should stay in readable C code and not
in weird bindings so that it reminds greppable and maintainable.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > What does "your code" mean? Duplicated in every driver?
> 
> Yes, interfaces to the DMA API should stay in readable C code and not
> in weird bindings so that it reminds greppable and maintainable.
> 

Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
corresponding C API.

One reason for that is that some requirements C APIs naturally have, can be
abstracted in a way, that the Rust compiler already ensures certain things and
hence drivers have less potential to produce errors.

(I understand this is very abstract and we can go into details and examples, if
you like.)

So, in the end, Rust drivers would just end up with each of them including a
copy of those abstractions, rather than using the C APIs from all over the place
within the driver.

This wouldn't help with what you ask for, it would just duplicate the "problem".

If you don't feel comfortable maintaining the Rust abstraction (or just don't
want to), that's fine. I don't think anyone expects you to do that, we can take
care of that instead.

From your perspective, I think you can just think of the Rust abstraction as a
single driver calling into the DMA API.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
> On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> > On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > > What does "your code" mean? Duplicated in every driver?
> > 
> > Yes, interfaces to the DMA API should stay in readable C code and not
> > in weird bindings so that it reminds greppable and maintainable.
> > 
> 
> Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
> corresponding C API.

Don't force me to deal with your shiny language of the day.  Maintaining
multi-language projects is a pain I have no interest in dealing with.
If you want to use something that's not C, be that assembly or rust you
write to C interfaces and deal with the impedence mismatch yourself as
far as I'm concerned.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Fri, Jan 10, 2025 at 09:39:55AM +0100, Christoph Hellwig wrote:
> On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
> > On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> > > On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > > > What does "your code" mean? Duplicated in every driver?
> > > 
> > > Yes, interfaces to the DMA API should stay in readable C code and not
> > > in weird bindings so that it reminds greppable and maintainable.
> > > 
> > 
> > Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
> > corresponding C API.
> 
> Don't force me to deal with your shiny language of the day.

Again, no one asks you to deal with or maintain this piece of Rust code.

> Maintaining
> multi-language projects is a pain I have no interest in dealing with.
> If you want to use something that's not C, be that assembly or rust you
> write to C interfaces and deal with the impedence mismatch yourself as
> far as I'm concerned.
> 

This is exactly what we're doing and proposing here, isn't it?

We wrote a single piece of Rust code that abstracts the C API for all Rust
drivers, which we offer to maintain ourselves.

What else are you asking for?

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Fri, Jan 10, 2025 at 11:41:54AM +0100, Danilo Krummrich wrote:
> On Fri, Jan 10, 2025 at 09:39:55AM +0100, Christoph Hellwig wrote:
> > On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
> > > On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> > > > On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > > > > What does "your code" mean? Duplicated in every driver?
> > > > 
> > > > Yes, interfaces to the DMA API should stay in readable C code and not
> > > > in weird bindings so that it reminds greppable and maintainable.
> > > > 
> > > 
> > > Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
> > > corresponding C API.
> > 
> > Don't force me to deal with your shiny language of the day.
> 
> Again, no one asks you to deal with or maintain this piece of Rust code.
> 
> > Maintaining
> > multi-language projects is a pain I have no interest in dealing with.
> > If you want to use something that's not C, be that assembly or rust you
> > write to C interfaces and deal with the impedence mismatch yourself as
> > far as I'm concerned.
> > 
> 
> This is exactly what we're doing and proposing here, isn't it?
> 
> We wrote a single piece of Rust code that abstracts the C API for all Rust
> drivers, which we offer to maintain ourselves.
> 
> What else are you asking for?

Since there hasn't been a reply so far, I assume that we're good with
maintaining the DMA Rust abstractions separately.

Hence, the next version of this patch series will have the corresponding
maintainer entry.

- Danilo

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Robin Murphy]

On 2025-01-16 1:17 pm, Danilo Krummrich wrote:
> On Fri, Jan 10, 2025 at 11:41:54AM +0100, Danilo Krummrich wrote:
>> On Fri, Jan 10, 2025 at 09:39:55AM +0100, Christoph Hellwig wrote:
>>> On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
>>>> On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
>>>>> On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
>>>>>> What does "your code" mean? Duplicated in every driver?
>>>>>
>>>>> Yes, interfaces to the DMA API should stay in readable C code and not
>>>>> in weird bindings so that it reminds greppable and maintainable.
>>>>>
>>>>
>>>> Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
>>>> corresponding C API.
>>>
>>> Don't force me to deal with your shiny language of the day.
>>
>> Again, no one asks you to deal with or maintain this piece of Rust code.
>>
>>> Maintaining
>>> multi-language projects is a pain I have no interest in dealing with.
>>> If you want to use something that's not C, be that assembly or rust you
>>> write to C interfaces and deal with the impedence mismatch yourself as
>>> far as I'm concerned.
>>>
>>
>> This is exactly what we're doing and proposing here, isn't it?
>>
>> We wrote a single piece of Rust code that abstracts the C API for all Rust
>> drivers, which we offer to maintain ourselves.
>>
>> What else are you asking for?
> 
> Since there hasn't been a reply so far, I assume that we're good with
> maintaining the DMA Rust abstractions separately.

Indeed, FWIW it seems like the appropriate level of abstraction to me, 
judging by the other wrappers living in rust/kernel/ already. As far as 
the interaction with C code goes, it appears to be a pretty 
straightforward midlayer consumer of the DMA API much like others we 
already have (e.g. videobuf2-dma-*), just one which happens to be a 
language binding rather than some other kind of functional abstraction.

There is a realistic chance that the C API will evolve in ways which 
break the binding, but as long as a) that won't break non-Rust builds, 
and b) Rust folks are happy to take responsibility for un-breaking the 
Rust build if and when it happens, then that seems reasonable IMO.

Thanks,
Robin.

> 
> Hence, the next version of this patch series will have the corresponding
> maintainer entry.
> 
> - Danilo

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Thu, Jan 16, 2025 at 01:57:50PM +0000, Robin Murphy wrote:
> On 2025-01-16 1:17 pm, Danilo Krummrich wrote:
> > On Fri, Jan 10, 2025 at 11:41:54AM +0100, Danilo Krummrich wrote:
> > > On Fri, Jan 10, 2025 at 09:39:55AM +0100, Christoph Hellwig wrote:
> > > > On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
> > > > > On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> > > > > > On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > > > > > > What does "your code" mean? Duplicated in every driver?
> > > > > > 
> > > > > > Yes, interfaces to the DMA API should stay in readable C code and not
> > > > > > in weird bindings so that it reminds greppable and maintainable.
> > > > > > 
> > > > > 
> > > > > Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
> > > > > corresponding C API.
> > > > 
> > > > Don't force me to deal with your shiny language of the day.
> > > 
> > > Again, no one asks you to deal with or maintain this piece of Rust code.
> > > 
> > > > Maintaining
> > > > multi-language projects is a pain I have no interest in dealing with.
> > > > If you want to use something that's not C, be that assembly or rust you
> > > > write to C interfaces and deal with the impedence mismatch yourself as
> > > > far as I'm concerned.
> > > > 
> > > 
> > > This is exactly what we're doing and proposing here, isn't it?
> > > 
> > > We wrote a single piece of Rust code that abstracts the C API for all Rust
> > > drivers, which we offer to maintain ourselves.
> > > 
> > > What else are you asking for?
> > 
> > Since there hasn't been a reply so far, I assume that we're good with
> > maintaining the DMA Rust abstractions separately.
> 
> Indeed, FWIW it seems like the appropriate level of abstraction to me,
> judging by the other wrappers living in rust/kernel/ already. As far as the
> interaction with C code goes, it appears to be a pretty straightforward
> midlayer consumer of the DMA API much like others we already have (e.g.
> videobuf2-dma-*), just one which happens to be a language binding rather
> than some other kind of functional abstraction.
> 
> There is a realistic chance that the C API will evolve in ways which break
> the binding, but as long as a) that won't break non-Rust builds, and b) Rust
> folks are happy to take responsibility for un-breaking the Rust build if and
> when it happens, then that seems reasonable IMO.

Surely you can expect maintainers of the Rust abstraction to help with
integrating API changes -- this isn't different compared to driver / component
maintainers helping with integrating fundamental API changes for their affected
driver / component, like you've mentioned videobuf2-dma stuff.

At last year's LPC I held a talk [1] about Rust in the kernel and there was a
question at the end where I was asked how I think about cases where Rust
abstraction break caused C API changes.

My answer was that I'm not too concerned about this. We can just rely on what
already happens every day in kernel development: people work together and
collaborate. There are a lot of very core components that are widely used and
depending on the complexity of the change may require the help of the users to
integrate changes. So, I don't think with Rust abstractions we're adding
anything that the kernel does not already has a strategy to deal with.

- Danilo

[1] https://youtu.be/3Igmx28B3BQ?si=wD0CP-zku4U6fAsN

> 
> Thanks,
> Robin.
> 
> > 
> > Hence, the next version of this patch series will have the corresponding
> > maintainer entry.
> > 
> > - Danilo
> 

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Simona Vetter]

On Thu, Jan 16, 2025 at 04:57:56PM +0100, Danilo Krummrich wrote:
> On Thu, Jan 16, 2025 at 01:57:50PM +0000, Robin Murphy wrote:
> > On 2025-01-16 1:17 pm, Danilo Krummrich wrote:
> > > On Fri, Jan 10, 2025 at 11:41:54AM +0100, Danilo Krummrich wrote:
> > > > On Fri, Jan 10, 2025 at 09:39:55AM +0100, Christoph Hellwig wrote:
> > > > > On Thu, Jan 09, 2025 at 09:49:47AM +0100, Danilo Krummrich wrote:
> > > > > > On Thu, Jan 09, 2025 at 09:08:12AM +0100, Christoph Hellwig wrote:
> > > > > > > On Wed, Jan 08, 2025 at 04:21:33PM +0100, Danilo Krummrich wrote:
> > > > > > > > What does "your code" mean? Duplicated in every driver?
> > > > > > > 
> > > > > > > Yes, interfaces to the DMA API should stay in readable C code and not
> > > > > > > in weird bindings so that it reminds greppable and maintainable.
> > > > > > > 
> > > > > > 
> > > > > > Rust drivers shouldn't use C APIs directly, but rather use an abstraction of the
> > > > > > corresponding C API.
> > > > > 
> > > > > Don't force me to deal with your shiny language of the day.
> > > > 
> > > > Again, no one asks you to deal with or maintain this piece of Rust code.
> > > > 
> > > > > Maintaining
> > > > > multi-language projects is a pain I have no interest in dealing with.
> > > > > If you want to use something that's not C, be that assembly or rust you
> > > > > write to C interfaces and deal with the impedence mismatch yourself as
> > > > > far as I'm concerned.
> > > > > 
> > > > 
> > > > This is exactly what we're doing and proposing here, isn't it?
> > > > 
> > > > We wrote a single piece of Rust code that abstracts the C API for all Rust
> > > > drivers, which we offer to maintain ourselves.
> > > > 
> > > > What else are you asking for?
> > > 
> > > Since there hasn't been a reply so far, I assume that we're good with
> > > maintaining the DMA Rust abstractions separately.
> > 
> > Indeed, FWIW it seems like the appropriate level of abstraction to me,
> > judging by the other wrappers living in rust/kernel/ already. As far as the
> > interaction with C code goes, it appears to be a pretty straightforward
> > midlayer consumer of the DMA API much like others we already have (e.g.
> > videobuf2-dma-*), just one which happens to be a language binding rather
> > than some other kind of functional abstraction.
> > 
> > There is a realistic chance that the C API will evolve in ways which break
> > the binding, but as long as a) that won't break non-Rust builds, and b) Rust
> > folks are happy to take responsibility for un-breaking the Rust build if and
> > when it happens, then that seems reasonable IMO.
> 
> Surely you can expect maintainers of the Rust abstraction to help with
> integrating API changes -- this isn't different compared to driver / component
> maintainers helping with integrating fundamental API changes for their affected
> driver / component, like you've mentioned videobuf2-dma stuff.
> 
> At last year's LPC I held a talk [1] about Rust in the kernel and there was a
> question at the end where I was asked how I think about cases where Rust
> abstraction break caused C API changes.
> 
> My answer was that I'm not too concerned about this. We can just rely on what
> already happens every day in kernel development: people work together and
> collaborate. There are a lot of very core components that are widely used and
> depending on the complexity of the change may require the help of the users to
> integrate changes. So, I don't think with Rust abstractions we're adding
> anything that the kernel does not already has a strategy to deal with.

I think when we know that an api change is in the works we should try to
anticipate that, and not add another complex user of a feature that's
likely going to disappear. For dma-api I guess the big thing is the
dma/page split that's been talked about since forever, instead of mixing
it all up into the scatterlist structure.

But with rust abstractions it should be pretty easy to make sure the rust
side interface is really clean here, and so any refactoring limited to
just the middlelayer. Plus this patch here is not even close to tackling
the sg based dma-api parts.

Same applies for rust wrapping dma-buf interfaces, that's just more of the
same.
-Sima

> 
> - Danilo
> 
> [1] https://youtu.be/3Igmx28B3BQ?si=wD0CP-zku4U6fAsN
> 
> > 
> > Thanks,
> > Robin.
> > 
> > > 
> > > Hence, the next version of this patch series will have the corresponding
> > > maintainer entry.
> > > 
> > > - Danilo
> > 

-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Abdiel Janulgue]

On 17/01/2025 15:56, Simona Vetter wrote:
> Plus this patch here is not even close to tackling
> the sg based dma-api parts.

The scatterlist portion of dma-api bindings are in the pipeline. The 
idea was to start upstreaming this basic component of the dma-api first.

Regards,
Abdiel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Daniel Almeida]

> On 17 Jan 2025, at 16:10, Abdiel Janulgue <abdiel.janulgue@gmail.com> wrote:
> 
> 
> On 17/01/2025 15:56, Simona Vetter wrote:
>> Plus this patch here is not even close to tackling
>> the sg based dma-api parts.
> 
> The scatterlist portion of dma-api bindings are in the pipeline. The idea was to start upstreaming this basic component of the dma-api first.
> 
> Regards,
> Abdiel
> 

Oh, I just realized you’re working on the scatterlist stuff. This will indeed come in handy!

Thanks for that.

— Daniel 

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Thu, Jan 16, 2025 at 02:17:24PM +0100, Danilo Krummrich wrote:
> Since there hasn't been a reply so far, I assume that we're good with
> maintaining the DMA Rust abstractions separately.

No, I'm not.  This was an explicit:

Nacked-by: Christoph Hellwig <hch@lst.de>

And I also do not want another maintainer.  If you want to make Linux
impossible to maintain due to a cross-language codebase do that in
your driver so that you have to do it instead of spreading this cancer
to core subsystems.  (where this cancer explicitly is a cross-language
codebase and not rust itself, just to escape the flameware brigade).

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Danilo Krummrich]

On Tue, Jan 28, 2025 at 10:23:34AM +0100, Christoph Hellwig wrote:
> On Thu, Jan 16, 2025 at 02:17:24PM +0100, Danilo Krummrich wrote:
> > Since there hasn't been a reply so far, I assume that we're good with
> > maintaining the DMA Rust abstractions separately.
> 
> No, I'm not.  This was an explicit:
> 
> Nacked-by: Christoph Hellwig <hch@lst.de>
> 
> And I also do not want another maintainer.  If you want to make Linux
> impossible to maintain due to a cross-language codebase do that in
> your driver so that you have to do it instead of spreading this cancer
> to core subsystems.  (where this cancer explicitly is a cross-language
> codebase and not rust itself, just to escape the flameware brigade).

I accept that you don't want to be involved with Rust in the kernel, which is
why we offered to maintain the Rust abstraction layer for the DMA coherent
allocator as a separate component (which it would be anyways) ourselves.

As explained previously, this component is just a user of the DMA coherent
allocator API (just like any other driver) and the reason we want this component
is because otherwise this abstraction layer would end up in every Rust driver
that needs the DMA coherent allocator as duplicated code, which for obvious
reasons isn't desirable.

Throughout this conversation I did not see technical arguments or concerns
against this, but I did recognize your opposition against Rust in the kernel
and cross-language projects.

Being a maintainer myself, I think it is outside the scope of a maintainer to
restrict the usage of a public kernel API for a certain entity arbitrarily and /
or by personal preference, which, as it appears to me, is the case here.

@Greg, @Linus: I believe this would be on your purview.

Please find the full thread in [1].

[1] https://lore.kernel.org/rust-for-linux/20250108135951.GA18074@lst.de/

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Wed, Jan 29, 2025 at 10:33:22PM +0100, Danilo Krummrich wrote:
> I accept that you don't want to be involved with Rust in the kernel, which is
> why we offered to maintain the Rust abstraction layer for the DMA coherent
> allocator as a separate component (which it would be anyways) ourselves.

Which doesn't help me a bit.  Every additional bit that the another
language creeps in drastically reduces the maintainability of the kernel
as an integrated project.  The only reason Linux managed to survive so
long is by not having internal boundaries, and adding another language
complely breaks this.  You might not like my answer, but I will do
everything I can do to stop this.  This is NOT because I hate Rust.
While not my favourite language it's definitively one of the best new
ones and I encourage people to use it for new projects where it fits.
I do not want it anywhere near a huge C code base that I need to
maintain.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Abdiel Janulgue]

Hi,

On 31/01/2025 09:57, Christoph Hellwig wrote:
> 
> Which doesn't help me a bit.  Every additional bit that the another
> language creeps in drastically reduces the maintainability of the kernel
> as an integrated project.  The only reason Linux managed to survive so
> long is by not having internal boundaries, and adding another language
> complely breaks this.  You might not like my answer, but I will do
> everything I can do to stop this.  This is NOT because I hate Rust.
> While not my favourite language it's definitively one of the best new
> ones and I encourage people to use it for new projects where it fits.
> I do not want it anywhere near a huge C code base that I need to
> maintain.
> 

I do acknowledge your reservations about the possible maintenance burden 
due to the introduction of a rust (or another language) consumer of the 
dma-api. But I was hoping that we could arrive at some sort of common 
ground?

If this rust middle-layer abstraction is unacceptable to you, could you 
perhaps suggest a solution so that all rust device driver don’t end up 
with redundant dma coherent allocator rust code? Could the rust team do 
something about it?

Thanks,

Abdiel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Christoph Hellwig]

On Mon, Feb 03, 2025 at 10:17:22AM +0200, Abdiel Janulgue wrote:
> I do acknowledge your reservations about the possible maintenance burden 
> due to the introduction of a rust (or another language) consumer of the 
> dma-api. But I was hoping that we could arrive at some sort of common 
> ground?

The common ground is that I have absolutely no interest in helping
to spread a multi-language code base.  I absolutely support using
Rust in new codebase, but I do not at all in Linux.

Thank you for your understanding!

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Philipp Stanner]

On Tue, 2025-01-28 at 10:23 +0100, Christoph Hellwig wrote:
> On Thu, Jan 16, 2025 at 02:17:24PM +0100, Danilo Krummrich wrote:
> > Since there hasn't been a reply so far, I assume that we're good
> > with
> > maintaining the DMA Rust abstractions separately.
> 
> No, I'm not.  This was an explicit:
> 
> Nacked-by: Christoph Hellwig <hch@lst.de>
> 
> And I also do not want another maintainer.  If you want to make Linux
> impossible to maintain due to a cross-language codebase do that in
> your driver so that you have to do it instead of spreading this
> cancer
> to core subsystems.  (where this cancer explicitly is a cross-
> language
> codebase and not rust itself, just to escape the flameware brigade).


Hallo,

would some sort of official statement by the "entire community"
reassure you that the burden of keeping Rust abstractions working with
any changes on the C side rests entirely on the Rust side's shoulders?
(because that's what the statements made by the latter seem to mean to
me)

As I see it, at least as long as Rust code is just drivers and
abstractions which sit on top of the core infrastructure, which is C,
breakages or a hypothetical unmaintainability would only affect the
Rust side, whereas the traditional kernel could go on working and being
maintained as it always did.

I'm thinking about something like a linux/LANGUAGES file, which
specifies what languages you can use and what the rules are. Written
down transparently black on white. As a counterpart to linux/LICENSES,
basically.


Philipp

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Daniel Almeida]

> 
> Hallo,
> 
> would some sort of official statement by the "entire community"
> reassure you that the burden of keeping Rust abstractions working with
> any changes on the C side rests entirely on the Rust side's shoulders?
> (because that's what the statements made by the latter seem to mean to
> me)
> 


This has been offered over and over in this thread, i.e.:


```
If you don't feel comfortable maintaining the Rust abstraction (or just don't
want to), that's fine. I don't think anyone expects you to do that, we can take
care of that instead.
```

https://lore.kernel.org/rust-for-linux/Z3-NqwAG_96yq8VD@pollux/

Communication is not the issue here.

Also, this person waited until v8 to give a single line NAK. This is not very nice.

— Daniel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Philipp Stanner]

On Thu, 2025-01-30 at 10:35 -0300, Daniel Almeida wrote:
> 
> > 
> > Hallo,
> > 
> > would some sort of official statement by the "entire community"
> > reassure you that the burden of keeping Rust abstractions working
> > with
> > any changes on the C side rests entirely on the Rust side's
> > shoulders?
> > (because that's what the statements made by the latter seem to mean
> > to
> > me)
> > 
> 
> 
> This has been offered over and over in this thread, i.e.:
> 
> 
> ```
> If you don't feel comfortable maintaining the Rust abstraction (or
> just don't
> want to), that's fine. I don't think anyone expects you to do that,
> we can take
> care of that instead.
> ```
> 
> https://lore.kernel.org/rust-for-linux/Z3-NqwAG_96yq8VD@pollux/


Yup, I've read the whole thread. I was referring to the statements in
those mails in my response.

My thinking is that such a file is the most official, most binding
statement / offer the project has.


P.



> 
> Communication is not the issue here.
> 
> Also, this person waited until v8 to give a single line NAK. This is
> not very nice.
> 
> — Daniel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Jason Gunthorpe]

On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
> would some sort of official statement by the "entire community"
> reassure you that the burden of keeping Rust abstractions working with
> any changes on the C side rests entirely on the Rust side's
> shoulders?

You'd have to reconcile that with the recent event where Linus defered
the MM pull request and some C patches were dropped because of rust
kbuild bugs:

https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/

It seems to me the message is now crystal clear, and the opposite of
what you claim.

All PRs to Linus must not break the rust build and the responsibilty
for that falls to all the maintainers. If the Rust team is not quick
enough to resolve any issues during the development window then
patches must be dropped before sending PRs, or Linus will refuse the
PR.

Effectively this seems to imply that patches changing some of the C
API cannot be merged by maintainers unless accompanied by matching
Rust hunks.

If there are different instructions to maintainers I would be
interested to know.

Thus, I would not describe this situation as "rests entirely on the
Rust side's shoulders".

Regardless, it seem to be where we are now, and I have it on my list
to follow Andrew and get a rust compiler setup..

Jason

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Greg KH]

On Thu, Jan 30, 2025 at 11:46:46AM -0400, Jason Gunthorpe wrote:
> On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
> > would some sort of official statement by the "entire community"
> > reassure you that the burden of keeping Rust abstractions working with
> > any changes on the C side rests entirely on the Rust side's
> > shoulders?
> 
> You'd have to reconcile that with the recent event where Linus defered
> the MM pull request and some C patches were dropped because of rust
> kbuild bugs:
> 
> https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/
> 
> It seems to me the message is now crystal clear, and the opposite of
> what you claim.
> 
> All PRs to Linus must not break the rust build and the responsibilty
> for that falls to all the maintainers. If the Rust team is not quick
> enough to resolve any issues during the development window then
> patches must be dropped before sending PRs, or Linus will refuse the
> PR.
> 
> Effectively this seems to imply that patches changing some of the C
> API cannot be merged by maintainers unless accompanied by matching
> Rust hunks.
> 
> If there are different instructions to maintainers I would be
> interested to know.

That's not the case, the one you point at above was a tooling issue that
people missed due to the holidays.  Fixing it up was simple enough and
people did so and moved on.

Once a core api changes in a tree and it hits linux-next and that blows
up a rust build, obviously people should notice it then and the rust
maintainers/developers have said they will fix it up.

So the claim remains the same here.  It's just like staging, api changes
to subsystems are allowed to break staging, and rust code, and
maintainers do NOT have to fix them up there, that's up to the staging
and rust maintainers/developers to do so.

thanks,

greg k-h

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Jason Gunthorpe]

On Thu, Jan 30, 2025 at 05:11:43PM +0100, Greg KH wrote:
> On Thu, Jan 30, 2025 at 11:46:46AM -0400, Jason Gunthorpe wrote:
> > On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
> > > would some sort of official statement by the "entire community"
> > > reassure you that the burden of keeping Rust abstractions working with
> > > any changes on the C side rests entirely on the Rust side's
> > > shoulders?
> > 
> > You'd have to reconcile that with the recent event where Linus defered
> > the MM pull request and some C patches were dropped because of rust
> > kbuild bugs:
> > 
> > https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/
> > 
> > It seems to me the message is now crystal clear, and the opposite of
> > what you claim.
> > 
> > All PRs to Linus must not break the rust build and the responsibilty
> > for that falls to all the maintainers. If the Rust team is not quick
> > enough to resolve any issues during the development window then
> > patches must be dropped before sending PRs, or Linus will refuse the
> > PR.
> > 
> > Effectively this seems to imply that patches changing some of the C
> > API cannot be merged by maintainers unless accompanied by matching
> > Rust hunks.
> > 
> > If there are different instructions to maintainers I would be
> > interested to know.
> 
> That's not the case, the one you point at above was a tooling issue that
> people missed due to the holidays.  Fixing it up was simple enough and
> people did so and moved on.

Regardless of holidays, you seem to be saying that Linus should have
accepted Andrew's PR and left rust with build failures?

I'm also not sure about moved on, Uros's previously accepted patches
are still unmerged:

Uros Bizjak (6):
      x86/kgdb: use IS_ERR_PCPU() macro
      compiler.h: introduce TYPEOF_UNQUAL() macro
      percpu: use TYPEOF_UNQUAL() in variable declarations
      percpu: use TYPEOF_UNQUAL() in *_cpu_ptr() accessors
      percpu: repurpose __percpu tag as a named address space qualifier
      percpu/x86: enable strict percpu checks via named AS qualifiers

Uros has now respun a v4 without a CONFIG_CC_HAS_TYPEOF_UNQUAL that
triggered the rust issue. The underlying Rust issue of mismatched
compilers messing up kconfig probes looks to still be open.

> So the claim remains the same here.  It's just like staging, api changes
> to subsystems are allowed to break staging, and rust code, and
> maintainers do NOT have to fix them up there, that's up to the staging
> and rust maintainers/developers to do so.

My reading of this paragraph makes me think you expect Linus to have
merged Andrew's PR and left the rust build broken so that the rust
maintainer/developers could fix it later?

Jason

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Greg KH]

On Thu, Jan 30, 2025 at 01:24:37PM -0400, Jason Gunthorpe wrote:
> On Thu, Jan 30, 2025 at 05:11:43PM +0100, Greg KH wrote:
> > On Thu, Jan 30, 2025 at 11:46:46AM -0400, Jason Gunthorpe wrote:
> > > On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
> > > > would some sort of official statement by the "entire community"
> > > > reassure you that the burden of keeping Rust abstractions working with
> > > > any changes on the C side rests entirely on the Rust side's
> > > > shoulders?
> > > 
> > > You'd have to reconcile that with the recent event where Linus defered
> > > the MM pull request and some C patches were dropped because of rust
> > > kbuild bugs:
> > > 
> > > https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/
> > > 
> > > It seems to me the message is now crystal clear, and the opposite of
> > > what you claim.
> > > 
> > > All PRs to Linus must not break the rust build and the responsibilty
> > > for that falls to all the maintainers. If the Rust team is not quick
> > > enough to resolve any issues during the development window then
> > > patches must be dropped before sending PRs, or Linus will refuse the
> > > PR.
> > > 
> > > Effectively this seems to imply that patches changing some of the C
> > > API cannot be merged by maintainers unless accompanied by matching
> > > Rust hunks.
> > > 
> > > If there are different instructions to maintainers I would be
> > > interested to know.
> > 
> > That's not the case, the one you point at above was a tooling issue that
> > people missed due to the holidays.  Fixing it up was simple enough and
> > people did so and moved on.
> 
> Regardless of holidays, you seem to be saying that Linus should have
> accepted Andrew's PR and left rust with build failures?

I can't answer for Linus, sorry.  But a generic "hey, this broke our
working toolchain builds" is something that is much much much different
than "an api changed so I now have to turn off this driver in my build"
issue.

thanks,

greg k-h

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Jason Gunthorpe]

On Fri, Jan 31, 2025 at 08:47:54AM +0100, Greg KH wrote:
> On Thu, Jan 30, 2025 at 01:24:37PM -0400, Jason Gunthorpe wrote:
> > On Thu, Jan 30, 2025 at 05:11:43PM +0100, Greg KH wrote:
> > > On Thu, Jan 30, 2025 at 11:46:46AM -0400, Jason Gunthorpe wrote:
> > > > On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
> > > > > would some sort of official statement by the "entire community"
> > > > > reassure you that the burden of keeping Rust abstractions working with
> > > > > any changes on the C side rests entirely on the Rust side's
> > > > > shoulders?
> > > > 
> > > > You'd have to reconcile that with the recent event where Linus defered
> > > > the MM pull request and some C patches were dropped because of rust
> > > > kbuild bugs:
> > > > 
> > > > https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/
> > > > 
> > > > It seems to me the message is now crystal clear, and the opposite of
> > > > what you claim.
> > > > 
> > > > All PRs to Linus must not break the rust build and the responsibilty
> > > > for that falls to all the maintainers. If the Rust team is not quick
> > > > enough to resolve any issues during the development window then
> > > > patches must be dropped before sending PRs, or Linus will refuse the
> > > > PR.
> > > > 
> > > > Effectively this seems to imply that patches changing some of the C
> > > > API cannot be merged by maintainers unless accompanied by matching
> > > > Rust hunks.
> > > > 
> > > > If there are different instructions to maintainers I would be
> > > > interested to know.
> > > 
> > > That's not the case, the one you point at above was a tooling issue that
> > > people missed due to the holidays.  Fixing it up was simple enough and
> > > people did so and moved on.
> > 
> > Regardless of holidays, you seem to be saying that Linus should have
> > accepted Andrew's PR and left rust with build failures?
> 
> I can't answer for Linus, sorry.  

Then I think we need a clear statement from Linus how he will be
working. If he is build testing rust or not.

Without that I don't think the Rust team should be saying "any changes
on the C side rests entirely on the Rust side's shoulders".

It is clearly not the process if Linus is build testing rust and
rejecting PRs that fail to build.

> But a generic "hey, this broke our
> working toolchain builds" is something that is much much much different

It broke some builds, it sounded like the typical rust build was not
effected because it used the same version of clang for C code and
bindgen. Linus was mixing gcc and clang in his build.

> than "an api changed so I now have to turn off this driver in my build"
> issue.

I tested this theory, allnoconfig x86 build, enable 64 bit, rust and
PCI only. *NO* Rust drivers enabled.

Make a hypothetical C API change:

       --- a/include/asm-generic/pci_iomap.h
       +++ b/include/asm-generic/pci_iomap.h
       @@ -18,7 +18,7 @@ extern void __iomem *pci_iomap_range(struct pci_dev *dev, int bar,
	extern void __iomem *pci_iomap_wc_range(struct pci_dev *dev, int bar,
					       unsigned long offset,
					       unsigned long maxlen);
       -extern void pci_iounmap(struct pci_dev *dev, void __iomem *);
       +extern void pci_iounmap(struct pci_dev *dev, void __iomem *, unsigned int flags);
	/* Create a virtual mapping cookie for a port on a given PCI device.
	 * Do not call this directly, it exists to make it easier for architectures
	 * to override */
       diff --git a/lib/iomap.c b/lib/iomap.c
       index 4f8b31baa5752a..5b63063a1a811f 100644
       --- a/lib/iomap.c
       +++ b/lib/iomap.c
       @@ -421,7 +421,7 @@ EXPORT_SYMBOL(ioport_unmap);
	#ifdef CONFIG_PCI
	/* Hide the details if this is a MMIO or PIO address space and just do what
	 * you expect in the correct way. */
       -void pci_iounmap(struct pci_dev *dev, void __iomem * addr)
       +void pci_iounmap(struct pci_dev *dev, void __iomem * addr, unsigned int flags)
	{
	       IO_COND(addr, /* nothing */, iounmap(addr));
	}

And the result is..

       error[E0061]: this function takes 3 arguments but 2 arguments were supplied
	  --> ../rust/kernel/pci.rs:320:13
	   |
       320 |             bindings::pci_iounmap(pdev.as_raw(), ioptr as _);
	   |             ^^^^^^^^^^^^^^^^^^^^^--------------------------- an argument of type `u32` is missing
	   |
       note: function defined here
	  --> /tmp/x/build/rust/bindings/bindings_generated.rs:3:1444598
	   |
       3   | ...> * mut ffi :: c_void ; } extern "C" { pub fn pci_iounmap (dev : * mut pci_dev , arg1 : * mut ffi :: c_v...
	   |                                                  ^^^^^^^^^^^
       help: provide the argument
	   |
       320 |             bindings::pci_iounmap(pdev.as_raw(), ioptr as _, /* u32 */);
	   |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Build fails.

Can't seem to fix this using kconfig without turning off CONFIG_RUST,
exactly the same outcome as Uros's case.

Doesn't seem "much much much different" to me.

This shouldn't be surprising. It doesn't work like C. Rust builds the
PCI bindings always once CONFIG_PCI is turned on. It doesn't matter if
no rust driver is being built that consumes those bindings. It won't
work like staging does where you can just turn off one driver.

Jason

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Hector Martin]

On 2025/01/31 22:54, Jason Gunthorpe wrote:
> On Fri, Jan 31, 2025 at 08:47:54AM +0100, Greg KH wrote:
>> On Thu, Jan 30, 2025 at 01:24:37PM -0400, Jason Gunthorpe wrote:
>>> On Thu, Jan 30, 2025 at 05:11:43PM +0100, Greg KH wrote:
>>>> On Thu, Jan 30, 2025 at 11:46:46AM -0400, Jason Gunthorpe wrote:
>>>>> On Thu, Jan 30, 2025 at 02:19:16PM +0100, Philipp Stanner wrote:
>>>>>> would some sort of official statement by the "entire community"
>>>>>> reassure you that the burden of keeping Rust abstractions working with
>>>>>> any changes on the C side rests entirely on the Rust side's
>>>>>> shoulders?
>>>>>
>>>>> You'd have to reconcile that with the recent event where Linus defered
>>>>> the MM pull request and some C patches were dropped because of rust
>>>>> kbuild bugs:
>>>>>
>>>>> https://lore.kernel.org/linux-mm/CAHk-=whddBhfi5DUi370W3pYs+z3r2E7KYuHjwR=a1eRig5Gxg@mail.gmail.com/
>>>>>
>>>>> It seems to me the message is now crystal clear, and the opposite of
>>>>> what you claim.
>>>>>
>>>>> All PRs to Linus must not break the rust build and the responsibilty
>>>>> for that falls to all the maintainers. If the Rust team is not quick
>>>>> enough to resolve any issues during the development window then
>>>>> patches must be dropped before sending PRs, or Linus will refuse the
>>>>> PR.
>>>>>
>>>>> Effectively this seems to imply that patches changing some of the C
>>>>> API cannot be merged by maintainers unless accompanied by matching
>>>>> Rust hunks.
>>>>>
>>>>> If there are different instructions to maintainers I would be
>>>>> interested to know.
>>>>
>>>> That's not the case, the one you point at above was a tooling issue that
>>>> people missed due to the holidays.  Fixing it up was simple enough and
>>>> people did so and moved on.
>>>
>>> Regardless of holidays, you seem to be saying that Linus should have
>>> accepted Andrew's PR and left rust with build failures?
>>
>> I can't answer for Linus, sorry.  
> 
> Then I think we need a clear statement from Linus how he will be
> working. If he is build testing rust or not.
> 
> Without that I don't think the Rust team should be saying "any changes
> on the C side rests entirely on the Rust side's shoulders".
> 
> It is clearly not the process if Linus is build testing rust and
> rejecting PRs that fail to build.

Adding Linus

My 2c: If Linus doesn't pipe up with an authoritative answer to this
thread, Miguel and the other Rust folks should just merge this series
once it is reviewed and ready, ignoring Christoph's overt attempt at
sabotaging the project. If Linus pulls it, what Christoph says doesn't
matter. If Linus doesn't pull it, the R4L project is essentially dead
until either Linus or Christoph make a move. Everything else is beating
around the bush.

Rust folks: Please don't waste your time and mental cycles on drama like
this. It's not worth your time. Either Linus likes it, or he doesn't.
Everything else is distractions orchestrated by a subset of saboteur
maintainers who are trying to demoralize you until you give up, because
they know they're going to be on the losing side of history sooner or
later. No amount of sabotage from old entrenched maintainers is going to
stop the world from moving forward towards memory-safe languages.

FWIW, in my opinion, the "cancer" comment from Christoph would be enough
to qualify for Code-of-Conduct action, but I doubt anything of the sort
will happen.

> 
>> But a generic "hey, this broke our
>> working toolchain builds" is something that is much much much different
> 
> It broke some builds, it sounded like the typical rust build was not
> effected because it used the same version of clang for C code and
> bindgen. Linus was mixing gcc and clang in his build.
> 
>> than "an api changed so I now have to turn off this driver in my build"
>> issue.
> 
> I tested this theory, allnoconfig x86 build, enable 64 bit, rust and
> PCI only. *NO* Rust drivers enabled.
> 
> Make a hypothetical C API change:
> 
>        --- a/include/asm-generic/pci_iomap.h
>        +++ b/include/asm-generic/pci_iomap.h
>        @@ -18,7 +18,7 @@ extern void __iomem *pci_iomap_range(struct pci_dev *dev, int bar,
> 	extern void __iomem *pci_iomap_wc_range(struct pci_dev *dev, int bar,
> 					       unsigned long offset,
> 					       unsigned long maxlen);
>        -extern void pci_iounmap(struct pci_dev *dev, void __iomem *);
>        +extern void pci_iounmap(struct pci_dev *dev, void __iomem *, unsigned int flags);
> 	/* Create a virtual mapping cookie for a port on a given PCI device.
> 	 * Do not call this directly, it exists to make it easier for architectures
> 	 * to override */
>        diff --git a/lib/iomap.c b/lib/iomap.c
>        index 4f8b31baa5752a..5b63063a1a811f 100644
>        --- a/lib/iomap.c
>        +++ b/lib/iomap.c
>        @@ -421,7 +421,7 @@ EXPORT_SYMBOL(ioport_unmap);
> 	#ifdef CONFIG_PCI
> 	/* Hide the details if this is a MMIO or PIO address space and just do what
> 	 * you expect in the correct way. */
>        -void pci_iounmap(struct pci_dev *dev, void __iomem * addr)
>        +void pci_iounmap(struct pci_dev *dev, void __iomem * addr, unsigned int flags)
> 	{
> 	       IO_COND(addr, /* nothing */, iounmap(addr));
> 	}
> 
> And the result is..
> 
>        error[E0061]: this function takes 3 arguments but 2 arguments were supplied
> 	  --> ../rust/kernel/pci.rs:320:13
> 	   |
>        320 |             bindings::pci_iounmap(pdev.as_raw(), ioptr as _);
> 	   |             ^^^^^^^^^^^^^^^^^^^^^--------------------------- an argument of type `u32` is missing
> 	   |
>        note: function defined here
> 	  --> /tmp/x/build/rust/bindings/bindings_generated.rs:3:1444598
> 	   |
>        3   | ...> * mut ffi :: c_void ; } extern "C" { pub fn pci_iounmap (dev : * mut pci_dev , arg1 : * mut ffi :: c_v...
> 	   |                                                  ^^^^^^^^^^^
>        help: provide the argument
> 	   |
>        320 |             bindings::pci_iounmap(pdev.as_raw(), ioptr as _, /* u32 */);
> 	   |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Build fails.
> 
> Can't seem to fix this using kconfig without turning off CONFIG_RUST,
> exactly the same outcome as Uros's case.
> 
> Doesn't seem "much much much different" to me.
> 
> This shouldn't be surprising. It doesn't work like C. Rust builds the
> PCI bindings always once CONFIG_PCI is turned on. It doesn't matter if
> no rust driver is being built that consumes those bindings. It won't
> work like staging does where you can just turn off one driver.
> 
> Jason
> 
> 

- Hector

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Jason Gunthorpe]

On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
> 
> My 2c: If Linus doesn't pipe up with an authoritative answer to this
> thread, Miguel and the other Rust folks should just merge this
> series

Please don't hijack this side thread.

This is about Linus's policy for merging C code that breaks Rust
builds. You should put this comment on the branch with Christoph's
NAK.

Thanks,
Jason

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Hector Martin]

On 2025/02/04 4:16, Jason Gunthorpe wrote:
> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
>>
>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>> thread, Miguel and the other Rust folks should just merge this
>> series
> 
> Please don't hijack this side thread.
> 
> This is about Linus's policy for merging C code that breaks Rust
> builds. You should put this comment on the branch with Christoph's
> NAK.
> 

Sorry for the thread mishap. It's kind of hard to redirect to the other
branch now though...

- Hector

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Paolo Bonzini]

On 2/3/25 19:46, Hector Martin wrote:
> Adding Linus
> 
> My 2c: If Linus doesn't pipe up with an authoritative answer to this
> thread, Miguel and the other Rust folks should just merge this series
> once it is reviewed and ready, ignoring Christoph's overt attempt at
> sabotaging the project.

Hold your horses...  I agree that they should just merge the series in 
the unlikely event that Linus doesn't chime in over the next two months, 
but I would very strongly suggest that it's sent to Linus (assuming he 
doesn't jump in now) as a separate pull request.

This is almost always a good thing to do when you have commits that 
merit specific attention (example: 
https://www.uwsg.indiana.edu/hypermail/linux/kernel/2109.2/10311.html).

My 2 cents as someone that is certainly not any kind of VIP, but has 
been in MAINTAINERS for quite a while.

> Rust folks: Please don't waste your time and mental cycles on drama like
> this. It's not worth your time. Either Linus likes it, or he doesn't.
> Everything else is distractions orchestrated by a subset of saboteur
> maintainers who are trying to demoralize you until you give up, because
> they know they're going to be on the losing side of history sooner or
> later. No amount of sabotage from old entrenched maintainers is going to
> stop the world from moving forward towards memory-safe languages.

I have a question, which is unrelated to my opinion of Rust for Linux: 
in what way do you think this tirade is actually helping?

> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
> to qualify for Code-of-Conduct action, but I doubt anything of the sort
> will happen.

FWIW I agree that it was borderline and probably on the wrong side of 
the edge.  But I am happy for one that Christoph has since expanded 
beyond the "cancer" comment, because that's at least a technical 
argument, unlike yours above.

Paolo

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Hector Martin]

On 2025/02/04 4:22, Paolo Bonzini wrote:
> On 2/3/25 19:46, Hector Martin wrote:
>> Adding Linus
>>
>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>> thread, Miguel and the other Rust folks should just merge this series
>> once it is reviewed and ready, ignoring Christoph's overt attempt at
>> sabotaging the project.
> 
> Hold your horses...  I agree that they should just merge the series in 
> the unlikely event that Linus doesn't chime in over the next two months, 
> but I would very strongly suggest that it's sent to Linus (assuming he 
> doesn't jump in now) as a separate pull request.
> 
> This is almost always a good thing to do when you have commits that 
> merit specific attention (example: 
> https://www.uwsg.indiana.edu/hypermail/linux/kernel/2109.2/10311.html).

Sure, as a separate PR is fine.

> My 2 cents as someone that is certainly not any kind of VIP, but has 
> been in MAINTAINERS for quite a while.
> 
>> Rust folks: Please don't waste your time and mental cycles on drama like
>> this. It's not worth your time. Either Linus likes it, or he doesn't.
>> Everything else is distractions orchestrated by a subset of saboteur
>> maintainers who are trying to demoralize you until you give up, because
>> they know they're going to be on the losing side of history sooner or
>> later. No amount of sabotage from old entrenched maintainers is going to
>> stop the world from moving forward towards memory-safe languages.
> 
> I have a question, which is unrelated to my opinion of Rust for Linux: 
> in what way do you think this tirade is actually helping?

There have already been high-profile departures from the Rust for Linux
project due to the open hostility of certain kernel maintainers. The
tension in the air is palpable, and so is the lowering morale.

I'm encouraging the people involved to stop playing nice and start
playing hardball in these cases, because playing nice does not work with
some people. Taking the high road only works with people who want to
cooperate and reach a solution that works for everyone. People who
openly do not want to cooperate nor reach any sort of reasonable
solution should be shunned and ignored, as is the case here.

> 
>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
>> will happen.
> 
> FWIW I agree that it was borderline and probably on the wrong side of 
> the edge.  But I am happy for one that Christoph has since expanded 
> beyond the "cancer" comment, because that's at least a technical 
> argument, unlike yours above.

My argument above may not be "technical" in the technology sense, but it
is quite specific and objective: Christoph has made it *very* clear that
he is not intending to reach any sort of workable solution. He has
decided that he doesn't want Rust in the Linux kernel and will do
whatever he can to stop it, which fundamentally clashes with the goals
of the R4L project in an irreconcilable way. He may have his own
"technical" reasoning for this, but this is irrelevant, because there is
no way to appeal to his technical concerns and the technical goals of
the R4L project simultaneously.

What he is doing is quite literally the dictionary definition of
"sabotage". Since his goal is to sabotage the R4L project, in its
fundamental goal, no amount of purely technical discussion will allow us
to reach a workable solution. Therefore, the only possible reaction is
social in nature: Ignore Christoph, and work around him.

- Hector

On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Simona Vetter]

On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
> Adding Linus
> 
> My 2c: If Linus doesn't pipe up with an authoritative answer to this
> thread, Miguel and the other Rust folks should just merge this series
> once it is reviewed and ready, ignoring Christoph's overt attempt at
> sabotaging the project. If Linus pulls it, what Christoph says doesn't
> matter. If Linus doesn't pull it, the R4L project is essentially dead
> until either Linus or Christoph make a move. Everything else is beating
> around the bush.
> 
> Rust folks: Please don't waste your time and mental cycles on drama like
> this. It's not worth your time. Either Linus likes it, or he doesn't.
> Everything else is distractions orchestrated by a subset of saboteur
> maintainers who are trying to demoralize you until you give up, because
> they know they're going to be on the losing side of history sooner or
> later. No amount of sabotage from old entrenched maintainers is going to
> stop the world from moving forward towards memory-safe languages.
> 
> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
> to qualify for Code-of-Conduct action, but I doubt anything of the sort
> will happen.

Yeah no.

https://chaos.social/@sima/113942119012147959

This was about you, because typing a proper answer takes a bit longer. It
was also about your toots on fedi, like this:

https://social.treehouse.systems/@marcan/113941468353031993

And "haha it's only a joke" does not work with your public profile and following.

I do understand the frustration and temptation to just burn it all to the
ground, head the call of the sirens, or maybe for me more pick up goat
farming in the Swiss Alps. But you can't have it both and expect to also
be part of and contribute to the same community. And personally I don't
appreciate getting drenched in gasoline while I'm trying to quench flames
on the ground.

And this isn't the first time or the second, by now it's a pretty clear
pattern over some years. And with the first I could explain why you react
like that and you had my full understanding, but eventually that runs a
bit thin as an excuse.  Now I'm left with the unlikely explanation that
you just like thundering in as the cavalry, fashionably late, maximally
destructive, because it entertains the masses on fedi or reddit or
wherever. I have no idea what you're trying to achieve here, I really
don't get it, but I am for sure fed up dealing with the fallout.

Cheers, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Dave Airlie]

On Thu, 6 Feb 2025 at 04:52, Simona Vetter <simona.vetter@ffwll.ch> wrote:
>
> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
> > Adding Linus
> >
> > My 2c: If Linus doesn't pipe up with an authoritative answer to this
> > thread, Miguel and the other Rust folks should just merge this series
> > once it is reviewed and ready, ignoring Christoph's overt attempt at
> > sabotaging the project. If Linus pulls it, what Christoph says doesn't
> > matter. If Linus doesn't pull it, the R4L project is essentially dead
> > until either Linus or Christoph make a move. Everything else is beating
> > around the bush.
> >
> > Rust folks: Please don't waste your time and mental cycles on drama like
> > this. It's not worth your time. Either Linus likes it, or he doesn't.
> > Everything else is distractions orchestrated by a subset of saboteur
> > maintainers who are trying to demoralize you until you give up, because
> > they know they're going to be on the losing side of history sooner or
> > later. No amount of sabotage from old entrenched maintainers is going to
> > stop the world from moving forward towards memory-safe languages.
> >
> > FWIW, in my opinion, the "cancer" comment from Christoph would be enough
> > to qualify for Code-of-Conduct action, but I doubt anything of the sort
> > will happen.
>
> Yeah no.
>
> https://chaos.social/@sima/113942119012147959
>
> This was about you, because typing a proper answer takes a bit longer. It
> was also about your toots on fedi, like this:
>
> https://social.treehouse.systems/@marcan/113941468353031993
>
> And "haha it's only a joke" does not work with your public profile and following.
>
> I do understand the frustration and temptation to just burn it all to the
> ground, head the call of the sirens, or maybe for me more pick up goat
> farming in the Swiss Alps. But you can't have it both and expect to also
> be part of and contribute to the same community. And personally I don't
> appreciate getting drenched in gasoline while I'm trying to quench flames
> on the ground.
>
> And this isn't the first time or the second, by now it's a pretty clear
> pattern over some years. And with the first I could explain why you react
> like that and you had my full understanding, but eventually that runs a
> bit thin as an excuse.  Now I'm left with the unlikely explanation that
> you just like thundering in as the cavalry, fashionably late, maximally
> destructive, because it entertains the masses on fedi or reddit or
> wherever. I have no idea what you're trying to achieve here, I really
> don't get it, but I am for sure fed up dealing with the fallout.
>

To back up Sima here, we don't need grandstanding, brigading, playing
to the crowd, streamer drama creation or any of that in discussions
around this.

The r4l team and drm maintainer team have this sort of thing in hand,
it's not like we don't understand the community of the Linux kernel,
and having this first reaction to blow shit up and dramatise it just
isn't helpful.

Being toxic on the right side of an argument is still toxic, please
try and be better, and maybe take a step back and consider is what you
are posting going to help the discussion or just adding pointless
drama to it.

Dave.

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/06 5:36, Dave Airlie wrote:
> On Thu, 6 Feb 2025 at 04:52, Simona Vetter <simona.vetter@ffwll.ch> wrote:
>>
>> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
>>> Adding Linus
>>>
>>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>>> thread, Miguel and the other Rust folks should just merge this series
>>> once it is reviewed and ready, ignoring Christoph's overt attempt at
>>> sabotaging the project. If Linus pulls it, what Christoph says doesn't
>>> matter. If Linus doesn't pull it, the R4L project is essentially dead
>>> until either Linus or Christoph make a move. Everything else is beating
>>> around the bush.
>>>
>>> Rust folks: Please don't waste your time and mental cycles on drama like
>>> this. It's not worth your time. Either Linus likes it, or he doesn't.
>>> Everything else is distractions orchestrated by a subset of saboteur
>>> maintainers who are trying to demoralize you until you give up, because
>>> they know they're going to be on the losing side of history sooner or
>>> later. No amount of sabotage from old entrenched maintainers is going to
>>> stop the world from moving forward towards memory-safe languages.
>>>
>>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
>>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
>>> will happen.
>>
>> Yeah no.
>>
>> https://chaos.social/@sima/113942119012147959
>>
>> This was about you, because typing a proper answer takes a bit longer. It
>> was also about your toots on fedi, like this:
>>
>> https://social.treehouse.systems/@marcan/113941468353031993
>>
>> And "haha it's only a joke" does not work with your public profile and following.
>>
>> I do understand the frustration and temptation to just burn it all to the
>> ground, head the call of the sirens, or maybe for me more pick up goat
>> farming in the Swiss Alps. But you can't have it both and expect to also
>> be part of and contribute to the same community. And personally I don't
>> appreciate getting drenched in gasoline while I'm trying to quench flames
>> on the ground.
>>
>> And this isn't the first time or the second, by now it's a pretty clear
>> pattern over some years. And with the first I could explain why you react
>> like that and you had my full understanding, but eventually that runs a
>> bit thin as an excuse.  Now I'm left with the unlikely explanation that
>> you just like thundering in as the cavalry, fashionably late, maximally
>> destructive, because it entertains the masses on fedi or reddit or
>> wherever. I have no idea what you're trying to achieve here, I really
>> don't get it, but I am for sure fed up dealing with the fallout.
>>
> 
> To back up Sima here, we don't need grandstanding, brigading, playing
> to the crowd, streamer drama creation or any of that in discussions
> around this.
> 
> The r4l team and drm maintainer team have this sort of thing in hand,
> it's not like we don't understand the community of the Linux kernel,
> and having this first reaction to blow shit up and dramatise it just
> isn't helpful.
> 
> Being toxic on the right side of an argument is still toxic, please
> try and be better, and maybe take a step back and consider is what you
> are posting going to help the discussion or just adding pointless
> drama to it.
> 
> Dave.

I'm tired.

I'm tired of seeing positive, technically impressive kernel projects
blockaded delayed by maintainers with no technical justification, and at
best end up moving along at a glacial pace.

I'm tired of seeing important contributors and maintainers give up and
throw the towel after enduring repeated misbehavior and hostility
towards their efforts from others.

I'm tired of getting messages, privately and publicly, from all kinds of
people, saying they won't touch the kernel with a 10-foot pole due to
the hostility and the baroque, regressive process.

I'm tired of seeing people get away with using words like "cancer" to
describe others' work, with zero repercussion.

I'm tired of *politely and calmly* calling out hostile and unwelcoming
behavior from maintainers and suggest ways to improve, only to be
ignored and nothing change (note: this refers to other instances, not
this instance).

I'm tired of having to spend hours or days of my time to upstream simple
things, because even the simplest of changes en up in a bikeshed.

I'm tired of having to manually format code instead of using clang-format.

I'm tired of drive-by nitpickers who send useless review comments on
code they don't take the time to understand.

I'm tired of having to review patches in an email client, where I can't
even tell which patches are for me to merge and not without writing
complex filtering rules to correlate email bodies with kernel subsystem
paths, which I don't have the time to write and maintain.

I'm tired of having to type a half dozen `b4` commands just to send a
change.

And I'm tired of hearing things will get better if I just "trust the
process" or let people work from within, while nothing seems to have
actually changed in years despite endless discussion about these
problems on the sidelines.

If shaming on social media does not work, then tell me what does,
because I'm out of ideas.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Linus Torvalds]

On Thu, 6 Feb 2025 at 01:19, Hector Martin <marcan@marcan.st> wrote:
>
> If shaming on social media does not work, then tell me what does,
> because I'm out of ideas.

How about you accept the fact that maybe the problem is you.

You think you know better. But the current process works.

It has problems, but problems are a fact of life.  There is no perfect.

However, I will say that the social media brigading just makes me not
want to have anything at all to do with your approach.

Because if we have issues in the kernel development model, then social
media sure as hell isn't the solution. The same way it sure as hell
wasn't the solution to politics.

Technical patches and discussions matter. Social media brigading - no
than\k you.

                 Linus

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Dr. Greg]

On Thu, Feb 06, 2025 at 09:58:36AM -0800, Linus Torvalds wrote:

Good morning to everyone.

> On Thu, 6 Feb 2025 at 01:19, Hector Martin <marcan@marcan.st> wrote:
> >
> > If shaming on social media does not work, then tell me what does,
> > because I'm out of ideas.

> Because if we have issues in the kernel development model, then
> social media sure as hell isn't the solution. The same way it sure
> as hell wasn't the solution to politics.
>
> Technical patches and discussions matter. Social media brigading - no
> thank you.

I truely wish that technical patches and discussions were the currency
that matter but I believe we are struggling with that, at least in
some venues in the kernel development process.

No one should construe that statement as an endorsement of berating
people on social media as the 'fix'.

I come at this from the perspective of having worked on Linux since
around December of 1991.  I first met you and Tove in 1995 at the Free
Software Conference at MIT that Stallman sponsored.  When we first
met, I told you that cancer patients in North Dakota were enjoying
more time with their families because of what we were able to do with
Linux and optimizing medical processes at our Cancer Center.

RedHat paid me to speak at a number of conferences in the 90's talking
about how Linux was going to dominate enterprise computing, given that
it was about technology people doing the 'right' technology thing.

I'm seeing things that make me regret those words.

Probably the last technical contribution of my career is leading an
initiative to provide the Linux community a generic security modeling
architecture.  Not to supplant or replace anything currently being
done, but to provide a flexible alternative to the development of
alternate and/or customized workload models, particularly in this era
of machine learning and modeling.

Four patch series over two years, as of yesterday, not a single line
of code ever reviewed.

For a contribution that touches nothing outside of its own directory
and does nothing unless people choose to execute a workload under its
control.

We were meticulous in our submissions to avoid wasting maintainers
time.  We even waited two months without hearing a word before we sent
an inquiry as to the status of one of the submissions.  We were told,
rather curtly, that anything we sent would likely be ignored if we
ever inquired about them.

We tried to engage, perhaps to excess, in technical discussions
attempting to explain why and how we chose to implement what we were
proposing.  Including input from advisors who are running production
IT systems that feel that there needs to be better approaches to
addressing their security needs.

There were never any relevant technical exchanges.  The discussion
consisted of, we have decided to do things a certain way, no
discussion, if you don't like that you should really consider doing
something other than submitting to upstream Linux.

The all powerful sub-system maintainer model works well if the big
technology companies can employ omniscient individuals in these roles,
but those types are a bit hard to come by.  Lacking that, there is the
tangible risk of stifling innovation and Linux is the only place that
innovation can occur in the operating system space.

Not sure what the fix is, from a project management perspective the
technology industry has never faced a challenge like this.  The fork
model, which was the classic protection in open-source, doesn't work
at this scale.

We have a Code Of Conduct that we can't scream or hurl four letter
words and insults at one another.  Maybe it already exists but a Code
Of Standards for maintainers would seem to be an imperative if we are
going to move forward productively.  Jim are you listening?

Obviously respect and open-mindedness to new ideas appears to be the
grease that makes all of this run smoothly.  Unfortunately that seems
to be about as rare a commodity as omniscience in our industry.

>                  Linus

Linus, best wishes to you and your family, it has been a fascinating
ride and thing to watch.

As always,
Dr. Greg

The Quixote Project - Flailing at the Travails of Cybersecurity
              https://github.com/Quixote-Project

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Steven Rostedt]

On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote:
> 
> Not sure what the fix is, from a project management perspective the
> technology industry has never faced a challenge like this.  The fork
> model, which was the classic protection in open-source, doesn't work
> at this scale.

Maybe not quite a fork, but I wonder if the Rust project did something similar
to what PREEMPT_RT did. That was to keep an out of tree patch. The full patch
was just merged last September after being out of tree for a good 20 years.

In the beginning there was a few things in that patch that Christoph was
against, but over time he became accepting of it.

Yes, being out of tree is very difficult because you have to constantly rebase
(we actually found quilt being a better tool than git in this case!). But it
also gives you full flexibility to try new approaches. Just because something
is out of tree doesn't mean it can't be published and used. Red Hat and SUSE,
as well as many others shipped PREEMPT_RT while it was out of tree.

Note, even though PREEMPT_RT started in 2004 and wasn't fully merged until
2024, it slowly did creep in bit by bit. For example, here's a few things that
came from the RT patch, and each was rewritten at least 3 times to become
acceptable by the upstream maintainers:

  - NOHZ
  - High res timers
  - threaded interrupts
  - mutex code (yes, before RT everything used a semaphore)
  - lockdep
  - ftrace
  - generic interrupt code
  - generic timer code
  - priority inheritance
  - SCHED_DEADLINE
  - RT push/pull scheduling

and more.

Point being, if you are having issues with getting code upstream, don't be
afraid to make it out of tree. Just make it easy for others to use. And slowly
move your code into mainline. This is also a way to prove beyond a doubt how
useful your code is. The number of users is a technical argument. Linus does
pull things in when there is a large user base, even over other maintainer's
objections (see sched_ext).

It worked for us, it can work for you ;-)

-- Steve

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Steven Rostedt]

On Fri, Feb 07, 2025 at 11:26:50PM -0500, Steven Rostedt wrote:
> 
> Note, even though PREEMPT_RT started in 2004 and wasn't fully merged until
> 2024, it slowly did creep in bit by bit. For example, here's a few things that
> came from the RT patch, and each was rewritten at least 3 times to become
> acceptable by the upstream maintainers:
> 
>   - NOHZ
>   - High res timers
>   - threaded interrupts
>   - mutex code (yes, before RT everything used a semaphore)
>   - lockdep
>   - ftrace
>   - generic interrupt code
>   - generic timer code
>   - priority inheritance
>   - SCHED_DEADLINE
>   - RT push/pull scheduling
> 
> and more.
> 

Here's a little bit of Linux trivia. KVM was first introduced to Linux via the
RT patch. Because it was such a new technology and they didn't want to break
the Linux workflow, we agreed to take their changes so that they could try out
different methods and have users without being committed to something and have
their changes break upstream Linux.

Sound familiar?

-- Steve

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/08 13:26, Steven Rostedt wrote:
> On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote:
>>
>> Not sure what the fix is, from a project management perspective the
>> technology industry has never faced a challenge like this.  The fork
>> model, which was the classic protection in open-source, doesn't work
>> at this scale.
> 
> Maybe not quite a fork, but I wonder if the Rust project did something similar
> to what PREEMPT_RT did. That was to keep an out of tree patch. The full patch
> was just merged last September after being out of tree for a good 20 years.
> 
> In the beginning there was a few things in that patch that Christoph was
> against, but over time he became accepting of it.
> 
> Yes, being out of tree is very difficult because you have to constantly rebase
> (we actually found quilt being a better tool than git in this case!). But it
> also gives you full flexibility to try new approaches. Just because something
> is out of tree doesn't mean it can't be published and used. Red Hat and SUSE,
> as well as many others shipped PREEMPT_RT while it was out of tree.

By kernel and mesa policy, while a drm/ kernel driver is not merged with
a stable UAPI, the mesa side cannot be merged/enabled either. That means
that every day the driver isn't upstream, our users suffer due to
various container technologies not shipping the driver in userspace
since they ship upstream mesa.

The process is *designed* to encourage upstreaming early and *punish*
downstream trees.

It's not just distros shipping, or even typical container technologies
like Flatpak (we have a workaround for that specific one). We'd need to
get stuff like Android/AOSP to ship the patched userspace driver so it
works in Waydroid, which some people want to use. It is an impossible task.

Never mind that distros aren't, in fact, inclined to ship out-of-tree
kernels. PREEMPT_RT was an exception due to its general usefulness, and
even then was not available everywhere, and had a reasonable excuse for
being out-of-tree so long due to its intrusiveness in core code. Rust
and the Rust drivers aren't even remotely as intrusive, they just stay
off to the side in their own directories. Nobody wants to ship
downstream kernels just for random hardware support.

Out of tree doesn't work. People, both in the kernel and out of the
kernel, want things in tree.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Icenowy Zheng]

在 2025-02-08星期六的 17:31 +0900，Hector Martin写道：
> 
> By kernel and mesa policy, while a drm/ kernel driver is not merged
> with
> a stable UAPI, the mesa side cannot be merged/enabled either. That
> means
> that every day the driver isn't upstream, our users suffer due to
> various container technologies not shipping the driver in userspace
> since they ship upstream mesa.
> 
> The process is *designed* to encourage upstreaming early and *punish*
> downstream trees.

Well, at least some Mesa drivers are developed against some "vendor"
(even not community downstream) kernel drivers, e.g. powervr one;
although in the Asahi case there's no such vendor thing (because the HW
vendor is Darwin-only). In addition, I think at least some early
etnaviv development is based on drm/etnaviv drivers w/ in-kernel
version code earlier than the first in-kernel-tree version code (Well
drm version code rarely changes, but I think in the case of etnaviv it
really represents the UAPI).

Furtherly, the monorepo nature of Linux kernel means to refactor an
interface, it's usually the person changing the callee that need to
change all callers to satify the interface change; having Rust code in
tree calling the interface effectively means adding the responsibility
of fixing the Rust code when the interface changes, which could be not
acceptable by the C-only maintainers. In regards of adding a
maintainer, having more maintainers means more communication. The
situation of the current problem really shows how difficult
communication is, right?

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Danilo Krummrich]

On Mon, Feb 10, 2025 at 05:41:30PM +0800, Icenowy Zheng wrote:
> Furtherly, the monorepo nature of Linux kernel means to refactor an
> interface, it's usually the person changing the callee that need to
> change all callers to satify the interface change; having Rust code in
> tree calling the interface effectively means adding the responsibility
> of fixing the Rust code when the interface changes, which could be not
> acceptable by the C-only maintainers. In regards of adding a
> maintainer, having more maintainers means more communication.

This is exactly the same as for every new driver / component, regardless of
whether it is written in C or in Rust.

It is absolutely common that driver maintainers help with integrating core API
changes, if necessary.

I don't see why the same process should not work for Rust abstractions.

(Additionally, in this particular case even one of the reviewers of
DMA MAPPING HELPERS offered to be a reviewer of the Rust abstractions too, in
order to keep eye on the DMA API angle.)

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Icenowy Zheng]

在 2025-02-10星期一的 11:24 +0100，Danilo Krummrich写道：
> On Mon, Feb 10, 2025 at 05:41:30PM +0800, Icenowy Zheng wrote:
> > Furtherly, the monorepo nature of Linux kernel means to refactor an
> > interface, it's usually the person changing the callee that need to
> > change all callers to satify the interface change; having Rust code
> > in
> > tree calling the interface effectively means adding the
> > responsibility
> > of fixing the Rust code when the interface changes, which could be
> > not
> > acceptable by the C-only maintainers. In regards of adding a
> > maintainer, having more maintainers means more communication.
> 
> This is exactly the same as for every new driver / component,
> regardless of
> whether it is written in C or in Rust.
> 
> It is absolutely common that driver maintainers help with integrating
> core API
> changes, if necessary.
> 
> I don't see why the same process should not work for Rust
> abstractions.

Because integrating API changes could involve change to contexts of API
calls, which could be difficult for Rust situation, especially when
it's not required for core API maintainers to be able to write Rust.

> 
> (Additionally, in this particular case even one of the reviewers of
> DMA MAPPING HELPERS offered to be a reviewer of the Rust abstractions
> too, in
> order to keep eye on the DMA API angle.)

Sorry, but I did a fact check on this, and I found that the only
"reviewer" of DMA MAPPING HELPERS is Robin Murphy, he has only one
reply in this thread, and the reply only says "Indeed, FWIW it seems
like the appropriate level of abstraction to me, 
judging by the other wrappers living in rust/kernel/ already", he
didn't offer to be a reviewer, and he still says "Rust folks are happy
to take responsibility for un-breaking the 
Rust build if and when it happens".

He is just saying he's going to accept the abstraction, which should be
not able to forcibly override Christoph's explicit NACK here.

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Abdiel Janulgue]

Hi,

On 13/02/2025 05:49, Icenowy Zheng wrote:
> 
> Sorry, but I did a fact check on this, and I found that the only
> "reviewer" of DMA MAPPING HELPERS is Robin Murphy, he has only one
> reply in this thread, and the reply only says "Indeed, FWIW it seems
> like the appropriate level of abstraction to me,
> judging by the other wrappers living in rust/kernel/ already", he
> didn't offer to be a reviewer, 

Robin did offer:

https://lore.kernel.org/rust-for-linux/4956d01e-2d06-4edd-813b-9da94b482069@arm.com/

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Icenowy Zheng]

在 2025-02-13星期四的 08:41 +0200，Abdiel Janulgue写道：
> Hi,
> 
> On 13/02/2025 05:49, Icenowy Zheng wrote:
> > 
> > Sorry, but I did a fact check on this, and I found that the only
> > "reviewer" of DMA MAPPING HELPERS is Robin Murphy, he has only one
> > reply in this thread, and the reply only says "Indeed, FWIW it
> > seems
> > like the appropriate level of abstraction to me,
> > judging by the other wrappers living in rust/kernel/ already", he
> > didn't offer to be a reviewer, 
> 
> Robin did offer:
> 
> https://lore.kernel.org/rust-for-linux/4956d01e-2d06-4edd-813b-9da94b482069@arm.com/

Well okay it's a further thing.

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Danilo Krummrich]

On Thu, Feb 13, 2025 at 11:49:20AM +0800, Icenowy Zheng wrote:
> 在 2025-02-10星期一的 11:24 +0100，Danilo Krummrich写道：
> > 
> > (Additionally, in this particular case even one of the reviewers of
> > DMA MAPPING HELPERS offered to be a reviewer of the Rust abstractions
> > too, in
> > order to keep eye on the DMA API angle.)
> 
> Sorry, but I did a fact check on this, and I found that the only
> "reviewer" of DMA MAPPING HELPERS is Robin Murphy, he has only one
> reply in this thread, and the reply only says "Indeed, FWIW it seems
> like the appropriate level of abstraction to me, 
> judging by the other wrappers living in rust/kernel/ already", he
> didn't offer to be a reviewer,

As Abdiel pointed out already, he did offer it in [1].

But that's not the relevant part, but I think how you handled being in doubt is
relevant.

I think the correct way would have been to just ask for a pointer that proves
the statement in question.

Instead you just went ahead with the big words "fact check" and then even got it
wrong. In your "fact check" you did not put any disclaimer to e.g. indicate that
you might not have the full picture, etc.

Ultimately, the way you replied to this, comes across as an immediate accuse of
lying.

I really think that we should *not* pick up this way of arguing that nowadays
became all too present.

- Danilo

[1] https://lore.kernel.org/rust-for-linux/4956d01e-2d06-4edd-813b-9da94b482069@arm.com/

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Theodore Ts'o]

On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote:
> 
> The all powerful sub-system maintainer model works well if the big
> technology companies can employ omniscient individuals in these roles,
> but those types are a bit hard to come by.

I'll let you in a secret.  The maintainers are not "all-powerfui".  We
are the "thin blue line" that is trying to keep the code to be
maintainable and high quality.  Like most leaders of volunteer
organization, whether it is the Internet Engineerint Task Force (the
standards body for the Internet), we actually have very little power.
We can not *command* people to work on retiring technical debt, or to
improve testing infrastructure, or work on some particular feature
that we'd very like for our users.

All we can do is stop things from being accepted (either in our
subsystem, or the imprimatur of the IETF).  Hopefully, we're only
stopping bad things from progressing, but that *really* is all we can
actually do.

One of the things which gets very frustrating from the maintainer's
perspective is development teams that are only interested in their pet
feature, and we *know*, through very bitter experience, that 95+% of
the time, once the code is accepted, the engineers which contribute
the code will disappear, never to be seen again.  As a result, a very
common dynamic is that maintainers will exercise the one and only
power which they have --- which is to refuse to accept code until it
is pretty much perfect --- since once we accept the code, we instantly
lose all leverge, and the contributors will be disappear, and we will
be left with the responsibility of cleanig up the mess.  (And once
there are users, we can't even rip out the code, since that would be a
user-visible regression.)

Occasionally, there is an accusation that different standards that
might apply for people who are in the "gold old buys club".  But what
isn't appreciated, is that it is precisely because people who are
long-term members of the community are trusted to stick around and
will support code that the have sponsored.  For example, Darrick Wong,
who contributed ext4 metadata checksuming support before we moved on
to become the XFS maintainer, is still reviewing and making
contributions to code that he contributed many years to ext4.  I've
been known to submit fixes to test bugs for xfs-specific tests in
xfstests that I discovered when running daily spinner tests on the
linux git tree.

Just in the past two weeks, I've spent 15+ hours working on cleaning
up a casefold "security fix" that had nasty on-disk backwards
compatibility issues.  Part of it was that it was ext4 (although I
discovered that there was also fsck.f2fs problems that had been
overlooked), but the *other* part of why I spent so much time was that
I sponsored the casefolding patches, and so I felt I certain moral
responsibility to make sure the code was continued to be maintained.

(And note, this has nothing to do with who employs me; the 15-20 hours
that I spent working on creating the fix and the test scripts was
purely on my own time, on the weekend, or late at night.  The time
that I spend doing this isn't reflected in my performance reviews, or
promotion propects --- in fact, arguably, over the past 15 years, this
has probably slowed down my promotion prospects; I've only been
promoted once since joining said big tech company...)

> Not sure what the fix is, from a project management perspective the
> technology industry has never faced a challenge like this.  The fork
> model, which was the classic protection in open-source, doesn't work
> at this scale.

Well, here's my suggestion.  Teams that want to get features,
especially ones that might be potentially disruptive, into the tree,
need to spend time becoming part of the *community*.  This means that
they need to participate in part of the joint effort to keep the code
maintainable and high quaity --- even if it isn't part of their
company's short-term goals, or directly related to their pet feature
that they are trying to get upstream.

This was the trust of my "Beyond Upstream First" presentation which I
gave to the Linux Foundation Member Summit last fall[1].

[1] https://docs.google.com/presentation/d/11rMc8PBeyMItV6hv31OHSZ626_6FCZxjX6ZxEAehCpQ/edit#slide=id.p

Now, I'll say that the Rust developers are making strides in the right
direction here.  Miguel has stood for election for the Technical
Adviory Board, and has been contributing in various ways beyond just
Rust for Linux.  Ultimately, Cristoph's concern is that Rust is going
to make life harder for maintainers because of particular build breaks
getting in the way of the very limited bandwidth that Maintainers have
(and again, a lot of the work that we do gets done on our own personal
time; it's rare that even those maintainers employed by a "big
technology company" have complete freedom to work on whatever they
want; and they certainly don't have minions employed to do all of the
grunt work to support code maintenance work, especially if we let
crappy code slip past us in the name of "time to market" concerns.)

So I think we'll get past this.  It might take some more effort, and
more trust building --- on both sides --- but I'm fairly optismistic
that it will happen.  It might not happe as *fast* as people might
like; as Linus pointed out, it took ten years for the Clang compiler
to be considered 100% fully supported --- and this was without needing
to worrying about language issues, including an upstream language
community which refuses to make any kind of backwards compatibility
guarantees, and which is *actively* hostile to a second Rust compiler
implementation (I suspect because it might limit their ability to make
arbitrary backwards-incompatble language changes).

> Obviously respect and open-mindedness to new ideas appears to be the
> grease that makes all of this run smoothly.  Unfortunately that seems
> to be about as rare a commodity as omniscience in our industry.

The other thing which is super rare is people and companies who care
about tech debt cleanup, code maintainability, and code quality.
Instead of complaining about maintainers for who are unreasonably
caring about these things, when they are desparately under-resourced
to do as good of a job as they industry demands, how about meeting us
half-way and *helping* us with these sort of long-term code health
issues?  Maybe if you engage us as part of the community, we'll be a
lot more open to adding changes that might increase the code
maintenance burden?

Cheers,

					- Ted

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Danilo Krummrich]

Ted,

On Sat, Feb 08, 2025 at 03:44:16PM -0500, Theodore Ts'o wrote:
> This was the trust of my "Beyond Upstream First" presentation which I
> gave to the Linux Foundation Member Summit last fall[1].
> 
> [1] https://docs.google.com/presentation/d/11rMc8PBeyMItV6hv31OHSZ626_6FCZxjX6ZxEAehCpQ/edit#slide=id.p
> 
> Now, I'll say that the Rust developers are making strides in the right
> direction here.

Please let me add that a lot of those Rust kernel developers also have been
kernel developers before there has been Rust in the kernel and do still
contribute and even maintain C components, drivers and / or subsystems.

I think missing trust should not be a relevent factor in this matter.

> Miguel has stood for election for the Technical
> Adviory Board, and has been contributing in various ways beyond just
> Rust for Linux.

Thanks for pointing this out.

- Danilo

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[comex]

On Feb 8, 2025, at 12:44 PM, Theodore Ts'o <tytso@mit.edu> wrote:
> 
> including an upstream language
> community which refuses to make any kind of backwards compatibility
> guarantees, and which is *actively* hostile to a second Rust compiler
> implementation (I suspect because it might limit their ability to make
> arbitrary backwards-incompatble language changes).

Rust does, of course, make all sorts of backward compatibility guarantees for stabilized features.

The list of unstable features used by Rust for Linux is still long, but getting shorter.

https://github.com/Rust-for-Linux/linux/issues/2

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[David Airlie]

On Sun, Feb 9, 2025 at 6:48 AM Theodore Ts'o <tytso@mit.edu> wrote:
>
> On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote:
> >
> > The all powerful sub-system maintainer model works well if the big
> > technology companies can employ omniscient individuals in these roles,
> > but those types are a bit hard to come by.
>
> I'll let you in a secret.  The maintainers are not "all-powerfui".  We
> are the "thin blue line" that is trying to keep the code to be
> maintainable and high quality.  Like most leaders of volunteer
> organization, whether it is the Internet Engineerint Task Force (the
> standards body for the Internet), we actually have very little power.
> We can not *command* people to work on retiring technical debt, or to
> improve testing infrastructure, or work on some particular feature
> that we'd very like for our users.

Just as a courtesy to others can we not use the "thin blue line"
analogy in this community, it has had some bad connotation thrown on
it in the US context over the past few years, and I'd rather not as a
maintainer be aligned with current connotation/interpretations of it,
despite having family involved in our local force.

I'm open to suggestions for any better analogies.

Dave.

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Simona Vetter]

On Thu, Feb 13, 2025 at 08:20:52PM +1000, David Airlie wrote:
> On Sun, Feb 9, 2025 at 6:48 AM Theodore Ts'o <tytso@mit.edu> wrote:
> >
> > On Fri, Feb 07, 2025 at 06:16:38AM -0600, Dr. Greg wrote:
> > >
> > > The all powerful sub-system maintainer model works well if the big
> > > technology companies can employ omniscient individuals in these roles,
> > > but those types are a bit hard to come by.
> >
> > I'll let you in a secret.  The maintainers are not "all-powerfui".  We
> > are the "thin blue line" that is trying to keep the code to be
> > maintainable and high quality.  Like most leaders of volunteer
> > organization, whether it is the Internet Engineerint Task Force (the
> > standards body for the Internet), we actually have very little power.
> > We can not *command* people to work on retiring technical debt, or to
> > improve testing infrastructure, or work on some particular feature
> > that we'd very like for our users.
> 
> Just as a courtesy to others can we not use the "thin blue line"
> analogy in this community, it has had some bad connotation thrown on
> it in the US context over the past few years, and I'd rather not as a
> maintainer be aligned with current connotation/interpretations of it,
> despite having family involved in our local force.

Agreed. I dropped a bit the ball on this, because at first I thought this
was only posted on part of the thread that wasn't cc'ed to dri-devel - I
can't pick a fight with the kernel community at large for everything that
happens. And then I let Dave handle this, because I couldn't come up with
anything that's not a nuclear table flip. 2 weeks later I'm still not
better, so let me instead express positively what kind of maintainership I
strive for by linking to an old talk of mine:

https://blog.ffwll.ch/2017/01/maintainers-dont-scale.html

> I'm open to suggestions for any better analogies.

Better analogy aside, I fundamentally disagree with understanding
maintainership as a gatekeeper role that exists to keep the chaos out. My
goal is to help build a community where people enjoy collaborating, and
then gtfo so I don't hinder them. I think the talk I linked above is
holding up quite well even years later, but the last part really embodies
that idea, so let me just quote that:

    Be a Steward, Not a Lord

    I think one of key advantages of open source is that people stick
    around for a very long time. Even when they switch jobs or move
    around. Maybe the usual “for life” qualifier isn’t really a great
    choice, since it sounds more like a mandatory sentence than something
    done by choice. What I object to is the “dictator” part, since if your
    goal is to grow a great community and maybe reach world domination,
    then you as the maintainer need to serve that community. And not that
    the community serves you.

Cheers, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Jason Gunthorpe]

On Thu, Feb 20, 2025 at 05:24:01PM +0100, Simona Vetter wrote:
> Better analogy aside, I fundamentally disagree with understanding
> maintainership as a gatekeeper role that exists to keep the chaos out. My
> goal is to help build a community where people enjoy collaborating, and
> then gtfo so I don't hinder them. I think the talk I linked above is
> holding up quite well even years later, but the last part really embodies
> that idea, so let me just quote that:
> 
>     Be a Steward, Not a Lord
> 
>     I think one of key advantages of open source is that people stick
>     around for a very long time. Even when they switch jobs or move
>     around. Maybe the usual “for life” qualifier isn’t really a great
>     choice, since it sounds more like a mandatory sentence than something
>     done by choice. What I object to is the “dictator” part, since if your
>     goal is to grow a great community and maybe reach world domination,
>     then you as the maintainer need to serve that community. And not that
>     the community serves you.

+1

I agree and try, as best I can, to embody this.

Jason

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Jarkko Sakkinen]

On Thu, 2025-02-20 at 12:37 -0400, Jason Gunthorpe wrote:
> >     I think one of key advantages of open source is that people
> > stick
> >     around for a very long time. Even when they switch jobs or move
> >     around. Maybe the usual “for life” qualifier isn’t really a
> > great
> >     choice, since it sounds more like a mandatory sentence than
> > something
> >     done by choice. What I object to is the “dictator” part, since
> > if your
> >     goal is to grow a great community and maybe reach world
> > domination,
> >     then you as the maintainer need to serve that community. And
> > not that
> >     the community serves you.
> 
> +1
> 
> I agree and try, as best I can, to embody this.

I keep four simple rules of email response list these days when
I response to LKML:

1. Be honest.
2. Address your concerns.
3. Ask for help where you need it.
4. Admit your possible misconceptions (getting things wrong is not a mistake).

It's quite easy actually shift away from these even if you
don't do it purposely. So I actually just read what I'm going
to respond and reflect to these :-) It's anyhow usually best
to keep a short break and breath-in/outs before triggering
send, right?

The fourth one addresses so called "toxic positive" responses
as addresses in the associated Mastodon thread [1] :-)

> Jason

[1] https://social.kernel.org/notice/ArH99Q2ErS20vJB7Tc

BR, Jarkko

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Ronja Meyer]

On 08.02.2025 21:44, Theodore Ts'o wrote:
> I'll let you in a secret.  The maintainers are not "all-powerfui".  We
> are the "thin blue line" that is trying to keep the code to be
> maintainable and high quality.  Like most leaders of volunteer
> organization, whether it is the Internet Engineerint Task Force (the
> standards body for the Internet), we actually have very little power.
 > […] structurelessness becomes a way of masking power, and […] is 
usually most strongly advocated by those who are the most powerful 
(whether they are conscious of their power or not). As long as the 
structure of the group is informal, the rules of how decisions are made 
are known only to a few and awareness of power is limited to those who 
know the rules. Those who do not know the rules and are not chosen for 
initiation must remain in confusion, or suffer from paranoid delusions 
that something is happening of which they are not quite aware. […]

 From "The Tyranny of Structurelessness", written by Jo Freeman in 1970. 
https://www.jofreeman.com/joreen/tyranny.htm

- Ronja

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[33KK]

Honestly, how about you accept the fact that maybe the problem is you?

You're not doing any good by just scolding Hector for "social media 
brigading"
and not addressing the core issue, especially considering that LKML 
drama always
ends up on social media and on reaction content farms anyway, without the
"brigading", and Hector wasn't the one who started the drama.

You can't expect people to just keep going on and on in circular 
conversations
on the mailing list with somebody who isn't even engaging in good faith. 
These
conversations are about as technical as high school bullying, with you 
being the
teacher who punishes the one who tried to speak out, because they didn't 
want to
deal with figuring out who is right or wrong, and the victim is trying their
best to work out a solution for an unsolvable problem, since there's no 
solution
that the other side would accept.

Obviously, the people involved are tired of having this shit show happen 
again and
again and of course they will end up going to vent on social media, 
break down,
or leave, since they're not heard on the mailing list, and their mental 
health,
time and energy is clearly not respected by any of the adults present in the
room, judging by the complete inaction on the part of Linux's leadership.

You can't just pretend that multiple people leaving because of this is not a
problem. You can't just have people burn their time, energy and mental 
health on
pointless "discussions" and end up burning out, breaking down or leaving 
instead
of getting anything done. You can't just keep having your maintainers 
break down
one after another. This is not normal. This is not healthy. The process 
isn't
"working".

Just letting these "discussions" keep playing out like this is terrible
leadership, and is clearly not working well for mental health of people 
trying
to get involved with Linux.

You either want the R4L project to succeed and have new maintainers want 
to get
involved with the project, and you tell the hostile C maintainers to 
shut up and
cope with it, or you want the R4L project to fail and constantly have these
pointless "discussions" which boil down to some out of touch maintainer 
bullying
someone about something that should've been resolved long ago.

And as far as the *barely* technical arguments in the thread go:

The code is clearly exactly where it should be, in rust/kernel/dma, 
where else
is this supposed to go? Duplicating the bindings in multiple drivers 
would just
make the situation much worse, since now you have to worry about breaking
multiple copies of the bindings. R4L also proposed that they will 
maintain the
bindings.

Is the minor inconvenience of another maintainer having to review breaking
changes to DMA interfaces or some similar process to prevent build breakages
such a big deal compared to the pretty significant benefits Rust 
provides? The
Rust-based drivers were developed very quickly and ended up being way 
extremely
stable with relatively little effort compared to writing the same driver 
in C.

I think the benefits are pretty clear, if you can write a driver without
spending most of the time barely managing to tracking memory ownership 
in your
head, making sure you're not misusing underdocumented kernel APIs, etc.
obviously you're going to be significantly more productive, and it's 
also easier
for a new maintainer to get involved, since it's harder to break something.
Also, when writing safe bindings you have to double-check the safety of 
the API,
which helps to notice unsoundness in the C interface itself.

 > The only reason Linux managed to survive so long is by not having 
internal
 > boundaries

The actual reason Linux managed to survive so long is because of people 
working
together. Nobody understands the entirety of the kernel source, 
maintenance of
subsystems is split between different people that work together. What 
Christoph
is doing here, is pretty much opposite of that. He is actively trying to 
prevent
R4L from writing bindings to DMA, which are crucial to the project. 
Effectively
saying that the project should just go away entirely.

Christoph pretty clearly has no valid argument here.


NOTE: I am not affiliated in any way with R4L or Linux maintainers, I'm 
just a
Linux user watching this shitshow unfold, and am very disappointed about how
it's being handled.

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Danilo Krummrich]

On Thu, Feb 06, 2025 at 06:19:42PM +0900, Hector Martin wrote:
> 
> If shaming on social media does not work, then tell me what does,

Most importantly be *consistent* with good technical arguments, calmly focus on
your actual matter rather than escalating any surrounding details.

Accept that sometimes things can't be reached directly, but additional work is
needed to change the preconditions.

Goals aren't reached by burning bridges, but by building them. Sometimes you
may not be able to build a bridge where you would like to. But you can still
look for alternative routes with and within the community.

Surely, it does take time and energy, but certainly there's no shortcut.

- Danilo

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/07 4:37, Danilo Krummrich wrote:
> On Thu, Feb 06, 2025 at 06:19:42PM +0900, Hector Martin wrote:
>>
>> If shaming on social media does not work, then tell me what does,
> 
> Most importantly be *consistent* with good technical arguments, calmly focus on
> your actual matter rather than escalating any surrounding details.
> 
> Accept that sometimes things can't be reached directly, but additional work is
> needed to change the preconditions.
> 
> Goals aren't reached by burning bridges, but by building them. Sometimes you
> may not be able to build a bridge where you would like to. But you can still
> look for alternative routes with and within the community.
> 
> Surely, it does take time and energy, but certainly there's no shortcut.

I'm not talking about individual technical problems. You cannot solve
social and community problems with technical arguments.

Yes, given enough patience and technical argumentation, you can get code
into the kernel, or maybe even get some existing code refactored. That
doesn't change the fact that the process is often quite terrible,
hostile to newcomers, demoralizing, and by wide consensus much worse
than practically any other FOSS project, even those of similar scope to
the kernel.

And what I see, is very little effort to improve that status quo, or at
least very little that yields any actual change that isn't just
band-aids (e.g. tooling like b4, which is nice and appreciated, but
doesn't fix any underlying issues). And that's not going to change no
matter how many long technical arguments we have on the MLs (or even off
MLs, since MLs are also not particularly good for this, and I've seen
multiple arguments only reach a resolution after being redirected to IRC).

But I've used up all my spoons for this, and clearly Linus doesn't think
there's a problem in this thread worth replying to other than myself, so
I'm giving up on fighting for any change or being part of the kernel
maintainer community. Whether the rest of the kernel community chooses
to continue to live in an ugly bubble or actually try to fix some of
these systemic issues, is up to them.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Konstantin Ryabitsev]

On Fri, Feb 07, 2025 at 05:16:28AM +0900, Hector Martin wrote:
> And what I see, is very little effort to improve that status quo, or at
> least very little that yields any actual change that isn't just
> band-aids (e.g. tooling like b4, which is nice and appreciated, but
> doesn't fix any underlying issues). And that's not going to change no
> matter how many long technical arguments we have on the MLs (or even off
> MLs, since MLs are also not particularly good for this, and I've seen
> multiple arguments only reach a resolution after being redirected to IRC).

From my perspective, there are several camps clashing when it comes to the
kernel development model. One is people who are (rightfully) pointing out that
using the mailing lists was fine 20 years ago, but the world of software
development has vastly moved on to forges.

The other camp is people who (also rightfully) point out that kernel
development has always been decentralized and we should resist all attempts to
get ourselves into a position where Linux is dependent on any single
Benevolent Entity (Github, Gitlab, LF, kernel.org, etc), because this would
give that entity too much political or commercial control or, at the very
least, introduce SPoFs.

At best, I can hope to make both camps grumpily agree to coexist.

I *am* very wary of Benevolent Entities, because we have too many very recent
examples of companies "realigning priorities" when political winds shift.
Programs and initiatives that have until recently been poster board examples
of progress and benevolence are shuttered and defunded. I am concerned that
we're only a couple of mood swings away from someone deciding that free
software should not be allowed to exist because it benefits America's foes.
Many of us remember all too well when large tech giants treated Linux as a
"cancer" to be opposed, and I can certainly see that idea easily re-entering
some Big Brain in Charge.

From my perspective, I would like to ensure that Linux development can
continue without a hard dependency on a single centralized forge -- whether
controlled by a large commercial entity, or even a standalone one that is
operated by kernel.org. It's becoming shockingly difficult to operate a public
resource on the web unless you're willing to put it behind a large commercial
CDN that will protect you from hostile bots (and if you do that, you're back
to depending on the whims of a Benevolent Entity).

We're trying to get lore.kernel.org to the point where it's like a global
messaging bus that is indexed and searchable. Currently, you mostly have to
send things to a mailing list for them to end up on lore, but it's gradually
becoming less and less the case. We're already bridging with bugzilla and we
should be able to bridge with forges soon, too (currently delayed again
because I'm scrambling to move kernel.org frontends away from Equinix). Who
knows, we may be actually leapfrogging the forge era of software development
straight into "AI" agents era -- but that remains to be seen.

Anyway, all of this is to say that I'm happy that you've found b4 useful, but
I wouldn't view it as a band-aid -- it's just a very small and email-centric
way to interact with the kernel lore.

-K

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/08 2:14, Konstantin Ryabitsev wrote:
> On Fri, Feb 07, 2025 at 05:16:28AM +0900, Hector Martin wrote:
>> And what I see, is very little effort to improve that status quo, or at
>> least very little that yields any actual change that isn't just
>> band-aids (e.g. tooling like b4, which is nice and appreciated, but
>> doesn't fix any underlying issues). And that's not going to change no
>> matter how many long technical arguments we have on the MLs (or even off
>> MLs, since MLs are also not particularly good for this, and I've seen
>> multiple arguments only reach a resolution after being redirected to IRC).
> 
> From my perspective, there are several camps clashing when it comes to the
> kernel development model. One is people who are (rightfully) pointing out that
> using the mailing lists was fine 20 years ago, but the world of software
> development has vastly moved on to forges.
> 
> The other camp is people who (also rightfully) point out that kernel
> development has always been decentralized and we should resist all attempts to
> get ourselves into a position where Linux is dependent on any single
> Benevolent Entity (Github, Gitlab, LF, kernel.org, etc), because this would
> give that entity too much political or commercial control or, at the very
> least, introduce SPoFs.
> 
> At best, I can hope to make both camps grumpily agree to coexist.
> 
> I *am* very wary of Benevolent Entities, because we have too many very recent
> examples of companies "realigning priorities" when political winds shift.
> Programs and initiatives that have until recently been poster board examples
> of progress and benevolence are shuttered and defunded. I am concerned that
> we're only a couple of mood swings away from someone deciding that free
> software should not be allowed to exist because it benefits America's foes.
> Many of us remember all too well when large tech giants treated Linux as a
> "cancer" to be opposed, and I can certainly see that idea easily re-entering
> some Big Brain in Charge.
> 
> From my perspective, I would like to ensure that Linux development can
> continue without a hard dependency on a single centralized forge -- whether
> controlled by a large commercial entity, or even a standalone one that is
> operated by kernel.org. It's becoming shockingly difficult to operate a public
> resource on the web unless you're willing to put it behind a large commercial
> CDN that will protect you from hostile bots (and if you do that, you're back
> to depending on the whims of a Benevolent Entity).
> 
> We're trying to get lore.kernel.org to the point where it's like a global
> messaging bus that is indexed and searchable. Currently, you mostly have to
> send things to a mailing list for them to end up on lore, but it's gradually
> becoming less and less the case. We're already bridging with bugzilla and we
> should be able to bridge with forges soon, too (currently delayed again
> because I'm scrambling to move kernel.org frontends away from Equinix). Who
> knows, we may be actually leapfrogging the forge era of software development
> straight into "AI" agents era -- but that remains to be seen.
> 
> Anyway, all of this is to say that I'm happy that you've found b4 useful, but
> I wouldn't view it as a band-aid -- it's just a very small and email-centric
> way to interact with the kernel lore.
> 

The centralization concern is valid, but there are technical solutions
to this, such as forge federation. It's possible to set up a forge
environment to be less of a SPoF, such as by ensuring all data is open
and archiveable to allow for migration and backup restore, even by third
parties (you can make practically ~all forge data public except for
login passwords, and we have email-based reset processes for those).
It's also possible to simply shard, and let different subsystems choose
their own forge infrastructure, so downtime has a more limited effect.

Meanwhile, for better or worse, much of Linux infra *is* centralized -
for example, the mailing lists themselves, and a lot of the Git hosting.

There's also the fact that there is enough support for Linux that
finding a Benevolent Entity to provide stuff like DDoS protection is not
particularly hard, and such services are easily replaceable should
priorities change (and indeed are the kind of service that can only be
provided by a large entity, whether paid or sponsored, by nature). Heck,
even I have a Benevolent Entity sponsoring a CDN frontend for Asahi
sites and installer images (bunny.net, they're nice, highly recommended).

At the end of the day, I do not believe a theoretical breakdown of Linux
infra would be a major long-term setback to Linux kernel development.
The kernel already survived the great BitKeeper to Git migration, and
this time there's no proprietary software to be yanked out from under
the kernel (at least assuming the forge of choice is FOSS and
self-hostable, which seems like a no-brainer given the concerns). As
long as everyone has the Git trees, infrastructure can be rebuilt. As
long as everyone has email and git, development can continue ad-hoc.
Sure, it would cause a blip, but it's not like we would switch to a
forge, the forge would go down for a few days, and it would not be the
end of the world. Just ask the freedesktop folks (a good case study on
how infra is hard, but not impossible, and even if imperfect, even if at
times frustrating, it still functions, and well enough to still be
wildly positive compared to not having that infra at all).

But I'm afraid you'll find much if not most of the true opposition to
forges is not technical, it is philosophical or preference-based (even
though it may be presented as technical opposition, sometimes to
intentionally mislead). This is, in fact, quite a mirror of the R4L
situation, where technical arguments ("show me you can write a real
driver") quickly lead to non-technical arguments when solutions are
proposed ("it's cancer").

I actually considered moving soc/apple development to a forge personally
in the near future (obviously not my call to make any more), and I was
fully expecting a pile of pushback, "because that's not how we do things
here". Who knows, I might have gotten a "fuck you, either you accept
email patches or I remove you from MAINTAINTERS" from Linus.

All that said, thanks for b4. I might have given up much earlier, not
due to flamewars but due to exhaustion with poor tooling, if it didn't
exist.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Konstantin Ryabitsev]

On Sat, Feb 08, 2025 at 03:02:11AM +0900, Hector Martin wrote:
> The centralization concern is valid, but there are technical solutions
> to this, such as forge federation. 

Unfortunately, forge federation is really only between Forgejo instances and
is still pretty nascent, afaict. The most promising development in
decentralized forges that I've seen is Radicle, but I'm yet to try it again
ever since they got collectively drunk on bitcoin coolaid a few years ago.

> Meanwhile, for better or worse, much of Linux infra *is* centralized -
> for example, the mailing lists themselves, and a lot of the Git hosting.

Yes, but it's at least resilient. If someone knocks out vger.kernel.org,
kernel development will still continue because maintainers are still cc'd
directly via email, and email is still the only widely adopted federated
platform that we have, with nothing else coming anywhere close.

> At the end of the day, I do not believe a theoretical breakdown of Linux
> infra would be a major long-term setback to Linux kernel development.

We know it's the case when kernel.org went off the air for months in 2011. :)
Let's keep it that way!

> But I'm afraid you'll find much if not most of the true opposition to
> forges is not technical, it is philosophical or preference-based (even
> though it may be presented as technical opposition, sometimes to
> intentionally mislead). This is, in fact, quite a mirror of the R4L
> situation, where technical arguments ("show me you can write a real
> driver") quickly lead to non-technical arguments when solutions are
> proposed ("it's cancer").
> 
> I actually considered moving soc/apple development to a forge personally
> in the near future (obviously not my call to make any more), and I was
> fully expecting a pile of pushback, "because that's not how we do things
> here". Who knows, I might have gotten a "fuck you, either you accept
> email patches or I remove you from MAINTAINTERS" from Linus.

It is my goal to be able to give subsystems a way to use forges without it
impacting how they interact with upstream or handle tree-wide changes. That
is, once I'm done moving things from one Benevolent Company to another.

> All that said, thanks for b4. I might have given up much earlier, not
> due to flamewars but due to exhaustion with poor tooling, if it didn't
> exist.

I'm happy to hear that.

Best wishes,
-K

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Neal Gompa]

On Friday, February 7, 2025 1:16:11 PM Eastern Standard Time Konstantin 
Ryabitsev wrote:
> On Sat, Feb 08, 2025 at 03:02:11AM +0900, Hector Martin wrote:
> > The centralization concern is valid, but there are technical solutions
> > to this, such as forge federation.
> 
> Unfortunately, forge federation is really only between Forgejo instances and
> is still pretty nascent, afaict. The most promising development in
> decentralized forges that I've seen is Radicle, but I'm yet to try it again
> ever since they got collectively drunk on bitcoin coolaid a few years ago.

Strictly speaking, this isn't true. Forgefed federation was tested very early 
on with Codeberg's Gitea fork (before it was called Forgejo) and Pagure with 
its Forgefed extension[1]. The initial design and development was across three 
forge systems (Vervis, Forgejo, and Pagure).

Even before that, Pagure has data portability built into its design in a way 
no other forge has today: all project metadata is stored as JSON data checked 
into specialized Git repos. That means it's trivial to mirror and archive 
projects to have redundant instances or failover instances to support whatever 
reliability and high availability guarantees you'd like.

And insofar as cross system contributions, Pagure supports making pull 
requests that pull from arbitrary Git servers (as long as the Pagure instance 
can connect to it) with its remote pull request feature. That has been a core 
feature for nearly the entire time it has existed.

It's not that it wasn't possible, it's just nobody cares. In the 11 years 
since Pagure was created, literally no other forge has even intimated that 
they want even *some* of the concepts Pagure has. Extensibility and 
portability are simply not important concepts to anyone anymore.

If it was, maybe more people would have been excited about Pagure, I'd have 
more contributors to that project, and there'd be more deployments out there 
showing it off.

It is what it is, I guess. ☹️

[1]: https://pagure.io/pagure-forgefed

> > Meanwhile, for better or worse, much of Linux infra *is* centralized -
> > for example, the mailing lists themselves, and a lot of the Git hosting.
> 
> Yes, but it's at least resilient. If someone knocks out vger.kernel.org,
> kernel development will still continue because maintainers are still cc'd
> directly via email, and email is still the only widely adopted federated
> platform that we have, with nothing else coming anywhere close.
> 

That's really a self-inflicted choice. Adoption is always a catch-22. If you 
really wanted federated forge development and have some aspect of 
decentralized development, you always can even with forges or other tools. 
It's just not something being considered or explored in the Linux kernel 
community, despite how much some folks dislike the email-based workflow.

> > At the end of the day, I do not believe a theoretical breakdown of Linux
> > infra would be a major long-term setback to Linux kernel development.
> 
> We know it's the case when kernel.org went off the air for months in 2011.
> :) Let's keep it that way!
> 
> > But I'm afraid you'll find much if not most of the true opposition to
> > forges is not technical, it is philosophical or preference-based (even
> > though it may be presented as technical opposition, sometimes to
> > intentionally mislead). This is, in fact, quite a mirror of the R4L
> > situation, where technical arguments ("show me you can write a real
> > driver") quickly lead to non-technical arguments when solutions are
> > proposed ("it's cancer").
> > 
> > I actually considered moving soc/apple development to a forge personally
> > in the near future (obviously not my call to make any more), and I was
> > fully expecting a pile of pushback, "because that's not how we do things
> > here". Who knows, I might have gotten a "fuck you, either you accept
> > email patches or I remove you from MAINTAINTERS" from Linus.
> 
> It is my goal to be able to give subsystems a way to use forges without it
> impacting how they interact with upstream or handle tree-wide changes. That
> is, once I'm done moving things from one Benevolent Company to another.
> 

Honestly, this is probably not possible. If a subsystem moves to a forge 
workflow, it pretty much means tree-wide changes need to work partially that 
way too.


-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On Sun, Feb 09, 2025 at 03:25:26AM -0500, Neal Gompa wrote:
> On Friday, February 7, 2025 1:16:11 PM Eastern Standard Time Konstantin 
> Ryabitsev wrote:

> > It is my goal to be able to give subsystems a way to use forges without it
> > impacting how they interact with upstream or handle tree-wide changes. That
> > is, once I'm done moving things from one Benevolent Company to another.

> Honestly, this is probably not possible. If a subsystem moves to a forge 
> workflow, it pretty much means tree-wide changes need to work partially that 
> way too.

We do actually have some people using forges already, for example the
SOF people do a bunch of their review on github and then turn that into
patch serieses which they send via the normal email route when they're
happy with them.  I think tree wide stuff flows in via back merges or
rebases, one of the benefits of delegation is that it's not my problem.
This all works perfectly well from my side, as far as I know it's fine
for the SOF people too.  It certainly doesn't seem insurmountable.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Neal Gompa]

On Mon, Feb 10, 2025 at 12:28 PM Mark Brown <broonie@kernel.org> wrote:
>
> On Sun, Feb 09, 2025 at 03:25:26AM -0500, Neal Gompa wrote:
> > On Friday, February 7, 2025 1:16:11 PM Eastern Standard Time Konstantin
> > Ryabitsev wrote:
>
> > > It is my goal to be able to give subsystems a way to use forges without it
> > > impacting how they interact with upstream or handle tree-wide changes. That
> > > is, once I'm done moving things from one Benevolent Company to another.
>
> > Honestly, this is probably not possible. If a subsystem moves to a forge
> > workflow, it pretty much means tree-wide changes need to work partially that
> > way too.
>
> We do actually have some people using forges already, for example the
> SOF people do a bunch of their review on github and then turn that into
> patch serieses which they send via the normal email route when they're
> happy with them.  I think tree wide stuff flows in via back merges or
> rebases, one of the benefits of delegation is that it's not my problem.
> This all works perfectly well from my side, as far as I know it's fine
> for the SOF people too.  It certainly doesn't seem insurmountable.

It might be working as long as a subsystem continues to allow
receiving patches via email. As soon as a subsystem decides to stop
doing that (which is absolutely their right given the model of
subsystem maintenance that the Linux project has), I think this will
break down very quickly.

I wonder which team will be the first one to do it. It's not a
question of if, but when.


-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Al Viro]

On Fri, Feb 14, 2025 at 02:10:57AM -0500, Neal Gompa wrote:
> On Mon, Feb 10, 2025 at 12:28 PM Mark Brown <broonie@kernel.org> wrote:
> >
> > On Sun, Feb 09, 2025 at 03:25:26AM -0500, Neal Gompa wrote:
> > > On Friday, February 7, 2025 1:16:11 PM Eastern Standard Time Konstantin
> > > Ryabitsev wrote:
> >
> > > > It is my goal to be able to give subsystems a way to use forges without it
> > > > impacting how they interact with upstream or handle tree-wide changes. That
> > > > is, once I'm done moving things from one Benevolent Company to another.
> >
> > > Honestly, this is probably not possible. If a subsystem moves to a forge
> > > workflow, it pretty much means tree-wide changes need to work partially that
> > > way too.
> >
> > We do actually have some people using forges already, for example the
> > SOF people do a bunch of their review on github and then turn that into
> > patch serieses which they send via the normal email route when they're
> > happy with them.  I think tree wide stuff flows in via back merges or
> > rebases, one of the benefits of delegation is that it's not my problem.
> > This all works perfectly well from my side, as far as I know it's fine
> > for the SOF people too.  It certainly doesn't seem insurmountable.
> 
> It might be working as long as a subsystem continues to allow
> receiving patches via email. As soon as a subsystem decides to stop
> doing that (which is absolutely their right given the model of
> subsystem maintenance that the Linux project has), I think this will
> break down very quickly.
> 
> I wonder which team will be the first one to do it. It's not a
> question of if, but when.

Then it will be a matter of patches affecting their subtree getting merged
through the mainline.  All there is to it...

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

On Fri, Feb 14, 2025 at 02:10:57AM -0500, Neal Gompa wrote:
> On Mon, Feb 10, 2025 at 12:28 PM Mark Brown <broonie@kernel.org> wrote:

> > We do actually have some people using forges already, for example the
> > SOF people do a bunch of their review on github and then turn that into
> > patch serieses which they send via the normal email route when they're
> > happy with them.  I think tree wide stuff flows in via back merges or
> > rebases, one of the benefits of delegation is that it's not my problem.
> > This all works perfectly well from my side, as far as I know it's fine
> > for the SOF people too.  It certainly doesn't seem insurmountable.

> It might be working as long as a subsystem continues to allow
> receiving patches via email. As soon as a subsystem decides to stop
> doing that (which is absolutely their right given the model of
> subsystem maintenance that the Linux project has), I think this will
> break down very quickly.

Eh, probably they'll just get bypassed for the affected patches if they
really just drop everything on the floor.  That's effectively what's
happening with SOF in that I take patches for it (which seems to work
for everyone, people do review stuff that comes in on the list).  That
tends to be what happens if people are unresponsive.  More likely
there'f be some bridging effort of some kind with pressure applied to
get substantial work done on the forge.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Linus Torvalds]

On Fri, 7 Feb 2025 at 10:02, Hector Martin <marcan@marcan.st> wrote:
>
> Meanwhile, for better or worse, much of Linux infra *is* centralized -
> for example, the mailing lists themselves, and a lot of the Git hosting.

The mailing lists are mostly on kernel.org, but the git hosting most
certainly is not centralized in any way.

The kernel.org git repositories used to be special in that I didn't
require signed tags for them, because I trusted the user maintenance.
But I was encouraging signed tags even back then, and once it got to
the point where most were signed anyway, I just made it a rule. So now
kernel.org isn't special even in that respect.

Now, kernel.org is very much _convenient_. And you see that in the
stats: of my pulls in the last year, 85% have been from kernel.org.
But that is very much because it is convenient, not because it's
centralized.

But that still leaves the 15% that aren't kernel.org.

Since I did the stats, in case anybody is interested, the top
non-kernel.org hosts for my pulls are github.com, git.samba.org,
gitlab.freedesktop.org, evilpiepirate.org, git.infradead.org and
git.lwn.net (and there's a handful of other ones in there).

(And while I did the stats just for *my* pulls, if you look at total
merges over-all, the non-korg repositories are actually at 20% - I
think my percentages are higher simply because I tend to pull from
mostly top-level maintainers, and those are even more likely to use a
kernel.org account).

More importtantly, not being centralized was very much a basic tenet
of git, so *if* git.kernel.org were to become problematic, it's very
easy to move git repositories anywhere else. Very much by design.

              Linus

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/08 3:33, Linus Torvalds wrote:
> On Fri, 7 Feb 2025 at 10:02, Hector Martin <marcan@marcan.st> wrote:
>>
>> Meanwhile, for better or worse, much of Linux infra *is* centralized -
>> for example, the mailing lists themselves, and a lot of the Git hosting.
> 
> The mailing lists are mostly on kernel.org, but the git hosting most
> certainly is not centralized in any way.
> 
> The kernel.org git repositories used to be special in that I didn't
> require signed tags for them, because I trusted the user maintenance.
> But I was encouraging signed tags even back then, and once it got to
> the point where most were signed anyway, I just made it a rule. So now
> kernel.org isn't special even in that respect.
> 
> Now, kernel.org is very much _convenient_. And you see that in the
> stats: of my pulls in the last year, 85% have been from kernel.org.
> But that is very much because it is convenient, not because it's
> centralized.
> 
> But that still leaves the 15% that aren't kernel.org.

For all intents and purposes, 85% centralized might as well be fully
centralized. That is, any downtime on kernel.org will affect the
community effectively the same as downtime on a true central SPOF would.

> More importtantly, not being centralized was very much a basic tenet
> of git, so *if* git.kernel.org were to become problematic, it's very
> easy to move git repositories anywhere else. Very much by design.

And this is why, over focus on *decentraliation*, I think we should be
focusing on *recoverability* and *data availability*.

There are two distinct scenarios. If kernel.org goes down for a while,
that screws with people's workflow. They can find alternatives, but it
will cause immediate, acute disruption, even while using "decentralized"
git. You still need to tell people where the new repos are, reconfigure
remotes, etc.. If the disruption is expected to be short-lived, a lot of
people will choose to just ride it out, rather than find alternatives,
or will only use alternatives in an ad-hoc manner where strictly
necessary. Nobody does a "true" complete migration to another Git
hosting when faced with a brief, temporary outage (heck, even just the
time it takes to push a Linux Git tree from scratch to somewhere new is
much longer than ideal, more so if your internet connection isn't the
best or the routing to the server in question isn't great). At most you
push whatever you're working on as needed somewhere else.

If kernel.org goes down permanently or long term (or becomes
problematic), that's when you start doing a full migration. And thanks
to the Git design, this is possible - not because it's decentralized per
se, but because everyone has local copies of almost everything, and they
can easily be restored onto a new service.

The same goes for a forge scenario. If it goes down briefly, you fix it,
and you can still use email to communicate with people ad hoc and send
git changes around through any other hosting. It's still Git at the end
of the day. Emailed patches and Git trees aren't going to become
unusable just because most or all of the regular development happens on
a forge.

And if it goes down permanently or long term, or becomes problematic,
what really matters isn't that it's "decentralized" as in federation.
It's that the data is available, in much the same way it is for
lore.kernel.org, and that people are actively backing it up (and it
isn't hard to arrange for a handful of folks/organizations to have up to
date mirrors, you don't need a Git-like scenario where literally
everyone has full repo dumps; heck, I have a big NAS at home, I'd
volunteer myself). Then, the service can be restored elsewhere when
needed. If the issue is with the specific forge software used, you can
always develop a migration plan to a different software (which, if the
risk is managed properly, shouldn't be a time-critical process anyway).

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Dr. David Alan Gilbert]

* Hector Martin (marcan@marcan.st) wrote:
> On 2025/02/08 2:14, Konstantin Ryabitsev wrote:
> > On Fri, Feb 07, 2025 at 05:16:28AM +0900, Hector Martin wrote:
> >> And what I see, is very little effort to improve that status quo, or at
> >> least very little that yields any actual change that isn't just
> >> band-aids (e.g. tooling like b4, which is nice and appreciated, but
> >> doesn't fix any underlying issues). And that's not going to change no
> >> matter how many long technical arguments we have on the MLs (or even off
> >> MLs, since MLs are also not particularly good for this, and I've seen
> >> multiple arguments only reach a resolution after being redirected to IRC).
> > 
> > From my perspective, there are several camps clashing when it comes to the
> > kernel development model. One is people who are (rightfully) pointing out that
> > using the mailing lists was fine 20 years ago, but the world of software
> > development has vastly moved on to forges.
> > 
> > The other camp is people who (also rightfully) point out that kernel
> > development has always been decentralized and we should resist all attempts to
> > get ourselves into a position where Linux is dependent on any single
> > Benevolent Entity (Github, Gitlab, LF, kernel.org, etc), because this would
> > give that entity too much political or commercial control or, at the very
> > least, introduce SPoFs.
> > 
> > At best, I can hope to make both camps grumpily agree to coexist.
> > 
> > I *am* very wary of Benevolent Entities, because we have too many very recent
> > examples of companies "realigning priorities" when political winds shift.
> > Programs and initiatives that have until recently been poster board examples
> > of progress and benevolence are shuttered and defunded. I am concerned that
> > we're only a couple of mood swings away from someone deciding that free
> > software should not be allowed to exist because it benefits America's foes.
> > Many of us remember all too well when large tech giants treated Linux as a
> > "cancer" to be opposed, and I can certainly see that idea easily re-entering
> > some Big Brain in Charge.
> > 
> > From my perspective, I would like to ensure that Linux development can
> > continue without a hard dependency on a single centralized forge -- whether
> > controlled by a large commercial entity, or even a standalone one that is
> > operated by kernel.org. It's becoming shockingly difficult to operate a public
> > resource on the web unless you're willing to put it behind a large commercial
> > CDN that will protect you from hostile bots (and if you do that, you're back
> > to depending on the whims of a Benevolent Entity).
> > 
> > We're trying to get lore.kernel.org to the point where it's like a global
> > messaging bus that is indexed and searchable. Currently, you mostly have to
> > send things to a mailing list for them to end up on lore, but it's gradually
> > becoming less and less the case. We're already bridging with bugzilla and we
> > should be able to bridge with forges soon, too (currently delayed again
> > because I'm scrambling to move kernel.org frontends away from Equinix). Who
> > knows, we may be actually leapfrogging the forge era of software development
> > straight into "AI" agents era -- but that remains to be seen.
> > 
> > Anyway, all of this is to say that I'm happy that you've found b4 useful, but
> > I wouldn't view it as a band-aid -- it's just a very small and email-centric
> > way to interact with the kernel lore.
> > 
> 
> The centralization concern is valid, but there are technical solutions
> to this, such as forge federation. It's possible to set up a forge
> environment to be less of a SPoF, such as by ensuring all data is open
> and archiveable to allow for migration and backup restore, even by third
> parties (you can make practically ~all forge data public except for
> login passwords, and we have email-based reset processes for those).
> It's also possible to simply shard, and let different subsystems choose
> their own forge infrastructure, so downtime has a more limited effect.
> 
> Meanwhile, for better or worse, much of Linux infra *is* centralized -
> for example, the mailing lists themselves, and a lot of the Git hosting.

Although, many of the subsystems have their own patchworks or other systems
anyway dotted in random places.

It's actually something I find quite hard, it might seem there is
*a* Linux contribution process, but if you do fixups or devices all over
rather than in one subsystem you end up tripping over the oddities
of each maintainer; then trying to figure out when they're prepared
to take a patch, or where to check for whether they've taken it,
or whether to expect it to turn up in -next can all be quite random.


<snip>

> - Hector

Dave

> 
> 
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/06 3:52, Simona Vetter wrote:
> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
>> Adding Linus
>>
>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>> thread, Miguel and the other Rust folks should just merge this series
>> once it is reviewed and ready, ignoring Christoph's overt attempt at
>> sabotaging the project. If Linus pulls it, what Christoph says doesn't
>> matter. If Linus doesn't pull it, the R4L project is essentially dead
>> until either Linus or Christoph make a move. Everything else is beating
>> around the bush.
>>
>> Rust folks: Please don't waste your time and mental cycles on drama like
>> this. It's not worth your time. Either Linus likes it, or he doesn't.
>> Everything else is distractions orchestrated by a subset of saboteur
>> maintainers who are trying to demoralize you until you give up, because
>> they know they're going to be on the losing side of history sooner or
>> later. No amount of sabotage from old entrenched maintainers is going to
>> stop the world from moving forward towards memory-safe languages.
>>
>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
>> will happen.
> 
> Yeah no.
> 
> https://chaos.social/@sima/113942119012147959
> 
> This was about you, because typing a proper answer takes a bit longer. It
> was also about your toots on fedi, like this:
> 
> https://social.treehouse.systems/@marcan/113941468353031993
> 
> And "haha it's only a joke" does not work with your public profile and following.
> 
> I do understand the frustration and temptation to just burn it all to the
> ground, head the call of the sirens, or maybe for me more pick up goat
> farming in the Swiss Alps. But you can't have it both and expect to also
> be part of and contribute to the same community. And personally I don't
> appreciate getting drenched in gasoline while I'm trying to quench flames
> on the ground.
> 
> And this isn't the first time or the second, by now it's a pretty clear
> pattern over some years. And with the first I could explain why you react
> like that and you had my full understanding, but eventually that runs a
> bit thin as an excuse.  Now I'm left with the unlikely explanation that
> you just like thundering in as the cavalry, fashionably late, maximally
> destructive, because it entertains the masses on fedi or reddit or
> wherever. I have no idea what you're trying to achieve here, I really
> don't get it, but I am for sure fed up dealing with the fallout.
> 
> Cheers, Sima

Since we're into complaining about social media posts on the list now,
I'm going to link one of yours:

https://chaos.social/@sima/113961496955043901

An R4L maintainer quit after multiple rounds of hostility, including
some completely unacceptable public comments bordering on downright
harassment from tytso, a situation which was widely publicized.

If, according to you, that's "shockingly smooth" with "no outsized
drama" other than "what I stirred up" (I guess you also missed the whole
LWN post about this thread before I posted anything on Fedi? That's how
I even found out about this), then I'm glad I'm disengaging with this
community. The Stockholm syndrome and utter denial is palpable.

I thought I could work with you on these issues; I don't know what
happened over the past year since we last had a private conversation
about this, but clearly you've changed your mind.

Thus, as you kindly requested of me in social media,

https://chaos.social/@sima/113961283260455876

I will now proceed to fuck off. Good luck.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/07 18:41, Hector Martin wrote:
> On 2025/02/06 3:52, Simona Vetter wrote:
>> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
>>> Adding Linus
>>>
>>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>>> thread, Miguel and the other Rust folks should just merge this series
>>> once it is reviewed and ready, ignoring Christoph's overt attempt at
>>> sabotaging the project. If Linus pulls it, what Christoph says doesn't
>>> matter. If Linus doesn't pull it, the R4L project is essentially dead
>>> until either Linus or Christoph make a move. Everything else is beating
>>> around the bush.
>>>
>>> Rust folks: Please don't waste your time and mental cycles on drama like
>>> this. It's not worth your time. Either Linus likes it, or he doesn't.
>>> Everything else is distractions orchestrated by a subset of saboteur
>>> maintainers who are trying to demoralize you until you give up, because
>>> they know they're going to be on the losing side of history sooner or
>>> later. No amount of sabotage from old entrenched maintainers is going to
>>> stop the world from moving forward towards memory-safe languages.
>>>
>>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
>>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
>>> will happen.
>>
>> Yeah no.
>>
>> https://chaos.social/@sima/113942119012147959
>>
>> This was about you, because typing a proper answer takes a bit longer. It
>> was also about your toots on fedi, like this:
>>
>> https://social.treehouse.systems/@marcan/113941468353031993
>>
>> And "haha it's only a joke" does not work with your public profile and following.
>>
>> I do understand the frustration and temptation to just burn it all to the
>> ground, head the call of the sirens, or maybe for me more pick up goat
>> farming in the Swiss Alps. But you can't have it both and expect to also
>> be part of and contribute to the same community. And personally I don't
>> appreciate getting drenched in gasoline while I'm trying to quench flames
>> on the ground.
>>
>> And this isn't the first time or the second, by now it's a pretty clear
>> pattern over some years. And with the first I could explain why you react
>> like that and you had my full understanding, but eventually that runs a
>> bit thin as an excuse.  Now I'm left with the unlikely explanation that
>> you just like thundering in as the cavalry, fashionably late, maximally
>> destructive, because it entertains the masses on fedi or reddit or
>> wherever. I have no idea what you're trying to achieve here, I really
>> don't get it, but I am for sure fed up dealing with the fallout.
>>
>> Cheers, Sima
> 
> Since we're into complaining about social media posts on the list now,
> I'm going to link one of yours:
> 
> https://chaos.social/@sima/113961496955043901
> 
> An R4L maintainer quit after multiple rounds of hostility, including
> some completely unacceptable public comments bordering on downright
> harassment from tytso, a situation which was widely publicized.
> 
> If, according to you, that's "shockingly smooth" with "no outsized
> drama" other than "what I stirred up" (I guess you also missed the whole
> LWN post about this thread before I posted anything on Fedi? That's how
> I even found out about this), then I'm glad I'm disengaging with this
> community. The Stockholm syndrome and utter denial is palpable.
> 
> I thought I could work with you on these issues; I don't know what
> happened over the past year since we last had a private conversation
> about this, but clearly you've changed your mind.
> 
> Thus, as you kindly requested of me in social media,
> 
> https://chaos.social/@sima/113961283260455876
> 
> I will now proceed to fuck off. Good luck.

Oh, and one last thing. I've heard that, in whatever backroom
conversations you've been having about this situation, there has
apparently been a belief, explicit or implicit, that I am in any way
employed to work on the Linux kernel.

Unlike most people in this thread, I don't enjoy the luxury of a cushy
tech job that pays me to deal with this community. I am supported
exclusively by donations, which incidentally, have been steadily
*de*creasing since the start of the Asahi Linux project. The project has
zero corporate sponsorship.

And I do believe the fact that essentially all high-level Linux kernel
maintainers and contributors are paid by corporations to do it is a
major factor that has caused this community to become wildly out of
touch with what it means to be a community FOSS project.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Greg KH]

On Fri, Feb 07, 2025 at 07:20:03PM +0900, Hector Martin wrote:
> And I do believe the fact that essentially all high-level Linux kernel
> maintainers and contributors are paid by corporations to do it is a
> major factor that has caused this community to become wildly out of
> touch with what it means to be a community FOSS project.

Please note, that ever since I started keeping track of this type of
thing, way back in 2003 or so, it has been the case that over 80% of the
contributions come from company-funded developers.  Which means it
really goes back before that as well.

And that's good, it means that we have backing to do this properly, from
the companies that benifit from it.  To not have that would make it much
harder for any of this to work properly at all.

So don't try to play the "you all work for companies" card, that isn't
going to fly as obviously we all speak for ourselves here, and our
companies _know_ they can't tell us what to do, but they give us insight
into the problems that they have with Linux in order for us to help
change it to make it better for everyone.

Because again, everyone has the same problems (individuals and
companies), and so solving it for one "group", solves it for everyone.

thanks,

greg k-h

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Simona Vetter]

On Fri, Feb 07, 2025 at 07:20:03PM +0900, Hector Martin wrote:
> On 2025/02/07 18:41, Hector Martin wrote:
> > On 2025/02/06 3:52, Simona Vetter wrote:
> >> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
> >>> Adding Linus
> >>>
> >>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
> >>> thread, Miguel and the other Rust folks should just merge this series
> >>> once it is reviewed and ready, ignoring Christoph's overt attempt at
> >>> sabotaging the project. If Linus pulls it, what Christoph says doesn't
> >>> matter. If Linus doesn't pull it, the R4L project is essentially dead
> >>> until either Linus or Christoph make a move. Everything else is beating
> >>> around the bush.
> >>>
> >>> Rust folks: Please don't waste your time and mental cycles on drama like
> >>> this. It's not worth your time. Either Linus likes it, or he doesn't.
> >>> Everything else is distractions orchestrated by a subset of saboteur
> >>> maintainers who are trying to demoralize you until you give up, because
> >>> they know they're going to be on the losing side of history sooner or
> >>> later. No amount of sabotage from old entrenched maintainers is going to
> >>> stop the world from moving forward towards memory-safe languages.
> >>>
> >>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
> >>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
> >>> will happen.
> >>
> >> Yeah no.
> >>
> >> https://chaos.social/@sima/113942119012147959
> >>
> >> This was about you, because typing a proper answer takes a bit longer. It
> >> was also about your toots on fedi, like this:
> >>
> >> https://social.treehouse.systems/@marcan/113941468353031993
> >>
> >> And "haha it's only a joke" does not work with your public profile and following.
> >>
> >> I do understand the frustration and temptation to just burn it all to the
> >> ground, head the call of the sirens, or maybe for me more pick up goat
> >> farming in the Swiss Alps. But you can't have it both and expect to also
> >> be part of and contribute to the same community. And personally I don't
> >> appreciate getting drenched in gasoline while I'm trying to quench flames
> >> on the ground.
> >>
> >> And this isn't the first time or the second, by now it's a pretty clear
> >> pattern over some years. And with the first I could explain why you react
> >> like that and you had my full understanding, but eventually that runs a
> >> bit thin as an excuse.  Now I'm left with the unlikely explanation that
> >> you just like thundering in as the cavalry, fashionably late, maximally
> >> destructive, because it entertains the masses on fedi or reddit or
> >> wherever. I have no idea what you're trying to achieve here, I really
> >> don't get it, but I am for sure fed up dealing with the fallout.
> >>
> >> Cheers, Sima
> > 
> > Since we're into complaining about social media posts on the list now,
> > I'm going to link one of yours:
> > 
> > https://chaos.social/@sima/113961496955043901
> > 
> > An R4L maintainer quit after multiple rounds of hostility, including
> > some completely unacceptable public comments bordering on downright
> > harassment from tytso, a situation which was widely publicized.
> > 
> > If, according to you, that's "shockingly smooth" with "no outsized
> > drama" other than "what I stirred up" (I guess you also missed the whole
> > LWN post about this thread before I posted anything on Fedi? That's how
> > I even found out about this), then I'm glad I'm disengaging with this
> > community. The Stockholm syndrome and utter denial is palpable.

Yes, the kernel's conduct is still not where I think it needs to be. And
there are still cases occuring with some regularity which would be
unacceptable on dri-devel and sanctioned under the freedesktop.org code of
conduct. But that's not really new, it's not really specific to the r4l
project, and despite that it's still not good enough, it's definitely much
better than 10 years ago.

That all aside, I put the "outsized" very intentionally in my comment. It
carries a lot more than just the above paragraph in just one small word.

> > I thought I could work with you on these issues; I don't know what
> > happened over the past year since we last had a private conversation
> > about this, but clearly you've changed your mind.

I got seriously annoyed with you about two years ago for the first time,
but kept chatting for quite a while longer. Nothing changed. Like I said,
I'm not complaining about this specific case here, but about a pattern
that you've established consistently over a fair amount of time.

And if you're wondering why you didn't realize this: It's impossible to
change people by telling them they're wrong. What does tend to work is
explaining different perspectives, so that they can figure it out
themselves. And sometimes that's just way too subtle to ever register.

> > Thus, as you kindly requested of me in social media,
> > 
> > https://chaos.social/@sima/113961283260455876
> > 
> > I will now proceed to fuck off. Good luck.

I wasn't fully awake yet this morning, but I have no issue standing behind
that thread in it's entirety. Lightly edited for typos below, with one
sentence added for a bit more context:

This isn't about "being civil" or "friendly" and even less about "don't
call out". This is about calling out in an ineffective and imprecise way,
so that the people actually trying to change things are busy patching up
your collateral damage instead of implementing real changes, while all
you've achieved is airing your frustration for internet drama points.

Aside from that I've learned through private feedback since my first mail
here that your social media brigading has had much more of a chilling
effect on contributors than I assumed, which is not acceptable.

When you're that harmful with your calling out, eventually I'm going to be
fed up, and you get a live round shot across your bow. And if that then
causes you to ragequit, because you can't actually deal with the heat
you've been dishing out coming back around the corner:

Fuck off.

Or as Dave Airlie put it upthread in this conversation:

"Being toxic on the right side of an argument is still toxic, [...]"

So please do call out broken things, do change things, I've been doing it
for well over a decade in the linux kernel. But do it in a way that you're
not just becoming part of the problem and making it bigger.

> Oh, and one last thing. I've heard that, in whatever backroom
> conversations you've been having about this situation, there has
> apparently been a belief, explicit or implicit, that I am in any way
> employed to work on the Linux kernel.
> 
> Unlike most people in this thread, I don't enjoy the luxury of a cushy
> tech job that pays me to deal with this community. I am supported
> exclusively by donations, which incidentally, have been steadily
> *de*creasing since the start of the Asahi Linux project. The project has
> zero corporate sponsorship.
> 
> And I do believe the fact that essentially all high-level Linux kernel
> maintainers and contributors are paid by corporations to do it is a
> major factor that has caused this community to become wildly out of
> touch with what it means to be a community FOSS project.

I'm really not in favor of chiding people for how they earn money, because
this world isn't fair and critizing people for how they get their bills
paid is generally not ok. But this is a bit too much to just let it slide.

It's fairly plausible that you need the social media brigading to generate
attention that you can convert into enough donations to support the asahi
project. But someone has to clean up the mess your shitstorms create, it's
sure not you, which means my and other people's mental health essentially
pay your bills. And I really do not appreciate that dynamic.

Cheers, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Hector Martin]

On 2025/02/07 22:49, Simona Vetter wrote:
> On Fri, Feb 07, 2025 at 07:20:03PM +0900, Hector Martin wrote:
>> On 2025/02/07 18:41, Hector Martin wrote:
>>> On 2025/02/06 3:52, Simona Vetter wrote:
>>>> On Tue, Feb 04, 2025 at 03:46:14AM +0900, Hector Martin wrote:
>>>>> Adding Linus
>>>>>
>>>>> My 2c: If Linus doesn't pipe up with an authoritative answer to this
>>>>> thread, Miguel and the other Rust folks should just merge this series
>>>>> once it is reviewed and ready, ignoring Christoph's overt attempt at
>>>>> sabotaging the project. If Linus pulls it, what Christoph says doesn't
>>>>> matter. If Linus doesn't pull it, the R4L project is essentially dead
>>>>> until either Linus or Christoph make a move. Everything else is beating
>>>>> around the bush.
>>>>>
>>>>> Rust folks: Please don't waste your time and mental cycles on drama like
>>>>> this. It's not worth your time. Either Linus likes it, or he doesn't.
>>>>> Everything else is distractions orchestrated by a subset of saboteur
>>>>> maintainers who are trying to demoralize you until you give up, because
>>>>> they know they're going to be on the losing side of history sooner or
>>>>> later. No amount of sabotage from old entrenched maintainers is going to
>>>>> stop the world from moving forward towards memory-safe languages.
>>>>>
>>>>> FWIW, in my opinion, the "cancer" comment from Christoph would be enough
>>>>> to qualify for Code-of-Conduct action, but I doubt anything of the sort
>>>>> will happen.
>>>>
>>>> Yeah no.
>>>>
>>>> https://chaos.social/@sima/113942119012147959
>>>>
>>>> This was about you, because typing a proper answer takes a bit longer. It
>>>> was also about your toots on fedi, like this:
>>>>
>>>> https://social.treehouse.systems/@marcan/113941468353031993
>>>>
>>>> And "haha it's only a joke" does not work with your public profile and following.
>>>>
>>>> I do understand the frustration and temptation to just burn it all to the
>>>> ground, head the call of the sirens, or maybe for me more pick up goat
>>>> farming in the Swiss Alps. But you can't have it both and expect to also
>>>> be part of and contribute to the same community. And personally I don't
>>>> appreciate getting drenched in gasoline while I'm trying to quench flames
>>>> on the ground.
>>>>
>>>> And this isn't the first time or the second, by now it's a pretty clear
>>>> pattern over some years. And with the first I could explain why you react
>>>> like that and you had my full understanding, but eventually that runs a
>>>> bit thin as an excuse.  Now I'm left with the unlikely explanation that
>>>> you just like thundering in as the cavalry, fashionably late, maximally
>>>> destructive, because it entertains the masses on fedi or reddit or
>>>> wherever. I have no idea what you're trying to achieve here, I really
>>>> don't get it, but I am for sure fed up dealing with the fallout.
>>>>
>>>> Cheers, Sima
>>>
>>> Since we're into complaining about social media posts on the list now,
>>> I'm going to link one of yours:
>>>
>>> https://chaos.social/@sima/113961496955043901
>>>
>>> An R4L maintainer quit after multiple rounds of hostility, including
>>> some completely unacceptable public comments bordering on downright
>>> harassment from tytso, a situation which was widely publicized.
>>>
>>> If, according to you, that's "shockingly smooth" with "no outsized
>>> drama" other than "what I stirred up" (I guess you also missed the whole
>>> LWN post about this thread before I posted anything on Fedi? That's how
>>> I even found out about this), then I'm glad I'm disengaging with this
>>> community. The Stockholm syndrome and utter denial is palpable.
> 
> Yes, the kernel's conduct is still not where I think it needs to be. And
> there are still cases occuring with some regularity which would be
> unacceptable on dri-devel and sanctioned under the freedesktop.org code of
> conduct. But that's not really new, it's not really specific to the r4l
> project, and despite that it's still not good enough, it's definitely much
> better than 10 years ago.
> 
> That all aside, I put the "outsized" very intentionally in my comment. It
> carries a lot more than just the above paragraph in just one small word.
> 
>>> I thought I could work with you on these issues; I don't know what
>>> happened over the past year since we last had a private conversation
>>> about this, but clearly you've changed your mind.
> 
> I got seriously annoyed with you about two years ago for the first time,
> but kept chatting for quite a while longer. Nothing changed. Like I said,
> I'm not complaining about this specific case here, but about a pattern
> that you've established consistently over a fair amount of time.
> 
> And if you're wondering why you didn't realize this: It's impossible to
> change people by telling them they're wrong. What does tend to work is
> explaining different perspectives, so that they can figure it out
> themselves. And sometimes that's just way too subtle to ever register.
> 
>>> Thus, as you kindly requested of me in social media,
>>>
>>> https://chaos.social/@sima/113961283260455876
>>>
>>> I will now proceed to fuck off. Good luck.
> 
> I wasn't fully awake yet this morning, but I have no issue standing behind
> that thread in it's entirety. Lightly edited for typos below, with one
> sentence added for a bit more context:
> 
> This isn't about "being civil" or "friendly" and even less about "don't
> call out". This is about calling out in an ineffective and imprecise way,
> so that the people actually trying to change things are busy patching up
> your collateral damage instead of implementing real changes, while all
> you've achieved is airing your frustration for internet drama points.
> 
> Aside from that I've learned through private feedback since my first mail
> here that your social media brigading has had much more of a chilling
> effect on contributors than I assumed, which is not acceptable.
> 
> When you're that harmful with your calling out, eventually I'm going to be
> fed up, and you get a live round shot across your bow. And if that then
> causes you to ragequit, because you can't actually deal with the heat
> you've been dishing out coming back around the corner:
> 
> Fuck off.
> 
> Or as Dave Airlie put it upthread in this conversation:
> 
> "Being toxic on the right side of an argument is still toxic, [...]"
> 
> So please do call out broken things, do change things, I've been doing it
> for well over a decade in the linux kernel. But do it in a way that you're
> not just becoming part of the problem and making it bigger.
> 
>> Oh, and one last thing. I've heard that, in whatever backroom
>> conversations you've been having about this situation, there has
>> apparently been a belief, explicit or implicit, that I am in any way
>> employed to work on the Linux kernel.
>>
>> Unlike most people in this thread, I don't enjoy the luxury of a cushy
>> tech job that pays me to deal with this community. I am supported
>> exclusively by donations, which incidentally, have been steadily
>> *de*creasing since the start of the Asahi Linux project. The project has
>> zero corporate sponsorship.
>>
>> And I do believe the fact that essentially all high-level Linux kernel
>> maintainers and contributors are paid by corporations to do it is a
>> major factor that has caused this community to become wildly out of
>> touch with what it means to be a community FOSS project.
> 
> I'm really not in favor of chiding people for how they earn money, because
> this world isn't fair and critizing people for how they get their bills
> paid is generally not ok. But this is a bit too much to just let it slide.
> 
> It's fairly plausible that you need the social media brigading to generate
> attention that you can convert into enough donations to support the asahi
> project. But someone has to clean up the mess your shitstorms create, it's
> sure not you, which means my and other people's mental health essentially
> pay your bills. And I really do not appreciate that dynamic.
> 

If I were trying to get more funds, I'd be spending my time finding
corporate sponsors, not documenting the failings of the Linux kernel
community in public. Believe it or not, no, the latter doesn't get me
more donations.

Given you have now decided to add "baseless public accusations from a
position of power" to the list of abuses perpetrated by Linux kernel
maintainers, I think it's about time we stop having this conversation.

- Hector

Re: On community influencing (was Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.)

[Simona Vetter]

On Fri, Feb 07, 2025 at 02:49:50PM +0100, Simona Vetter wrote:
> When you're that harmful with your calling out, eventually I'm going to be
> fed up, and you get a live round shot across your bow. And if that then
> causes you to ragequit, because you can't actually deal with the heat
> you've been dishing out coming back around the corner:
> 
> Fuck off.

With some sleep, a w/e and slightly less ill temper: As I'd say in Swiss
German, this was suboptimal wording. Or in plain English, a disaster, I
screwed up.

I do not rescind the sentiment I expressed though. I cannot stand people
who punch where they're stronger, and eschew conflict and fold the moment
someone hits back.

> Or as Dave Airlie put it upthread in this conversation:
> 
> "Being toxic on the right side of an argument is still toxic, [...]"

The irony has not entirely escaped me. Oh well.

Remorsefully, Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Carlos Bilbao]

Hello,

On 1/8/25 09:16, Miguel Ojeda wrote:
> On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
>>
>> No rust code in kernel/dma, please.
> 
> What do you suggest?

This is it. What do people suggest? This thread has received a lot of
attention -- maybe it's an opportunity for the community to brainstorm.
Here's an idea:

Some maintainers clearly hate wrappers/bindings to Rust, while others don't
mind as long as the R4L folks commit to the maintenance duties. Depending on
the subsystem, it will be a completely different conversation. It seems to
me that right now, every time the R4L folks interact with a new subsystem,
they need to understand the maintainer's stance. That looks like an
exhausting process.

Would it make sense to have a C middleman that Rust calls instead of
binding directly to the C funcs? This dispatcher would provide a simpler,
stable API with fixed function signatures, even if the C functions change.
If a C func is removed/changed, only the C dispatcher would need to be
adapted, but until that happens, a new error could be returned to the Rust
side (not ideal, but Rust doesn't just break). 

This would obviously impose some limitations on the Rust side, but it might
be less conflictive than direct bindings. Also, in Abdiel's case here (for
example), he'd explicitly take on the maintenance of the Rust abstraction
and its DMA dispatchers, leaving no ambiguity about ownership.

> 
> Thanks!
> 
> Cheers,
> Miguel
> 
> 

Thanks,
Carlos

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[David Airlie]

On Sun, Feb 9, 2025 at 9:55 AM Carlos Bilbao <bilbao@vt.edu> wrote:
>
> Hello,
>
> On 1/8/25 09:16, Miguel Ojeda wrote:
> > On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
> >>
> >> No rust code in kernel/dma, please.
> >
> > What do you suggest?
>
> This is it. What do people suggest? This thread has received a lot of
> attention -- maybe it's an opportunity for the community to brainstorm.
> Here's an idea:
>
> Some maintainers clearly hate wrappers/bindings to Rust, while others don't
> mind as long as the R4L folks commit to the maintenance duties. Depending on
> the subsystem, it will be a completely different conversation. It seems to
> me that right now, every time the R4L folks interact with a new subsystem,
> they need to understand the maintainer's stance. That looks like an
> exhausting process.
>

That is the process we are committed to, everyone has a voice and gets
to be heard, and we move forward in the appropriate manner with each
maintainer. Bypassing maintainers is the last resort, not engaging at
all with maintainers isn't a productive way forward either. Your below
attempt is trying to create a technical solution to a social problem.

> Would it make sense to have a C middleman that Rust calls instead of
> binding directly to the C funcs? This dispatcher would provide a simpler,
> stable API with fixed function signatures, even if the C functions change.
> If a C func is removed/changed, only the C dispatcher would need to be
> adapted, but until that happens, a new error could be returned to the Rust
> side (not ideal, but Rust doesn't just break).
>
> This would obviously impose some limitations on the Rust side, but it might
> be less conflictive than direct bindings. Also, in Abdiel's case here (for
> example), he'd explicitly take on the maintenance of the Rust abstraction
> and its DMA dispatchers, leaving no ambiguity about ownership.
>

This just adds overheads for everyone to little advantage for the
cases where maintainers are fine with all the other approaches.

Also the bindings code in rust is usually pretty trivial to change,
it's not like you can really abstract things that far, changing APIs
is hard no matter what, understanding the nuances in every C driver
can takes months of work adding rust bindings to your changes is
rarely going to be the limiting factor. We have learned over the years
to not make API changes that are subtle or hidden, and can leave out
of tree or in development users broken in subtle ways and as long as
we keep making API changes like that, updating the rust code is the
least of anyone's worries. This isn't like having to learn Mandarin vs
English, it's code written with a different syntax, it's not even
perl.

Dave.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Carlos Bilbao]

Hello David,

On 2/9/25 00:44, David Airlie wrote:
> On Sun, Feb 9, 2025 at 9:55 AM Carlos Bilbao <bilbao@vt.edu> wrote:
>>
>> Hello,
>>
>> On 1/8/25 09:16, Miguel Ojeda wrote:
>>> On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> No rust code in kernel/dma, please.
>>>
>>> What do you suggest?
>>
>> This is it. What do people suggest? This thread has received a lot of
>> attention -- maybe it's an opportunity for the community to brainstorm.
>> Here's an idea:
>>
>> Some maintainers clearly hate wrappers/bindings to Rust, while others don't
>> mind as long as the R4L folks commit to the maintenance duties. Depending on
>> the subsystem, it will be a completely different conversation. It seems to
>> me that right now, every time the R4L folks interact with a new subsystem,
>> they need to understand the maintainer's stance. That looks like an
>> exhausting process.
>>
> 
> That is the process we are committed to, everyone has a voice and gets
> to be heard, and we move forward in the appropriate manner with each
> maintainer. Bypassing maintainers is the last resort, not engaging at
> all with maintainers isn't a productive way forward either. Your below
> attempt is trying to create a technical solution to a social problem.
> 
>> Would it make sense to have a C middleman that Rust calls instead of
>> binding directly to the C funcs? This dispatcher would provide a simpler,
>> stable API with fixed function signatures, even if the C functions change.
>> If a C func is removed/changed, only the C dispatcher would need to be
>> adapted, but until that happens, a new error could be returned to the Rust
>> side (not ideal, but Rust doesn't just break).
>>
>> This would obviously impose some limitations on the Rust side, but it might
>> be less conflictive than direct bindings. Also, in Abdiel's case here (for
>> example), he'd explicitly take on the maintenance of the Rust abstraction
>> and its DMA dispatchers, leaving no ambiguity about ownership.
>>
> 
> This just adds overheads for everyone to little advantage for the
> cases where maintainers are fine with all the other approaches.

I'm not sure there's a need to worry about overhead in this hypothetical
case if the dispatcher is merely a translator and doesn't validate inputs,
copy buffers, or such. It would be less than calling from userspace.

What I meant to say (I wasn't clear before) was for this solution to apply
to cases where maintainers are firmly against all alternatives. Obviously,
if all the parties involved are happy, direct bindings are preferable.


> 
> Also the bindings code in rust is usually pretty trivial to change,
> it's not like you can really abstract things that far, changing APIs
> is hard no matter what, understanding the nuances in every C driver
> can takes months of work adding rust bindings to your changes is
> rarely going to be the limiting factor. We have learned over the years
> to not make API changes that are subtle or hidden, and can leave out
> of tree or in development users broken in subtle ways and as long as
> we keep making API changes like that, updating the rust code is the
> least of anyone's worries. This isn't like having to learn Mandarin vs
> English, it's code written with a different syntax, it's not even
> perl.

I'm not sure there's a need to worry about overhead in this hypothetical
case if the dispatcher is merely a translator and doesn't validate inputs,
copy buffers, or such. It would be less than calling from userspace.

What I meant to say (and wasn't clear before) is that this solution could
apply to cases where maintainers are firmly against all better
alternatives. Of course, if all the parties involved are happy with direct
bindings, that's preferable. But as you said, this is a social problem, so
maybe it's not a one-size-fits-all solution.

I agree that in some cases, abstracting the API could be complicated, but
I’d focus on keeping it simple and providing Rust with the minimum subset
it needs to keep things moving forward. Abdiel's PR is a good example --
by keeping the functions in his CoherentAllocation, we keep that option,
but lose the granularity of DMA mapping attributes (which would be some
default in the C dispatcher).

I agree that making subtle API changes is risky, and it would def. be
something to keep an eye on.

And yes, with this approach, some Rust developers may eventually miss
certain DMA functionalities in the future; but that would still be better
than the current situation, where _all_ Rust devs are missing all DMA
capabilities.

> 
> Dave.
> 

Thanks,
Carlos

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Carlos Bilbao]

On 2/9/25 10:19, Carlos Bilbao wrote:

> Hello David,
>
> On 2/9/25 00:44, David Airlie wrote:
>> On Sun, Feb 9, 2025 at 9:55 AM Carlos Bilbao <bilbao@vt.edu> wrote:
>>> Hello,
>>>
>>> On 1/8/25 09:16, Miguel Ojeda wrote:
>>>> On Wed, Jan 8, 2025 at 3:00 PM Christoph Hellwig <hch@lst.de> wrote:
>>>>> No rust code in kernel/dma, please.
>>>> What do you suggest?
>>> This is it. What do people suggest? This thread has received a lot of
>>> attention -- maybe it's an opportunity for the community to brainstorm.
>>> Here's an idea:
>>>
>>> Some maintainers clearly hate wrappers/bindings to Rust, while others don't
>>> mind as long as the R4L folks commit to the maintenance duties. Depending on
>>> the subsystem, it will be a completely different conversation. It seems to
>>> me that right now, every time the R4L folks interact with a new subsystem,
>>> they need to understand the maintainer's stance. That looks like an
>>> exhausting process.
>>>
>> That is the process we are committed to, everyone has a voice and gets
>> to be heard, and we move forward in the appropriate manner with each
>> maintainer. Bypassing maintainers is the last resort, not engaging at
>> all with maintainers isn't a productive way forward either. Your below
>> attempt is trying to create a technical solution to a social problem.
>>
>>> Would it make sense to have a C middleman that Rust calls instead of
>>> binding directly to the C funcs? This dispatcher would provide a simpler,
>>> stable API with fixed function signatures, even if the C functions change.
>>> If a C func is removed/changed, only the C dispatcher would need to be
>>> adapted, but until that happens, a new error could be returned to the Rust
>>> side (not ideal, but Rust doesn't just break).
>>>
>>> This would obviously impose some limitations on the Rust side, but it might
>>> be less conflictive than direct bindings. Also, in Abdiel's case here (for
>>> example), he'd explicitly take on the maintenance of the Rust abstraction
>>> and its DMA dispatchers, leaving no ambiguity about ownership.
>>>
>> This just adds overheads for everyone to little advantage for the
>> cases where maintainers are fine with all the other approaches.
> I'm not sure there's a need to worry about overhead in this hypothetical
> case if the dispatcher is merely a translator and doesn't validate inputs,
> copy buffers, or such. It would be less than calling from userspace.
>
> What I meant to say (I wasn't clear before) was for this solution to apply
> to cases where maintainers are firmly against all alternatives. Obviously,
> if all the parties involved are happy, direct bindings are preferable.
>
>
>> Also the bindings code in rust is usually pretty trivial to change,
>> it's not like you can really abstract things that far, changing APIs
>> is hard no matter what, understanding the nuances in every C driver
>> can takes months of work adding rust bindings to your changes is
>> rarely going to be the limiting factor. We have learned over the years
>> to not make API changes that are subtle or hidden, and can leave out
>> of tree or in development users broken in subtle ways and as long as
>> we keep making API changes like that, updating the rust code is the
>> least of anyone's worries. This isn't like having to learn Mandarin vs
>> English, it's code written with a different syntax, it's not even
>> perl.
> I'm not sure there's a need to worry about overhead in this hypothetical
> case if the dispatcher is merely a translator and doesn't validate inputs,
> copy buffers, or such. It would be less than calling from userspace.
>
> What I meant to say (and wasn't clear before) is that this solution could
> apply to cases where maintainers are firmly against all better
> alternatives. Of course, if all the parties involved are happy with direct
> bindings, that's preferable. But as you said, this is a social problem, so
> maybe it's not a one-size-fits-all solution.


I accidentally copied this twice after rephrasing, not trying to be
annoying x).


>
> I agree that in some cases, abstracting the API could be complicated, but
> I’d focus on keeping it simple and providing Rust with the minimum subset
> it needs to keep things moving forward. Abdiel's PR is a good example --
> by keeping the functions in his CoherentAllocation, we keep that option,
> but lose the granularity of DMA mapping attributes (which would be some
> default in the C dispatcher).
>
> I agree that making subtle API changes is risky, and it would def. be
> something to keep an eye on.
>
> And yes, with this approach, some Rust developers may eventually miss
> certain DMA functionalities in the future; but that would still be better
> than the current situation, where _all_ Rust devs are missing all DMA
> capabilities.
>
>> Dave.
>>
> Thanks,
> Carlos

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Daniel Sedlak]

On 1/8/25 1:27 PM, Abdiel Janulgue wrote:
> +/// Possible attributes associated with a DMA mapping.
> +///
> +/// They can be combined with the operators `|`, `&`, and `!`.
> +///
> +/// Values can be used from the [`attrs`] module.
> +#[derive(Clone, Copy, PartialEq)]
> +pub struct Attrs(pub u32);

Shouldn't this be non-pub? It seems unfortunate to leak C representation 
to the Rust world and allow its construction without going through 
constructor method. Also #[repr(transparent)] may be well suited? [1]

Link: https://doc.rust-lang.org/nomicon/other-reprs.html#reprtransparent 
[1]
> +
> +impl Attrs {
> +    /// Get the raw representation of this attribute.
> +    pub(crate) fn as_raw(self) -> usize {
> +        self.0.try_into().unwrap()
> +    }

Shouldn't be the return type u32 instead of usize? Or at least to my 
understanding the raw means C representation which should be the type 
that bindgen generates.
> +
> +    /// Check whether `flags` is contained in `self`.
> +    pub fn contains(self, flags: Attrs) -> bool {
> +        (self & flags) == flags
> +    }
> +}
Daniel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Daniel Almeida]

Hi Abdiel,

I was trying to test this again, but it apparently doesn’t compile for the reasons
Daniel Sedlak pointed out above.


Send a new version and I’ll test it for you.

— Daniel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Abdiel Janulgue]

On 08/01/2025 21:09, Daniel Almeida wrote:
> I was trying to test this again, but it apparently doesn’t compile for the reasons
> Daniel Sedlak pointed out above.
> 

Okay, now I noticed the problem.

bindgen generates this rust signature for dma_alloc_attrs():

extern "C" {
     pub fn dma_alloc_attrs(
         dev: *mut device,
         size: usize,
         dma_handle: *mut dma_addr_t,
         flag: gfp_t,
         attrs: ffi::c_ulong,
     ) -> *mut ffi::c_void;
}

We are interested in the last argument 'attrs'. For some reason it seems 
ffi::c_ulong is defined as usize at least on top of `rust-next` commit 
0c5928deada15a8d075516e6e0d9ee19011bb000.

In previous versions ffi::c_ulong used to be u64. I also noticed that 
core::ffi::c_ulong is u64 while crate::ffi::c_ulong is usize. In the 
previous versions they're both u64.

So my mistake here as is I tried to match the signature manually by 
using usize as suggested by the compiler. Maybe we should stick to 
ffi::c_ulong return type for Attrs::as_raw() to be in sync the 
bindgen-generated interface?

Regards,
Abdiel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Miguel Ojeda]

On Thu, Jan 9, 2025 at 12:14 PM Abdiel Janulgue
<abdiel.janulgue@gmail.com> wrote:
>
> We are interested in the last argument 'attrs'. For some reason it seems
> ffi::c_ulong is defined as usize at least on top of `rust-next` commit
> 0c5928deada15a8d075516e6e0d9ee19011bb000.

We are moving to a custom mapping instead of using `core::ffi`
directly. Please see commit 1bae8729e50a ("rust: map `long` to `isize`
and `char` to `u8`"):

    The following FFI types are replaced compared to `core::ffi`:

    1. `char` type is now always mapped to `u8`, since kernel uses
       `-funsigned-char` on the C code. `core::ffi` maps it to platform
       default ABI, which can be either signed or unsigned.

    2. `long` is now always mapped to `isize`. It's very common in the
       kernel to use `long` to represent a pointer-sized integer, and in
       fact `intptr_t` is a typedef of `long` in the kernel. Enforce this
       mapping rather than mapping to `i32/i64` depending on platform can
       save us a lot of unnecessary casts.

    +    // In the kernel, `intptr_t` is defined to be `long` in
         all platforms, so we can map the type to
    +    // `isize`.
    +    c_long = isize;
    +    c_ulong = usize;

Cheers,
Miguel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Miguel Ojeda]

On Thu, Jan 9, 2025 at 12:14 PM Abdiel Janulgue
<abdiel.janulgue@gmail.com> wrote:
>
> using usize as suggested by the compiler. Maybe we should stick to
> ffi::c_ulong return type for Attrs::as_raw() to be in sync the
> bindgen-generated interface?

In the past, `bindgen` also generated code using the `core::ffi::`
mapping, but now uses it uses the `ffi` (the crate) instead (starting
with commit d072acda4862 ("rust: use custom FFI integer types") in
mainline).

Note that in mainline, `ffi` (the crate) still redirects to
`core::ffi`, since we split the change in 2 kernel cycles to make it
easier to migrate. But in `rust-next`, `ffi` (the crate) already uses
the new mapping.

The idea is that you can already use `ffi` (the crate) -- so please use that.

I hope that helps!

Cheers,
Miguel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Abdiel Janulgue]

On 09/01/2025 13:32, Miguel Ojeda wrote:
> 
> Note that in mainline, `ffi` (the crate) still redirects to
> `core::ffi`, since we split the change in 2 kernel cycles to make it
> easier to migrate. But in `rust-next`, `ffi` (the crate) already uses
> the new mapping.
> 
> The idea is that you can already use `ffi` (the crate) -- so please use that.
> 
> I hope that helps!
> 
> Cheers,
> Miguel

Thanks, sticking to ffi::c_ulong does indeed fix this.

Regards,
Abdiel

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[kernel test robot]

Hi Abdiel,

kernel test robot noticed the following build errors:

[auto build test ERROR on v6.13-rc6]
[also build test ERROR on linus/master]
[cannot apply to next-20250110]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Abdiel-Janulgue/rust-error-Add-EOVERFLOW/20250108-202929
base:   v6.13-rc6
patch link:    https://lore.kernel.org/r/20250108122825.136021-3-abdiel.janulgue%40gmail.com
patch subject: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.
config: um-randconfig-001-20250112 (https://download.01.org/0day-ci/archive/20250112/202501120855.PXuydC6T-lkp@intel.com/config)
compiler: clang version 20.0.0git (https://github.com/llvm/llvm-project f5cd181ffbb7cb61d582fe130d46580d5969d47a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250112/202501120855.PXuydC6T-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202501120855.PXuydC6T-lkp@intel.com/

All errors (new ones prefixed by >>):

>> error[E0425]: cannot find function `dma_alloc_attrs` in crate `bindings`
   --> rust/kernel/dma.rs:145:23
   |
   145   |               bindings::dma_alloc_attrs(
   |                         ^^^^^^^^^^^^^^^ help: a function with a similar name exists: `dma_alloc_pages`
   |
   ::: rust/bindings/bindings_generated.rs:39433:5
   |
   39433 | /     pub fn dma_alloc_pages(
   39434 | |         dev: *mut device,
   39435 | |         size: usize,
   39436 | |         dma_handle: *mut dma_addr_t,
   39437 | |         dir: dma_data_direction,
   39438 | |         gfp: gfp_t,
   39439 | |     ) -> *mut page;
   | |__________________- similarly named function `dma_alloc_pages` defined here
--
>> error[E0425]: cannot find function `dma_free_attrs` in crate `bindings`
   --> rust/kernel/dma.rs:262:23
   |
   262   |               bindings::dma_free_attrs(
   |                         ^^^^^^^^^^^^^^ help: a function with a similar name exists: `dma_free_pages`
   |
   ::: rust/bindings/bindings_generated.rs:39442:5
   |
   39442 | /     pub fn dma_free_pages(
   39443 | |         dev: *mut device,
   39444 | |         size: usize,
   39445 | |         page: *mut page,
   39446 | |         dma_handle: dma_addr_t,
   39447 | |         dir: dma_data_direction,
   39448 | |     );
   | |_____- similarly named function `dma_free_pages` defined here

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Thomas Hampton]

On 1/8/2025 6:27 AM, Abdiel Janulgue wrote:

> +    /// Returns the base address, dma handle, attributes and the size of the allocated region.
> +    /// The caller takes ownership of the returned resources, i.e., will have the responsibility
> +    /// in calling `bindings::dma_free_attrs`.
> +    pub fn into_parts(self) -> (*mut T, bindings::dma_addr_t, usize, usize) {
> +        let size = self.count * core::mem::size_of::<T>();
> +        let ret = (
> +            self.cpu_addr,
> +            self.dma_handle,
> +            self.dma_attrs.as_raw(),
> +            size,
> +        );
> +        // Drop the device's reference count associated with this object. This is needed as no
> +        // destructor will be called on this object once this function returns.
> +        // SAFETY: the device pointer is still valid as of this point due to the type invariants
> +        // on `CoherentAllocation`.
> +        unsafe { bindings::put_device(self.dev.as_raw()) }
> +        core::mem::forget(self);
> +        ret
> +    }

Would it be feasible and prudent to avoid exposing `pub fn
into_parts`, `pub fn first_ptr`, `pub fn first_ptr_mut`?

It seems like some sort of functional access method might be better
for keeping future Rust code maintainable and Rusty.

For example, making client code from:

> {
>    let ctrl_id = unsafe { &*(id.first_ptr() as *const NvmeIdCtrl) };
>    number_of_namespaces = ctrl_id.nn.into();
>    mdts = ctrl_id.mdts;
> }

Into something like,
pub fn expose(self, op: F)
  where F: Fn(*const T) {
    op(self.cpu_addr);
}

let number_of_namespaces;
let mdts;
id.expose(|data| {
let ctrl_id = &*(data as *const NvmeIdCtrl);
    number_of_namespaces = ctrl_id.nn.into();
    mdts = ctrl_id.mdts;
});

And to avoid code docs like:

> /// The caller <...> will have the responsibility in calling `bindings::dma_free_attrs`.

Re: [PATCH v8 2/2] rust: add dma coherent allocator abstraction.

[Thomas Hampton]

On Tue, Feb 4, 2025 at 10:54 AM Thomas Hampton
<thomas.j.hampton@gmail.com> wrote:
>
On second thought, this wouldn't prevent the construction of a closure
that simply extracts the pointer itself, nevermind.

Re: [PATCH v8 0/2] Add dma coherent allocator abstraction

[Pyrex]

I'm nervy about the Rust code here.

- read()'s comment says it takes a snapshot, but it doesn't

- read()'s name implies it does a read, but it doesn't. It returns a 
live, dangerous view

- into_parts()'s comment claims to decrement the refcount. One, it 
doesn't. Two, it probably shouldn't, if it's supposed to transfer ownership.

- write() shouldn't take an immutable receiver without unsafe

- write() is unsound if used with the slice from read()

- the mutation in write() breaks read() without contradicting its 
`Safety` requirements

- write() memcpys T, which isn't explicitly Copy

This doesn't have to be this unsafe. AsBytes + FromBytes implies Copy 
(or at least it _should_) -- so the view could be of Cell.

Re: [PATCH v8 0/2] Add dma coherent allocator abstraction

[Danilo Krummrich]

Hi,

On Mon, Feb 10, 2025 at 12:54:25AM -0800, Pyrex wrote:
> I'm nervy about the Rust code here.

Code reviews are always welcome!

However, please make sure to consider the following.

When sending feedback, please try to keep up with the latest version [1] of the
patchset.

Please send your comments inline keeping the relevant context and removing the
irrelevant parts in your reply. Otherwise it is impossible to have a discussion
that everyone can follow.

Thanks!

[1] https://lore.kernel.org/rust-for-linux/20250123104333.1340512-1-abdiel.janulgue@gmail.com/