Re: [RFC 24/24] WIP: rust/audio: add GStreamer backend

[Markus Armbruster <armbru@redhat.com>]

marcandre.lureau@redhat.com writes:

> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> This patch introduce a rust/audio crate that replaces QEMU audio/
> mixing/resampling code with GStreamer and Rust. It could potentially
> remove the need for all the system-specific audio API implementation,
> since GStreamer has audio elements for
> ALSA/Pipewire/PulseAudio/jack/OSX/WASAPI etc (removing ~10k loc).
>
> TODO:
> - test on various system, with various configuration to see if this
>   backend can replace the other QEMU audio backends
> - add a spicesink/spicesrc to handle spice, or rewrite spice to use
>   the capture approach used by VNC code. Or drop capture support, and
>   use custom qemusrc/qemusink for both Spice and VNC, lowering the feature
>   and behaviour disparity.
> - build-sys: make gstreamer optional
> - build-sys: loadable module support
> - investigate dropping get_buffer_size_out()
> - investigate improving emulated devices to not require regular
>   timers (appsrc need-data is called once)
> - add generic audio backend tests
> - more tests for the mixing/liveadder behaviour (synchronization)
> - other: replace audio/dbus with a rust implementation (not using gstreamer)
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  qapi/audio.json               |   29 +
>  audio/audio-driver_template.h |    2 +
>  rust/audio/wrapper.h          |   27 +
>  audio/audio.c                 |    5 +
>  Cargo.lock                    |  572 ++++++++++++++++--
>  Cargo.toml                    |    6 +
>  audio/trace-events            |    5 +
>  rust/audio/Cargo.toml         |   29 +
>  rust/audio/build.rs           |   49 ++
>  rust/audio/meson.build        |   75 +++
>  rust/audio/src/audio.rs       |  516 ++++++++++++++++
>  rust/audio/src/bindings.rs    |   32 +
>  rust/audio/src/gstreamer.rs   | 1070 +++++++++++++++++++++++++++++++++
>  rust/audio/src/lib.rs         |   99 +++
>  rust/meson.build              |    6 +
>  15 files changed, 2467 insertions(+), 55 deletions(-)
>  create mode 100644 rust/audio/wrapper.h
>  create mode 100644 rust/audio/Cargo.toml
>  create mode 100644 rust/audio/build.rs
>  create mode 100644 rust/audio/meson.build
>  create mode 100644 rust/audio/src/audio.rs
>  create mode 100644 rust/audio/src/bindings.rs
>  create mode 100644 rust/audio/src/gstreamer.rs
>  create mode 100644 rust/audio/src/lib.rs
>
> diff --git a/qapi/audio.json b/qapi/audio.json
> index 2df87b9710..76dc7cbfa6 100644
> --- a/qapi/audio.json
> +++ b/qapi/audio.json
> @@ -128,6 +128,33 @@
>      '*out':       'AudiodevAlsaPerDirectionOptions',
>      '*threshold': 'uint32' } }
>  
> +    ##
> +    # @AudiodevGStreamerOptions:
> +    #
> +    # Options of the GStreamer audio backend.
> +    #
> +    # @in: options of the capture stream
> +    #
> +    # @out: options of the playback stream
> +    #
> +    # @sink: the name of the GStreamer sink element to use
> +    #        (default 'autoaudiosink')
> +    #
> +    # @source: the name of the GStreamer source element to use
> +    #        (default 'autoaudiosrc')

Are 'autoaudiosink' and 'autoaudiosrc' well-known GStreamer names, or
arbitrary?

> +    #
> +    # Since: 11.0
> +    ##
> +    { 'struct': 'AudiodevGStreamerOptions',
> +      'data': {
> +        '*in':        'AudiodevPerDirectionOptions',
> +        '*out':       'AudiodevPerDirectionOptions',
> +        '*sink':      'str',
> +        '*source':    'str'
> +      }
> +    }

Unindent by four, and format the doc comment like this:

##
# @AudiodevGStreamerOptions:
#
# Options of the GStreamer audio backend.
#
# @in: options of the capture stream
#
# @out: options of the playback stream
#
# @sink: the name of the GStreamer sink element to use
#     (default 'autoaudiosink')
#
# @source: the name of the GStreamer source element to use
#     (default 'autoaudiosrc')
#
# Since: 11.0
##


> +
> +
>  ##
>  # @AudiodevSndioOptions:
>  #
> @@ -484,6 +511,7 @@
>              { 'name': 'sdl', 'if': 'CONFIG_AUDIO_SDL' },
>              { 'name': 'sndio', 'if': 'CONFIG_AUDIO_SNDIO' },
>              { 'name': 'spice', 'if': 'CONFIG_SPICE' },
> +            { 'name': 'gstreamer' },

Short form suffices:

               'gstreamer',

>              'wav' ] }
>  
>  ##
> @@ -530,6 +558,7 @@
>                     'if': 'CONFIG_AUDIO_SNDIO' },
>      'spice':     { 'type': 'AudiodevGenericOptions',
>                     'if': 'CONFIG_SPICE' },
> +    'gstreamer': { 'type': 'AudiodevGStreamerOptions' },

Short form suffices:

       'gstreamer': 'AudiodevGStreamerOptions',

>      'wav':       'AudiodevWavOptions' } }
>  
>  ##

[...]