From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [10.110.1.242]) by lists.oasis-open.org (Postfix) with ESMTP id 565DF9863B3 for ; Wed, 11 Jan 2023 17:04:19 +0000 (UTC) Message-ID: Date: Wed, 11 Jan 2023 18:04:10 +0100 References: <20221208072325.2259940-1-acourbot@chromium.org> From: Alexander Gordeev In-Reply-To: <20221208072325.2259940-1-acourbot@chromium.org> MIME-Version: 1.0 Subject: [virtio-dev] Re: [RFC PATCH v6] virtio-video: Add virtio video device specification Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable To: Alexandre Courbot , virtio-dev@lists.oasis-open.org, Keiichi Watanabe , =?UTF-8?Q?Alex_Benn=c3=a9e?= Cc: Marcin Wojtas , =?UTF-8?Q?Matti_M=c3=b6ll?= , Andrew Gazizov , Enrico Granata , Gustavo Padovan , Peter Griffin , =?UTF-8?Q?Bart=c5=82omiej_Grzesik?= , Tomasz Figa List-ID: Hello, Sorry for the delay. I'm still gathering data in the old emails... On 08.12.22 08:23, Alexandre Courbot wrote: > Add the specification of the video decoder and encoder devices, which > can be used to provide host-accelerated video operations to the guest. > > Signed-off-by: Keiichi Watanabe > Signed-off-by: Alexandre Courbot > -- > Here is the long-overdue new revision of the virtio-video RFC. This > version reorganizes the specification quite a bit and tries to simplify > the protocol further. Nonetheless, it still results in a rather long (17 > pages) specification for just these devices, even though the spec is not > fully complete (I want to rethink the formats descriptions, and some > parameters need to be added for the encoder device). > > I would like to get some high-level feedback on this version and maybe > propose to do things a bit differently before people invest too much > time reviewing this in depth. While rewriting this document, it became > more and more obvious that this is just a different, and maybe a bit > simpler, reimplementation of the V4L2 stateless decoder protocol [1]. I > am now wondering whether it would not make more sense to rewrite this > specification as just a way to transport V4L2 requests over virtio, > similarly to how virtio-fs does with the FUSE protocol [2]. > > At the time we started writing this implementation, the V4L2 protocols > for decoders and encoders were not set in stone yet, but now that they > are it might make sense to reconsider. Switching to this solution would > greatly shorten the virtio-video device spec, and also provide a way to > support other kind of V4L2 devices like cameras or image processors at > no extra cost. > > Note that doing so would not require that either the host or guest uses > V4L2 - the virtio video device would just emulate a V4L2 device over > virtio. A few adaptations would need to be done regarding how memory > types work, but otherwise I believe most of V4L2 could be used as-is. > > Please share your thoughts about this, and I will either explore this > idea further with a prototype, or keep moving the present spec forward, > hopefully at a faster pace. > > Due to the RFC state of this patch I have refrained from referencing the > normative statements in conformance.tex - I will do that as a final step > once the spec is mostly agreed on. > > [1] https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url= =3Dhttps%3a%2f%2fdocs.kernel.org%2fuserspace%2dapi%2fmedia%2fv4l%2fdev%2dst= ateless%2ddecoder.html&umid=3D8ec8d8c9-b83c-40de-9337-a377056fe2af&auth=3D5= 3c7c7de28b92dfd96e93d9dd61a23e634d2fbec-e98508782bc1c9aa6b2e4a9df9d4dd170f9= a5ffa > [2] https://github.com/oasis-tcs/virtio-spec/blob/master/virtio-fs.tex > > Full PDF: > https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=3Dht= tps%3a%2f%2fdrive.google.com%2ffile%2fd%2f1HRVDiDdo50%2dS9X5tWgzmT90FJRHoB1= dN%2fview%3fusp%3dsharing&umid=3D8ec8d8c9-b83c-40de-9337-a377056fe2af&auth= =3D53c7c7de28b92dfd96e93d9dd61a23e634d2fbec-e315af79a067165e908bf1d803441eb= 181e2f375 > > PDF of video section only: > https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=3Dht= tps%3a%2f%2fdrive.google.com%2ffile%2fd%2f1Sm6LSqvKqQiwYmDE9BXZ0po3XTKnKYlD= %2fview%3fusp%3dsharing&umid=3D8ec8d8c9-b83c-40de-9337-a377056fe2af&auth=3D= 53c7c7de28b92dfd96e93d9dd61a23e634d2fbec-b5c45bb4b6ccc5b73ea2a54e26f151d617= 22d0df > --- > content.tex | 1 + > virtio-video.tex | 1420 ++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 1421 insertions(+) > create mode 100644 virtio-video.tex > > ...snip... > > +\subsection{Supported formats}\label{sec:Device Types / Video Device / S= upported formats} > + > +Bitstream and image formats are identified by their fourcc code, which > +is a four-bytes ASCII sequence uniquely identifying the format and its > +properties. > + > +\subsubsection{Bitstream formats}\label{sec:Device Types / Video Device = / Supported formats / Bitstream formats} > + > +The fourcc code of each supported bitstream format is given, as well as > +the unit of data requested in each input resource for the decoder, or > +produced in each output resource for the encoder. > + > +\begin{description} > +\item[\field{MPG2}] > +MPEG2 encoded stream. One Access Unit per resource. > +\item[\field{H264}] > +H.264 encoded stream. One NAL unit per resource. > +\item[\field{HEVC}] > +HEVC encoded stream. One NAL unit per resource. > +\item[\field{VP80}] > +VP8 encoded stream. One frame per resource. > +\item[\field{VP90}] > +VP9 encoded stream. One frame per resource. > +\end{description} > + Actually I'm not sure where do these fourcc codes for bitstream formats come from. In one of the old comments I found a reference to https://www.rfc-editor.org/rfc/rfc2361 But it doesn't define MPG2, HEVC, VP80, VP90. Also there is this comment to virtio-video v1 from Tomasz Figa and the related discussion: https://markmail.org/message/gc6h25acct22niut#query:+page:1+mid:et4l3ni7qjq= hiygo+state:results He wrote that it is not worth it because there are so many conflicting sets of fourcc codes. I can see that fourcc codes were not used in virtio-video spec draft versions 1 to 5. So now it looks quite weird to see them here. Probably this is because you'd like to mimic v4l2 interface more closely? > +\subsubsection{Image formats}\label{sec:Device Types / Video Device / Su= pported formats / Image formats} > + > +The fourcc code of each supported image format is given, as well as its > +number of planes, physical buffers, and eventual subsampling. > + > +\begin{description} > +\item[\field{RGB3}] > +one RGB plane where each component takes one byte, i.e.~3 bytes per > +pixel. > +\item[\field{NV12}] > +one Y plane followed by interleaved U and V data, in a single buffer. > +4:2:0 subsampling. > +\item[\field{NV12}] > +same as \field{NV12} but using two separate buffers for the Y and UV > +planes. > +\item[\field{YU12}] > +one Y plane followed by one Cb plane, followed by one Cr plane, in a > +single buffer. 4:2:0 subsampling. > +\item[\field{YM12}] > +same as \field{YU12} but using three separate buffers for the Y, U and V > +planes. > +\end{description} -- Alexander Gordeev Senior Software Engineer OpenSynergy GmbH Rotherstr. 20, 10245 Berlin Phone: +49 30 60 98 54 0 - 88 Fax: +49 (30) 60 98 54 0 - 99 EMail: alexander.gordeev@opensynergy.com www.opensynergy.com Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616= B Gesch=C3=A4ftsf=C3=BChrer/Managing Director: R=C3=A9gis Adjamah Please mind our privacy notice pursuant to Art. 13 GDPR. // Unsere= Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier. --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org