[ANN] Request for Topics and registration for a Media Summit September 16th

[ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

Hi all,

We will organize another Media Summit on Monday September 16th to coincide with
the Open Source Summit Europe in Vienna:

https://events.linuxfoundation.org/open-source-summit-europe/

Avnet Silica has very kindly offered to host this summit at their Vienna
office, which is about 35 minutes by public transport from the OSSE venue.

Location:

https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu

The meeting room can hold 18 people and has video conferencing support (MS Teams).

That said, I want to keep remote participation to a minimum. This yearly summit is meant
for active media developers to meet up face-to-face and to discuss media subsystem issues.
But if you are an active media developer, but are not able to attend in person, then this
is an option.

If you have a topic that you want to discuss, just 'Reply All' to this announcement.
It would be very much appreciated if you can also add a guesstimate of the time
you need for your topic.

If you want to attend the meeting (either in person or remote), then send an email to me
directly. Since the number of seats is limited, I may have to put people on a waiting list.
Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
what to expect.

Priority goes to presenters and the core media maintainers. If multiple people of the same
company want to attend, then I may ask to limit attendance to one or two people.

It is hard to predict how many people want to attend, so I'll see how it goes.

Regards,

	Hans

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Ricardo Ribalda]

Hi Hans

Thanks for organizing this

On Mon, 6 May 2024 at 13:33, Hans Verkuil <hverkuil@xs4all.nl> wrote:

>
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.


I would like to talk about media-ci and the multi-committer model.

I am planning to attend in person and I think we will need around 1
hour for this topic

Best regards!



-- 
Ricardo Ribalda

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

Hi Hans,

On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
> Hi all,
> 
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.

A big thank you to Avnet !

> 
> Location:
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> 
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
> 
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.

With the recently merged support for streams in the subdev API, I would
like to discuss the next steps we plan for subdevs, state, and usage of
the media controller device to submit requests.

> If you want to attend the meeting (either in person or remote), then send an email to me
> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> what to expect.
> 
> Priority goes to presenters and the core media maintainers. If multiple people of the same
> company want to attend, then I may ask to limit attendance to one or two people.
> 
> It is hard to predict how many people want to attend, so I'll see how it goes.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Sean Young]

On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
> Hi all,
> 
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.
> 
> Location:
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> 
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
> 
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.

So I've been working away on new tooling for infrared and it's nearing
completion. I'd like to talk about:

 - What it is and what it can do (love to hear any feedback of course)
 - Where is should be hosted? (I hope gitlab fdo, who do I ask)
 - What needs to be in place for a release?
 - This tool replaces ir-ctl and ir-keytable. How we phase them out?

I hope we can do this in 30 minutes and I should be able to attend in person.

Alternatively we can also discuss this over email so I don't want to bump
more important things.


Sean

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tommaso Merciai]

Hi Hans,
Thanks for organizing this summit.

On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
> Hi all,
> 
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.
> 
> Location:
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> 
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
> 
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.
> 
> If you want to attend the meeting (either in person or remote), then send an email to me
> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> what to expect.
> 
> Priority goes to presenters and the core media maintainers. If multiple people of the same
> company want to attend, then I may ask to limit attendance to one or two people.

If is possible Martin and me from Avnet Silica we are planning to attend in person the meeting.
We are working on some topics to present.

Thanks in advance,
Tommaso

> 
> It is hard to predict how many people want to attend, so I'll see how it goes.
> 
> Regards,
> 
> 	Hans
> 

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 07/05/2024 08:40, Tommaso Merciai wrote:
> Hi Hans,
> Thanks for organizing this summit.
> 
> On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
>> Hi all,
>>
>> We will organize another Media Summit on Monday September 16th to coincide with
>> the Open Source Summit Europe in Vienna:
>>
>> https://events.linuxfoundation.org/open-source-summit-europe/
>>
>> Avnet Silica has very kindly offered to host this summit at their Vienna
>> office, which is about 35 minutes by public transport from the OSSE venue.
>>
>> Location:
>>
>> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
>>
>> The meeting room can hold 18 people and has video conferencing support (MS Teams).
>>
>> That said, I want to keep remote participation to a minimum. This yearly summit is meant
>> for active media developers to meet up face-to-face and to discuss media subsystem issues.
>> But if you are an active media developer, but are not able to attend in person, then this
>> is an option.
>>
>> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
>> It would be very much appreciated if you can also add a guesstimate of the time
>> you need for your topic.
>>
>> If you want to attend the meeting (either in person or remote), then send an email to me
>> directly. Since the number of seats is limited, I may have to put people on a waiting list.
>> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
>> what to expect.
>>
>> Priority goes to presenters and the core media maintainers. If multiple people of the same
>> company want to attend, then I may ask to limit attendance to one or two people.
> 
> If is possible Martin and me from Avnet Silica we are planning to attend in person the meeting.
> We are working on some topics to present.

That should be possible! Esp. since you are hosting this event :-)

Regards,

	Hans

> 
> Thanks in advance,
> Tommaso
> 
>>
>> It is hard to predict how many people want to attend, so I'll see how it goes.
>>
>> Regards,
>>
>> 	Hans
>>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tommaso Merciai]

On Wed, May 08, 2024 at 09:16:04AM +0200, Hans Verkuil wrote:
> On 07/05/2024 08:40, Tommaso Merciai wrote:
> > Hi Hans,
> > Thanks for organizing this summit.
> > 
> > On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
> >> Hi all,
> >>
> >> We will organize another Media Summit on Monday September 16th to coincide with
> >> the Open Source Summit Europe in Vienna:
> >>
> >> https://events.linuxfoundation.org/open-source-summit-europe/
> >>
> >> Avnet Silica has very kindly offered to host this summit at their Vienna
> >> office, which is about 35 minutes by public transport from the OSSE venue.
> >>
> >> Location:
> >>
> >> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> >>
> >> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> >>
> >> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> >> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> >> But if you are an active media developer, but are not able to attend in person, then this
> >> is an option.
> >>
> >> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> >> It would be very much appreciated if you can also add a guesstimate of the time
> >> you need for your topic.
> >>
> >> If you want to attend the meeting (either in person or remote), then send an email to me
> >> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> >> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> >> what to expect.
> >>
> >> Priority goes to presenters and the core media maintainers. If multiple people of the same
> >> company want to attend, then I may ask to limit attendance to one or two people.
> > 
> > If is possible Martin and me from Avnet Silica we are planning to attend in person the meeting.
> > We are working on some topics to present.
> 
> That should be possible! Esp. since you are hosting this event :-)

Thanks for the confirmation :)

Regards,
Tommaso

> 
> Regards,
> 
> 	Hans
> 
> > 
> > Thanks in advance,
> > Tommaso
> > 
> >>
> >> It is hard to predict how many people want to attend, so I'll see how it goes.
> >>
> >> Regards,
> >>
> >> 	Hans
> >>
> 

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Daniel Almeida]

Hi Hans, all,

I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].

Please note that these are new submissions that are unrelated with what was discussed last year.

30 minutes will do.

[0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
[1] https://lwn.net/Articles/970565

— Daniel

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
<daniel.almeida@collabora.com> wrote:
>
> Hi Hans, all,
>
> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
>
> Please note that these are new submissions that are unrelated with what was discussed last year.
>
> 30 minutes will do.
>
> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> [1] https://lwn.net/Articles/970565

Somewhat related to the topic: I see potential for a quite big
redesign of the videobuf2 framework going forward and recently with
more Rust adoption I'm starting to think it could benefit from being
implemented in Rust, since we would have to rewrite it quite a bit
anyway. Especially since it's a part of the subsystem that has to deal
with memory management, object lifetime and asynchronousness quite a
lot and we had a history of issues there. So it could be interesting
to hear everyone's thoughts.

That said, I wouldn't be able to travel this time unfortunately, so it
would be nice if we could arrange this topic in a time slot friendly
for remote attendance from Japan. Also +Hidenori Kobayashi from my
team who would also be interested in joining remotely.

Best,
Tomasz

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 6/12/24 06:12, Tomasz Figa wrote:
> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> <daniel.almeida@collabora.com> wrote:
>>
>> Hi Hans, all,
>>
>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
>>
>> Please note that these are new submissions that are unrelated with what was discussed last year.
>>
>> 30 minutes will do.
>>
>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
>> [1] https://lwn.net/Articles/970565
> 
> Somewhat related to the topic: I see potential for a quite big
> redesign of the videobuf2 framework going forward and recently with
> more Rust adoption I'm starting to think it could benefit from being
> implemented in Rust, since we would have to rewrite it quite a bit
> anyway. Especially since it's a part of the subsystem that has to deal
> with memory management, object lifetime and asynchronousness quite a
> lot and we had a history of issues there. So it could be interesting
> to hear everyone's thoughts.

I think it is far too soon to write a framework like that in Rust. To be
honest, I won't even consider it until Linus officially accepts Rust as a
second language in the kernel, instead of as an experiment.

The vb2 framework can certainly use some more work, and esp. better support
for codecs, since that's where the main pain is at the moment.

But I would need to see a proper proposal first. I assume that's what you
plan to present?

> That said, I wouldn't be able to travel this time unfortunately, so it
> would be nice if we could arrange this topic in a time slot friendly
> for remote attendance from Japan. Also +Hidenori Kobayashi from my
> team who would also be interested in joining remotely.

That would mean a slot in the morning, right? Since Japan is 7 hours ahead
of CEST.

Regards,

	Hans

> 
> Best,
> Tomasz

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Steve Cho]

I would be presenting 2 short topics (15mins each) following Hans's suggestion.

- V4L2 testing on Chromium using virtual video decode driver (VISL)
- V4L2 video decoding testing with KernelCI

On Wed, Jun 12, 2024 at 3:46 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 6/12/24 06:12, Tomasz Figa wrote:
> > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> > <daniel.almeida@collabora.com> wrote:
> >>
> >> Hi Hans, all,
> >>
> >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> >>
> >> Please note that these are new submissions that are unrelated with what was discussed last year.
> >>
> >> 30 minutes will do.
> >>
> >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> >> [1] https://lwn.net/Articles/970565
> >
> > Somewhat related to the topic: I see potential for a quite big
> > redesign of the videobuf2 framework going forward and recently with
> > more Rust adoption I'm starting to think it could benefit from being
> > implemented in Rust, since we would have to rewrite it quite a bit
> > anyway. Especially since it's a part of the subsystem that has to deal
> > with memory management, object lifetime and asynchronousness quite a
> > lot and we had a history of issues there. So it could be interesting
> > to hear everyone's thoughts.
>
> I think it is far too soon to write a framework like that in Rust. To be
> honest, I won't even consider it until Linus officially accepts Rust as a
> second language in the kernel, instead of as an experiment.
>
> The vb2 framework can certainly use some more work, and esp. better support
> for codecs, since that's where the main pain is at the moment.
>
> But I would need to see a proper proposal first. I assume that's what you
> plan to present?
>
> > That said, I wouldn't be able to travel this time unfortunately, so it
> > would be nice if we could arrange this topic in a time slot friendly
> > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > team who would also be interested in joining remotely.
>
> That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> of CEST.
>
> Regards,
>
>         Hans
>
> >
> > Best,
> > Tomasz
>
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2024 08:46:50 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> On 6/12/24 06:12, Tomasz Figa wrote:
> > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> > <daniel.almeida@collabora.com> wrote:  
> >>
> >> Hi Hans, all,
> >>
> >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> >>
> >> Please note that these are new submissions that are unrelated with what was discussed last year.
> >>
> >> 30 minutes will do.
> >>
> >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> >> [1] https://lwn.net/Articles/970565  
> > 
> > Somewhat related to the topic: I see potential for a quite big
> > redesign of the videobuf2 framework going forward and recently with
> > more Rust adoption I'm starting to think it could benefit from being
> > implemented in Rust, since we would have to rewrite it quite a bit
> > anyway. Especially since it's a part of the subsystem that has to deal
> > with memory management, object lifetime and asynchronousness quite a
> > lot and we had a history of issues there. So it could be interesting
> > to hear everyone's thoughts.  
> 
> I think it is far too soon to write a framework like that in Rust.

Agreed. I don't object redesigns in C to make it better - which could have
some colateral effect of making things easier for a future Rust adoption,
but such changes should be justified by themselves, and not because of a
language change.

See: redesigns at the core will potentially affect lots of drivers,
so it needs very good technical reasons why doing it. Plus, it requires 
comprehensive tests with different types of hardware/drivers to reduce the
risk of regressions. Depending on the changes, it may require extra tests
with devices that are outside complex camera world: radio, analog and digital 
TV drivers - and even some input devices that use VB2 - to ensure that
nothing broke.

> To be
> honest, I won't even consider it until Linus officially accepts Rust as a
> second language in the kernel, instead of as an experiment.

This is not enough: if the core starts to use a second language, all media
developers will be affected and will be required to have expertise on such
language. That's not something that should happen without careful
analysis and plans that should include a gradual roll-up, lost of tests
with the affected drivers including the legacy ones and some strategy to
quickly solve regression issues.

It is not a matter of what language is better. Instead, it is a matter of
not affecting code maintenance during the (probably long) transition period
and beyond.

If you see the past history, the transition from V4L to V4L2 took more than 10
years - being possible to be done only with the help of libv4l, plus a
lot of backward-compat code that we added. Still there were several
regressions and we even had to quickly patch the Kernel and/or some apps
that were using the uAPI on different ways.

Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.

On both cases, there were very good technical reasons for the transition,
in terms of missing needed features, broken memory models and serious
troubles that utterly causing VB1 to not work well on non-x86 hardware.

In the end, the authors of the core changes need to acquire legacy hardware
and to do lots of driver-specific changes to avoid breaking existing stuff.
Hans and I had to dedicate a lot of time and efforts on such transitions, 
as it required a lot of work.

I can tell you: there's no fun on such changes: typically, companies won't
pay someone to do changes on drivers for legacy hardware, specially
when there are no real benefits, which is the case here, as the final result
is just to keep the existing drivers to work with existing hardware,
usually without any new features. So, the ones behind such core changes
have to commit fixing drivers usually on their spare time.

> The vb2 framework can certainly use some more work, and esp. better support
> for codecs, since that's where the main pain is at the moment.
> 
> But I would need to see a proper proposal first. I assume that's what you
> plan to present?
> 
> > That said, I wouldn't be able to travel this time unfortunately, so it
> > would be nice if we could arrange this topic in a time slot friendly
> > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > team who would also be interested in joining remotely.  
> 
> That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> of CEST.
> 
> Regards,
> 
> 	Hans
> 
> > 
> > Best,
> > Tomasz  
> 


Thanks,
Mauro

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab
<mchehab@kernel.org> wrote:
>
> Em Wed, 12 Jun 2024 08:46:50 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>
> > On 6/12/24 06:12, Tomasz Figa wrote:
> > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> > > <daniel.almeida@collabora.com> wrote:
> > >>
> > >> Hi Hans, all,
> > >>
> > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > >>
> > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > >>
> > >> 30 minutes will do.
> > >>
> > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > >> [1] https://lwn.net/Articles/970565
> > >
> > > Somewhat related to the topic: I see potential for a quite big
> > > redesign of the videobuf2 framework going forward and recently with
> > > more Rust adoption I'm starting to think it could benefit from being
> > > implemented in Rust, since we would have to rewrite it quite a bit
> > > anyway. Especially since it's a part of the subsystem that has to deal
> > > with memory management, object lifetime and asynchronousness quite a
> > > lot and we had a history of issues there. So it could be interesting
> > > to hear everyone's thoughts.
> >
> > I think it is far too soon to write a framework like that in Rust.
>
> Agreed. I don't object redesigns in C to make it better - which could have
> some colateral effect of making things easier for a future Rust adoption,
> but such changes should be justified by themselves, and not because of a
> language change.

No, the thought of redesign doesn't come from the language change,
it's the other way around. Since rewriting a lot of the code already,
why not do it in a language that is generally considered better.

>
> See: redesigns at the core will potentially affect lots of drivers,
> so it needs very good technical reasons why doing it. Plus, it requires
> comprehensive tests with different types of hardware/drivers to reduce the
> risk of regressions. Depending on the changes, it may require extra tests
> with devices that are outside complex camera world: radio, analog and digital
> TV drivers - and even some input devices that use VB2 - to ensure that
> nothing broke.

We don't have to do it in an all-or-nothing way. We can start with an
experimental new implementation in Rust, which could be gradually
tested. It could even be done the same way as the vb -> vb2
transition, although I suspect it wouldn't really be necessary, as I
would like to see it more like a drop-in replacement. In general I
think the API exposed outside of the framework wouldn't really change
that much, it's more about the internal design.

>
> > To be
> > honest, I won't even consider it until Linus officially accepts Rust as a
> > second language in the kernel, instead of as an experiment.
>
> This is not enough: if the core starts to use a second language, all media
> developers will be affected and will be required to have expertise on such
> language.

Let's be realistic, how many developers are actively touching vb2 these days?

> That's not something that should happen without careful
> analysis and plans that should include a gradual roll-up, lost of tests
> with the affected drivers including the legacy ones and some strategy to
> quickly solve regression issues.

That said, I agree. It needs proper discussion and planning. That's
why I'm proposing this as a topic. :)
Moreover the redesign itself also needs proper discussion and is more
of a long term goal, not something to land in the next few days.

>
> It is not a matter of what language is better. Instead, it is a matter of
> not affecting code maintenance during the (probably long) transition period
> and beyond.
>
> If you see the past history, the transition from V4L to V4L2 took more than 10
> years - being possible to be done only with the help of libv4l, plus a
> lot of backward-compat code that we added. Still there were several
> regressions and we even had to quickly patch the Kernel and/or some apps
> that were using the uAPI on different ways.

That's a different situation, because UAPI is involved.

>
> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
>

Yes, vb -> vb2 would be a more appropriate comparison.

> On both cases, there were very good technical reasons for the transition,
> in terms of missing needed features, broken memory models and serious
> troubles that utterly causing VB1 to not work well on non-x86 hardware.
>

It's a very similar situation now, vb2 doesn't work well on modern
hardware, but I still have hopes that it can be fixed without
affecting the driver-facing behavior. (We would probably need to
develop some unit tests that validate the driver-facing behavior to
ensure that.)

> In the end, the authors of the core changes need to acquire legacy hardware
> and to do lots of driver-specific changes to avoid breaking existing stuff.
> Hans and I had to dedicate a lot of time and efforts on such transitions,
> as it required a lot of work.
>
> I can tell you: there's no fun on such changes: typically, companies won't
> pay someone to do changes on drivers for legacy hardware, specially
> when there are no real benefits, which is the case here, as the final result
> is just to keep the existing drivers to work with existing hardware,
> usually without any new features. So, the ones behind such core changes
> have to commit fixing drivers usually on their spare time.
>

I don't get that argument. Wouldn't the same apply to any core change?
I think the reason we have driver maintainers is that they can help
with testing. Moreover, we need to invest into testing infrastructure
(which is what people have been doing recently via Media CI) to make
such changes less painful. Otherwise the subsystem will just bit-rot
and become useful for modern use cases.

> > The vb2 framework can certainly use some more work, and esp. better support
> > for codecs, since that's where the main pain is at the moment.
> >
> > But I would need to see a proper proposal first. I assume that's what you
> > plan to present?
> >
> > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > would be nice if we could arrange this topic in a time slot friendly
> > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > team who would also be interested in joining remotely.
> >
> > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > of CEST.
> >
> > Regards,
> >
> >       Hans
> >
> > >
> > > Best,
> > > Tomasz
> >
>
>
> Thanks,
> Mauro

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

Hi Tomasz,

On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > >>
> > > >> Hi Hans, all,
> > > >>
> > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > >>
> > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > >>
> > > >> 30 minutes will do.
> > > >>
> > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > >> [1] https://lwn.net/Articles/970565
> > > >
> > > > Somewhat related to the topic: I see potential for a quite big
> > > > redesign of the videobuf2 framework going forward and recently with
> > > > more Rust adoption I'm starting to think it could benefit from being
> > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > with memory management, object lifetime and asynchronousness quite a
> > > > lot and we had a history of issues there. So it could be interesting
> > > > to hear everyone's thoughts.
> > >
> > > I think it is far too soon to write a framework like that in Rust.
> >
> > Agreed. I don't object redesigns in C to make it better - which could have
> > some colateral effect of making things easier for a future Rust adoption,
> > but such changes should be justified by themselves, and not because of a
> > language change.
> 
> No, the thought of redesign doesn't come from the language change,
> it's the other way around. Since rewriting a lot of the code already,
> why not do it in a language that is generally considered better.
> 
> > See: redesigns at the core will potentially affect lots of drivers,
> > so it needs very good technical reasons why doing it. Plus, it requires
> > comprehensive tests with different types of hardware/drivers to reduce the
> > risk of regressions. Depending on the changes, it may require extra tests
> > with devices that are outside complex camera world: radio, analog and digital
> > TV drivers - and even some input devices that use VB2 - to ensure that
> > nothing broke.
> 
> We don't have to do it in an all-or-nothing way. We can start with an
> experimental new implementation in Rust, which could be gradually
> tested. It could even be done the same way as the vb -> vb2
> transition, although I suspect it wouldn't really be necessary, as I
> would like to see it more like a drop-in replacement. In general I
> think the API exposed outside of the framework wouldn't really change
> that much, it's more about the internal design.
> 
> > > To be
> > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > second language in the kernel, instead of as an experiment.
> >
> > This is not enough: if the core starts to use a second language, all media
> > developers will be affected and will be required to have expertise on such
> > language.
> 
> Let's be realistic, how many developers are actively touching vb2 these days?
> 
> > That's not something that should happen without careful
> > analysis and plans that should include a gradual roll-up, lost of tests
> > with the affected drivers including the legacy ones and some strategy to
> > quickly solve regression issues.
> 
> That said, I agree. It needs proper discussion and planning. That's
> why I'm proposing this as a topic. :)
> Moreover the redesign itself also needs proper discussion and is more
> of a long term goal, not something to land in the next few days.

Focussing on this topic, if we're brainstorming memory management for
media devices, I'd like to throw in a controversial idea. In addition to
being clearer on the fact that USERPTR is deprecated, I would like to
deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
centralized buffer allocator, instead of having multiple allocation APIs
scattered in different places. There are design ideas in gralloc that we
could benefit from.

> > It is not a matter of what language is better. Instead, it is a matter of
> > not affecting code maintenance during the (probably long) transition period
> > and beyond.
> >
> > If you see the past history, the transition from V4L to V4L2 took more than 10
> > years - being possible to be done only with the help of libv4l, plus a
> > lot of backward-compat code that we added. Still there were several
> > regressions and we even had to quickly patch the Kernel and/or some apps
> > that were using the uAPI on different ways.
> 
> That's a different situation, because UAPI is involved.
> 
> > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> 
> Yes, vb -> vb2 would be a more appropriate comparison.
> 
> > On both cases, there were very good technical reasons for the transition,
> > in terms of missing needed features, broken memory models and serious
> > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> 
> It's a very similar situation now, vb2 doesn't work well on modern
> hardware, but I still have hopes that it can be fixed without
> affecting the driver-facing behavior. (We would probably need to
> develop some unit tests that validate the driver-facing behavior to
> ensure that.)
> 
> > In the end, the authors of the core changes need to acquire legacy hardware
> > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > as it required a lot of work.
> >
> > I can tell you: there's no fun on such changes: typically, companies won't
> > pay someone to do changes on drivers for legacy hardware, specially
> > when there are no real benefits, which is the case here, as the final result
> > is just to keep the existing drivers to work with existing hardware,
> > usually without any new features. So, the ones behind such core changes
> > have to commit fixing drivers usually on their spare time.
> 
> I don't get that argument. Wouldn't the same apply to any core change?
> I think the reason we have driver maintainers is that they can help
> with testing. Moreover, we need to invest into testing infrastructure
> (which is what people have been doing recently via Media CI) to make
> such changes less painful. Otherwise the subsystem will just bit-rot
> and become useful for modern use cases.

I've recently seen an increase in people experimenting with sourdough,
kombucha, kimchi and other fermentation techniques, so rotting isn't
always negative [*], but I assume you meant useless here, not useful :-)

* I'll draw the line at surströmming.

> > > The vb2 framework can certainly use some more work, and esp. better support
> > > for codecs, since that's where the main pain is at the moment.
> > >
> > > But I would need to see a proper proposal first. I assume that's what you
> > > plan to present?
> > >
> > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > would be nice if we could arrange this topic in a time slot friendly
> > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > team who would also be interested in joining remotely.
> > >
> > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > of CEST.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Tomasz,
>
> On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > > >>
> > > > >> Hi Hans, all,
> > > > >>
> > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > > >>
> > > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > > >>
> > > > >> 30 minutes will do.
> > > > >>
> > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > > >> [1] https://lwn.net/Articles/970565
> > > > >
> > > > > Somewhat related to the topic: I see potential for a quite big
> > > > > redesign of the videobuf2 framework going forward and recently with
> > > > > more Rust adoption I'm starting to think it could benefit from being
> > > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > > with memory management, object lifetime and asynchronousness quite a
> > > > > lot and we had a history of issues there. So it could be interesting
> > > > > to hear everyone's thoughts.
> > > >
> > > > I think it is far too soon to write a framework like that in Rust.
> > >
> > > Agreed. I don't object redesigns in C to make it better - which could have
> > > some colateral effect of making things easier for a future Rust adoption,
> > > but such changes should be justified by themselves, and not because of a
> > > language change.
> >
> > No, the thought of redesign doesn't come from the language change,
> > it's the other way around. Since rewriting a lot of the code already,
> > why not do it in a language that is generally considered better.
> >
> > > See: redesigns at the core will potentially affect lots of drivers,
> > > so it needs very good technical reasons why doing it. Plus, it requires
> > > comprehensive tests with different types of hardware/drivers to reduce the
> > > risk of regressions. Depending on the changes, it may require extra tests
> > > with devices that are outside complex camera world: radio, analog and digital
> > > TV drivers - and even some input devices that use VB2 - to ensure that
> > > nothing broke.
> >
> > We don't have to do it in an all-or-nothing way. We can start with an
> > experimental new implementation in Rust, which could be gradually
> > tested. It could even be done the same way as the vb -> vb2
> > transition, although I suspect it wouldn't really be necessary, as I
> > would like to see it more like a drop-in replacement. In general I
> > think the API exposed outside of the framework wouldn't really change
> > that much, it's more about the internal design.
> >
> > > > To be
> > > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > > second language in the kernel, instead of as an experiment.
> > >
> > > This is not enough: if the core starts to use a second language, all media
> > > developers will be affected and will be required to have expertise on such
> > > language.
> >
> > Let's be realistic, how many developers are actively touching vb2 these days?
> >
> > > That's not something that should happen without careful
> > > analysis and plans that should include a gradual roll-up, lost of tests
> > > with the affected drivers including the legacy ones and some strategy to
> > > quickly solve regression issues.
> >
> > That said, I agree. It needs proper discussion and planning. That's
> > why I'm proposing this as a topic. :)
> > Moreover the redesign itself also needs proper discussion and is more
> > of a long term goal, not something to land in the next few days.
>
> Focussing on this topic, if we're brainstorming memory management for
> media devices, I'd like to throw in a controversial idea. In addition to
> being clearer on the fact that USERPTR is deprecated,

Definitely. This has been long overdue.

>  I would like to
> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> centralized buffer allocator, instead of having multiple allocation APIs
> scattered in different places. There are design ideas in gralloc that we
> could benefit from.
>

Given that we now have DMA-buf heaps in mainline, it sounds much more
realistic. It would certainly help eliminate some issues in the vb2
layer, such as vb2-dma-sg having its own open coded page allocation
that can't handle DMA addressing limitations (which can be fixed with
some additions to the DMA API, but eliminating the problem altogether
is way better than any other solution.)

That said, as we already use a centralized DMA-buf allocator in
ChromeOS and don't really care about the MMAP mode, I'm definitely
biased here. We would need to hear from people working on userspace
which still uses it (if there is any).

> > > It is not a matter of what language is better. Instead, it is a matter of
> > > not affecting code maintenance during the (probably long) transition period
> > > and beyond.
> > >
> > > If you see the past history, the transition from V4L to V4L2 took more than 10
> > > years - being possible to be done only with the help of libv4l, plus a
> > > lot of backward-compat code that we added. Still there were several
> > > regressions and we even had to quickly patch the Kernel and/or some apps
> > > that were using the uAPI on different ways.
> >
> > That's a different situation, because UAPI is involved.
> >
> > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> >
> > Yes, vb -> vb2 would be a more appropriate comparison.
> >
> > > On both cases, there were very good technical reasons for the transition,
> > > in terms of missing needed features, broken memory models and serious
> > > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> >
> > It's a very similar situation now, vb2 doesn't work well on modern
> > hardware, but I still have hopes that it can be fixed without
> > affecting the driver-facing behavior. (We would probably need to
> > develop some unit tests that validate the driver-facing behavior to
> > ensure that.)
> >
> > > In the end, the authors of the core changes need to acquire legacy hardware
> > > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > > as it required a lot of work.
> > >
> > > I can tell you: there's no fun on such changes: typically, companies won't
> > > pay someone to do changes on drivers for legacy hardware, specially
> > > when there are no real benefits, which is the case here, as the final result
> > > is just to keep the existing drivers to work with existing hardware,
> > > usually without any new features. So, the ones behind such core changes
> > > have to commit fixing drivers usually on their spare time.
> >
> > I don't get that argument. Wouldn't the same apply to any core change?
> > I think the reason we have driver maintainers is that they can help
> > with testing. Moreover, we need to invest into testing infrastructure
> > (which is what people have been doing recently via Media CI) to make
> > such changes less painful. Otherwise the subsystem will just bit-rot
> > and become useful for modern use cases.
>
> I've recently seen an increase in people experimenting with sourdough,
> kombucha, kimchi and other fermentation techniques, so rotting isn't
> always negative [*], but I assume you meant useless here, not useful :-)

Yeah, definitely. I'd love it if bit-rotting led to computer software
becoming more useful, but sadly it's rarely the case.

Best regards,
Tomasz

>
> * I'll draw the line at surströmming.
>
> > > > The vb2 framework can certainly use some more work, and esp. better support
> > > > for codecs, since that's where the main pain is at the moment.
> > > >
> > > > But I would need to see a proper proposal first. I assume that's what you
> > > > plan to present?
> > > >
> > > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > > would be nice if we could arrange this topic in a time slot friendly
> > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > > team who would also be interested in joining remotely.
> > > >
> > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > > of CEST.
>
> --
> Regards,
>
> Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

On Wed, Jun 12, 2024 at 06:01:07PM +0900, Tomasz Figa wrote:
> On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > > > >>
> > > > > >> Hi Hans, all,
> > > > > >>
> > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > > > >>
> > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > > > >>
> > > > > >> 30 minutes will do.
> > > > > >>
> > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > > > >> [1] https://lwn.net/Articles/970565
> > > > > >
> > > > > > Somewhat related to the topic: I see potential for a quite big
> > > > > > redesign of the videobuf2 framework going forward and recently with
> > > > > > more Rust adoption I'm starting to think it could benefit from being
> > > > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > > > with memory management, object lifetime and asynchronousness quite a
> > > > > > lot and we had a history of issues there. So it could be interesting
> > > > > > to hear everyone's thoughts.
> > > > >
> > > > > I think it is far too soon to write a framework like that in Rust.
> > > >
> > > > Agreed. I don't object redesigns in C to make it better - which could have
> > > > some colateral effect of making things easier for a future Rust adoption,
> > > > but such changes should be justified by themselves, and not because of a
> > > > language change.
> > >
> > > No, the thought of redesign doesn't come from the language change,
> > > it's the other way around. Since rewriting a lot of the code already,
> > > why not do it in a language that is generally considered better.
> > >
> > > > See: redesigns at the core will potentially affect lots of drivers,
> > > > so it needs very good technical reasons why doing it. Plus, it requires
> > > > comprehensive tests with different types of hardware/drivers to reduce the
> > > > risk of regressions. Depending on the changes, it may require extra tests
> > > > with devices that are outside complex camera world: radio, analog and digital
> > > > TV drivers - and even some input devices that use VB2 - to ensure that
> > > > nothing broke.
> > >
> > > We don't have to do it in an all-or-nothing way. We can start with an
> > > experimental new implementation in Rust, which could be gradually
> > > tested. It could even be done the same way as the vb -> vb2
> > > transition, although I suspect it wouldn't really be necessary, as I
> > > would like to see it more like a drop-in replacement. In general I
> > > think the API exposed outside of the framework wouldn't really change
> > > that much, it's more about the internal design.
> > >
> > > > > To be
> > > > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > > > second language in the kernel, instead of as an experiment.
> > > >
> > > > This is not enough: if the core starts to use a second language, all media
> > > > developers will be affected and will be required to have expertise on such
> > > > language.
> > >
> > > Let's be realistic, how many developers are actively touching vb2 these days?
> > >
> > > > That's not something that should happen without careful
> > > > analysis and plans that should include a gradual roll-up, lost of tests
> > > > with the affected drivers including the legacy ones and some strategy to
> > > > quickly solve regression issues.
> > >
> > > That said, I agree. It needs proper discussion and planning. That's
> > > why I'm proposing this as a topic. :)
> > > Moreover the redesign itself also needs proper discussion and is more
> > > of a long term goal, not something to land in the next few days.
> >
> > Focussing on this topic, if we're brainstorming memory management for
> > media devices, I'd like to throw in a controversial idea. In addition to
> > being clearer on the fact that USERPTR is deprecated,
> 
> Definitely. This has been long overdue.
> 
> >  I would like to
> > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > centralized buffer allocator, instead of having multiple allocation APIs
> > scattered in different places. There are design ideas in gralloc that we
> > could benefit from.
> 
> Given that we now have DMA-buf heaps in mainline, it sounds much more
> realistic. It would certainly help eliminate some issues in the vb2
> layer, such as vb2-dma-sg having its own open coded page allocation
> that can't handle DMA addressing limitations (which can be fixed with
> some additions to the DMA API, but eliminating the problem altogether
> is way better than any other solution.)

There are (at least) two issues we'll have to solve to make DMA heaps
more universally usable:

- Memory allocation from DMA heaps isn't accounted in the process
  resource limits. This is one of the blockers for getting distributions
  to enable the heaps. We'll have to fix that.

- We need a userspace library to pick the right heap based on the memory
  constraints of the devices that you'll want to share the buffer with.
  This will require exposing those constraints to userspace somehow.

I'm sure there will be more issues, but solving issues is what we do :-)

> That said, as we already use a centralized DMA-buf allocator in
> ChromeOS and don't really care about the MMAP mode, I'm definitely
> biased here. We would need to hear from people working on userspace
> which still uses it (if there is any).
> 
> > > > It is not a matter of what language is better. Instead, it is a matter of
> > > > not affecting code maintenance during the (probably long) transition period
> > > > and beyond.
> > > >
> > > > If you see the past history, the transition from V4L to V4L2 took more than 10
> > > > years - being possible to be done only with the help of libv4l, plus a
> > > > lot of backward-compat code that we added. Still there were several
> > > > regressions and we even had to quickly patch the Kernel and/or some apps
> > > > that were using the uAPI on different ways.
> > >
> > > That's a different situation, because UAPI is involved.
> > >
> > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> > >
> > > Yes, vb -> vb2 would be a more appropriate comparison.
> > >
> > > > On both cases, there were very good technical reasons for the transition,
> > > > in terms of missing needed features, broken memory models and serious
> > > > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> > >
> > > It's a very similar situation now, vb2 doesn't work well on modern
> > > hardware, but I still have hopes that it can be fixed without
> > > affecting the driver-facing behavior. (We would probably need to
> > > develop some unit tests that validate the driver-facing behavior to
> > > ensure that.)
> > >
> > > > In the end, the authors of the core changes need to acquire legacy hardware
> > > > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > > > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > > > as it required a lot of work.
> > > >
> > > > I can tell you: there's no fun on such changes: typically, companies won't
> > > > pay someone to do changes on drivers for legacy hardware, specially
> > > > when there are no real benefits, which is the case here, as the final result
> > > > is just to keep the existing drivers to work with existing hardware,
> > > > usually without any new features. So, the ones behind such core changes
> > > > have to commit fixing drivers usually on their spare time.
> > >
> > > I don't get that argument. Wouldn't the same apply to any core change?
> > > I think the reason we have driver maintainers is that they can help
> > > with testing. Moreover, we need to invest into testing infrastructure
> > > (which is what people have been doing recently via Media CI) to make
> > > such changes less painful. Otherwise the subsystem will just bit-rot
> > > and become useful for modern use cases.
> >
> > I've recently seen an increase in people experimenting with sourdough,
> > kombucha, kimchi and other fermentation techniques, so rotting isn't
> > always negative [*], but I assume you meant useless here, not useful :-)
> 
> Yeah, definitely. I'd love it if bit-rotting led to computer software
> becoming more useful, but sadly it's rarely the case.
> 
> > * I'll draw the line at surströmming.
> >
> > > > > The vb2 framework can certainly use some more work, and esp. better support
> > > > > for codecs, since that's where the main pain is at the moment.
> > > > >
> > > > > But I would need to see a proper proposal first. I assume that's what you
> > > > > plan to present?
> > > > >
> > > > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > > > would be nice if we could arrange this topic in a time slot friendly
> > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > > > team who would also be interested in joining remotely.
> > > > >
> > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > > > of CEST.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Wed, Jun 12, 2024 at 6:20 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> On Wed, Jun 12, 2024 at 06:01:07PM +0900, Tomasz Figa wrote:
> > On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart wrote:
> > > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> > > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > > > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > > > > >>
> > > > > > >> Hi Hans, all,
> > > > > > >>
> > > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > > > > >>
> > > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > > > > >>
> > > > > > >> 30 minutes will do.
> > > > > > >>
> > > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > > > > >> [1] https://lwn.net/Articles/970565
> > > > > > >
> > > > > > > Somewhat related to the topic: I see potential for a quite big
> > > > > > > redesign of the videobuf2 framework going forward and recently with
> > > > > > > more Rust adoption I'm starting to think it could benefit from being
> > > > > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > > > > with memory management, object lifetime and asynchronousness quite a
> > > > > > > lot and we had a history of issues there. So it could be interesting
> > > > > > > to hear everyone's thoughts.
> > > > > >
> > > > > > I think it is far too soon to write a framework like that in Rust.
> > > > >
> > > > > Agreed. I don't object redesigns in C to make it better - which could have
> > > > > some colateral effect of making things easier for a future Rust adoption,
> > > > > but such changes should be justified by themselves, and not because of a
> > > > > language change.
> > > >
> > > > No, the thought of redesign doesn't come from the language change,
> > > > it's the other way around. Since rewriting a lot of the code already,
> > > > why not do it in a language that is generally considered better.
> > > >
> > > > > See: redesigns at the core will potentially affect lots of drivers,
> > > > > so it needs very good technical reasons why doing it. Plus, it requires
> > > > > comprehensive tests with different types of hardware/drivers to reduce the
> > > > > risk of regressions. Depending on the changes, it may require extra tests
> > > > > with devices that are outside complex camera world: radio, analog and digital
> > > > > TV drivers - and even some input devices that use VB2 - to ensure that
> > > > > nothing broke.
> > > >
> > > > We don't have to do it in an all-or-nothing way. We can start with an
> > > > experimental new implementation in Rust, which could be gradually
> > > > tested. It could even be done the same way as the vb -> vb2
> > > > transition, although I suspect it wouldn't really be necessary, as I
> > > > would like to see it more like a drop-in replacement. In general I
> > > > think the API exposed outside of the framework wouldn't really change
> > > > that much, it's more about the internal design.
> > > >
> > > > > > To be
> > > > > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > > > > second language in the kernel, instead of as an experiment.
> > > > >
> > > > > This is not enough: if the core starts to use a second language, all media
> > > > > developers will be affected and will be required to have expertise on such
> > > > > language.
> > > >
> > > > Let's be realistic, how many developers are actively touching vb2 these days?
> > > >
> > > > > That's not something that should happen without careful
> > > > > analysis and plans that should include a gradual roll-up, lost of tests
> > > > > with the affected drivers including the legacy ones and some strategy to
> > > > > quickly solve regression issues.
> > > >
> > > > That said, I agree. It needs proper discussion and planning. That's
> > > > why I'm proposing this as a topic. :)
> > > > Moreover the redesign itself also needs proper discussion and is more
> > > > of a long term goal, not something to land in the next few days.
> > >
> > > Focussing on this topic, if we're brainstorming memory management for
> > > media devices, I'd like to throw in a controversial idea. In addition to
> > > being clearer on the fact that USERPTR is deprecated,
> >
> > Definitely. This has been long overdue.
> >
> > >  I would like to
> > > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > > centralized buffer allocator, instead of having multiple allocation APIs
> > > scattered in different places. There are design ideas in gralloc that we
> > > could benefit from.
> >
> > Given that we now have DMA-buf heaps in mainline, it sounds much more
> > realistic. It would certainly help eliminate some issues in the vb2
> > layer, such as vb2-dma-sg having its own open coded page allocation
> > that can't handle DMA addressing limitations (which can be fixed with
> > some additions to the DMA API, but eliminating the problem altogether
> > is way better than any other solution.)
>
> There are (at least) two issues we'll have to solve to make DMA heaps
> more universally usable:
>
> - Memory allocation from DMA heaps isn't accounted in the process
>   resource limits. This is one of the blockers for getting distributions
>   to enable the heaps. We'll have to fix that.

I think the same applies to MMAP buffers today or am I missing something?

That said, it's something that has to be fixed anyway. DMA heaps would
make it easier to fix, since we would have one central place where the
accounting could be implemented.

>
> - We need a userspace library to pick the right heap based on the memory
>   constraints of the devices that you'll want to share the buffer with.
>   This will require exposing those constraints to userspace somehow.

That is not a new problem either. Today with MMAP we also need the
userspace to somehow choose the right exporter. So we could start with
each video node pointing to the right heap (probably for each plane
separately?), which would provide the same level of functionality as
with MMAP today, but using a uniform API and a central implementation
in the kernel.

Then we could follow up with exposing more information, if needed, to
help the userspace make a more informed decision.

>
> I'm sure there will be more issues, but solving issues is what we do :-)
>
> > That said, as we already use a centralized DMA-buf allocator in
> > ChromeOS and don't really care about the MMAP mode, I'm definitely
> > biased here. We would need to hear from people working on userspace
> > which still uses it (if there is any).
> >
> > > > > It is not a matter of what language is better. Instead, it is a matter of
> > > > > not affecting code maintenance during the (probably long) transition period
> > > > > and beyond.
> > > > >
> > > > > If you see the past history, the transition from V4L to V4L2 took more than 10
> > > > > years - being possible to be done only with the help of libv4l, plus a
> > > > > lot of backward-compat code that we added. Still there were several
> > > > > regressions and we even had to quickly patch the Kernel and/or some apps
> > > > > that were using the uAPI on different ways.
> > > >
> > > > That's a different situation, because UAPI is involved.
> > > >
> > > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> > > >
> > > > Yes, vb -> vb2 would be a more appropriate comparison.
> > > >
> > > > > On both cases, there were very good technical reasons for the transition,
> > > > > in terms of missing needed features, broken memory models and serious
> > > > > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> > > >
> > > > It's a very similar situation now, vb2 doesn't work well on modern
> > > > hardware, but I still have hopes that it can be fixed without
> > > > affecting the driver-facing behavior. (We would probably need to
> > > > develop some unit tests that validate the driver-facing behavior to
> > > > ensure that.)
> > > >
> > > > > In the end, the authors of the core changes need to acquire legacy hardware
> > > > > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > > > > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > > > > as it required a lot of work.
> > > > >
> > > > > I can tell you: there's no fun on such changes: typically, companies won't
> > > > > pay someone to do changes on drivers for legacy hardware, specially
> > > > > when there are no real benefits, which is the case here, as the final result
> > > > > is just to keep the existing drivers to work with existing hardware,
> > > > > usually without any new features. So, the ones behind such core changes
> > > > > have to commit fixing drivers usually on their spare time.
> > > >
> > > > I don't get that argument. Wouldn't the same apply to any core change?
> > > > I think the reason we have driver maintainers is that they can help
> > > > with testing. Moreover, we need to invest into testing infrastructure
> > > > (which is what people have been doing recently via Media CI) to make
> > > > such changes less painful. Otherwise the subsystem will just bit-rot
> > > > and become useful for modern use cases.
> > >
> > > I've recently seen an increase in people experimenting with sourdough,
> > > kombucha, kimchi and other fermentation techniques, so rotting isn't
> > > always negative [*], but I assume you meant useless here, not useful :-)
> >
> > Yeah, definitely. I'd love it if bit-rotting led to computer software
> > becoming more useful, but sadly it's rarely the case.
> >
> > > * I'll draw the line at surströmming.
> > >
> > > > > > The vb2 framework can certainly use some more work, and esp. better support
> > > > > > for codecs, since that's where the main pain is at the moment.
> > > > > >
> > > > > > But I would need to see a proper proposal first. I assume that's what you
> > > > > > plan to present?
> > > > > >
> > > > > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > > > > would be nice if we could arrange this topic in a time slot friendly
> > > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > > > > team who would also be interested in joining remotely.
> > > > > >
> > > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > > > > of CEST.
>
> --
> Regards,
>
> Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

On Wed, Jun 12, 2024 at 06:33:14PM +0900, Tomasz Figa wrote:
> On Wed, Jun 12, 2024 at 6:20 PM Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 06:01:07PM +0900, Tomasz Figa wrote:
> > > On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart wrote:
> > > > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> > > > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > > > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > > > > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > > > > > >>
> > > > > > > >> Hi Hans, all,
> > > > > > > >>
> > > > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > > > > > >>
> > > > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > > > > > >>
> > > > > > > >> 30 minutes will do.
> > > > > > > >>
> > > > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > > > > > >> [1] https://lwn.net/Articles/970565
> > > > > > > >
> > > > > > > > Somewhat related to the topic: I see potential for a quite big
> > > > > > > > redesign of the videobuf2 framework going forward and recently with
> > > > > > > > more Rust adoption I'm starting to think it could benefit from being
> > > > > > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > > > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > > > > > with memory management, object lifetime and asynchronousness quite a
> > > > > > > > lot and we had a history of issues there. So it could be interesting
> > > > > > > > to hear everyone's thoughts.
> > > > > > >
> > > > > > > I think it is far too soon to write a framework like that in Rust.
> > > > > >
> > > > > > Agreed. I don't object redesigns in C to make it better - which could have
> > > > > > some colateral effect of making things easier for a future Rust adoption,
> > > > > > but such changes should be justified by themselves, and not because of a
> > > > > > language change.
> > > > >
> > > > > No, the thought of redesign doesn't come from the language change,
> > > > > it's the other way around. Since rewriting a lot of the code already,
> > > > > why not do it in a language that is generally considered better.
> > > > >
> > > > > > See: redesigns at the core will potentially affect lots of drivers,
> > > > > > so it needs very good technical reasons why doing it. Plus, it requires
> > > > > > comprehensive tests with different types of hardware/drivers to reduce the
> > > > > > risk of regressions. Depending on the changes, it may require extra tests
> > > > > > with devices that are outside complex camera world: radio, analog and digital
> > > > > > TV drivers - and even some input devices that use VB2 - to ensure that
> > > > > > nothing broke.
> > > > >
> > > > > We don't have to do it in an all-or-nothing way. We can start with an
> > > > > experimental new implementation in Rust, which could be gradually
> > > > > tested. It could even be done the same way as the vb -> vb2
> > > > > transition, although I suspect it wouldn't really be necessary, as I
> > > > > would like to see it more like a drop-in replacement. In general I
> > > > > think the API exposed outside of the framework wouldn't really change
> > > > > that much, it's more about the internal design.
> > > > >
> > > > > > > To be
> > > > > > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > > > > > second language in the kernel, instead of as an experiment.
> > > > > >
> > > > > > This is not enough: if the core starts to use a second language, all media
> > > > > > developers will be affected and will be required to have expertise on such
> > > > > > language.
> > > > >
> > > > > Let's be realistic, how many developers are actively touching vb2 these days?
> > > > >
> > > > > > That's not something that should happen without careful
> > > > > > analysis and plans that should include a gradual roll-up, lost of tests
> > > > > > with the affected drivers including the legacy ones and some strategy to
> > > > > > quickly solve regression issues.
> > > > >
> > > > > That said, I agree. It needs proper discussion and planning. That's
> > > > > why I'm proposing this as a topic. :)
> > > > > Moreover the redesign itself also needs proper discussion and is more
> > > > > of a long term goal, not something to land in the next few days.
> > > >
> > > > Focussing on this topic, if we're brainstorming memory management for
> > > > media devices, I'd like to throw in a controversial idea. In addition to
> > > > being clearer on the fact that USERPTR is deprecated,
> > >
> > > Definitely. This has been long overdue.
> > >
> > > >  I would like to
> > > > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > > > centralized buffer allocator, instead of having multiple allocation APIs
> > > > scattered in different places. There are design ideas in gralloc that we
> > > > could benefit from.
> > >
> > > Given that we now have DMA-buf heaps in mainline, it sounds much more
> > > realistic. It would certainly help eliminate some issues in the vb2
> > > layer, such as vb2-dma-sg having its own open coded page allocation
> > > that can't handle DMA addressing limitations (which can be fixed with
> > > some additions to the DMA API, but eliminating the problem altogether
> > > is way better than any other solution.)
> >
> > There are (at least) two issues we'll have to solve to make DMA heaps
> > more universally usable:
> >
> > - Memory allocation from DMA heaps isn't accounted in the process
> >   resource limits. This is one of the blockers for getting distributions
> >   to enable the heaps. We'll have to fix that.
> 
> I think the same applies to MMAP buffers today or am I missing something?

Correct. We could fix that too, but I'd rather focus on DMA heaps.

> That said, it's something that has to be fixed anyway. DMA heaps would
> make it easier to fix, since we would have one central place where the
> accounting could be implemented.
> 
> > - We need a userspace library to pick the right heap based on the memory
> >   constraints of the devices that you'll want to share the buffer with.
> >   This will require exposing those constraints to userspace somehow.
> 
> That is not a new problem either. Today with MMAP we also need the
> userspace to somehow choose the right exporter. So we could start with
> each video node pointing to the right heap (probably for each plane
> separately?), which would provide the same level of functionality as
> with MMAP today, but using a uniform API and a central implementation
> in the kernel.
> 
> Then we could follow up with exposing more information, if needed, to
> help the userspace make a more informed decision.

The two issues can be decoupled for the implementation phase, true, but
I think we'll have to showcase a design that solves them both if we want
to get adoption fo this API shift.

> > I'm sure there will be more issues, but solving issues is what we do :-)
> >
> > > That said, as we already use a centralized DMA-buf allocator in
> > > ChromeOS and don't really care about the MMAP mode, I'm definitely
> > > biased here. We would need to hear from people working on userspace
> > > which still uses it (if there is any).
> > >
> > > > > > It is not a matter of what language is better. Instead, it is a matter of
> > > > > > not affecting code maintenance during the (probably long) transition period
> > > > > > and beyond.
> > > > > >
> > > > > > If you see the past history, the transition from V4L to V4L2 took more than 10
> > > > > > years - being possible to be done only with the help of libv4l, plus a
> > > > > > lot of backward-compat code that we added. Still there were several
> > > > > > regressions and we even had to quickly patch the Kernel and/or some apps
> > > > > > that were using the uAPI on different ways.
> > > > >
> > > > > That's a different situation, because UAPI is involved.
> > > > >
> > > > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> > > > >
> > > > > Yes, vb -> vb2 would be a more appropriate comparison.
> > > > >
> > > > > > On both cases, there were very good technical reasons for the transition,
> > > > > > in terms of missing needed features, broken memory models and serious
> > > > > > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> > > > >
> > > > > It's a very similar situation now, vb2 doesn't work well on modern
> > > > > hardware, but I still have hopes that it can be fixed without
> > > > > affecting the driver-facing behavior. (We would probably need to
> > > > > develop some unit tests that validate the driver-facing behavior to
> > > > > ensure that.)
> > > > >
> > > > > > In the end, the authors of the core changes need to acquire legacy hardware
> > > > > > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > > > > > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > > > > > as it required a lot of work.
> > > > > >
> > > > > > I can tell you: there's no fun on such changes: typically, companies won't
> > > > > > pay someone to do changes on drivers for legacy hardware, specially
> > > > > > when there are no real benefits, which is the case here, as the final result
> > > > > > is just to keep the existing drivers to work with existing hardware,
> > > > > > usually without any new features. So, the ones behind such core changes
> > > > > > have to commit fixing drivers usually on their spare time.
> > > > >
> > > > > I don't get that argument. Wouldn't the same apply to any core change?
> > > > > I think the reason we have driver maintainers is that they can help
> > > > > with testing. Moreover, we need to invest into testing infrastructure
> > > > > (which is what people have been doing recently via Media CI) to make
> > > > > such changes less painful. Otherwise the subsystem will just bit-rot
> > > > > and become useful for modern use cases.
> > > >
> > > > I've recently seen an increase in people experimenting with sourdough,
> > > > kombucha, kimchi and other fermentation techniques, so rotting isn't
> > > > always negative [*], but I assume you meant useless here, not useful :-)
> > >
> > > Yeah, definitely. I'd love it if bit-rotting led to computer software
> > > becoming more useful, but sadly it's rarely the case.
> > >
> > > > * I'll draw the line at surströmming.
> > > >
> > > > > > > The vb2 framework can certainly use some more work, and esp. better support
> > > > > > > for codecs, since that's where the main pain is at the moment.
> > > > > > >
> > > > > > > But I would need to see a proper proposal first. I assume that's what you
> > > > > > > plan to present?
> > > > > > >
> > > > > > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > > > > > would be nice if we could arrange this topic in a time slot friendly
> > > > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > > > > > team who would also be interested in joining remotely.
> > > > > > >
> > > > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > > > > > of CEST.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Nicolas Dufresne]

Hi Tomasz,

Le mercredi 12 juin 2024 à 18:01 +0900, Tomasz Figa a écrit :
> >   I would like to
> > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > centralized buffer allocator, instead of having multiple allocation APIs
> > scattered in different places. There are design ideas in gralloc that we
> > could benefit from.
> > 
> 
> Given that we now have DMA-buf heaps in mainline, it sounds much more
> realistic. It would certainly help eliminate some issues in the vb2
> layer, such as vb2-dma-sg having its own open coded page allocation
> that can't handle DMA addressing limitations (which can be fixed with
> some additions to the DMA API, but eliminating the problem altogether
> is way better than any other solution.)
> 
> That said, as we already use a centralized DMA-buf allocator in
> ChromeOS and don't really care about the MMAP mode, I'm definitely
> biased here. We would need to hear from people working on userspace
> which still uses it (if there is any).

This aspect is what makes the V4L2 support in chromium really sad to non
ChromeOS users. ChromeOS makes arbitrary decisions like that codec video node
must be named video-dec0/enc0 (there is solutions to that using udev and
topologies btw), and decided that minigbm is the only way allocate memory for
these codecs on Linux. In practice, in every project where we need Chromium V4L2
codecs, we endup having to patch it to do MMAP/dmabuf export support and EGL
import support (the other way around).

I'm not bashing against ChromeOS, I just want to underline that this is far from
a solve problem, and no one is actively working on it. I don't see minigbm
becoming an acceptable goto library, since EOL (in Chromebook term) hardware get
removed and you need a CLA to contribute. I also strongly disagree that the
calculation of auxiliary buffer needed for codecs reference frame should be done
in a third party library. The driver must validate these sizes, and is the
logical place to have this logic, not a third party system allocator. On that
front though, its a bit like the video-decX dev node naming, there is a generic
solution, since the size is exposed through TRY/S_FMT calls already.

I'll repeat that as often as needed, there is a lot of gaps toward removing mmap
(dmabuf exportation) in CODEC drivers which ChromeOS only have hacks around and
no generic solution suitable for generic linux community. In FFMPEG and
GStreamer, we use mmap + dmabuf export because that is actually usable in a
generic manner today.

Nicolas

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

On Wed, Jun 12, 2024 at 04:09:53PM -0400, Nicolas Dufresne wrote:
> Le mercredi 12 juin 2024 à 18:01 +0900, Tomasz Figa a écrit :
> > >   I would like to
> > > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > > centralized buffer allocator, instead of having multiple allocation APIs
> > > scattered in different places. There are design ideas in gralloc that we
> > > could benefit from.
> > > 
> > 
> > Given that we now have DMA-buf heaps in mainline, it sounds much more
> > realistic. It would certainly help eliminate some issues in the vb2
> > layer, such as vb2-dma-sg having its own open coded page allocation
> > that can't handle DMA addressing limitations (which can be fixed with
> > some additions to the DMA API, but eliminating the problem altogether
> > is way better than any other solution.)
> > 
> > That said, as we already use a centralized DMA-buf allocator in
> > ChromeOS and don't really care about the MMAP mode, I'm definitely
> > biased here. We would need to hear from people working on userspace
> > which still uses it (if there is any).
> 
> This aspect is what makes the V4L2 support in chromium really sad to non
> ChromeOS users. ChromeOS makes arbitrary decisions like that codec video node
> must be named video-dec0/enc0 (there is solutions to that using udev and
> topologies btw), and decided that minigbm is the only way allocate memory for
> these codecs on Linux. In practice, in every project where we need Chromium V4L2
> codecs, we endup having to patch it to do MMAP/dmabuf export support and EGL
> import support (the other way around).
> 
> I'm not bashing against ChromeOS, I just want to underline that this is far from
> a solve problem,

I think we all agree on this. There's an extensive amount of work
required in this area.

> and no one is actively working on it.

That may change though, I've heard that some of the people involved in a
previous attempt are considering resuming the work. It of course doesn't
provide a guarantee of success.

> I don't see minigbm
> becoming an acceptable goto library, since EOL (in Chromebook term) hardware get
> removed and you need a CLA to contribute. I also strongly disagree that the
> calculation of auxiliary buffer needed for codecs reference frame should be done
> in a third party library. The driver must validate these sizes, and is the
> logical place to have this logic, not a third party system allocator. On that
> front though, its a bit like the video-decX dev node naming, there is a generic
> solution, since the size is exposed through TRY/S_FMT calls already.

I think the selection of an allocator (in a nutshell, what DMA heap to
use) and the selection of buffer parameters (size, format, stride,
modifiers, ...) are two related but distinct problems. We need to design
a solution that will address both, but keep them distinct so that one
can still use the future generic allocator API without having to
delegate all the allocation parameters decisions to a third-party
component.

> I'll repeat that as often as needed, there is a lot of gaps toward removing mmap
> (dmabuf exportation) in CODEC drivers which ChromeOS only have hacks around and
> no generic solution suitable for generic linux community. In FFMPEG and
> GStreamer, we use mmap + dmabuf export because that is actually usable in a
> generic manner today.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Chen-Yu Tsai]

On Wed, Jun 12, 2024 at 5:02 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> On Wed, Jun 12, 2024 at 5:34 PM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Tomasz,
> >
> > On Wed, Jun 12, 2024 at 05:22:34PM +0900, Tomasz Figa wrote:
> > > On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > > > Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > > > > On 6/12/24 06:12, Tomasz Figa wrote:
> > > > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > > > > >>
> > > > > >> Hi Hans, all,
> > > > > >>
> > > > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > > > >>
> > > > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > > > >>
> > > > > >> 30 minutes will do.
> > > > > >>
> > > > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > > > >> [1] https://lwn.net/Articles/970565
> > > > > >
> > > > > > Somewhat related to the topic: I see potential for a quite big
> > > > > > redesign of the videobuf2 framework going forward and recently with
> > > > > > more Rust adoption I'm starting to think it could benefit from being
> > > > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > > > with memory management, object lifetime and asynchronousness quite a
> > > > > > lot and we had a history of issues there. So it could be interesting
> > > > > > to hear everyone's thoughts.
> > > > >
> > > > > I think it is far too soon to write a framework like that in Rust.
> > > >
> > > > Agreed. I don't object redesigns in C to make it better - which could have
> > > > some colateral effect of making things easier for a future Rust adoption,
> > > > but such changes should be justified by themselves, and not because of a
> > > > language change.
> > >
> > > No, the thought of redesign doesn't come from the language change,
> > > it's the other way around. Since rewriting a lot of the code already,
> > > why not do it in a language that is generally considered better.
> > >
> > > > See: redesigns at the core will potentially affect lots of drivers,
> > > > so it needs very good technical reasons why doing it. Plus, it requires
> > > > comprehensive tests with different types of hardware/drivers to reduce the
> > > > risk of regressions. Depending on the changes, it may require extra tests
> > > > with devices that are outside complex camera world: radio, analog and digital
> > > > TV drivers - and even some input devices that use VB2 - to ensure that
> > > > nothing broke.
> > >
> > > We don't have to do it in an all-or-nothing way. We can start with an
> > > experimental new implementation in Rust, which could be gradually
> > > tested. It could even be done the same way as the vb -> vb2
> > > transition, although I suspect it wouldn't really be necessary, as I
> > > would like to see it more like a drop-in replacement. In general I
> > > think the API exposed outside of the framework wouldn't really change
> > > that much, it's more about the internal design.
> > >
> > > > > To be
> > > > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > > > second language in the kernel, instead of as an experiment.
> > > >
> > > > This is not enough: if the core starts to use a second language, all media
> > > > developers will be affected and will be required to have expertise on such
> > > > language.
> > >
> > > Let's be realistic, how many developers are actively touching vb2 these days?
> > >
> > > > That's not something that should happen without careful
> > > > analysis and plans that should include a gradual roll-up, lost of tests
> > > > with the affected drivers including the legacy ones and some strategy to
> > > > quickly solve regression issues.
> > >
> > > That said, I agree. It needs proper discussion and planning. That's
> > > why I'm proposing this as a topic. :)
> > > Moreover the redesign itself also needs proper discussion and is more
> > > of a long term goal, not something to land in the next few days.
> >
> > Focussing on this topic, if we're brainstorming memory management for
> > media devices, I'd like to throw in a controversial idea. In addition to
> > being clearer on the fact that USERPTR is deprecated,
>
> Definitely. This has been long overdue.
>
> >  I would like to
> > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > centralized buffer allocator, instead of having multiple allocation APIs
> > scattered in different places. There are design ideas in gralloc that we
> > could benefit from.
> >
>
> Given that we now have DMA-buf heaps in mainline, it sounds much more
> realistic. It would certainly help eliminate some issues in the vb2
> layer, such as vb2-dma-sg having its own open coded page allocation
> that can't handle DMA addressing limitations (which can be fixed with
> some additions to the DMA API, but eliminating the problem altogether
> is way better than any other solution.)
>
> That said, as we already use a centralized DMA-buf allocator in
> ChromeOS and don't really care about the MMAP mode, I'm definitely
> biased here. We would need to hear from people working on userspace
> which still uses it (if there is any).

That's not entirely true. In the decode path, if a post-processor is
needed (mostly for detiling), then the decoder capture buffers are
allocated with the MMAP mode.


ChenYu

> > > > It is not a matter of what language is better. Instead, it is a matter of
> > > > not affecting code maintenance during the (probably long) transition period
> > > > and beyond.
> > > >
> > > > If you see the past history, the transition from V4L to V4L2 took more than 10
> > > > years - being possible to be done only with the help of libv4l, plus a
> > > > lot of backward-compat code that we added. Still there were several
> > > > regressions and we even had to quickly patch the Kernel and/or some apps
> > > > that were using the uAPI on different ways.
> > >
> > > That's a different situation, because UAPI is involved.
> > >
> > > > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> > >
> > > Yes, vb -> vb2 would be a more appropriate comparison.
> > >
> > > > On both cases, there were very good technical reasons for the transition,
> > > > in terms of missing needed features, broken memory models and serious
> > > > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> > >
> > > It's a very similar situation now, vb2 doesn't work well on modern
> > > hardware, but I still have hopes that it can be fixed without
> > > affecting the driver-facing behavior. (We would probably need to
> > > develop some unit tests that validate the driver-facing behavior to
> > > ensure that.)
> > >
> > > > In the end, the authors of the core changes need to acquire legacy hardware
> > > > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > > > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > > > as it required a lot of work.
> > > >
> > > > I can tell you: there's no fun on such changes: typically, companies won't
> > > > pay someone to do changes on drivers for legacy hardware, specially
> > > > when there are no real benefits, which is the case here, as the final result
> > > > is just to keep the existing drivers to work with existing hardware,
> > > > usually without any new features. So, the ones behind such core changes
> > > > have to commit fixing drivers usually on their spare time.
> > >
> > > I don't get that argument. Wouldn't the same apply to any core change?
> > > I think the reason we have driver maintainers is that they can help
> > > with testing. Moreover, we need to invest into testing infrastructure
> > > (which is what people have been doing recently via Media CI) to make
> > > such changes less painful. Otherwise the subsystem will just bit-rot
> > > and become useful for modern use cases.
> >
> > I've recently seen an increase in people experimenting with sourdough,
> > kombucha, kimchi and other fermentation techniques, so rotting isn't
> > always negative [*], but I assume you meant useless here, not useful :-)
>
> Yeah, definitely. I'd love it if bit-rotting led to computer software
> becoming more useful, but sadly it's rarely the case.
>
> Best regards,
> Tomasz
>
> >
> > * I'll draw the line at surströmming.
> >
> > > > > The vb2 framework can certainly use some more work, and esp. better support
> > > > > for codecs, since that's where the main pain is at the moment.
> > > > >
> > > > > But I would need to see a proper proposal first. I assume that's what you
> > > > > plan to present?
> > > > >
> > > > > > That said, I wouldn't be able to travel this time unfortunately, so it
> > > > > > would be nice if we could arrange this topic in a time slot friendly
> > > > > > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > > > > > team who would also be interested in joining remotely.
> > > > >
> > > > > That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> > > > > of CEST.
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2024 11:34:30 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> Focussing on this topic, if we're brainstorming memory management for
> media devices, I'd like to throw in a controversial idea. In addition to
> being clearer on the fact that USERPTR is deprecated, I would like to
> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> centralized buffer allocator, instead of having multiple allocation APIs
> scattered in different places. There are design ideas in gralloc that we
> could benefit from.

Deprecating USERPTR is doable, as not many apps use it, and they're
mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
V4L2 core is a different history: lots of different userspace programs,
including browsers and proprietary apps like zoom, etc. rely on MMAP
support. We can only consider deprecating MMAP once applications switch 
to DMABUF.

Thanks,
Mauro

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

Hi Mauro,

On Wed, Jun 12, 2024 at 10:44:06PM +0200, Mauro Carvalho Chehab wrote:
> Em Wed, 12 Jun 2024 11:34:30 +0300 Laurent Pinchart escreveu:
> 
> > Focussing on this topic, if we're brainstorming memory management for
> > media devices, I'd like to throw in a controversial idea. In addition to
> > being clearer on the fact that USERPTR is deprecated, I would like to
> > deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> > centralized buffer allocator, instead of having multiple allocation APIs
> > scattered in different places. There are design ideas in gralloc that we
> > could benefit from.
> 
> Deprecating USERPTR is doable, as not many apps use it, and they're
> mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
> V4L2 core is a different history: lots of different userspace programs,
> including browsers and proprietary apps like zoom, etc. rely on MMAP
> support. We can only consider deprecating MMAP once applications switch 
> to DMABUF.

Deprecating doesn't mean dropping it right away, it means telling
application developers that DMABUF is the recommended way. We will still
have to support MMAP for a long time, including fixing bugs in it, as
that will be a long transition. And it first requires solving the
problem of centralizing allocation for DMABUF. It won't happen
overnight, but I'm trying to gather support for the idea, and get people
to collaborate on solving the technical problems that are currently
blocking this long term evolution. If the media subsystem endorsed the
effort, basically saying publicly that we are fine deprecating MMAP in
principle once a good replacement will be available, it may help. I
don't expect the deprecation to happen before at least two years, and
the removal from the kernel would probably take another 10 to 15 years
:-)

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 12/06/2024 22:52, Laurent Pinchart wrote:
> Hi Mauro,
> 
> On Wed, Jun 12, 2024 at 10:44:06PM +0200, Mauro Carvalho Chehab wrote:
>> Em Wed, 12 Jun 2024 11:34:30 +0300 Laurent Pinchart escreveu:
>>
>>> Focussing on this topic, if we're brainstorming memory management for
>>> media devices, I'd like to throw in a controversial idea. In addition to
>>> being clearer on the fact that USERPTR is deprecated, I would like to
>>> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
>>> centralized buffer allocator, instead of having multiple allocation APIs
>>> scattered in different places. There are design ideas in gralloc that we
>>> could benefit from.
>>
>> Deprecating USERPTR is doable, as not many apps use it, and they're
>> mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
>> V4L2 core is a different history: lots of different userspace programs,
>> including browsers and proprietary apps like zoom, etc. rely on MMAP
>> support. We can only consider deprecating MMAP once applications switch 
>> to DMABUF.
> 
> Deprecating doesn't mean dropping it right away, it means telling
> application developers that DMABUF is the recommended way. We will still
> have to support MMAP for a long time, including fixing bugs in it, as
> that will be a long transition. And it first requires solving the
> problem of centralizing allocation for DMABUF. It won't happen
> overnight, but I'm trying to gather support for the idea, and get people
> to collaborate on solving the technical problems that are currently
> blocking this long term evolution. If the media subsystem endorsed the
> effort, basically saying publicly that we are fine deprecating MMAP in
> principle once a good replacement will be available, it may help. I
> don't expect the deprecation to happen before at least two years, and
> the removal from the kernel would probably take another 10 to 15 years
> :-)
> 

IMHO you cannot removed MMAP support: it is the only streaming I/O method
that is supported by all drivers, whereas DMABUF isn't. And many, many userspace
applications use that. Nor does it pose problems: it just works.

USERPTR support is another matter: there have been problems with it, and
the vb2 code is hard to understand and to support.

I wouldn't shed a tear if it disappears. The strategy would be to first
make sure any driver supporting USERPTR also supports DMABUF, and then
put USERPTR under a kernel config option. Initially it would default to y,
but issue a warning, and later (after a few years) it can default to n
and eventually be removed.

Regards,

	Hans

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

Hi Hans,

On Thu, Jun 13, 2024 at 09:08:55AM +0200, Hans Verkuil wrote:
> On 12/06/2024 22:52, Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 10:44:06PM +0200, Mauro Carvalho Chehab wrote:
> >> Em Wed, 12 Jun 2024 11:34:30 +0300 Laurent Pinchart escreveu:
> >>
> >>> Focussing on this topic, if we're brainstorming memory management for
> >>> media devices, I'd like to throw in a controversial idea. In addition to
> >>> being clearer on the fact that USERPTR is deprecated, I would like to
> >>> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> >>> centralized buffer allocator, instead of having multiple allocation APIs
> >>> scattered in different places. There are design ideas in gralloc that we
> >>> could benefit from.
> >>
> >> Deprecating USERPTR is doable, as not many apps use it, and they're
> >> mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
> >> V4L2 core is a different history: lots of different userspace programs,
> >> including browsers and proprietary apps like zoom, etc. rely on MMAP
> >> support. We can only consider deprecating MMAP once applications switch 
> >> to DMABUF.
> > 
> > Deprecating doesn't mean dropping it right away, it means telling
> > application developers that DMABUF is the recommended way. We will still
> > have to support MMAP for a long time, including fixing bugs in it, as
> > that will be a long transition. And it first requires solving the
> > problem of centralizing allocation for DMABUF. It won't happen
> > overnight, but I'm trying to gather support for the idea, and get people
> > to collaborate on solving the technical problems that are currently
> > blocking this long term evolution. If the media subsystem endorsed the
> > effort, basically saying publicly that we are fine deprecating MMAP in
> > principle once a good replacement will be available, it may help. I
> > don't expect the deprecation to happen before at least two years, and
> > the removal from the kernel would probably take another 10 to 15 years
> > :-)
> 
> IMHO you cannot removed MMAP support: it is the only streaming I/O method
> that is supported by all drivers, whereas DMABUF isn't. And many, many userspace
> applications use that. Nor does it pose problems: it just works.

I may have failed to get my point across properly, so I'll try again :-)

What I would like to do is

1. Explore how we can implement a centralized allocator that
applications can use on any Linux system to provide dmabuf instances.

2. Implement that allocator.

3. Deprecate MMAP, meaning documenting that users of V4L2 should use the
centralized allocator and DMABUF. No code change in V4L2, no removal of
MMAP, and bugs in MMAP support would keep being addressed.

4. 5-10 years later, start scheduling MMAP removal, as in setting a date
for it.

5. 5-10 years more in the future, drop MMAP when nobody will be using it
anymore.

It's phases 1 to 3 that I'm the most interested in. 4 and 5 are just
about dropping code *when* MMAP isn't used anymore *iff* that ever
happens.

> USERPTR support is another matter: there have been problems with it, and
> the vb2 code is hard to understand and to support.
> 
> I wouldn't shed a tear if it disappears. The strategy would be to first
> make sure any driver supporting USERPTR also supports DMABUF, and then
> put USERPTR under a kernel config option. Initially it would default to y,
> but issue a warning, and later (after a few years) it can default to n
> and eventually be removed.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 13/06/2024 11:12, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Thu, Jun 13, 2024 at 09:08:55AM +0200, Hans Verkuil wrote:
>> On 12/06/2024 22:52, Laurent Pinchart wrote:
>>> On Wed, Jun 12, 2024 at 10:44:06PM +0200, Mauro Carvalho Chehab wrote:
>>>> Em Wed, 12 Jun 2024 11:34:30 +0300 Laurent Pinchart escreveu:
>>>>
>>>>> Focussing on this topic, if we're brainstorming memory management for
>>>>> media devices, I'd like to throw in a controversial idea. In addition to
>>>>> being clearer on the fact that USERPTR is deprecated, I would like to
>>>>> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
>>>>> centralized buffer allocator, instead of having multiple allocation APIs
>>>>> scattered in different places. There are design ideas in gralloc that we
>>>>> could benefit from.
>>>>
>>>> Deprecating USERPTR is doable, as not many apps use it, and they're
>>>> mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
>>>> V4L2 core is a different history: lots of different userspace programs,
>>>> including browsers and proprietary apps like zoom, etc. rely on MMAP
>>>> support. We can only consider deprecating MMAP once applications switch 
>>>> to DMABUF.
>>>
>>> Deprecating doesn't mean dropping it right away, it means telling
>>> application developers that DMABUF is the recommended way. We will still
>>> have to support MMAP for a long time, including fixing bugs in it, as
>>> that will be a long transition. And it first requires solving the
>>> problem of centralizing allocation for DMABUF. It won't happen
>>> overnight, but I'm trying to gather support for the idea, and get people
>>> to collaborate on solving the technical problems that are currently
>>> blocking this long term evolution. If the media subsystem endorsed the
>>> effort, basically saying publicly that we are fine deprecating MMAP in
>>> principle once a good replacement will be available, it may help. I
>>> don't expect the deprecation to happen before at least two years, and
>>> the removal from the kernel would probably take another 10 to 15 years
>>> :-)
>>
>> IMHO you cannot removed MMAP support: it is the only streaming I/O method
>> that is supported by all drivers, whereas DMABUF isn't. And many, many userspace
>> applications use that. Nor does it pose problems: it just works.
> 
> I may have failed to get my point across properly, so I'll try again :-)
> 
> What I would like to do is
> 
> 1. Explore how we can implement a centralized allocator that
> applications can use on any Linux system to provide dmabuf instances.

Would be nice.

> 
> 2. Implement that allocator.

Good.

2.5: start adding DMABUF support to all drivers that do not yet
support this.

> 
> 3. Deprecate MMAP, meaning documenting that users of V4L2 should use the
> centralized allocator and DMABUF. No code change in V4L2, no removal of
> MMAP, and bugs in MMAP support would keep being addressed.
> 
> 4. 5-10 years later, start scheduling MMAP removal, as in setting a date
> for it.
> 
> 5. 5-10 years more in the future, drop MMAP when nobody will be using it
> anymore.

3-5: seems pointless to me. It would break many userspace applications.

Frankly, MMAP just works, and has since forever. Perhaps at some point
we might switch to dmabufs internally to vb2 (that would require that
the driver can call that central allocator!) if that would simplify
matters, but in the uAPI we need to keep MMAP.

Regards,

	Hans

> 
> It's phases 1 to 3 that I'm the most interested in. 4 and 5 are just
> about dropping code *when* MMAP isn't used anymore *iff* that ever
> happens.
> 
>> USERPTR support is another matter: there have been problems with it, and
>> the vb2 code is hard to understand and to support.
>>
>> I wouldn't shed a tear if it disappears. The strategy would be to first
>> make sure any driver supporting USERPTR also supports DMABUF, and then
>> put USERPTR under a kernel config option. Initially it would default to y,
>> but issue a warning, and later (after a few years) it can default to n
>> and eventually be removed.
> 

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

On Thu, Jun 13, 2024 at 11:38:47AM +0200, Hans Verkuil wrote:
> On 13/06/2024 11:12, Laurent Pinchart wrote:
> > On Thu, Jun 13, 2024 at 09:08:55AM +0200, Hans Verkuil wrote:
> >> On 12/06/2024 22:52, Laurent Pinchart wrote:
> >>> On Wed, Jun 12, 2024 at 10:44:06PM +0200, Mauro Carvalho Chehab wrote:
> >>>> Em Wed, 12 Jun 2024 11:34:30 +0300 Laurent Pinchart escreveu:
> >>>>
> >>>>> Focussing on this topic, if we're brainstorming memory management for
> >>>>> media devices, I'd like to throw in a controversial idea. In addition to
> >>>>> being clearer on the fact that USERPTR is deprecated, I would like to
> >>>>> deprecate MMAP too and only focus on DMABUF. I believe Linux needs a
> >>>>> centralized buffer allocator, instead of having multiple allocation APIs
> >>>>> scattered in different places. There are design ideas in gralloc that we
> >>>>> could benefit from.
> >>>>
> >>>> Deprecating USERPTR is doable, as not many apps use it, and they're
> >>>> mostly focused on complex camera/ARM scenario. Now, deprecating MMAP at 
> >>>> V4L2 core is a different history: lots of different userspace programs,
> >>>> including browsers and proprietary apps like zoom, etc. rely on MMAP
> >>>> support. We can only consider deprecating MMAP once applications switch 
> >>>> to DMABUF.
> >>>
> >>> Deprecating doesn't mean dropping it right away, it means telling
> >>> application developers that DMABUF is the recommended way. We will still
> >>> have to support MMAP for a long time, including fixing bugs in it, as
> >>> that will be a long transition. And it first requires solving the
> >>> problem of centralizing allocation for DMABUF. It won't happen
> >>> overnight, but I'm trying to gather support for the idea, and get people
> >>> to collaborate on solving the technical problems that are currently
> >>> blocking this long term evolution. If the media subsystem endorsed the
> >>> effort, basically saying publicly that we are fine deprecating MMAP in
> >>> principle once a good replacement will be available, it may help. I
> >>> don't expect the deprecation to happen before at least two years, and
> >>> the removal from the kernel would probably take another 10 to 15 years
> >>> :-)
> >>
> >> IMHO you cannot removed MMAP support: it is the only streaming I/O method
> >> that is supported by all drivers, whereas DMABUF isn't. And many, many userspace
> >> applications use that. Nor does it pose problems: it just works.
> > 
> > I may have failed to get my point across properly, so I'll try again :-)
> > 
> > What I would like to do is
> > 
> > 1. Explore how we can implement a centralized allocator that
> > applications can use on any Linux system to provide dmabuf instances.
> 
> Would be nice.
> 
> > 2. Implement that allocator.
> 
> Good.
> 
> 2.5: start adding DMABUF support to all drivers that do not yet
> support this.
> 
> > 3. Deprecate MMAP, meaning documenting that users of V4L2 should use the
> > centralized allocator and DMABUF. No code change in V4L2, no removal of
> > MMAP, and bugs in MMAP support would keep being addressed.
> > 
> > 4. 5-10 years later, start scheduling MMAP removal, as in setting a date
> > for it.
> > 
> > 5. 5-10 years more in the future, drop MMAP when nobody will be using it
> > anymore.
> 
> 3-5: seems pointless to me. It would break many userspace applications.
> 
> Frankly, MMAP just works, and has since forever. Perhaps at some point
> we might switch to dmabufs internally to vb2 (that would require that
> the driver can call that central allocator!) if that would simplify
> matters, but in the uAPI we need to keep MMAP.

3 is important in my opinion, it's about telling, once 1-2 are in place,
that new applications should use the new API. It doesn't break anything
because it doesn't change any code in the kernel. The only thing that
will change is the V4L2 documentation, to tell users "please use DMABUF
for new applications". Nothing more than that.

4-5 are not strictly necessary, and are for much later anyway. They're
about dropping a feature once all users will be gone. If it happens, it
won't break anything because nobody will notice :-)

> > It's phases 1 to 3 that I'm the most interested in. 4 and 5 are just
> > about dropping code *when* MMAP isn't used anymore *iff* that ever
> > happens.
> > 
> >> USERPTR support is another matter: there have been problems with it, and
> >> the vb2 code is hard to understand and to support.
> >>
> >> I wouldn't shed a tear if it disappears. The strategy would be to first
> >> make sure any driver supporting USERPTR also supports DMABUF, and then
> >> put USERPTR under a kernel config option. Initially it would default to y,
> >> but issue a warning, and later (after a few years) it can default to n
> >> and eventually be removed.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2024 17:22:34 +0900
Tomasz Figa <tfiga@chromium.org> escreveu:

> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab
> <mchehab@kernel.org> wrote:
> >
> > Em Wed, 12 Jun 2024 08:46:50 +0200
> > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >  
> > > On 6/12/24 06:12, Tomasz Figa wrote:  
> > > > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> > > > <daniel.almeida@collabora.com> wrote:  
> > > >>
> > > >> Hi Hans, all,
> > > >>
> > > >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > > >>
> > > >> Please note that these are new submissions that are unrelated with what was discussed last year.
> > > >>
> > > >> 30 minutes will do.
> > > >>
> > > >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > > >> [1] https://lwn.net/Articles/970565  
> > > >
> > > > Somewhat related to the topic: I see potential for a quite big
> > > > redesign of the videobuf2 framework going forward and recently with
> > > > more Rust adoption I'm starting to think it could benefit from being
> > > > implemented in Rust, since we would have to rewrite it quite a bit
> > > > anyway. Especially since it's a part of the subsystem that has to deal
> > > > with memory management, object lifetime and asynchronousness quite a
> > > > lot and we had a history of issues there. So it could be interesting
> > > > to hear everyone's thoughts.  
> > >
> > > I think it is far too soon to write a framework like that in Rust.  
> >
> > Agreed. I don't object redesigns in C to make it better - which could have
> > some colateral effect of making things easier for a future Rust adoption,
> > but such changes should be justified by themselves, and not because of a
> > language change.  
> 
> No, the thought of redesign doesn't come from the language change,
> it's the other way around. Since rewriting a lot of the code already,
> why not do it in a language that is generally considered better.

As Hans said, Rast has experimental support. We can't have drivers
depending on experimental stuff.

> 
> >
> > See: redesigns at the core will potentially affect lots of drivers,
> > so it needs very good technical reasons why doing it. Plus, it requires
> > comprehensive tests with different types of hardware/drivers to reduce the
> > risk of regressions. Depending on the changes, it may require extra tests
> > with devices that are outside complex camera world: radio, analog and digital
> > TV drivers - and even some input devices that use VB2 - to ensure that
> > nothing broke.  
> 
> We don't have to do it in an all-or-nothing way. We can start with an
> experimental new implementation in Rust, which could be gradually
> tested. It could even be done the same way as the vb -> vb2
> transition, although I suspect it wouldn't really be necessary, as I
> would like to see it more like a drop-in replacement. In general I
> think the API exposed outside of the framework wouldn't really change
> that much, it's more about the internal design.

Having two implementations of the same logic doesn't sound reasonable,
as it doubles the maintainership effort: all changes done on one
implementation needs to be moved to the other one.

Btw, we also have seem this problem before with VB and, up to some
sense, with VB2, as some drivers used to have their own buffer
handling implementation that usually started from a VB or VB2 fork.

So, if VB2 has issues, let's fix it in C code.

> > > To be
> > > honest, I won't even consider it until Linus officially accepts Rust as a
> > > second language in the kernel, instead of as an experiment.  
> >
> > This is not enough: if the core starts to use a second language, all media
> > developers will be affected and will be required to have expertise on such
> > language.  
> 
> Let's be realistic, how many developers are actively touching vb2 these days?

How many developers don't need VB2? Hopefully none :-)

> > That's not something that should happen without careful
> > analysis and plans that should include a gradual roll-up, lost of tests
> > with the affected drivers including the legacy ones and some strategy to
> > quickly solve regression issues.  
> 
> That said, I agree. It needs proper discussion and planning. That's
> why I'm proposing this as a topic. :)
> Moreover the redesign itself also needs proper discussion and is more
> of a long term goal, not something to land in the next few days.
> 
> >
> > It is not a matter of what language is better. Instead, it is a matter of
> > not affecting code maintenance during the (probably long) transition period
> > and beyond.
> >
> > If you see the past history, the transition from V4L to V4L2 took more than 10
> > years - being possible to be done only with the help of libv4l, plus a
> > lot of backward-compat code that we added. Still there were several
> > regressions and we even had to quickly patch the Kernel and/or some apps
> > that were using the uAPI on different ways.  
> 
> That's a different situation, because UAPI is involved.

It is different, but similar, up to some sense, as a change at VB2 
implementation will likely affect its kAPI, its behavior or both.

The point I'm underlining is that core redesigns do affect existing
drivers usually on unexpected ways.

> 
> >
> > Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> >  
> 
> Yes, vb -> vb2 would be a more appropriate comparison.
> 
> > On both cases, there were very good technical reasons for the transition,
> > in terms of missing needed features, broken memory models and serious
> > troubles that utterly causing VB1 to not work well on non-x86 hardware.
> >  
> 
> It's a very similar situation now, vb2 doesn't work well on modern
> hardware, but I still have hopes that it can be fixed without
> affecting the driver-facing behavior. (We would probably need to
> develop some unit tests that validate the driver-facing behavior to
> ensure that.)
> 
> > In the end, the authors of the core changes need to acquire legacy hardware
> > and to do lots of driver-specific changes to avoid breaking existing stuff.
> > Hans and I had to dedicate a lot of time and efforts on such transitions,
> > as it required a lot of work.
> >
> > I can tell you: there's no fun on such changes: typically, companies won't
> > pay someone to do changes on drivers for legacy hardware, specially
> > when there are no real benefits, which is the case here, as the final result
> > is just to keep the existing drivers to work with existing hardware,
> > usually without any new features. So, the ones behind such core changes
> > have to commit fixing drivers usually on their spare time.
> >  
> 
> I don't get that argument. Wouldn't the same apply to any core change?

It depends of the type of change. For instance, an addition of a new
V4L2 control should not cause regressions to existing drivers. The
same would be true if one adds a new memory allocation component for
VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..):
only drivers using the new way would be affected.

> I think the reason we have driver maintainers is that they can help
> with testing. Moreover, we need to invest into testing infrastructure
> (which is what people have been doing recently via Media CI) to make
> such changes less painful. Otherwise the subsystem will just bit-rot
> and become useful for modern use cases.

Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover
actual drivers. For that, we would need to invest on a CI solution
integrated with lots of different hardware pieces, to check for actual
driver regressions.

On one of my previous work, the company I used to work had that: they
had some monitors display some things, and the camera captured input
were compared to what the monitor were actually displaying. Doable, but
expensive. 

Regards,
Mauro

Thanks,
Mauro

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 12/06/2024 22:35, Mauro Carvalho Chehab wrote:
> Em Wed, 12 Jun 2024 17:22:34 +0900
> Tomasz Figa <tfiga@chromium.org> escreveu:
> 
>> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab
>> <mchehab@kernel.org> wrote:
>>>
>>> Em Wed, 12 Jun 2024 08:46:50 +0200
>>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>>>  
>>>> On 6/12/24 06:12, Tomasz Figa wrote:  
>>>>> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
>>>>> <daniel.almeida@collabora.com> wrote:  
>>>>>>
>>>>>> Hi Hans, all,
>>>>>>
>>>>>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
>>>>>>
>>>>>> Please note that these are new submissions that are unrelated with what was discussed last year.
>>>>>>
>>>>>> 30 minutes will do.
>>>>>>
>>>>>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
>>>>>> [1] https://lwn.net/Articles/970565  
>>>>>
>>>>> Somewhat related to the topic: I see potential for a quite big
>>>>> redesign of the videobuf2 framework going forward and recently with
>>>>> more Rust adoption I'm starting to think it could benefit from being
>>>>> implemented in Rust, since we would have to rewrite it quite a bit
>>>>> anyway. Especially since it's a part of the subsystem that has to deal
>>>>> with memory management, object lifetime and asynchronousness quite a
>>>>> lot and we had a history of issues there. So it could be interesting
>>>>> to hear everyone's thoughts.  
>>>>
>>>> I think it is far too soon to write a framework like that in Rust.  
>>>
>>> Agreed. I don't object redesigns in C to make it better - which could have
>>> some colateral effect of making things easier for a future Rust adoption,
>>> but such changes should be justified by themselves, and not because of a
>>> language change.  
>>
>> No, the thought of redesign doesn't come from the language change,
>> it's the other way around. Since rewriting a lot of the code already,
>> why not do it in a language that is generally considered better.
> 
> As Hans said, Rast has experimental support. We can't have drivers
> depending on experimental stuff.

Indeed.

While discussing Rust for experimental drivers or codec libraries is
interesting (and I am doing a Rust course, so hopefully I have a better
understanding of what's involved by the upcoming media summit), using
it for core media frameworks is simply a hard NACK until Linus blesses
Rust as a second kernel language.

So don't spend your valuable time on that.

> 
>>
>>>
>>> See: redesigns at the core will potentially affect lots of drivers,
>>> so it needs very good technical reasons why doing it. Plus, it requires
>>> comprehensive tests with different types of hardware/drivers to reduce the
>>> risk of regressions. Depending on the changes, it may require extra tests
>>> with devices that are outside complex camera world: radio, analog and digital
>>> TV drivers - and even some input devices that use VB2 - to ensure that
>>> nothing broke.  
>>
>> We don't have to do it in an all-or-nothing way. We can start with an
>> experimental new implementation in Rust, which could be gradually
>> tested. It could even be done the same way as the vb -> vb2
>> transition, although I suspect it wouldn't really be necessary, as I
>> would like to see it more like a drop-in replacement. In general I
>> think the API exposed outside of the framework wouldn't really change
>> that much, it's more about the internal design.

It makes no sense to have a C and a Rust version of vb2. This framework
is critical to all drivers, and we're not going to support two versions
and fix bugs/add features in two places. Again, it's a hard NACK. Don't
waste time on that.

If there are ideas to make vb2 better, then I am all for that.

I just want to mention two things here:

For most drivers, using vb2 is just fine: the work a driver needs to do is
quite straightforward. Exceptions are codec drivers and possibly complex
camera drivers when they need to use requests (not certain yet).

Internally vb2 is quite complex, but that's because what it does is quite
complex. And that's fine. If the internal structure can be improved to
make it less complex, then I'm all for that, but there is no magic bullet
(including using Rust instead of C) that suddenly makes everything simple.

Generally I prefer to have the complexity in core frameworks, that will
only make life easier for the driver developers.

To summarize:

Until Rust is accepted by Linus as a second kernel language, as media
maintainer I will NACK core media frameworks written in Rust. I won't
spend time on it, it's an immediate NACK from me.

Note that this doesn't imply that once Linus *does* accept Rust, that we
are OK with core frameworks written in Rust. But that will be a separate
discussion once that happens.

Regards,

	Hans

> 
> Having two implementations of the same logic doesn't sound reasonable,
> as it doubles the maintainership effort: all changes done on one
> implementation needs to be moved to the other one.
> 
> Btw, we also have seem this problem before with VB and, up to some
> sense, with VB2, as some drivers used to have their own buffer
> handling implementation that usually started from a VB or VB2 fork.
> 
> So, if VB2 has issues, let's fix it in C code.
> 
>>>> To be
>>>> honest, I won't even consider it until Linus officially accepts Rust as a
>>>> second language in the kernel, instead of as an experiment.  
>>>
>>> This is not enough: if the core starts to use a second language, all media
>>> developers will be affected and will be required to have expertise on such
>>> language.  
>>
>> Let's be realistic, how many developers are actively touching vb2 these days?
> 
> How many developers don't need VB2? Hopefully none :-)
> 
>>> That's not something that should happen without careful
>>> analysis and plans that should include a gradual roll-up, lost of tests
>>> with the affected drivers including the legacy ones and some strategy to
>>> quickly solve regression issues.  
>>
>> That said, I agree. It needs proper discussion and planning. That's
>> why I'm proposing this as a topic. :)
>> Moreover the redesign itself also needs proper discussion and is more
>> of a long term goal, not something to land in the next few days.
>>
>>>
>>> It is not a matter of what language is better. Instead, it is a matter of
>>> not affecting code maintenance during the (probably long) transition period
>>> and beyond.
>>>
>>> If you see the past history, the transition from V4L to V4L2 took more than 10
>>> years - being possible to be done only with the help of libv4l, plus a
>>> lot of backward-compat code that we added. Still there were several
>>> regressions and we even had to quickly patch the Kernel and/or some apps
>>> that were using the uAPI on different ways.  
>>
>> That's a different situation, because UAPI is involved.
> 
> It is different, but similar, up to some sense, as a change at VB2 
> implementation will likely affect its kAPI, its behavior or both.
> 
> The point I'm underlining is that core redesigns do affect existing
> drivers usually on unexpected ways.
> 
>>
>>>
>>> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
>>>  
>>
>> Yes, vb -> vb2 would be a more appropriate comparison.
>>
>>> On both cases, there were very good technical reasons for the transition,
>>> in terms of missing needed features, broken memory models and serious
>>> troubles that utterly causing VB1 to not work well on non-x86 hardware.
>>>  
>>
>> It's a very similar situation now, vb2 doesn't work well on modern
>> hardware, but I still have hopes that it can be fixed without
>> affecting the driver-facing behavior. (We would probably need to
>> develop some unit tests that validate the driver-facing behavior to
>> ensure that.)
>>
>>> In the end, the authors of the core changes need to acquire legacy hardware
>>> and to do lots of driver-specific changes to avoid breaking existing stuff.
>>> Hans and I had to dedicate a lot of time and efforts on such transitions,
>>> as it required a lot of work.
>>>
>>> I can tell you: there's no fun on such changes: typically, companies won't
>>> pay someone to do changes on drivers for legacy hardware, specially
>>> when there are no real benefits, which is the case here, as the final result
>>> is just to keep the existing drivers to work with existing hardware,
>>> usually without any new features. So, the ones behind such core changes
>>> have to commit fixing drivers usually on their spare time.
>>>  
>>
>> I don't get that argument. Wouldn't the same apply to any core change?
> 
> It depends of the type of change. For instance, an addition of a new
> V4L2 control should not cause regressions to existing drivers. The
> same would be true if one adds a new memory allocation component for
> VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..):
> only drivers using the new way would be affected.
> 
>> I think the reason we have driver maintainers is that they can help
>> with testing. Moreover, we need to invest into testing infrastructure
>> (which is what people have been doing recently via Media CI) to make
>> such changes less painful. Otherwise the subsystem will just bit-rot
>> and become useful for modern use cases.
> 
> Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover
> actual drivers. For that, we would need to invest on a CI solution
> integrated with lots of different hardware pieces, to check for actual
> driver regressions.
> 
> On one of my previous work, the company I used to work had that: they
> had some monitors display some things, and the camera captured input
> were compared to what the monitor were actually displaying. Doable, but
> expensive. 
> 
> Regards,
> Mauro
> 
> Thanks,
> Mauro

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Thu, Jun 13, 2024 at 4:38 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 12/06/2024 22:35, Mauro Carvalho Chehab wrote:
> > Em Wed, 12 Jun 2024 17:22:34 +0900
> > Tomasz Figa <tfiga@chromium.org> escreveu:
> >
> >> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab
> >> <mchehab@kernel.org> wrote:
> >>>
> >>> Em Wed, 12 Jun 2024 08:46:50 +0200
> >>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >>>
> >>>> On 6/12/24 06:12, Tomasz Figa wrote:
> >>>>> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> >>>>> <daniel.almeida@collabora.com> wrote:
> >>>>>>
> >>>>>> Hi Hans, all,
> >>>>>>
> >>>>>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> >>>>>>
> >>>>>> Please note that these are new submissions that are unrelated with what was discussed last year.
> >>>>>>
> >>>>>> 30 minutes will do.
> >>>>>>
> >>>>>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> >>>>>> [1] https://lwn.net/Articles/970565
> >>>>>
> >>>>> Somewhat related to the topic: I see potential for a quite big
> >>>>> redesign of the videobuf2 framework going forward and recently with
> >>>>> more Rust adoption I'm starting to think it could benefit from being
> >>>>> implemented in Rust, since we would have to rewrite it quite a bit
> >>>>> anyway. Especially since it's a part of the subsystem that has to deal
> >>>>> with memory management, object lifetime and asynchronousness quite a
> >>>>> lot and we had a history of issues there. So it could be interesting
> >>>>> to hear everyone's thoughts.
> >>>>
> >>>> I think it is far too soon to write a framework like that in Rust.
> >>>
> >>> Agreed. I don't object redesigns in C to make it better - which could have
> >>> some colateral effect of making things easier for a future Rust adoption,
> >>> but such changes should be justified by themselves, and not because of a
> >>> language change.
> >>
> >> No, the thought of redesign doesn't come from the language change,
> >> it's the other way around. Since rewriting a lot of the code already,
> >> why not do it in a language that is generally considered better.
> >
> > As Hans said, Rast has experimental support. We can't have drivers
> > depending on experimental stuff.
>
> Indeed.
>
> While discussing Rust for experimental drivers or codec libraries is
> interesting (and I am doing a Rust course, so hopefully I have a better
> understanding of what's involved by the upcoming media summit), using
> it for core media frameworks is simply a hard NACK until Linus blesses
> Rust as a second kernel language.
>
> So don't spend your valuable time on that.
>

Alright. I'm fine with C as well, although it's a shame that
eventually when Rust becomes a first-class citizen we'll be left with
a lot of legacy code base. Anyway, I guess let's wait until that
happens first. :)

> >
> >>
> >>>
> >>> See: redesigns at the core will potentially affect lots of drivers,
> >>> so it needs very good technical reasons why doing it. Plus, it requires
> >>> comprehensive tests with different types of hardware/drivers to reduce the
> >>> risk of regressions. Depending on the changes, it may require extra tests
> >>> with devices that are outside complex camera world: radio, analog and digital
> >>> TV drivers - and even some input devices that use VB2 - to ensure that
> >>> nothing broke.
> >>
> >> We don't have to do it in an all-or-nothing way. We can start with an
> >> experimental new implementation in Rust, which could be gradually
> >> tested. It could even be done the same way as the vb -> vb2
> >> transition, although I suspect it wouldn't really be necessary, as I
> >> would like to see it more like a drop-in replacement. In general I
> >> think the API exposed outside of the framework wouldn't really change
> >> that much, it's more about the internal design.
>
> It makes no sense to have a C and a Rust version of vb2. This framework
> is critical to all drivers, and we're not going to support two versions
> and fix bugs/add features in two places. Again, it's a hard NACK. Don't
> waste time on that.
>
> If there are ideas to make vb2 better, then I am all for that.
>
> I just want to mention two things here:
>
> For most drivers, using vb2 is just fine: the work a driver needs to do is
> quite straightforward. Exceptions are codec drivers and possibly complex
> camera drivers when they need to use requests (not certain yet).

Do we have any data that suggests that non-codec and non-complex
camera drivers are actually "most drivers"?

Anyway, I agree that "using" vb2 is indeed fine and I want us to keep
using it. But whether it actually works well is a different story.
Things become problematic as soon as someone intends to run something
more complex than yavta, e.g. exporting and importing DMA-bufs is
involved.

So putting aside the Rust discussion (that wasn't really the core
point), could I get 30 minutes to cover the vb2 pain points and how we
could fix them? Or should we just work on that and send patches?
Either works for me. (In fact we started already, e.g. via the
duplicate plane mapping patch series).

>
> Internally vb2 is quite complex, but that's because what it does is quite
> complex. And that's fine. If the internal structure can be improved to
> make it less complex, then I'm all for that, but there is no magic bullet
> (including using Rust instead of C) that suddenly makes everything simple.
>
> Generally I prefer to have the complexity in core frameworks, that will
> only make life easier for the driver developers.

I prefer complexity neither in drivers nor core frameworks, but we
can't have everything. ;)

I agree that buffer management is a complex problem, so we can't avoid
some level of complexity, although there is certainly room for
improvement in vb2. That also wasn't the core reason for the proposed
redesign. The core point is about the functional issues.

>
> To summarize:
>
> Until Rust is accepted by Linus as a second kernel language, as media
> maintainer I will NACK core media frameworks written in Rust. I won't
> spend time on it, it's an immediate NACK from me.
>
> Note that this doesn't imply that once Linus *does* accept Rust, that we
> are OK with core frameworks written in Rust. But that will be a separate
> discussion once that happens.

Ack.

Best regards,
Tomasz

>
> Regards,
>
>         Hans
>
> >
> > Having two implementations of the same logic doesn't sound reasonable,
> > as it doubles the maintainership effort: all changes done on one
> > implementation needs to be moved to the other one.
> >
> > Btw, we also have seem this problem before with VB and, up to some
> > sense, with VB2, as some drivers used to have their own buffer
> > handling implementation that usually started from a VB or VB2 fork.
> >
> > So, if VB2 has issues, let's fix it in C code.
> >
> >>>> To be
> >>>> honest, I won't even consider it until Linus officially accepts Rust as a
> >>>> second language in the kernel, instead of as an experiment.
> >>>
> >>> This is not enough: if the core starts to use a second language, all media
> >>> developers will be affected and will be required to have expertise on such
> >>> language.
> >>
> >> Let's be realistic, how many developers are actively touching vb2 these days?
> >
> > How many developers don't need VB2? Hopefully none :-)
> >
> >>> That's not something that should happen without careful
> >>> analysis and plans that should include a gradual roll-up, lost of tests
> >>> with the affected drivers including the legacy ones and some strategy to
> >>> quickly solve regression issues.
> >>
> >> That said, I agree. It needs proper discussion and planning. That's
> >> why I'm proposing this as a topic. :)
> >> Moreover the redesign itself also needs proper discussion and is more
> >> of a long term goal, not something to land in the next few days.
> >>
> >>>
> >>> It is not a matter of what language is better. Instead, it is a matter of
> >>> not affecting code maintenance during the (probably long) transition period
> >>> and beyond.
> >>>
> >>> If you see the past history, the transition from V4L to V4L2 took more than 10
> >>> years - being possible to be done only with the help of libv4l, plus a
> >>> lot of backward-compat code that we added. Still there were several
> >>> regressions and we even had to quickly patch the Kernel and/or some apps
> >>> that were using the uAPI on different ways.
> >>
> >> That's a different situation, because UAPI is involved.
> >
> > It is different, but similar, up to some sense, as a change at VB2
> > implementation will likely affect its kAPI, its behavior or both.
> >
> > The point I'm underlining is that core redesigns do affect existing
> > drivers usually on unexpected ways.
> >
> >>
> >>>
> >>> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> >>>
> >>
> >> Yes, vb -> vb2 would be a more appropriate comparison.
> >>
> >>> On both cases, there were very good technical reasons for the transition,
> >>> in terms of missing needed features, broken memory models and serious
> >>> troubles that utterly causing VB1 to not work well on non-x86 hardware.
> >>>
> >>
> >> It's a very similar situation now, vb2 doesn't work well on modern
> >> hardware, but I still have hopes that it can be fixed without
> >> affecting the driver-facing behavior. (We would probably need to
> >> develop some unit tests that validate the driver-facing behavior to
> >> ensure that.)
> >>
> >>> In the end, the authors of the core changes need to acquire legacy hardware
> >>> and to do lots of driver-specific changes to avoid breaking existing stuff.
> >>> Hans and I had to dedicate a lot of time and efforts on such transitions,
> >>> as it required a lot of work.
> >>>
> >>> I can tell you: there's no fun on such changes: typically, companies won't
> >>> pay someone to do changes on drivers for legacy hardware, specially
> >>> when there are no real benefits, which is the case here, as the final result
> >>> is just to keep the existing drivers to work with existing hardware,
> >>> usually without any new features. So, the ones behind such core changes
> >>> have to commit fixing drivers usually on their spare time.
> >>>
> >>
> >> I don't get that argument. Wouldn't the same apply to any core change?
> >
> > It depends of the type of change. For instance, an addition of a new
> > V4L2 control should not cause regressions to existing drivers. The
> > same would be true if one adds a new memory allocation component for
> > VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..):
> > only drivers using the new way would be affected.
> >
> >> I think the reason we have driver maintainers is that they can help
> >> with testing. Moreover, we need to invest into testing infrastructure
> >> (which is what people have been doing recently via Media CI) to make
> >> such changes less painful. Otherwise the subsystem will just bit-rot
> >> and become useful for modern use cases.
> >
> > Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover
> > actual drivers. For that, we would need to invest on a CI solution
> > integrated with lots of different hardware pieces, to check for actual
> > driver regressions.
> >
> > On one of my previous work, the company I used to work had that: they
> > had some monitors display some things, and the camera captured input
> > were compared to what the monitor were actually displaying. Doable, but
> > expensive.
> >
> > Regards,
> > Mauro
> >
> > Thanks,
> > Mauro
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

On 13/06/2024 10:14, Tomasz Figa wrote:
> On Thu, Jun 13, 2024 at 4:38 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 12/06/2024 22:35, Mauro Carvalho Chehab wrote:
>>> Em Wed, 12 Jun 2024 17:22:34 +0900
>>> Tomasz Figa <tfiga@chromium.org> escreveu:
>>>
>>>> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab
>>>> <mchehab@kernel.org> wrote:
>>>>>
>>>>> Em Wed, 12 Jun 2024 08:46:50 +0200
>>>>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>>>>>
>>>>>> On 6/12/24 06:12, Tomasz Figa wrote:
>>>>>>> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
>>>>>>> <daniel.almeida@collabora.com> wrote:
>>>>>>>>
>>>>>>>> Hi Hans, all,
>>>>>>>>
>>>>>>>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
>>>>>>>>
>>>>>>>> Please note that these are new submissions that are unrelated with what was discussed last year.
>>>>>>>>
>>>>>>>> 30 minutes will do.
>>>>>>>>
>>>>>>>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
>>>>>>>> [1] https://lwn.net/Articles/970565
>>>>>>>
>>>>>>> Somewhat related to the topic: I see potential for a quite big
>>>>>>> redesign of the videobuf2 framework going forward and recently with
>>>>>>> more Rust adoption I'm starting to think it could benefit from being
>>>>>>> implemented in Rust, since we would have to rewrite it quite a bit
>>>>>>> anyway. Especially since it's a part of the subsystem that has to deal
>>>>>>> with memory management, object lifetime and asynchronousness quite a
>>>>>>> lot and we had a history of issues there. So it could be interesting
>>>>>>> to hear everyone's thoughts.
>>>>>>
>>>>>> I think it is far too soon to write a framework like that in Rust.
>>>>>
>>>>> Agreed. I don't object redesigns in C to make it better - which could have
>>>>> some colateral effect of making things easier for a future Rust adoption,
>>>>> but such changes should be justified by themselves, and not because of a
>>>>> language change.
>>>>
>>>> No, the thought of redesign doesn't come from the language change,
>>>> it's the other way around. Since rewriting a lot of the code already,
>>>> why not do it in a language that is generally considered better.
>>>
>>> As Hans said, Rast has experimental support. We can't have drivers
>>> depending on experimental stuff.
>>
>> Indeed.
>>
>> While discussing Rust for experimental drivers or codec libraries is
>> interesting (and I am doing a Rust course, so hopefully I have a better
>> understanding of what's involved by the upcoming media summit), using
>> it for core media frameworks is simply a hard NACK until Linus blesses
>> Rust as a second kernel language.
>>
>> So don't spend your valuable time on that.
>>
> 
> Alright. I'm fine with C as well, although it's a shame that
> eventually when Rust becomes a first-class citizen we'll be left with
> a lot of legacy code base. Anyway, I guess let's wait until that
> happens first. :)
> 
>>>
>>>>
>>>>>
>>>>> See: redesigns at the core will potentially affect lots of drivers,
>>>>> so it needs very good technical reasons why doing it. Plus, it requires
>>>>> comprehensive tests with different types of hardware/drivers to reduce the
>>>>> risk of regressions. Depending on the changes, it may require extra tests
>>>>> with devices that are outside complex camera world: radio, analog and digital
>>>>> TV drivers - and even some input devices that use VB2 - to ensure that
>>>>> nothing broke.
>>>>
>>>> We don't have to do it in an all-or-nothing way. We can start with an
>>>> experimental new implementation in Rust, which could be gradually
>>>> tested. It could even be done the same way as the vb -> vb2
>>>> transition, although I suspect it wouldn't really be necessary, as I
>>>> would like to see it more like a drop-in replacement. In general I
>>>> think the API exposed outside of the framework wouldn't really change
>>>> that much, it's more about the internal design.
>>
>> It makes no sense to have a C and a Rust version of vb2. This framework
>> is critical to all drivers, and we're not going to support two versions
>> and fix bugs/add features in two places. Again, it's a hard NACK. Don't
>> waste time on that.
>>
>> If there are ideas to make vb2 better, then I am all for that.
>>
>> I just want to mention two things here:
>>
>> For most drivers, using vb2 is just fine: the work a driver needs to do is
>> quite straightforward. Exceptions are codec drivers and possibly complex
>> camera drivers when they need to use requests (not certain yet).
> 
> Do we have any data that suggests that non-codec and non-complex
> camera drivers are actually "most drivers"?

Anything with a simple video pipeline: webcams and PCI/USB video capture
cards in particular. All devices you probably never work with since they
are primary PC oriented and not for embedded systems/SoCs.

I have drawers full of them...

Codec drivers are a minority.

> 
> Anyway, I agree that "using" vb2 is indeed fine and I want us to keep
> using it. But whether it actually works well is a different story.
> Things become problematic as soon as someone intends to run something
> more complex than yavta, e.g. exporting and importing DMA-bufs is
> involved.
> 
> So putting aside the Rust discussion (that wasn't really the core
> point), could I get 30 minutes to cover the vb2 pain points and how we
> could fix them? Or should we just work on that and send patches?
> Either works for me. (In fact we started already, e.g. via the
> duplicate plane mapping patch series).

Sure! Can you make a new reply to this topic with a proposal? This is
hidden inside this longer thread, so I'd rather have a fresh post.

Regards,

	Hans

> 
>>
>> Internally vb2 is quite complex, but that's because what it does is quite
>> complex. And that's fine. If the internal structure can be improved to
>> make it less complex, then I'm all for that, but there is no magic bullet
>> (including using Rust instead of C) that suddenly makes everything simple.
>>
>> Generally I prefer to have the complexity in core frameworks, that will
>> only make life easier for the driver developers.
> 
> I prefer complexity neither in drivers nor core frameworks, but we
> can't have everything. ;)
> 
> I agree that buffer management is a complex problem, so we can't avoid
> some level of complexity, although there is certainly room for
> improvement in vb2. That also wasn't the core reason for the proposed
> redesign. The core point is about the functional issues.
> 
>>
>> To summarize:
>>
>> Until Rust is accepted by Linus as a second kernel language, as media
>> maintainer I will NACK core media frameworks written in Rust. I won't
>> spend time on it, it's an immediate NACK from me.
>>
>> Note that this doesn't imply that once Linus *does* accept Rust, that we
>> are OK with core frameworks written in Rust. But that will be a separate
>> discussion once that happens.
> 
> Ack.
> 
> Best regards,
> Tomasz
> 
>>
>> Regards,
>>
>>         Hans
>>
>>>
>>> Having two implementations of the same logic doesn't sound reasonable,
>>> as it doubles the maintainership effort: all changes done on one
>>> implementation needs to be moved to the other one.
>>>
>>> Btw, we also have seem this problem before with VB and, up to some
>>> sense, with VB2, as some drivers used to have their own buffer
>>> handling implementation that usually started from a VB or VB2 fork.
>>>
>>> So, if VB2 has issues, let's fix it in C code.
>>>
>>>>>> To be
>>>>>> honest, I won't even consider it until Linus officially accepts Rust as a
>>>>>> second language in the kernel, instead of as an experiment.
>>>>>
>>>>> This is not enough: if the core starts to use a second language, all media
>>>>> developers will be affected and will be required to have expertise on such
>>>>> language.
>>>>
>>>> Let's be realistic, how many developers are actively touching vb2 these days?
>>>
>>> How many developers don't need VB2? Hopefully none :-)
>>>
>>>>> That's not something that should happen without careful
>>>>> analysis and plans that should include a gradual roll-up, lost of tests
>>>>> with the affected drivers including the legacy ones and some strategy to
>>>>> quickly solve regression issues.
>>>>
>>>> That said, I agree. It needs proper discussion and planning. That's
>>>> why I'm proposing this as a topic. :)
>>>> Moreover the redesign itself also needs proper discussion and is more
>>>> of a long term goal, not something to land in the next few days.
>>>>
>>>>>
>>>>> It is not a matter of what language is better. Instead, it is a matter of
>>>>> not affecting code maintenance during the (probably long) transition period
>>>>> and beyond.
>>>>>
>>>>> If you see the past history, the transition from V4L to V4L2 took more than 10
>>>>> years - being possible to be done only with the help of libv4l, plus a
>>>>> lot of backward-compat code that we added. Still there were several
>>>>> regressions and we even had to quickly patch the Kernel and/or some apps
>>>>> that were using the uAPI on different ways.
>>>>
>>>> That's a different situation, because UAPI is involved.
>>>
>>> It is different, but similar, up to some sense, as a change at VB2
>>> implementation will likely affect its kAPI, its behavior or both.
>>>
>>> The point I'm underlining is that core redesigns do affect existing
>>> drivers usually on unexpected ways.
>>>
>>>>
>>>>>
>>>>> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
>>>>>
>>>>
>>>> Yes, vb -> vb2 would be a more appropriate comparison.
>>>>
>>>>> On both cases, there were very good technical reasons for the transition,
>>>>> in terms of missing needed features, broken memory models and serious
>>>>> troubles that utterly causing VB1 to not work well on non-x86 hardware.
>>>>>
>>>>
>>>> It's a very similar situation now, vb2 doesn't work well on modern
>>>> hardware, but I still have hopes that it can be fixed without
>>>> affecting the driver-facing behavior. (We would probably need to
>>>> develop some unit tests that validate the driver-facing behavior to
>>>> ensure that.)
>>>>
>>>>> In the end, the authors of the core changes need to acquire legacy hardware
>>>>> and to do lots of driver-specific changes to avoid breaking existing stuff.
>>>>> Hans and I had to dedicate a lot of time and efforts on such transitions,
>>>>> as it required a lot of work.
>>>>>
>>>>> I can tell you: there's no fun on such changes: typically, companies won't
>>>>> pay someone to do changes on drivers for legacy hardware, specially
>>>>> when there are no real benefits, which is the case here, as the final result
>>>>> is just to keep the existing drivers to work with existing hardware,
>>>>> usually without any new features. So, the ones behind such core changes
>>>>> have to commit fixing drivers usually on their spare time.
>>>>>
>>>>
>>>> I don't get that argument. Wouldn't the same apply to any core change?
>>>
>>> It depends of the type of change. For instance, an addition of a new
>>> V4L2 control should not cause regressions to existing drivers. The
>>> same would be true if one adds a new memory allocation component for
>>> VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..):
>>> only drivers using the new way would be affected.
>>>
>>>> I think the reason we have driver maintainers is that they can help
>>>> with testing. Moreover, we need to invest into testing infrastructure
>>>> (which is what people have been doing recently via Media CI) to make
>>>> such changes less painful. Otherwise the subsystem will just bit-rot
>>>> and become useful for modern use cases.
>>>
>>> Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover
>>> actual drivers. For that, we would need to invest on a CI solution
>>> integrated with lots of different hardware pieces, to check for actual
>>> driver regressions.
>>>
>>> On one of my previous work, the company I used to work had that: they
>>> had some monitors display some things, and the camera captured input
>>> were compared to what the monitor were actually displaying. Doable, but
>>> expensive.
>>>
>>> Regards,
>>> Mauro
>>>
>>> Thanks,
>>> Mauro
>>
> 

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

On Thu, Jun 13, 2024 at 05:14:45PM +0900, Tomasz Figa wrote:
> On Thu, Jun 13, 2024 at 4:38 PM Hans Verkuil wrote:
> > On 12/06/2024 22:35, Mauro Carvalho Chehab wrote:
> > > Em Wed, 12 Jun 2024 17:22:34 +0900 Tomasz Figa escreveu:
> > >> On Wed, Jun 12, 2024 at 4:54 PM Mauro Carvalho Chehab wrote:
> > >>> Em Wed, 12 Jun 2024 08:46:50 +0200 Hans Verkuil escreveu:
> > >>>> On 6/12/24 06:12, Tomasz Figa wrote:
> > >>>>> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> > >>>>>>
> > >>>>>> Hi Hans, all,
> > >>>>>>
> > >>>>>> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> > >>>>>>
> > >>>>>> Please note that these are new submissions that are unrelated with what was discussed last year.
> > >>>>>>
> > >>>>>> 30 minutes will do.
> > >>>>>>
> > >>>>>> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > >>>>>> [1] https://lwn.net/Articles/970565
> > >>>>>
> > >>>>> Somewhat related to the topic: I see potential for a quite big
> > >>>>> redesign of the videobuf2 framework going forward and recently with
> > >>>>> more Rust adoption I'm starting to think it could benefit from being
> > >>>>> implemented in Rust, since we would have to rewrite it quite a bit
> > >>>>> anyway. Especially since it's a part of the subsystem that has to deal
> > >>>>> with memory management, object lifetime and asynchronousness quite a
> > >>>>> lot and we had a history of issues there. So it could be interesting
> > >>>>> to hear everyone's thoughts.
> > >>>>
> > >>>> I think it is far too soon to write a framework like that in Rust.
> > >>>
> > >>> Agreed. I don't object redesigns in C to make it better - which could have
> > >>> some colateral effect of making things easier for a future Rust adoption,
> > >>> but such changes should be justified by themselves, and not because of a
> > >>> language change.
> > >>
> > >> No, the thought of redesign doesn't come from the language change,
> > >> it's the other way around. Since rewriting a lot of the code already,
> > >> why not do it in a language that is generally considered better.
> > >
> > > As Hans said, Rast has experimental support. We can't have drivers
> > > depending on experimental stuff.
> >
> > Indeed.
> >
> > While discussing Rust for experimental drivers or codec libraries is
> > interesting (and I am doing a Rust course, so hopefully I have a better
> > understanding of what's involved by the upcoming media summit), using
> > it for core media frameworks is simply a hard NACK until Linus blesses
> > Rust as a second kernel language.
> >
> > So don't spend your valuable time on that.
> 
> Alright. I'm fine with C as well, although it's a shame that
> eventually when Rust becomes a first-class citizen we'll be left with
> a lot of legacy code base. Anyway, I guess let's wait until that
> happens first. :)

You make it sound like any global acceptance of rust will automatically
make C a second class citizen. That may not help getting more people to
accept a second language :-)

> > >>> See: redesigns at the core will potentially affect lots of drivers,
> > >>> so it needs very good technical reasons why doing it. Plus, it requires
> > >>> comprehensive tests with different types of hardware/drivers to reduce the
> > >>> risk of regressions. Depending on the changes, it may require extra tests
> > >>> with devices that are outside complex camera world: radio, analog and digital
> > >>> TV drivers - and even some input devices that use VB2 - to ensure that
> > >>> nothing broke.
> > >>
> > >> We don't have to do it in an all-or-nothing way. We can start with an
> > >> experimental new implementation in Rust, which could be gradually
> > >> tested. It could even be done the same way as the vb -> vb2
> > >> transition, although I suspect it wouldn't really be necessary, as I
> > >> would like to see it more like a drop-in replacement. In general I
> > >> think the API exposed outside of the framework wouldn't really change
> > >> that much, it's more about the internal design.
> >
> > It makes no sense to have a C and a Rust version of vb2. This framework
> > is critical to all drivers, and we're not going to support two versions
> > and fix bugs/add features in two places. Again, it's a hard NACK. Don't
> > waste time on that.
> >
> > If there are ideas to make vb2 better, then I am all for that.
> >
> > I just want to mention two things here:
> >
> > For most drivers, using vb2 is just fine: the work a driver needs to do is
> > quite straightforward. Exceptions are codec drivers and possibly complex
> > camera drivers when they need to use requests (not certain yet).

For camera drivers, I plan to experiment with the request API at some
point, and I think the current way it's handled in helpers through the
subsystem, including in vb2, will not be a good match for the needs. As
far as I understand, when a request is queued, core code then dispatches
calls to vb2 and control operations corresponding to the request
contents. What we will likely need instead is using a top-level entry
point and getting data out of the request manually, not through
callbacks. I'll report more on this once I start experimenting.

> Do we have any data that suggests that non-codec and non-complex
> camera drivers are actually "most drivers"?
> 
> Anyway, I agree that "using" vb2 is indeed fine and I want us to keep
> using it. But whether it actually works well is a different story.
> Things become problematic as soon as someone intends to run something
> more complex than yavta, e.g. exporting and importing DMA-bufs is
> involved.
> 
> So putting aside the Rust discussion (that wasn't really the core
> point), could I get 30 minutes to cover the vb2 pain points and how we
> could fix them? Or should we just work on that and send patches?
> Either works for me. (In fact we started already, e.g. via the
> duplicate plane mapping patch series).
> 
> > Internally vb2 is quite complex, but that's because what it does is quite
> > complex. And that's fine. If the internal structure can be improved to
> > make it less complex, then I'm all for that, but there is no magic bullet
> > (including using Rust instead of C) that suddenly makes everything simple.
> >
> > Generally I prefer to have the complexity in core frameworks, that will
> > only make life easier for the driver developers.
> 
> I prefer complexity neither in drivers nor core frameworks, but we
> can't have everything. ;)
> 
> I agree that buffer management is a complex problem, so we can't avoid
> some level of complexity, although there is certainly room for
> improvement in vb2. That also wasn't the core reason for the proposed
> redesign. The core point is about the functional issues.
> 
> > To summarize:
> >
> > Until Rust is accepted by Linus as a second kernel language, as media
> > maintainer I will NACK core media frameworks written in Rust. I won't
> > spend time on it, it's an immediate NACK from me.
> >
> > Note that this doesn't imply that once Linus *does* accept Rust, that we
> > are OK with core frameworks written in Rust. But that will be a separate
> > discussion once that happens.
> 
> Ack.
> 
> > > Having two implementations of the same logic doesn't sound reasonable,
> > > as it doubles the maintainership effort: all changes done on one
> > > implementation needs to be moved to the other one.
> > >
> > > Btw, we also have seem this problem before with VB and, up to some
> > > sense, with VB2, as some drivers used to have their own buffer
> > > handling implementation that usually started from a VB or VB2 fork.
> > >
> > > So, if VB2 has issues, let's fix it in C code.
> > >
> > >>>> To be
> > >>>> honest, I won't even consider it until Linus officially accepts Rust as a
> > >>>> second language in the kernel, instead of as an experiment.
> > >>>
> > >>> This is not enough: if the core starts to use a second language, all media
> > >>> developers will be affected and will be required to have expertise on such
> > >>> language.
> > >>
> > >> Let's be realistic, how many developers are actively touching vb2 these days?
> > >
> > > How many developers don't need VB2? Hopefully none :-)
> > >
> > >>> That's not something that should happen without careful
> > >>> analysis and plans that should include a gradual roll-up, lost of tests
> > >>> with the affected drivers including the legacy ones and some strategy to
> > >>> quickly solve regression issues.
> > >>
> > >> That said, I agree. It needs proper discussion and planning. That's
> > >> why I'm proposing this as a topic. :)
> > >> Moreover the redesign itself also needs proper discussion and is more
> > >> of a long term goal, not something to land in the next few days.
> > >>
> > >>>
> > >>> It is not a matter of what language is better. Instead, it is a matter of
> > >>> not affecting code maintenance during the (probably long) transition period
> > >>> and beyond.
> > >>>
> > >>> If you see the past history, the transition from V4L to V4L2 took more than 10
> > >>> years - being possible to be done only with the help of libv4l, plus a
> > >>> lot of backward-compat code that we added. Still there were several
> > >>> regressions and we even had to quickly patch the Kernel and/or some apps
> > >>> that were using the uAPI on different ways.
> > >>
> > >> That's a different situation, because UAPI is involved.
> > >
> > > It is different, but similar, up to some sense, as a change at VB2
> > > implementation will likely affect its kAPI, its behavior or both.
> > >
> > > The point I'm underlining is that core redesigns do affect existing
> > > drivers usually on unexpected ways.
> > >
> > >>
> > >>>
> > >>> Yet, the transition from VB1 to VB2 was also painful, and took a lot of time.
> > >>>
> > >>
> > >> Yes, vb -> vb2 would be a more appropriate comparison.
> > >>
> > >>> On both cases, there were very good technical reasons for the transition,
> > >>> in terms of missing needed features, broken memory models and serious
> > >>> troubles that utterly causing VB1 to not work well on non-x86 hardware.
> > >>>
> > >>
> > >> It's a very similar situation now, vb2 doesn't work well on modern
> > >> hardware, but I still have hopes that it can be fixed without
> > >> affecting the driver-facing behavior. (We would probably need to
> > >> develop some unit tests that validate the driver-facing behavior to
> > >> ensure that.)
> > >>
> > >>> In the end, the authors of the core changes need to acquire legacy hardware
> > >>> and to do lots of driver-specific changes to avoid breaking existing stuff.
> > >>> Hans and I had to dedicate a lot of time and efforts on such transitions,
> > >>> as it required a lot of work.
> > >>>
> > >>> I can tell you: there's no fun on such changes: typically, companies won't
> > >>> pay someone to do changes on drivers for legacy hardware, specially
> > >>> when there are no real benefits, which is the case here, as the final result
> > >>> is just to keep the existing drivers to work with existing hardware,
> > >>> usually without any new features. So, the ones behind such core changes
> > >>> have to commit fixing drivers usually on their spare time.
> > >>>
> > >>
> > >> I don't get that argument. Wouldn't the same apply to any core change?
> > >
> > > It depends of the type of change. For instance, an addition of a new
> > > V4L2 control should not cause regressions to existing drivers. The
> > > same would be true if one adds a new memory allocation component for
> > > VB2 (e. g. something similar to videobuf2-vmalloc.c/videobuf2-dma-sg.c/..):
> > > only drivers using the new way would be affected.
> > >
> > >> I think the reason we have driver maintainers is that they can help
> > >> with testing. Moreover, we need to invest into testing infrastructure
> > >> (which is what people have been doing recently via Media CI) to make
> > >> such changes less painful. Otherwise the subsystem will just bit-rot
> > >> and become useful for modern use cases.
> > >
> > > Using CI to check for uAPI/kAPI changes is helpful, but it doesn't cover
> > > actual drivers. For that, we would need to invest on a CI solution
> > > integrated with lots of different hardware pieces, to check for actual
> > > driver regressions.
> > >
> > > On one of my previous work, the company I used to work had that: they
> > > had some monitors display some things, and the camera captured input
> > > were compared to what the monitor were actually displaying. Doable, but
> > > expensive.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

On Wed, Jun 12, 2024 at 3:47 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 6/12/24 06:12, Tomasz Figa wrote:
> > On Wed, May 15, 2024 at 1:19 AM Daniel Almeida
> > <daniel.almeida@collabora.com> wrote:
> >>
> >> Hi Hans, all,
> >>
> >> I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> >>
> >> Please note that these are new submissions that are unrelated with what was discussed last year.
> >>
> >> 30 minutes will do.
> >>
> >> [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> >> [1] https://lwn.net/Articles/970565
> >
> > Somewhat related to the topic: I see potential for a quite big
> > redesign of the videobuf2 framework going forward and recently with
> > more Rust adoption I'm starting to think it could benefit from being
> > implemented in Rust, since we would have to rewrite it quite a bit
> > anyway. Especially since it's a part of the subsystem that has to deal
> > with memory management, object lifetime and asynchronousness quite a
> > lot and we had a history of issues there. So it could be interesting
> > to hear everyone's thoughts.
>
> I think it is far too soon to write a framework like that in Rust. To be
> honest, I won't even consider it until Linus officially accepts Rust as a
> second language in the kernel, instead of as an experiment.
>

Given the speed of development and adoption in drivers, I would
consider this just a matter of time. And the redesign that I mentioned
would likely need quite some time to prepare as well.

In addition, the more secure, redesigned version could coexist with
the C version (as an experiment, same kind as Rust support today), as
I would expect that the driver API would likely stay the same. The
redesign would be mostly about how things are handled internally in
the framework.

> The vb2 framework can certainly use some more work, and esp. better support
> for codecs, since that's where the main pain is at the moment.
>
> But I would need to see a proper proposal first. I assume that's what you
> plan to present?
>

Actually, if I could get a 30 minutes slot, I could indeed present a
list of problems that need to be addressed in vb2 and some proposals
on how to do that. (And why it may require going as far as a
redesign.)

> > That said, I wouldn't be able to travel this time unfortunately, so it
> > would be nice if we could arrange this topic in a time slot friendly
> > for remote attendance from Japan. Also +Hidenori Kobayashi from my
> > team who would also be interested in joining remotely.
>
> That would mean a slot in the morning, right? Since Japan is 7 hours ahead
> of CEST.

Correct. I can probably stay online until around 8pm, which should be
1pm in Vienna. (So I guess until lunchtime?)

>
> Regards,
>
>         Hans
>
> >
> > Best,
> > Tomasz
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Laurent Pinchart]

Hi Tomasz,

On Wed, Jun 12, 2024 at 01:12:29PM +0900, Tomasz Figa wrote:
> On Wed, May 15, 2024 at 1:19 AM Daniel Almeida wrote:
> >
> > Hi Hans, all,
> >
> > I’d like to attend in person and discuss the use of Rust in the subsystem, especially in light of [0] and [1].
> >
> > Please note that these are new submissions that are unrelated with what was discussed last year.
> >
> > 30 minutes will do.
> >
> > [0] https://lwn.net/ml/linux-media/20240227215146.46487-1-daniel.almeida@collabora.com/
> > [1] https://lwn.net/Articles/970565
> 
> Somewhat related to the topic: I see potential for a quite big
> redesign of the videobuf2 framework going forward and recently with
> more Rust adoption I'm starting to think it could benefit from being
> implemented in Rust, since we would have to rewrite it quite a bit
> anyway. Especially since it's a part of the subsystem that has to deal
> with memory management, object lifetime and asynchronousness quite a
> lot and we had a history of issues there. So it could be interesting
> to hear everyone's thoughts.

More than the language choice, what picked my interest here is why you
htink vb2 should be rewritten. Do you have a high-level overview
document that explains the rationale and what you think the new vb2
should look like ?

> That said, I wouldn't be able to travel this time unfortunately, so it
> would be nice if we could arrange this topic in a time slot friendly
> for remote attendance from Japan. Also +Hidenori Kobayashi from my
> team who would also be interested in joining remotely.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

Hi all,

On 5/6/24 13:33, Hans Verkuil wrote:
> Hi all,
> 
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.
> 
> Location:
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> 
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
> 
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.
> 
> If you want to attend the meeting (either in person or remote), then send an email to me
> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> what to expect.
> 
> Priority goes to presenters and the core media maintainers. If multiple people of the same
> company want to attend, then I may ask to limit attendance to one or two people.
> 
> It is hard to predict how many people want to attend, so I'll see how it goes.

Just a quick update:

After just 9 days I already have 5 people who want to present something.
And currently I have 10 people who want to attend in person and one remotely.

So I am confident that we should be able to fill that room :-)

Regards,

	Hans

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Steve Cho]

Hi Hans,

If allowed, I would like to talk about how V4L2 virtual video decode
driver is used to enhance testing on Chromium.
I expect it to be 30mins.

Steve

On Wed, May 15, 2024 at 2:32 AM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> Hi all,
>
> On 5/6/24 13:33, Hans Verkuil wrote:
> > Hi all,
> >
> > We will organize another Media Summit on Monday September 16th to coincide with
> > the Open Source Summit Europe in Vienna:
> >
> > https://events.linuxfoundation.org/open-source-summit-europe/
> >
> > Avnet Silica has very kindly offered to host this summit at their Vienna
> > office, which is about 35 minutes by public transport from the OSSE venue.
> >
> > Location:
> >
> > https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> >
> > The meeting room can hold 18 people and has video conferencing support (MS Teams).
> >
> > That said, I want to keep remote participation to a minimum. This yearly summit is meant
> > for active media developers to meet up face-to-face and to discuss media subsystem issues.
> > But if you are an active media developer, but are not able to attend in person, then this
> > is an option.
> >
> > If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> > It would be very much appreciated if you can also add a guesstimate of the time
> > you need for your topic.
> >
> > If you want to attend the meeting (either in person or remote), then send an email to me
> > directly. Since the number of seats is limited, I may have to put people on a waiting list.
> > Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> > what to expect.
> >
> > Priority goes to presenters and the core media maintainers. If multiple people of the same
> > company want to attend, then I may ask to limit attendance to one or two people.
> >
> > It is hard to predict how many people want to attend, so I'll see how it goes.
>
> Just a quick update:
>
> After just 9 days I already have 5 people who want to present something.
> And currently I have 10 people who want to attend in person and one remotely.
>
> So I am confident that we should be able to fill that room :-)
>
> Regards,
>
>         Hans
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Hans Verkuil]

Hi all,

I just wanted to give an update of the current status of the Media Summit on Monday
September 16th at the Avnet Silica office.

At this moment I have 7 attendees who want to present something, so that should
fill the day. There is probably time for one or two more topics, so if you have
something you want to present, just reply to this email with a proposal.

I will finalize the schedule and attendance list by mid-July.

There are currently 11 in-person attendees (so there is still room) and 4
remote participants.

Regards,

	Hans


On 06/05/2024 13:33, Hans Verkuil wrote:
> Hi all,
> 
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.
> 
> Location:
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
> 
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
> 
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.
> 
> If you want to attend the meeting (either in person or remote), then send an email to me
> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> what to expect.
> 
> Priority goes to presenters and the core media maintainers. If multiple people of the same
> company want to attend, then I may ask to limit attendance to one or two people.
> 
> It is hard to predict how many people want to attend, so I'll see how it goes.
> 
> Regards,
> 
> 	Hans
> 

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Tomasz Figa]

Hi Hans and everyone,

[re-sending in plain text...]

On Mon, May 6, 2024 at 8:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> Hi all,
>
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
>
> https://events.linuxfoundation.org/open-source-summit-europe/
>
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.
>
> Location:
>
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
>
> The meeting room can hold 18 people and has video conferencing support (MS Teams).
>
> That said, I want to keep remote participation to a minimum. This yearly summit is meant
> for active media developers to meet up face-to-face and to discuss media subsystem issues.
> But if you are an active media developer, but are not able to attend in person, then this
> is an option.
>
> If you have a topic that you want to discuss, just 'Reply All' to this announcement.
> It would be very much appreciated if you can also add a guesstimate of the time
> you need for your topic.

I'd like to propose the following topic:

Current state of videobuf2, its limitations and the paths forward.

In the session I'd go over the issues with the current design and
implementation of vb2 during my experience as a maintainer and a
driver developer. I have a couple of ideas on how to address those and
would also like to share those and get some initial high level
feedback on the direction.

I think I need 30 minutes, but unfortunately I can't travel at the
time if the conference and need to join remotely.

Best,
Tomasz

>
> If you want to attend the meeting (either in person or remote), then send an email to me
> directly. Since the number of seats is limited, I may have to put people on a waiting list.
> Please let me know sooner rather than later (ideally by mid-July) so I have a good idea
> what to expect.
>
> Priority goes to presenters and the core media maintainers. If multiple people of the same
> company want to attend, then I may ask to limit attendance to one or two people.
>
> It is hard to predict how many people want to attend, so I'll see how it goes.
>
> Regards,
>
>         Hans
>

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Sakari Ailus]

Hi Hans,

On Mon, May 06, 2024 at 01:33:32PM +0200, Hans Verkuil wrote:
> We will organize another Media Summit on Monday September 16th to coincide with
> the Open Source Summit Europe in Vienna:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/
> 
> Avnet Silica has very kindly offered to host this summit at their Vienna
> office, which is about 35 minutes by public transport from the OSSE venue.

I'd be happy to join, either in person (likely) or remotely otherwise. I'll
let you know once I have more information.

-- 
Kind regards,

Sakari Ailus

Re: [ANN] Request for Topics and registration for a Media Summit September 16th

[Sakari Ailus]

Hi Hans,

On Fri, Jun 28, 2024 at 11:59:42AM +0000, Sakari Ailus wrote:
> I'd be happy to join, either in person (likely) or remotely otherwise. I'll
> let you know once I have more information.

I'll be attending in person.

-- 
Sakari Ailus