[ANN] Media Summit September 16th: Draft Agenda (v3)

[ANN] Media Summit September 16th: Draft Agenda (v3)

[Hans Verkuil]

(Apologies for posting a v3 right after v2, I forgot to add Suresh to the attendee list, that's corrected in v3)

Hi all,

Here is my third stab at an agenda for the media summit. As always, it
is subject to change and all times are guesstimates!

The media summit will be held on Monday September 16th. Avnet Silica has very
kindly offered to host this summit at their Vienna office, which is about 35
minutes by public transport from the Open Source Summit Europe venue
(https://events.linuxfoundation.org/open-source-summit-europe/OSSE).

Avnet Silica Office Location:

Schönbrunner Str. 297/307, 1120 Vienna, Austria

https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu

Refreshments are available during the day.

The meeting room is sponsored by Avnet Silica. Much appreciated!

Regarding the face mask policy: we will follow the same guidance that the
Linux Foundation gives for the EOSS conference:

https://events.linuxfoundation.org/open-source-summit-europe/attend/health-and-safety/#onsite-health-and-safety


In-Person Attendees:

Sakari Ailus <sakari.ailus@linux.intel.com> (Intel)
Daniel Almeida <daniel.almeida@collabora.com> (Collabora)
Mauro Carvalho Chehab <mchehab@kernel.org> (Media Kernel Maintainer)
Sebastian Fricke <sebastian.fricke@collabora.com> (Collabora)
Martin Hecht <martin.hecht@avnet.eu> (Avnet)
Hu, Jerry W <jerry.w.hu@intel.com> (Intel)
Tommaso Merciai <tomm.merciai@gmail.com> (Avnet)
Jacopo Mondi <jacopo.mondi@ideasonboard.com> (Ideas On Board)
Benjamin Mugnier <benjamin.mugnier@foss.st.com> (ST Electronics)
Laurent Pinchart <laurent.pinchart@ideasonboard.com> (Ideas On Board)
Ricardo Ribalda <ribalda@chromium.org> (Google)
Michael Tretter <m.tretter@pengutronix.de> (Pengutronix)
Suresh Vankadara <svankada@qti.qualcomm.com> (Qualcomm)
Hans Verkuil <hverkuil-cisco@xs4all.nl> (Cisco Systems Norway)
Alain Volmat <alain.volmat@foss.st.com> (ST Electronics) (TBC)
Sean Young <sean@mess.org>
Jerry W Hu <jerry.w.hu@intel.com> (Intel)

Remote Attendees (using MS Teams):

Steve Cho <stevecho@chromium.org> (Google)
Nas Chung <nas.chung@chipsnmedia.com> (Chips & Media)
Tomasz Figa <tfiga@chromium.org> (Google)
Hidenori Kobayashi <hidenorik@chromium.org> (Google)
Jai Luthra <j-luthra@ti.com> (TI)

Note: information on how to connect remotely will come later.

If any information above is incorrect, or if I missed someone, then please let me know.

We are currently 16 confirmed in-person participants and one TBC. The maximum is 18 people,
so we're almost full. If you want to join in-person, then contact me and I'll put you on a
waitlist. The attendee list should be finalized by the end of August.

Draft agenda:

8:45-9:15: get settled :-)

9:15-9:25: Hans: Quick introduction

9:25-10:00: Steve Cho:

- V4L2 testing on Chromium using virtual video decode driver (VISL)
- V4L2 video decoding testing with KernelCI

10:00-11:00: Ricardo: multi-committer model using gitlab

11:00-11:15: break

11:15-12:15: Jacopo: Multi-context support in V4L2

12:15-13:30: Lunch

13:30-14:00: Tomasz: Current state of videobuf2, its limitations and the paths forward.

14:00-14:45: Laurent: subdevs, state, and usage of the media controller device to submit requests.

14:45-15:00: break

15:00-15:30: Sean: new tooling for infrared:

- What it is and what it can do (love to hear any feedback of course)
- Where it should be hosted? (I hope gitlab fdo, who do I ask)
- What needs to be in place for a release?
- This tool replaces ir-ctl and ir-keytable. How we phase them out?

15:30-16:00: Daniel: Rust in the media subsystem

16:00-16:15: break

16:15-16:30: Hans: UVC maintenance

16:30-18:00: TBD

Please reply with corrections, questions, etc. to this email. I'll update the agenda
over time. Again, these times are very preliminary.

Regards,

	Hans

Re: [ANN] Media Summit September 16th: Draft Agenda (v3)

[Laurent Pinchart]

Hello,

On Tue, Aug 13, 2024 at 10:17:59AM +0200, Hans Verkuil wrote:
> (Apologies for posting a v3 right after v2, I forgot to add Suresh to the attendee list, that's corrected in v3)
> 
> Hi all,
> 
> Here is my third stab at an agenda for the media summit. As always, it
> is subject to change and all times are guesstimates!
> 
> The media summit will be held on Monday September 16th. Avnet Silica has very
> kindly offered to host this summit at their Vienna office, which is about 35
> minutes by public transport from the Open Source Summit Europe venue
> (https://events.linuxfoundation.org/open-source-summit-europe/OSSE).
> 
> Avnet Silica Office Location:
> 
> Schönbrunner Str. 297/307, 1120 Vienna, Austria
> 
> https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> 
> Refreshments are available during the day.
> 
> The meeting room is sponsored by Avnet Silica. Much appreciated!
> 
> Regarding the face mask policy: we will follow the same guidance that the
> Linux Foundation gives for the EOSS conference:
> 
> https://events.linuxfoundation.org/open-source-summit-europe/attend/health-and-safety/#onsite-health-and-safety
> 
> 
> In-Person Attendees:
> 
> Sakari Ailus <sakari.ailus@linux.intel.com> (Intel)
> Daniel Almeida <daniel.almeida@collabora.com> (Collabora)
> Mauro Carvalho Chehab <mchehab@kernel.org> (Media Kernel Maintainer)
> Sebastian Fricke <sebastian.fricke@collabora.com> (Collabora)
> Martin Hecht <martin.hecht@avnet.eu> (Avnet)
> Hu, Jerry W <jerry.w.hu@intel.com> (Intel)
> Tommaso Merciai <tomm.merciai@gmail.com> (Avnet)
> Jacopo Mondi <jacopo.mondi@ideasonboard.com> (Ideas On Board)
> Benjamin Mugnier <benjamin.mugnier@foss.st.com> (ST Electronics)
> Laurent Pinchart <laurent.pinchart@ideasonboard.com> (Ideas On Board)
> Ricardo Ribalda <ribalda@chromium.org> (Google)
> Michael Tretter <m.tretter@pengutronix.de> (Pengutronix)
> Suresh Vankadara <svankada@qti.qualcomm.com> (Qualcomm)
> Hans Verkuil <hverkuil-cisco@xs4all.nl> (Cisco Systems Norway)
> Alain Volmat <alain.volmat@foss.st.com> (ST Electronics) (TBC)
> Sean Young <sean@mess.org>
> Jerry W Hu <jerry.w.hu@intel.com> (Intel)
> 
> Remote Attendees (using MS Teams):
> 
> Steve Cho <stevecho@chromium.org> (Google)
> Nas Chung <nas.chung@chipsnmedia.com> (Chips & Media)
> Tomasz Figa <tfiga@chromium.org> (Google)
> Hidenori Kobayashi <hidenorik@chromium.org> (Google)
> Jai Luthra <j-luthra@ti.com> (TI)
> 
> Note: information on how to connect remotely will come later.
> 
> If any information above is incorrect, or if I missed someone, then please let me know.
> 
> We are currently 16 confirmed in-person participants and one TBC. The maximum is 18 people,
> so we're almost full. If you want to join in-person, then contact me and I'll put you on a
> waitlist. The attendee list should be finalized by the end of August.
> 
> Draft agenda:
> 
> 8:45-9:15: get settled :-)
> 
> 9:15-9:25: Hans: Quick introduction
> 
> 9:25-10:00: Steve Cho:
> 
> - V4L2 testing on Chromium using virtual video decode driver (VISL)
> - V4L2 video decoding testing with KernelCI
> 
> 10:00-11:00: Ricardo: multi-committer model using gitlab
> 
> 11:00-11:15: break
> 
> 11:15-12:15: Jacopo: Multi-context support in V4L2
> 
> 12:15-13:30: Lunch
> 
> 13:30-14:00: Tomasz: Current state of videobuf2, its limitations and the paths forward.
> 
> 14:00-14:45: Laurent: subdevs, state, and usage of the media controller device to submit requests.
> 
> 14:45-15:00: break
> 
> 15:00-15:30: Sean: new tooling for infrared:
> 
> - What it is and what it can do (love to hear any feedback of course)
> - Where it should be hosted? (I hope gitlab fdo, who do I ask)
> - What needs to be in place for a release?
> - This tool replaces ir-ctl and ir-keytable. How we phase them out?
> 
> 15:30-16:00: Daniel: Rust in the media subsystem
> 
> 16:00-16:15: break
> 
> 16:15-16:30: Hans: UVC maintenance
> 
> 16:30-18:00: TBD

Here's a candidate topic for this time slot:

Should media drivers depend on CONFIG_PM?

Supporting both CONFIG_PM and !CONFIG_PM in drivers requires cumbersome
constructs, most likely leading to bugs because !CONFIG_PM is hardly
ever tested. The issue can be at least partly attributed to deficiencies
in the runtime PM and driver core APIs that should make this task easier
for drivers, but that will not realistically change any time soon.

In !CONFIG_PM kernels, drivers using runtime PM power up the device at
probe time, and keep it powered until remove time. The increased power
consumption really makes !CONFIG_PM a niche use case, if a use case at
all.

For those reasons, I would like to propose depending on CONFIG_PM for
media drivers. In an ideal world this could be done for the whole
subsystem. However, some architectures don't support CONFIG_PM, namely

- alpha
- csky
- hexagon
- m68k
- microblaze
- nios2
- openrisc
- parisc
- s390
- sparc (32-bit version only, sparc64 supports CONFIG_PM)

I assume we would get complains of the media subsystem became unusable
on those architectures. The decision could be made per driver, or per
category of drivers. I'm in particular interested in avoiding the churn
of supporting !CONFIG_PM in camera sensor drivers, and in platform
drivers that are used only on platforms that support CONFIG_PM.

I'm aware that asking this question may open the door to a more annoying
one. If we decide that we need to keep supporting those platforms in
camera sensor drivers, and that keeping the camera sensor powered up
unconditionally is not good enough, then we will have to reconsider the
move to runtime PM for sensor drivers that we started years ago (and
haven't completed yet).

> Please reply with corrections, questions, etc. to this email. I'll update the agenda
> over time. Again, these times are very preliminary.

-- 
Regards,

Laurent Pinchart

Re: [ANN] Media Summit September 16th: Draft Agenda (v3)

[Mauro Carvalho Chehab]

Em Mon, 26 Aug 2024 01:24:55 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> Hello,
> 
> On Tue, Aug 13, 2024 at 10:17:59AM +0200, Hans Verkuil wrote:
> > (Apologies for posting a v3 right after v2, I forgot to add Suresh to the attendee list, that's corrected in v3)
> > 
> > Hi all,
> > 
> > Here is my third stab at an agenda for the media summit. As always, it
> > is subject to change and all times are guesstimates!
> > 
> > The media summit will be held on Monday September 16th. Avnet Silica has very
> > kindly offered to host this summit at their Vienna office, which is about 35
> > minutes by public transport from the Open Source Summit Europe venue
> > (https://events.linuxfoundation.org/open-source-summit-europe/OSSE).
> > 
> > Avnet Silica Office Location:
> > 
> > Schönbrunner Str. 297/307, 1120 Vienna, Austria
> > 
> > https://www.google.com/maps/place/Avnet+EMG+Elektronische+Bauteile+GmbH+(Silica)/@48.183203,16.3100937,15z/data=!4m6!3m5!1s0x476da80e20b26d5b:0x2c5d2a77bbd43334!8m2!3d48.1832035!4d16.320372!16s%2Fg%2F1tcy32vt?entry=ttu
> > 
> > Refreshments are available during the day.
> > 
> > The meeting room is sponsored by Avnet Silica. Much appreciated!
> > 
> > Regarding the face mask policy: we will follow the same guidance that the
> > Linux Foundation gives for the EOSS conference:
> > 
> > https://events.linuxfoundation.org/open-source-summit-europe/attend/health-and-safety/#onsite-health-and-safety
> > 
> > 
> > In-Person Attendees:
> > 
> > Sakari Ailus <sakari.ailus@linux.intel.com> (Intel)
> > Daniel Almeida <daniel.almeida@collabora.com> (Collabora)
> > Mauro Carvalho Chehab <mchehab@kernel.org> (Media Kernel Maintainer)
> > Sebastian Fricke <sebastian.fricke@collabora.com> (Collabora)
> > Martin Hecht <martin.hecht@avnet.eu> (Avnet)
> > Hu, Jerry W <jerry.w.hu@intel.com> (Intel)
> > Tommaso Merciai <tomm.merciai@gmail.com> (Avnet)
> > Jacopo Mondi <jacopo.mondi@ideasonboard.com> (Ideas On Board)
> > Benjamin Mugnier <benjamin.mugnier@foss.st.com> (ST Electronics)
> > Laurent Pinchart <laurent.pinchart@ideasonboard.com> (Ideas On Board)
> > Ricardo Ribalda <ribalda@chromium.org> (Google)
> > Michael Tretter <m.tretter@pengutronix.de> (Pengutronix)
> > Suresh Vankadara <svankada@qti.qualcomm.com> (Qualcomm)
> > Hans Verkuil <hverkuil-cisco@xs4all.nl> (Cisco Systems Norway)
> > Alain Volmat <alain.volmat@foss.st.com> (ST Electronics) (TBC)
> > Sean Young <sean@mess.org>
> > Jerry W Hu <jerry.w.hu@intel.com> (Intel)
> > 
> > Remote Attendees (using MS Teams):
> > 
> > Steve Cho <stevecho@chromium.org> (Google)
> > Nas Chung <nas.chung@chipsnmedia.com> (Chips & Media)
> > Tomasz Figa <tfiga@chromium.org> (Google)
> > Hidenori Kobayashi <hidenorik@chromium.org> (Google)
> > Jai Luthra <j-luthra@ti.com> (TI)
> > 
> > Note: information on how to connect remotely will come later.
> > 
> > If any information above is incorrect, or if I missed someone, then please let me know.
> > 
> > We are currently 16 confirmed in-person participants and one TBC. The maximum is 18 people,
> > so we're almost full. If you want to join in-person, then contact me and I'll put you on a
> > waitlist. The attendee list should be finalized by the end of August.
> > 
> > Draft agenda:
> > 
> > 8:45-9:15: get settled :-)
> > 
> > 9:15-9:25: Hans: Quick introduction
> > 
> > 9:25-10:00: Steve Cho:
> > 
> > - V4L2 testing on Chromium using virtual video decode driver (VISL)
> > - V4L2 video decoding testing with KernelCI
> > 
> > 10:00-11:00: Ricardo: multi-committer model using gitlab
> > 
> > 11:00-11:15: break
> > 
> > 11:15-12:15: Jacopo: Multi-context support in V4L2
> > 
> > 12:15-13:30: Lunch
> > 
> > 13:30-14:00: Tomasz: Current state of videobuf2, its limitations and the paths forward.
> > 
> > 14:00-14:45: Laurent: subdevs, state, and usage of the media controller device to submit requests.
> > 
> > 14:45-15:00: break
> > 
> > 15:00-15:30: Sean: new tooling for infrared:
> > 
> > - What it is and what it can do (love to hear any feedback of course)
> > - Where it should be hosted? (I hope gitlab fdo, who do I ask)
> > - What needs to be in place for a release?
> > - This tool replaces ir-ctl and ir-keytable. How we phase them out?
> > 
> > 15:30-16:00: Daniel: Rust in the media subsystem
> > 
> > 16:00-16:15: break
> > 
> > 16:15-16:30: Hans: UVC maintenance
> > 
> > 16:30-18:00: TBD  
> 
> Here's a candidate topic for this time slot:
> 
> Should media drivers depend on CONFIG_PM?
> 
> Supporting both CONFIG_PM and !CONFIG_PM in drivers requires cumbersome
> constructs, most likely leading to bugs because !CONFIG_PM is hardly
> ever tested. The issue can be at least partly attributed to deficiencies
> in the runtime PM and driver core APIs that should make this task easier
> for drivers, but that will not realistically change any time soon.
> 
> In !CONFIG_PM kernels, drivers using runtime PM power up the device at
> probe time, and keep it powered until remove time. The increased power
> consumption really makes !CONFIG_PM a niche use case, if a use case at
> all.
> 
> For those reasons, I would like to propose depending on CONFIG_PM for
> media drivers. In an ideal world this could be done for the whole
> subsystem. 

This is not an option. Not all drivers depend on it. Some of them will
never need it - like most USB and PCI tv/capture cards and digital tv ones.

> However, some architectures don't support CONFIG_PM, namely
> 
> - alpha
> - csky
> - hexagon
> - m68k
> - microblaze
> - nios2
> - openrisc
> - parisc
> - s390
> - sparc (32-bit version only, sparc64 supports CONFIG_PM)

Well, arch-dependent drivers, like the ones made to run with ARM SoC
could depend on PM, but then there are sensor drivers and such which
are somewhat independent.
> 
> I assume we would get complains of the media subsystem became unusable
> on those architectures. The decision could be made per driver, or per
> category of drivers. I'm in particular interested in avoiding the churn
> of supporting !CONFIG_PM in camera sensor drivers, and in platform
> drivers that are used only on platforms that support CONFIG_PM.

There are a couple of sensor drivers that are used by em28xx, which, as far
as I remember, doesn't use PM. So, even for sensors it could be problematic.

> 
> I'm aware that asking this question may open the door to a more annoying
> one. If we decide that we need to keep supporting those platforms in
> camera sensor drivers, and that keeping the camera sensor powered up
> unconditionally is not good enough, then we will have to reconsider the
> move to runtime PM for sensor drivers that we started years ago (and
> haven't completed yet).

Having PM support for sensor drivers makes sense, provided that it won't
break support of the existing drivers using them.
> 
> > Please reply with corrections, questions, etc. to this email. I'll update the agenda
> > over time. Again, these times are very preliminary.  
> 

Re: [ANN] Media Summit September 16th: Draft Agenda (v3)

[Laurent Pinchart]

Hi Mauro,

On Mon, Aug 26, 2024 at 04:03:14AM +0200, Mauro Carvalho Chehab wrote:
> Em Mon, 26 Aug 2024 01:24:55 +0300 Laurent Pinchart escreveu:
> > On Tue, Aug 13, 2024 at 10:17:59AM +0200, Hans Verkuil wrote:

[snip]

> > > 16:30-18:00: TBD  
> > 
> > Here's a candidate topic for this time slot:
> > 
> > Should media drivers depend on CONFIG_PM?
> > 
> > Supporting both CONFIG_PM and !CONFIG_PM in drivers requires cumbersome
> > constructs, most likely leading to bugs because !CONFIG_PM is hardly
> > ever tested. The issue can be at least partly attributed to deficiencies
> > in the runtime PM and driver core APIs that should make this task easier
> > for drivers, but that will not realistically change any time soon.
> > 
> > In !CONFIG_PM kernels, drivers using runtime PM power up the device at
> > probe time, and keep it powered until remove time. The increased power
> > consumption really makes !CONFIG_PM a niche use case, if a use case at
> > all.
> > 
> > For those reasons, I would like to propose depending on CONFIG_PM for
> > media drivers. In an ideal world this could be done for the whole
> > subsystem. 
> 
> This is not an option. Not all drivers depend on it. Some of them will
> never need it - like most USB and PCI tv/capture cards and digital tv ones.

That by itself, I think, isn't a problem. We could technical depend on
CONFIG_PM even if not all drivers make active use of it. Whether or not
we want to is the question I hope this discussion will answer (and I
have an increasingly strong feeling the answer will be negative, but
let's see).

> > However, some architectures don't support CONFIG_PM, namely
> > 
> > - alpha
> > - csky
> > - hexagon
> > - m68k
> > - microblaze
> > - nios2
> > - openrisc
> > - parisc
> > - s390
> > - sparc (32-bit version only, sparc64 supports CONFIG_PM)
> 
> Well, arch-dependent drivers, like the ones made to run with ARM SoC
> could depend on PM, but then there are sensor drivers and such which
> are somewhat independent.
> 
> > I assume we would get complains of the media subsystem became unusable
> > on those architectures. The decision could be made per driver, or per
> > category of drivers. I'm in particular interested in avoiding the churn
> > of supporting !CONFIG_PM in camera sensor drivers, and in platform
> > drivers that are used only on platforms that support CONFIG_PM.
> 
> There are a couple of sensor drivers that are used by em28xx, which, as far
> as I remember, doesn't use PM. So, even for sensors it could be problematic.

I've checked the em28xx driver. Unless I'm mistaken, the only driver it
uses from drivers/media/i2c/ is the ov2640. Still, that's more than
zero, so this would certainly need to be taken into account.

In the em28xx case, power management of the camera sensor is handled by
the em28xx driver, outside of the camera sensor driver. Switching to
usage of runtime PM in the ov2640 driver and keeping the sensor "powered
up" from probe() onwards wouldn't be an issue, as the power on would be
a no-op (the ov2640 driver will have no regulator, clock or GPIO to
control in this case). Depending on CONFIG_PM, however, could be an
issue, even if I doubt anyone would use an em28xx-based device on a
platform that doesn't enable CONFIG_PM.

> > I'm aware that asking this question may open the door to a more annoying
> > one. If we decide that we need to keep supporting those platforms in
> > camera sensor drivers, and that keeping the camera sensor powered up
> > unconditionally is not good enough, then we will have to reconsider the
> > move to runtime PM for sensor drivers that we started years ago (and
> > haven't completed yet).
> 
> Having PM support for sensor drivers makes sense, provided that it won't
> break support of the existing drivers using them.
>
> > > Please reply with corrections, questions, etc. to this email. I'll update the agenda
> > > over time. Again, these times are very preliminary.  

-- 
Regards,

Laurent Pinchart