[RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
for devices that are source of an external flash strobe for flash
devices. This part seems to be missing in current Flash control
class, i.e. a means for configuring devices that are not camera
flash drivers but involve a flash related functionality.

The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
to determine the flash control behavior, for instance, at an image
sensor device.

The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
a flash subdev is present in the system.

Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
---

Hi Sakari,

My apologies for not bringing this earlier when you were designing
the Flash control API.
It seems like a use case were a sensor controller drives a strobe
signal for a Flash and the sensor side requires some set up doesn't
quite fit in the Flash Control API.

Or is there already a control allowing to set Flash strobe mode at
the sensor to: OFF, ON (per each exposed frame), AUTO ?

--

Regards,
Sylwester
---
 Documentation/DocBook/media/v4l/controls.xml |   30 ++++++++++++++++++++++++++
 drivers/media/video/v4l2-ctrls.c             |    2 +
 include/linux/videodev2.h                    |    7 ++++++
 3 files changed, 39 insertions(+), 0 deletions(-)

diff --git a/Documentation/DocBook/media/v4l/controls.xml b/Documentation/DocBook/media/v4l/controls.xml
index 48a0434..1745187 100644
--- a/Documentation/DocBook/media/v4l/controls.xml
+++ b/Documentation/DocBook/media/v4l/controls.xml
@@ -3462,6 +3462,36 @@ interface and may change in the future.</para>
 	    after strobe during which another strobe will not be
 	    possible. This is a read-only control.</entry>
 	  </row>
+	  <row>
+	    <entry spanname="id"><constant>V4L2_CID_FLASH_HW_STROBE_MODE</constant></entry>
+	    <entry>menu</entry>
+	  </row>
+	  <row id="v4l2-flash-hw-strobe-mode">
+	    <entry spanname="descr">Determines the mode of hardware flash strobe
+	    at the device external to a flash controller, e.g. image sensor. This
+	    control has effect only when <constant>V4L2_CID_FLASH_STROBE_SOURCE
+	    </constant> is set to <constant>V4L2_FLASH_STROBE_SOURCE_EXTERNAL
+	    </constant> at the flash controller.
+            </entry>
+	  </row>
+	  <row>
+	    <entrytbl spanname="descr" cols="2">
+	      <tbody valign="top">
+		<row>
+		  <entry><constant>V4L2_FLASH_HW_STROBE_MODE_OFF</constant></entry>
+		  <entry>Flash strobe disabled.</entry>
+		</row>
+		<row>
+		  <entry><constant>V4L2_FLASH_HW_STROBE_MODE_ON</constant></entry>
+		  <entry>Flash strobe enabled for each exposed frame.</entry>
+		</row>
+		<row>
+		  <entry><constant>V4L2_FLASH_HW_STROBE_MODE_AUTO</constant></entry>
+		  <entry>Flash strobe determined automatically.</entry>
+		</row>
+	      </tbody>
+	    </entrytbl>
+	  </row>
 	  <row><entry></entry></row>
 	</tbody>
       </tgroup>
diff --git a/drivers/media/video/v4l2-ctrls.c b/drivers/media/video/v4l2-ctrls.c
index 96ec73d..0d188e3 100644
--- a/drivers/media/video/v4l2-ctrls.c
+++ b/drivers/media/video/v4l2-ctrls.c
@@ -619,6 +619,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_FLASH_FAULT:		return "Faults";
 	case V4L2_CID_FLASH_CHARGE:		return "Charge";
 	case V4L2_CID_FLASH_READY:		return "Ready to strobe";
+	case V4L2_CID_FLASH_HW_STROBE_MODE:	return "Hardware strobe mode";

 	default:
 		return NULL;
@@ -699,6 +700,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_TUNE_PREEMPHASIS:
 	case V4L2_CID_FLASH_LED_MODE:
 	case V4L2_CID_FLASH_STROBE_SOURCE:
+	case V4L2_CID_FLASH_HW_STROBE_MODE:
 	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
 	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE:
 	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h
index d43149c..7ffb47d 100644
--- a/include/linux/videodev2.h
+++ b/include/linux/videodev2.h
@@ -1705,6 +1705,13 @@ enum v4l2_flash_strobe_source {
 #define V4L2_CID_FLASH_CHARGE			(V4L2_CID_FLASH_CLASS_BASE + 11)
 #define V4L2_CID_FLASH_READY			(V4L2_CID_FLASH_CLASS_BASE + 12)

+#define V4L2_CID_FLASH_HW_STROBE_MODE		(V4L2_CID_FLASH_CLASS_BASE + 13)
+enum v4l2_flash_hw_strobe_mode {
+	V4L2_FLASH_HW_STROBE_OFF,
+	V4L2_FLASH_HW_STROBE_ON,
+	V4L2_FLASH_HW_STROBE_AUTO,
+};
+
 /*
  *	T U N I N G
  */
--
1.7.4.1

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

Thanks for the RFC.

On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
> for devices that are source of an external flash strobe for flash
> devices. This part seems to be missing in current Flash control
> class, i.e. a means for configuring devices that are not camera
> flash drivers but involve a flash related functionality.
> 
> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
> to determine the flash control behavior, for instance, at an image
> sensor device.
> 
> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
> a flash subdev is present in the system.
> 
> Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
> ---
> 
> Hi Sakari,
> 
> My apologies for not bringing this earlier when you were designing
> the Flash control API.
> It seems like a use case were a sensor controller drives a strobe
> signal for a Flash and the sensor side requires some set up doesn't
> quite fit in the Flash Control API.
> 
> Or is there already a control allowing to set Flash strobe mode at
> the sensor to: OFF, ON (per each exposed frame), AUTO ?

The flash API defines the API for the flash, not for the sensor which might
be controlling the flash through the hardware strobe pin. I left that out
deliberately before I could see what kind of controls would be needed for
that.

If I understand you correctly, this control is intended to configure the
flash strobe per-frame? That may be somewhat hardware-dependent.

Some hardware is able to strobe the flash for the "next possible frame" or
for the first frame when the streaming is started. In either of the cases,
the frames before and after the one exposed with the flash typically are
ruined because the flash has exposed only a part of them. You typically want
to discard such frames.

The timing control of the flash strobe fully depends on the type of the
flash: LED flash typically remains on for the whole duration of the frame
exposure, whereas on xenon flash the full frame must be being exposed when
the flash is being fired.

Also different use cases may require different flash timing handling. [1]

Some sensors have a synchronised electrical shutter (or what was it called,
something like that anyway); that causes the exposure of all the lines of
the sensor to stop at the same time. This effectively eliminates the rolling
shutter effect. The user should know whether (s)he is using synchronised
shutter or rolling shutter since that affects the timing a lot.

How the control of the hardware strobe should look like to the user?

I don't think the flash handling can be fully expressed by a single control
--- except for end user applications. They very likely don't want to know
about all the flash timing related details.

Are you able tell more about your use case? How about the sensor providing
the hardware strobe signal?

[1] http://www.spinics.net/lists/linux-media/msg31363.html

Cheers,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

Hi Sakari,

On 12/05/2011 11:41 PM, Sakari Ailus wrote:
> On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
>> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
>> for devices that are source of an external flash strobe for flash
>> devices. This part seems to be missing in current Flash control
>> class, i.e. a means for configuring devices that are not camera
>> flash drivers but involve a flash related functionality.
>>
>> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
>> to determine the flash control behavior, for instance, at an image
>> sensor device.
>>
>> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
>> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
>> a flash subdev is present in the system.
>>
>> Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
>> ---
>>
>> Hi Sakari,
>>
>> My apologies for not bringing this earlier when you were designing
>> the Flash control API.
>> It seems like a use case were a sensor controller drives a strobe
>> signal for a Flash and the sensor side requires some set up doesn't
>> quite fit in the Flash Control API.
>>
>> Or is there already a control allowing to set Flash strobe mode at
>> the sensor to: OFF, ON (per each exposed frame), AUTO ?

Thank you for the in-depth opinion (and sorry for the delayed response).

> The flash API defines the API for the flash, not for the sensor which might
> be controlling the flash through the hardware strobe pin. I left that out
> deliberately before I could see what kind of controls would be needed for
> that.
> 
> If I understand you correctly, this control is intended to configure the
> flash strobe per-frame? That may be somewhat hardware-dependent.

Yes, per captured frame. Actually the controls I proposed were meant to select
specific flash strobe algorithm. What refinements could be relevant for those
algorithms may be a different question. Something like the proposed controls
is really almost all that is offered by many of hardware we use.

> Some hardware is able to strobe the flash for the "next possible frame" or
> for the first frame when the streaming is started. In either of the cases,
> the frames before and after the one exposed with the flash typically are
> ruined because the flash has exposed only a part of them. You typically want
> to discard such frames.

Is this the case for Xenon flash as well, or LED only ?

I think the fact that we're using video capture like interface for still capture
adds complexity in such cases.

> The timing control of the flash strobe fully depends on the type of the
> flash: LED flash typically remains on for the whole duration of the frame
> exposure, whereas on xenon flash the full frame must be being exposed when
> the flash is being fired.

Indeed, I should have separated the LED and Xenon case in the first place.

Do you think we could start with separate menu controls for LED and Xenon
flash strobe, e.g.

V4L2_FLASH_LED_STROBE_MODE,
V4L2_FLASH_XENON_STROBE_MODE

and then think of what controls would be needed for each particular mode
under these menus ?

> Also different use cases may require different flash timing handling. [1]

I think we need to be able to specify flash strobe delay relative to exposure
start in absolute time and relative to exposure time units.

> 
> Some sensors have a synchronised electrical shutter (or what was it called,
> something like that anyway); that causes the exposure of all the lines of

I guess you mean two-curtain type shutter, like the one described here:

http://camerapedia.wikia.com/wiki/Focal_plane_shutter
http://www.photozone.de/hi-speed-flash-sync

> the sensor to stop at the same time. This effectively eliminates the rolling
> shutter effect. The user should know whether (s)he is using synchronised
> shutter or rolling shutter since that affects the timing a lot.
> 
> How the control of the hardware strobe should look like to the user?
> 
> I don't think the flash handling can be fully expressed by a single control
> --- except for end user applications. They very likely don't want to know
> about all the flash timing related details.

Agreed.

> 
> Are you able tell more about your use case? How about the sensor providing
> the hardware strobe signal?

As a light source a high intensity white LED is used. The LED current control
circuit is directly controlled by a sensor, let's say for simplicity through
one pin.
Now all the magic happens in the sensor firmware and the user can only select
flash programs, e.g. always on/off or auto. I've seen the front curtain and
rear curtain modes used here and there. As you may know these are used in
"slow sync" flash case, where the sensor is fired at the beginning or at the
end of long exposure period.

For example S5K6AA sensor provides following options in REG_TC_FLS_Mode
register for LED flash strobe:

0: TC_FLASH_DISABLE,
1: TC_FLASH_CONT_ENABLE,  // Always on
2: TC_FLASH_PULSE_ENABLE, // Use burst pulse on every capture
3: TC_FLASH_PULSE_AUTO    // Sensor controls the Flash status (burst mode)

For option 3 there is also a register:

REG_TC_FLS_Threshold - Set flash activation threshold in normalized
                       brightness units

For Xenon flash

REG_TC_FLS_XenonMode (Set Xenon flash mode):

0: TC_XENON_DISABLE,
1: TC_XENON_ONE_STROBE, // Use one strobe
2: TC_XENON_PRE_FLASH   // Use n strobes for pre-flash and another one, full

REG_TC_FLS_XenonPreFlashCnt - Number of Xenon pre-flash strobes


And this sensor has also register to trigger still (single- or multi-frame)
capture (REG_TC_GP_EnableCapture), i.e. switch from low resolution/high frame
rate operation to higher resolution image capture.


> [1] http://www.spinics.net/lists/linux-media/msg31363.html


--
Regards,
Sylwester

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

On Sat, Dec 10, 2011 at 02:55:19PM +0100, Sylwester Nawrocki wrote:
> Hi Sakari,
> 
> On 12/05/2011 11:41 PM, Sakari Ailus wrote:
> > On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
> >> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
> >> for devices that are source of an external flash strobe for flash
> >> devices. This part seems to be missing in current Flash control
> >> class, i.e. a means for configuring devices that are not camera
> >> flash drivers but involve a flash related functionality.
> >>
> >> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
> >> to determine the flash control behavior, for instance, at an image
> >> sensor device.
> >>
> >> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
> >> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
> >> a flash subdev is present in the system.
> >>
> >> Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
> >> ---
> >>
> >> Hi Sakari,
> >>
> >> My apologies for not bringing this earlier when you were designing
> >> the Flash control API.
> >> It seems like a use case were a sensor controller drives a strobe
> >> signal for a Flash and the sensor side requires some set up doesn't
> >> quite fit in the Flash Control API.
> >>
> >> Or is there already a control allowing to set Flash strobe mode at
> >> the sensor to: OFF, ON (per each exposed frame), AUTO ?
> 
> Thank you for the in-depth opinion (and sorry for the delayed response).

You're welcome! Thanks for bringing up the topic! :-)

> > The flash API defines the API for the flash, not for the sensor which might
> > be controlling the flash through the hardware strobe pin. I left that out
> > deliberately before I could see what kind of controls would be needed for
> > that.
> > 
> > If I understand you correctly, this control is intended to configure the
> > flash strobe per-frame? That may be somewhat hardware-dependent.
> 
> Yes, per captured frame. Actually the controls I proposed were meant to select
> specific flash strobe algorithm. What refinements could be relevant for those
> algorithms may be a different question. Something like the proposed controls
> is really almost all that is offered by many of hardware we use.



> > Some hardware is able to strobe the flash for the "next possible frame" or
> > for the first frame when the streaming is started. In either of the cases,
> > the frames before and after the one exposed with the flash typically are
> > ruined because the flash has exposed only a part of them. You typically want
> > to discard such frames.
> 
> Is this the case for Xenon flash as well, or LED only ?

Both xenon and LED.

> I think the fact that we're using video capture like interface for still capture
> adds complexity in such cases.

It also adds flexibility. You can expose one frame with xenon flash w/o
stopping streaming.

> > flash: LED flash typically remains on for the whole duration of the frame
> > exposure, whereas on xenon flash the full frame must be being exposed when
> > the flash is being fired.
> 
> Indeed, I should have separated the LED and Xenon case in the first place.
> 
> Do you think we could start with separate menu controls for LED and Xenon
> flash strobe, e.g.
> 
> V4L2_FLASH_LED_STROBE_MODE,
> V4L2_FLASH_XENON_STROBE_MODE
> 
> and then think of what controls would be needed for each particular mode
> under these menus ?

Do we need to separate them? I don't think they're very different, with the
possible exception of intensity control. For xenon flashes the intensity is
at least sometimes controlled by the strobe pulse length.

Still, flash strobe start timing and length control are the same.

> > Also different use cases may require different flash timing handling. [1]
> 
> I think we need to be able to specify flash strobe delay relative to exposure
> start in absolute time and relative to exposure time units.

I agree. I actually just sent a few patches which could be relevant to this,
you're cc'd (patches 1 and 2):

<URL:http://www.mail-archive.com/linux-media@vger.kernel.org/msg39798.html>

What units your sensor uses naturally?

> > Some sensors have a synchronised electrical shutter (or what was it called,
> > something like that anyway); that causes the exposure of all the lines of
> 
> I guess you mean two-curtain type shutter, like the one described here:
> 
> http://camerapedia.wikia.com/wiki/Focal_plane_shutter
> http://www.photozone.de/hi-speed-flash-sync

No, but I wasn't actually aware of that. :)

This is what I meant:

<URL:http://forums.dpreview.com/forums/read.asp?forum=1037&message=37167063>

> > the sensor to stop at the same time. This effectively eliminates the rolling
> > shutter effect. The user should know whether (s)he is using synchronised
> > shutter or rolling shutter since that affects the timing a lot.
> > 
> > How the control of the hardware strobe should look like to the user?
> > 
> > I don't think the flash handling can be fully expressed by a single control
> > --- except for end user applications. They very likely don't want to know
> > about all the flash timing related details.
> 
> Agreed.
> 
> > 
> > Are you able tell more about your use case? How about the sensor providing
> > the hardware strobe signal?
> 
> As a light source a high intensity white LED is used. The LED current control
> circuit is directly controlled by a sensor, let's say for simplicity through
> one pin.

Can there be use for more? That said, I like simple things. ;)

> Now all the magic happens in the sensor firmware and the user can only select
> flash programs, e.g. always on/off or auto. I've seen the front curtain and
> rear curtain modes used here and there. As you may know these are used in
> "slow sync" flash case, where the sensor is fired at the beginning or at the
> end of long exposure period.

Right. I think I'll need to refresh my memory on the flash usage.

> For example S5K6AA sensor provides following options in REG_TC_FLS_Mode
> register for LED flash strobe:
> 
> 0: TC_FLASH_DISABLE,
> 1: TC_FLASH_CONT_ENABLE,  // Always on
> 2: TC_FLASH_PULSE_ENABLE, // Use burst pulse on every capture
> 3: TC_FLASH_PULSE_AUTO    // Sensor controls the Flash status (burst mode)
> 
> For option 3 there is also a register:
> 
> REG_TC_FLS_Threshold - Set flash activation threshold in normalized
>                        brightness units
> 
> For Xenon flash
> 
> REG_TC_FLS_XenonMode (Set Xenon flash mode):
> 
> 0: TC_XENON_DISABLE,
> 1: TC_XENON_ONE_STROBE, // Use one strobe
> 2: TC_XENON_PRE_FLASH   // Use n strobes for pre-flash and another one, full
> 
> REG_TC_FLS_XenonPreFlashCnt - Number of Xenon pre-flash strobes
> 
> 
> And this sensor has also register to trigger still (single- or multi-frame)
> capture (REG_TC_GP_EnableCapture), i.e. switch from low resolution/high frame
> rate operation to higher resolution image capture.

Some of that functionality might fit better behind private ioctls instead.
We'll first need to see what looks more or less generic. I'll get back to
the topic in the near future.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

Hi Sakari,

On 12/14/2011 10:51 PM, Sakari Ailus wrote:
> On Sat, Dec 10, 2011 at 02:55:19PM +0100, Sylwester Nawrocki wrote:
>> On 12/05/2011 11:41 PM, Sakari Ailus wrote:
>>> On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
>>>> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
>>>> for devices that are source of an external flash strobe for flash
>>>> devices. This part seems to be missing in current Flash control
>>>> class, i.e. a means for configuring devices that are not camera
>>>> flash drivers but involve a flash related functionality.
>>>>
>>>> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
>>>> to determine the flash control behavior, for instance, at an image
>>>> sensor device.
>>>>
>>>> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
>>>> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
>>>> a flash subdev is present in the system.
>>>>
>>>> Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
>>>> ---
>>>>
>>>> Hi Sakari,
>>>>
>>>> My apologies for not bringing this earlier when you were designing
>>>> the Flash control API.
>>>> It seems like a use case were a sensor controller drives a strobe
>>>> signal for a Flash and the sensor side requires some set up doesn't
>>>> quite fit in the Flash Control API.
>>>>
>>>> Or is there already a control allowing to set Flash strobe mode at
>>>> the sensor to: OFF, ON (per each exposed frame), AUTO ?
>>
>> Thank you for the in-depth opinion (and sorry for the delayed response).
> 
> You're welcome! Thanks for bringing up the topic! :-)
> 
>>> The flash API defines the API for the flash, not for the sensor which might
>>> be controlling the flash through the hardware strobe pin. I left that out
>>> deliberately before I could see what kind of controls would be needed for
>>> that.
>>>
>>> If I understand you correctly, this control is intended to configure the
>>> flash strobe per-frame? That may be somewhat hardware-dependent.
>>
>> Yes, per captured frame. Actually the controls I proposed were meant to select
>> specific flash strobe algorithm. What refinements could be relevant for those
>> algorithms may be a different question. Something like the proposed controls
>> is really almost all that is offered by many of hardware we use.
> 
> 
> 
>>> Some hardware is able to strobe the flash for the "next possible frame" or
>>> for the first frame when the streaming is started. In either of the cases,
>>> the frames before and after the one exposed with the flash typically are
>>> ruined because the flash has exposed only a part of them. You typically want
>>> to discard such frames.
>>
>> Is this the case for Xenon flash as well, or LED only ?
> 
> Both xenon and LED.
> 
>> I think the fact that we're using video capture like interface for still 
>> capture adds complexity in such cases.
> 
> It also adds flexibility. You can expose one frame with xenon flash w/o
> stopping streaming.

Yes, but only if you're able to control an image sensor and ISP at very low
level.

>>> flash: LED flash typically remains on for the whole duration of the frame
>>> exposure, whereas on xenon flash the full frame must be being exposed when
>>> the flash is being fired.
>>
>> Indeed, I should have separated the LED and Xenon case in the first place.
>>
>> Do you think we could start with separate menu controls for LED and Xenon
>> flash strobe, e.g.
>>
>> V4L2_FLASH_LED_STROBE_MODE,
>> V4L2_FLASH_XENON_STROBE_MODE
>>
>> and then think of what controls would be needed for each particular mode
>> under these menus ?
> 
> Do we need to separate them? I don't think they're very different, with the

Perhaps we don't need to separate them this much. But the differences are
significant IHMO. However it could be enough if we pass information about
the flash type in sensor's platform data. (I wrote this before I've seen
your smiapp driver patches;)).

> possible exception of intensity control. For xenon flashes the intensity is
> at least sometimes controlled by the strobe pulse length.
> 
> Still, flash strobe start timing and length control are the same.

But strobe control signal characteristics may be different for Xenon lamp,
due to pre-flash strobes for example ? Does pre-flash really apply to
LED flashes as well ?

>>> Also different use cases may require different flash timing handling. [1]
>>
>> I think we need to be able to specify flash strobe delay relative to exposure
>> start in absolute time and relative to exposure time units.
> 
> I agree. I actually just sent a few patches which could be relevant to this,
> you're cc'd (patches 1 and 2):
> 
> <URL:http://www.mail-archive.com/linux-media@vger.kernel.org/msg39798.html>
> 
> What units your sensor uses naturally?

The one which allows to specify flash strobe delay and its length just accepts
absolute time with resolution determined by sensor master clock period.

But in many cases with a high brightness LED as flash the delay is handled by
sensor firmware with only relatively high level interface exposed to the user.

>>> Some sensors have a synchronised electrical shutter (or what was it called,
>>> something like that anyway); that causes the exposure of all the lines of
>>
>> I guess you mean two-curtain type shutter, like the one described here:
>>
>> http://camerapedia.wikia.com/wiki/Focal_plane_shutter
>> http://www.photozone.de/hi-speed-flash-sync
> 
> No, but I wasn't actually aware of that. :)
> 
> This is what I meant:
> 
> <URL:http://forums.dpreview.com/forums/read.asp?forum=1037&message=37167063>

Hmm, the characteristic of this kind of shutter might be similar to mechanical
shutter. So there would be at least 2 types of shutter from flash POV:
 - electronics rolling shutter,
 - mechanical (electronic synchronized) shutter.

Then maybe we could develop a control indicating shutter type:

V4L2_CID_FLASH_SHUTTER_TYPE
  V4L2_FLASH_ELECTRONIC_ROLLING_SHUTTER
  V4L2_FLASH_MECHANICAL_SHUTTER

But I don't know what to do with this information exactly at this moment.. :-)

>>> the sensor to stop at the same time. This effectively eliminates the rolling
>>> shutter effect. The user should know whether (s)he is using synchronised
>>> shutter or rolling shutter since that affects the timing a lot.
>>>
>>> How the control of the hardware strobe should look like to the user?
>>>
>>> I don't think the flash handling can be fully expressed by a single control
>>> --- except for end user applications. They very likely don't want to know
>>> about all the flash timing related details.
>>
>> Agreed.
>>
>>>
>>> Are you able tell more about your use case? How about the sensor providing
>>> the hardware strobe signal?
>>
>> As a light source a high intensity white LED is used. The LED current control
>> circuit is directly controlled by a sensor, let's say for simplicity through
>> one pin.
> 
> Can there be use for more? That said, I like simple things. ;)

:-) Yes, physically. However logically it's safe to assume we have only one for
strobe. Some Flash LED drivers might have more pins for operation mode control
like capacitor precharge, etc. E.g. like this one:

URL: http://www.maxim-ic.com/datasheet/index.mvp/id/4163

Some may use additional pin for intensity control (PWM), whereas for some the
intensity is controlled through I2C (and we might not always have a Flash subdev
driver for them - for instance a led class driver).

URL : http://para.maxim-ic.com/en/results.mvp?fam=whiteled


-- 
Regards,
Sylwester

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

On Wed, Dec 28, 2011 at 05:06:55PM +0100, Sylwester Nawrocki wrote:
> On 12/14/2011 10:51 PM, Sakari Ailus wrote:
> > On Sat, Dec 10, 2011 at 02:55:19PM +0100, Sylwester Nawrocki wrote:
> >> On 12/05/2011 11:41 PM, Sakari Ailus wrote:
> >>> On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
> >>>> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
> >>>> for devices that are source of an external flash strobe for flash
> >>>> devices. This part seems to be missing in current Flash control
> >>>> class, i.e. a means for configuring devices that are not camera
> >>>> flash drivers but involve a flash related functionality.
> >>>>
> >>>> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
> >>>> to determine the flash control behavior, for instance, at an image
> >>>> sensor device.
> >>>>
> >>>> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
> >>>> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
> >>>> a flash subdev is present in the system.
> >>>>
> >>>> Signed-off-by: Sylwester Nawrocki <snjw23@gmail.com>
> >>>> ---
> >>>>
> >>>> Hi Sakari,
> >>>>
> >>>> My apologies for not bringing this earlier when you were designing
> >>>> the Flash control API.
> >>>> It seems like a use case were a sensor controller drives a strobe
> >>>> signal for a Flash and the sensor side requires some set up doesn't
> >>>> quite fit in the Flash Control API.
> >>>>
> >>>> Or is there already a control allowing to set Flash strobe mode at
> >>>> the sensor to: OFF, ON (per each exposed frame), AUTO ?
> >>
> >> Thank you for the in-depth opinion (and sorry for the delayed response).
> > 
> > You're welcome! Thanks for bringing up the topic! :-)
> > 
> >>> The flash API defines the API for the flash, not for the sensor which might
> >>> be controlling the flash through the hardware strobe pin. I left that out
> >>> deliberately before I could see what kind of controls would be needed for
> >>> that.
> >>>
> >>> If I understand you correctly, this control is intended to configure the
> >>> flash strobe per-frame? That may be somewhat hardware-dependent.
> >>
> >> Yes, per captured frame. Actually the controls I proposed were meant to select
> >> specific flash strobe algorithm. What refinements could be relevant for those
> >> algorithms may be a different question. Something like the proposed controls
> >> is really almost all that is offered by many of hardware we use.
> > 
> > 
> > 
> >>> Some hardware is able to strobe the flash for the "next possible frame" or
> >>> for the first frame when the streaming is started. In either of the cases,
> >>> the frames before and after the one exposed with the flash typically are
> >>> ruined because the flash has exposed only a part of them. You typically want
> >>> to discard such frames.
> >>
> >> Is this the case for Xenon flash as well, or LED only ?
> > 
> > Both xenon and LED.
> > 
> >> I think the fact that we're using video capture like interface for still 
> >> capture adds complexity in such cases.
> > 
> > It also adds flexibility. You can expose one frame with xenon flash w/o
> > stopping streaming.
> 
> Yes, but only if you're able to control an image sensor and ISP at very low
> level.

For ISP I don't see it would matter, but for sensor, yes.

In general most don't provide a way to expose frames in a stream with flash
--- instead the stream has to be stopped first. I think this is what 
users would expect to do in regular cases.

If flash exposure isn't possible while streaming, the sensor can always tell
the strobe control is busy. Do you think that would work for you?

> >>> flash: LED flash typically remains on for the whole duration of the frame
> >>> exposure, whereas on xenon flash the full frame must be being exposed when
> >>> the flash is being fired.
> >>
> >> Indeed, I should have separated the LED and Xenon case in the first place.
> >>
> >> Do you think we could start with separate menu controls for LED and Xenon
> >> flash strobe, e.g.
> >>
> >> V4L2_FLASH_LED_STROBE_MODE,
> >> V4L2_FLASH_XENON_STROBE_MODE
> >>
> >> and then think of what controls would be needed for each particular mode
> >> under these menus ?
> > 
> > Do we need to separate them? I don't think they're very different, with the
> 
> Perhaps we don't need to separate them this much. But the differences are
> significant IHMO. However it could be enough if we pass information about
> the flash type in sensor's platform data. (I wrote this before I've seen
> your smiapp driver patches;)).

Yeah; that might make sense. Smart sensors might even require information on
the flash type etc. to be able to control it properly.

> > possible exception of intensity control. For xenon flashes the intensity is
> > at least sometimes controlled by the strobe pulse length.
> > 
> > Still, flash strobe start timing and length control are the same.
> 
> But strobe control signal characteristics may be different for Xenon lamp,
> due to pre-flash strobes for example ? Does pre-flash really apply to
> LED flashes as well ?

Yes; we do that on the Nokia N9 and, I think, on the N900 as well.

I think that from the sensor's point of view the low level controls are
still the same, as well as the high level ones. I have to admit I haven't
worked on xenon flash devices but I have neither seen any flash control
functionality (related to hardware, I mean; there are some controls which
are valid for xenon only, such as charging the capacitor on/of) which would
only be valid for either flash types.

> >>> Also different use cases may require different flash timing handling. [1]
> >>
> >> I think we need to be able to specify flash strobe delay relative to exposure
> >> start in absolute time and relative to exposure time units.
> > 
> > I agree. I actually just sent a few patches which could be relevant to this,
> > you're cc'd (patches 1 and 2):
> > 
> > <URL:http://www.mail-archive.com/linux-media@vger.kernel.org/msg39798.html>
> > 
> > What units your sensor uses naturally?
> 
> The one which allows to specify flash strobe delay and its length just accepts
> absolute time with resolution determined by sensor master clock period.

Same on the SMIA++.

> But in many cases with a high brightness LED as flash the delay is handled by
> sensor firmware with only relatively high level interface exposed to the user.
> 
> >>> Some sensors have a synchronised electrical shutter (or what was it called,
> >>> something like that anyway); that causes the exposure of all the lines of
> >>
> >> I guess you mean two-curtain type shutter, like the one described here:
> >>
> >> http://camerapedia.wikia.com/wiki/Focal_plane_shutter
> >> http://www.photozone.de/hi-speed-flash-sync
> > 
> > No, but I wasn't actually aware of that. :)
> > 
> > This is what I meant:
> > 
> > <URL:http://forums.dpreview.com/forums/read.asp?forum=1037&message=37167063>
> 
> Hmm, the characteristic of this kind of shutter might be similar to mechanical
> shutter. So there would be at least 2 types of shutter from flash POV:
>  - electronics rolling shutter,
>  - mechanical (electronic synchronized) shutter.
> 
> Then maybe we could develop a control indicating shutter type:
> 
> V4L2_CID_FLASH_SHUTTER_TYPE
>   V4L2_FLASH_ELECTRONIC_ROLLING_SHUTTER
>   V4L2_FLASH_MECHANICAL_SHUTTER

I agree; we should expose the type. Once we have ones with synchronous
shutters. :-)

Synchronous shutters are just as good as mechanical ones --- or likely even
better; the exposure time can be controlled precisely, unlike on mechanical
shutters.

> But I don't know what to do with this information exactly at this moment.. :-)

With the addition of the new sensor configuration controls, this allows to
configure the flash timing so that the exposure only spans the desired area
of the image at the desired point of time and duration.

> >>> the sensor to stop at the same time. This effectively eliminates the rolling
> >>> shutter effect. The user should know whether (s)he is using synchronised
> >>> shutter or rolling shutter since that affects the timing a lot.
> >>>
> >>> How the control of the hardware strobe should look like to the user?
> >>>
> >>> I don't think the flash handling can be fully expressed by a single control
> >>> --- except for end user applications. They very likely don't want to know
> >>> about all the flash timing related details.
> >>
> >> Agreed.
> >>
> >>>
> >>> Are you able tell more about your use case? How about the sensor providing
> >>> the hardware strobe signal?
> >>
> >> As a light source a high intensity white LED is used. The LED current control
> >> circuit is directly controlled by a sensor, let's say for simplicity through
> >> one pin.
> > 
> > Can there be use for more? That said, I like simple things. ;)
> 
> :-) Yes, physically. However logically it's safe to assume we have only one for
> strobe. Some Flash LED drivers might have more pins for operation mode control
> like capacitor precharge, etc. E.g. like this one:
> 
> URL: http://www.maxim-ic.com/datasheet/index.mvp/id/4163

This is an interesting one. I would have preferred I2C control interface,
but whatever... :-) Sounds like the capacitor charge control might not be
valid only for xenon flashes after all. ;-)

> Some may use additional pin for intensity control (PWM), whereas for some the
> intensity is controlled through I2C (and we might not always have a Flash subdev
> driver for them - for instance a led class driver).
> 
> URL : http://para.maxim-ic.com/en/results.mvp?fam=whiteled

But having a led class driver makes it somewhat difficult to work with V4L2,
doesn't it? The pwm source likely isn't a gpio typically?

Cheers,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

Hi Sakari,

My apologies for reviving so old thread ;)

It unfortunately didn't end with any conclusion but we need 
a functionality similar to the provided by the $subject patch 
for multiple different {camera sensor + ISP} devices.

On 12/10/2011 02:55 PM, Sylwester Nawrocki wrote:
> Hi Sakari,
> 
> On 12/05/2011 11:41 PM, Sakari Ailus wrote:
>> On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
>>> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
>>> for devices that are source of an external flash strobe for flash
>>> devices. This part seems to be missing in current Flash control
>>> class, i.e. a means for configuring devices that are not camera
>>> flash drivers but involve a flash related functionality.
>>>
>>> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
>>> to determine the flash control behavior, for instance, at an image
>>> sensor device.
>>>
>>> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
>>> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
>>> a flash subdev is present in the system.
>>>
>>> Signed-off-by: Sylwester Nawrocki<snjw23@gmail.com>
>>> ---
>>>
>>> Hi Sakari,
>>>
>>> My apologies for not bringing this earlier when you were designing
>>> the Flash control API.
>>> It seems like a use case were a sensor controller drives a strobe
>>> signal for a Flash and the sensor side requires some set up doesn't
>>> quite fit in the Flash Control API.
>>>
>>> Or is there already a control allowing to set Flash strobe mode at
>>> the sensor to: OFF, ON (per each exposed frame), AUTO ?
> 
> Thank you for the in-depth opinion (and sorry for the delayed response).
> 
>> The flash API defines the API for the flash, not for the sensor which might
>> be controlling the flash through the hardware strobe pin. I left that out
>> deliberately before I could see what kind of controls would be needed for
>> that.
>>
>> If I understand you correctly, this control is intended to configure the
>> flash strobe per-frame? That may be somewhat hardware-dependent.
> 
> Yes, per captured frame. Actually the controls I proposed were meant to select
> specific flash strobe algorithm. What refinements could be relevant for those
> algorithms may be a different question. Something like the proposed controls
> is really almost all that is offered by many of hardware we use.
> 
>> Some hardware is able to strobe the flash for the "next possible frame" or
>> for the first frame when the streaming is started. In either of the cases,
>> the frames before and after the one exposed with the flash typically are
>> ruined because the flash has exposed only a part of them. You typically want
>> to discard such frames.
> 
> Is this the case for Xenon flash as well, or LED only ?
> 
> I think the fact that we're using video capture like interface for still capture
> adds complexity in such cases.
> 
>> The timing control of the flash strobe fully depends on the type of the
>> flash: LED flash typically remains on for the whole duration of the frame
>> exposure, whereas on xenon flash the full frame must be being exposed when
>> the flash is being fired.
> 
> Indeed, I should have separated the LED and Xenon case in the first place.
> 
> Do you think we could start with separate menu controls for LED and Xenon
> flash strobe, e.g.
> 
> V4L2_FLASH_LED_STROBE_MODE,
> V4L2_FLASH_XENON_STROBE_MODE
> 
> and then think of what controls would be needed for each particular mode
> under these menus ?
> 
>> Also different use cases may require different flash timing handling. [1]
> 
> I think we need to be able to specify flash strobe delay relative to exposure
> start in absolute time and relative to exposure time units.
> 
>>
>> Some sensors have a synchronised electrical shutter (or what was it called,
>> something like that anyway); that causes the exposure of all the lines of
> 
> I guess you mean two-curtain type shutter, like the one described here:
> 
> http://camerapedia.wikia.com/wiki/Focal_plane_shutter
> http://www.photozone.de/hi-speed-flash-sync
> 
>> the sensor to stop at the same time. This effectively eliminates the rolling
>> shutter effect. The user should know whether (s)he is using synchronised
>> shutter or rolling shutter since that affects the timing a lot.
>>
>> How the control of the hardware strobe should look like to the user?
>>
>> I don't think the flash handling can be fully expressed by a single control
>> --- except for end user applications. They very likely don't want to know
>> about all the flash timing related details.
> 
> Agreed.
> 
>>
>> Are you able tell more about your use case? How about the sensor providing
>> the hardware strobe signal?
> 
> As a light source a high intensity white LED is used. The LED current control
> circuit is directly controlled by a sensor, let's say for simplicity through
> one pin.
> Now all the magic happens in the sensor firmware and the user can only select
> flash programs, e.g. always on/off or auto. I've seen the front curtain and
> rear curtain modes used here and there. As you may know these are used in
> "slow sync" flash case, where the sensor is fired at the beginning or at the
> end of long exposure period.
> 
> For example S5K6AA sensor provides following options in REG_TC_FLS_Mode
> register for LED flash strobe:
> 
> 0: TC_FLASH_DISABLE,
> 1: TC_FLASH_CONT_ENABLE,  // Always on
> 2: TC_FLASH_PULSE_ENABLE, // Use burst pulse on every capture
> 3: TC_FLASH_PULSE_AUTO    // Sensor controls the Flash status (burst mode)
> 
> For option 3 there is also a register:
> 
> REG_TC_FLS_Threshold - Set flash activation threshold in normalized
>                         brightness units
> 
> For Xenon flash
> 
> REG_TC_FLS_XenonMode (Set Xenon flash mode):
> 
> 0: TC_XENON_DISABLE,
> 1: TC_XENON_ONE_STROBE, // Use one strobe
> 2: TC_XENON_PRE_FLASH   // Use n strobes for pre-flash and another one, full
> 
> REG_TC_FLS_XenonPreFlashCnt - Number of Xenon pre-flash strobes
> 
> 
> And this sensor has also register to trigger still (single- or multi-frame)
> capture (REG_TC_GP_EnableCapture), i.e. switch from low resolution/high frame
> rate operation to higher resolution image capture.
> 
> 
>> [1] http://www.spinics.net/lists/linux-media/msg31363.html

I'm wondering whether we should separate somehow, e.g. by using different
prefix, controls related to the flash controller and controls of camera
device controlling the flash in firmware ? Perhaps it's not necessary.
 
And should we develop distinct set of flash controls for camera devices,
i.e. not the flash controllers itself ? Or may we reuse the existing
flash controls [1] there ?

Having studied multiple Samsung sensor drivers in Android kernels, it 
seems all of them basically expose 4 flash modes:

1. The flash strobe is off (the flash is always off)
2. The flash strobe is on (the flash is fired with each snapshot,
   exact timing is determined in the camera firmware)
3. The flash strobe is controlled automatically by the sensor 
   (the flash may get fired or not, depending on light conditions, etc.)
4. The flash is on continuously (torch function)

There is even a dedicated common control in the private header for samsung
devices [1] - V4L2_CID_CAMERA_FLASH_MODE.

Having studied the existing flash controls [1] I'm wondering whether
a control like

V4L2_CID_FLASH_HW_STROBE_AUTO

with options:
	V4L2_FLASH_HW_STROBE_OFF
	V4L2_FLASH_HW_STROBE_AUTO
	V4L2_FLASH_HW_STROBE_ON


would make sense ? It would cover above cases 1, 3, 2.

Then I'm not sure what to do exactly with the Torch option.
 
Perhaps a boolean control like V4L2_FLASH_TORCH_ENABLED would do ?

Then V4L2_CID_FLASH_HW_STROBE_AUTO and V4L2_FLASH_TORCH_ENABLED could
possibly form a cluster where required.

What do you think ?

--
Thanks,
Sylwester

[1] http://linuxtv.org/downloads/v4l-dvb-apis/extended-controls.html#flash-controls
[2] https://android.googlesource.com/kernel/samsung/+/665612374970f5dba43b65510b52b72130818541/include/linux/videodev2_samsung.h

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

On 10/13/2012 08:52 PM, Sylwester Nawrocki wrote:
> Hi Sakari,
>
> My apologies for reviving so old thread ;)
>
> It unfortunately didn't end with any conclusion but we need
> a functionality similar to the provided by the $subject patch
> for multiple different {camera sensor + ISP} devices.
>
> On 12/10/2011 02:55 PM, Sylwester Nawrocki wrote:
>> Hi Sakari,
>>
>> On 12/05/2011 11:41 PM, Sakari Ailus wrote:
>>> On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
>>>> The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
>>>> for devices that are source of an external flash strobe for flash
>>>> devices. This part seems to be missing in current Flash control
>>>> class, i.e. a means for configuring devices that are not camera
>>>> flash drivers but involve a flash related functionality.
>>>>
>>>> The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
>>>> to determine the flash control behavior, for instance, at an image
>>>> sensor device.
>>>>
>>>> The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
>>>> is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
>>>> a flash subdev is present in the system.
>>>>
>>>> Signed-off-by: Sylwester Nawrocki<snjw23@gmail.com>
>>>> ---
>>>>
>>>> Hi Sakari,
>>>>
>>>> My apologies for not bringing this earlier when you were designing
>>>> the Flash control API.
>>>> It seems like a use case were a sensor controller drives a strobe
>>>> signal for a Flash and the sensor side requires some set up doesn't
>>>> quite fit in the Flash Control API.
>>>>
>>>> Or is there already a control allowing to set Flash strobe mode at
>>>> the sensor to: OFF, ON (per each exposed frame), AUTO ?
>>
>> Thank you for the in-depth opinion (and sorry for the delayed response).
>>
>>> The flash API defines the API for the flash, not for the sensor which might
>>> be controlling the flash through the hardware strobe pin. I left that out
>>> deliberately before I could see what kind of controls would be needed for
>>> that.
>>>
>>> If I understand you correctly, this control is intended to configure the
>>> flash strobe per-frame? That may be somewhat hardware-dependent.
>>
>> Yes, per captured frame. Actually the controls I proposed were meant to select
>> specific flash strobe algorithm. What refinements could be relevant for those
>> algorithms may be a different question. Something like the proposed controls
>> is really almost all that is offered by many of hardware we use.
>>
>>> Some hardware is able to strobe the flash for the "next possible frame" or
>>> for the first frame when the streaming is started. In either of the cases,
>>> the frames before and after the one exposed with the flash typically are
>>> ruined because the flash has exposed only a part of them. You typically want
>>> to discard such frames.
>>
>> Is this the case for Xenon flash as well, or LED only ?
>>
>> I think the fact that we're using video capture like interface for still capture
>> adds complexity in such cases.
>>
>>> The timing control of the flash strobe fully depends on the type of the
>>> flash: LED flash typically remains on for the whole duration of the frame
>>> exposure, whereas on xenon flash the full frame must be being exposed when
>>> the flash is being fired.
>>
>> Indeed, I should have separated the LED and Xenon case in the first place.
>>
>> Do you think we could start with separate menu controls for LED and Xenon
>> flash strobe, e.g.
>>
>> V4L2_FLASH_LED_STROBE_MODE,
>> V4L2_FLASH_XENON_STROBE_MODE
>>
>> and then think of what controls would be needed for each particular mode
>> under these menus ?
>>
>>> Also different use cases may require different flash timing handling. [1]
>>
>> I think we need to be able to specify flash strobe delay relative to exposure
>> start in absolute time and relative to exposure time units.
>>
>>>
>>> Some sensors have a synchronised electrical shutter (or what was it called,
>>> something like that anyway); that causes the exposure of all the lines of
>>
>> I guess you mean two-curtain type shutter, like the one described here:
>>
>> http://camerapedia.wikia.com/wiki/Focal_plane_shutter
>> http://www.photozone.de/hi-speed-flash-sync
>>
>>> the sensor to stop at the same time. This effectively eliminates the rolling
>>> shutter effect. The user should know whether (s)he is using synchronised
>>> shutter or rolling shutter since that affects the timing a lot.
>>>
>>> How the control of the hardware strobe should look like to the user?
>>>
>>> I don't think the flash handling can be fully expressed by a single control
>>> --- except for end user applications. They very likely don't want to know
>>> about all the flash timing related details.
>>
>> Agreed.
>>
>>>
>>> Are you able tell more about your use case? How about the sensor providing
>>> the hardware strobe signal?
>>
>> As a light source a high intensity white LED is used. The LED current control
>> circuit is directly controlled by a sensor, let's say for simplicity through
>> one pin.
>> Now all the magic happens in the sensor firmware and the user can only select
>> flash programs, e.g. always on/off or auto. I've seen the front curtain and
>> rear curtain modes used here and there. As you may know these are used in
>> "slow sync" flash case, where the sensor is fired at the beginning or at the
>> end of long exposure period.
>>
>> For example S5K6AA sensor provides following options in REG_TC_FLS_Mode
>> register for LED flash strobe:
>>
>> 0: TC_FLASH_DISABLE,
>> 1: TC_FLASH_CONT_ENABLE,  // Always on
>> 2: TC_FLASH_PULSE_ENABLE, // Use burst pulse on every capture
>> 3: TC_FLASH_PULSE_AUTO    // Sensor controls the Flash status (burst mode)
>>
>> For option 3 there is also a register:
>>
>> REG_TC_FLS_Threshold - Set flash activation threshold in normalized
>>                          brightness units
>>
>> For Xenon flash
>>
>> REG_TC_FLS_XenonMode (Set Xenon flash mode):
>>
>> 0: TC_XENON_DISABLE,
>> 1: TC_XENON_ONE_STROBE, // Use one strobe
>> 2: TC_XENON_PRE_FLASH   // Use n strobes for pre-flash and another one, full
>>
>> REG_TC_FLS_XenonPreFlashCnt - Number of Xenon pre-flash strobes
>>
>>
>> And this sensor has also register to trigger still (single- or multi-frame)
>> capture (REG_TC_GP_EnableCapture), i.e. switch from low resolution/high frame
>> rate operation to higher resolution image capture.
>>
>>
>>> [1] http://www.spinics.net/lists/linux-media/msg31363.html
>
> I'm wondering whether we should separate somehow, e.g. by using different
> prefix, controls related to the flash controller and controls of camera
> device controlling the flash in firmware ? Perhaps it's not necessary.
>
> And should we develop distinct set of flash controls for camera devices,
> i.e. not the flash controllers itself ? Or may we reuse the existing
> flash controls [1] there ?

Hm, please ignore this, of course the flash control class as currently
defined is intended to support configuration with the flash strobe
controlled by the sensor (Synchronised LED flash).

> Having studied multiple Samsung sensor drivers in Android kernels, it
> seems all of them basically expose 4 flash modes:
>
> 1. The flash strobe is off (the flash is always off)
> 2. The flash strobe is on (the flash is fired with each snapshot,
>     exact timing is determined in the camera firmware)
> 3. The flash strobe is controlled automatically by the sensor
>     (the flash may get fired or not, depending on light conditions, etc.)
> 4. The flash is on continuously (torch function)
>
> There is even a dedicated common control in the private header for samsung
> devices [1] - V4L2_CID_CAMERA_FLASH_MODE.
>
> Having studied the existing flash controls [1] I'm wondering whether
> a control like
>
> V4L2_CID_FLASH_HW_STROBE_AUTO
>
> with options:
> 	V4L2_FLASH_HW_STROBE_OFF
> 	V4L2_FLASH_HW_STROBE_AUTO
> 	V4L2_FLASH_HW_STROBE_ON

The _HW prefix probably needs to be removed there.

> would make sense ? It would cover above cases 1, 3, 2.
>
> Then I'm not sure what to do exactly with the Torch option.
>
> Perhaps a boolean control like V4L2_FLASH_TORCH_ENABLED would do ?

OTOH it seems V4L2_CID_FLASH_LED_MODE and V4L2_CID_FLASH_TORCH_INTENSITY
could do the same. V4L2_CID_FLASH_LED_MODE would be switching between
Flash and Torch function and V4L2_CID_FLASH_TORCH_INTENSITY would be
turning flash LED on/off.

Answering myself, likely a bad sign... :-)


Regards,
Sylwester

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

On Sun, Oct 14, 2012 at 11:16:11AM +0200, Sylwester Nawrocki wrote:
> On 10/13/2012 08:52 PM, Sylwester Nawrocki wrote:
> >My apologies for reviving so old thread ;)

This indeed is a very, very old one! ;-)

> >It unfortunately didn't end with any conclusion but we need
> >a functionality similar to the provided by the $subject patch
> >for multiple different {camera sensor + ISP} devices.
> >
> >On 12/10/2011 02:55 PM, Sylwester Nawrocki wrote:
> >>Hi Sakari,
> >>
> >>On 12/05/2011 11:41 PM, Sakari Ailus wrote:
> >>>On Mon, Dec 05, 2011 at 08:56:46PM +0100, Sylwester Nawrocki wrote:
> >>>>The V4L2_CID_FLASH_HW_STROBE_MODE mode control is intended
> >>>>for devices that are source of an external flash strobe for flash
> >>>>devices. This part seems to be missing in current Flash control
> >>>>class, i.e. a means for configuring devices that are not camera
> >>>>flash drivers but involve a flash related functionality.
> >>>>
> >>>>The V4L2_CID_FLASH_HW_STROBE_MODE control enables the user
> >>>>to determine the flash control behavior, for instance, at an image
> >>>>sensor device.
> >>>>
> >>>>The control has effect only when V4L2_CID_FLASH_STROBE_SOURCE control
> >>>>is set to V4L2_FLASH_STROBE_SOURCE_EXTERNAL at a flash subdev, if
> >>>>a flash subdev is present in the system.
> >>>>
> >>>>Signed-off-by: Sylwester Nawrocki<snjw23@gmail.com>
> >>>>---
> >>>>
> >>>>Hi Sakari,
> >>>>
> >>>>My apologies for not bringing this earlier when you were designing
> >>>>the Flash control API.
> >>>>It seems like a use case were a sensor controller drives a strobe
> >>>>signal for a Flash and the sensor side requires some set up doesn't
> >>>>quite fit in the Flash Control API.
> >>>>
> >>>>Or is there already a control allowing to set Flash strobe mode at
> >>>>the sensor to: OFF, ON (per each exposed frame), AUTO ?
> >>
> >>Thank you for the in-depth opinion (and sorry for the delayed response).
> >>
> >>>The flash API defines the API for the flash, not for the sensor which might
> >>>be controlling the flash through the hardware strobe pin. I left that out
> >>>deliberately before I could see what kind of controls would be needed for
> >>>that.
> >>>
> >>>If I understand you correctly, this control is intended to configure the
> >>>flash strobe per-frame? That may be somewhat hardware-dependent.
> >>
> >>Yes, per captured frame. Actually the controls I proposed were meant to select
> >>specific flash strobe algorithm. What refinements could be relevant for those
> >>algorithms may be a different question. Something like the proposed controls
> >>is really almost all that is offered by many of hardware we use.
> >>
> >>>Some hardware is able to strobe the flash for the "next possible frame" or
> >>>for the first frame when the streaming is started. In either of the cases,
> >>>the frames before and after the one exposed with the flash typically are
> >>>ruined because the flash has exposed only a part of them. You typically want
> >>>to discard such frames.
> >>
> >>Is this the case for Xenon flash as well, or LED only ?
> >>
> >>I think the fact that we're using video capture like interface for still capture
> >>adds complexity in such cases.
> >>
> >>>The timing control of the flash strobe fully depends on the type of the
> >>>flash: LED flash typically remains on for the whole duration of the frame
> >>>exposure, whereas on xenon flash the full frame must be being exposed when
> >>>the flash is being fired.
> >>
> >>Indeed, I should have separated the LED and Xenon case in the first place.
> >>
> >>Do you think we could start with separate menu controls for LED and Xenon
> >>flash strobe, e.g.
> >>
> >>V4L2_FLASH_LED_STROBE_MODE,
> >>V4L2_FLASH_XENON_STROBE_MODE
> >>
> >>and then think of what controls would be needed for each particular mode
> >>under these menus ?
> >>
> >>>Also different use cases may require different flash timing handling. [1]
> >>
> >>I think we need to be able to specify flash strobe delay relative to exposure
> >>start in absolute time and relative to exposure time units.
> >>
> >>>
> >>>Some sensors have a synchronised electrical shutter (or what was it called,
> >>>something like that anyway); that causes the exposure of all the lines of
> >>
> >>I guess you mean two-curtain type shutter, like the one described here:
> >>
> >>http://camerapedia.wikia.com/wiki/Focal_plane_shutter
> >>http://www.photozone.de/hi-speed-flash-sync
> >>
> >>>the sensor to stop at the same time. This effectively eliminates the rolling
> >>>shutter effect. The user should know whether (s)he is using synchronised
> >>>shutter or rolling shutter since that affects the timing a lot.
> >>>
> >>>How the control of the hardware strobe should look like to the user?
> >>>
> >>>I don't think the flash handling can be fully expressed by a single control
> >>>--- except for end user applications. They very likely don't want to know
> >>>about all the flash timing related details.
> >>
> >>Agreed.
> >>
> >>>
> >>>Are you able tell more about your use case? How about the sensor providing
> >>>the hardware strobe signal?
> >>
> >>As a light source a high intensity white LED is used. The LED current control
> >>circuit is directly controlled by a sensor, let's say for simplicity through
> >>one pin.
> >>Now all the magic happens in the sensor firmware and the user can only select
> >>flash programs, e.g. always on/off or auto. I've seen the front curtain and
> >>rear curtain modes used here and there. As you may know these are used in
> >>"slow sync" flash case, where the sensor is fired at the beginning or at the
> >>end of long exposure period.
> >>
> >>For example S5K6AA sensor provides following options in REG_TC_FLS_Mode
> >>register for LED flash strobe:
> >>
> >>0: TC_FLASH_DISABLE,
> >>1: TC_FLASH_CONT_ENABLE,  // Always on
> >>2: TC_FLASH_PULSE_ENABLE, // Use burst pulse on every capture
> >>3: TC_FLASH_PULSE_AUTO    // Sensor controls the Flash status (burst mode)
> >>
> >>For option 3 there is also a register:
> >>
> >>REG_TC_FLS_Threshold - Set flash activation threshold in normalized
> >>                         brightness units
> >>
> >>For Xenon flash
> >>
> >>REG_TC_FLS_XenonMode (Set Xenon flash mode):
> >>
> >>0: TC_XENON_DISABLE,
> >>1: TC_XENON_ONE_STROBE, // Use one strobe
> >>2: TC_XENON_PRE_FLASH   // Use n strobes for pre-flash and another one, full
> >>
> >>REG_TC_FLS_XenonPreFlashCnt - Number of Xenon pre-flash strobes
> >>
> >>
> >>And this sensor has also register to trigger still (single- or multi-frame)
> >>capture (REG_TC_GP_EnableCapture), i.e. switch from low resolution/high frame
> >>rate operation to higher resolution image capture.
> >>
> >>
> >>>[1] http://www.spinics.net/lists/linux-media/msg31363.html
> >
> >I'm wondering whether we should separate somehow, e.g. by using different
> >prefix, controls related to the flash controller and controls of camera
> >device controlling the flash in firmware ? Perhaps it's not necessary.
> >
> >And should we develop distinct set of flash controls for camera devices,
> >i.e. not the flash controllers itself ? Or may we reuse the existing
> >flash controls [1] there ?
> 
> Hm, please ignore this, of course the flash control class as currently
> defined is intended to support configuration with the flash strobe
> controlled by the sensor (Synchronised LED flash).

Currently the flash control reference states that "The V4L2 flash controls
are intended to provide generic access to flash controller devices. Flash
controller devices are typically used in digital cameras".

Whether or not higher level controls should be part of the same class is a
valid question. The controls intended to expose certain frames with flash
are quite different from those used to control the low-level flash chip: the
user is fully responsible for timing and the flash sequence.

For higher level controls that could be implemented using the low-level
controls for the end user, the user would likely prefer to say things like
"please give me a frame exposed with flash". Since the timing is no longer
implemented by the user, the user would need to know which frames have been
exposed and how, at least in a general case. Getting around this could
involve configuring the sensor before starting streaming. Perhaps this is an
assumption we could accept now, before we have proper means for passing
frame-related parameters to user space.

> >Having studied multiple Samsung sensor drivers in Android kernels, it
> >seems all of them basically expose 4 flash modes:
> >
> >1. The flash strobe is off (the flash is always off)
> >2. The flash strobe is on (the flash is fired with each snapshot,
> >    exact timing is determined in the camera firmware)
> >3. The flash strobe is controlled automatically by the sensor
> >    (the flash may get fired or not, depending on light conditions, etc.)
> >4. The flash is on continuously (torch function)
> >
> >There is even a dedicated common control in the private header for samsung
> >devices [1] - V4L2_CID_CAMERA_FLASH_MODE.
> >
> >Having studied the existing flash controls [1] I'm wondering whether
> >a control like
> >
> >V4L2_CID_FLASH_HW_STROBE_AUTO
> >
> >with options:
> >	V4L2_FLASH_HW_STROBE_OFF
> >	V4L2_FLASH_HW_STROBE_AUTO
> >	V4L2_FLASH_HW_STROBE_ON
> 
> The _HW prefix probably needs to be removed there.

V4L2_CID_FLASH_STROBE_AUTO sounds good to me as such.

Do you always need to streamoff first, after which these the sensors perform
a single capture with flash enabled, or how does it work? How about the
subsequent captures; will they be done with flash enabled as well (requiring
a possible flash cool-off time) or will the flash be disabled for them?

For something more complex we'd require something else than a control
interface; possibly a private IOCTL to set frame-specific flash parameters.

Albeit I think that this control would be very different from what the rest
of the controls in the class do, I don't see a better place for it either. I
wouldn't mind hearing others' opinions, though.

> >would make sense ? It would cover above cases 1, 3, 2.
> >
> >Then I'm not sure what to do exactly with the Torch option.
> >
> >Perhaps a boolean control like V4L2_FLASH_TORCH_ENABLED would do ?
> 
> OTOH it seems V4L2_CID_FLASH_LED_MODE and V4L2_CID_FLASH_TORCH_INTENSITY
> could do the same. V4L2_CID_FLASH_LED_MODE would be switching between
> Flash and Torch function and V4L2_CID_FLASH_TORCH_INTENSITY would be
> turning flash LED on/off.

You don't even need to set torch intensity. I'd suppose this is maximum by
default, or may not be controllable in the first place. All you need to do
to turn the flash off is to set the V4L2_CID_FLASH_LED_MODE to
V4L2_FLASH_LED_MODE_NONE. If you want torch, set it to
V4L2_FLASH_LED_MODE_TORCH.

I wonder if TC_FLASH_CONT_ENABLE enables torch, or something else.

What about pre-flash? :-)

Kind regards,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

Hi Sakari,

On 10/14/2012 08:30 PM, Sakari Ailus wrote:
> Currently the flash control reference states that "The V4L2 flash controls
> are intended to provide generic access to flash controller devices. Flash
> controller devices are typically used in digital cameras".
> 
> Whether or not higher level controls should be part of the same class is a
> valid question. The controls intended to expose certain frames with flash
> are quite different from those used to control the low-level flash chip: the
> user is fully responsible for timing and the flash sequence.
> 
> For higher level controls that could be implemented using the low-level
> controls for the end user, the user would likely prefer to say things like
> "please give me a frame exposed with flash". Since the timing is no longer
> implemented by the user, the user would need to know which frames have been
> exposed and how, at least in a general case. Getting around this could
> involve configuring the sensor before starting streaming. Perhaps this is an
> assumption we could accept now, before we have proper means for passing
> frame-related parameters to user space.

Yes, right. This auto strobe control seems to be a higher level one, since
we have a firmware program that is taking care of some things that normally
would be done through the existing Flash class controls by a user space
application/library.

I'm not really sure if we need a new class. It's even hard to name it.
I don't see such an auto strobe control as a high level one, from an end 
application POV. It's more like the existing controls are low level.

With V4L2 device profiles, when eventually we create them... it might not
matter if we create separate flash control class for flash controllers,
camera sensors, each one low or high level.

I agree with your remarks regarding pre-configuring a sensor. Frame meta
data could carry relevant information, but so far we don't have standard
solution for that.

>>> V4L2_CID_FLASH_HW_STROBE_AUTO
>>>
>>> with options:
>>> 	V4L2_FLASH_HW_STROBE_OFF
>>> 	V4L2_FLASH_HW_STROBE_AUTO
>>> 	V4L2_FLASH_HW_STROBE_ON
>>
>> The _HW prefix probably needs to be removed there.
> 
> V4L2_CID_FLASH_STROBE_AUTO sounds good to me as such.
> 
> Do you always need to streamoff first, after which these the sensors perform
> a single capture with flash enabled, or how does it work? How about the
> subsequent captures; will they be done with flash enabled as well (requiring
> a possible flash cool-off time) or will the flash be disabled for them?

Currently a streamoff is needed in general. The pipeline needs to be
reconfigured for different resolution/image format. The flash should be 
fired as configured per each capture request. I imagine sensor could delay 
actual capture when the Flash is not ready, due to required cool-off/
re-charge time. Such an information would need to be provided to the sensor 
somehow, e.g. with a private ioctl. I don't have much experience yet with 
more advanced Flash circuits/controls. 

So V4L2_FLASH_STROBE_ON would mean, for each executed capture request the 
Flash is to be triggered unconditionally.

> For something more complex we'd require something else than a control
> interface; possibly a private IOCTL to set frame-specific flash parameters.

Yes, I agree with that. Per frame meta data is needed to figure out, e.g. 
which frame has been exposed with flash and which one without.

> Albeit I think that this control would be very different from what the rest
> of the controls in the class do, I don't see a better place for it either. I
> wouldn't mind hearing others' opinions, though.

Maybe we could consider just documenting this kind of controls in a separate 
section ?

>> OTOH it seems V4L2_CID_FLASH_LED_MODE and V4L2_CID_FLASH_TORCH_INTENSITY
>> could do the same. V4L2_CID_FLASH_LED_MODE would be switching between
>> Flash and Torch function and V4L2_CID_FLASH_TORCH_INTENSITY would be
>> turning flash LED on/off.
> 
> You don't even need to set torch intensity. I'd suppose this is maximum by
> default, or may not be controllable in the first place. All you need to do
> to turn the flash off is to set the V4L2_CID_FLASH_LED_MODE to
> V4L2_FLASH_LED_MODE_NONE. If you want torch, set it to
> V4L2_FLASH_LED_MODE_TORCH.

Indeed, thanks for opening my eyes. :)

> I wonder if TC_FLASH_CONT_ENABLE enables torch, or something else.

Yes, it just enables the LED permanently.

> What about pre-flash? :-)

Good question, I think an interface to adjust this feature is needed as well.
So parameters like pre-flash count, time period between pre-and main flash 
strobe, or pre-flash status can be configured/retrieved. My knowledge is 
rather limited on that topic yet though.

--
Regards,
Sylwester

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

On Tue, Oct 16, 2012 at 11:45:59PM +0200, Sylwester Nawrocki wrote:
> On 10/14/2012 08:30 PM, Sakari Ailus wrote:
> > Currently the flash control reference states that "The V4L2 flash controls
> > are intended to provide generic access to flash controller devices. Flash
> > controller devices are typically used in digital cameras".
> > 
> > Whether or not higher level controls should be part of the same class is a
> > valid question. The controls intended to expose certain frames with flash
> > are quite different from those used to control the low-level flash chip: the
> > user is fully responsible for timing and the flash sequence.
> > 
> > For higher level controls that could be implemented using the low-level
> > controls for the end user, the user would likely prefer to say things like
> > "please give me a frame exposed with flash". Since the timing is no longer
> > implemented by the user, the user would need to know which frames have been
> > exposed and how, at least in a general case. Getting around this could
> > involve configuring the sensor before starting streaming. Perhaps this is an
> > assumption we could accept now, before we have proper means for passing
> > frame-related parameters to user space.
> 
> Yes, right. This auto strobe control seems to be a higher level one, since
> we have a firmware program that is taking care of some things that normally
> would be done through the existing Flash class controls by a user space
> application/library.
> 
> I'm not really sure if we need a new class. It's even hard to name it.
> I don't see such an auto strobe control as a high level one, from an end 
> application POV. It's more like the existing controls are low level.

After thinking about it awhile, an alternative I see to this is to put it to
the camera class. It's got other high level controls as well, those related
to e.g. AF. I have to admit I'm not certain which one would be a better
choice in the long run. I'm leaning towards the camera class, though.

Say, if you'd create low level controls for lens movement, you probably
wouldn't put them to the existing camera class. They wouldn't seem to fit
there.

Laurent, Hans, others: do you have an opinion on this?

> With V4L2 device profiles, when eventually we create them... it might not
> matter if we create separate flash control class for flash controllers,
> camera sensors, each one low or high level.
> 
> I agree with your remarks regarding pre-configuring a sensor. Frame meta
> data could carry relevant information, but so far we don't have standard
> solution for that.

Agreed. But as long as we assume there's streamoff, we don't even need one.

> >>> V4L2_CID_FLASH_HW_STROBE_AUTO
> >>>
> >>> with options:
> >>> 	V4L2_FLASH_HW_STROBE_OFF
> >>> 	V4L2_FLASH_HW_STROBE_AUTO
> >>> 	V4L2_FLASH_HW_STROBE_ON
> >>
> >> The _HW prefix probably needs to be removed there.
> > 
> > V4L2_CID_FLASH_STROBE_AUTO sounds good to me as such.
> > 
> > Do you always need to streamoff first, after which these the sensors perform
> > a single capture with flash enabled, or how does it work? How about the
> > subsequent captures; will they be done with flash enabled as well (requiring
> > a possible flash cool-off time) or will the flash be disabled for them?
> 
> Currently a streamoff is needed in general. The pipeline needs to be
> reconfigured for different resolution/image format. The flash should be 
> fired as configured per each capture request. I imagine sensor could delay 
> actual capture when the Flash is not ready, due to required cool-off/
> re-charge time. Such an information would need to be provided to the sensor 
> somehow, e.g. with a private ioctl. I don't have much experience yet with 
> more advanced Flash circuits/controls. 

The simple flash controller chips I'm familiar with don't have such fancy
features. One just has to be patient with commanding the LED. :)

If there was a sensor that can do this automatically, then why not?

> So V4L2_FLASH_STROBE_ON would mean, for each executed capture request the 
> Flash is to be triggered unconditionally.
> 
> > For something more complex we'd require something else than a control
> > interface; possibly a private IOCTL to set frame-specific flash parameters.
> 
> Yes, I agree with that. Per frame meta data is needed to figure out, e.g. 
> which frame has been exposed with flash and which one without.

This was discussed in Cambourne last year. The frame parameters could be
given with a single IOCTL (possibly private or not), and the results would
be available on a plane of a multi-plane buffer. More in the meeting notes.

<URL:http://www.digipedia.pl/usenet/thread/18550/849/>

> > Albeit I think that this control would be very different from what the rest
> > of the controls in the class do, I don't see a better place for it either. I
> > wouldn't mind hearing others' opinions, though.
> 
> Maybe we could consider just documenting this kind of controls in a separate 
> section ?
> 
> >> OTOH it seems V4L2_CID_FLASH_LED_MODE and V4L2_CID_FLASH_TORCH_INTENSITY
> >> could do the same. V4L2_CID_FLASH_LED_MODE would be switching between
> >> Flash and Torch function and V4L2_CID_FLASH_TORCH_INTENSITY would be
> >> turning flash LED on/off.
> > 
> > You don't even need to set torch intensity. I'd suppose this is maximum by
> > default, or may not be controllable in the first place. All you need to do
> > to turn the flash off is to set the V4L2_CID_FLASH_LED_MODE to
> > V4L2_FLASH_LED_MODE_NONE. If you want torch, set it to
> > V4L2_FLASH_LED_MODE_TORCH.
> 
> Indeed, thanks for opening my eyes. :)
> 
> > I wonder if TC_FLASH_CONT_ENABLE enables torch, or something else.
> 
> Yes, it just enables the LED permanently.
> 
> > What about pre-flash? :-)
> 
> Good question, I think an interface to adjust this feature is needed as well.
> So parameters like pre-flash count, time period between pre-and main flash 
> strobe, or pre-flash status can be configured/retrieved. My knowledge is 
> rather limited on that topic yet though.

Same for me. I somehow feel we're not binding ourselves if we implement such
a control without considering pre-flash. I reckon pre-flash is often done
with a different sensor configuration (output resolution and frame rate).

Kind regards,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sylwester Nawrocki]

Hi Sakari,

On 10/23/2012 10:07 PM, Sakari Ailus wrote:
> On Tue, Oct 16, 2012 at 11:45:59PM +0200, Sylwester Nawrocki wrote:
>> On 10/14/2012 08:30 PM, Sakari Ailus wrote:
>>> Currently the flash control reference states that "The V4L2 flash controls
>>> are intended to provide generic access to flash controller devices. Flash
>>> controller devices are typically used in digital cameras".
>>>
>>> Whether or not higher level controls should be part of the same class is a
>>> valid question. The controls intended to expose certain frames with flash
>>> are quite different from those used to control the low-level flash chip: the
>>> user is fully responsible for timing and the flash sequence.
>>>
>>> For higher level controls that could be implemented using the low-level
>>> controls for the end user, the user would likely prefer to say things like
>>> "please give me a frame exposed with flash". Since the timing is no longer
>>> implemented by the user, the user would need to know which frames have been
>>> exposed and how, at least in a general case. Getting around this could
>>> involve configuring the sensor before starting streaming. Perhaps this is an
>>> assumption we could accept now, before we have proper means for passing
>>> frame-related parameters to user space.
>>
>> Yes, right. This auto strobe control seems to be a higher level one, since
>> we have a firmware program that is taking care of some things that normally
>> would be done through the existing Flash class controls by a user space
>> application/library.
>>
>> I'm not really sure if we need a new class. It's even hard to name it.
>> I don't see such an auto strobe control as a high level one, from an end 
>> application POV. It's more like the existing controls are low level.
> 
> After thinking about it awhile, an alternative I see to this is to put it to
> the camera class. It's got other high level controls as well, those related
> to e.g. AF. I have to admit I'm not certain which one would be a better
> choice in the long run. I'm leaning towards the camera class, though.

At first I thought it might be fine to put this control in the camera class.
But didn't we agree we classify controls by functionality, not by where
they are used ?
Since we already have a Flash controller functionality class it seems to me
more correct, or less wrong, to put V4L2_CID_FLASH_STROBE_AUTO there.

In an example configuration, where there is a Flash controller subdev and
the Flash controller is strobed in hardware by a camera sensor module, we
would have some Flash functionality control at the camera sensor subdev
and the Flash subdev. Let's focus on the camera subdev for a moment. 
It would have V4L2_CID_FLASH_LED_MODE - to switch between Flash and Torch
mode, and V4L2_CID_FLASH_STROBE_AUTO - to determine the flash strobing 
behaviour. It would look fishy to have one of these controls in the Camera 
class and the other in the Flash class. I would find it hard to explain
to someone new learning about v4l2.

> Say, if you'd create low level controls for lens movement, you probably
> wouldn't put them to the existing camera class. They wouldn't seem to fit
> there.
> 
> Laurent, Hans, others: do you have an opinion on this?
> 
>> With V4L2 device profiles, when eventually we create them... it might not
>> matter if we create separate flash control class for flash controllers,
>> camera sensors, each one low or high level.
>>
>> I agree with your remarks regarding pre-configuring a sensor. Frame meta
>> data could carry relevant information, but so far we don't have standard
>> solution for that.
> 
> Agreed. But as long as we assume there's streamoff, we don't even need one.
> 
>>>>> V4L2_CID_FLASH_HW_STROBE_AUTO
>>>>>
>>>>> with options:
>>>>> 	V4L2_FLASH_HW_STROBE_OFF
>>>>> 	V4L2_FLASH_HW_STROBE_AUTO
>>>>> 	V4L2_FLASH_HW_STROBE_ON
>>>>
>>>> The _HW prefix probably needs to be removed there.
>>>
>>> V4L2_CID_FLASH_STROBE_AUTO sounds good to me as such.
>>>
>>> Do you always need to streamoff first, after which these the sensors perform
>>> a single capture with flash enabled, or how does it work? How about the
>>> subsequent captures; will they be done with flash enabled as well (requiring
>>> a possible flash cool-off time) or will the flash be disabled for them?
>>
>> Currently a streamoff is needed in general. The pipeline needs to be
>> reconfigured for different resolution/image format. The flash should be 
>> fired as configured per each capture request. I imagine sensor could delay 
>> actual capture when the Flash is not ready, due to required cool-off/
>> re-charge time. Such an information would need to be provided to the sensor 
>> somehow, e.g. with a private ioctl. I don't have much experience yet with 
>> more advanced Flash circuits/controls. 
> 
> The simple flash controller chips I'm familiar with don't have such fancy
> features. One just has to be patient with commanding the LED. :)
> 
> If there was a sensor that can do this automatically, then why not?
> 
>> So V4L2_FLASH_STROBE_ON would mean, for each executed capture request the 
>> Flash is to be triggered unconditionally.
>>
>>> For something more complex we'd require something else than a control
>>> interface; possibly a private IOCTL to set frame-specific flash parameters.
>>
>> Yes, I agree with that. Per frame meta data is needed to figure out, e.g. 
>> which frame has been exposed with flash and which one without.
> 
> This was discussed in Cambourne last year. The frame parameters could be
> given with a single IOCTL (possibly private or not), and the results would
> be available on a plane of a multi-plane buffer. More in the meeting notes.
> 
> <URL:http://www.digipedia.pl/usenet/thread/18550/849/>

Thanks. I was thinking about adding support in drivers for an application 
request to allocate one more plane in addition to what implies a particular 
fourcc, for capturing meta-data. Then, e.g. a camera specific plugin to the
v4l2 library would implement standard callbacks that would return specific 
information based on the meta data.
We will likely need something like this for the s5c73m3 type of sensors.
If they are to be used by standard applications then some device specific
library must be present in user space.

>>> Albeit I think that this control would be very different from what the rest
>>> of the controls in the class do, I don't see a better place for it either. I
>>> wouldn't mind hearing others' opinions, though.
>>
>> Maybe we could consider just documenting this kind of controls in a separate 
>> section ?
>>
>>>> OTOH it seems V4L2_CID_FLASH_LED_MODE and V4L2_CID_FLASH_TORCH_INTENSITY
>>>> could do the same. V4L2_CID_FLASH_LED_MODE would be switching between
>>>> Flash and Torch function and V4L2_CID_FLASH_TORCH_INTENSITY would be
>>>> turning flash LED on/off.
>>>
>>> You don't even need to set torch intensity. I'd suppose this is maximum by
>>> default, or may not be controllable in the first place. All you need to do
>>> to turn the flash off is to set the V4L2_CID_FLASH_LED_MODE to
>>> V4L2_FLASH_LED_MODE_NONE. If you want torch, set it to
>>> V4L2_FLASH_LED_MODE_TORCH.
>>
>> Indeed, thanks for opening my eyes. :)
>>
>>> I wonder if TC_FLASH_CONT_ENABLE enables torch, or something else.
>>
>> Yes, it just enables the LED permanently.
>>
>>> What about pre-flash? :-)
>>
>> Good question, I think an interface to adjust this feature is needed as well.
>> So parameters like pre-flash count, time period between pre-and main flash 
>> strobe, or pre-flash status can be configured/retrieved. My knowledge is 
>> rather limited on that topic yet though.
> 
> Same for me. I somehow feel we're not binding ourselves if we implement such
> a control without considering pre-flash. I reckon pre-flash is often done
> with a different sensor configuration (output resolution and frame rate).

--
Regards,
Sylwester

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Sakari Ailus]

Hi Sylwester,

On Wed, Oct 24, 2012 at 11:04:11AM +0200, Sylwester Nawrocki wrote:
> On 10/23/2012 10:07 PM, Sakari Ailus wrote:
> > On Tue, Oct 16, 2012 at 11:45:59PM +0200, Sylwester Nawrocki wrote:
> >> On 10/14/2012 08:30 PM, Sakari Ailus wrote:
> >>> Currently the flash control reference states that "The V4L2 flash controls
> >>> are intended to provide generic access to flash controller devices. Flash
> >>> controller devices are typically used in digital cameras".
> >>>
> >>> Whether or not higher level controls should be part of the same class is a
> >>> valid question. The controls intended to expose certain frames with flash
> >>> are quite different from those used to control the low-level flash chip: the
> >>> user is fully responsible for timing and the flash sequence.
> >>>
> >>> For higher level controls that could be implemented using the low-level
> >>> controls for the end user, the user would likely prefer to say things like
> >>> "please give me a frame exposed with flash". Since the timing is no longer
> >>> implemented by the user, the user would need to know which frames have been
> >>> exposed and how, at least in a general case. Getting around this could
> >>> involve configuring the sensor before starting streaming. Perhaps this is an
> >>> assumption we could accept now, before we have proper means for passing
> >>> frame-related parameters to user space.
> >>
> >> Yes, right. This auto strobe control seems to be a higher level one, since
> >> we have a firmware program that is taking care of some things that normally
> >> would be done through the existing Flash class controls by a user space
> >> application/library.
> >>
> >> I'm not really sure if we need a new class. It's even hard to name it.
> >> I don't see such an auto strobe control as a high level one, from an end 
> >> application POV. It's more like the existing controls are low level.
> > 
> > After thinking about it awhile, an alternative I see to this is to put it to
> > the camera class. It's got other high level controls as well, those related
> > to e.g. AF. I have to admit I'm not certain which one would be a better
> > choice in the long run. I'm leaning towards the camera class, though.
> 
> At first I thought it might be fine to put this control in the camera class.
> But didn't we agree we classify controls by functionality, not by where
> they are used ?
> Since we already have a Flash controller functionality class it seems to me
> more correct, or less wrong, to put V4L2_CID_FLASH_STROBE_AUTO there.
> 
> In an example configuration, where there is a Flash controller subdev and
> the Flash controller is strobed in hardware by a camera sensor module, we
> would have some Flash functionality control at the camera sensor subdev
> and the Flash subdev. Let's focus on the camera subdev for a moment. 
> It would have V4L2_CID_FLASH_LED_MODE - to switch between Flash and Torch
> mode, and V4L2_CID_FLASH_STROBE_AUTO - to determine the flash strobing 
> behaviour. It would look fishy to have one of these controls in the Camera 
> class and the other in the Flash class. I would find it hard to explain
> to someone new learning about v4l2.

Good points. The flash mode control is still used even if the flash timing
would be handles by the sensor silently.

I'm fine with putting it to the flash class.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC/PATCH] v4l: Add V4L2_CID_FLASH_HW_STROBE_MODE control

[Laurent Pinchart]

Hi Sakari,

On Tuesday 13 November 2012 23:53:46 Sakari Ailus wrote:
> On Wed, Oct 24, 2012 at 11:04:11AM +0200, Sylwester Nawrocki wrote:
> > On 10/23/2012 10:07 PM, Sakari Ailus wrote:
> > > On Tue, Oct 16, 2012 at 11:45:59PM +0200, Sylwester Nawrocki wrote:
> > >> On 10/14/2012 08:30 PM, Sakari Ailus wrote:
> > >>> Currently the flash control reference states that "The V4L2 flash
> > >>> controls are intended to provide generic access to flash controller
> > >>> devices. Flash controller devices are typically used in digital
> > >>> cameras".
> > >>> 
> > >>> Whether or not higher level controls should be part of the same class
> > >>> is a valid question. The controls intended to expose certain frames
> > >>> with flash are quite different from those used to control the low-
> > >>> level flash chip: the user is fully responsible for timing and the
> > >>> flash sequence.
> > >>> 
> > >>> For higher level controls that could be implemented using the
> > >>> low-level controls for the end user, the user would likely prefer to
> > >>> say things like "please give me a frame exposed with flash". Since the
> > >>> timing is no longer implemented by the user, the user would need to
> > >>> know which frames have been exposed and how, at least in a general
> > >>> case. Getting around this could involve configuring the sensor before
> > >>> starting streaming. Perhaps this is an assumption we could accept now,
> > >>> before we have proper means for passing frame-related parameters to
> > >>> user space.
> > >> 
> > >> Yes, right. This auto strobe control seems to be a higher level one,
> > >> since we have a firmware program that is taking care of some things
> > >> that normally would be done through the existing Flash class controls
> > >> by a user space application/library.
> > >> 
> > >> I'm not really sure if we need a new class. It's even hard to name it.
> > >> I don't see such an auto strobe control as a high level one, from an
> > >> end application POV. It's more like the existing controls are low
> > >> level.
> > > 
> > > After thinking about it awhile, an alternative I see to this is to put
> > > it to the camera class. It's got other high level controls as well,
> > > those related to e.g. AF. I have to admit I'm not certain which one
> > > would be a better choice in the long run. I'm leaning towards the
> > > camera class, though.
> >
> > At first I thought it might be fine to put this control in the camera
> > class. But didn't we agree we classify controls by functionality, not by
> > where they are used ?
> > Since we already have a Flash controller functionality class it seems to
> > me more correct, or less wrong, to put V4L2_CID_FLASH_STROBE_AUTO there.
> > 
> > In an example configuration, where there is a Flash controller subdev and
> > the Flash controller is strobed in hardware by a camera sensor module, we
> > would have some Flash functionality control at the camera sensor subdev
> > and the Flash subdev. Let's focus on the camera subdev for a moment.
> > It would have V4L2_CID_FLASH_LED_MODE - to switch between Flash and Torch
> > mode, and V4L2_CID_FLASH_STROBE_AUTO - to determine the flash strobing
> > behaviour. It would look fishy to have one of these controls in the Camera
> > class and the other in the Flash class. I would find it hard to explain
> > to someone new learning about v4l2.
> 
> Good points. The flash mode control is still used even if the flash timing
> would be handles by the sensor silently.
> 
> I'm fine with putting it to the flash class.

I agree with that as well. We should update the flash control class 
documentation to explain that flash controls apply to both flash controller 
devices and to devices that control the flash controller such as sensors (with 
a better wording than that of course :-)).

-- 
Regards,

Laurent Pinchart