Creating a V4L driver for a USB camera

Creating a V4L driver for a USB camera

[Erik de Castro Lopo]

Hi all,

I'm a senior software engineer [0] with a small startup. Our product
is Linux based and makes use of a 3M pixel camera. Unfortunately, the
camera we have been using for the last 3 years is no longer being
produced.

We have found two candidate replacement cameras, one with a binary
only driver and user space library and one with a windows driver
but no Linux driver.

My questions:

 - How difficult is it to create a GPL V4L driver for a USB camera
   by snooping the USB traffic of the device when connected to
   a windows machine? The intention is to merge this work into
   the V4L mainline and ultimately the kernel.

 - How much work is involved in the above for someone experienced
   in writing V4L drivers?

 - Are there people involved with the V4L project that would be
   willing to undertake this project under contract?

Obviously, answers to the last question can be send directly to
me :-). The others are probably best sent to the list.

Cheers,
Erik

[0] I am also a FOSS developer. My main two projects are
       http://www.mega-nerd.com/libsndfile/
       http://www.mega-nerd.com/libsamplerate/
    I am also involved with Xiph.org and the Debian project.
-- 
=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Re: Creating a V4L driver for a USB camera

[Erik Andrén]

Hi,

2009/6/3 Erik de Castro Lopo <erik@bcode.com>:
> Hi all,
>
> I'm a senior software engineer [0] with a small startup. Our product
> is Linux based and makes use of a 3M pixel camera. Unfortunately, the
> camera we have been using for the last 3 years is no longer being
> produced.
>
> We have found two candidate replacement cameras, one with a binary
> only driver and user space library and one with a windows driver
> but no Linux driver.
>
> My questions:
>
>  - How difficult is it to create a GPL V4L driver for a USB camera
>   by snooping the USB traffic of the device when connected to
>   a windows machine? The intention is to merge this work into
>   the V4L mainline and ultimately the kernel.

Do you have any datasheet available on what usb bridge / sensor that is used?
If the chipsets are undocumented and some proprietary image
compression technique is used, the time to reverse-engineer them can
be quite lengthy.

Best regards,
Erik

Re: Creating a V4L driver for a USB camera

[Hans de Goede]

Hi,

On 06/03/2009 06:13 AM, Erik de Castro Lopo wrote:
> Hi all,
>
> I'm a senior software engineer [0] with a small startup. Our product
> is Linux based and makes use of a 3M pixel camera. Unfortunately, the
> camera we have been using for the last 3 years is no longer being
> produced.
>
> We have found two candidate replacement cameras, one with a binary
> only driver and user space library and one with a windows driver
> but no Linux driver.
>
> My questions:
>
>   - How difficult is it to create a GPL V4L driver for a USB camera
>     by snooping the USB traffic of the device when connected to
>     a windows machine? The intention is to merge this work into
>     the V4L mainline and ultimately the kernel.
>

That depends mainly on the format of the image data by the cam,
if the cam sends raw bayer data, or raw yuv / rgb then this is doable,
if it uses plain JPEG it is also doable. If it uses some custom
compression then you need a wizzkid to crack the code. I've tried
this myself, and I failed, you really need someone with the right
mindset to reverse engineer a compression algorithm. Merely being
a good programmer is not enough.

>   - How much work is involved in the above for someone experienced
>     in writing V4L drivers?

This can vary wildly, assuming the video data format is a known one,
a wild estimate would be that this takes 2 fulltime weeks (with hands
on hardware access). But it could be that it takes much longer if
somehow the cam is strange (or worse, like buggy).

>   - Are there people involved with the V4L project that would be
>     willing to undertake this project under contract?
>

Your welcome to send me a couple of cams, that is usually all the
payment I expect, I also don't make any promises (I do this on top
of my dayjob). But first things first, what are the usb-id's of
the cams, can you send me (offlist) the windows drivers ? Chances are
the chipset used is already supported.

Regards,

Hans

Re: Creating a V4L driver for a USB camera

[Erik de Castro Lopo]

On Wed, 3 Jun 2009 16:18:33 +1000
Erik Andrén <erik.andren@gmail.com> wrote:

> Do you have any datasheet available on what usb bridge / sensor that is used?

The USB device itself comes up as :

    Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc

The sensor is a Micron MT9T001P12STC and I have the data sheet for it.

I've asked the manufacturer for source code to the windows driver
and docs/source/whatever for the USB interface.

> If the chipsets are undocumented and some proprietary image
> compression technique is used, the time to reverse-engineer them can
> be quite lengthy.

I happen to know that the sensor/camera (via the windows driver) can
provide raw bayer data which is what I'm after (our application is
machine vision and bayer works best).

Cheers,
Erik
-- 
=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Re: Creating a V4L driver for a USB camera

[Theodore Kilgore]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1584 bytes --]



On Thu, 4 Jun 2009, Erik de Castro Lopo wrote:

> On Wed, 3 Jun 2009 16:18:33 +1000
> Erik Andrén <erik.andren@gmail.com> wrote:
>
>> Do you have any datasheet available on what usb bridge / sensor that is used?
>
> The USB device itself comes up as :
>
>    Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc
>
> The sensor is a Micron MT9T001P12STC and I have the data sheet for it.
>
> I've asked the manufacturer for source code to the windows driver
> and docs/source/whatever for the USB interface.
>
>> If the chipsets are undocumented and some proprietary image
>> compression technique is used, the time to reverse-engineer them can
>> be quite lengthy.
>
> I happen to know that the sensor/camera (via the windows driver) can
> provide raw bayer data which is what I'm after (our application is
> machine vision and bayer works best).
>
> Cheers,
> Erik

If this is the case, then it ought not to be terribly difficult to write a 
basic driver. If you wanted still camera support, with which I have a bit 
more experience than with streaming support, I would say it would take me 
about a week or so. The real obstacle is proprietary data compression 
which is usually totally undocumented, as others have already said.

Of course, I said above "basic" driver. That does not include things like 
color balance, contrast, or brightness controls. Such would probably take 
a little bit longer.

>From what I read here, I think you will have good luck if you follow 
through with this project.


Theodore Kilgore

(Greetings from the monthly meeting of the East Alabama LUG)

Re: Creating a V4L driver for a USB camera

[Erik de Castro Lopo]

On Thu, 4 Jun 2009 11:28:38 +1000
Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote:

> If this is the case, then it ought not to be terribly difficult to write a 
> basic driver. If you wanted still camera support, with which I have a bit 
> more experience than with streaming support,

Yep, only interested in still images ATM.

> Of course, I said above "basic" driver. That does not include things like 
> color balance, contrast, or brightness controls. Such would probably take 
> a little bit longer.

Als need contol over things like this. We have pretty good control
over the lighting the camera works under so we tweak contrast/brightness/
whatever in the camera to provide the bext possible image to the image
processing.

Cheers,
Erik
-- 
=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Re: Creating a V4L driver for a USB camera

[Theodore Kilgore]

On Thu, 4 Jun 2009, Erik de Castro Lopo wrote:

> On Thu, 4 Jun 2009 11:28:38 +1000
> Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote:
>
>> If this is the case, then it ought not to be terribly difficult to write a
>> basic driver. If you wanted still camera support, with which I have a bit
>> more experience than with streaming support,
>
> Yep, only interested in still images ATM.
>
>> Of course, I said above "basic" driver. That does not include things like
>> color balance, contrast, or brightness controls. Such would probably take
>> a little bit longer.
>
> Als need contol over things like this. We have pretty good control
> over the lighting the camera works under so we tweak contrast/brightness/
> whatever in the camera to provide the bext possible image to the image
> processing.

Well, if you are interested in using the camera as a still camera, then 
probably you ought also to send an inquiry over to

gphoto-devel@lists.sourceforge.net

because that is, basically, where the still camera support is done, not 
here. At the same time, though, you have gotten some responses here from 
some pretty clever folks who have volunteered to help. So if I were you I 
would certainly not want to drop those offers into the sea.

The reason for the distinction between V4L  type of things and Gphoto type 
of things is the old and time-honored rigid distinction between 
kernelspace and userspace in Linux. Basically, the idea is that a video 
device or a streaming camera (webcam) requires a kernel device to be 
created, usually named /dev/videoX where X is a number, and if you have 
just one such device hooked up then X=0, of course. But userspace 
applications do not require that, and a device can be communicated with 
through libusb, for example, or even can be communicated with directly.

So, Gphoto is a project which supports still cameras, using libusb as its 
infrastructure. It was conceived and intended to support proprietary 
cameras, which do not use any standard communication protocol. More 
recently, it has also evolved into supporting the cameras which use the 
PTP protocol. If your camera is proprietary or uses PTP and you want to 
get still photos off it, then Gphoto is the way to go. If it can also 
stream and you want _that_ supported, then here is the right place.

Many still cameras, though, are not using a proprietary or a PTP 
communication protocol, but instead are using Mass Storage, usually with 
protocol Transparent SCSI, and with Bulk Transport. Such still cameras do 
not in general need any particular driver software at all. Rather, one 
gets access to such a camera by mounting it. For, as a USB device it acts 
just like any USB external hard drive or flash drive and the photos on it 
are files ready to download.

Unfortunately, if you have mentioned what are the characteristics of your 
camera as a USB device, then that mention did not register on me. What 
might really help in getting started (whichever way this is going) is the 
full output of lsusb. Here, for example, is part of the output from a 
camera, from lsusb -v:

Bus 005 Device 002: ID 093a:010f Pixart Imaging, Inc. Argus 
DC-1610/DC-1620/Emprex PCD3600/Philips P44417B keychain camera/Precision 
Mini,Model HA513A/Vivitar Vivicam 55
Device Descriptor:
   bLength                18
   bDescriptorType         1
   bcdUSB               1.10
   bDeviceClass          255 Vendor Specific Class
   bDeviceSubClass       255 Vendor Specific Subclass
   bDeviceProtocol       255 Vendor Specific Protocol
   bMaxPacketSize0         8
   idVendor           0x093a Pixart Imaging, Inc.
   idProduct          0x010f Argus DC-1610/DC-1620/Emprex PCD3600/Philips 
P44417B keychain camera/Precision Mini,Model HA513A/Vivitar Vivicam 55
   bcdDevice            1.00
   iManufacturer           0
   iProduct                2 USB Dual-mode Camera

Note in particular these lines

   bDeviceClass          255 Vendor Specific Class
   bDeviceSubClass       255 Vendor Specific Subclass
   bDeviceProtocol       255 Vendor Specific Protocol

which tell that, whatever else, this camera needs to have a driver 
provided or else one can do nothing at all with it. It does have a 
stillcam driver, in fact, in libgphoto2. A webcam driver for it is under 
development, too, over here, since the camera is "Dual-mode" and can be 
used both as a still camera and as a webcam.

Now, here is another camera:

Bus 005 Device 003: ID 2770:915d NHJ, Ltd Cyberpix S-210S / Little Tikes 
My Real Digital Camera
Device Descriptor:
   bLength                18
   bDescriptorType         1
   bcdUSB               1.10
   bDeviceClass            0 (Defined at Interface level)
   bDeviceSubClass         0
   bDeviceProtocol         0
   bMaxPacketSize0         8
   idVendor           0x2770 NHJ, Ltd
   idProduct          0x915d Cyberpix S-210S / Little Tikes My Real Digital 
Camera

So, the note says "Defined at Interface level" so we can scroll down a bit 
and we see

       bInterfaceClass         8 Mass Storage
       bInterfaceSubClass      6 SCSI
       bInterfaceProtocol     80 Bulk (Zip)

so in other words, to get access to this camera one must mount it. Period. 
No particular additional support is needed for this second camera, and, as 
a matter of fact, no additional support is possible, either, neither from 
the kernel nor from a project like libgphoto.

So the reason for these examples is, it is this kind of information which 
one must use to judge the situation, and it is this kind of information 
which one would have to provide in order to begin to get definitive 
answers. This, in fact, is where one must start.

Hope this helps.

Theodore Kilgore

Re: Creating a V4L driver for a USB camera

[Erik de Castro Lopo]

On Thu, 4 Jun 2009 14:02:37 +1000
Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote:

> Well, if you are interested in using the camera as a still camera, then 
> probably you ought also to send an inquiry over to
> 
> gphoto-devel@lists.sourceforge.net
> 
> because that is, basically, where the still camera support is done, not 
> here.

Well our current camera has a V4L based driver so we'd like to stick
to that :-).

> So the reason for these examples is, it is this kind of information which 
> one must use to judge the situation, and it is this kind of information 
> which one would have to provide in order to begin to get definitive 
> answers. This, in fact, is where one must start.

Ok, to the lsusb -v info:

    Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc. 
    Device Descriptor:
      bLength                18
      bDescriptorType         1
      bcdUSB               2.00
      bDeviceClass            0 (Defined at Interface level)
      bDeviceSubClass         0 
      bDeviceProtocol         0 
      bMaxPacketSize0        64
      idVendor           0x0547 Anchor Chips, Inc.
      idProduct          0x8031 
      bcdDevice            0.00
      iManufacturer           1 
      iProduct                2 
      iSerial                 0 
      bNumConfigurations      1
      Configuration Descriptor:
        bLength                 9
        bDescriptorType         2
        wTotalLength           32
        bNumInterfaces          1
        bConfigurationValue     1
        iConfiguration          0 
        bmAttributes         0x80
          (Bus Powered)
        MaxPower              100mA
        Interface Descriptor:
          bLength                 9
          bDescriptorType         4
          bInterfaceNumber        0
          bAlternateSetting       0
          bNumEndpoints           2
          bInterfaceClass       255 Vendor Specific Class
          bInterfaceSubClass      0 
          bInterfaceProtocol      0 
          iInterface              0 
          Endpoint Descriptor:
            bLength                 7
            bDescriptorType         5
            bEndpointAddress     0x81  EP 1 IN
            bmAttributes            3
              Transfer Type            Interrupt
              Synch Type               None
              Usage Type               Data
            wMaxPacketSize     0x0004  1x 4 bytes
            bInterval               0
          Endpoint Descriptor:
            bLength                 7
            bDescriptorType         5
            bEndpointAddress     0x82  EP 2 IN
            bmAttributes            2
              Transfer Type            Bulk
              Synch Type               None
              Usage Type               Data
            wMaxPacketSize     0x0200  1x 512 bytes
            bInterval               0


The "Vendor Specific Class" above suggests that this camera does not
behave like a proper USV video or still camera, but rather uses its
own protocol (just like the camera we are replacing).

I have managed to convince the manufactuer of the fact that its a
good idea to provide some information and/or windows source code,
but as yet I can't predict how good that information will be.

Erik
-- 
=======================
erik de castro lopo
senior design engineer

bCODE
level 2, 2a glen street
milsons point
sydney nsw 2061
australia

tel +61 (0)2 9954 4411
fax +61 (0)2 9954 4422
www.bcode.com

Re: Creating a V4L driver for a USB camera

[Theodore Kilgore]

On Thu, 4 Jun 2009, Erik de Castro Lopo wrote:

> On Thu, 4 Jun 2009 14:02:37 +1000
> Theodore Kilgore <kilgota@banach.math.auburn.edu> wrote:
>
>> Well, if you are interested in using the camera as a still camera, then
>> probably you ought also to send an inquiry over to
>>
>> gphoto-devel@lists.sourceforge.net
>>
>> because that is, basically, where the still camera support is done, not
>> here.
>
> Well our current camera has a V4L based driver so we'd like to stick
> to that :-).

As I explained, it is a matter of what one wants to do with the camera 
which kind of driver one wants to construct. Also it is a matter of what 
exactly that the particular camera will do. To take the two extreme cases:

-- if you have a camera which will not act as a webcam, a V4L driver will 
probably not materialize.

-- if you have a webcam which has no ability to act as a still camera, 
then no Gphoto driver will materialize.

And if the camera can do both of the above, then it is possible (and it 
occurs, too) that the camera has a Gphoto driver for its still camera 
functionality and it has a V4L driver for its webcam functionality. But 
the still camera and webcam functions are conceptually different and 
require distinct methodologies to support them in Linux.

In other words, the kind of software support which is required by new 
camera X is determined by the properties of camera X alone. That was my 
point.

> Ok, to the lsusb -v info:
>
>    Bus 001 Device 011: ID 0547:8031 Anchor Chips, Inc.
>    Device Descriptor:
>      bLength                18
>      bDescriptorType         1
>      bcdUSB               2.00
>      bDeviceClass            0 (Defined at Interface level)
>      bDeviceSubClass         0
>      bDeviceProtocol         0
>      bMaxPacketSize0        64
>      idVendor           0x0547 Anchor Chips, Inc.
>      idProduct          0x8031
>      bcdDevice            0.00
>      iManufacturer           1
>      iProduct                2
>      iSerial                 0
>      bNumConfigurations      1
>      Configuration Descriptor:
>        bLength                 9
>        bDescriptorType         2
>        wTotalLength           32
>        bNumInterfaces          1
>        bConfigurationValue     1
>        iConfiguration          0
>        bmAttributes         0x80
>          (Bus Powered)
>        MaxPower              100mA
>        Interface Descriptor:
>          bLength                 9
>          bDescriptorType         4
>          bInterfaceNumber        0
>          bAlternateSetting       0
>          bNumEndpoints           2
>          bInterfaceClass       255 Vendor Specific Class
>          bInterfaceSubClass      0
>          bInterfaceProtocol      0
>          iInterface              0
>          Endpoint Descriptor:
>            bLength                 7
>            bDescriptorType         5
>            bEndpointAddress     0x81  EP 1 IN
>            bmAttributes            3
>              Transfer Type            Interrupt
>              Synch Type               None
>              Usage Type               Data
>            wMaxPacketSize     0x0004  1x 4 bytes
>            bInterval               0
>          Endpoint Descriptor:
>            bLength                 7
>            bDescriptorType         5
>            bEndpointAddress     0x82  EP 2 IN
>            bmAttributes            2
>              Transfer Type            Bulk
>              Synch Type               None
>              Usage Type               Data
>            wMaxPacketSize     0x0200  1x 512 bytes
>            bInterval               0
>
>
> The "Vendor Specific Class" above suggests that this camera does not
> behave like a proper USV video or still camera, but rather uses its
> own protocol (just like the camera we are replacing).

Yes. That seems quite clear.

>
> I have managed to convince the manufactuer of the fact that its a
> good idea to provide some information and/or windows source code,
> but as yet I can't predict how good that information will be.

If the manufacturer truly provides meaningful information, that would be 
very good. Therefore, I restrain myself from expressing general pessimism 
about the prospects.

Theodore Kilgore