Seeking guidance for Adding ISP Driver for RK3568

Seeking guidance for Adding ISP Driver for RK3568

[Yogesh Hegde]

Hi,

I am seeking your expert guidance and assistance in adding the ISP driver for 
RK3568 to the Linux Kernel.

Before I start, I have a few questions regarding the process of 
adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. 
I greatly appreciate any insights you can provide on the following points:

1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399
   has already been upstreamed, I would like to understand if the ISP driver 
   for RK3568 will be added to the existing RK3399 ISP driver or if it will be 
   a completely new driver ? 

2. Upstreaming process and staging: I would like to know if the upstreaming 
   process for the RK3568 ISP driver will involve starting with the downstream 
   driver and adding it to the staging area. From there, would it be merged 
   into the media subsystem or follow a different process altogether? 

3. Permission requirements from Rockchip: To ensure a smooth upstreaming 
   process, I would like to know if I need any permissions or approvals from 
   Rockchip, the manufacturer of the RK3568 chipset. If permissions are 
   required, could you please guide me on how to obtain them? 

I would also like to mention that while this is my first time upstreaming a 
full driver, I do have some experience working with the media subsystem. 
I am willing to devote the necessary time and effort to learn and contribute 
effectively to the upstreaming process also I have Rock 3A development board, 
which will be helpful in testing the ISP driver. 

Thank you very much for considering my request. I eagerly await your responses 
and any direction you and the mailing list members can provide.

Thanks & Regards,
Yogesh

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Seeking guidance for Adding ISP Driver for RK3568

[Michael Riesch]

Hi Yogesh,

Cc: Laurent as I had a chat with him about 1.

On 7/11/23 09:11, Yogesh Hegde wrote:
> Hi,
> 
> I am seeking your expert guidance and assistance in adding the ISP driver for 
> RK3568 to the Linux Kernel.
> 
> Before I start, I have a few questions regarding the process of 
> adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. 
> I greatly appreciate any insights you can provide on the following points:
> 
> 1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399
>    has already been upstreamed, I would like to understand if the ISP driver 
>    for RK3568 will be added to the existing RK3399 ISP driver or if it will be 
>    a completely new driver ?

That's a very good question, but unfortunately there is no clear answer
to it. We were able to hack the existing mainline rkisp1 driver so that
it works with the RK3568 ISP (rkisp v2.1), but the result is far from
being mainline material.

I would say that there is enough overlap that the RK3568 ISP code should
enter drivers/media/platform/rockchip/rkisp1/ but on the other hand the
differences are so significant that there should be separate
rkisp2-{csi,params,...}.c files.

Example for overlap: With some refactoring applied, rkisp1-capture.c
could support the RK3568 ISP as well.

Example for significant changes: the CSI subdevice is fundamentally
different, so most likely there will be a rkisp2-csi.c.

Example for a mixture of both: a new stats/params interface to user
space will be required for the RK3568 ISP, but if this is designed to be
general enough, it might be reused for the older ISP versions at some point.

> 2. Upstreaming process and staging: I would like to know if the upstreaming 
>    process for the RK3568 ISP driver will involve starting with the downstream 
>    driver and adding it to the staging area. From there, would it be merged 
>    into the media subsystem or follow a different process altogether?

I think the tendency is towards skipping staging, but I don't have a say
on that one.

> 3. Permission requirements from Rockchip: To ensure a smooth upstreaming 
>    process, I would like to know if I need any permissions or approvals from 
>    Rockchip, the manufacturer of the RK3568 chipset. If permissions are 
>    required, could you please guide me on how to obtain them?

Downstream code is GPL'ed, so it should be fine.

> I would also like to mention that while this is my first time upstreaming a 
> full driver, I do have some experience working with the media subsystem.

Cool! Just be aware that you picked a complex beast for your first full
driver.

> I am willing to devote the necessary time and effort to learn and contribute 
> effectively to the upstreaming process also I have Rock 3A development board, 
> which will be helpful in testing the ISP driver.

What camera sensor hardware do you have?

Best regards,
Michael

> 
> Thank you very much for considering my request. I eagerly await your responses 
> and any direction you and the mailing list members can provide.
> 
> Thanks & Regards,
> Yogesh
> 
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Seeking guidance for Adding ISP Driver for RK3568

[Yogesh Hegde]

Hi Michael, 
On Tue, Jul 11, 2023 at 11:29:34AM +0200, Michael Riesch wrote:
> Hi Yogesh,
> 
> Cc: Laurent as I had a chat with him about 1.
> 
> On 7/11/23 09:11, Yogesh Hegde wrote:
> > Hi,
> > 
> > I am seeking your expert guidance and assistance in adding the ISP driver for 
> > RK3568 to the Linux Kernel.
> > 
> > Before I start, I have a few questions regarding the process of 
> > adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. 
> > I greatly appreciate any insights you can provide on the following points:
> > 
> > 1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399
> >    has already been upstreamed, I would like to understand if the ISP driver 
> >    for RK3568 will be added to the existing RK3399 ISP driver or if it will be 
> >    a completely new driver ?
> 
> That's a very good question, but unfortunately there is no clear answer
> to it. We were able to hack the existing mainline rkisp1 driver so that
> it works with the RK3568 ISP (rkisp v2.1), but the result is far from
> being mainline material.
> 
> I would say that there is enough overlap that the RK3568 ISP code should
> enter drivers/media/platform/rockchip/rkisp1/ but on the other hand the
> differences are so significant that there should be separate
> rkisp2-{csi,params,...}.c files.
> 
> Example for overlap: With some refactoring applied, rkisp1-capture.c
> could support the RK3568 ISP as well.
> 
> Example for significant changes: the CSI subdevice is fundamentally
> different, so most likely there will be a rkisp2-csi.c.
> 
> Example for a mixture of both: a new stats/params interface to user
> space will be required for the RK3568 ISP, but if this is designed to be
> general enough, it might be reused for the older ISP versions at some point.
> 
> > 2. Upstreaming process and staging: I would like to know if the upstreaming 
> >    process for the RK3568 ISP driver will involve starting with the downstream 
> >    driver and adding it to the staging area. From there, would it be merged 
> >    into the media subsystem or follow a different process altogether?
> 
> I think the tendency is towards skipping staging, but I don't have a say
> on that one.
> 
> > 3. Permission requirements from Rockchip: To ensure a smooth upstreaming 
> >    process, I would like to know if I need any permissions or approvals from 
> >    Rockchip, the manufacturer of the RK3568 chipset. If permissions are 
> >    required, could you please guide me on how to obtain them?
> 
> Downstream code is GPL'ed, so it should be fine.
> 
> > I would also like to mention that while this is my first time upstreaming a 
> > full driver, I do have some experience working with the media subsystem.
> 
> Cool! Just be aware that you picked a complex beast for your first full
> driver.
> 
Thank you for taking the time to respond and provide clarity regarding the 
process.

> What camera sensor hardware do you have?

I have Raspi Cam v1.3 (OV5647) and Raspberry Pi Camera module 2 (IMX219).
From the Rock 3A schematics, the MIPI CSI port is Raspberry Pi compatible. 


Thanks and Regards
Yogesh

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip