* Memory allocation in Video4Linux
@ 2011-02-09 6:23 Wang, Wen W
2011-02-09 6:30 ` Kanigeri, Hari K
` (2 more replies)
0 siblings, 3 replies; 19+ messages in thread
From: Wang, Wen W @ 2011-02-09 6:23 UTC (permalink / raw)
To: linux-media@vger.kernel.org,
umg-meego-handset-kernel@umglistsvr.jf.intel.com
Cc: Jozef Kruger, Zhang, Xiaolin
Hi,
We are developing the image processor driver for Intel Medfield platform.
We have received some comments on memory management that we should use standard Linux kernel interfaces for this, since we are doing everything by ourselves including memory allocation (based on pages), page table management, virtual address management and etc.
So can you please help give some advice or suggestion on using standard kernel interface for memory management?
The processor has a MMU on-chip with same virtual address range as IA. The processor will access system memory (read and write) through MMU and page table. The memory consumption of the driver could be quite big especially for high resolution (14MP) with certain features turned on.
For example: advanced ISP with XNR and yuv444 output, at 14MP this uses:
1 RAW16: 2*14 = 28MB
1 qplane6: 6/4 * 14 = 21MB
1 yuv420_16: 2 * 1.5 * 14 = 42MB
1 yuv420: 1.5 * 14 = 21MB
1 yuv444: 3 * 14 = 42MB
total: 154MB.
Thanks
Wen
^ permalink raw reply [flat|nested] 19+ messages in thread* RE: Memory allocation in Video4Linux 2011-02-09 6:23 Memory allocation in Video4Linux Wang, Wen W @ 2011-02-09 6:30 ` Kanigeri, Hari K 2011-02-09 6:30 ` Yang, Jianwei 2011-02-09 10:19 ` Mauro Carvalho Chehab 2 siblings, 0 replies; 19+ messages in thread From: Kanigeri, Hari K @ 2011-02-09 6:30 UTC (permalink / raw) To: Wang, Wen W, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger > -----Original Message----- > From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On > Behalf Of Wang, Wen W > Sent: Wednesday, February 09, 2011 11:53 AM > To: linux-media@vger.kernel.org; umg-meego-handset- > kernel@umglistsvr.jf.intel.com > Cc: Jozef Kruger > Subject: [Umg-meego-handset-kernel] Memory allocation in Video4Linux > > Hi, > > We are developing the image processor driver for Intel Medfield > platform. > > We have received some comments on memory management that we should use > standard Linux kernel interfaces for this, since we are doing > everything by ourselves including memory allocation (based on pages), > page table management, virtual address management and etc. > > So can you please help give some advice or suggestion on using standard > kernel interface for memory management? Not sure if this meets your requirements, but check IOMMU driver. IOMMU driver is responsible to map the user buffers to Device's virtual address. Thank you, Best regards, Hari ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 6:23 Memory allocation in Video4Linux Wang, Wen W 2011-02-09 6:30 ` Kanigeri, Hari K @ 2011-02-09 6:30 ` Yang, Jianwei 2011-02-09 6:32 ` Wang, Wen W 2011-02-09 10:19 ` Mauro Carvalho Chehab 2 siblings, 1 reply; 19+ messages in thread From: Yang, Jianwei @ 2011-02-09 6:30 UTC (permalink / raw) To: Wang, Wen W, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger Curious for the below, why it is a sum? Will you use all of format at the same time? > 1 RAW16: 2*14 = 28MB > 1 qplane6: 6/4 * 14 = 21MB > 1 yuv420_16: 2 * 1.5 * 14 = 42MB > 1 yuv420: 1.5 * 14 = 21MB > 1 yuv444: 3 * 14 = 42MB > total: 154MB. > -----Original Message----- > From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On > Behalf Of Wang, Wen W > Sent: Wednesday, February 09, 2011 2:23 PM > To: linux-media@vger.kernel.org; umg-meego-handset- > kernel@umglistsvr.jf.intel.com > Cc: Jozef Kruger > Subject: [Umg-meego-handset-kernel] Memory allocation in Video4Linux > > Hi, > > We are developing the image processor driver for Intel Medfield platform. > > We have received some comments on memory management that we should use > standard Linux kernel interfaces for this, since we are doing everything > by ourselves including memory allocation (based on pages), page table > management, virtual address management and etc. > > So can you please help give some advice or suggestion on using standard > kernel interface for memory management? > > The processor has a MMU on-chip with same virtual address range as IA. The > processor will access system memory (read and write) through MMU and page > table. The memory consumption of the driver could be quite big especially > for high resolution (14MP) with certain features turned on. > For example: advanced ISP with XNR and yuv444 output, at 14MP this uses: > 1 RAW16: 2*14 = 28MB > 1 qplane6: 6/4 * 14 = 21MB > 1 yuv420_16: 2 * 1.5 * 14 = 42MB > 1 yuv420: 1.5 * 14 = 21MB > 1 yuv444: 3 * 14 = 42MB > total: 154MB. > > Thanks > Wen > _______________________________________________ > Umg-meego-handset-kernel mailing list > Umg-meego-handset-kernel@umglistsvr.jf.intel.com > http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 6:30 ` Yang, Jianwei @ 2011-02-09 6:32 ` Wang, Wen W 2011-02-09 6:49 ` Iyer, Sundar 0 siblings, 1 reply; 19+ messages in thread From: Wang, Wen W @ 2011-02-09 6:32 UTC (permalink / raw) To: Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger Yes. Some are internal frame and yuv444 is the output. >-----Original Message----- >From: Yang, Jianwei >Sent: 2011年2月9日 14:31 >To: Wang, Wen W; linux-media@vger.kernel.org; >umg-meego-handset-kernel@umglistsvr.jf.intel.com >Cc: Jozef Kruger >Subject: RE: Memory allocation in Video4Linux > >Curious for the below, why it is a sum? Will you use all of format at the same time? > >> 1 RAW16: 2*14 = 28MB >> 1 qplane6: 6/4 * 14 = 21MB >> 1 yuv420_16: 2 * 1.5 * 14 = 42MB >> 1 yuv420: 1.5 * 14 = 21MB >> 1 yuv444: 3 * 14 = 42MB >> total: 154MB. > >> -----Original Message----- >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On >> Behalf Of Wang, Wen W >> Sent: Wednesday, February 09, 2011 2:23 PM >> To: linux-media@vger.kernel.org; umg-meego-handset- >> kernel@umglistsvr.jf.intel.com >> Cc: Jozef Kruger >> Subject: [Umg-meego-handset-kernel] Memory allocation in Video4Linux >> >> Hi, >> >> We are developing the image processor driver for Intel Medfield platform. >> >> We have received some comments on memory management that we should use >> standard Linux kernel interfaces for this, since we are doing everything >> by ourselves including memory allocation (based on pages), page table >> management, virtual address management and etc. >> >> So can you please help give some advice or suggestion on using standard >> kernel interface for memory management? >> >> The processor has a MMU on-chip with same virtual address range as IA. The >> processor will access system memory (read and write) through MMU and page >> table. The memory consumption of the driver could be quite big especially >> for high resolution (14MP) with certain features turned on. >> For example: advanced ISP with XNR and yuv444 output, at 14MP this uses: >> 1 RAW16: 2*14 = 28MB >> 1 qplane6: 6/4 * 14 = 21MB >> 1 yuv420_16: 2 * 1.5 * 14 = 42MB >> 1 yuv420: 1.5 * 14 = 21MB >> 1 yuv444: 3 * 14 = 42MB >> total: 154MB. >> >> Thanks >> Wen >> _______________________________________________ >> Umg-meego-handset-kernel mailing list >> Umg-meego-handset-kernel@umglistsvr.jf.intel.com >> http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 6:32 ` Wang, Wen W @ 2011-02-09 6:49 ` Iyer, Sundar 2011-02-09 7:22 ` Kanigeri, Hari K 0 siblings, 1 reply; 19+ messages in thread From: Iyer, Sundar @ 2011-02-09 6:49 UTC (permalink / raw) To: Wang, Wen W, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger I remember some Continous Memory Allocator (CMA) being iterated down a few versions on some mailing lists? IIRC, it is also for large buffers and management for video IPs. Cheers! > -----Original Message----- > From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com [mailto:umg- > meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On Behalf Of Wang, Wen W > Sent: Wednesday, February 09, 2011 12:03 PM > To: Yang, Jianwei; linux-media@vger.kernel.org; umg-meego-handset- > kernel@umglistsvr.jf.intel.com > Cc: Jozef Kruger > Subject: Re: [Umg-meego-handset-kernel] Memory allocation in Video4Linux > > Yes. Some are internal frame and yuv444 is the output. > > >-----Original Message----- > >From: Yang, Jianwei > >Sent: 2011年2月9日 14:31 > >To: Wang, Wen W; linux-media@vger.kernel.org; > >umg-meego-handset-kernel@umglistsvr.jf.intel.com > >Cc: Jozef Kruger > >Subject: RE: Memory allocation in Video4Linux > > > >Curious for the below, why it is a sum? Will you use all of format at the same time? > > > >> 1 RAW16: 2*14 = 28MB > >> 1 qplane6: 6/4 * 14 = 21MB > >> 1 yuv420_16: 2 * 1.5 * 14 = 42MB > >> 1 yuv420: 1.5 * 14 = 21MB > >> 1 yuv444: 3 * 14 = 42MB > >> total: 154MB. > > > >> -----Original Message----- > >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On > >> Behalf Of Wang, Wen W > >> Sent: Wednesday, February 09, 2011 2:23 PM > >> To: linux-media@vger.kernel.org; umg-meego-handset- > >> kernel@umglistsvr.jf.intel.com > >> Cc: Jozef Kruger > >> Subject: [Umg-meego-handset-kernel] Memory allocation in Video4Linux > >> > >> Hi, > >> > >> We are developing the image processor driver for Intel Medfield platform. > >> > >> We have received some comments on memory management that we should use > >> standard Linux kernel interfaces for this, since we are doing everything > >> by ourselves including memory allocation (based on pages), page table > >> management, virtual address management and etc. > >> > >> So can you please help give some advice or suggestion on using standard > >> kernel interface for memory management? > >> > >> The processor has a MMU on-chip with same virtual address range as IA. The > >> processor will access system memory (read and write) through MMU and page > >> table. The memory consumption of the driver could be quite big especially > >> for high resolution (14MP) with certain features turned on. > >> For example: advanced ISP with XNR and yuv444 output, at 14MP this uses: > >> 1 RAW16: 2*14 = 28MB > >> 1 qplane6: 6/4 * 14 = 21MB > >> 1 yuv420_16: 2 * 1.5 * 14 = 42MB > >> 1 yuv420: 1.5 * 14 = 21MB > >> 1 yuv444: 3 * 14 = 42MB > >> total: 154MB. > >> > >> Thanks > >> Wen > >> _______________________________________________ > >> Umg-meego-handset-kernel mailing list > >> Umg-meego-handset-kernel@umglistsvr.jf.intel.com > >> http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel > _______________________________________________ > Umg-meego-handset-kernel mailing list > Umg-meego-handset-kernel@umglistsvr.jf.intel.com > http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 6:49 ` Iyer, Sundar @ 2011-02-09 7:22 ` Kanigeri, Hari K 2011-02-09 7:27 ` Wang, Wen W 0 siblings, 1 reply; 19+ messages in thread From: Kanigeri, Hari K @ 2011-02-09 7:22 UTC (permalink / raw) To: Iyer, Sundar, Wang, Wen W, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger > -----Original Message----- > From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On > Behalf Of Iyer, Sundar > Sent: Wednesday, February 09, 2011 12:20 PM > To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- > handset-kernel@umglistsvr.jf.intel.com > Cc: Jozef Kruger > Subject: Re: [Umg-meego-handset-kernel] Memory allocation in > Video4Linux > > I remember some Continous Memory Allocator (CMA) being iterated down a > few versions on > some mailing lists? IIRC, it is also for large buffers and management > for video IPs. I believe CMA is for allocating physically contiguous memory and from what Wen mentioned he also needs virtual memory management, which the IOMMU will provide. Please check the open source discussion on CMA, the last I heard CMA proposal was shot down. Reference: http://www.spinics.net/lists/linux-media/msg26875.html Wen, how are you currently allocating physical memory ? Thank you, Best regards, Hari ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 7:22 ` Kanigeri, Hari K @ 2011-02-09 7:27 ` Wang, Wen W 2011-02-09 7:51 ` Hans Verkuil 2011-02-10 7:59 ` Gao, Bin 0 siblings, 2 replies; 19+ messages in thread From: Wang, Wen W @ 2011-02-09 7:27 UTC (permalink / raw) To: Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger Hi Hari, You are right. What we need is virtual address. Currently we alloc pages (alloc_pages()) for any request. Store those pages for an image buffer into a list. We also manage the virtual address for ISP by ourself (the range from 0 to 4GB) and the page table for our MMU which is independent to system MMU page table. Thanks Wen >-----Original Message----- >From: Kanigeri, Hari K >Sent: 2011年2月9日 15:22 >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; >umg-meego-handset-kernel@umglistsvr.jf.intel.com >Cc: Jozef Kruger >Subject: RE: Memory allocation in Video4Linux > > > >> -----Original Message----- >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On >> Behalf Of Iyer, Sundar >> Sent: Wednesday, February 09, 2011 12:20 PM >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- >> handset-kernel@umglistsvr.jf.intel.com >> Cc: Jozef Kruger >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in >> Video4Linux >> >> I remember some Continous Memory Allocator (CMA) being iterated down a >> few versions on >> some mailing lists? IIRC, it is also for large buffers and management >> for video IPs. > >I believe CMA is for allocating physically contiguous memory and from what Wen >mentioned he also needs virtual memory management, which the IOMMU will >provide. Please check the open source discussion on CMA, the last I heard CMA >proposal was shot down. >Reference: http://www.spinics.net/lists/linux-media/msg26875.html > >Wen, how are you currently allocating physical memory ? > > >Thank you, >Best regards, >Hari ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-09 7:27 ` Wang, Wen W @ 2011-02-09 7:51 ` Hans Verkuil 2011-02-09 7:55 ` Wang, Wen W 2011-02-09 7:58 ` Jonghun Han 2011-02-10 7:59 ` Gao, Bin 1 sibling, 2 replies; 19+ messages in thread From: Hans Verkuil @ 2011-02-09 7:51 UTC (permalink / raw) To: Wang, Wen W, Jozef Kruger Cc: Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org On Wednesday, February 09, 2011 08:27:27 Wang, Wen W wrote: > Hi Hari, > > You are right. What we need is virtual address. > > Currently we alloc pages (alloc_pages()) for any request. Store those pages for an image buffer into a list. We also manage the virtual address for ISP by ourself (the range from 0 to 4GB) and the page table for our MMU which is independent to system MMU page table. Assuming you are using video4linux for this driver, then you should take a look at the new videobuf2 framework that will appear in 2.6.39. It is already in the media tree (http://git.linuxtv.org/media_tree.git, see include/media/videobuf2-core.h). It is much better than the old videobuf framework, and in particular gives the driver much more control on how memory is allocated and used. Regards, Hans > > Thanks > Wen > > >-----Original Message----- > >From: Kanigeri, Hari K > >Sent: 2011年2月9日 15:22 > >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; > >umg-meego-handset-kernel@umglistsvr.jf.intel.com > >Cc: Jozef Kruger > >Subject: RE: Memory allocation in Video4Linux > > > > > > > >> -----Original Message----- > >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On > >> Behalf Of Iyer, Sundar > >> Sent: Wednesday, February 09, 2011 12:20 PM > >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- > >> handset-kernel@umglistsvr.jf.intel.com > >> Cc: Jozef Kruger > >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in > >> Video4Linux > >> > >> I remember some Continous Memory Allocator (CMA) being iterated down a > >> few versions on > >> some mailing lists? IIRC, it is also for large buffers and management > >> for video IPs. > > > >I believe CMA is for allocating physically contiguous memory and from what Wen > >mentioned he also needs virtual memory management, which the IOMMU will > >provide. Please check the open source discussion on CMA, the last I heard CMA > >proposal was shot down. > >Reference: http://www.spinics.net/lists/linux-media/msg26875.html > > > >Wen, how are you currently allocating physical memory ? > > > > > >Thank you, > >Best regards, > >Hari > �翳�.n�����+%��遍荻\x17�w��.n��伐�{炳g����n�r■�����ㄨ&｛�夸z罐����zf＂������������赙z_璁\x0f�:+v�)撸� > > -- Hans Verkuil - video4linux developer - sponsored by Cisco ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 7:51 ` Hans Verkuil @ 2011-02-09 7:55 ` Wang, Wen W 2011-02-10 10:06 ` Sakari Ailus 2011-02-09 7:58 ` Jonghun Han 1 sibling, 1 reply; 19+ messages in thread From: Wang, Wen W @ 2011-02-09 7:55 UTC (permalink / raw) To: Hans Verkuil, Jozef Kruger Cc: Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org Hi Hans, Thanks for your point and videobuf2 is also what I want to use. But since our development kernel version right now is still 2.6.35, I need to find way to work with current videobuf framework. Thanks Wen >-----Original Message----- >From: Hans Verkuil [mailto:hverkuil@xs4all.nl] >Sent: 2011年2月9日 15:52 >To: Wang, Wen W; Jozef Kruger >Cc: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org >Subject: Re: Memory allocation in Video4Linux > >On Wednesday, February 09, 2011 08:27:27 Wang, Wen W wrote: >> Hi Hari, >> >> You are right. What we need is virtual address. >> >> Currently we alloc pages (alloc_pages()) for any request. Store those pages for an >image buffer into a list. We also manage the virtual address for ISP by ourself (the >range from 0 to 4GB) and the page table for our MMU which is independent to system >MMU page table. > >Assuming you are using video4linux for this driver, then you should take a >look at the new videobuf2 framework that will appear in 2.6.39. It is already >in the media tree (http://git.linuxtv.org/media_tree.git, see >include/media/videobuf2-core.h). > >It is much better than the old videobuf framework, and in particular gives >the driver much more control on how memory is allocated and used. > >Regards, > > Hans > >> >> Thanks >> Wen >> >> >-----Original Message----- >> >From: Kanigeri, Hari K >> >Sent: 2011年2月9日 15:22 >> >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; >> >umg-meego-handset-kernel@umglistsvr.jf.intel.com >> >Cc: Jozef Kruger >> >Subject: RE: Memory allocation in Video4Linux >> > >> > >> > >> >> -----Original Message----- >> >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >> >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On >> >> Behalf Of Iyer, Sundar >> >> Sent: Wednesday, February 09, 2011 12:20 PM >> >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- >> >> handset-kernel@umglistsvr.jf.intel.com >> >> Cc: Jozef Kruger >> >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in >> >> Video4Linux >> >> >> >> I remember some Continous Memory Allocator (CMA) being iterated down a >> >> few versions on >> >> some mailing lists? IIRC, it is also for large buffers and management >> >> for video IPs. >> > >> >I believe CMA is for allocating physically contiguous memory and from what Wen >> >mentioned he also needs virtual memory management, which the IOMMU will >> >provide. Please check the open source discussion on CMA, the last I heard CMA >> >proposal was shot down. >> >Reference: http://www.spinics.net/lists/linux-media/msg26875.html >> > >> >Wen, how are you currently allocating physical memory ? >> > >> > >> >Thank you, >> >Best regards, >> >Hari >> �翳�.n�����+%��遍荻\x17�w��.n��伐�{炳g����n�r■�����ㄨ&｛� >夸z罐����zf＂������������赙z_璁\x0f�:+v�)撸� >> >> > >-- >Hans Verkuil - video4linux developer - sponsored by Cisco ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-09 7:55 ` Wang, Wen W @ 2011-02-10 10:06 ` Sakari Ailus 0 siblings, 0 replies; 19+ messages in thread From: Sakari Ailus @ 2011-02-10 10:06 UTC (permalink / raw) To: Wang, Wen W Cc: Hans Verkuil, Jozef Kruger, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org Wang, Wen W wrote: > Hi Hans, > > Thanks for your point and videobuf2 is also what I want to use. > > But since our development kernel version right now is still 2.6.35, I > need to find way to work with current videobuf framework. Hi Wen, If you're bound to use that kernel version, then one option to consider might be backporting the videobuf2 to that kernel. 2.6.35 isn't so old. You'd save work by not implementing the buffer handling on your own AND also, you wouldn't later need to port the driver to use videobuf2. I would suggest not to start using the old videobuf at this point. With the current ISPs with their own MMUs it really does more harm than good. Regards, -- Sakari Ailus sakari.ailus@maxwell.research.nokia.com ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 7:51 ` Hans Verkuil 2011-02-09 7:55 ` Wang, Wen W @ 2011-02-09 7:58 ` Jonghun Han 2011-02-09 8:10 ` Wang, Wen W 1 sibling, 1 reply; 19+ messages in thread From: Jonghun Han @ 2011-02-09 7:58 UTC (permalink / raw) To: 'Hans Verkuil', 'Wang, Wen W', 'Jozef Kruger' Cc: 'Kanigeri, Hari K', 'Iyer, Sundar', 'Yang, Jianwei', linux-media, 조경호/S/W Solution개발팀(S.LSI)/E4(선임)/삼성전자 Hi, Maybe VCM is helpful for you. Please refer to the following URL. http://marc.info/?l=linux-mm&m=129255940319217&w=2 Best regards, Jonghun Han Wednesday, February 09, 2011 4:52 PM Hans Verkuil wrote: > -----Original Message----- > From: linux-media-owner@vger.kernel.org [mailto:linux-media- > owner@vger.kernel.org] On Behalf Of Hans Verkuil > Sent: Wednesday, February 09, 2011 4:52 PM > To: Wang, Wen W; Jozef Kruger > Cc: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org > Subject: Re: Memory allocation in Video4Linux > > On Wednesday, February 09, 2011 08:27:27 Wang, Wen W wrote: > > Hi Hari, > > > > You are right. What we need is virtual address. > > > > Currently we alloc pages (alloc_pages()) for any request. Store those pages for > an image buffer into a list. We also manage the virtual address for ISP by ourself > (the range from 0 to 4GB) and the page table for our MMU which is independent to > system MMU page table. > > Assuming you are using video4linux for this driver, then you should take a look at > the new videobuf2 framework that will appear in 2.6.39. It is already in the media > tree (http://git.linuxtv.org/media_tree.git, see include/media/videobuf2-core.h). > > It is much better than the old videobuf framework, and in particular gives the driver > much more control on how memory is allocated and used. > > Regards, > > Hans > > > > > Thanks > > Wen > > > > >-----Original Message----- > > >From: Kanigeri, Hari K > > >Sent: 2011年2月9日 15:22 > > >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; > > >linux-media@vger.kernel.org; > > >umg-meego-handset-kernel@umglistsvr.jf.intel.com > > >Cc: Jozef Kruger > > >Subject: RE: Memory allocation in Video4Linux > > > > > > > > > > > >> -----Original Message----- > > >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com > > >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] > > >> On Behalf Of Iyer, Sundar > > >> Sent: Wednesday, February 09, 2011 12:20 PM > > >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; > > >> umg-meego- handset-kernel@umglistsvr.jf.intel.com > > >> Cc: Jozef Kruger > > >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in > > >> Video4Linux > > >> > > >> I remember some Continous Memory Allocator (CMA) being iterated > > >> down a few versions on some mailing lists? IIRC, it is also for > > >> large buffers and management for video IPs. > > > > > >I believe CMA is for allocating physically contiguous memory and from > > >what Wen mentioned he also needs virtual memory management, which the > > >IOMMU will provide. Please check the open source discussion on CMA, > > >the last I heard CMA proposal was shot down. > > >Reference: http://www.spinics.net/lists/linux-media/msg26875.html > > > > > >Wen, how are you currently allocating physical memory ? > > > > > > > > >Thank you, > > >Best regards, > > >Hari > > 翳 .n +% 遍荻\x17 w .n 伐 {炳g n r■ ㄨ&｛ 夸z罐 zf＂ > > 赙z_璁\x0f :+v )撸 > > > > > > -- > Hans Verkuil - video4linux developer - sponsored by Cisco > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in the body > of a message to majordomo@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 7:58 ` Jonghun Han @ 2011-02-09 8:10 ` Wang, Wen W 0 siblings, 0 replies; 19+ messages in thread From: Wang, Wen W @ 2011-02-09 8:10 UTC (permalink / raw) To: Jonghun Han, 'Hans Verkuil', 'Jozef Kruger' Cc: Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, 조경호/S/W Solution개발팀(S.LSI)/E4(선임)/삼성전자 Hi Han, Thank you very much. I will check the link. Thanks Wen >-----Original Message----- >From: Jonghun Han [mailto:jonghun.han@samsung.com] >Sent: 2011年2月9日 15:58 >To: 'Hans Verkuil'; Wang, Wen W; 'Jozef Kruger' >Cc: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org; >조경호/S/W Solution개발팀(S.LSI)/E4(선임)/삼성전자 >Subject: RE: Memory allocation in Video4Linux > > >Hi, > >Maybe VCM is helpful for you. Please refer to the following URL. >http://marc.info/?l=linux-mm&m=129255940319217&w=2 > >Best regards, > > Jonghun Han > > >Wednesday, February 09, 2011 4:52 PM Hans Verkuil wrote: > >> -----Original Message----- >> From: linux-media-owner@vger.kernel.org [mailto:linux-media- >> owner@vger.kernel.org] On Behalf Of Hans Verkuil >> Sent: Wednesday, February 09, 2011 4:52 PM >> To: Wang, Wen W; Jozef Kruger >> Cc: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org >> Subject: Re: Memory allocation in Video4Linux >> >> On Wednesday, February 09, 2011 08:27:27 Wang, Wen W wrote: >> > Hi Hari, >> > >> > You are right. What we need is virtual address. >> > >> > Currently we alloc pages (alloc_pages()) for any request. Store those pages for >> an image buffer into a list. We also manage the virtual address for ISP by ourself >> (the range from 0 to 4GB) and the page table for our MMU which is independent to >> system MMU page table. >> >> Assuming you are using video4linux for this driver, then you should take a look at >> the new videobuf2 framework that will appear in 2.6.39. It is already in the media >> tree (http://git.linuxtv.org/media_tree.git, see include/media/videobuf2-core.h). >> >> It is much better than the old videobuf framework, and in particular gives the driver >> much more control on how memory is allocated and used. >> >> Regards, >> >> Hans >> >> > >> > Thanks >> > Wen >> > >> > >-----Original Message----- >> > >From: Kanigeri, Hari K >> > >Sent: 2011年2月9日 15:22 >> > >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; >> > >linux-media@vger.kernel.org; >> > >umg-meego-handset-kernel@umglistsvr.jf.intel.com >> > >Cc: Jozef Kruger >> > >Subject: RE: Memory allocation in Video4Linux >> > > >> > > >> > > >> > >> -----Original Message----- >> > >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >> > >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] >> > >> On Behalf Of Iyer, Sundar >> > >> Sent: Wednesday, February 09, 2011 12:20 PM >> > >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; >> > >> umg-meego- handset-kernel@umglistsvr.jf.intel.com >> > >> Cc: Jozef Kruger >> > >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in >> > >> Video4Linux >> > >> >> > >> I remember some Continous Memory Allocator (CMA) being iterated >> > >> down a few versions on some mailing lists? IIRC, it is also for >> > >> large buffers and management for video IPs. >> > > >> > >I believe CMA is for allocating physically contiguous memory and from >> > >what Wen mentioned he also needs virtual memory management, which the >> > >IOMMU will provide. Please check the open source discussion on CMA, >> > >the last I heard CMA proposal was shot down. >> > >Reference: http://www.spinics.net/lists/linux-media/msg26875.html >> > > >> > >Wen, how are you currently allocating physical memory ? >> > > >> > > >> > >Thank you, >> > >Best regards, >> > >Hari >> > 翳 .n +% 遍荻\x17 w .n 伐 {炳g n r■ ㄨ&｛ 夸z罐 >zf＂ >> > 赙z_璁\x0f :+v )撸 >> > >> > >> >> -- >> Hans Verkuil - video4linux developer - sponsored by Cisco >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-media" in the body >> of a message to majordomo@vger.kernel.org More majordomo info at >> http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-09 7:27 ` Wang, Wen W 2011-02-09 7:51 ` Hans Verkuil @ 2011-02-10 7:59 ` Gao, Bin 2011-02-10 8:08 ` Jozef Kruger 2011-02-10 9:29 ` Laurent Pinchart 1 sibling, 2 replies; 19+ messages in thread From: Gao, Bin @ 2011-02-10 7:59 UTC (permalink / raw) To: Wang, Wen W, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger Penwell has IOMMU feature? As far as I know only part of Intel server processors have this feature which is designed originally for VT(virtualization technology). Wen, Can you refer to other ISP Soc drivers and see how they are dealing with this issue? I don't understand why you need to manage MMU inside ISP, I think the real problem is how can we allocate a large number of memory pages from IA side where ISP can access to by DMA. Any ISP document can be shared to help us understand what's the problem? -Bin -----Original Message----- From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On Behalf Of Wang, Wen W Sent: Tuesday, February 08, 2011 11:27 PM To: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego-handset-kernel@umglistsvr.jf.intel.com Cc: Jozef Kruger Subject: Re: [Umg-meego-handset-kernel] Memory allocation in Video4Linux Hi Hari, You are right. What we need is virtual address. Currently we alloc pages (alloc_pages()) for any request. Store those pages for an image buffer into a list. We also manage the virtual address for ISP by ourself (the range from 0 to 4GB) and the page table for our MMU which is independent to system MMU page table. Thanks Wen >-----Original Message----- >From: Kanigeri, Hari K >Sent: 2011年2月9日 15:22 >To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; >umg-meego-handset-kernel@umglistsvr.jf.intel.com >Cc: Jozef Kruger >Subject: RE: Memory allocation in Video4Linux > > > >> -----Original Message----- >> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On >> Behalf Of Iyer, Sundar >> Sent: Wednesday, February 09, 2011 12:20 PM >> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- >> handset-kernel@umglistsvr.jf.intel.com >> Cc: Jozef Kruger >> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in >> Video4Linux >> >> I remember some Continous Memory Allocator (CMA) being iterated down a >> few versions on >> some mailing lists? IIRC, it is also for large buffers and management >> for video IPs. > >I believe CMA is for allocating physically contiguous memory and from what Wen >mentioned he also needs virtual memory management, which the IOMMU will >provide. Please check the open source discussion on CMA, the last I heard CMA >proposal was shot down. >Reference: http://www.spinics.net/lists/linux-media/msg26875.html > >Wen, how are you currently allocating physical memory ? > > >Thank you, >Best regards, >Hari _______________________________________________ Umg-meego-handset-kernel mailing list Umg-meego-handset-kernel@umglistsvr.jf.intel.com http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-10 7:59 ` Gao, Bin @ 2011-02-10 8:08 ` Jozef Kruger 2011-02-10 9:29 ` Laurent Pinchart 1 sibling, 0 replies; 19+ messages in thread From: Jozef Kruger @ 2011-02-10 8:08 UTC (permalink / raw) To: Gao, Bin Cc: Wang, Wen W, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com Hi Bin, the ISP in Medfield has it's own MMU. This needs to be programmed. So in fact we have 2 problems: 1. how to allocate large memory address space (allocating pages is not the problem). 2. if we re-use OS functionality for this, we need to keep the ISP MMU up to date. This means it needs to be programmed with the L1 page table base address and the TLB needs to be flushed whenever something changes in the page tables. I don't think you need any ISP specific documentation for this. The ISP MMU behaves the same as the IA MMU does. Best regards, Jozef On 02/09/2011 11:59 PM, Gao, Bin wrote: > Penwell has IOMMU feature? > As far as I know only part of Intel server processors have this feature which is designed originally for VT(virtualization technology). > > Wen, > Can you refer to other ISP Soc drivers and see how they are dealing with this issue? > I don't understand why you need to manage MMU inside ISP, I think the real problem is how can we allocate a large number of memory pages from IA side where ISP can access to by DMA. > Any ISP document can be shared to help us understand what's the problem? > > -Bin > > -----Original Message----- > From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On Behalf Of Wang, Wen W > Sent: Tuesday, February 08, 2011 11:27 PM > To: Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego-handset-kernel@umglistsvr.jf.intel.com > Cc: Jozef Kruger > Subject: Re: [Umg-meego-handset-kernel] Memory allocation in Video4Linux > > Hi Hari, > > You are right. What we need is virtual address. > > Currently we alloc pages (alloc_pages()) for any request. Store those pages for an image buffer into a list. We also manage the virtual address for ISP by ourself (the range from 0 to 4GB) and the page table for our MMU which is independent to system MMU page table. > > Thanks > Wen > >> -----Original Message----- >> From: Kanigeri, Hari K >> Sent: 2011年2月9日 15:22 >> To: Iyer, Sundar; Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; >> umg-meego-handset-kernel@umglistsvr.jf.intel.com >> Cc: Jozef Kruger >> Subject: RE: Memory allocation in Video4Linux >> >> >> >>> -----Original Message----- >>> From: umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com >>> [mailto:umg-meego-handset-kernel-bounces@umglistsvr.jf.intel.com] On >>> Behalf Of Iyer, Sundar >>> Sent: Wednesday, February 09, 2011 12:20 PM >>> To: Wang, Wen W; Yang, Jianwei; linux-media@vger.kernel.org; umg-meego- >>> handset-kernel@umglistsvr.jf.intel.com >>> Cc: Jozef Kruger >>> Subject: Re: [Umg-meego-handset-kernel] Memory allocation in >>> Video4Linux >>> >>> I remember some Continous Memory Allocator (CMA) being iterated down a >>> few versions on >>> some mailing lists? IIRC, it is also for large buffers and management >>> for video IPs. >> I believe CMA is for allocating physically contiguous memory and from what Wen >> mentioned he also needs virtual memory management, which the IOMMU will >> provide. Please check the open source discussion on CMA, the last I heard CMA >> proposal was shot down. >> Reference: http://www.spinics.net/lists/linux-media/msg26875.html >> >> Wen, how are you currently allocating physical memory ? >> >> >> Thank you, >> Best regards, >> Hari > _______________________________________________ > Umg-meego-handset-kernel mailing list > Umg-meego-handset-kernel@umglistsvr.jf.intel.com > http://umglistsvr.jf.intel.com/mailman/listinfo/umg-meego-handset-kernel -- Jozef Kruger Senior Customer Solutions Architect Silicon Hive Inc. 2025 Gateway Place, Suite 230 San Jose, CA 95110 phone: +1 408 512 2969 cell: +1 408 644 7533 fax: +1 408 437 6014 jozef.kruger@siliconhive.com http://www.siliconhive.com ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-10 7:59 ` Gao, Bin 2011-02-10 8:08 ` Jozef Kruger @ 2011-02-10 9:29 ` Laurent Pinchart 2011-02-10 9:44 ` Wang, Wen W 1 sibling, 1 reply; 19+ messages in thread From: Laurent Pinchart @ 2011-02-10 9:29 UTC (permalink / raw) To: Gao, Bin Cc: Wang, Wen W, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com, Jozef Kruger Hi Wen, On Thursday 10 February 2011 08:59:38 Gao, Bin wrote: > Penwell has IOMMU feature? > As far as I know only part of Intel server processors have this feature > which is designed originally for VT(virtualization technology). > > Wen, > Can you refer to other ISP Soc drivers and see how they are dealing with > this issue? I don't understand why you need to manage MMU inside ISP, I > think the real problem is how can we allocate a large number of memory > pages from IA side where ISP can access to by DMA. Any ISP document can be > shared to help us understand what's the problem? I second this request. It's not totally clear to me from this mail thread whether your ISP has its own MMU (which would then be considered as an IOMMU), or accesses the system memory through other means (continuous buffers, scatter-gather lists, ...). If the Medfield ISP includes an IOMMU, the IOMMU layer is probably what you want to use. Memory can then be allocated using a simple vmalloc(), and pages can be mapped to the ISP memory space using the IOMMU. Please note that, unless the ISP IOMMU is already supported by the Linux kernel, you will obviously need to implement an IOMMU driver for it. What you shouldn't do is put that code in the ISP driver (unless hardware design doesn't give you a choice about that, but I need to see more documentation to answer this question). -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: Memory allocation in Video4Linux 2011-02-10 9:29 ` Laurent Pinchart @ 2011-02-10 9:44 ` Wang, Wen W 2011-02-10 10:00 ` Laurent Pinchart 0 siblings, 1 reply; 19+ messages in thread From: Wang, Wen W @ 2011-02-10 9:44 UTC (permalink / raw) To: Laurent Pinchart, Gao, Bin Cc: Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com, Jozef Kruger, Zhang, Xiaolin, Xie, Cindy Hi Laurent, You make this very clear, thank you! The ISP on Medfield do have its own IOMMU. And I also think an IOMMU layer for ISP is needed. I'm not quite understand "unless hardware design doesn't give you a choice about that,". Can you explain more on that? Also regarding to the VCMM (Virtual Contiguous Memory Manager) or CMA, is it also an option? Thanks Wen >-----Original Message----- >From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com] >Sent: 2011年2月10日 17:29 >To: Gao, Bin >Cc: Wang, Wen W; Kanigeri, Hari K; Iyer, Sundar; Yang, Jianwei; >linux-media@vger.kernel.org; umg-meego-handset-kernel@umglistsvr.jf.intel.com; >Jozef Kruger >Subject: Re: Memory allocation in Video4Linux > >Hi Wen, > >On Thursday 10 February 2011 08:59:38 Gao, Bin wrote: >> Penwell has IOMMU feature? >> As far as I know only part of Intel server processors have this feature >> which is designed originally for VT(virtualization technology). >> >> Wen, >> Can you refer to other ISP Soc drivers and see how they are dealing with >> this issue? I don't understand why you need to manage MMU inside ISP, I >> think the real problem is how can we allocate a large number of memory >> pages from IA side where ISP can access to by DMA. Any ISP document can be >> shared to help us understand what's the problem? > >I second this request. It's not totally clear to me from this mail thread >whether your ISP has its own MMU (which would then be considered as an IOMMU), >or accesses the system memory through other means (continuous buffers, >scatter-gather lists, ...). > >If the Medfield ISP includes an IOMMU, the IOMMU layer is probably what you >want to use. Memory can then be allocated using a simple vmalloc(), and pages >can be mapped to the ISP memory space using the IOMMU. > >Please note that, unless the ISP IOMMU is already supported by the Linux >kernel, you will obviously need to implement an IOMMU driver for it. What you >shouldn't do is put that code in the ISP driver (unless hardware design >doesn't give you a choice about that, but I need to see more documentation to >answer this question). > >-- >Regards, > >Laurent Pinchart ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-10 9:44 ` Wang, Wen W @ 2011-02-10 10:00 ` Laurent Pinchart 2011-02-11 2:07 ` KyongHo Cho 0 siblings, 1 reply; 19+ messages in thread From: Laurent Pinchart @ 2011-02-10 10:00 UTC (permalink / raw) To: Wang, Wen W Cc: Gao, Bin, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com, Jozef Kruger, Zhang, Xiaolin, Xie, Cindy Hi Wen, On Thursday 10 February 2011 10:44:58 Wang, Wen W wrote: > Hi Laurent, > > You make this very clear, thank you! > > The ISP on Medfield do have its own IOMMU. And I also think an IOMMU layer > for ISP is needed. > > I'm not quite understand "unless hardware design doesn't give you a choice > about that,". Can you explain more on that? Basically, Linux aims for driver reusability. That means that separate pieces of hardware, even when they are designed to work together, should be handled by separate, indepedent drivers. This might make the initial development a bit longer, as proper abstraction APIs need to be designed when they don't exist, but it cuts down time to market later when a hardware block is reused in a different silicon or board. A good example of that approach is I2C sensors. They're supported by drivers that are completely independent of the ISP they are connected to, and use the V4L2 subdev in-kernel API to communicate with the ISP in a hardware-indepedent way. IOMMU is such an API. It lets you implement support for your particular IOMMU in a self-contained drivers, which can then be used by device drivers (such as the ISP driver). For this to work, the hardware needs to have at least some level of separation between the different components. With I2C that's easy, the sensor can be controlled completely indepedently from the ISP. With the ISP IOMMU, it would more or less depend on how the registers are layed out in memory. If the ISP IOMMU registers are grouped together and separated from the other ISP registers, you should be fine. If they are mixed with ISP registers (an hypotetical bad case would for instance be if the ISP IOMMU required you to set bits in the TLB entries that describe the format of the video data stored in a page), such an abstraction would be much more difficult to achieve, and sometimes even impossible. > Also regarding to the VCMM (Virtual Contiguous Memory Manager) or CMA, is > it also an option? I'm not sure about VCMM, it seems to be an attempt to unify memory management across IOMMUs and system MMU. I don't think you need to worry about it now, IOMMU should be enough for your needs. CMA, as its name implies, is a contiguous memory allocator. As your ISP has an IOMMU, you don't need to allocate contiguous memory, so CMA isn't useful for your hardware. -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-10 10:00 ` Laurent Pinchart @ 2011-02-11 2:07 ` KyongHo Cho 0 siblings, 0 replies; 19+ messages in thread From: KyongHo Cho @ 2011-02-11 2:07 UTC (permalink / raw) To: Wang, Wen W Cc: Gao, Bin, Kanigeri, Hari K, Iyer, Sundar, Yang, Jianwei, linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com, Jozef Kruger, Zhang, Xiaolin, Xie, Cindy Hi Wen and Laurent, > >> Also regarding to the VCMM (Virtual Contiguous Memory Manager) or CMA, is >> it also an option? > > I'm not sure about VCMM, it seems to be an attempt to unify memory management > across IOMMUs and system MMU. I don't think you need to worry about it now, > IOMMU should be enough for your needs. > > CMA, as its name implies, is a contiguous memory allocator. As your ISP has an > IOMMU, you don't need to allocate contiguous memory, so CMA isn't useful for > your hardware. > VCM(aka. VCMM) is a virtual memory (device memory) allocator and manager when a system has multiple MMUs. Since physical memory allocation (and virtual memory allocation also) for processes by O/S is not suitable for peripheral devices, VCM separates allocating physical memory and virtual memory. It also gives you chances to override the default behavior of the allocators. If you concern about device memory management with IOMMU, I think VCM is one of good options even though a system has only one IOMMU to deal with. It is not merged into the Linux kernel because applications of VCM is very restricted and just few people are interested in it. KyongHo. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Memory allocation in Video4Linux 2011-02-09 6:23 Memory allocation in Video4Linux Wang, Wen W 2011-02-09 6:30 ` Kanigeri, Hari K 2011-02-09 6:30 ` Yang, Jianwei @ 2011-02-09 10:19 ` Mauro Carvalho Chehab 2 siblings, 0 replies; 19+ messages in thread From: Mauro Carvalho Chehab @ 2011-02-09 10:19 UTC (permalink / raw) To: Wang, Wen W Cc: linux-media@vger.kernel.org, umg-meego-handset-kernel@umglistsvr.jf.intel.com, Jozef Kruger, Zhang, Xiaolin Hi Wen, Em 09-02-2011 04:23, Wang, Wen W escreveu: > Hi, > > We are developing the image processor driver for Intel Medfield platform. > > We have received some comments on memory management that we should use standard Linux kernel interfaces for this, since we are doing everything by ourselves including memory allocation (based on pages), page table management, virtual address management and etc. > > So can you please help give some advice or suggestion on using standard kernel interface for memory management? > > The processor has a MMU on-chip with same virtual address range as IA. The processor will access system memory (read and write) through MMU and page table. The memory consumption of the driver could be quite big especially for high resolution (14MP) with certain features turned on. > For example: advanced ISP with XNR and yuv444 output, at 14MP this uses: > 1 RAW16: 2*14 = 28MB > 1 qplane6: 6/4 * 14 = 21MB > 1 yuv420_16: 2 * 1.5 * 14 = 42MB > 1 yuv420: 1.5 * 14 = 21MB > 1 yuv444: 3 * 14 = 42MB > total: 154MB. You should take a look at the videobuf2 for buffer management. It is flexible enough to be used on embedded hardware, as it splits the memory management on a separate module. In particular, the CMA allocator was designed to handle memory management on complex designs. You'll find extensive discussions about CMA and videobuf2 in the ML. The videobuf2 is already at linux-next and will be available for 2.6.39. Not sure about the status of the CMA allocator, as the patches are handled via another upstream tree. I hope that helps. Cheers, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2011-02-11 2:07 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-02-09 6:23 Memory allocation in Video4Linux Wang, Wen W 2011-02-09 6:30 ` Kanigeri, Hari K 2011-02-09 6:30 ` Yang, Jianwei 2011-02-09 6:32 ` Wang, Wen W 2011-02-09 6:49 ` Iyer, Sundar 2011-02-09 7:22 ` Kanigeri, Hari K 2011-02-09 7:27 ` Wang, Wen W 2011-02-09 7:51 ` Hans Verkuil 2011-02-09 7:55 ` Wang, Wen W 2011-02-10 10:06 ` Sakari Ailus 2011-02-09 7:58 ` Jonghun Han 2011-02-09 8:10 ` Wang, Wen W 2011-02-10 7:59 ` Gao, Bin 2011-02-10 8:08 ` Jozef Kruger 2011-02-10 9:29 ` Laurent Pinchart 2011-02-10 9:44 ` Wang, Wen W 2011-02-10 10:00 ` Laurent Pinchart 2011-02-11 2:07 ` KyongHo Cho 2011-02-09 10:19 ` Mauro Carvalho Chehab
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox