Linux 2.6.9-ac3

Linux 2.6.9-ac3

[Alan Cox]

ftp://ftp.kernel.org/pub/linux/kernel/people/alan/linux-2.6/2.6.9/

2.6.9-ac3
o	Fix syncppp/async ppp problems with new hangup	(Paul Fulghum)
o	Fix broken parport_pc unload			(Andrea Arcangeli)
o	Security fix for smbfs leak/overrun		(Urban Widmark)
o	Stop i8xx_tco making some boxes reboot on load	(wim@iguana)
o	Fix cpia/module tools deadlock			(Peter Pregler)
o	Fix missing suid_dumpable export		(Alan Cox)

2.6.9-ac2
o	Fix invalid kernel version stupidity		(Adrian Bunk)
o	Compiler ICE workaround/fixup			(Linus Torvalds)
o	Fix network DoS bug in 2.6.9			(Herbert Xu)
	| Suggested by Sami Farin
o	Flash lights on panic as in 2.4			(Andi Kleen)

2.6.9-ac1

Security Fixes
o	Set VM_IO on areas that are temporarily		(Alan Cox)
	marked PageReserved (Serious bug)
o	Lock ide-proc against driver unload		(Alan Cox)
	(very low severity)

Bug Fixes
o	Working IDE locking				(Alan Cox)
	| And a great deal of review by Bartlomiej
o	Handle E7xxx boxes with USB legacy flaws	(Alan Cox)
	
Functionality
o	Allow booting with "irqpoll" or "irqfixup"	(Alan Cox)
	on systems with broken IRQ tables.
o	Support for setuid core dumping in some		(Alan Cox)
	environments (off by default)
o	Support for drives that don't report geometry
o	IT8212 support (raid and passthrough)		(Alan Cox)
o	Allow IDE to grab all unknown generic IDE	(Alan Cox)
	devices (boot with "all-generic-ide")
o	Restore PWC driver				(Luc Saillard)

Other
o	Small pending tty clean-up to moxa		(Alan Cox)
o	Put VIA Velocity (tm) adapters under gigabit	(VIA)

Re: Linux 2.6.9-ac3

[Mark Lord]

 >Bug Fixes
 >o	Working IDE locking		(Alan Cox)
 >	| And a great deal of review by Bartlomiej

Mmm.. may still have some issues.

Here's what "cardctl eject" now gives on unload of ide-cs:

bad: scheduling while atomic!
  [<c02a0f7e>] schedule+0x4de/0x4f0
  [<c0119451>] __wake_up_common+0x41/0x60
  [<c02a1079>] wait_for_completion+0x99/0xf0
  [<c01193f0>] default_wake_function+0x0/0x20
  [<c01193f0>] default_wake_function+0x0/0x20
  [<c012c8a8>] queue_work+0x68/0x80
  [<c012c791>] call_usermodehelper+0xe1/0xf0
  [<c012c640>] __call_usermodehelper+0x0/0x70
  [<c01bdc7d>] kset_hotplug+0x1dd/0x240
  [<c01bdd3a>] kobject_hotplug+0x5a/0x60
  [<c01be07b>] kobject_del+0x1b/0x40
  [<c01be0b3>] kobject_unregister+0x13/0x30
  [<c018cef9>] del_gendisk+0x39/0xe0
  [<c0223cd8>] idedisk_cleanup+0x48/0x60
  [<c0212fd6>] __ide_unregister_hwif+0x506/0x5c0
  [<c0196be8>] ext3_mark_iloc_dirty+0x28/0x40
  [<c02130b6>] ide_unregister_hwif+0x26/0x40
  [<e092da23>] ide_release+0x73/0x80 [ide_cs]
  [<c0118992>] activate_task+0x62/0x80
  [<e092d1b6>] ide_detach+0x86/0xa0 [ide_cs]
  [<e08b5b19>] unbind_request+0xc9/0xd0 [ds]
  [<e08b62fd>] ds_ioctl+0x3dd/0x690 [ds]
  [<c029c25d>] unix_dgram_sendmsg+0x36d/0x570
  [<c0230ce0>] sock_sendmsg+0xe0/0x100
  [<c013bd8b>] generic_file_aio_write_nolock+0x27b/0x4b0
  [<c0185a7b>] proc_destroy_inode+0x1b/0x20
  [<c0172b75>] destroy_inode+0x35/0x60
  [<c0173ef2>] iput+0x62/0x90
  [<c0185a7b>] proc_destroy_inode+0x1b/0x20
  [<c0172b75>] destroy_inode+0x35/0x60
  [<c0147bb3>] zap_pmd_range+0x63/0x80
  [<c0147c23>] unmap_page_range+0x53/0x80
  [<c0147d36>] unmap_vmas+0xe6/0x1d0
  [<c014a4d7>] remove_vm_struct+0x77/0xa0
  [<c014bf1f>] unmap_vma_list+0x1f/0x30
  [<c014c2df>] do_munmap+0x14f/0x190
  [<c016ba30>] sys_ioctl+0x100/0x270
  [<c01060d9>] sysenter_past_esp+0x52/0x71

I see a similar dump when using delkin_cb on 2.6.9-ac3 as well.

Cheers
-- 
Mark Lord
(hdparm keeper & the original "Linux IDE Guy")

[patch 2.6.9-ac3] ide: fixes for 2.6.9-ac3 ide issues

[Mark Lord]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

The attached patch fixes two problems with 2.6.9-ac3:

1. restores the missing spin_unlock_irq() line, and
2. removes hwif from /proc/ide/ as part of __ide_unregister_hwif().

(and also removes a bogus compiler warning)

Signed-off-by: Mark Lord <lkml@rtr.ca>
-- 
Mark Lord
(hdparm keeper & the original "Linux IDE Guy")

[-- Attachment #2: ac3-ide-fixes.patch --]
[-- Type: text/plain, Size: 2667 bytes --]

diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/ide/ide.c linux/drivers/ide/ide.c
--- linux-2.6.9-ac3/drivers/ide/ide.c	2004-10-21 20:45:22.000000000 -0400
+++ linux/drivers/ide/ide.c	2004-10-21 21:28:54.000000000 -0400
@@ -862,7 +862,7 @@
 
 int __ide_unregister_hwif(ide_hwif_t *hwif)
 {
-	ide_drive_t *drive;
+	ide_drive_t *drive = NULL; /* keep compiler happy */
 	ide_hwif_t *g;
 	static ide_hwif_t tmp_hwif; /* protected by ide_cfg_sem */
 	ide_hwgroup_t *hwgroup;
@@ -901,6 +901,7 @@
 	was_present = hwif->present;	 
 	hwif->present = 0;
 
+	spin_unlock_irq(&ide_lock);
 	up(&ide_setting_sem);
 
 	for (unit = 0; unit < MAX_DRIVES; ++unit) {
@@ -912,6 +913,7 @@
 
 #ifdef CONFIG_PROC_FS
 	destroy_proc_ide_drives(hwif);
+	destroy_proc_ide_interface(hwif);
 #endif
 
 	spin_lock_irq(&ide_lock);
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/ide/ide-proc.c linux/drivers/ide/ide-proc.c
--- linux-2.6.9-ac3/drivers/ide/ide-proc.c	2004-10-21 20:45:22.000000000 -0400
+++ linux/drivers/ide/ide-proc.c	2004-10-21 21:25:06.000000000 -0400
@@ -821,24 +821,30 @@
 EXPORT_SYMBOL_GPL(ide_pci_create_host_proc);
 #endif
 
+void destroy_proc_ide_interface(ide_hwif_t *hwif)
+{
+	int exist = (hwif->proc != NULL);
+#if 0
+	if (!hwif->present)
+		continue;
+#endif
+	if (exist) {
+		destroy_proc_ide_drives(hwif);
+		ide_remove_proc_entries(hwif->proc, hwif_entries);
+		remove_proc_entry(hwif->name, proc_ide_root);
+		hwif->proc = NULL;
+	}
+}
+
+EXPORT_SYMBOL(destroy_proc_ide_interface);
+
 void destroy_proc_ide_interfaces(void)
 {
 	int	h;
 
 	for (h = 0; h < MAX_HWIFS; h++) {
 		ide_hwif_t *hwif = &ide_hwifs[h];
-		int exist = (hwif->proc != NULL);
-#if 0
-		if (!hwif->present)
-			continue;
-#endif
-		if (exist) {
-			destroy_proc_ide_drives(hwif);
-			ide_remove_proc_entries(hwif->proc, hwif_entries);
-			remove_proc_entry(hwif->name, proc_ide_root);
-			hwif->proc = NULL;
-		} else
-			continue;
+		destroy_proc_ide_interface(hwif);
 	}
 }
 
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/include/linux/ide.h linux/include/linux/ide.h
--- linux-2.6.9-ac3/include/linux/ide.h	2004-10-21 20:45:22.000000000 -0400
+++ linux/include/linux/ide.h	2004-10-21 21:27:39.000000000 -0400
@@ -1076,6 +1076,7 @@
 extern void proc_ide_create(void);
 extern void proc_ide_destroy(void);
 extern void destroy_proc_ide_drives(ide_hwif_t *);
+extern void destroy_proc_ide_interface(ide_hwif_t *);
 extern void create_proc_ide_interfaces(void);
 extern void ide_add_proc_entries(struct proc_dir_entry *, ide_proc_entry_t *, void *);
 extern void ide_remove_proc_entries(struct proc_dir_entry *, ide_proc_entry_t *);

Re: Linux 2.6.9-ac3

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 221 bytes --]

Alan Cox wrote:
> ftp://ftp.kernel.org/pub/linux/kernel/people/alan/linux-2.6/2.6.9/

Since you're collecting and may end up being the source of all patches 
stable; any chance of a patch tarball too please?

Cheers,
Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

[PATCH 2.6.9-ac3] delkin_cb: new driver for Cardbus IDE CF Adapter

[Mark Lord]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

This is the first 2.6 release of a PCI-IDE module to support
the Delkin/ASKA/Workbit Cardbus CompactFlash<-->IDE adaptor.
It may (or not) also work for other versions of this hardware (eg. SD).

This module is based on the existing ide-cs module,
and works fine for me (with other fixes previously posted for -ac3).

Patch is against 2.6.9-ac3.

Signed-off-by: Mark Lord <lkml@rtr.ca>
-- 
Mark Lord
(hdparm keeper & the original "Linux IDE Guy")

[-- Attachment #2: delkin_cb-2.6.9-ac3.patch --]
[-- Type: text/plain, Size: 6453 bytes --]

diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/ide/Kconfig linux/drivers/ide/Kconfig
--- linux-2.6.9-ac3/drivers/ide/Kconfig	2004-10-21 20:45:22.000000000 -0400
+++ linux/drivers/ide/Kconfig	2004-10-21 20:25:05.000000000 -0400
@@ -166,6 +166,13 @@
 	  Support for outboard IDE disks, tape drives, and CD-ROM drives
 	  connected through a  PCMCIA card.
 
+config BLK_DEV_DELKIN
+	tristate "Cardbus IDE support (Delkin/ASKA/Workbit)"
+	depends on PCMCIA && PCI
+	help
+	  Support for Delkin, ASKA, and Workbit Cardbus CompactFlash
+	  Adapters.  This may also work for similar SD and XD adapters.
+
 config BLK_DEV_IDECD
 	tristate "Include IDE/ATAPI CDROM support"
 	---help---
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/ide/pci/delkin_cb.c linux/drivers/ide/pci/delkin_cb.c
--- linux-2.6.9-ac3/drivers/ide/pci/delkin_cb.c	1969-12-31 19:00:00.000000000 -0500
+++ linux/drivers/ide/pci/delkin_cb.c	2004-10-21 21:44:45.000000000 -0400
@@ -0,0 +1,139 @@
+/*
+ *  linux/drivers/ide/pci/delkin_cb.c
+ *
+ *  Created 20 Oct 2004 by Mark Lord
+ *
+ *  Basic support for Delkin/ASKA/Workbit Cardbus CompactFlash adapter
+ *
+ *  Modeled after the 16-bit PCMCIA driver: ide-cs.c
+ *
+ *  This is slightly peculiar, in that it is a PCI driver,
+ *  but is NOT an IDE PCI driver -- the IDE layer does not directly
+ *  support hot insertion/removal of PCI interfaces, so this driver
+ *  is unable to use the IDE PCI interfaces.  Instead, it uses the
+ *  same interfaces as the ide-cs (PCMCIA) driver uses.
+ *  On the plus side, the driver is also smaller/simpler this way.
+ *
+ *  This file is subject to the terms and conditions of the GNU General Public
+ *  License.  See the file COPYING in the main directory of this archive for
+ *  more details.
+ */
+#include <linux/config.h>
+#include <linux/types.h>
+#include <linux/module.h>
+#include <linux/mm.h>
+#include <linux/blkdev.h>
+#include <linux/hdreg.h>
+#include <linux/ide.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <asm/io.h>
+
+/*
+ * No chip documentation has yet been found,
+ * so these configuration values were pulled from
+ * a running Win98 system using "debug".
+ * This gives around 3MByte/second read performance,
+ * which is about 2/3 of what the chip is capable of.
+ *
+ * There is also a 4KByte mmio region on the card,
+ * but its purpose has yet to be reverse-engineered.
+ */
+static const u8 setup[] = {
+	0x00, 0x05, 0xbe, 0x01, 0x20, 0x8f, 0x00, 0x00,
+	0xa4, 0x1f, 0xb3, 0x1b, 0x00, 0x00, 0x00, 0x80,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+	0x00, 0x00, 0x00, 0x00, 0xa4, 0x83, 0x02, 0x13,
+};
+
+static int __devinit
+delkin_cb_probe (struct pci_dev *dev, const struct pci_device_id *id)
+{
+	unsigned long base;
+	hw_regs_t hw;
+	ide_hwif_t *hwif = NULL;
+	ide_drive_t *drive;
+	int i, rc;
+
+	rc = pci_enable_device(dev);
+	if (rc) {
+		printk(KERN_ERR "delkin_cb: pci_enable_device failed (%d)\n", rc);
+		return rc;
+	}
+	rc = pci_request_regions(dev, "delkin_cb");
+	if (rc) {
+		printk(KERN_ERR "delkin_cb: pci_request_regions failed (%d)\n", rc);
+		pci_disable_device(dev);
+		return rc;
+	}
+	base = pci_resource_start(dev, 0);
+	outb(0x02, base + 0x1e);	/* set nIEN to block interrupts */
+	inb(base + 0x17);		/* read status to clear interrupts */
+	for (i = 0; i < sizeof(setup); ++i) {
+		if (setup[i])
+			outb(setup[i], base + i);
+	}
+	pci_release_regions(dev);	/* IDE layer handles regions itself */
+
+	memset(&hw, 0, sizeof(hw));
+	ide_std_init_ports(&hw, base + 0x10, base + 0x1e);
+	hw.irq = dev->irq;
+	hw.chipset = ide_pci;		/* this enables IRQ sharing */
+
+	rc = ide_register_hw(&hw, &hwif);
+	if (rc < 0) {
+		printk(KERN_ERR "delkin_cb: ide_register_hw failed (%d)\n", rc);
+		return -ENODEV;
+	}
+	pci_set_drvdata(dev, hwif);
+	hwif->pci_dev = dev;
+	drive = &hwif->drives[0];
+	if (drive->present) {
+		drive->io_32bit = 1;
+		drive->unmask   = 1;
+	}
+	return 0;
+}
+
+static void
+delkin_cb_remove (struct pci_dev *dev)
+{
+	ide_hwif_t *hwif = pci_get_drvdata(dev);
+
+	if (hwif)
+		ide_unregister_hwif(hwif);
+	pci_disable_device(dev);
+}
+
+static struct pci_device_id delkin_cb_pci_tbl[] __devinitdata = {
+	{ PCI_VENDOR_ID_WORKBIT, PCI_DEVICE_ID_WORKBIT_CB, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0},
+	{ 0, },
+};
+MODULE_DEVICE_TABLE(pci, delkin_cb_pci_tbl);
+
+static struct pci_driver driver = {
+	.name		= "Delkin/ASKA/Workbit Cardbus IDE",
+	.id_table	= delkin_cb_pci_tbl,
+	.probe		= delkin_cb_probe,
+	.remove		= delkin_cb_remove,
+};
+
+static int
+delkin_cb_init (void)
+{
+	return pci_module_init(&driver);
+}
+
+static void
+delkin_cb_exit (void)
+{
+	pci_unregister_driver(&driver);
+}
+
+module_init(delkin_cb_init);
+module_exit(delkin_cb_exit);
+
+MODULE_AUTHOR("Mark Lord");
+MODULE_DESCRIPTION("Basic support for Delkin/ASKA/Workbit Cardbus IDE");
+MODULE_LICENSE("GPL");
+
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/ide/pci/Makefile linux/drivers/ide/pci/Makefile
--- linux-2.6.9-ac3/drivers/ide/pci/Makefile	2004-10-18 17:53:42.000000000 -0400
+++ linux/drivers/ide/pci/Makefile	2004-10-21 19:32:57.000000000 -0400
@@ -9,6 +9,7 @@
 obj-$(CONFIG_BLK_DEV_CS5530)		+= cs5530.o
 obj-$(CONFIG_BLK_DEV_SC1200)		+= sc1200.o
 obj-$(CONFIG_BLK_DEV_CY82C693)		+= cy82c693.o
+obj-$(CONFIG_BLK_DEV_DELKIN)		+= delkin_cb.o
 obj-$(CONFIG_BLK_DEV_HPT34X)		+= hpt34x.o
 obj-$(CONFIG_BLK_DEV_HPT366)		+= hpt366.o
 #obj-$(CONFIG_BLK_DEV_HPT37X)		+= hpt37x.o
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/drivers/scsi/nsp32.h linux/drivers/scsi/nsp32.h
--- linux-2.6.9-ac3/drivers/scsi/nsp32.h	2004-10-18 17:53:05.000000000 -0400
+++ linux/drivers/scsi/nsp32.h	2004-10-21 19:32:57.000000000 -0400
@@ -22,7 +22,6 @@
  * VENDOR/DEVICE ID
  */
 #define PCI_VENDOR_ID_IODATA  0x10fc
-#define PCI_VENDOR_ID_WORKBIT 0x1145
 
 #define PCI_DEVICE_ID_NINJASCSI_32BI_CBSC_II   0x0005
 #define PCI_DEVICE_ID_NINJASCSI_32BI_KME       0xf007
diff -u --recursive --new-file --exclude='.*' linux-2.6.9-ac3/include/linux/pci_ids.h linux/include/linux/pci_ids.h
--- linux-2.6.9-ac3/include/linux/pci_ids.h	2004-10-21 20:45:22.000000000 -0400
+++ linux/include/linux/pci_ids.h	2004-10-21 20:25:05.000000000 -0400
@@ -2331,3 +2331,6 @@
 #define PCI_DEVICE_ID_ARK_STING		0xa091
 #define PCI_DEVICE_ID_ARK_STINGARK	0xa099
 #define PCI_DEVICE_ID_ARK_2000MT	0xa0a1
+
+#define PCI_VENDOR_ID_WORKBIT		0x1145
+#define PCI_DEVICE_ID_WORKBIT_CB	0xf021

Re: Linux 2.6.9-ac3

[Jens Axboe]

On Fri, Oct 22 2004, Alan Cox wrote:
> ftp://ftp.kernel.org/pub/linux/kernel/people/alan/linux-2.6/2.6.9/

If you're collection ide patchlets for later inclusion, please include
this as well. It makes sure we remove the ideX directory in proc when an
interface is removed. With ide-cs and repeated insert/reject cycles, we
end up with tons of ide2 entries.

Signed-off-by: Jens Axboe <axboe@suse.de>

--- /opt/kernel/linux-2.6/include/linux/ide.h	2004-10-11 12:19:40.000000000 +0200
+++ linux-2.6.9/include/linux/ide.h	2004-10-20 17:25:05.000000000 +0200
@@ -1063,6 +1063,7 @@
 extern void proc_ide_create(void);
 extern void proc_ide_destroy(void);
 extern void destroy_proc_ide_drives(ide_hwif_t *);
+extern void destroy_proc_ide_interfaces(void);
 extern void create_proc_ide_interfaces(void);
 extern void ide_add_proc_entries(struct proc_dir_entry *, ide_proc_entry_t *, void *);
 extern void ide_remove_proc_entries(struct proc_dir_entry *, ide_proc_entry_t *);
--- /opt/kernel/linux-2.6/drivers/ide/ide.c	2004-10-11 12:19:29.000000000 +0200
+++ linux-2.6.9/drivers/ide/ide.c	2004-10-20 17:25:54.000000000 +0200
@@ -798,6 +798,7 @@
 
 #ifdef CONFIG_PROC_FS
 	destroy_proc_ide_drives(hwif);
+	destroy_proc_ide_interfaces();
 #endif
 
 	hwgroup = hwif->hwgroup;
--- /opt/kernel/linux-2.6/drivers/ide/ide-proc.c	2004-10-11 12:19:29.000000000 +0200
+++ linux-2.6.9/drivers/ide/ide-proc.c	2004-10-20 17:31:02.040529272 +0200
@@ -731,10 +731,9 @@
 	for (h = 0; h < MAX_HWIFS; h++) {
 		ide_hwif_t *hwif = &ide_hwifs[h];
 		int exist = (hwif->proc != NULL);
-#if 0
-		if (!hwif->present)
+
+		if (hwif->present)
 			continue;
-#endif
 		if (exist) {
 			destroy_proc_ide_drives(hwif);
 			ide_remove_proc_entries(hwif->proc, hwif_entries);

-- 
Jens Axboe

Re: Linux 2.6.9-ac3

[Luca Risolia]

> o       Restore PWC driver                              (Luc Saillard)

This driver does decompression in kernel space, which is not
allowed. That part has to be removed from the driver before
asking for the inclusion in the mainline kernel.

Regards,
	Luca Risolia

Re: Linux 2.6.9-ac3

[Luc Saillard]

On Fri, Oct 22, 2004 at 10:13:35AM +0200, Luca Risolia wrote:
> > o       Restore PWC driver                              (Luc Saillard)
> 
> This driver does decompression in kernel space, which is not
> allowed. That part has to be removed from the driver before
> asking for the inclusion in the mainline kernel.

I know this problem, but without a user API like ALSA, each driver need to
implement the decompression module. When the driver will support v4l2, we can
return the compressed stream to the user land. I want a v4l3, which is
designed as ALSA does for soundcard, with a API for userland and kernelland.
In the meantime, i can put a module option, if the user want the
decompression in the kernel mode.

Just for your information, many other part in the kernel use
decompression/compression like PPP, video, ... Have you look about the
algorithm use in decompression ?, it's just a lookup table with some
pre-calculated values. Ok you lost ~60Kbytes of kernel memory. I you look into
the logitech windows drivers, you will a see a big fat library name openvc
(apt-cache search opencv). I don't want to include all operations but if we
haven't a good userland library, our webcam is useless on Linux. 
Please, if you can provide some help to put this in userland (provides
patches for mplayer, xawtv, kame, gnomeeting, ...), i'll be glad to remove
the offending code.

Luc

opencv:
 The Open Computer Vision Library is a collection of algorithms and sample
 code for various computer vision problems. The library is compatible with
 IPL (Intel's Image Processing Library) and, if available, can use IPP
 (Intel's Integrated Performance Primitives) for better performance.
 .
 OpenCV provides low level portable data types and operators, and a set
 of high level functionalities for video acquisition, image processing and
 analysis, structural analysis, motion analysis and object tracking, object
 recognition, camera calibration and 3D reconstruction.
(taken from apt-cache show libopencv-doc)

Re: Linux 2.6.9-ac3

[Luca Risolia]

On Fri, 22 Oct 2004 11:21:02 +0200
Luc Saillard <luc@saillard.org> wrote:

> On Fri, Oct 22, 2004 at 10:13:35AM +0200, Luca Risolia wrote:
> > > o       Restore PWC driver                              (Luc Saillard)
> > 
> > This driver does decompression in kernel space, which is not
> > allowed. That part has to be removed from the driver before
> > asking for the inclusion in the mainline kernel.
> 
> I know this problem, but without a user API like ALSA, each driver need to
> implement the decompression module. When the driver will support v4l2, we can
> return the compressed stream to the user land. I want a v4l3, which is
> designed as ALSA does for soundcard, with a API for userland and kernelland.
> In the meantime, i can put a module option, if the user want the
> decompression in the kernel mode.

Either port the driver to V4L2, which handles decompression stuff well,
or provide a separate downloadble GPL'ed module. Other drivers in the
mainline kernel observe this rule; the pwc case is not an exception.
Also, this matter has been already discussed many times in the v4l
mailing list: no video decompression at all in kernel space, even if
*optional* through an *indipendent* module. I doubt Morton or Linus will
ever accept this version of pwc driver, since they did accept a patch disabling
colorspace conversion from a driver recently.

> Just for your information, many other part in the kernel use
> decompression/compression like PPP, video, ... Have you look about the
> algorithm use in decompression ?, it's just a lookup table with some
> pre-calculated values. Ok you lost ~60Kbytes of kernel memory. I you look into
> the logitech windows drivers, you will a see a big fat library name openvc
> (apt-cache search opencv). I don't want to include all operations but if we
> haven't a good userland library, our webcam is useless on Linux. 
> Please, if you can provide some help to put this in userland (provides
> patches for mplayer, xawtv, kame, gnomeeting, ...), i'll be glad to remove
> the offending code.

It sounds logic that none would help to fix user applications, if
we kept including things like decompression in each module in the kernel.

If it ever happens that this driver is accepted, be also prepared to
accept patches adding decompression and colorspace conversions for
every video driver I am aware of, starting from the ones I have already
written to the ones I'll submit in the future.

Regards,
	Luca Risolia

> 
> Luc
> 
> opencv:
>  The Open Computer Vision Library is a collection of algorithms and sample
>  code for various computer vision problems. The library is compatible with
>  IPL (Intel's Image Processing Library) and, if available, can use IPP
>  (Intel's Integrated Performance Primitives) for better performance.
>  .
>  OpenCV provides low level portable data types and operators, and a set
>  of high level functionalities for video acquisition, image processing and
>  analysis, structural analysis, motion analysis and object tracking, object
>  recognition, camera calibration and 3D reconstruction.
> (taken from apt-cache show libopencv-doc)
> 

Re: Linux 2.6.9-ac3

[Xavier Bestel]

Luc Saillard <luc@saillard.org> wrote:
> On Fri, Oct 22, 2004 at 10:13:35AM +0200, Luca Risolia wrote:
> > > o       Restore PWC driver                              (Luc Saillard)
> > 
> > This driver does decompression in kernel space, which is not
> > allowed. That part has to be removed from the driver before
> > asking for the inclusion in the mainline kernel.
> 
> I know this problem, but without a user API like ALSA, each driver need to
> implement the decompression module. When the driver will support v4l2, we can
> return the compressed stream to the user land. I want a v4l3, which is
> designed as ALSA does for soundcard, with a API for userland and kernelland.

Why not use gstreamer as a userland API ? You deliver compressed video
through v4l2, then write a decompression plugin specific to your
chipset.

	Xav

Re: Linux 2.6.9-ac3

[Luc Saillard]

On Fri, Oct 22, 2004 at 02:34:55PM +0200, Xavier Bestel wrote:
> Luc Saillard <luc@saillard.org> wrote:
> > I know this problem, but without a user API like ALSA, each driver need to
> > implement the decompression module. When the driver will support v4l2, we can
> > return the compressed stream to the user land. I want a v4l3, which is
> > designed as ALSA does for soundcard, with a API for userland and kernelland.
> 
> Why not use gstreamer as a userland API ? You deliver compressed video
> through v4l2, then write a decompression plugin specific to your
> chipset.
 
Because i don't know very well gstreamer and not a lot of applications are
use it. But i can try to produce a plugin for it, but i don't know a lot of
application (video) that use gstreamer. I can try to make a plugin for
gstreamer, i'll put on my TODO.

I try gstreamer with amarok to play sound using alsa, and this does't work
(segfault). Gstreamer seems too big to be the default for every applications,
think that you can put a webcam on a top appliance, with little memory, space
disk, and NO XML :-)

Luc

Re: Linux 2.6.9-ac3

[Ronald S. Bultje]

Hi Luc,

On Fri, 2004-10-22 at 15:47, Luc Saillard wrote:
> I try gstreamer with amarok to play sound using alsa, and this does't work
> (segfault). Gstreamer seems too big to be the default for every applications,
> think that you can put a webcam on a top appliance, with little memory, space
> disk, and NO XML :-)

That is all possible. I understand that you don't want XML and such in
your embedded devices, and you don't want loadable modules, and you
basically want no bloat at all. To some extent, including all of the
above-mentioned, that is already possible using GStreamer. Some of us
(yes, I'm a GStreamer developer otherwise I wouldn't care to reply) have
worked on this in the past, and some of us are still working on
GStreamer-based solutions in embedded devices.

> > Luc Saillard <luc@saillard.org> wrote:
> > > I know this problem, but without a user API like ALSA, each driver need to
> > > implement the decompression module. When the driver will support v4l2, we can
> > > return the compressed stream to the user land. I want a v4l3, which is
> > > designed as ALSA does for soundcard, with a API for userland and kernelland.

It works for ALSA because audio is as simple as it gets. As soon as you
throw in some soundcard with no PCM support but only
my_nice_media_audio_format, it doesn't work. So seriously, tell me what
you'd need to make this work for video (and specifically all those
brands of webcams)?

* since webcams have custom compression algos, you need dynamically
loadable libraries and extendable type systems. OK, so since we're in
for embedded, we'll also need a very powerful system that can help us do
all this but suited for embedded.
* since people will want to touch every single detail, you will need a
lowlevel API which is basically a userspace wrapper around POSIX.
Doubtful.
* and of course a nice highlevel API for us weenies that don't get it.
* And because of the combination of the above taken (too bloated)
together with the fact that v4l2 is actually documented, pretty much
nobody will use it.
* so nobody will write plugins for his custom webcam format and well,
from here on you get the point: it won't work.
* did I mention how good fragmentation is for our public image to the
corporate world? :).

For more fun, read the video4linux-list@redhat.com archives. It's fun to
re-read flamewars from the past. Let's not redo them, it's a waste of
time.

Now, of course, you want a solution that will work for your particular
case, which appears to be some kind fo embedded thing with your specific
cam. So just use v4l2, implement a custom module in your embedded
application that decodes from the cam-format in userspace and you're
done! Oh, you want to use the cam in Gnome-Meeting? Then let's go for
the GStreamer-approach anyway, I don't think Gnome-Meeting cares about
XML. :).

Ronald

PS v4l3? Let's first port the remaining drivers (e.g. zr36120, qc-usb)
to v4l2, ok? :).

-- 
Ronald S. Bultje <rbultje@ronald.bitfreak.net>

Re: Linux 2.6.9-ac3

[David Weinehall]

On Fri, Oct 22, 2004 at 03:33:27PM +0200, Luc Saillard wrote:

[snip]

> I try gstreamer with amarok to play sound using alsa, and this does't work
> (segfault). Gstreamer seems too big to be the default for every applications,

[snip]

This sounds more like a problem with Amarok though; I'm
listening to music right now using Rhythmbox, which uses gstreamer as a
backend, and I'm using ALSA (granted, I have esd as an intermediate
layer).


Regards: David Weinehall
-- 
 /) David Weinehall <tao@acc.umu.se> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Re: Linux 2.6.9-ac3

[Gergely Nagy]

On Fri, 2004-10-22 at 14:34 +0200, Xavier Bestel wrote:
> Luc Saillard <luc@saillard.org> wrote:
> > On Fri, Oct 22, 2004 at 10:13:35AM +0200, Luca Risolia wrote:
> > > > o       Restore PWC driver                              (Luc Saillard)
> > > 
> > > This driver does decompression in kernel space, which is not
> > > allowed. That part has to be removed from the driver before
> > > asking for the inclusion in the mainline kernel.
> > 
> > I know this problem, but without a user API like ALSA, each driver need to
> > implement the decompression module. When the driver will support v4l2, we can
> > return the compressed stream to the user land. I want a v4l3, which is
> > designed as ALSA does for soundcard, with a API for userland and kernelland.
> 
> Why not use gstreamer as a userland API ? You deliver compressed video
> through v4l2, then write a decompression plugin specific to your
> chipset.

gstreamer might work for your average desktop application, and a
gstreamer plugin for this might be very handy indeed. However, there are
setups where gstreamer is just too bloated, and there's no way I could
squeeze it on the box's filesystem (think in the 8-16Mb range for
available space).

A small decoding library and a gstreamer plugin on top of that would be
the best solution, I think.

However, until there's such a thing, Luc's driver works fine for me and
my employer.

(Also, keep in mind that PWC thingies are pretty common in some circles,
and owners of these devices would like to see a working driver now.
Instead of a perfect one a year from now. Therefore, the driver staging
in the -ac tree is a very good thing, imho.)

-- 
Gergely Nagy

Re: Linux 2.6.9-ac3

[Luc Saillard]

On Fri, Oct 22, 2004 at 02:30:36PM +0200, Luca Risolia wrote:
> Either port the driver to V4L2, which handles decompression stuff well,
no, it only provide a number (that needs to be reserved) to the user API, but
if the user API doesn't exist, application will not support your camera. I
can provide a .c and .h for the decompression module, but since the
decompression table depend of the resolution,quality choose by the webcam
(thinks as quality fine,normal,low). I can't provide with v4l2 a generic
control for this.
> or provide a separate downloadble GPL'ed module. Other drivers in the
> mainline kernel observe this rule; the pwc case is not an exception.
The driver, in it's current status, is GPL. Can you give me some examples
about "other drivers" ?
> Also, this matter has been already discussed many times in the v4l
> mailing list: no video decompression at all in kernel space, even if
> *optional* through an *indipendent* module.
It's easy for you to say that, because your driver returns a native format.
Since nobody cares about webcam, and stream compressed, if today, i remove
the table in the module, i'll wait one year ? two years for app that support
my webcam ?
> I doubt Morton or Linus will ever accept this version of pwc driver, since
> they did accept a patch disabling colorspace conversion from a driver
> recently.
my driver only output a standart video stream, i don't convert colorspace.
Decompression ops use table and offset in this table. The one thing i agree
about decompression, is for kernel preemption.

> It sounds logic that none would help to fix user applications, if
> we kept including things like decompression in each module in the kernel.
And it's logic too, to let the driver do the decompression because you WANT
your device working. So please, i want to remove decompression, but i want
people be able to use camera. People have trouble to compile the module
outside of the kernel (and you just need to type make; make install). If you
need to compile, gnomeeting, kame, kde, xawtv yourself to use the camera,
i'll not able to take time to add features on this drivers. 
 
Luc

Re: Linux 2.6.9-ac3

[Luca Risolia]

On Fri, 22 Oct 2004 15:16:39 +0200
Luc Saillard <luc@saillard.org> wrote:

> On Fri, Oct 22, 2004 at 02:30:36PM +0200, Luca Risolia wrote:
> > Either port the driver to V4L2, which handles decompression stuff well,

> no, it only provide a number (that needs to be reserved) to the user API, but
> if the user API doesn't exist, application will not support your camera. I
> can provide a .c and .h for the decompression module, but since the
> decompression table depend of the resolution,quality choose by the webcam
> (thinks as quality fine,normal,low). I can't provide with v4l2 a generic
> control for this.

Modify the V4L2 API by adding the correct generic interface.

> > or provide a separate downloadble GPL'ed module. Other drivers in the
> > mainline kernel observe this rule; the pwc case is not an exception.

> The driver, in it's current status, is GPL. Can you give me some examples
> about "other drivers" ?

At least two: w9968cf, ov511.

> > Also, this matter has been already discussed many times in the v4l
> > mailing list: no video decompression at all in kernel space, even if
> > *optional* through an *indipendent* module.

> It's easy for you to say that, because your driver returns a native format.
> Since nobody cares about webcam, and stream compressed, if today, i remove
> the table in the module, i'll wait one year ? two years for app that support
> my webcam ?

No, it's not easy for me to say that: the w9968cf driver is not really usefull
without the optional module available *outside* the kernel, since native UYVY
is not supported in userland at all. Furthermore, having had a separate module
for these things, is one more reason why none never patched any of the existing
applications.

Another example: sometime ago I had to patch the V4L2 API to add Bayer format
support to V4L2. This saved me from implementing colorspace conversion in the
sn9c102 driver. The patch was *one line* long and one of its results is that
now someone is working on gnomemeeting/pwlib to implement Bayer-to-* color
conversions that will make thousands of webcams work on Linux with this
application.

Regards,
	Luca Risolia

Re: Linux 2.6.9-ac3

[Gerd Knorr]

Luca Risolia <luca.risolia@studio.unibo.it> writes:

> If it ever happens that this driver is accepted, be also prepared to
> accept patches adding decompression and colorspace conversions for
> every video driver I am aware of, starting from the ones I have already
> written to the ones I'll submit in the future.

Uhm, colorspace conversions and decompression are two different
problems.  Drivers should support at least one of the v4l2 standard
formats.  They might have to do decompression in kernel space to to
that.

colorspace conversion should be the job of the userspace app, I think
everyone agrees on that.  Same goes for any *standard* compression
formats, i.e. webcams which simply use jpeg for example (where the
driver can say "Hey, I'll give you V4L2_PIX_FMT_JPEG" and the decoding
can easily be done using libjpeg.

The corner case are the vendor-specific compressions.  IMHO it doesn't
make much sense to attempt to implement every strange format some
engineer invented in every v4l2 application.  Especially if there is
no free implementation of it (which is the reason the non-gpl pwcx
module was created IIRC).

  Gerd

-- 
return -ENOSIG;

Re: Linux 2.6.9-ac3

[Alan Cox]

On Gwe, 2004-10-22 at 15:10, Gerd Knorr wrote:
> The corner case are the vendor-specific compressions.  IMHO it doesn't
> make much sense to attempt to implement every strange format some
> engineer invented in every v4l2 application.  Especially if there is
> no free implementation of it (which is the reason the non-gpl pwcx
> module was created IIRC).

The pwc formats look like they can be done a lot faster in MMX, which
argues for some format of user space exposure and a set of format idents
for "vendor foo, protocol 0" etc

Re: Linux 2.6.9-ac3

[Gerd Knorr]

On Fri, Oct 22, 2004 at 04:51:23PM +0100, Alan Cox wrote:
> On Gwe, 2004-10-22 at 15:10, Gerd Knorr wrote:
> > The corner case are the vendor-specific compressions.  IMHO it doesn't
> > make much sense to attempt to implement every strange format some
> > engineer invented in every v4l2 application.  Especially if there is
> > no free implementation of it (which is the reason the non-gpl pwcx
> > module was created IIRC).
> 
> The pwc formats look like they can be done a lot faster in MMX, which
> argues for some format of user space exposure and a set of format idents
> for "vendor foo, protocol 0" etc

We'll also need a libv4l2-vendorstuff then (*one* libary for *all* these
vendor formats), otherwise that isn't going to work.  If someone is
willing to create & maintain such a library -- fine with me.  I'll
happily hand out v4l2 vendor format ID's and agree do drop stuff from
kernel space then.  But asking the apps to decode stuff in userspace
without providing a way to do so isn't a good idea.

  Gerd

-- 
return -ENOSIG;

Re: Linux 2.6.9-ac3

[Luca Risolia]

On Fri, 22 Oct 2004 20:14:57 +0200
Gerd Knorr <kraxel@bytesex.org> wrote:

> We'll also need a libv4l2-vendorstuff then (*one* libary for *all* these
> vendor formats), otherwise that isn't going to work.  If someone is
> willing to create & maintain such a library -- fine with me.  I'll
> happily hand out v4l2 vendor format ID's and agree do drop stuff from
> kernel space then.  But asking the apps to decode stuff in userspace
> without providing a way to do so isn't a good idea.

Okay, developers interested in pwc might want to start this library, so
that only a basic driver can be included in the kernel.

Luca Risolia

Re: Linux 2.6.9-ac3

[Alan Cox]

On Gwe, 2004-10-22 at 09:13, Luca Risolia wrote:
> > o       Restore PWC driver                              (Luc Saillard)
> 
> This driver does decompression in kernel space, which is not
> allowed. That part has to be removed from the driver before
> asking for the inclusion in the mainline kernel.

I agree but lets get there a step at a time.

Re: Linux 2.6.9-ac3

[Luca Risolia]

On Fri, 22 Oct 2004 12:09:37 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> On Gwe, 2004-10-22 at 09:13, Luca Risolia wrote:
> > > o       Restore PWC driver                              (Luc Saillard)
> > 
> > This driver does decompression in kernel space, which is not
> > allowed. That part has to be removed from the driver before
> > asking for the inclusion in the mainline kernel.
> 
> I agree but lets get there a step at a time.

This is a step backward.

Re: Linux 2.6.9-ac3

[Alan Cox]

On Gwe, 2004-10-22 at 14:00, Luca Risolia wrote:
> On Fri, 22 Oct 2004 12:09:37 +0100
> > I agree but lets get there a step at a time.
> 
> This is a step backward.

In your opinion only

Re: Linux 2.6.9-ac3

[Bill Davidsen]

Luca Risolia wrote:
>>o       Restore PWC driver                              (Luc Saillard)
> 
> 
> This driver does decompression in kernel space, which is not
> allowed. That part has to be removed from the driver before
> asking for the inclusion in the mainline kernel.

What do you mean by "not allowed?" Clearly it would nice if it were in 
user space, but it would have to be in EVERY user application to be 
useful. We have compression in kernel for ppp, and there's only one 
significant use for that, requiring that every application support every 
vendor hardware makes it a non-scalable NxM problem.

The ideal solution would be to convert vendor format to neutral format 
ala netpbm, and then let the applications handle that format (or a small 
set of formats). It sounds as if this driver is essentially doing that.

This is not hardware which virtually every system includes, so size is 
not a big issue here. I think that CPU hogging is a valid concern, 
perhaps that would be a good thing to address rather than taking the 
"wait a few years for support" approach.

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

Re: Linux 2.6.9-ac3

[Luca Risolia]

On Sat, 23 Oct 2004 10:54:25 -0400
Bill Davidsen <davidsen@tmr.com> wrote:

> Luca Risolia wrote:
> >>o       Restore PWC driver                              (Luc Saillard)
> > 
> > 
> > This driver does decompression in kernel space, which is not
> > allowed. That part has to be removed from the driver before
> > asking for the inclusion in the mainline kernel.
> 
> What do you mean by "not allowed?"

http://marc.theaimsgroup.com/?l=linux-video&m=108627734619978&w=2

Also note how Alan Cox seems not to be actually coherent with his
previous opinions.

 Clearly it would nice if it were in 
> user space, but it would have to be in EVERY user application to be 
> useful. We have compression in kernel for ppp, and there's only one 
> significant use for that, requiring that every application support every 
> vendor hardware makes it a non-scalable NxM problem.

Hmm..What about a common library finally?

Luca Risolia

Re: Linux 2.6.9-ac3

[Bill Davidsen]

Luca Risolia wrote:
> On Sat, 23 Oct 2004 10:54:25 -0400
> Bill Davidsen <davidsen@tmr.com> wrote:
> 
> 
>>Luca Risolia wrote:
>>
>>>>o       Restore PWC driver                              (Luc Saillard)
>>>
>>>
>>>This driver does decompression in kernel space, which is not
>>>allowed. That part has to be removed from the driver before
>>>asking for the inclusion in the mainline kernel.
>>
>>What do you mean by "not allowed?"
> 
> 
> http://marc.theaimsgroup.com/?l=linux-video&m=108627734619978&w=2
> 
> Also note how Alan Cox seems not to be actually coherent with his
> previous opinions.

If you're a Republican he's "wishy-washy", if you're a Democrat he's 
"flexible and adaptable to changing conditions." If you're not a US 
resident you're confused by the previous sentence, please ignore it's an 
in-joke (or tragedy).
> 
>  Clearly it would nice if it were in 
> 
>>user space, but it would have to be in EVERY user application to be 
>>useful. We have compression in kernel for ppp, and there's only one 
>>significant use for that, requiring that every application support every 
>>vendor hardware makes it a non-scalable NxM problem.
> 
> 
> Hmm..What about a common library finally?

That sounds like the eventual solution. A vendor to common format 
conversion library, and with luck someone will be clever and let the 
driver select it in a nice acceptable way. Thought: after open an ioctl 
to tell you which conversion to use? The optimal mechanics are 
inobvious, but I think the library is the right idea.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Linux 2.6.9-ac3

[Luc Saillard]

On Mon, Oct 25, 2004 at 06:22:21PM -0400, Bill Davidsen wrote:
> >Hmm..What about a common library finally?
> 
> That sounds like the eventual solution. A vendor to common format 
> conversion library, and with luck someone will be clever and let the 
> driver select it in a nice acceptable way. Thought: after open an ioctl 
> to tell you which conversion to use? The optimal mechanics are 
> inobvious, but I think the library is the right idea.

It's difficult to create a API that will be use for anyone. I try to be
application developper and want to grab an image from a device. I need to
know the list of webcams, the list of capture devices, ... and want to
display some sliders to control the camera like red,blue,gamma, ...
I want one format even if it's not supported by the driver.

What's the goal of our API ?
 - open,close, enumerate any device (usb,pci, firewire, ...) on the host.
 - convert any format (i'm YetAnotherCameraApp, want a RGB image, but the
   driver only support YUV)
 - can recognize special video source and activate some features.
 - can be use by any program or language ?
 - provide API to change red, blue, balance, gamma, ... without using ioctl.
   Perhaps like Alsa, provinding two levels: - a hardware interface, and a
   software interface.
 - provides some additionals features, like flipping image, change
   colorspace, use MMX ...
 - Please insert any features you want.

The ffmpeg project have already some source code to convert data between
various format without loosing to much information. (Think YUV to RGB, and
RGB to YUV410P).

When i'll finished to support v4l2 for the PWC driver, i'll try to begin a
little library to be use and a sample application. But this takes times ...

Luc