RE: 2.6.16-rc4: known regressions

RE: 2.6.16-rc4: known regressions

[Yu, Luming]

> >Subject    : S3 sleep hangs the second time - 600X
>> >References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
>> >Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
>> >Status     : problematic commit identified,
>> >             further discussion is in the bug
>> 
>> The real problem is there are some bugs hidden by ec_intr=0.
>> ec_intr=1 just get these bug  just exposed, and we need to fix them. 
>
>From a users' point of view, these are regressions from 
>2.6.15, and not 
>all of them might be fixed in time for 2.6.16.
>
>What is a possible short term solution/workaround for 2.6.16?

ec_intr=0 is a reasonable workaround for this box,
if we couldn't root-cause and fix the real problem on time.

>Can we go back to default to polling mode in 2.6.16?
>

No, don't do this.  There are other laptops need this. And I didn't
get regression report that is root-caused to enabling ec_intr=1 by
default. If you argue bug 5989, 6075 could be,  I think
the truth is, for 5989, we need to fix thermal and processor driver
issue.
for 6075, we need to fix interrupt issue.

So far, I don't think we need o fall back.

Thanks,
Luming

.

RE: 2.6.16-rc4: known regressions

[Yu, Luming]

> >Subject    : S3 sleep hangs the second time - 600X
>> >References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
>> >Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
>> >Status     : problematic commit identified,
>> >             further discussion is in the bug
>> 
>> The real problem is there are some bugs hidden by ec_intr=0.
>> ec_intr=1 just get these bug  just exposed, and we need to fix them. 
>
>From a users' point of view, these are regressions from 
>2.6.15, and not 
>all of them might be fixed in time for 2.6.16.
>
>What is a possible short term solution/workaround for 2.6.16?

ec_intr=0 is a reasonable workaround for this box,
if we couldn't root-cause and fix the real problem on time.

>Can we go back to default to polling mode in 2.6.16?
>

No, don't do this.  There are other laptops need this. And I didn't
get regression report that is root-caused to enabling ec_intr=1 by
default. If you argue bug 5989, 6075 could be,  I think
the truth is, for 5989, we need to fix thermal and processor driver
issue.
for 6075, we need to fix interrupt issue.

So far, I don't think we need o fall back.

Thanks,
Luming

.

Re: 2.6.16-rc4: known regressions

[Adrian Bunk]

On Wed, Feb 22, 2006 at 02:55:10PM +0800, Yu, Luming wrote:
> > >Subject    : S3 sleep hangs the second time - 600X
> >> >References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
> >> >Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
> >> >Status     : problematic commit identified,
> >> >             further discussion is in the bug
> >> 
> >> The real problem is there are some bugs hidden by ec_intr=0.
> >> ec_intr=1 just get these bug  just exposed, and we need to fix them. 
> >
> >From a users' point of view, these are regressions from 
> >2.6.15, and not 
> >all of them might be fixed in time for 2.6.16.
> >
> >What is a possible short term solution/workaround for 2.6.16?
> 
> ec_intr=0 is a reasonable workaround for this box,
> if we couldn't root-cause and fix the real problem on time.
> 
> >Can we go back to default to polling mode in 2.6.16?
> 
> No, don't do this.  There are other laptops need this. And I didn't
> get regression report that is root-caused to enabling ec_intr=1 by
> default. If you argue bug 5989, 6075 could be,  I think
> the truth is, for 5989, we need to fix thermal and processor driver
> issue.

We do both agree that defaulting to polling mode is not a long term 
solution.

The question is what to do until it's resolved - assuming that issues 
like 5989 might not be fixed in time for 2.6.16.

Breaking setups working with the defaults under 2.6.15 in 2.6.16 doesn't 
sound that good.

> for 6075, we need to fix interrupt issue.

As far as I understand 6075, the submitter already tried ec_intr=0 
without success.

> So far, I don't think we need o fall back.
> 
> Thanks,
> Luming

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

[RFC: 2.6.16 patch] drivers/acpi/ec.c: default to polling mode for 2.6.16

[Adrian Bunk]

On Wed, Feb 22, 2006 at 01:23:17PM +0100, Adrian Bunk wrote:
> On Wed, Feb 22, 2006 at 02:55:10PM +0800, Yu, Luming wrote:
> > > >Subject    : S3 sleep hangs the second time - 600X
> > >> >References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
> > >> >Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
> > >> >Status     : problematic commit identified,
> > >> >             further discussion is in the bug
> > >> 
> > >> The real problem is there are some bugs hidden by ec_intr=0.
> > >> ec_intr=1 just get these bug  just exposed, and we need to fix them. 
> > >
> > >From a users' point of view, these are regressions from 
> > >2.6.15, and not 
> > >all of them might be fixed in time for 2.6.16.
> > >
> > >What is a possible short term solution/workaround for 2.6.16?
> > 
> > ec_intr=0 is a reasonable workaround for this box,
> > if we couldn't root-cause and fix the real problem on time.
> > 
> > >Can we go back to default to polling mode in 2.6.16?
> > 
> > No, don't do this.  There are other laptops need this. And I didn't
> > get regression report that is root-caused to enabling ec_intr=1 by
> > default. If you argue bug 5989, 6075 could be,  I think
> > the truth is, for 5989, we need to fix thermal and processor driver
> > issue.
> 
> We do both agree that defaulting to polling mode is not a long term 
> solution.
> 
> The question is what to do until it's resolved - assuming that issues 
> like 5989 might not be fixed in time for 2.6.16.
> 
> Breaking setups working with the defaults under 2.6.15 in 2.6.16 doesn't 
> sound that good.
> 
> > for 6075, we need to fix interrupt issue.
> 
> As far as I understand 6075, the submitter already tried ec_intr=0 
> without success.
>...

Let me suggest the following patch for going back to default to polling 
mode in 2.6.16.

The idea is to get this patch into 2.6.16, immediately revert it in 
Len's ACPI tree, and properly fix all issues before 2.6.17.

This way, there will be less regressions when the changed default is in 
a stable kernel.

cu
Adrian


<--  snip  -->


The changed default seems to cause regressions (see 
http://bugzilla.kernel.org/show_bug.cgi?id=5989).

Let's change the default back to polling mode for one more stable 
kernel.

Signed-off-by: Adrian Bunk <bunk@stusta.de>

---

 Documentation/kernel-parameters.txt |    4 ++--
 drivers/acpi/ec.c                   |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

--- linux-2.6.16-rc4-mm2-full/Documentation/kernel-parameters.txt.old	2006-02-25 16:01:51.000000000 +0100
+++ linux-2.6.16-rc4-mm2-full/Documentation/kernel-parameters.txt	2006-02-25 16:02:04.000000000 +0100
@@ -460,8 +460,8 @@
 
 	ec_intr=	[HW,ACPI] ACPI Embedded Controller interrupt mode
 			Format: <int>
-			0: polling mode
-			non-0: interrupt mode (default)
+			0: polling mode (default)
+			non-0: interrupt mode
 
 	eda=		[HW,PS2]
 
--- linux-2.6.16-rc4-mm2-full/drivers/acpi/ec.c.old	2006-02-25 16:00:22.000000000 +0100
+++ linux-2.6.16-rc4-mm2-full/drivers/acpi/ec.c	2006-02-25 16:01:24.000000000 +0100
@@ -73,7 +73,7 @@
 	.class = ACPI_EC_CLASS,
 	.ids = ACPI_EC_HID,
 	.ops = {
-		.add = acpi_ec_intr_add,
+		.add = acpi_ec_poll_add,
 		.remove = acpi_ec_remove,
 		.start = acpi_ec_start,
 		.stop = acpi_ec_stop,
@@ -147,7 +147,7 @@
 
 /* External interfaces use first EC only, so remember */
 static struct acpi_device *first_ec;
-static int acpi_ec_poll_mode = EC_INTR;
+static int acpi_ec_poll_mode = EC_POLL;
 
 /* --------------------------------------------------------------------------
                              Transaction Management

RE: 2.6.16-rc4: known regressions

[Yu, Luming]

>
>Subject    : S3 sleep hangs the second time - 600X
>References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
>Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
>Status     : problematic commit identified,
>             further discussion is in the bug

The real problem is there are some bugs hidden by ec_intr=0.
ec_intr=1 just get these bug  just exposed, and we need to fix them. 

Thanks,
Luming

RE: 2.6.16-rc4: known regressions

[Yu, Luming]

>
>Subject    : S3 sleep hangs the second time - 600X
>References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
>Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
>Status     : problematic commit identified,
>             further discussion is in the bug

The real problem is there are some bugs hidden by ec_intr=0.
ec_intr=1 just get these bug  just exposed, and we need to fix them. 

Thanks,
Luming

RE: 2.6.16-rc4: known regressions

[Yu, Luming]

>
>Subject    : S3 sleep hangs the second time - 600X
>References : http://bugzilla.kernel.org/show_bug.cgi?id=3D5989
>Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
>Status     : problematic commit identified,
>             further discussion is in the bug

The real problem is there are some bugs hidden by ec_intr=3D0.
ec_intr=3D1 just get these bug  just exposed, and we need to fix them.=20

Thanks,
Luming

Re: 2.6.16-rc4: known regressions

[Adrian Bunk]

On Wed, Feb 22, 2006 at 10:39:01AM +0800, Yu, Luming wrote:
> >
> >Subject    : S3 sleep hangs the second time - 600X
> >References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
> >Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
> >Status     : problematic commit identified,
> >             further discussion is in the bug
> 
> The real problem is there are some bugs hidden by ec_intr=0.
> ec_intr=1 just get these bug  just exposed, and we need to fix them. 

>From a users' point of view, these are regressions from 2.6.15, and not 
all of them might be fixed in time for 2.6.16.

What is a possible short term solution/workaround for 2.6.16?
Can we go back to default to polling mode in 2.6.16?

> Thanks,
> Luming

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Linux 2.6.16-rc4

[Linus Torvalds]

Since it's a long weekend in the US, and since I know you'll all be bored 
to tears without a new release to play with, I'm cutting 2.6.16-rc4 a few 
days early.

[ Actually, since -rc2 and -rc3 were both late, we're still off the normal 
  "once a week" release schedule, but let's ignore that ]

The bulk of it is some SCSI driver updates (have you ever seen a diffstat 
without the qla driver taking the top spot? No? Neither have I), with a 
few architectures thrown in, and a number of mostly trivial fixes for good 
measure.

It is, as we say in the business, a "perfect" release. No bugs. No sirree. 
But just to verify that, you should all immediately download and test it. 
Since, quite frankly, what else do you have to do over the weekend?

		Linus

---
adam radford:
      [SCSI] 3ware 9000 driver >4GB memory fix

Adrian Bunk:
      fix a typo in the CPU_H8300H dependencies
      arch/sh/Kconfig: fix the ISA_DMA_API dependencies

Adrian Drzewiecki:
      [BRIDGE]: Fix deadlock in br_stp_disable_bridge

Alan Cox:
      Fix locking error in esp

Alan Stern:
      usb-storage: new unusual_devs entry
      usb-storage: unusual_devs entry
      USB: unusual_devs.h entry: TrekStor i.Beat
      USB: unusual_devs.h entry: iAUDIO M5

Albert D. Cahalan:
      x86: document sysenter path

Albert Lee:
      libata: minor fix for 2.6.16-rc3

Alessandro Zummo:
      Input: ixp4xx-beeper - fix compile error

Andi Kleen:
      x86_64: Update defconfig
      x86_64: Add boot option to disable randomized mappings and cleanup
      x86_64: Don't call do_exit with interrupts disabled after IRET exception
      x86_64: Don't enable ATI apicmaintimer workaround when the machine has C2 or C3
      x86_64: Disable tsc when apicpmtimer is active
      x86_64: Resolve the RIP of an early exception using kallsyms
      x86_64: Relax SRAT covers all memory check a bit
      x86_64: Always pass full number of nodes to NUMA hash computation
      Handle all and empty zones when setting up custom zonelists for mbind

Andreas Herrmann:
      [SCSI] zfcp: get rid of physical_wwpn and physical_s_id
      [SCSI] zfcp: fix adapter erp when link is unplugged
      [SCSI] zfcp: fix: avoid race between fc_remote_port_add and scsi_add_device

Andreas Schwab:
      [IA64] Remove duplicate EXPORT_SYMBOLs

Andrew Morton:
      [APPLETALK]: warning fix
      ide: touch softlockup detector during pio
      swsusp: nuke noisy message
      smctr warning fix
      select: time comparison fixes

Andrew Vasquez:
      [SCSI] qla2xxx: Add port-speed FC transport attribute.
      [SCSI] qla2xxx: Add host port-type FC transport attribute.
      [SCSI] qla2xxx: Add host-statistics FC transport attributes.
      [SCSI] qla2xxx: Add beacon support via class-device attribute.
      [SCSI] qla2xxx: Return correct data-len during NVRAM retrieval.
      [SCSI] qla2xxx: Add support to retrieve/update HBA option-rom.
      qla2xxx: Close window on race between rport removal and fcport transition.
      qla2xxx: Remove bogus debug-code.
      qla2xxx: Pass input-buffer length to Get-ID-List mailbox command.
      qla2xxx: Correct lun assignment during IOCB submission.

Arthur Othieno:
      Input: kill remnants of 98kbd{,-io} and 98spkr
      [SERIAL] Documentation/jsm.txt is a no show.

Ashok Raj:
      [IA64] Dont set NR_CPUS for cpu_possible_map when CPU hotplug is enabled.
      [IA64] Count disabled cpus as potential hot-pluggable CPUs
      [IA64] Count disabled cpus as potential hot-pluggable CPUs

Atsushi Nemoto:
      [MIPS] Rewrite get_wchan and its helper functions using kallsyms_lookup.
      [MIPS] Add protected_blast_icache_range, blast_icache_range, etc.
      [MIPS] Fix typo in _sys32_rt_sigreturn and _sysn32_rt_sigreturn.

Benjamin LaHaise:
      kbuild: revert "fix make -jN with multiple targets with O=..."

Bjorn Helgaas:
      HPET: handle multiple ACPI EXTENDED_IRQ resources
      mmconfig: add kernel parameter documentation
      ACPI: fix vendor resource length computation

Brian King:
      [SCSI] ipr: Fix adapter initialization failure

Chen, Kenneth W:
      sched: revert "filter affine wakeups"

Chris Wright:
      sys_mbind sanity checking

Christian Lindner:
      USB: PL2303: Leadtek 9531 GPS-Mouse

Christian Trefzer:
      neofb: avoid resetting display config on unblank
      neofb: avoid resetting display config on unblank (v2)

Chuck Ebbert:
      i386: fix singlestepping though a syscall

Cornelia Huck:
      s390: ccw device disbanding
      s390: fix assignment instead of check in ccw_device_set_online()

Dan Williams:
      wireless/atmel: fix setting TX key only in ENCODEEXT
      wireless/atmel: fix Open System authentication process bugs
      Necessary evil to get sata_vsc to initialize with Intel iq3124h hba

Daniel Yeisley:
      x86_64: early initialization of cpu_to_node

Dave Jones:
      [CPUFREQ] cpufreq_notify_transition cleanup.
      [CPUFREQ] Whitespace/CodingStyle cleanups
      Remove "RV370 5B60 [Radeon X300 (PCIE)]" from DRI list
      [ATM]: Ratelimit atmsvc failure messages
      [IPV4] ICMP: Invert default for invalid icmp msgs sysctl
      [P8023]: Fix tainting of kernel.

David Brownell:
      USB: fix up the usb early handoff logic for EHCI
      USB: sl811_cs needs platform_device conversion too
      Input: ads7846 - assorted updates

David Gibson:
      powerpc: Fix accidentally-working typo in __pud_free_tlb

David Howells:
      FRV: Miscellaneous fixes
      FRV: Use virtual interrupt disablement

David S. Miller:
      [SPARC]: sys_newfstatat --> sys_fstatat64
      [NET]: Revert skb_copy_datagram_iovec() recursion elimination.

Dean Nelson:
      [IA64-SGI] enforce proper ordering of callouts by XPC

Dean Roe:
      [IA64-SGI] fix the size of __sn_cnodeid_to_nasid

Dmitry Torokhov:
      Input: trackpoint - enable devices connected to external port
      Input: ads7846 - convert to to dynamic input_dev allocation

Francois Romieu:
      sis190: early setting of the pci driver private data

Frank Pavlic:
      s390: lcs performance enhancements
      s390: some qeth driver fixes

Gerald Britton:
      x86: fix oprofile kernel callgraph regression

Harald Welte:
      [NETFILTER] Fix Kconfig menu level for x_tables

hawkes@sgi.com:
      [IA64] ia64: simplify and fix udelay()

Heiko Carstens:
      s390: fix __delay implementation
      s390: fix preempt_count of idle thread with cpu hotplug
      s390: additional_cpus parameter
      s390: possible_cpus parameter
      s390: smp initialization speed
      s390: sys32_fstatat -> sys32_fstatat64

Herbert Poetzl:
      [ARM] remove duplicate #includes

Herbert Xu:
      [IPSEC]: Fix strange IPsec freeze.

Horms:
      [IA64] support panic_on_oops sysctl

Hugh Dickins:
      compound page: use page[1].lru
      compound page: default destructor
      compound page: no access_process_vm check

Ingo Molnar:
      hrtimer: round up relative start time on low-res arches
      Introduce CONFIG_DEFAULT_MIGRATION_COST

Jack Steiner:
      [IA64] Missing check for TIF_WORK if trace/audit enabled

Jamal Hadi Salim:
      [NETLINK] genetlink: Fix bugs spotted by Andrew Morton.

James Bottomley:
      add scsi_execute_in_process_context() API
      [SCSI] fix wrong context bugs in SCSI
      fix x86 topology export in sysfs for subarchitectures

Jan Beulich:
      x86_64: make touch_nmi_watchdog() not touch impossible cpus' private data

Jay Vosburgh:
      bonding: fix a locking bug in bond_release

Jean Delvare:
      vt8231: Fix sysfs temperature interface
      w83781d: Use real-time status registers
      w83627hf: Document the reset module parameter
      it87: Fix oops on removal
      i2c: Drop outdated probe/remove code in i2c-isa

Jean Tourrilhes:
      Wavelan_cs bitfield fixes

Jeff Mahoney:
      reiserfs: fix potential (unlikely) oops in reiserfs_get_acl

Jens Axboe:
      [SCSI] gdth: don't map zero-length requests
      Add missing FUA write to sata_mv dma command list

Jes Sorensen:
      [IA64-SGI] sn2 minor fixes and cleanups
      [IA64] remove obsolete corporate address
      [IA64-SGI] remove compile time warning

Jim Keniston:
      kprobes: Update Documentation/kprobes.txt

Joe Perches:
      [IRDA]: Ratelimit messages.

Johannes Berg:
      allow windfarm_pm112 module to load

Joshua Giles:
      [SCSI] megaraid_sas: register 16 byte CDB capability

Joshua Kinard:
      Fix SGI O2 compile error in drivers/video/gbefb.c

Ju, Seokmann:
      [SCSI] megaraid_legacy: kobject_register failure

Karsten Keil:
      Fix NULL pointer dereference in isdn_tty_at_cout

Kurt Hackel:
      ocfs2: recheck recovery state after getting lock
      ocfs2: fix release of ast never reserved
      ocfs2: add dlm_wait_for_node_death
      ocfs2: manually grant remote recovery lock
      ocfs2: detach from heartbeat events before freeing mle

Kyle McMartin:
      [PARISC] Convert ccio-dma.c to use seq_file
      [PARISC] Convert sba_iommu.c to use seq_file
      [PARISC] Stub out pselect6/ppoll until TIF_RESTORE_SIGMASK is done
      sys_newfstatat -> sys_fstatat64

Linus Torvalds:
      Handle holes in node mask in node fallback list setup
      Linux v2.6.16-rc4

Maciej W. Rozycki:
      [MIPS] Fix CPU type bitmasks for MIPS III, IV and V.

Marcel Holtmann:
      [Bluetooth] Reduce L2CAP MTU for RFCOMM connections
      [Bluetooth] Fix NULL pointer dereferences of the HCI socket
      [Bluetooth] Fix firmware loading problem of BT3C driver

Marcel Selhorst:
      Infineon TPM: IO-port leakage fix, WTX-bugfix

Mark Fasheh:
      jbd: revert checkpoint list changes
      ocfs2: only checkpoint journal when asked to

Mark Haverkamp:
      [SCSI] aacraid: reduce device probe warnings
      [SCSI] aacraid: Update global function names
      [SCSI] aacraid: use no_uld_attach flag

Mark Maule:
      [IA64-SGI] export sn_pcidev_info_get

Martin Michlmayr:
      [ARM] 3337/1: Fix NSLU2 flash support according to window size configuration patch

Matthew Wilcox:
      [SCSI] sym2: Mask off opcode from RBC

Maxim Shchetynin:
      [SCSI] zfcp: fix logging during device reset

Meelis Roos:
      Input: logips2pp - add new signature (99)

Michael Hund:
      USB: add new device ids to ldusb
      USB: change ldusb's experimental state

Michael S. Tsirkin:
      IPoIB: Don't start send-only joins while multicast thread is stopped
      IPoIB: Fix another send-only join race
      madvise MADV_DONTFORK/MADV_DOFORK
      add asm-generic/mman.h

Mike Christie:
      [SCSI] iscsi update: cleanup iscsi class interface
      [SCSI] iscsi update: pass correct skb to skb_trim
      [SCSI] iscsi update: setup pool before using
      [SCSI] iscsi update: set deamon pid earlier
      [SCSI] iscsi update: rm conn lock
      [SCSI] iscsi update: set correct state at creation time
      [SCSI] iscsi update: fix mgmt pool err path release
      [SCSI] iscsi update: use gfp_t
      [SCSI] iscsi update: rm unused sessions list

Miklos Szeredi:
      fuse: fix bug in aborted fuse_release_end()

Moore, Eric:
      [SCSI] fusion - mptctl - MPTCOMMAND - adding function types.
      [SCSI] fusion - mptctl - adding support for bus_type=SAS
      [SCSI] fusion - mtctl - change to wait_event_timeout
      [SCSI] fusion - mptctl - Event Log Fix
      [SCSI] fusion - mptctl -sense width fix
      [SCSI] fusion - mptctl - backplane istwi fix
      [SCSI] fusion - mptctl -firmware download fix
      [SCSI] fusion - mptctl -adding asyn event notification support

NeilBrown:
      Fix over-zealous tag clearing in radix_tree_delete

Nicolas DICHTEL:
      [IPV6] Don't store dst_entry for RAW socket

Nicolas Pitre:
      [ARM] 3338/1: old ABI compat: sys_socketcall
      [ARM] 3339/1: ARM EABI: make unmuxed syscalls visible

Oleg Nesterov:
      fix kill_proc_info() vs CLONE_THREAD race
      fix kill_proc_info() vs fork() theoretical race
      fix zap_thread's ptrace related problems

Patrick McHardy:
      [NETFILTER]: Fix xfrm lookup after SNAT
      [XFRM]: Fix SNAT-related crash in xfrm4_output_finish
      [NETFILTER]: Don't invoke okfn in CONFIG_NETFILTER=n variant of nf_hook()

Paul Fulghum:
      tty reference count fix

Paul Jackson:
      cpuset: oops in exit on null cpuset fix

Paul Mackerras:
      Provide an interface for getting the current tick length

Peter Osterlund:
      pktcdvd: Don't spam the kernel log when nothing is wrong
      pktcdvd: Allow non-writable media to be mounted
      pktcdvd: Don't unlock the door if the disc is in use
      pktcdvd: Reduce stack usage

Peter Staubach:
      fix deadlock in ext2

Phil Dibowitz:
      USB: unusual-devs bugfix

Rafael J. Wysocki:
      swsusp: fix breakage with swap on LVM

Ralf Baechle:
      [MIPS] RM200: Give RM200 it's own timex.h.
      [MIPS] Update docs to reflect the latest status of the Alchemy IDE driver.
      [MIPS] Fold non-__mips64 case into CONFIG_32BIT case.
      [MIPS] Remove commented out function prom_build_cpu_map.
      [MIPS] More uaccess.h fixes with gcc >= 4.0.1.
      [MIPS] Get rid of kludgery needed to keep stdargs of old compilers working.
      [MIPS] MT: Fix c0 back-to-back hazard.
      [MIPS] MT: Propagate config7 into VPE.
      [SERIAL] Fix typo in comment

Ralph Campbell:
      IB/mad: Handle DR SMPs with a LID routed part

Roland Dreier:
      IB/mthca: Don't print debugging info until we have all values
      IPoIB: Yet another fix for send-only joins
      IB/mthca: bump driver version and release date

Roman Zippel:
      hrtimer: fix multiple macro argument expansion

Russell King:
      [ARM] Fix SMP initialisation oops
      [MMC] mmci: allow small data transfers

Stephen Hemminger:
      [BRIDGE]: Better fix for netfilter missing symbol has_bridge_parent
      sk98lin: no d-link support (kconfig)
      skge: no longer experimental
      skge: speed setting
      sky2: speed setting fix

Steve French:
      CIFS: fix cifs_user_read oops when null SMB response on forcedirectio mount

Sumant Patro:
      [SCSI] megaraid_sas: support for 1078 type controller added

Thomas Koeller:
      [MIPS] RM9000: Fix buggy I-cache workaround.

Thomas Meyer:
      x86: gitignore some autogenerated files for i386

Thomas Renninger:
      [CPUFREQ] Check for not initialized freq on cpufreq changes
      [CPUFREQ] Check whether driver init did not initialize current freq

Tim Hockin:
      Remove KERN_INFO from middle of printk line

Trond Myklebust:
      NLM: Fix the NLM_GRANTED callback checks

Yasuyuki Kozakai:
      [NETFILTER]: x_tables: fix dependencies of conntrack related modules
      [NETFILTER]: nf_conntrack: move registration of __nf_ct_attach
      [NETFILTER]: nf_conntrack: attach conntrack to TCP RST generated by ip6t_REJECT
      [NETFILTER]: nf_conntrack: attach conntrack to locally generated ICMPv6 error
      [NETFILTER]: nf_conntrack: Fix TCP/UDP HW checksum handling for IPv6 packet

Yoichi Yuasa:
      MIPS 32bit machines need fstatat64 support.

2.6.16-rc4: known regressions

[Adrian Bunk]

This email lists some known regressions in 2.6.16-rc4 compared to 2.6.15.

If you find your name in the Cc header, you are either submitter of one 
of the bugs, maintainer of an affectected subsystem or driver, a patch 
of you was declared guilty for a breakage or I'm considering you in any 
other way possibly involved with one or more of these issues.


Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
Submitter  : John Stultz <johnstul@us.ibm.com>
Status     : still present in -git two days ago


Subject    : S3 sleep hangs the second time - 600X
References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
Status     : problematic commit identified,
             further discussion is in the bug

Subject    : psmouse starts losing sync in 2.6.16-rc2
References : http://lkml.org/lkml/2006/2/5/50
Submitter  : Meelis Roos <mroos@linux.ee>
Handled-By : Dmitry Torokhov <dmitry.torokhov@gmail.com>
Status     : Dmitry: Working on various manifestations of this one.
                     At worst we will have to disable resync by default
                     before 2.6.16 final is out and continue in 2.6.17 cycle.



cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

2.6.16-rc4: known regressions

[Adrian Bunk]

This email lists some known regressions in 2.6.16-rc4 compared to 2.6.15.

If you find your name in the Cc header, you are either submitter of one 
of the bugs, maintainer of an affectected subsystem or driver, a patch 
of you was declared guilty for a breakage or I'm considering you in any 
other way possibly involved with one or more of these issues.


Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
Submitter  : John Stultz <johnstul@us.ibm.com>
Status     : still present in -git two days ago


Subject    : S3 sleep hangs the second time - 600X
References : http://bugzilla.kernel.org/show_bug.cgi?id=5989
Submitter  : Sanjoy Mahajan <sanjoy@mrao.cam.ac.uk>
Status     : problematic commit identified,
             further discussion is in the bug

Subject    : psmouse starts losing sync in 2.6.16-rc2
References : http://lkml.org/lkml/2006/2/5/50
Submitter  : Meelis Roos <mroos@linux.ee>
Handled-By : Dmitry Torokhov <dmitry.torokhov@gmail.com>
Status     : Dmitry: Working on various manifestations of this one.
                     At worst we will have to disable resync by default
                     before 2.6.16 final is out and continue in 2.6.17 cycle.



cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
> Submitter  : John Stultz <johnstul@us.ibm.com>
> Status     : still present in -git two days ago

This is not ppc only. I have the exact same problem on Gentoo
Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
Haven't had the time to investigate further yet, sorry.

                          Pekka

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> References : http://bugzilla.kernel.org/show_bug.cgi?id=3D6021
> Submitter  : John Stultz <johnstul@us.ibm.com>
> Status     : still present in -git two days ago

This is not ppc only. I have the exact same problem on Gentoo
Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
Haven't had the time to investigate further yet, sorry.

                          Pekka

Re: 2.6.16-rc4: known regressions

[Adrian Bunk]

On Sun, Feb 19, 2006 at 01:06:45PM +0200, Pekka Enberg wrote:
> On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> > Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> > References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
> > Submitter  : John Stultz <johnstul@us.ibm.com>
> > Status     : still present in -git two days ago
> 
> This is not ppc only. I have the exact same problem on Gentoo
> Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
> Haven't had the time to investigate further yet, sorry.

Thanks for this information.

Robert, can you look at this problem?

>                           Pekka

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> > > Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> > > References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
> > > Submitter  : John Stultz <johnstul@us.ibm.com>
> > > Status     : still present in -git two days ago

On Sun, Feb 19, 2006 at 01:06:45PM +0200, Pekka Enberg wrote:
> > This is not ppc only. I have the exact same problem on Gentoo
> > Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
> > Haven't had the time to investigate further yet, sorry.

On Sun, 2006-02-19 at 15:54 +0100, Adrian Bunk wrote:
> Thanks for this information.
> 
> Robert, can you look at this problem?

I am doing git bisect now. Will post complete results when I have them.
Here's current git bisect log if that's any help:

penberg ~/devel/kernel/2.6-git 2 git bisect log
git-bisect start
# bad: [f3bcf72eb85aba88a7bd0a6116dd0b5418590dbe] Linux v2.6.16-rc1
git-bisect bad f3bcf72eb85aba88a7bd0a6116dd0b5418590dbe
# good: [dab47a31f42a23d2b374e1cd7d0b797e8e08b23d] Linux v2.6.15
git-bisect good dab47a31f42a23d2b374e1cd7d0b797e8e08b23d
# bad: [64ca9004b819ab87648dbfc78f3ef49ee491343e] Make vm86 support optional
git-bisect bad 64ca9004b819ab87648dbfc78f3ef49ee491343e
# bad: [0a75c23a009ff65f651532cecc16675d05f4de37] Merge with http://kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
git-bisect bad 0a75c23a009ff65f651532cecc16675d05f4de37
# bad: [d779188d2baf436e67fe8816fca2ef53d246900f] Merge branch 'upstream-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/netdev-2.6
git-bisect bad d779188d2baf436e67fe8816fca2ef53d246900f

			Pekka

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> > > Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> > > References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
> > > Submitter  : John Stultz <johnstul@us.ibm.com>
> > > Status     : still present in -git two days ago

On Sun, Feb 19, 2006 at 01:06:45PM +0200, Pekka Enberg wrote:
> > This is not ppc only. I have the exact same problem on Gentoo
> > Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
> > Haven't had the time to investigate further yet, sorry.

Here's the result of git bisect:

ba9dc657af86d05d2971633e57d1f6f94ed60472 is first bad commit
diff-tree ba9dc657af86d05d2971633e57d1f6f94ed60472 (from 733260ff9c45bd4db60f45d17e8560a4a68dff4d)
Author: Greg Kroah-Hartman <gregkh@suse.de>
Date:   Wed Nov 16 13:41:28 2005 -0800

    [PATCH] USB: allow usb drivers to disable dynamic ids

    This lets drivers, like the usb-serial ones, disable the ability to add
    ids from sysfs.

    The usb-serial drivers are "odd" in that they are really usb-serial bus
    drivers, not usb bus drivers, so the dynamic id logic will have to go
    into the usb-serial bus core for those drivers to get that ability.

    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

:040000 040000 ed98c56f9d575c69ff8d590f336ab259be360230 1ffa39f0d0afdf7deda0cf4270f29fa6af1a5d5c M      drivers
:040000 040000 cae5649115b1ea49c732bc940009161f222042af c6bb3c2357a4bfe3dab8bc900a38b4ea344207cd M      include

Please note that with the above changeset, hal and udev daemons refuse
to start up during boot so I don't even have /dev/sda2 when plugging in
ipod. Earlier in the bisect (and with plain 2.6.16-rc1), both daemons
started up fine and the device file appeared, but no icon appeared at
Gnome desktop. Debugging that with /usr/bin/ipod, there were no hal
events in the log. In any case, it certainly looks like the USB merge
broke my setup.

I have included the full git bisect log and my config below.

			Pekka

git-bisect start
# bad: [f3bcf72eb85aba88a7bd0a6116dd0b5418590dbe] Linux v2.6.16-rc1
git-bisect bad f3bcf72eb85aba88a7bd0a6116dd0b5418590dbe
# good: [dab47a31f42a23d2b374e1cd7d0b797e8e08b23d] Linux v2.6.15
git-bisect good dab47a31f42a23d2b374e1cd7d0b797e8e08b23d
# bad: [64ca9004b819ab87648dbfc78f3ef49ee491343e] Make vm86 support optional
git-bisect bad 64ca9004b819ab87648dbfc78f3ef49ee491343e
# bad: [0a75c23a009ff65f651532cecc16675d05f4de37] Merge with http://kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
git-bisect bad 0a75c23a009ff65f651532cecc16675d05f4de37
# bad: [d779188d2baf436e67fe8816fca2ef53d246900f] Merge branch 'upstream-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/netdev-2.6
git-bisect bad d779188d2baf436e67fe8816fca2ef53d246900f
# bad: [d347da0deffa1d8f88f0d270eab040e4707c9916] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6
git-bisect bad d347da0deffa1d8f88f0d270eab040e4707c9916
# bad: [c6c88bbde4d8b2ffe9886b7130b2e23781d424e5] Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb-2.6
git-bisect bad c6c88bbde4d8b2ffe9886b7130b2e23781d424e5
# bad: [95f209f93663103db2a8fb989e226ac68a98b036] USB: pl2303_update_line_status data length fix
git-bisect bad 95f209f93663103db2a8fb989e226ac68a98b036
# bad: [ba9dc657af86d05d2971633e57d1f6f94ed60472] USB: allow usb drivers to disable dynamic ids
git-bisect bad ba9dc657af86d05d2971633e57d1f6f94ed60472
# good: [baefbc39d8e23942cc10db92f5bc42e3476f6bc1] USB: wakeup flag updates (2/3) uhci-hcd
git-bisect good baefbc39d8e23942cc10db92f5bc42e3476f6bc1
# good: [8364d6b0be2dbbf162c6aea79615b5025a0d67c2] USB: dummy_hcd: rename variables
git-bisect good 8364d6b0be2dbbf162c6aea79615b5025a0d67c2
# good: [5ba35bd8f9a4fa6b92ef707826c47a1466ece460] USB: make bias writeable in libusual
git-bisect good 5ba35bd8f9a4fa6b92ef707826c47a1466ece460
# good: [733260ff9c45bd4db60f45d17e8560a4a68dff4d] USB: add dynamic id functionality to USB core
git-bisect good 733260ff9c45bd4db60f45d17e8560a4a68dff4d

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.15
# Sun Feb 19 22:30:09 2006
#
CONFIG_X86_32=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
# CONFIG_CLEAN_COMPILE is not set
CONFIG_BROKEN=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32

#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_HOTPLUG=y
CONFIG_KOBJECT_UEVENT=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y

#
# Block layer
#
# CONFIG_LBD is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
# CONFIG_SMP is not set
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=m
CONFIG_X86_MCE_P4THERMAL=y
CONFIG_TOSHIBA=m
CONFIG_I8K=m
# CONFIG_X86_REBOOTFIXUPS is not set
CONFIG_MICROCODE=m
CONFIG_X86_MSR=m
# CONFIG_X86_CPUID is not set

#
# Firmware Drivers
#
CONFIG_EDD=m
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
# CONFIG_SPARSEMEM_STATIC is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
# CONFIG_REGPARM is not set
# CONFIG_SECCOMP is not set
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_PHYSICAL_START=0x100000
# CONFIG_KEXEC is not set

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
# CONFIG_PM_DEBUG is not set
CONFIG_SOFTWARE_SUSPEND=y
CONFIG_PM_STD_PARTITION=""

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
# CONFIG_ACPI_SLEEP is not set
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
# CONFIG_ACPI_HOTKEY is not set
CONFIG_ACPI_FAN=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_THERMAL=m
# CONFIG_ACPI_ASUS is not set
CONFIG_ACPI_IBM=m
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
# CONFIG_X86_PM_TIMER is not set
# CONFIG_ACPI_CONTAINER is not set

#
# APM (Advanced Power Management) BIOS Support
#
# CONFIG_APM is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCIEPORTBUS is not set
# CONFIG_PCI_MSI is not set
# CONFIG_PCI_LEGACY_PROC is not set
# CONFIG_PCI_DEBUG is not set
CONFIG_ISA_DMA_API=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set

#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_AOUT is not set
# CONFIG_BINFMT_MISC is not set

#
# Networking
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
# CONFIG_XFRM_USER is not set
CONFIG_NET_KEY=y
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=y
CONFIG_INET_ESP=y
CONFIG_INET_IPCOMP=y
CONFIG_INET_TUNNEL=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_BIC=y

#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
CONFIG_IPV6=y
# CONFIG_IPV6_PRIVACY is not set
# CONFIG_INET6_AH is not set
# CONFIG_INET6_ESP is not set
# CONFIG_INET6_IPCOMP is not set
# CONFIG_INET6_TUNNEL is not set
# CONFIG_IPV6_TUNNEL is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set

#
# Core Netfilter Configuration
#
# CONFIG_NETFILTER_NETLINK is not set

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
# CONFIG_IP_NF_CT_ACCT is not set
# CONFIG_IP_NF_CONNTRACK_MARK is not set
# CONFIG_IP_NF_CONNTRACK_EVENTS is not set
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
# CONFIG_IP_NF_NETBIOS_NS is not set
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
# CONFIG_IP_NF_PPTP is not set
CONFIG_IP_NF_QUEUE=m
# CONFIG_IP_NF_IPTABLES is not set
# CONFIG_IP_NF_ARPTABLES is not set

#
# IPv6: Netfilter Configuration (EXPERIMENTAL)
#
CONFIG_IP6_NF_QUEUE=m
# CONFIG_IP6_NF_IPTABLES is not set

#
# DCCP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP is not set

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
CONFIG_VLAN_8021Q=m
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set

#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_IEEE80211 is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set

#
# Connector - unified userspace <-> kernelspace linker
#
CONFIG_CONNECTOR=m

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
CONFIG_PARPORT_NOT_PC=y
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_1284=y

#
# Plug and Play support
#
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_PNPACPI=y

#
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=8192
CONFIG_BLK_DEV_INITRD=y
# CONFIG_CDROM_PKTCDVD is not set
# CONFIG_ATA_OVER_ETH is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
# CONFIG_IDEDISK_MULTI_MODE is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
CONFIG_BLK_DEV_IDEFLOPPY=m
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
# CONFIG_BLK_DEV_IDEPNP is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
CONFIG_BLK_DEV_ALI15X3=y
# CONFIG_WDC_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_CS5535 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_IT821X is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set

#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set

#
# SCSI low-level drivers
#
# CONFIG_ISCSI_TCP is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_AHCI is not set
# CONFIG_SCSI_SATA_SVW is not set
CONFIG_SCSI_ATA_PIIX=m
# CONFIG_SCSI_SATA_MV is not set
# CONFIG_SCSI_SATA_NV is not set
# CONFIG_SCSI_PDC_ADMA is not set
# CONFIG_SCSI_SATA_QSTOR is not set
# CONFIG_SCSI_SATA_PROMISE is not set
# CONFIG_SCSI_SATA_SX4 is not set
# CONFIG_SCSI_SATA_SIL is not set
# CONFIG_SCSI_SATA_SIL24 is not set
# CONFIG_SCSI_SATA_SIS is not set
# CONFIG_SCSI_SATA_ULI is not set
# CONFIG_SCSI_SATA_VIA is not set
# CONFIG_SCSI_SATA_VITESSE is not set
CONFIG_SCSI_SATA_INTEL_COMBINED=y
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA24XX is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set

#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_SPI is not set
# CONFIG_FUSION_FC is not set
# CONFIG_FUSION_SAS is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Network device support
#
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m
# CONFIG_NET_SB1000 is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# PHY device support
#
# CONFIG_PHYLIB is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
# CONFIG_NET_VENDOR_3COM is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
CONFIG_NATSEMI=m
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_NET_POCKET is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set

#
# Ethernet (10000 Mbit)
#
# CONFIG_CHELSIO_T1 is not set
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
CONFIG_GAMEPORT=m
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_FM801 is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_NONSTANDARD=y
# CONFIG_COMPUTONE is not set
# CONFIG_ROCKETPORT is not set
# CONFIG_CYCLADES is not set
# CONFIG_DIGIEPCA is not set
# CONFIG_MOXA_INTELLIO is not set
# CONFIG_MOXA_SMARTIO is not set
# CONFIG_ISI is not set
# CONFIG_SYNCLINK is not set
# CONFIG_SYNCLINKMP is not set
# CONFIG_N_HDLC is not set
# CONFIG_RISCOM8 is not set
# CONFIG_SPECIALIX is not set
# CONFIG_SX is not set
# CONFIG_RIO is not set
# CONFIG_STALDRV is not set

#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
# CONFIG_PPDEV is not set
# CONFIG_TIPAR is not set

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=m
# CONFIG_AGP_ALI is not set
CONFIG_AGP_ATI=m
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not set
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=m
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set

#
# TPM devices
#
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set

#
# I2C support
#
CONFIG_I2C=y
# CONFIG_I2C_CHARDEV is not set

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=y
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_PARPORT is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
# CONFIG_I2C_VOODOO3 is not set
# CONFIG_I2C_PCA_ISA is not set

#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_RTC_X1205_I2C is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Hardware Monitoring support
#
CONFIG_HWMON=y
# CONFIG_HWMON_VID is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia Capabilities Port drivers
#

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_MACMODES is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_VESA is not set
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_I810 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON_OLD is not set
CONFIG_FB_RADEON=y
CONFIG_FB_RADEON_I2C=y
# CONFIG_FB_RADEON_DEBUG is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_CYBLA is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y

#
# Logo configuration
#
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not set
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Sound
#
CONFIG_SOUND=m

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_AC97_BUS=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_GENERIC_DRIVER=y

#
# Generic devices
#
CONFIG_SND_MPU401_UART=m
CONFIG_SND_DUMMY=m
CONFIG_SND_VIRMIDI=m
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# PCI devices
#
CONFIG_SND_ALI5451=m
CONFIG_SND_ATIIXP=m
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
CONFIG_SND_TRIDENT=m
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
# CONFIG_SND_INTEL8X0 is not set
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_HDA_INTEL is not set

#
# USB devices
#
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_USX2Y is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_BANDWIDTH is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_ISP116X_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_BIG_ENDIAN is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_SL811_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_OBSOLETE_OSS_USB_DRIVER is not set
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m

#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#

#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
# CONFIG_USB_STORAGE_USBAT is not set
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
# CONFIG_USB_LIBUSUAL is not set

#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
# CONFIG_HID_FF is not set
CONFIG_USB_HIDDEV=y

#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_ACECAD is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_ITMTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_YEALINK is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set
# CONFIG_USB_KEYSPAN_REMOTE is not set
# CONFIG_USB_APPLETOUCH is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set

#
# Video4Linux support is needed for USB Multimedia device support
#

#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=m
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_RNDIS_HOST is not set
# CONFIG_USB_NET_CDC_SUBSET is not set
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_MON=y

#
# USB port drivers
#
CONFIG_USB_USS720=m

#
# USB Serial Converter support
#
# CONFIG_USB_SERIAL is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
CONFIG_USB_AUERSWALD=m
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
CONFIG_USB_LCD=m
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETKIT is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TEST is not set

#
# USB DSL modem support
#

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# MMC/SD Card support
#
# CONFIG_MMC is not set

#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set

#
# SN Devices
#

#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_FS_POSIX_ACL is not set
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_INOTIFY=y
# CONFIG_QUOTA is not set
CONFIG_DNOTIFY=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=y
# CONFIG_FUSE_FS is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=m
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
# CONFIG_RELAYFS_FS is not set

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
CONFIG_HFSPLUS_FS=m
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
# CONFIG_NFS_FS is not set
# CONFIG_NFSD is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
# CONFIG_9P_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m

#
# Instrumentation Support
#
CONFIG_PROFILING=y
CONFIG_OPROFILE=m
CONFIG_KPROBES=y

#
# Kernel hacking
#
# CONFIG_PRINTK_TIME is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_LOG_BUF_SHIFT=14
# CONFIG_DETECT_SOFTLOCKUP is not set
# CONFIG_SCHEDSTATS is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_PREEMPT is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_DEBUG_VM is not set
CONFIG_FRAME_POINTER=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set

#
# Page alloc debug is incompatible with Software Suspend on i386
#
CONFIG_4KSTACKS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_TGR192 is not set
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
# CONFIG_CRYPTO_AES_586 is not set
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
# CONFIG_CRYPTO_TEA is not set
CONFIG_CRYPTO_ARC4=m
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
# CONFIG_CRYPTO_TEST is not set

#
# Hardware crypto devices
#
# CONFIG_CRYPTO_DEV_PADLOCK is not set

#
# Library routines
#
# CONFIG_CRC_CCITT is not set
# CONFIG_CRC16 is not set
CONFIG_CRC32=y
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_BIOS_REBOOT=y

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Sun, Feb 19, 2006 at 11:14:13PM +0200, Pekka Enberg wrote:
> On 2/18/06, Adrian Bunk <bunk@stusta.de> wrote:
> > > > Subject    : gnome-volume-manager broken on powerpc since 2.6.16-rc1
> > > > References : http://bugzilla.kernel.org/show_bug.cgi?id=6021
> > > > Submitter  : John Stultz <johnstul@us.ibm.com>
> > > > Status     : still present in -git two days ago
> 
> On Sun, Feb 19, 2006 at 01:06:45PM +0200, Pekka Enberg wrote:
> > > This is not ppc only. I have the exact same problem on Gentoo
> > > Linux/x86. No ipod events on 2.6.16-rc1, whereas 2.6.15 works fine.
> > > Haven't had the time to investigate further yet, sorry.
> 
> Here's the result of git bisect:
> 
> ba9dc657af86d05d2971633e57d1f6f94ed60472 is first bad commit
> diff-tree ba9dc657af86d05d2971633e57d1f6f94ed60472 (from 733260ff9c45bd4db60f45d17e8560a4a68dff4d)
> Author: Greg Kroah-Hartman <gregkh@suse.de>
> Date:   Wed Nov 16 13:41:28 2005 -0800
> 
>     [PATCH] USB: allow usb drivers to disable dynamic ids
> 
>     This lets drivers, like the usb-serial ones, disable the ability to add
>     ids from sysfs.
> 
>     The usb-serial drivers are "odd" in that they are really usb-serial bus
>     drivers, not usb bus drivers, so the dynamic id logic will have to go
>     into the usb-serial bus core for those drivers to get that ability.
> 
>     Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
> 
> :040000 040000 ed98c56f9d575c69ff8d590f336ab259be360230 1ffa39f0d0afdf7deda0cf4270f29fa6af1a5d5c M      drivers
> :040000 040000 cae5649115b1ea49c732bc940009161f222042af c6bb3c2357a4bfe3dab8bc900a38b4ea344207cd M      include
> 
> Please note that with the above changeset, hal and udev daemons refuse
> to start up during boot so I don't even have /dev/sda2 when plugging in
> ipod.

That's _really_ odd, as hal, udev and dbus all work just fine on my
machines with the above changeset (actually with 2.6.16-rc4).  And that
changeset should not have caused anything to change with regards to the
core uevent code, as it's a usb-serial change only.

And you don't even have CONFIG_USB_SERIAL enabled...  Very odd.

If you revert this one patch, on top of a clean 2.6.16-rc4, do things
start working for you again?

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Pekka J Enberg]

On Sun, 19 Feb 2006, Greg KH wrote:
> That's _really_ odd, as hal, udev and dbus all work just fine on my
> machines with the above changeset (actually with 2.6.16-rc4).  And that
> changeset should not have caused anything to change with regards to the
> core uevent code, as it's a usb-serial change only.
> 
> And you don't even have CONFIG_USB_SERIAL enabled...  Very odd.
> 
> If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> start working for you again?

Probably not. For some reason, I had udev and hal failing for plain 2.6.15 
too until I did make mrproper. So the bisect results are probably not 
reliable. I'll do that again this evening with mrproper. Uh.

			Pekka

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

On Sun, 2006-02-19 at 17:02 -0800, Greg KH wrote:
> If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> start working for you again?

Okey dokey, bisecting with mrproper took little longer than expected but
I found the bad changeset:

033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
Author: Kay Sievers <kay.sievers@suse.de>
Date:   Fri Nov 11 06:09:55 2005 +0100

    [PATCH] remove mount/umount uevents from superblock handling

    The names of these events have been confusing from the beginning
    on, as they have been more like claim/release events. We needed these
    events for noticing HAL if storage devices have been mounted.

    Thanks to Al, we have the proper solution now and can poll()
    /proc/mounts instead to get notfied about mount tree changes.

    Signed-off-by: Kay Sievers <kay.sievers@suse.de>
    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

:040000 040000 0397ffe6fdbffa290e2d22f5f59c5a8cc2861ef0 44e4df27174f1dfb125481f9ce4471edaa93c57d M      fs
:040000 040000 94178115d8c608d6247bed322efeddf19047ad99 55655fc60af26aed51a7d3ddee452c859ec3c501 M      include
:040000 040000 40bd1625895ddca1d39381887587f4465abed222 397331a2cd3077cd3f41901f4af74ab6ad3fb13a M      lib

Reverting the above from yesterday's git head fixes the problem for me
and I get my Ipod icon in Gnome again. I am running udev 079, dbus 060,
hal 0.5.5.1, and gnome-volume-manager 1.5.4 on Gentoo/x86. The patch
should probably be reverted for 2.6.16 because it breaks userspace.

				Pekka

Re: 2.6.16-rc4: known regressions

[Kay Sievers]

On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> On Sun, 2006-02-19 at 17:02 -0800, Greg KH wrote:
> > If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> > start working for you again?
> 
> Okey dokey, bisecting with mrproper took little longer than expected but
> I found the bad changeset:
> 
> 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> Author: Kay Sievers <kay.sievers@suse.de>
> Date:   Fri Nov 11 06:09:55 2005 +0100
> 
>     [PATCH] remove mount/umount uevents from superblock handling

Upgrade HAL, it's too old for that kernel.

Kay

Re: 2.6.16-rc4: known regressions

[Andrew Morton]

Kay Sievers <kay.sievers@suse.de> wrote:
>
> On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> > On Sun, 2006-02-19 at 17:02 -0800, Greg KH wrote:
> > > If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> > > start working for you again?
> > 
> > Okey dokey, bisecting with mrproper took little longer than expected but
> > I found the bad changeset:

Thanks - it helps heaps.

> > 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> > diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> > Author: Kay Sievers <kay.sievers@suse.de>
> > Date:   Fri Nov 11 06:09:55 2005 +0100
> > 
> >     [PATCH] remove mount/umount uevents from superblock handling
> 
> Upgrade HAL, it's too old for that kernel.
> 

We broke back-compatibility.  The changelog _failed to tell us_ that we
were breaking back-compatibility.  The patch wouldn't have been applied if
we'd been told that.  At least, not without a lot of careful thought.

The fact that the changelog failed to tell us this makes one suspect that
the breakage was inadvertent.


So no, upgrading HAL is not a good answer.  Please fix the kernel.

Re: 2.6.16-rc4: known regressions

[Kay Sievers]

On Tue, Feb 21, 2006 at 03:33:05PM -0800, Andrew Morton wrote:
> Kay Sievers <kay.sievers@suse.de> wrote:
> >
> > On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> > > On Sun, 2006-02-19 at 17:02 -0800, Greg KH wrote:
> > > > If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> > > > start working for you again?
> > > 
> > > Okey dokey, bisecting with mrproper took little longer than expected but
> > > I found the bad changeset:
> 
> Thanks - it helps heaps.
> 
> > > 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> > > diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> > > Author: Kay Sievers <kay.sievers@suse.de>
> > > Date:   Fri Nov 11 06:09:55 2005 +0100
> > > 
> > >     [PATCH] remove mount/umount uevents from superblock handling
> > 
> > Upgrade HAL, it's too old for that kernel.
> > 
> 
> We broke back-compatibility.  The changelog _failed to tell us_ that we
> were breaking back-compatibility.  The patch wouldn't have been applied if
> we'd been told that.  At least, not without a lot of careful thought.
> 
> The fact that the changelog failed to tell us this makes one suspect that
> the breakage was inadvertent.
> 
> 
> So no, upgrading HAL is not a good answer.  Please fix the kernel.

No, these events I added for HAL and they were wrong as pointed out by
Al Viro and with his help we replaced them by a better solution, which
actually is the "fix".

HAL was prepared to make use of the new events and needs to be upgraded
when the kernel gets upgraded. This happens all the time as long as we
try to find the right way to interact with the kernel and need to
change the interfaces.

HAL is tightly bound to the kernel and this will countinuously happen in
the future too, cause we can't solve the problem that you don't know how
an interface works without a user and if you don't put it in the kernel you
certainly don't have a user.

Interfaces get stable by becoming good enough and not by someone that
declares them as stable. If you have a problem with that, please introduce
a list of stable interfaces where people can rely on and I will not put
any of the new interfaces that are under construction on that list. We
need to be able to use new interfaces and change them until we know that
they are good enough. It is very unlikely to get complex interfaces
right in the first place. In the mount event case they have been identified
as the wrong way to do it and got replaced.

Thanks,
Kay

Re: 2.6.16-rc4: known regressions

[Mark Lord]

Kay Sievers wrote:
>
> HAL is tightly bound to the kernel and this will countinuously happen in
> the future too, cause we can't solve the problem that you don't know how
> an interface works without a user and if you don't put it in the kernel you
> certainly don't have a user.

The usual approach is to overlap the old/new interfaces,
so that userspace continues to work.  After a few kernel revisions,
then nuke the old interface.

What, you say, you're replacing an existing userspace API in-situ,
rather than creating a new one??

Humbug..

Re: 2.6.16-rc4: known regressions

[Andrew Morton]

Kay Sievers <kay.sievers@suse.de> wrote:
>
> > We broke back-compatibility.  The changelog _failed to tell us_ that we
> > were breaking back-compatibility.  The patch wouldn't have been applied if
> > we'd been told that.  At least, not without a lot of careful thought.
> > 
> > The fact that the changelog failed to tell us this makes one suspect that
> > the breakage was inadvertent.
> > 
> > 
> > So no, upgrading HAL is not a good answer.  Please fix the kernel.
> 
> [ bunch of special-pleading ]
>

None of that matters or is relevant.

You took a kernel interface which was present in 2.6.10, 2.6.11, 2.6.12,
2.6.13, 2.6.14 and 2.6.15 and changed it in a non-compatible way, without
telling us that it was non-compatible and without even notifying people
that we'd gone and broken existing userspace.

We.  Don't.  Do. That.

Please either restore the old events so we can have a 6-12 month transition
period or revert the patch.

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Tue, 21 Feb 2006, Andrew Morton wrote:
> 
> We.  Don't.  Do. That.
> 
> Please either restore the old events so we can have a 6-12 month transition
> period or revert the patch.

I agree.

This stupid argument of "HAL is part of the kernel, so we can break it" is 
_bogus_. 

The fact is, if changing the kernel breaks user-space, it's a regression. 
IT DOES NOT MATTER WHETHER IT'S IN /sbin/hotplug OR ANYTHING ELSE. If it 
was installed by a distribution, it's user-space. If it got installed by 
"vmlinux", it's the kernel.

The only piece of user-space code we ship with the kernel is the system 
call trampoline etc that the kernel sets up. THOSE interfaces we can 
really change, because it changes with the kernel.

		Linus

Re: 2.6.16-rc4: known regressions

[Con Kolivas]

On Wednesday 22 February 2006 11:34, Linus Torvalds wrote:
> On Tue, 21 Feb 2006, Andrew Morton wrote:
> > We.  Don't.  Do. That.
> >
> > Please either restore the old events so we can have a 6-12 month
> > transition period or revert the patch.
>
> I agree.
>
> This stupid argument of "HAL is part of the kernel, so we can break it" is
> _bogus_.
>
> The fact is, if changing the kernel breaks user-space, it's a regression.
> IT DOES NOT MATTER WHETHER IT'S IN /sbin/hotplug OR ANYTHING ELSE. If it
> was installed by a distribution, it's user-space. If it got installed by
> "vmlinux", it's the kernel.
>
> The only piece of user-space code we ship with the kernel is the system
> call trampoline etc that the kernel sets up. THOSE interfaces we can
> really change, because it changes with the kernel.

Sanity.

It would be a sad day when no distribution in existence can support the 
current kernel except for a bleeding edge source based thing upgraded daily.

Cheers,
Con

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Tue, 21 Feb 2006, Linus Torvalds wrote:
> 
> The only piece of user-space code we ship with the kernel is the system 
> call trampoline etc that the kernel sets up. THOSE interfaces we can 
> really change, because it changes with the kernel.

Side note: if people want to, we could have other "trampolines" like that, 
so that we could have more user-level code that gets distributed with the 
kernel. It doesn't have to be something that gets mapped into every binary 
either: we could - if we wanted to - have things like shared libraries or 
helper shell scripts or whatever that we expose in /sys/shlib/ that are 
kernel-version dependent.

Then we could perhaps change more things, just because we could change the 
wrappers that actually use them together with the kernel.

To some degree, /initrd was supposed to do things like that, and in 
theory, it still could. However, realistically, 99% of any /initrd is more 
about the distribution than the kernel, so right now we have to count 
/initrd as a distribution thing, not a kernel thing.

		Linus

Re: 2.6.16-rc4: known regressions

[Theodore Ts'o]

On Tue, Feb 21, 2006 at 05:06:48PM -0800, Linus Torvalds wrote:
> 
> To some degree, /initrd was supposed to do things like that, and in 
> theory, it still could. However, realistically, 99% of any /initrd is more 
> about the distribution than the kernel, so right now we have to count 
> /initrd as a distribution thing, not a kernel thing.

... and if we're truly going to be pouring more and more complexity
into initrd (such as userspace swsusp), then (a) we probably should
make it more of a kernel-specific thing, and not a distro-specific
thing, since without that you can be pretty much guaranteed that more
and more people will be using and testing swsusp2, and not uswsusp,
and (b) we need to add _way_ better debugging provisions so that if
something dies in early boot, you don't go pulling out your hair
trying to figure out what went wrong, and having to spend a good 20
minutes or so between each try-to-fix the initrd, watch the boot fail,
reboot into a working setup, cursing Red Hat's nash, modifying the
initrd, and trying again.

Usually I break the loop by giving up, and ripping out whatever kernel
feature requires initrd, such as dm, and installing on hard partitions
with all of the kernel modules I need compiled into the kernel.  I
still have no idea why mptscsi fails to detect SCSI disks when loaded
as a module via initrd on various bits of IBM hardware (including the
e326 and ls-20 blade), but works fine when compiled directly into the
kernel....

If we want more and more stuff to be poured into initrd, it's got to
be made easier to debug and consistent across distributions, such that
more people can test initrd configurations, and flush out the bugs,
never mind the question of programs like udev randomly breaking
between kernel releases.  Maybe it's time to consider moving all of
that into the kernel source; if they wanted to be treated as part of
the kernel, then let them liteally become part of the kernel from a
source code and release management perspective.

						- Ted

Re: 2.6.16-rc4: known regressions

[Joel Becker]

On Wed, Feb 22, 2006 at 06:21:58AM -0500, Theodore Ts'o wrote:
> with all of the kernel modules I need compiled into the kernel.  I
> still have no idea why mptscsi fails to detect SCSI disks when loaded
> as a module via initrd on various bits of IBM hardware (including the
> e326 and ls-20 blade), but works fine when compiled directly into the
> kernel....

Ted,
	Do you mean that you are using a distro (eg, RHEL4 or something)
with a mainline kernel?  We've seen something similar, and what we've
determined is happening is that insmod is returning before the module is
done initializing.  It's not that mptscsi fails to detect the disks.
Rather, it's still in the detection process when the boot process tries
to mount /.  So there's no / yet, and the thing hangs.  In the case we
see, it's some interaction between the RHEL4/SLES9 version of
module-init-tools and the latest version of the kernel.  Our first
attempt at fixing it was to change the linuxrc to sleep between each
insmod.  This works, but only if the modules load and initialize
themsleves fast enough.  Get a FC card in there, and it just doesn't
work.  So we've taken to compiling the modules in-kernel.

Joel

-- 

"Three o'clock is always too late or too early for anything you
 want to do."
        - Jean-Paul Sartre

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: 2.6.16-rc4: known regressions

[Theodore Ts'o]

On Wed, Feb 22, 2006 at 07:48:21AM -0800, Joel Becker wrote:
> On Wed, Feb 22, 2006 at 06:21:58AM -0500, Theodore Ts'o wrote:
> > with all of the kernel modules I need compiled into the kernel.  I
> > still have no idea why mptscsi fails to detect SCSI disks when loaded
> > as a module via initrd on various bits of IBM hardware (including the
> > e326 and ls-20 blade), but works fine when compiled directly into the
> > kernel....
> 
> Ted,
> 	Do you mean that you are using a distro (eg, RHEL4 or something)
> with a mainline kernel?  We've seen something similar, and what we've
> determined is happening is that insmod is returning before the module is
> done initializing.  It's not that mptscsi fails to detect the disks.
> Rather, it's still in the detection process when the boot process tries
> to mount /.  So there's no / yet, and the thing hangs.  

Yep, that's exactly what I'm doing; RHEL4U2 with a 2.6.14 or 2.6.15
kernel.  Thanks for the tip, that should help me investigate further!

> In the case we
> see, it's some interaction between the RHEL4/SLES9 version of
> module-init-tools and the latest version of the kernel.  Our first
> attempt at fixing it was to change the linuxrc to sleep between each
> insmod.  This works, but only if the modules load and initialize
> themsleves fast enough.  Get a FC card in there, and it just doesn't
> work.  So we've taken to compiling the modules in-kernel.

Sounds like this is another of example of system support programs
(insmod in this case) breaking with modern kernels.  Hopefully now
that Linus has laid down the law about how breaking userspace is
uncool, people will agree that it's a bug.  (That is unless Red Hat
made some kind of incompatible change and it's Red Hat's fault, but I
kinda doubt that.)  Anyway, I'll look into this some more and see why
you can't use a mainline kernel with RHEL4, at least not in this
configuration.

						- Ted

Re: 2.6.16-rc4: known regressions

[Gabor Gombas]

On Wed, Feb 22, 2006 at 11:25:34AM -0500, Theodore Ts'o wrote:

> Sounds like this is another of example of system support programs
> (insmod in this case) breaking with modern kernels.

I don't think isnmod is broken. It's job is to load a chunk of code into
the kernel, and it's doing just that.

The asynchronous device discovery is caused/required by hotplug. If you
can recreate the problem with a kernel that has CONFIG_HOTPLUG disabled,
then I agree that this is a kernel bug which should be fixed.

But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
this exact behavior, therefore you should better fix userspace to cope
with it. Your initrd should use the notification mechanisms provided by
the kernel to wait for the would-be root device really becoming
available; if it's not doing that, then IMHO you should not use a
CONFIG_HOTPLUG enabled kernel.

As I see, a lot of people spent a lot of work making the kernel
hotplug-friendly. Unfortunately a lot less work was done on the
userspace side, that's why there are still a lot of initscripts that
assume they can immediately access the device after insmod have
returned, even if they are running on a hotplug-enabled kernel.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, Gabor Gombas wrote:
> 
> I don't think isnmod is broken. It's job is to load a chunk of code into
> the kernel, and it's doing just that.
> 
> The asynchronous device discovery is caused/required by hotplug. If you
> can recreate the problem with a kernel that has CONFIG_HOTPLUG disabled,
> then I agree that this is a kernel bug which should be fixed.

I think it currently can happen without HOTPLUG too. In fact, 
CONFIG_HOTPLUG is really a "special drivers that do hot-plugging", not 
about "devices that show up on their own".

The thing is, "insmod" really just tends to introduce the driver to the 
kernel. It leaves it pretty open what that driver will actually _do_. And 
a lot of drivers tend to do discovery independently of actually plugging 
in the driver.

For example, any USB host driver will always discover its devices 
asynchronously, and has no dependency on CONFIG_HOTPLUG. It can take 
several seconds for all the hubs to have powered up and discovered what is 
behind them.

The same is true of most SCSI buses - CONFIG_HOTPLUG may talk about 
whether the actual _controller_ is hotpluggable, but not about whether a 
disk is, or how disk discovery takes place. 

Now, arguably "insmod" (both the user binary and the kernel side) is doing 
the right thing: it's inserting the driver. The fact that all the devices 
that the driver uses may not be immediately available is not insmod's 
issue. That's a very valid way to look at it.

At the same time, it's also arguable that from an ease of use standpoint, 
"insmod" should generally try to wait until the driver has enumerated what 
it knows about. That's a totally non-technical argument, but it's an 
equally valid standpoint.

Of course, the technical argument is that discovery can take a long long 
time (minutes to wait for disks to spin up etc), so if insmod were to wait 
for it all, we'd be really screwed and our bootup times would go through 
the roof.

The usability argument is that right now we don't have any easy way at all 
to wait for bus scanning to have finished, and that's a very valid 
argument. You could wait for the hotplug event, but since you don't even 
_know_ that you'll get such an event, that's really not a very good answer 
either.

We could improve. 

I _think_ that in this particular case, the best particular choice might 
be for the "mount" binary to be taught to re-try after a few seconds: 
either with a command line argument, or with just the early bootup initrd 
code being encouraged to have a loop like

	if (mounting root failed)
		echo "Please press F1 to continue"
		do
			read-keyboard-with-5-second-timeout
		while (mounting root failed)
	endif

so that the user would have to press a key (or we'd just re-try every five 
seconds).

That way, the boot wouldn't just fail immediately over something that can 
be fixed (sometimes the root partition might just be hot-pluggable too: 
"insert disk and continue" can be a valid way to handle issues).

		Linus

Re: 2.6.16-rc4: known regressions

[Christian Trefzer]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

Hi everyone,


On Wed, Feb 22, 2006 at 09:57:38AM -0800, Linus Torvalds wrote:
> 
> 
> I _think_ that in this particular case, the best particular choice might 
> be for the "mount" binary to be taught to re-try after a few seconds: 
> either with a command line argument, or with just the early bootup initrd 
> code being encouraged to have a loop like
> 
> 	if (mounting root failed)
> 		echo "Please press F1 to continue"
> 		do
> 			read-keyboard-with-5-second-timeout
> 		while (mounting root failed)
> 	endif
> 
> so that the user would have to press a key (or we'd just re-try every five 
> seconds).
> 
> That way, the boot wouldn't just fail immediately over something that can 
> be fixed (sometimes the root partition might just be hot-pluggable too: 
> "insert disk and continue" can be a valid way to handle issues).
> 

Is there a way to tell the kernel about which is the root device other
than through the kernel command line? If not, /init or /linuxrc could
parse /proc/cmdline for that and wait for the device node to appear.

Having the same trouble with my crude bloated glibc initramfs, I
resorted to waiting for /dev/hdX to appear after ide module insertion in
a loop with one second delay. AFAICS the loop is taken three times at
most on my slowest box, so five seconds seems a bit much, IMHO.

Trivial shell code goes like

echo -n "Waiting for root device to appear"
while [ ! -e $rootdevice ]
do
	sleep 1s
	echo -n "."
done
echo " OK."

The only downside here would be devices specified as hex numbers, and we
have an endless loop in case something went wrong. The latter can be
addressed with a counter checked against some sane limit.

Worked fine for me every time, except once when I build an image without
the IDE controller's module : /


Chris


[-- Attachment #2: Type: application/pgp-signature, Size: 829 bytes --]

Re: 2.6.16-rc4: known regressions

[Joel Becker]

On Wed, Feb 22, 2006 at 06:33:54PM +0100, Gabor Gombas wrote:
> I don't think isnmod is broken. It's job is to load a chunk of code into
> the kernel, and it's doing just that.
> 
> ...
>
> But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
> this exact behavior, therefore you should better fix userspace to cope
> with it. Your initrd should use the notification mechanisms provided by
> the kernel to wait for the would-be root device really becoming
> available; if it's not doing that, then IMHO you should not use a
> CONFIG_HOTPLUG enabled kernel.

        The issue isn't so much "insmod is right" vs "insmod is wrong".
It's that the behavior changed in a surprising fashion.  Red Hat's
kernel for RHEL4 (in our example) has CONFIG_HOTPLUG=y, yet it Just
Works.  A more recent kernel (.15 and .16 at least) with
CONFIG_HOTPLUG=y doesn't work.  Same disk drivers.  Same initramfs
script.
	We're discussing this very "kernel change broke userspace
expectations" issue.  You don't need to convince me that

  1. Insmod loads the driver
  2. Userspace initramfs sleeps waiting for hotplug
  3. Hotplug completes
  4. Userspace initramfs continues, using the now plugged devices.

is the "most correct".  It makes perfect technical sense.  If you were
starting from scratch, no one would be complaining, becuase no one would
see a problem.
	We're not starting from scratch.  We have large installed bases
of userspace code that expects a certain behavior from the kernel.
While it sometimes is necessary to say "you need to update", I think the
consensus is clear - only do that when it is necessary.  Don't call
people dumb or say they "need to update" just because you didn't
consider existing users.

Joel

-- 

 "I'm living so far beyond my income that we may almost be said
 to be living apart."
         - e e cummings

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 10:59:23AM -0800, Joel Becker wrote:
> On Wed, Feb 22, 2006 at 06:33:54PM +0100, Gabor Gombas wrote:
> > I don't think isnmod is broken. It's job is to load a chunk of code into
> > the kernel, and it's doing just that.
> > 
> > ...
> >
> > But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
> > this exact behavior, therefore you should better fix userspace to cope
> > with it. Your initrd should use the notification mechanisms provided by
> > the kernel to wait for the would-be root device really becoming
> > available; if it's not doing that, then IMHO you should not use a
> > CONFIG_HOTPLUG enabled kernel.
> 
>         The issue isn't so much "insmod is right" vs "insmod is wrong".
> It's that the behavior changed in a surprising fashion.  Red Hat's
> kernel for RHEL4 (in our example) has CONFIG_HOTPLUG=y, yet it Just
> Works.  A more recent kernel (.15 and .16 at least) with
> CONFIG_HOTPLUG=y doesn't work.  Same disk drivers.  Same initramfs
> script.

RHEL is a very different kernel from mainline (just like SLES is).  Have
you looked through their patches to see if they are including something
that causes this behavior?

What about trying a stock 2.6.6 or so kernel?  Does that work
differently from 2.6.15?

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Arjan van de Ven]

On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
> On Wed, Feb 22, 2006 at 10:59:23AM -0800, Joel Becker wrote:
> > On Wed, Feb 22, 2006 at 06:33:54PM +0100, Gabor Gombas wrote:
> > > I don't think isnmod is broken. It's job is to load a chunk of code into
> > > the kernel, and it's doing just that.
> > > 
> > > ...
> > >
> > > But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
> > > this exact behavior, therefore you should better fix userspace to cope
> > > with it. Your initrd should use the notification mechanisms provided by
> > > the kernel to wait for the would-be root device really becoming
> > > available; if it's not doing that, then IMHO you should not use a
> > > CONFIG_HOTPLUG enabled kernel.
> > 
> >         The issue isn't so much "insmod is right" vs "insmod is wrong".
> > It's that the behavior changed in a surprising fashion.  Red Hat's
> > kernel for RHEL4 (in our example) has CONFIG_HOTPLUG=y, yet it Just
> > Works.  A more recent kernel (.15 and .16 at least) with
> > CONFIG_HOTPLUG=y doesn't work.  Same disk drivers.  Same initramfs
> > script.
> 
> RHEL is a very different kernel from mainline (just like SLES is).  Have
> you looked through their patches to see if they are including something
> that causes this behavior?

I doubt it does; rhel isn't that much patches...

> 
> What about trying a stock 2.6.6 or so kernel?  Does that work
> differently from 2.6.15?

... however it's very much designed only for the kernel that comes with
it (with "it's" I mean all the userspace infrastructure); all the
changes and additions since 2.6.9 aren't incorporated so you probably
really want new alsa, new initscripts, new mkinitrd, new
module-init-tools. some because of abi changes since 2.6.9, others
because the kernel grew capabilities that are really needed for "nice"
behavior.

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 08:29:48PM +0100, Arjan van de Ven wrote:
> On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
> > What about trying a stock 2.6.6 or so kernel?  Does that work
> > differently from 2.6.15?
> 
> ... however it's very much designed only for the kernel that comes with
> it (with "it's" I mean all the userspace infrastructure); all the
> changes and additions since 2.6.9 aren't incorporated so you probably
> really want new alsa, new initscripts, new mkinitrd, new
> module-init-tools. some because of abi changes since 2.6.9, others
> because the kernel grew capabilities that are really needed for "nice"
> behavior.

I totally agree.  Distros are changing into two different groups these
days:
	- everything tied together and intregrated nicely for a specific
	  kernel version, userspace tool versions, etc.
	- flexible and works with multiple kernel versions, different
	  userspace tools, etc.

Distros in the first category are the "enterprise" releases (RHEL, SLES,
etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
possibly.)

More flexible distros that handle different kernel versions are Gentoo,
Debian, and probably Fedora.

And this is a natural progression as people try to provide a more
complete "solution" for users.

When people to complain that they can't run a "kernel-of-the-day" on
their "enterprise" distro, they are not realizing that that distro was
just not developed to support that kind of thing at all.

So, in short, if you are going to do kernel development, pick a distro
that handles different kernel versions.  Likewise, if you are doing
userspace development (X.org, HAL, KDE, Gnome, etc.) you pick a distro
that allows you to change that level of the stack.

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Jens Axboe]

On Wed, Feb 22 2006, Greg KH wrote:
> On Wed, Feb 22, 2006 at 08:29:48PM +0100, Arjan van de Ven wrote:
> > On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
> > > What about trying a stock 2.6.6 or so kernel?  Does that work
> > > differently from 2.6.15?
> > 
> > ... however it's very much designed only for the kernel that comes with
> > it (with "it's" I mean all the userspace infrastructure); all the
> > changes and additions since 2.6.9 aren't incorporated so you probably
> > really want new alsa, new initscripts, new mkinitrd, new
> > module-init-tools. some because of abi changes since 2.6.9, others
> > because the kernel grew capabilities that are really needed for "nice"
> > behavior.
> 
> I totally agree.  Distros are changing into two different groups these
> days:
> 	- everything tied together and intregrated nicely for a specific
> 	  kernel version, userspace tool versions, etc.
> 	- flexible and works with multiple kernel versions, different
> 	  userspace tools, etc.
> 
> Distros in the first category are the "enterprise" releases (RHEL, SLES,
> etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> possibly.)
> 
> More flexible distros that handle different kernel versions are Gentoo,
> Debian, and probably Fedora.
> 
> And this is a natural progression as people try to provide a more
> complete "solution" for users.
> 
> When people to complain that they can't run a "kernel-of-the-day" on
> their "enterprise" distro, they are not realizing that that distro was
> just not developed to support that kind of thing at all.

I have to disagree somewhat violently to that statement, I'm afraid :-)
At least for me, it's pretty much a requirement that I can put eg
2.6.18-rc2 on an enterprise install. It's a must to debug problems -
both ways, actually, testing both a new rc kernel on that enterprise
distro but also putting a vanilla kernel on the enterprise distro to
test something that fails with the distro kernel.

I'd absolutely hate if we got into a situation where you couldn't just
put a new vanilla kernel on SLESx. Calling it a complete solution to
just sounds like an excuse for breaking things that we don't have to.
Please lets not make things so fragile!

> So, in short, if you are going to do kernel development, pick a distro
> that handles different kernel versions.  Likewise, if you are doing
> userspace development (X.org, HAL, KDE, Gnome, etc.) you pick a distro
> that allows you to change that level of the stack.

Everything doesn't fit into those two boxes.

-- 
Jens Axboe

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 09:45:08PM +0100, Jens Axboe wrote:
> On Wed, Feb 22 2006, Greg KH wrote:
> > On Wed, Feb 22, 2006 at 08:29:48PM +0100, Arjan van de Ven wrote:
> > > On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
> > > > What about trying a stock 2.6.6 or so kernel?  Does that work
> > > > differently from 2.6.15?
> > > 
> > > ... however it's very much designed only for the kernel that comes with
> > > it (with "it's" I mean all the userspace infrastructure); all the
> > > changes and additions since 2.6.9 aren't incorporated so you probably
> > > really want new alsa, new initscripts, new mkinitrd, new
> > > module-init-tools. some because of abi changes since 2.6.9, others
> > > because the kernel grew capabilities that are really needed for "nice"
> > > behavior.
> > 
> > I totally agree.  Distros are changing into two different groups these
> > days:
> > 	- everything tied together and intregrated nicely for a specific
> > 	  kernel version, userspace tool versions, etc.
> > 	- flexible and works with multiple kernel versions, different
> > 	  userspace tools, etc.
> > 
> > Distros in the first category are the "enterprise" releases (RHEL, SLES,
> > etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> > possibly.)
> > 
> > More flexible distros that handle different kernel versions are Gentoo,
> > Debian, and probably Fedora.
> > 
> > And this is a natural progression as people try to provide a more
> > complete "solution" for users.
> > 
> > When people to complain that they can't run a "kernel-of-the-day" on
> > their "enterprise" distro, they are not realizing that that distro was
> > just not developed to support that kind of thing at all.
> 
> I have to disagree somewhat violently to that statement, I'm afraid :-)
> At least for me, it's pretty much a requirement that I can put eg
> 2.6.18-rc2 on an enterprise install. It's a must to debug problems -
> both ways, actually, testing both a new rc kernel on that enterprise
> distro but also putting a vanilla kernel on the enterprise distro to
> test something that fails with the distro kernel.

I agree that is is a _good_ thing that us kernel developers can do this,
and that it isn't impossible (I do the same thing.)  But we also aren't
worrying about the fact that our sound stopped working, or that the
desktop icons don't show up anymore if we plug in a new device.  We are
a very special case.

For any "user", they should not ever count on using a different kernel
than what was shipped with the system (or updates) for an "enterprise"
distro.  There are just too many little things that easily go wrong.

> I'd absolutely hate if we got into a situation where you couldn't just
> put a new vanilla kernel on SLESx. Calling it a complete solution to
> just sounds like an excuse for breaking things that we don't have to.
> Please lets not make things so fragile!

We are trying, but as everyone is so quick to point out, we (myself
included) mess up at times :)

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Jens Axboe]

On Wed, Feb 22 2006, Greg KH wrote:
> On Wed, Feb 22, 2006 at 09:45:08PM +0100, Jens Axboe wrote:
> > On Wed, Feb 22 2006, Greg KH wrote:
> > > On Wed, Feb 22, 2006 at 08:29:48PM +0100, Arjan van de Ven wrote:
> > > > On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
> > > > > What about trying a stock 2.6.6 or so kernel?  Does that work
> > > > > differently from 2.6.15?
> > > > 
> > > > ... however it's very much designed only for the kernel that comes with
> > > > it (with "it's" I mean all the userspace infrastructure); all the
> > > > changes and additions since 2.6.9 aren't incorporated so you probably
> > > > really want new alsa, new initscripts, new mkinitrd, new
> > > > module-init-tools. some because of abi changes since 2.6.9, others
> > > > because the kernel grew capabilities that are really needed for "nice"
> > > > behavior.
> > > 
> > > I totally agree.  Distros are changing into two different groups these
> > > days:
> > > 	- everything tied together and intregrated nicely for a specific
> > > 	  kernel version, userspace tool versions, etc.
> > > 	- flexible and works with multiple kernel versions, different
> > > 	  userspace tools, etc.
> > > 
> > > Distros in the first category are the "enterprise" releases (RHEL, SLES,
> > > etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> > > possibly.)
> > > 
> > > More flexible distros that handle different kernel versions are Gentoo,
> > > Debian, and probably Fedora.
> > > 
> > > And this is a natural progression as people try to provide a more
> > > complete "solution" for users.
> > > 
> > > When people to complain that they can't run a "kernel-of-the-day" on
> > > their "enterprise" distro, they are not realizing that that distro was
> > > just not developed to support that kind of thing at all.
> > 
> > I have to disagree somewhat violently to that statement, I'm afraid :-)
> > At least for me, it's pretty much a requirement that I can put eg
> > 2.6.18-rc2 on an enterprise install. It's a must to debug problems -
> > both ways, actually, testing both a new rc kernel on that enterprise
> > distro but also putting a vanilla kernel on the enterprise distro to
> > test something that fails with the distro kernel.
> 
> I agree that is is a _good_ thing that us kernel developers can do this,
> and that it isn't impossible (I do the same thing.)  But we also aren't
> worrying about the fact that our sound stopped working, or that the
> desktop icons don't show up anymore if we plug in a new device.  We are
> a very special case.

Definitely, when I built such a kernel I usually don't include sound or
usb :-)

> For any "user", they should not ever count on using a different kernel
> than what was shipped with the system (or updates) for an "enterprise"
> distro.  There are just too many little things that easily go wrong.

Sure, they move outside the supported zone very quickly if they do that.
But that's a different thing from the system actually not booting /
working with a vanilla kernel.

> > I'd absolutely hate if we got into a situation where you couldn't just
> > put a new vanilla kernel on SLESx. Calling it a complete solution to
> > just sounds like an excuse for breaking things that we don't have to.
> > Please lets not make things so fragile!
> 
> We are trying, but as everyone is so quick to point out, we (myself
> included) mess up at times :)

Mistakes happen for everybody, I think it's the intentional breakage
that people are yelling about :-)

-- 
Jens Axboe

Re: 2.6.16-rc4: known regressions

[Martin Bligh]

> I totally agree.  Distros are changing into two different groups these
> days:
> 	- everything tied together and intregrated nicely for a specific
> 	  kernel version, userspace tool versions, etc.
> 	- flexible and works with multiple kernel versions, different
> 	  userspace tools, etc.
> 
> Distros in the first category are the "enterprise" releases (RHEL, SLES,
> etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> possibly.)
> 
> More flexible distros that handle different kernel versions are Gentoo,
> Debian, and probably Fedora.
> 
> And this is a natural progression as people try to provide a more
> complete "solution" for users.
> 
> When people to complain that they can't run a "kernel-of-the-day" on
> their "enterprise" distro, they are not realizing that that distro was
> just not developed to support that kind of thing at all.
> 
> So, in short, if you are going to do kernel development, pick a distro
> that handles different kernel versions.  Likewise, if you are doing
> userspace development (X.org, HAL, KDE, Gnome, etc.) you pick a distro
> that allows you to change that level of the stack.

That sort of thing is going to make distros incredibly reluctant to 
update kernels, which just encourages them to operate inside their own
fiefdoms, rather than working together with mainline, which is what we
want.

Moreover, its' not just the big distros. It's every corporation with a
product based around Linux, which are far more numerous and smaller
operations. We *have* to encourage these people to work with us, else
we end up not getting bug fixes back upstream from them etc.

That means giving them stable, consistent userspace<->kernel APIs.

M.

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Thu, Feb 23, 2006 at 09:29:50AM -0800, Martin Bligh wrote:
> >I totally agree.  Distros are changing into two different groups these
> >days:
> >	- everything tied together and intregrated nicely for a specific
> >	  kernel version, userspace tool versions, etc.
> >	- flexible and works with multiple kernel versions, different
> >	  userspace tools, etc.
> >
> >Distros in the first category are the "enterprise" releases (RHEL, SLES,
> >etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> >possibly.)
> >
> >More flexible distros that handle different kernel versions are Gentoo,
> >Debian, and probably Fedora.
> >
> >And this is a natural progression as people try to provide a more
> >complete "solution" for users.
> >
> >When people to complain that they can't run a "kernel-of-the-day" on
> >their "enterprise" distro, they are not realizing that that distro was
> >just not developed to support that kind of thing at all.
> >
> >So, in short, if you are going to do kernel development, pick a distro
> >that handles different kernel versions.  Likewise, if you are doing
> >userspace development (X.org, HAL, KDE, Gnome, etc.) you pick a distro
> >that allows you to change that level of the stack.
> 
> That sort of thing is going to make distros incredibly reluctant to 
> update kernels, which just encourages them to operate inside their own
> fiefdoms, rather than working together with mainline, which is what we
> want.

They are very reluctant to upgrade kernels today, for released versions
of the distro, and so they backport kernel fixes and security updates to
that older kernel, just like all of the packages in a distro.  That's
they way they work.

But they work together with mainline as they need it for their _next_
release that happens again in 6 months or so, with the updated kernel
and everything else.

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Martin Bligh]

> They are very reluctant to upgrade kernels today, for released versions
> of the distro, and so they backport kernel fixes and security updates to
> that older kernel, just like all of the packages in a distro.  That's
> they way they work.
> 
> But they work together with mainline as they need it for their _next_
> release that happens again in 6 months or so, with the updated kernel
> and everything else.

I don't think that's universally true at all, either from my own 
observations, or that of others I have talked to.

What I see is people operating in their own silos, and being terrified
to upgrade because they have no idea what random breakage will have
occured in mainline. When they get into that mode, the not only don't
suck new kernels down, they don't push their own changes up.

That's not healthy, and whatever we can do to encourage them to play
with the rest of the world more nicely would be healthy, I think.

M.

Re: 2.6.16-rc4: known regressions

[Arjan van de Ven]

> They are very reluctant to upgrade kernels today, for released versions
> of the distro, and so they backport kernel fixes and security updates to
> that older kernel, just like all of the packages in a distro.  That's
> they way they work.


fedora actually regularly updates to newer kernels, as service to their
users. (And since kernels still fix more bugs than they create it's a
net win ;)

Re: 2.6.16-rc4: known regressions

[Benjamin LaHaise]

On Wed, Feb 22, 2006 at 11:40:24AM -0800, Greg KH wrote:
> I totally agree.  Distros are changing into two different groups these
> days:
> 	- everything tied together and intregrated nicely for a specific
> 	  kernel version, userspace tool versions, etc.
> 	- flexible and works with multiple kernel versions, different
> 	  userspace tools, etc.
> 
> Distros in the first category are the "enterprise" releases (RHEL, SLES,
> etc.), as well as some consumer oriented distros (SuSE, Ubuntu, Fedora
> possibly.)

That is a completely unreasonable position.  It is a requirement for those 
of us working on a variety of problems to be able to use new kernels on 
the "Enterprise" distributions in the market, as you have to be able to 
compare regressions and performance.  Swapping out all of userland just 
because hotplug can't get it's act together is *NOT* an option.

		-ben
-- 
"Ladies and gentlemen, I'm sorry to interrupt, but the police are here 
and they've asked us to stop the party."  Don't Email: <dont@kvack.org>.

Re: 2.6.16-rc4: known regressions

[Eric W. Biederman]

Greg KH <gregkh@suse.de> writes:

> On Wed, Feb 22, 2006 at 08:29:48PM +0100, Arjan van de Ven wrote:
>> On Wed, 2006-02-22 at 11:18 -0800, Greg KH wrote:
>> > What about trying a stock 2.6.6 or so kernel?  Does that work
>> > differently from 2.6.15?
>> 
>> ... however it's very much designed only for the kernel that comes with
>> it (with "it's" I mean all the userspace infrastructure); all the
>> changes and additions since 2.6.9 aren't incorporated so you probably
>> really want new alsa, new initscripts, new mkinitrd, new
>> module-init-tools. some because of abi changes since 2.6.9, others
>> because the kernel grew capabilities that are really needed for "nice"
>> behavior.

Not being able to take advantage of the latest whizz bang features
sound sane.  Kernel ABI changes do not sound sane.

> I totally agree.  Distros are changing into two different groups these
> days:
> 	- everything tied together and intregrated nicely for a specific
> 	  kernel version, userspace tool versions, etc.
> 	- flexible and works with multiple kernel versions, different
> 	  userspace tools, etc.
>
> And this is a natural progression as people try to provide a more
> complete "solution" for users.

Huh? I can understand it from the rational of a support contract, 
limiting the number of possibilities you have to deal with.  Beyond
that it just seems ludicrous.  If something is relied up by user
space it should be stable enough that you can upgrade your kernel.

It should be a case of not-supported but it ``should'' work.  So
anyone who is competent enough to do kernel development should 
have no problem getting things going.

> When people to complain that they can't run a "kernel-of-the-day" on
> their "enterprise" distro, they are not realizing that that distro was
> just not developed to support that kind of thing at all.

Which is a really bad tack to take.  It seriously forks the kernel
debugging and testing efforts.  Even with a distro that only wants
to support one kernel if people are having kernel hardware interaction
problems I will still suggest that they upgrade their kernel to
see if the problems are fixed in the newer kernels.  If there are
user space glitches but the drivers work, there are a lot more people
who can fix user space than can fix the kernel.

The historic rule is that if you need to know what needs to be changed
you read through Documentation/Changes.  If your distro has something
older you probably need to upgrade if not you don't.

> So, in short, if you are going to do kernel development, pick a distro
> that handles different kernel versions.  Likewise, if you are doing
> userspace development (X.org, HAL, KDE, Gnome, etc.) you pick a distro
> that allows you to change that level of the stack.

But this applies to kernel debugging so having a distro that is tied
intimately to their kernel is a problem for anyone running on recent
hardware.

Eric

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, Greg KH wrote:
> 
> RHEL is a very different kernel from mainline (just like SLES is).  Have
> you looked through their patches to see if they are including something
> that causes this behavior?

Quite apart from that, we have definitely had issues where pure timing 
makes a difference - the kernel does the same exact thing, but just 
switches the order of some driver initialization, so that when /sbin/init 
starts, some discovery is still on-going.

It's _rare_, but it's one kind of bug that the kernel really can't do a 
lot about. For example, for the longest time we held off from the 
scheduler running child programs before returning to the parent after a 
fork(), simply because that triggered a real race condition in "bash". 

Eventually, we could say "screw it, it's a user-level timing bug", but the 
point being that sometimes timing changes, and while we _can_ try to keep 
even timing-related behaviour like that similar, sometimes it just isn't 
possible.

It's quite possible that nothing has really "changed", and that some part 
of the kernel just finishes more quickly (or slowly), triggering this 
problem. 

		Linus

Re: 2.6.16-rc4: known regressions

[Andrew Morton]

Joel Becker <Joel.Becker@oracle.com> wrote:
>
> On Wed, Feb 22, 2006 at 06:33:54PM +0100, Gabor Gombas wrote:
> > I don't think isnmod is broken. It's job is to load a chunk of code into
> > the kernel, and it's doing just that.
> > 
> > ...
> >
> > But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
> > this exact behavior, therefore you should better fix userspace to cope
> > with it. Your initrd should use the notification mechanisms provided by
> > the kernel to wait for the would-be root device really becoming
> > available; if it's not doing that, then IMHO you should not use a
> > CONFIG_HOTPLUG enabled kernel.
> 
>         The issue isn't so much "insmod is right" vs "insmod is wrong".
> It's that the behavior changed in a surprising fashion.  Red Hat's
> kernel for RHEL4 (in our example) has CONFIG_HOTPLUG=y, yet it Just
> Works.  A more recent kernel (.15 and .16 at least) with
> CONFIG_HOTPLUG=y doesn't work.  Same disk drivers.  Same initramfs
> script.
> 	We're discussing this very "kernel change broke userspace
> expectations" issue.  You don't need to convince me that
> 
>   1. Insmod loads the driver
>   2. Userspace initramfs sleeps waiting for hotplug
>   3. Hotplug completes
>   4. Userspace initramfs continues, using the now plugged devices.

Yes, I tend to think that insmod should just block until all devices are
ready to be used.  insmod doesn't just "insert a module".  It runs that
module's init function.

The very common case is that userspace wants to use those devices
immediately upon return from insmod, and the handling of these
not-yet-ready devices from userspace is very hard - generally people just
stick lame sleeps in there to get around it.

If, for some strange and rare reason, people _really_ want the
return-when-its-not-ready-yet behaviour, they can do `insmod foo &', or we
give insmod a fork-then-exit option.

But right now the default (and unalterable) behaviour is the oddball,
rarely-desired and hard-to-handle one.

Re: 2.6.16-rc4: known regressions

[Arjan van de Ven]

> Yes, I tend to think that insmod should just block until all devices are
> ready to be used.  insmod doesn't just "insert a module".  It runs that
> module's init function.

that's just not always possible. Think USB, where you don't know the hub
is there.. until it had time to think about it for a bit after it got
power. 

> 
> But right now the default (and unalterable) behaviour is the oddball,
> rarely-desired and hard-to-handle one.

well there is one worse behavior: doing it sync for some, async for
others. That'd mean that the kernel needs to do all the work AND
userspace needs to do all the work *again*.

Lets at least pick ONE place where the work needs to be done ;)
(and if that is userspace, we can make a debug config option which
delays device announcing for a second or two just to help userland
developers debug their code)

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, Andrew Morton wrote:
> 
> Yes, I tend to think that insmod should just block until all devices are
> ready to be used.  insmod doesn't just "insert a module".  It runs that
> module's init function.

It really is very hard to accept the "blocking" behaviour.

Some things can take a _loong_ time to be ready, including even requiring 
user intervention. And even when scanning takes "only" a few seconds, if 
there are multiple modules, you really want to scan things in parallel. 

Not finding a disk is often a matter of timing out - not all buses even 
have any real "enumeration" capability, and enumeration literally ends up 
being "try these addresses, and if nothing answers in 500 msec, assume 
it's empty".

Now, 500 msec may not sound very bad, but it all adds up. I get unhappy if 
my bootup is a minute. I'd prefer booting up in a couple of seconds.

Also, how ready do you want things to be? Do you want to know the device 
is there ("disk at physical location X exists"), or do you want to have 
read the UUID off the disk and partitioned it? The latter is what is 
needed for a mount, but it's often a _lot_ more expensive than just 
knowing the disk is there, and it's not even necessarily needed in many 
circumstances.

For example, say that you have more than just a couple of disks attached 
to the system, but many of them are for non-critical stuff. You do not 
necessarily want to wait for them all to spin up at all. You usually only 
care about one of them - the root device.

		Linus

Re: 2.6.16-rc4: known regressions

[Andrew Morton]

Linus Torvalds <torvalds@osdl.org> wrote:
>
> For example, say that you have more than just a couple of disks attached 
>  to the system, but many of them are for non-critical stuff. You do not 
>  necessarily want to wait for them all to spin up at all. You usually only 
>  care about one of them - the root device.

Well yes, but I was suggesting that userspace be given the option - run
insmod asynchronously if it's a problem.

It all does indicate that our single module_init(no args) interface is too
coarse...

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 11:54:10AM -0800, Andrew Morton wrote:
> Joel Becker <Joel.Becker@oracle.com> wrote:
> >
> > On Wed, Feb 22, 2006 at 06:33:54PM +0100, Gabor Gombas wrote:
> > > I don't think isnmod is broken. It's job is to load a chunk of code into
> > > the kernel, and it's doing just that.
> > > 
> > > ...
> > >
> > > But if your kernel has CONFIG_HOTPLUG enabled, then _you_ have asked for
> > > this exact behavior, therefore you should better fix userspace to cope
> > > with it. Your initrd should use the notification mechanisms provided by
> > > the kernel to wait for the would-be root device really becoming
> > > available; if it's not doing that, then IMHO you should not use a
> > > CONFIG_HOTPLUG enabled kernel.
> > 
> >         The issue isn't so much "insmod is right" vs "insmod is wrong".
> > It's that the behavior changed in a surprising fashion.  Red Hat's
> > kernel for RHEL4 (in our example) has CONFIG_HOTPLUG=y, yet it Just
> > Works.  A more recent kernel (.15 and .16 at least) with
> > CONFIG_HOTPLUG=y doesn't work.  Same disk drivers.  Same initramfs
> > script.
> > 	We're discussing this very "kernel change broke userspace
> > expectations" issue.  You don't need to convince me that
> > 
> >   1. Insmod loads the driver
> >   2. Userspace initramfs sleeps waiting for hotplug
> >   3. Hotplug completes
> >   4. Userspace initramfs continues, using the now plugged devices.
> 
> Yes, I tend to think that insmod should just block until all devices are
> ready to be used.  insmod doesn't just "insert a module".  It runs that
> module's init function.
> 
> The very common case is that userspace wants to use those devices
> immediately upon return from insmod, and the handling of these
> not-yet-ready devices from userspace is very hard - generally people just
> stick lame sleeps in there to get around it.
> 
> If, for some strange and rare reason, people _really_ want the
> return-when-its-not-ready-yet behaviour, they can do `insmod foo &', or we
> give insmod a fork-then-exit option.
> 
> But right now the default (and unalterable) behaviour is the oddball,
> rarely-desired and hard-to-handle one.

But people are currently working right now to make a totally async boot
process that takes advantage of this behavior.  It's the only way we can
make the boot process faster.  They are working on stuff like:
	- only when the network device is reported found by the kernel
	  do the things that rely on networking start up
	- only when the filesystems are found, do the things that rely
	  on them being there (lvm, evms, dm, etc.)

And as others have pointed out, for a lot of busses, we just don't know
how long the device is going to take to be found.  For one of my boxes
with a flaky USB controller, it takes _minutes_ for devices to be
detected (yes, it is broken hardware, but the rest of the system works
just fine while it is off and trying to be detected.)

Another point is that some busses just don't know when all the devices
on it are found, as there is no such state.  USB is one such bus, and I
imagine firewire is another one.  So there is no real way for us to
delay at insmod time for all devices to be found.

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Jody McIntyre]

On Wed, Feb 22, 2006 at 12:26:38PM -0800, Greg KH wrote:
> 
> Another point is that some busses just don't know when all the devices
> on it are found, as there is no such state.  USB is one such bus, and I
> imagine firewire is another one.  So there is no real way for us to
> delay at insmod time for all devices to be found.

Actually, we know what's present as soon as a bus reset completes, which
is measured in microseconds (1394a) or 300ms at worst (1394-1995.)
There is the case of a device that has to boot and isn't initially
available, but it will issue a reset when it comes online and our
subsystem will do detection again if needed.

Cheers,
Jody

> thanks,
> 
> greg k-h
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 

Re: 2.6.16-rc4: known regressions

[Diego Calleja]

El Wed, 22 Feb 2006 11:54:10 -0800,
Andrew Morton <akpm@osdl.org> escribió:

> Yes, I tend to think that insmod should just block until all devices are
> ready to be used.  insmod doesn't just "insert a module".  It runs that
> module's init function.

However, in current systems a device is ready only iff the corresponding
sysfs tree has been created or a hotplug event has be launched, and that's
the one sane place where userspace can wait for "something". Drivers need to
setup the name of the sysfs classes, so if modules could <crack smoking> 
export some of that info to insmod maybe insmod could be taught to do wait
for things in userspace or wait for events coming from the $FOO.ko module

(ok, maybe ugly but sounds somewhat cleaner than just adding a
"sleep(magicnumber)" to insmod)

Re: 2.6.16-rc4: known regressions

[Russell King]

On Wed, Feb 22, 2006 at 11:54:10AM -0800, Andrew Morton wrote:
> Yes, I tend to think that insmod should just block until all devices are
> ready to be used.  insmod doesn't just "insert a module".  It runs that
> module's init function.

Not always possible.  In the case of PCMCIA, we've had to run things
asynchronously because of the #$@$#@ driver model locking issues -
otherwise adding a PCI (cardbus) device while we're in the probe for
the yenta device deadlocked.

I suspect other subsystems also suffered from this, which unfortunately
makes your otherwise good idea a pipedream.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 09:19:44PM +0000, Russell King wrote:
> On Wed, Feb 22, 2006 at 11:54:10AM -0800, Andrew Morton wrote:
> > Yes, I tend to think that insmod should just block until all devices are
> > ready to be used.  insmod doesn't just "insert a module".  It runs that
> > module's init function.
> 
> Not always possible.  In the case of PCMCIA, we've had to run things
> asynchronously because of the #$@$#@ driver model locking issues -
> otherwise adding a PCI (cardbus) device while we're in the probe for
> the yenta device deadlocked.

That issue should be fixed now :)

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Bryan O'Sullivan]

On Wed, 2006-02-22 at 07:48 -0800, Joel Becker wrote:

> 	Do you mean that you are using a distro (eg, RHEL4 or something)
> with a mainline kernel?  We've seen something similar, and what we've
> determined is happening is that insmod is returning before the module is
> done initializing.

Yep, we've seen this with other SCSI drivers.  Our solution was to add a
"sleep 15" after each modprobe in the initrd, since SCSI drivers often
take a while to pull their thumbs out.

	<b

Re: 2.6.16-rc4: known regressions

[Greg KH]

On Wed, Feb 22, 2006 at 06:21:58AM -0500, Theodore Ts'o wrote:
> If we want more and more stuff to be poured into initrd, it's got to
> be made easier to debug and consistent across distributions, such that
> more people can test initrd configurations, and flush out the bugs,
> never mind the question of programs like udev randomly breaking
> between kernel releases.  Maybe it's time to consider moving all of
> that into the kernel source; if they wanted to be treated as part of
> the kernel, then let them liteally become part of the kernel from a
> source code and release management perspective.

With the klibc patches that move even more stuff in the early boot
process out into userspace, I would tend to agree with you about this.
If those changes ever go in, we will already have the framework to keep
these tools in the tree, and a method for building them

Discussions about if you want to use klibc for a long-running daemon
like udevd is for another time :)

thanks,

greg k-h

Re: 2.6.16-rc4: known regressions

[Matthias Andree]

On Tue, 21 Feb 2006, Linus Torvalds wrote:

> Side note: if people want to, we could have other "trampolines" like that, 
> so that we could have more user-level code that gets distributed with the 
> kernel. It doesn't have to be something that gets mapped into every binary 
> either: we could - if we wanted to - have things like shared libraries or 
> helper shell scripts or whatever that we expose in /sys/shlib/ that are 
> kernel-version dependent.
> 
> Then we could perhaps change more things, just because we could change the 
> wrappers that actually use them together with the kernel.

Go for it, something like that is long overdue -- since you cannot be
bothered to fork experimental kernels as playgrounds which only merge
when the dust has settled.

Still, /sys has apparently changed in an incompatible way, too, Douglas
Gilbert was forced to update his userspace stuff because sysfs cahnged
layout. This madness of constantly breaking user space applications
needs to stop.

-- 
Matthias Andree

Re: 2.6.16-rc4: known regressions

[Paulo Marques]

Linus Torvalds wrote:
> [...]
> Side note: if people want to, we could have other "trampolines" like that, 
> so that we could have more user-level code that gets distributed with the 
> kernel. It doesn't have to be something that gets mapped into every binary 
> either: we could - if we wanted to - have things like shared libraries or 
> helper shell scripts or whatever that we expose in /sys/shlib/ that are 
> kernel-version dependent.

Do you envision this being used for stuff like libalsa, libusb, a v4l2 
lib, etc.?

I always felt that this kind of libraries are sort of "part of the 
kernel" in the sense that programs really do need them to interface with 
the kernel. (*)

If we had a privelidged libv4l2 library like that then things like 
format conversion and video encoding / decoding could be done in user 
space and we could provide a more "high level" standard interface for 
user programs.

This is the sort of thing that libalsa already does with audio software 
mixing (for instance) with the advantage that we need to keep the 
interface between libalsa and the kernel across kernel versions.

Of course, the interface exported by these libraries would now be the 
official kernel interface.

-- 
Paulo Marques - www.grupopie.com

Pointy-Haired Boss: I don't see anything that could stand in our way.
            Dilbert: Sanity? Reality? The laws of physics?

(*) Yeah, one can write programs that don't use the libraries, but that 
is just asking for trouble...

Re: 2.6.16-rc4: known regressions

[Diego Calleja]

El Wed, 22 Feb 2006 01:04:29 +0100,
Kay Sievers <kay.sievers@suse.de> escribió:

> HAL was prepared to make use of the new events and needs to be upgraded
> when the kernel gets upgraded. This happens all the time as long as we

I noticed there is not a hal entry in Documentation/Changes, if hal
really depends on the kernel (it does?) maybe it should be added there.

Re: 2.6.16-rc4: known regressions

[Pekka J Enberg]

On Tue, 21 Feb 2006, Kay Sievers wrote:
> > 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> > diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> > Author: Kay Sievers <kay.sievers@suse.de>
> > Date:   Fri Nov 11 06:09:55 2005 +0100
> > 
> >     [PATCH] remove mount/umount uevents from superblock handling

On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> Upgrade HAL, it's too old for that kernel.

It's what Gentoo stable is carrying. Thou shalt not break userspace!

			Pekka

Re: 2.6.16-rc4: known regressions

[Kay Sievers]

On Wed, Feb 22, 2006 at 09:06:10AM +0200, Pekka J Enberg wrote:
> On Tue, 21 Feb 2006, Kay Sievers wrote:
> > > 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> > > diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> > > Author: Kay Sievers <kay.sievers@suse.de>
> > > Date:   Fri Nov 11 06:09:55 2005 +0100
> > > 
> > >     [PATCH] remove mount/umount uevents from superblock handling
> 
> On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> > Upgrade HAL, it's too old for that kernel.
> 
> It's what Gentoo stable is carrying. Thou shalt not break userspace!

Well, that's part of the contract by using an experimental version of HAL,
it has nothing to do with the kernel, as long as it's under
construction, you need to follow the latest releases. There is no
other way to do it, cause nobody can get complex software right in the
first place. So if that's a problem, don't depend on HAL until we
release a 1.0 version which will give you the needed stability. Just bug
the gentoo packager to catch up, cause there are more dependencies
anyway, not only a specific kernel version. And we don't fix any
bugs in any experimental version before 1.0, so please just help moving that
project faster forward by using the latest version, if you want to use
HAL.

Kay

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, Kay Sievers wrote:
> 
> Well, that's part of the contract by using an experimental version of HAL,
> it has nothing to do with the kernel

NO NO NO!

Dammit, if this is the logic and mode of operation of HAL people, then we 
must stop accepting patches to the kernel from HAL people.

THIS IS NOT DEBATABLE.

If you cannot maintain a stable kernel interface, then you damn well 
should not send your patches in for inclusion in the standard kernel. Keep 
your own "HAL-unstable" kernel and ask people to test it there.

It really is that easy. Once a system call or other kernel interface goes 
into the standard kernel, it stays that way. It doesn't get switched 
around to break user space.

Bugs happen, and sometimes we break user space by mistake. Sometimes it 
really really is inevitable. But we NEVER EVER say what you say: "it's 
your own fault". It's _our_ fault, and it's _our_ problem to work out.

Guys: you now have two choices: fix it by sending me a patch and an 
explanation of what went wrong, or see the patch that broke things be 
reverted. And STOP THIS DAMN APOLOGIA. 

I'm fed up with hearing how "breaking user space is ok because it's HAL or 
hotplug". IT IS NOT OK. Get your damn act together, and stop blaming other 
people. 

If the interfaces were bad, we keep them around. Look in fs/stat.c some 
day. Realize that some of those interfaces are from 1991. They were bad, 
but that doesn't change _anything_. People used them, and we had 
implemented them.

			Linus

Re: 2.6.16-rc4: known regressions

[Arjan van de Ven]

On Wed, 2006-02-22 at 07:44 -0800, Linus Torvalds wrote:


[snip lots of good words about that breaking userspace ABIs is really
horrible]

I absolutely agree with what you say. HOWEVER hal is also terminally
broken. The thing they depend on is a *config option*. If they can't
deal with that config option not being enabled in a graceful way, that's
a series malfunction.


(and no this is not an excuse for breaking userspace ABIs at all,
although one can argue that this removing is almost the same as
disabling the config option)

Re: 2.6.16-rc4: known regressions

[Christoph Hellwig]

On Wed, Feb 22, 2006 at 05:03:07PM +0100, Arjan van de Ven wrote:
> On Wed, 2006-02-22 at 07:44 -0800, Linus Torvalds wrote:
> 
> 
> [snip lots of good words about that breaking userspace ABIs is really
> horrible]
> 
> I absolutely agree with what you say. HOWEVER hal is also terminally
> broken. The thing they depend on is a *config option*. If they can't
> deal with that config option not being enabled in a graceful way, that's
> a series malfunction.
> 
> 
> (and no this is not an excuse for breaking userspace ABIs at all,
> although one can argue that this removing is almost the same as
> disabling the config option)

And to continue the rant: the broken mount uevent feature (which
can't work right) got in without any serious review through the
driver model tree.  just as all those break udev/etc patches that
cause all these userland breakages for those people brave enough
to use udev and surrounding bits.

Folks, we need to stop breaking sysfs interface all the time.  Having
attributes on objects is real nice from many perspectives, but it's
also a burden because the internal object model is now seen by the
outside world.  That means anything involving sysfs needs a careful
design not random patching as the driver model core people appear to
do.

Re: 2.6.16-rc4: known regressions

[David Zeuthen]

On Wed, 2006-02-22 at 07:44 -0800, Linus Torvalds wrote:
> 
> On Wed, 22 Feb 2006, Kay Sievers wrote:
> > 
> > Well, that's part of the contract by using an experimental version of HAL,
> > it has nothing to do with the kernel
> 
> NO NO NO!
> 
> Dammit, if this is the logic and mode of operation of HAL people, then we 
> must stop accepting patches to the kernel from HAL people.
> 
> THIS IS NOT DEBATABLE.
> 
> If you cannot maintain a stable kernel interface, then you damn well 
> should not send your patches in for inclusion in the standard kernel. Keep 
> your own "HAL-unstable" kernel and ask people to test it there.

Oh, you know, I don't think that's exactly how it works; HAL is pretty
much at the mercy of what changes goes into the kernel. And, trust me,
the changes we need to cope with from your so-called stable API are not
so nice. 

But I realize these changes are important because it's progress and back
in 2.6.0 things were horribly broken for at least desktop workloads [1].
It also makes me release note that newer HAL releases require newer
kernel and udev releases and that's alright. In fact it's perfectly
fine. We get users to upgrade to the latest and greatest and we keep
making good progress. That's open source at it's finest I think.

For just one example of API breaking see

 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998

This breaks stuff for end users in a stable distribution. Not good.

> It really is that easy. Once a system call or other kernel interface goes 
> into the standard kernel, it stays that way. It doesn't get switched 
> around to break user space.
> 
> Bugs happen, and sometimes we break user space by mistake. Sometimes it 
> really really is inevitable. But we NEVER EVER say what you say: "it's 
> your own fault". It's _our_ fault, and it's _our_ problem to work out.
> 
> Guys: you now have two choices: fix it by sending me a patch and an 
> explanation of what went wrong, or see the patch that broke things be 
> reverted. And STOP THIS DAMN APOLOGIA. 
> 
> I'm fed up with hearing how "breaking user space is ok because it's HAL or 
> hotplug". IT IS NOT OK. Get your damn act together, and stop blaming other 
> people. 

I think maintaining a stable syscall interface makes sense. Didn't you
once say that only the syscall interface was supposed to be stable? Or
was that Greg KH? I can't remember...

And I also think that breaking things like sysfs can be alright as long
as you coordinate it with major users of it, e.g. udev and HAL. Please
realize how useful the changes sysfs changes from 2.6.0 to present were
and... and that there still is a lot of work left to make certain things
work for desktop workloads.

I even think changing things like in the RH bug I linked to above is
fine provided that you mentioned it in the release notes...

One day perhaps sysfs will be "just right" and you can mark it as being
stable. I just don't think we're there yet. And I see no reason
whatsoever to paint things as black and white as you do.

    David (Please keep me Cc'ed, I can't keep up with lkml)

[1] : plug in a USB hub with other hubs attached and 10-20 USB devices;
works a lot better with current kernels and udev than it ever did in
2.6.0 with /sbin/hotplug (fork bomb)

Re: 2.6.16-rc4: known regressions

[Christoph Hellwig]

On Wed, Feb 22, 2006 at 11:18:22AM -0500, David Zeuthen wrote:
> For just one example of API breaking see
> 
>  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998
> 
> This breaks stuff for end users in a stable distribution. Not good.

That's not an API breakage.  The API is left untouched, the driver
now just reports the attached device as what it is.  If HAL wasn't
a piece of cargo-cult programming crap you'd see in the relevant
standards what scsi device types exist, or even better stop relying
on such low-level knowledge.  It's a disk if sd_mod attaches to it
is a much better rule than relying on lowlevel SAM details.

Re: 2.6.16-rc4: known regressions

[David Zeuthen]

On Wed, 2006-02-22 at 16:35 +0000, Christoph Hellwig wrote:
> On Wed, Feb 22, 2006 at 11:18:22AM -0500, David Zeuthen wrote:
> > For just one example of API breaking see
> > 
> >  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998
> > 
> > This breaks stuff for end users in a stable distribution. Not good.
> 
> That's not an API breakage.  The API is left untouched, the driver
> now just reports the attached device as what it is.

ABI then, whatever..

>   If HAL wasn't
> a piece of cargo-cult programming crap 

We love you too Christoph.

> you'd see in the relevant
> standards what scsi device types exist, or even better stop relying
> on such low-level knowledge.  

Then don't export it unless it's useful. You did break ABI, don't try to
make up stupid excuses.

> It's a disk if sd_mod attaches to it
> is a much better rule than relying on lowlevel SAM details.

That's another way to do it.

    David

Re: 2.6.16-rc4: known regressions

[Christoph Hellwig]

On Wed, Feb 22, 2006 at 11:46:01AM -0500, David Zeuthen wrote:
> > you'd see in the relevant
> > standards what scsi device types exist, or even better stop relying
> > on such low-level knowledge.  
> 
> Then don't export it unless it's useful. You did break ABI, don't try to
> make up stupid excuses.

It's _not_ and abi break.  TYPE_RBC is a valid scsi disk device that could
happen on any SAM transport, not just firewire.  That sbp2 altered the
device type just papered over the crappy hal code as long as none of your
users had one of the non-firewire rbc devices.

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, David Zeuthen wrote:
> 
> Oh, you know, I don't think that's exactly how it works; HAL is pretty
> much at the mercy of what changes goes into the kernel. And, trust me,
> the changes we need to cope with from your so-called stable API are not
> so nice. 

Why do you "cope"?

Start complaining. If kernel changes screw up something, COMPLAIN. Loudly. 
They shouldn't.

> It also makes me release note that newer HAL releases require newer
> kernel and udev releases and that's alright.

It's _somewhat_ ok to have a well-defined one-way dependency. It's sad, 
but inevitable sometimes.

For example, the kernel does have a dependency on the compiler used to 
compile it. We try to avoid it as far as possible, but we've slowly been 
updating it, first from 1.40 to 2.75 to 2.9x and now to 3.1. But the 
kernel obviously shouldn't have any other run-time dependencies, because 
everything else is "on top of" the kernel.

What is NOT ok is to have a two-way dependency. If user-space HAL code 
depends on a new kernel, that's ok, although I suspect users would hope 
that it wouldn't be "kernel of the week", but more a "kernel of the last 
few months" thing. 

But if you have a TWO-WAY dependency, you're screwed. That means that you 
have to upgrade in lock-step, and that just IS NOT ACCEPTABLE. It's 
horrible for the user, but even more importantly, it's horrible for 
developers, because it means that you can't say "a bug happened" and do 
things like try to narrow it down with bisection or similar.

> For just one example of API breaking see
> 
>  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998

So the kernel obviously shouldn't be just randomly changing the type 
numbers around. 

The _real_ bug seems to be that some people think it is OK to do this kind 
of user-visible changes, without even considering the downstream, or 
indeed, without even telling anybody else (like Andrew or me) about it.

> This breaks stuff for end users in a stable distribution. Not good.

Indeed. Not good at all.

And yes, some of it may be just HAL being a fragile mess, and some of it 
may end up being just user-level code that must be made to be more robust 
("I see a new type I don't understand" "Ok, assume a lowest common 
denominator, and stop whining about it"). 

But a lot of it is definitely some kernel people being _waayy_ too 
cavalier about userspace-visible changes.

> I think maintaining a stable syscall interface makes sense. Didn't you
> once say that only the syscall interface was supposed to be stable? Or
> was that Greg KH? I can't remember...

It's _not_ just system calls. It's any user-visible stuff. That very much 
includes /proc, /sys, and any "kernel pipes" aka netlink etc bytestreams.

What is not stable is the _internal_ data structures. We break external 
modules, and we sometimes break even in-kernel drivers etc with abandon, 
if that is what it takes to fix something or make it prettier.

So fcntl and ioctl numbers etc are _inviolate_, because they are part of 
the system interface. As is /proc and /sys. We don't change them just 
because it's "convenient" to change them in the kernel. 

If /sys needs an extended type to describe the command set of a device, we 
do NOT just change an existing attribute in /sys. 

> And I also think that breaking things like sysfs can be alright as long
> as you coordinate it with major users of it, e.g. udev and HAL.

The major users are USERS. Not developers. It doesn't help to "coordinate" 
things, when what gets screwed is the end-user who no longer can upgrade 
his kernel without worrying that something might break.

THIS IS WHY WE MUST MAKE THE KERNEL INTERFACES STABLE!

If users cannot upgrade their kernels safely, we will have two totally
unacceptable end results:

 - users won't upgrade. They don't dare to, because it's too painful, and 
   they don't understand HAL or hotplug, or whatever. 

   If a developer cannot see that this is unacceptable, then that 
   developer is a nincompoop and needs to be educated.

 - users upgrade, and generate bug reports and waste other developers time 
   because those other developers didn't realize that the HAL cabal had 
   decided that that breakage was "ok".

   Or worse, they don't generate the bug reports, and then six months from 
   now, when they test again, and it's still broken, they generate a 
   really bad one ("it doesn't work") when everybody - including the HAL 
   cabal - has forgotten what it was all about.

   Again, if a developer cannot see that this is unacceptable, then that 
   developer is not playing along, and needs to have his mental compass 
   re-oriented.

The fact is, regressions are about 10x more costly than fixing old bugs. 
They cause problems downstream that just waste everybodys time. It's a 
_hell_ of a lot more efficient to spend extra time to keep old interfaces 
stable than it is to cause regressions.

> One day perhaps sysfs will be "just right" and you can mark it as being
> stable. I just don't think we're there yet. And I see no reason
> whatsoever to paint things as black and white as you do.

Nothing will _ever_ be "just right", and this has been going on too long. 
We had better get our act together.

		Linus

Re: 2.6.16-rc4: known regressions

[Linus Torvalds]

On Wed, 22 Feb 2006, Linus Torvalds wrote:
> 
> > For just one example of API breaking see
> > 
> >  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998
> 
> And yes, some of it may be just HAL being a fragile mess, and some of it 
> may end up being just user-level code that must be made to be more robust 
> ("I see a new type I don't understand" "Ok, assume a lowest common 
> denominator, and stop whining about it"). 

Btw, having looked at that bug report some more, I have to say that this 
particular one seems to be of the "HAL is just being an ass about things" 
variety.

Why the hell anybody would care about what the command transport type is, 
when all that matters is that it's a block device, I don't understand. The 
exact details of what kind of block device it is are totally secondary, 
and shouldn't affect basic desktop behaviour.

The patch (to HAL) that the bugzilla entry points to doesn't seem to make 
anything better either. It just adds _another_ magic case-statement. 
Instead, it should just default to doing something sane.

			Linus

Re: 2.6.16-rc4: known regressions

[Al Viro]

On Wed, Feb 22, 2006 at 09:31:59AM -0800, Linus Torvalds wrote:
> Why the hell anybody would care about what the command transport type is, 
> when all that matters is that it's a block device, I don't understand. The 
> exact details of what kind of block device it is are totally secondary, 
> and shouldn't affect basic desktop behaviour.

Actually, it's not about transport, it's about command _set_.  So there
is legitimate userland code that would want to know that (especially since
a lot of external enclosures have incredibly brittle and crappy firmware
and go tits-up when they see anything they don't recognize), but
	a) the last thing that code wants is to have TYPE_RBC mislabeled
as TYPE_DISK and
	b) hal has nothing to do with that.

The only place where _transport_ enters the picture is that RBC is very common
in e.g. firewire-to-IDE bridges, so sbp2 had to deal with it somehow.  And
instead of teaching sd.c to deal with those (it's very easy) it went ahead
and just marked those as type 0 (disk).  Almost worked...

Re: 2.6.16-rc4: known regressions

[John Stoffel]

>>>>> "Al" == Al Viro <viro@ftp.linux.org.uk> writes:

Al> The only place where _transport_ enters the picture is that RBC is
Al> very common in e.g. firewire-to-IDE bridges, so sbp2 had to deal
Al> with it somehow.  And instead of teaching sd.c to deal with those
Al> (it's very easy) it went ahead and just marked those as type 0
Al> (disk).  Almost worked...

I wonder if this change will fix all the problems I've had trying to
make my prolific chipset (crap) Firewire/USB enclosure to actually
work when used with firewire?  It's finally stable with USB, but
obviously not the best performance.

Gotta try this again sometime.

John

Re: 2.6.16-rc4: known regressions

[Al Viro]

On Wed, Feb 22, 2006 at 09:08:47AM -0800, Linus Torvalds wrote:
> >  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=175998
> 
> So the kernel obviously shouldn't be just randomly changing the type 
> numbers around. 

> The _real_ bug seems to be that some people think it is OK to do this kind 
> of user-visible changes, without even considering the downstream, or 
> indeed, without even telling anybody else (like Andrew or me) about it.

That's not quite true...

Some background: sbp2 took SCSI devices of type 14 (very reduced and slightly
incompatible version of "SCSI disk", fairly common for external disks) and
forcibly marked them as type 0.  Since sd.c had no way to tell whether it's
dealing with normal SCSI disk or with RBC one, it was unable to tell how
to find out whether the cache on that disk is write-through or write-behind
(that being one of the incompatibilities).

That leads to actual data corruption on reboot, BTW - some of these guys
simply lose the contents of cache at that.

Obvious fix?  Make sd.c deal with RBC (note that it's a valid SCSI type -
you bloody well can have it for a device attached to any SCSI bus, not
just firewire) and leave the sdev->type intact, so that sd.c could know
what's going on.  Right?

As it turns out, sdev->type is not just exposed to userland via sysfs
(that has legitimate uses), it's exposed to userland that happens to be
braindead.  There are two questions:
	* what commands does that device accept?
	* is there an sd<...> block device for it?
Both are valid for userland.  E.g. stuff like scsiinfo, etc., is issuing
SCSI commands via SG_IO.  And yes, knowing the device type is very, very
useful there.  For that we actually would want accurate type, for the same
reasons why we want it in sd.c.  The second question ("is there an sd.c
block device for this guy?") also is valid and has a sane answer in sysfs.

What got broken?  Code that used to assume that sd.c will never, ever handle
openly RBC disks.  As long as that remained true, userland could assume that
"sd fodder" and "has type 0" were the same.  Which was never guaranteed.

Re: 2.6.16-rc4: known regressions

[Christoph Hellwig]

On Wed, Feb 22, 2006 at 05:51:31PM +0000, Al Viro wrote:
> What got broken?  Code that used to assume that sd.c will never, ever handle
> openly RBC disks.  As long as that remained true, userland could assume that
> "sd fodder" and "has type 0" were the same.  Which was never guaranteed.

sd also has been handling TYPE_MOD forever, which HAL still doesn't deal
with.  Not that it should care at all about the scsi command type as
you mentioned..

Re: 2.6.16-rc4: known regressions

[Al Viro]

On Wed, Feb 22, 2006 at 05:55:06PM +0000, Christoph Hellwig wrote:
> On Wed, Feb 22, 2006 at 05:51:31PM +0000, Al Viro wrote:
> > What got broken?  Code that used to assume that sd.c will never, ever handle
> > openly RBC disks.  As long as that remained true, userland could assume that
> > "sd fodder" and "has type 0" were the same.  Which was never guaranteed.
> 
> sd also has been handling TYPE_MOD forever, which HAL still doesn't deal
> with.  Not that it should care at all about the scsi command type as
> you mentioned..

Oh, right - magneto-optical is also there.  I suspect that HAL doesn't
really care, along the lines of "they are all removable anyway"...

Re: 2.6.16-rc4: known regressions

[David Zeuthen]

(agreeing with you on lots of counts so cutting to the chase)

On Wed, 2006-02-22 at 09:08 -0800, Linus Torvalds wrote:
> THIS IS WHY WE MUST MAKE THE KERNEL INTERFACES STABLE!

You really need to sit down and define what you mean by stable then. For
example the syscall interface is somewhat well defined.

This is how the world looks like today from a consumer of sysfs with
little or no kernel programming experience. Just a few examples:

 1. There is little or no documentation sans kernel sources for
    the meaning nor range of the values of sysfs files. You have to
    lookup the kernel source... Maybe it would be useful to have
    some kind of auto-documentation feature that _forces_ kernel
    developers to provide docs along with the sysfs files..
    Then I would have

     /sys/block/hda/removable

    and 

     /sys/block/hda/docs_removable

    Of course make this an optional feature. Noteworthy this is how
    gconf in GNOME works.. sure, now I get flamed for bloat and, sure,
    this may be a stupid idea... Mostly I'm just trying to outline
    the problem here...

 2. Some of the interesting information we want isn't actually
    available even though the kernel has it already (can we please
    get an event when the kernel has finished scanning for partitions
    for example?). We're pretty fine getting some high-level information
    ourselves, like probing for file system for example.. but some
    things we can't really get at.. 

 3. User space needs to reorder hotplug events; that's fine but we get
    to play a lot of tricks because there are races everywhere (look at
    some of the udev rules for working around this).. I'm not sure how
    this is fixable...

 4. Back in 2.6.9 or 2.6.10 someone yanked the SCSI targets into the
    sysfs chain and that did break HAL. Depending on who you ask this
    is acceptable, other people says it's ABI breakage. Again, need to
    define what's stable means; I don't think it's good enough to just
    wait until it happens and then let yourself or some other high-level
    person decide whether a change is acceptable or not. We need
    predictability.

All these things.. what sysfs files to expect.. proper documentation..
what values a sysfs file can assume.. is, to me at least, all part of an
"interface"... an ABI.

The root problem, I think, here is really the lack of communication
between kernel developers and user space people. It's not like HAL is
closed source nor a lot of code. 

I believe that the problems we have in HAL are very fixable but the
thing is that with the "documentation" and "stability guarantees" that
the kernel today gives us... you pretty much have to be a kernel
developer to figure when or if some sysfs file can assume a new value..
or that there's a better sysfs file to check that this or that one...
Sorry, but I'd rather spend my time making the desktop bits use HAL to
implement a nice user-visible feature... than spending time figuring out
the exact semantics of some arcane file in sysfs. 

At this point I would welcome any kernel developer to help review the
HAL code and send patches for stupid assumptions made in HAL. I would
really appreciate it. I'm not kidding. Thanks.

    David

Re: 2.6.16-rc4: known regressions

[Al Viro]

On Wed, Feb 22, 2006 at 11:18:22AM -0500, David Zeuthen wrote:
> Oh, you know, I don't think that's exactly how it works; HAL is pretty
> much at the mercy of what changes goes into the kernel. And, trust me,
> the changes we need to cope with from your so-called stable API are not
> so nice. 
> 
> But I realize these changes are important because it's progress and back
> in 2.6.0 things were horribly broken for at least desktop workloads [1].
> It also makes me release note that newer HAL releases require newer
> kernel and udev releases and that's alright. In fact it's perfectly
> fine. We get users to upgrade to the latest and greatest and we keep
> making good progress. That's open source at it's finest I think.

No, it is not.  Just try to find a point where breakage had been introduced
into e.g. a driver, when known-good and known-bad versions are on the
different sides of your change requiring userland "upgrade".

Re: 2.6.16-rc4: known regressions

[grundig]

El Wed, 22 Feb 2006 11:18:22 -0500,
David Zeuthen <david@fubar.dk> escribió:


> But I realize these changes are important because it's progress and back
> in 2.6.0 things were horribly broken for at least desktop workloads [1].

As long as those changes don't come suddenly in the night, without a
transition period...

Check for Documentation/feature-removal-schedule.txt for some examples
of how to deprecate userland interfaces properly.

Re: 2.6.16-rc4: known regressions

[Martin Bligh]

David Zeuthen wrote:
> On Wed, 2006-02-22 at 07:44 -0800, Linus Torvalds wrote:
> 
>>On Wed, 22 Feb 2006, Kay Sievers wrote:
>>
>>>Well, that's part of the contract by using an experimental version of HAL,
>>>it has nothing to do with the kernel
>>
>>NO NO NO!
>>
>>Dammit, if this is the logic and mode of operation of HAL people, then we 
>>must stop accepting patches to the kernel from HAL people.
>>
>>THIS IS NOT DEBATABLE.
>>
>>If you cannot maintain a stable kernel interface, then you damn well 
>>should not send your patches in for inclusion in the standard kernel. Keep 
>>your own "HAL-unstable" kernel and ask people to test it there.
> 
> 
> Oh, you know, I don't think that's exactly how it works; HAL is pretty
> much at the mercy of what changes goes into the kernel. And, trust me,
> the changes we need to cope with from your so-called stable API are not
> so nice. 
> 
> But I realize these changes are important because it's progress and back
> in 2.6.0 things were horribly broken for at least desktop workloads [1].
> It also makes me release note that newer HAL releases require newer
> kernel and udev releases and that's alright. In fact it's perfectly
> fine. We get users to upgrade to the latest and greatest and we keep
> making good progress. That's open source at it's finest I think.

If it's all that fragile, surely it just means that someone picked the
wrong point at which to try to form the API abstraction?

Frankly, that seems to be the issue behind a lot of these problems -
people decide to shove stuff into userspace for some religions reason, 
without thinking about the API implications at all.

We don't have a sane way to package all the userspace crud together
with the microkernel that people are turning Linux into. Either people
quit pretending that divesting things to userspace is a solution to all
hard problems, or we create a packaging / bundling mechanism for all
this shite. Frankly, I prefer the former, but whichever ... it's
getting insane.

M.

Re: 2.6.16-rc4: known regressions

[Theodore Ts'o]

On Wed, Feb 22, 2006 at 09:14:59AM -0800, Martin Bligh wrote:
> >But I realize these changes are important because it's progress and back
> >in 2.6.0 things were horribly broken for at least desktop workloads [1].
> >It also makes me release note that newer HAL releases require newer
> >kernel and udev releases and that's alright. In fact it's perfectly
> >fine. We get users to upgrade to the latest and greatest and we keep
> >making good progress. That's open source at it's finest I think.
> 
> If it's all that fragile, surely it just means that someone picked the
> wrong point at which to try to form the API abstraction?
> 
> Frankly, that seems to be the issue behind a lot of these problems -
> people decide to shove stuff into userspace for some religions reason, 
> without thinking about the API implications at all.

Martin has hit the nail on the head.

There is currently a religion going on in some circles (we see it in
the uswsusp vs suspend2 debate) which states that moving functionality
to userspace is always better because it makes the kernel "simpler".
Well, maybe.  To the extent that we move policy to userspace, that is
(usually) goodness, but we have to weigh the resulting _interface_
complexity.  When you take a piece of work and split it up between the
kernel and userspace, by definition there will have to be some kind of
interface between the kernel and the userspace code.  

Some people assume the only thing that makes up the interface is
syscalls, ioctl's, and fnctls, but that's not true; /proc and /sys are
interfaces too.  And as Linus has stated, once we introduce an
interface, that's it; we have to maintain it forever.  No gratuitous
changes.  If that is too hard, because we can imagine potential
changes that might require us to change the interface, or painful
backwards compatibility kludges to maintain the old interface for at
least 12 months --- then maybe it was a bad idea to move certain
pieces of functionality into userspace in the first place.

> We don't have a sane way to package all the userspace crud together
> with the microkernel that people are turning Linux into. Either people
> quit pretending that divesting things to userspace is a solution to all
> hard problems, or we create a packaging / bundling mechanism for all
> this shite. Frankly, I prefer the former, but whichever ... it's
> getting insane.

Precisely.  These days, using initrd is an exercise in pain, and
wasted time; anything goes wrong, and there is no recovery whatsoever,
except a reboot back to a working setup, and if you have multiple SCSI
drivers, a 5-10 minute wait for the boot to cycle.  So I don't use it.
And if there is functionality that requires initrd's, as a general
rule I don't use it, and I don't test it.  And if it's just me being
stupid, then everyone can ignore me.  But if it's many people then
maybe folks should start considering that we either need to make
initrd more robust, and probably start bundling initrd setup's into
the kernel, or we should start reconsidering the whole plan of moving
more and more into initrd in the first place.

'						- Ted

Re: 2.6.16-rc4: known regressions

[Pekka Enberg]

Hi Kay,

On Wed, 2006-02-22 at 16:27 +0100, Kay Sievers wrote:
> Well, that's part of the contract by using an experimental version of HAL,
> it has nothing to do with the kernel, as long as it's under
> construction, you need to follow the latest releases.

That's not how you do it in a stable kernel. Please follow the proper
procedure and add it to Documentation/feature-removal-schedule.txt. In
the meanwhile, I sent a patch to Andrew to revert your change. Thanks.

			Pekka

Re: 2.6.16-rc4: known regressions

[Arjan van de Ven]

On Tue, 2006-02-21 at 23:57 +0100, Kay Sievers wrote:
> On Wed, Feb 22, 2006 at 12:51:01AM +0200, Pekka Enberg wrote:
> > On Sun, 2006-02-19 at 17:02 -0800, Greg KH wrote:
> > > If you revert this one patch, on top of a clean 2.6.16-rc4, do things
> > > start working for you again?
> > 
> > Okey dokey, bisecting with mrproper took little longer than expected but
> > I found the bad changeset:
> > 
> > 033b96fd30db52a710d97b06f87d16fc59fee0f1 is first bad commit
> > diff-tree 033b96fd30db52a710d97b06f87d16fc59fee0f1 (from 0f76e5acf9dc788e664056dda1e461f0bec93948)
> > Author: Kay Sievers <kay.sievers@suse.de>
> > Date:   Fri Nov 11 06:09:55 2005 +0100
> > 
> >     [PATCH] remove mount/umount uevents from superblock handling
> 
> Upgrade HAL, it's too old for that kernel.


thats not a good answer ;)