2.6.25: random stalls on certain hardware - regression?

2.6.25: random stalls on certain hardware - regression?

[Michael Tokarev]

[-- Attachment #1: Type: text/plain, Size: 3894 bytes --]

No hope to resolve this but still - maybe someone has an idea...

Hardware is -
  AMD Athlon x2-64 system
  Asus M2N-SLI DELUXE motherboard (nvidia MCP55)
  AMD BE-2400 CPU
  2x 2Gb mem
  Adaptec 3950B Ultra2 SCSI adapter (*)
  4x 36Gb Seagate SCSI disks
  2x nvidia GbE ethernet
  rtl8139 nic as 3rd one

When booting 2.6.25 - either 32 or 64 bits - the system freezes/hangs
at some random point.  So far there was about 10 hangs, some after
several minutes after boot, some are within hours.

The "hang" is a complete system freeze - on the console there's
still "... login:" prompt, but nothing works - keyboard is stuck
(numlock doesn't work), and server is not responding over network.
Sometimes ping to the server itself works, but definitely not routing.
(it's a server so no fancy stuff is loaded - X isn't even installed).

The kernel is vanilla 2.6.25 - tried several - .5, .8, .10 now -
the effect is the same.

2.6.24 and before worked without any glitch (2.6.24 is currently
running).  We tried different versions of BIOS (was 12something
before, tried 1304 and - currently - 1405, since 1502 is still
beta) - no difference at all.

The problem is that it is a production machine, and quite some
people depend on it (it's a remote office with only one server),
so I've very limited ability to try something.  Unfortunately not
git bisect, -- or at least I'm afraid to try it, both because of
possibility to have many reboots AND new freezes, and because
unstable kernel (2.6.25pre stuff) with possibility to break something.

Here's the modules:

ipt_REJECT              3968  2
xt_tcpudp               3712  1
xt_comment              2432  8
iptable_filter          3456  1
ip_tables              11536  1 iptable_filter
x_tables               13316  4 ipt_REJECT,xt_tcpudp,xt_comment,ip_tables
quota_v2                9728  6
xfs                   518456  1
raid0                   7040  1
raid10                 19456  1
usblp                  12416  0
raid456               116764  4
async_xor               3072  1 raid456
async_memcpy            2432  1 raid456
async_tx                2944  1 raid456
xor                    15756  2 raid456,async_xor
usb_storage            81984  0
it87                   20368  0
hwmon_vid               3584  1 it87
hwmon                   3100  1 it87
ohci_hcd               21380  0
ehci_hcd               31756  0
usbcore               119536  5 usblp,usb_storage,ohci_hcd,ehci_hcd
8250_pnp               10112  0
8250                   24196  1 8250_pnp
serial_core            18304  1 8250
forcedeth              46988  0
8139too                22656  0
mii                     5376  1 8139too
ext3                  119944  6
jbd                    39828  1 ext3
mbcache                 7300  1 ext3
aic7xxx               166968  32
scsi_transport_spi     22144  1 aic7xxx
raid1                  19712  3
md_mod                 69532  13 raid0,raid10,raid456,raid1
sd_mod                 25112  36
scsi_mod              127144  4 usb_storage,aic7xxx,scsi_transport_spi,sd_mod

i.e, nothing fancy at all, all the standard modules.

The difference in compilation - between all 2.6.25 and first 2.6.24 -
is the compiler, currently it's Debian 4.2.3-3 (testing), before it
was some older version.  But latest 2.6.24.7 were also compiled with
this very compiler and it works like usual.

Config is attached, just in case (i686smp).

Any idea how to start welcome.  Meanwhile I'll try current 2.6.26pre,
in a hope it will not destroy all our data.. ;)

Thanks!

(*) the hardware may seem a bit strange - an old SCSI controller on
a pretty modern system - but it was other, about 10 years old system
where the motherboard failed and has been replaced with this asus
one, but the drives (and the controller) are ok.  The mobo has enough
PCI slots (and 2 NICs) so that all the devices are here - something
I had.. difficulties to find.

/mjt

[-- Attachment #2: i686smp --]
[-- Type: text/plain, Size: 34902 bytes --]

3C515=m
4KSTACKS=y
60XX_WDT=n
64BIT=n
8139CP=m
8139TOO=m
8139TOO_8129=y
8139TOO_PIO=n
8139TOO_TUNE_TWISTER=n
8139_OLD_RX_RESET=n
AC3200=n
AC97_BUS=m
ACENIC=m
ACENIC_OMIT_TIGON_I=n
ACORN_PARTITION=n
ACPI=y
ACPI_AC=m
ACPI_ASUS=m
ACPI_BATTERY=m
ACPI_BAY=m
ACPI_BLACKLIST_YEAR=0
ACPI_BUTTON=m
ACPI_CONTAINER=m
ACPI_CUSTOM_DSDT=n
ACPI_DEBUG=n
ACPI_DOCK=m
ACPI_EC=y
ACPI_FAN=m
ACPI_HOTPLUG_CPU=y
ACPI_POWER=y
ACPI_PROCESSOR=m
ACPI_PROCFS=n
ACPI_PROCFS_POWER=n
ACPI_PROC_EVENT=y
ACPI_SBS=n
ACPI_SLEEP=y
ACPI_SYSFS_POWER=y
ACPI_SYSTEM=y
ACPI_THERMAL=m
ACPI_TOSHIBA=m
ACPI_VIDEO=m
ACPI_WMI=m
ACQUIRE_WDT=n
ADAPTEC_STARFIRE=n
ADFS_FS=n
ADM8211=n
ADVANTECH_WDT=n
AFFS_FS=n
AFS_FS=n
AF_RXRPC=n
AGP=m
AGP_ALI=m
AGP_AMD64=m
AGP_AMD=m
AGP_ATI=m
AGP_EFFICEON=n
AGP_INTEL=m
AGP_NVIDIA=m
AGP_SIS=m
AGP_SWORKS=m
AGP_VIA=m
AIC79XX_CMDS_PER_DEVICE=32
AIC79XX_DEBUG_ENABLE=n
AIC79XX_DEBUG_MASK=0
AIC79XX_REG_PRETTY_PRINT=y
AIC79XX_RESET_DELAY_MS=5000
AIC7XXX_CMDS_PER_DEVICE=32
AIC7XXX_DEBUG_ENABLE=n
AIC7XXX_DEBUG_MASK=0
AIC7XXX_REG_PRETTY_PRINT=y
AIC7XXX_RESET_DELAY_MS=5000
AIRO=m
AIRO_CS=n
ALIM1535_WDT=n
ALIM7101_WDT=n
AMD8111_ETH=n
AMIGA_PARTITION=n
ANON_INODES=y
APM=n
APPLICOM=n
APRICOT=n
ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
ARCH_ENABLE_MEMORY_HOTPLUG=y
ARCH_FLATMEM_ENABLE=y
ARCH_HAS_CPU_IDLE_WAIT=y
ARCH_HAS_CPU_RELAX=y
ARCH_HAS_ILOG2_U32=n
ARCH_HAS_ILOG2_U64=n
ARCH_HIBERNATION_POSSIBLE=y
ARCH_MAY_HAVE_PC_FDC=y
ARCH_POPULATES_NODE_MAP=y
ARCH_SELECT_MEMORY_MODEL=y
ARCH_SPARSEMEM_ENABLE=y
ARCH_SUPPORTS_AOUT=y
ARCH_SUPPORTS_MSI=y
ARCH_SUSPEND_POSSIBLE=y
ARCNET=n
ARPD=n
ASK_IP_FIB_HASH=y
ASYNC_CORE=m
ASYNC_MEMCPY=m
ASYNC_XOR=m
AT1700=n
ATA=m
ATALK=n
ATARI_PARTITION=n
ATA_ACPI=y
ATA_GENERIC=n
ATA_NONSTANDARD=n
ATA_OVER_ETH=n
ATA_PIIX=m
ATH5K=m
ATL1=m
ATM=m
ATMEL=n
ATM_BR2684=n
ATM_CLIP=m
ATM_CLIP_NO_ICMP=n
ATM_DRIVERS=n
ATM_LANE=n
ATP=n
AUDIT=n
AUDIT_ARCH=n
AUTOFS4_FS=m
AUTOFS_FS=n
AUXDISPLAY=n
B43=m
B43LEGACY=m
B43LEGACY_DEBUG=y
B43LEGACY_DMA=y
B43LEGACY_DMA_AND_PIO_MODE=y
B43LEGACY_DMA_MODE=n
B43LEGACY_PCICORE_AUTOSELECT=y
B43LEGACY_PCI_AUTOSELECT=y
B43LEGACY_PIO=y
B43LEGACY_PIO_MODE=n
B43LEGACY_RFKILL=y
B43_DEBUG=n
B43_PCICORE_AUTOSELECT=y
B43_PCI_AUTOSELECT=y
B43_PCMCIA=n
B43_RFKILL=y
B44=n
BACKLIGHT_CLASS_DEVICE=m
BACKLIGHT_CORGI=n
BACKLIGHT_LCD_SUPPORT=n
BACKLIGHT_PROGEAR=n
BACKTRACE_SELF_TEST=n
BASE_FULL=y
BASE_SMALL=0
BATTERY_DS2760=n
BEFS_FS=n
BFS_FS=n
BINFMT_AOUT=n
BINFMT_ELF=y
BINFMT_MISC=m
BITREVERSE=y
BLK_CPQ_CISS_DA=m
BLK_CPQ_DA=n
BLK_DEV=y
BLK_DEV_3W_XXXX_RAID=n
BLK_DEV_4DRIVES=n
BLK_DEV_AEC62XX=n
BLK_DEV_ALI14XX=n
BLK_DEV_ALI15X3=n
BLK_DEV_AMD74XX=m
BLK_DEV_ATIIXP=m
BLK_DEV_BSG=n
BLK_DEV_CMD640=m
BLK_DEV_CMD640_ENHANCED=n
BLK_DEV_CMD64X=m
BLK_DEV_COW_COMMON=n
BLK_DEV_CRYPTOLOOP=n
BLK_DEV_CS5520=n
BLK_DEV_CS5530=n
BLK_DEV_CS5535=m
BLK_DEV_CY82C693=n
BLK_DEV_DAC960=n
BLK_DEV_DELKIN=n
BLK_DEV_DM=m
BLK_DEV_DTC2278=n
BLK_DEV_FD=m
BLK_DEV_GENERIC=m
BLK_DEV_HD=n
BLK_DEV_HD_IDE=n
BLK_DEV_HPT34X=n
BLK_DEV_HPT366=n
BLK_DEV_HT6560B=n
BLK_DEV_IDE=m
BLK_DEV_IDEACPI=y
BLK_DEV_IDECD=m
BLK_DEV_IDECD_VERBOSE_ERRORS=y
BLK_DEV_IDECS=n
BLK_DEV_IDEDISK=m
BLK_DEV_IDEDMA=y
BLK_DEV_IDEDMA_PCI=y
BLK_DEV_IDEDMA_SFF=y
BLK_DEV_IDEFLOPPY=m
BLK_DEV_IDEPCI=y
BLK_DEV_IDEPNP=n
BLK_DEV_IDESCSI=m
BLK_DEV_IDETAPE=m
BLK_DEV_IDE_SATA=n
BLK_DEV_INITRD=y
BLK_DEV_IO_TRACE=n
BLK_DEV_IT8213=n
BLK_DEV_IT821X=n
BLK_DEV_JMICRON=n
BLK_DEV_LOOP=m
BLK_DEV_MD=m
BLK_DEV_NBD=m
BLK_DEV_NS87415=n
BLK_DEV_OFFBOARD=n
BLK_DEV_OPTI621=n
BLK_DEV_PDC202XX_NEW=n
BLK_DEV_PDC202XX_OLD=n
BLK_DEV_PIIX=m
BLK_DEV_PLATFORM=m
BLK_DEV_QD65XX=n
BLK_DEV_RAM=m
BLK_DEV_RAM_COUNT=16
BLK_DEV_RAM_SIZE=4096
BLK_DEV_RZ1000=n
BLK_DEV_SC1200=n
BLK_DEV_SD=m
BLK_DEV_SIIMAGE=m
BLK_DEV_SIS5513=m
BLK_DEV_SLC90E66=n
BLK_DEV_SR=m
BLK_DEV_SR_VENDOR=n
BLK_DEV_SVWKS=m
BLK_DEV_SX8=n
BLK_DEV_TC86C001=n
BLK_DEV_TRIFLEX=n
BLK_DEV_TRM290=n
BLK_DEV_UB=n
BLK_DEV_UMC8672=n
BLK_DEV_UMEM=n
BLK_DEV_VIA82CXXX=m
BLK_DEV_XD=n
BLK_DEV_XIP=n
BLOCK=y
BNX2=m
BNX2X=n
BONDING=m
BOOT_PRINTK_DELAY=n
BOUNCE=y
BRIDGE=m
BRIDGE_EBT_802_3=m
BRIDGE_EBT_AMONG=m
BRIDGE_EBT_ARP=m
BRIDGE_EBT_ARPREPLY=m
BRIDGE_EBT_BROUTE=m
BRIDGE_EBT_DNAT=m
BRIDGE_EBT_IP=m
BRIDGE_EBT_LIMIT=m
BRIDGE_EBT_LOG=m
BRIDGE_EBT_MARK=m
BRIDGE_EBT_MARK_T=m
BRIDGE_EBT_PKTTYPE=m
BRIDGE_EBT_REDIRECT=m
BRIDGE_EBT_SNAT=m
BRIDGE_EBT_STP=m
BRIDGE_EBT_T_FILTER=m
BRIDGE_EBT_T_NAT=m
BRIDGE_EBT_ULOG=m
BRIDGE_EBT_VLAN=m
BRIDGE_NETFILTER=y
BRIDGE_NF_EBTABLES=m
BROADCOM_PHY=m
BSD_DISKLABEL=n
BSD_PROCESS_ACCT=y
BSD_PROCESS_ACCT_V3=n
BT=n
BUG=y
CAN=n
CARDBUS=y
CARDMAN_4000=n
CARDMAN_4040=n
CASSINI=n
CC_OPTIMIZE_FOR_SIZE=n
CDROM_PKTCDVD=m
CDROM_PKTCDVD_BUFFERS=8
CDROM_PKTCDVD_WCACHE=n
CFG80211=m
CGROUPS=n
CGROUP_SCHED=n
CHELSIO_T1=n
CHELSIO_T3=n
CHR_DEV_OSST=n
CHR_DEV_SCH=n
CHR_DEV_SG=m
CHR_DEV_ST=m
CICADA_PHY=m
CIFS=n
CISS_SCSI_TAPE=n
CLASSIC_RCU=y
CLOCKSOURCE_WATCHDOG=y
CLS_U32_MARK=y
CLS_U32_PERF=y
CODA_FS=n
COMPAT_BRK=n
COMPAT_VDSO=n
COMPUTONE=n
CONFIGFS_FS=n
CONNECTOR=n
CPA_DEBUG=n
CPU5_WDT=n
CPU_FREQ=y
CPU_FREQ_DEBUG=n
CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=n
CPU_FREQ_DEFAULT_GOV_ONDEMAND=n
CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
CPU_FREQ_DEFAULT_GOV_USERSPACE=n
CPU_FREQ_GOV_CONSERVATIVE=m
CPU_FREQ_GOV_ONDEMAND=m
CPU_FREQ_GOV_PERFORMANCE=y
CPU_FREQ_GOV_POWERSAVE=m
CPU_FREQ_GOV_USERSPACE=m
CPU_FREQ_STAT=y
CPU_FREQ_STAT_DETAILS=n
CPU_FREQ_TABLE=y
CPU_IDLE=y
CPU_IDLE_GOV_LADDER=y
CPU_IDLE_GOV_MENU=y
CRAMFS=m
CRASH_DUMP=n
CRC16=n
CRC32=y
CRC7=n
CRC_CCITT=m
CRC_ITU_T=m
CRYPTO=y
CRYPTO_AEAD=m
CRYPTO_AES=m
CRYPTO_AES_586=m
CRYPTO_ALGAPI=m
CRYPTO_ANUBIS=n
CRYPTO_ARC4=m
CRYPTO_AUTHENC=m
CRYPTO_BLKCIPHER=m
CRYPTO_BLOWFISH=m
CRYPTO_CAMELLIA=n
CRYPTO_CAST5=m
CRYPTO_CAST6=m
CRYPTO_CBC=m
CRYPTO_CCM=n
CRYPTO_CRC32C=m
CRYPTO_CRYPTD=m
CRYPTO_CTR=n
CRYPTO_DEFLATE=m
CRYPTO_DES=m
CRYPTO_DEV_GEODE=n
CRYPTO_DEV_HIFN_795X=m
CRYPTO_DEV_HIFN_795X_RNG=y
CRYPTO_DEV_PADLOCK=m
CRYPTO_DEV_PADLOCK_AES=m
CRYPTO_DEV_PADLOCK_SHA=m
CRYPTO_ECB=m
CRYPTO_FCRYPT=n
CRYPTO_GCM=n
CRYPTO_GF128MUL=m
CRYPTO_HASH=m
CRYPTO_HMAC=m
CRYPTO_HW=y
CRYPTO_KHAZAD=m
CRYPTO_LRW=m
CRYPTO_LZO=m
CRYPTO_MANAGER=m
CRYPTO_MD4=m
CRYPTO_MD5=m
CRYPTO_MICHAEL_MIC=m
CRYPTO_NULL=m
CRYPTO_PCBC=n
CRYPTO_SALSA20=n
CRYPTO_SALSA20_586=n
CRYPTO_SEED=n
CRYPTO_SEQIV=n
CRYPTO_SERPENT=m
CRYPTO_SHA1=m
CRYPTO_SHA256=m
CRYPTO_SHA512=m
CRYPTO_TEA=m
CRYPTO_TEST=m
CRYPTO_TGR192=m
CRYPTO_TWOFISH=m
CRYPTO_TWOFISH_586=m
CRYPTO_TWOFISH_COMMON=m
CRYPTO_WP512=m
CRYPTO_XCBC=m
CRYPTO_XTS=n
CS5535_GPIO=n
CS89x0=n
CYCLADES=n
DAB=n
DAVICOM_PHY=m
DCDBAS=n
DE2104X=n
DE4X5=m
DE600=n
DE620=n
DEBUG_BUGVERBOSE=y
DEBUG_DEVRES=n
DEBUG_DRIVER=n
DEBUG_FS=n
DEBUG_HIGHMEM=n
DEBUG_INFO=n
DEBUG_KERNEL=y
DEBUG_KOBJECT=n
DEBUG_LIST=n
DEBUG_LOCKING_API_SELFTESTS=n
DEBUG_LOCK_ALLOC=n
DEBUG_MUTEXES=n
DEBUG_NX_TEST=n
DEBUG_PAGEALLOC=n
DEBUG_RODATA=n
DEBUG_RT_MUTEXES=n
DEBUG_SG=n
DEBUG_SHIRQ=n
DEBUG_SPINLOCK=n
DEBUG_SPINLOCK_SLEEP=n
DEBUG_STACKOVERFLOW=n
DEBUG_STACK_USAGE=n
DEBUG_VM=n
DECNET=n
DEFAULT_AS=n
DEFAULT_CFQ=y
DEFAULT_DEADLINE=n
DEFAULT_IOSCHED="cfq"
DEFAULT_IO_DELAY_TYPE=0
DEFAULT_NOOP=n
DEFAULT_TCP_CONG="cubic"
DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
DELL_RBU=n
DEPCA=n
DETECT_SOFTLOCKUP=y
DEVPORT=y
DIGIEPCA=n
DISCONTIGMEM_MANUAL=n
DISPLAY_SUPPORT=n
DL2K=m
DLM=n
DM9102=m
DMADEVICES=n
DMI=y
DMIID=y
DM_CRYPT=m
DM_DEBUG=n
DM_DELAY=n
DM_MIRROR=m
DM_MULTIPATH=n
DM_SNAPSHOT=m
DM_UEVENT=y
DM_ZERO=n
DNOTIFY=y
DOUBLEFAULT=y
DRM=m
DRM_I810=m
DRM_I830=m
DRM_I915=m
DRM_MGA=m
DRM_R128=m
DRM_RADEON=m
DRM_SAVAGE=m
DRM_SIS=n
DRM_TDFX=n
DRM_VIA=m
DS1682=n
DTLK=n
DUMMY=m
DUMMY_CONSOLE=y
DVB_CORE=n
DVB_TDA18271=m
DVB_TDA827X=m
E1000=m
E1000E=m
E1000E_ENABLED=y
E1000_DISABLE_PACKET_SPLIT=n
E1000_NAPI=n
E100=m
E2100=n
EARLY_PRINTK=n
ECONET=n
EDAC=y
EDAC_AMD76X=m
EDAC_DEBUG=n
EDAC_E752X=m
EDAC_E7XXX=m
EDAC_I3000=m
EDAC_I5000=m
EDAC_I82860=m
EDAC_I82875P=m
EDAC_I82975X=m
EDAC_MM_EDAC=m
EDAC_R82600=m
EDD=n
EEPRO100=n
EEPROM_93CX6=m
EEXPRESS=n
EEXPRESS_PRO=n
EFI=n
EFI_PARTITION=y
EFS_FS=n
EISA=n
EL16=n
EL1=n
EL2=n
EL3=m
ELF_CORE=y
ELPLUS=n
EMBEDDED=y
ENABLE_MUST_CHECK=y
ENABLE_WARN_DEPRECATED=y
ENCLOSURE_SERVICES=n
EPIC100=m
EPOLL=y
EQUALIZER=m
ESPSERIAL=n
ETH16I=n
EUROTECH_WDT=n
EVENTFD=y
EWRK3=n
EXPERIMENTAL=y
EXPORTFS=m
EXT2_FS=m
EXT2_FS_POSIX_ACL=y
EXT2_FS_SECURITY=n
EXT2_FS_XATTR=y
EXT2_FS_XIP=n
EXT3_FS=m
EXT3_FS_POSIX_ACL=y
EXT3_FS_SECURITY=n
EXT3_FS_XATTR=y
EXT4DEV_FS=n
FAIR_GROUP_SCHED=y
FAST_CMPXCHG_LOCAL=y
FAT_DEFAULT_CODEPAGE=437
FAT_DEFAULT_IOCHARSET="iso8859-1"
FAT_FS=m
FAULT_INJECTION=n
FB=y
FB_3DFX=n
FB_ARC=n
FB_ARK=n
FB_ASILIANT=n
FB_ATY128=m
FB_ATY128_BACKLIGHT=n
FB_ATY=m
FB_ATY_BACKLIGHT=n
FB_ATY_CT=y
FB_ATY_GENERIC_LCD=y
FB_ATY_GX=y
FB_BACKLIGHT=n
FB_CARILLO_RANCH=n
FB_CFB_COPYAREA=m
FB_CFB_FILLRECT=m
FB_CFB_IMAGEBLIT=m
FB_CFB_REV_PIXELS_IN_BYTE=n
FB_CIRRUS=n
FB_CYBER2000=n
FB_CYBLA=m
FB_DDC=m
FB_DEFERRED_IO=y
FB_EFI=n
FB_GEODE=n
FB_HECUBA=n
FB_HGA=n
FB_I810=m
FB_I810_GTF=n
FB_IMSTT=n
FB_INTEL=m
FB_INTEL_DEBUG=n
FB_INTEL_I2C=y
FB_KYRO=n
FB_LE80578=m
FB_MACMODES=n
FB_MATROX=m
FB_MATROX_G=y
FB_MATROX_I2C=m
FB_MATROX_MAVEN=n
FB_MATROX_MILLENIUM=y
FB_MATROX_MULTIHEAD=n
FB_MATROX_MYSTIQUE=y
FB_MODE_HELPERS=y
FB_NEOMAGIC=n
FB_NVIDIA=m
FB_NVIDIA_BACKLIGHT=n
FB_NVIDIA_DEBUG=n
FB_NVIDIA_I2C=y
FB_PM2=n
FB_PM3=n
FB_RADEON=m
FB_RADEON_BACKLIGHT=n
FB_RADEON_DEBUG=n
FB_RADEON_I2C=y
FB_RIVA=m
FB_RIVA_BACKLIGHT=n
FB_RIVA_DEBUG=n
FB_RIVA_I2C=y
FB_S1D13XXX=n
FB_S3=m
FB_SAVAGE=m
FB_SAVAGE_ACCEL=y
FB_SAVAGE_I2C=y
FB_SIS=m
FB_SIS_300=y
FB_SIS_315=y
FB_SVGALIB=m
FB_SYS_COPYAREA=n
FB_SYS_FILLRECT=n
FB_SYS_FOPS=n
FB_SYS_IMAGEBLIT=n
FB_TILEBLITTING=y
FB_TRIDENT=m
FB_TRIDENT_ACCEL=y
FB_VESA=n
FB_VGA16=n
FB_VIRTUAL=n
FB_VOODOO1=n
FB_VT8623=m
FDDI=n
FEALNX=n
FIB_RULES=y
FIREWIRE=n
FIRMWARE_EDID=y
FIX_EARLYCON_MEM=y
FLATMEM=y
FLATMEM_MANUAL=y
FLAT_NODE_MEM_MAP=y
FONTS=n
FONT_8x16=y
FONT_8x8=y
FORCEDETH=m
FORCEDETH_NAPI=n
FRAMEBUFFER_CONSOLE=m
FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=n
FRAMEBUFFER_CONSOLE_ROTATION=n
FRAME_POINTER=n
FS_MBCACHE=m
FS_POSIX_ACL=y
FUJITSU_LAPTOP=m
FUSE_FS=m
FUSION=y
FUSION_CTL=m
FUSION_FC=n
FUSION_LOGGING=y
FUSION_MAX_SGE=128
FUSION_SAS=m
FUSION_SPI=m
FUTEX=y
FW_LOADER=m
GACT_PROB=y
GAMEPORT=n
GENERIC_BUG=y
GENERIC_CALIBRATE_DELAY=y
GENERIC_CLOCKEVENTS=y
GENERIC_CLOCKEVENTS_BROADCAST=y
GENERIC_CLOCKEVENTS_BUILD=y
GENERIC_CMOS_UPDATE=y
GENERIC_CPU=n
GENERIC_GPIO=n
GENERIC_HARDIRQS=y
GENERIC_HWEIGHT=y
GENERIC_IOMAP=y
GENERIC_IRQ_PROBE=y
GENERIC_ISA_DMA=y
GENERIC_LOCKBREAK=n
GENERIC_PENDING_IRQ=y
GENERIC_TIME=y
GENERIC_TIME_VSYSCALL=n
GFS2_FS=n
GROUP_SCHED=y
HAMACHI=m
HAMRADIO=n
HANGCHECK_TIMER=n
HAPPYMEAL=n
HAS_DMA=y
HAS_IOMEM=y
HAS_IOPORT=y
HAVE_IDE=y
HAVE_KPROBES=y
HAVE_KRETPROBES=y
HAVE_KVM=y
HAVE_LATENCYTOP_SUPPORT=y
HAVE_OPROFILE=y
HAVE_SETUP_PER_CPU_AREA=n
HEADERS_CHECK=n
HERMES=m
HFSPLUS_FS=n
HFS_FS=n
HIBERNATION=y
HID=m
HIDRAW=y
HID_DEBUG=n
HID_FF=n
HID_SUPPORT=y
HIGHMEM4G=y
HIGHMEM64G=n
HIGHMEM=y
HIGHPTE=n
HIGH_RES_TIMERS=y
HIPPI=n
HOSTAP=n
HOTPLUG=y
HOTPLUG_CPU=y
HOTPLUG_PCI=n
HP100=n
HPET=n
HPET_EMULATE_RTC=y
HPET_TIMER=y
HPFS_FS=n
HPLAN=n
HPLAN_PLUS=n
HP_WATCHDOG=m
HT_IRQ=y
HUGETLBFS=n
HUGETLB_PAGE=n
HWMON=m
HWMON_DEBUG_CHIP=n
HWMON_VID=m
HW_CONSOLE=y
HW_RANDOM=m
HW_RANDOM_AMD=m
HW_RANDOM_GEODE=n
HW_RANDOM_INTEL=m
HW_RANDOM_VIA=m
HZ=100
HZ_1000=n
HZ_100=y
HZ_250=n
HZ_300=n
I2C=m
I2C_ALGOBIT=m
I2C_ALGOPCA=m
I2C_ALGOPCF=m
I2C_ALI1535=m
I2C_ALI1563=m
I2C_ALI15X3=m
I2C_AMD756=m
I2C_AMD756_S4882=m
I2C_AMD8111=m
I2C_BOARDINFO=y
I2C_CHARDEV=m
I2C_DEBUG_ALGO=n
I2C_DEBUG_BUS=n
I2C_DEBUG_CHIP=n
I2C_DEBUG_CORE=n
I2C_I801=m
I2C_I810=m
I2C_NFORCE2=m
I2C_OCORES=n
I2C_PARPORT=m
I2C_PARPORT_LIGHT=n
I2C_PCA_ISA=n
I2C_PIIX4=m
I2C_PROSAVAGE=m
I2C_SAVAGE4=m
I2C_SIMTEC=m
I2C_SIS5595=m
I2C_SIS630=m
I2C_SIS96X=m
I2C_STUB=n
I2C_TAOS_EVM=n
I2C_TINY_USB=n
I2C_VIA=m
I2C_VIAPRO=m
I2C_VOODOO3=m
I2O=n
I6300ESB_WDT=m
I82092=m
I82365=m
I8K=m
IB700_WDT=n
IBMASR=m
IBM_ASM=n
IBM_NEW_EMAC_EMAC4=n
IBM_NEW_EMAC_RGMII=n
IBM_NEW_EMAC_TAH=n
IBM_NEW_EMAC_ZMII=n
ICPLUS_PHY=m
IDE=m
IDEDISK_MULTI_MODE=y
IDE_ARCH_OBSOLETE_INIT=y
IDE_GENERIC=m
IDE_MAX_HWIFS=4
IDE_PROC_FS=n
IDE_TASK_IOCTL=y
IEEE1394=n
IEEE80211=n
IFB=m
IGB=m
IKCONFIG=n
INET6_AH=m
INET6_ESP=m
INET6_IPCOMP=m
INET6_TUNNEL=m
INET6_XFRM_MODE_BEET=m
INET6_XFRM_MODE_ROUTEOPTIMIZATION=n
INET6_XFRM_MODE_TRANSPORT=m
INET6_XFRM_MODE_TUNNEL=m
INET6_XFRM_TUNNEL=m
INET=y
INET_AH=m
INET_DIAG=m
INET_ESP=m
INET_IPCOMP=m
INET_LRO=n
INET_TCP_DIAG=m
INET_TUNNEL=m
INET_XFRM_MODE_BEET=m
INET_XFRM_MODE_TRANSPORT=m
INET_XFRM_MODE_TUNNEL=m
INET_XFRM_TUNNEL=m
INFINIBAND=n
INITRAMFS_SOURCE=""
INIT_ENV_ARG_LIMIT=32
INOTIFY=y
INOTIFY_USER=y
INPUT=y
INPUT_ATI_REMOTE2=n
INPUT_ATI_REMOTE=n
INPUT_ATLAS_BTNS=n
INPUT_EVBUG=n
INPUT_EVDEV=m
INPUT_FF_MEMLESS=n
INPUT_JOYDEV=n
INPUT_JOYSTICK=n
INPUT_KEYBOARD=y
INPUT_KEYSPAN_REMOTE=n
INPUT_MISC=y
INPUT_MOUSE=y
INPUT_MOUSEDEV=m
INPUT_MOUSEDEV_PSAUX=y
INPUT_MOUSEDEV_SCREEN_X=1024
INPUT_MOUSEDEV_SCREEN_Y=768
INPUT_PCSPKR=y
INPUT_POLLDEV=m
INPUT_POWERMATE=n
INPUT_TABLET=n
INPUT_TOUCHSCREEN=n
INPUT_UINPUT=n
INPUT_WISTRON_BTNS=n
INPUT_YEALINK=n
INTEL_MENLOW=m
IOMMU_HELPER=n
IOSCHED_AS=n
IOSCHED_CFQ=y
IOSCHED_DEADLINE=y
IOSCHED_NOOP=y
IO_DELAY_0X80=y
IO_DELAY_0XED=n
IO_DELAY_NONE=n
IO_DELAY_TYPE_0X80=0
IO_DELAY_TYPE_0XED=1
IO_DELAY_TYPE_NONE=3
IO_DELAY_TYPE_UDELAY=2
IO_DELAY_UDELAY=n
IP1000=m
IP6_NF_FILTER=m
IP6_NF_IPTABLES=m
IP6_NF_MANGLE=m
IP6_NF_MATCH_AH=m
IP6_NF_MATCH_EUI64=m
IP6_NF_MATCH_FRAG=m
IP6_NF_MATCH_HL=m
IP6_NF_MATCH_IPV6HEADER=m
IP6_NF_MATCH_MH=m
IP6_NF_MATCH_OPTS=m
IP6_NF_MATCH_RT=m
IP6_NF_QUEUE=m
IP6_NF_RAW=m
IP6_NF_TARGET_HL=m
IP6_NF_TARGET_LOG=m
IP6_NF_TARGET_REJECT=m
IPMI_DEVICE_INTERFACE=m
IPMI_HANDLER=m
IPMI_PANIC_EVENT=n
IPMI_POWEROFF=m
IPMI_SI=m
IPMI_WATCHDOG=m
IPV6=m
IPV6_MIP6=n
IPV6_MULTIPLE_TABLES=n
IPV6_OPTIMISTIC_DAD=n
IPV6_PRIVACY=y
IPV6_ROUTER_PREF=n
IPV6_SIT=m
IPV6_TUNNEL=m
IPW2100=n
IPW2200=n
IPWIRELESS=n
IPX=n
IP_ADVANCED_ROUTER=y
IP_DCCP=n
IP_FIB_HASH=y
IP_FIB_TRIE=n
IP_MROUTE=y
IP_MULTICAST=y
IP_MULTIPLE_TABLES=y
IP_NF_ARPFILTER=m
IP_NF_ARPTABLES=m
IP_NF_ARP_MANGLE=m
IP_NF_FILTER=m
IP_NF_IPTABLES=m
IP_NF_MANGLE=m
IP_NF_MATCH_ADDRTYPE=m
IP_NF_MATCH_AH=m
IP_NF_MATCH_ECN=m
IP_NF_MATCH_RECENT=m
IP_NF_MATCH_SET=m
IP_NF_MATCH_TTL=m
IP_NF_QUEUE=m
IP_NF_RAW=m
IP_NF_SET=m
IP_NF_SET_HASHSIZE=1024
IP_NF_SET_IPHASH=m
IP_NF_SET_IPMAP=m
IP_NF_SET_IPPORTHASH=m
IP_NF_SET_IPTREE=m
IP_NF_SET_IPTREEMAP=m
IP_NF_SET_MACIPMAP=m
IP_NF_SET_MAX=256
IP_NF_SET_NETHASH=m
IP_NF_SET_PORTMAP=m
IP_NF_TARGET_CLUSTERIP=m
IP_NF_TARGET_ECN=m
IP_NF_TARGET_LOG=m
IP_NF_TARGET_MASQUERADE=m
IP_NF_TARGET_NETMAP=m
IP_NF_TARGET_REDIRECT=m
IP_NF_TARGET_REJECT=m
IP_NF_TARGET_SET=m
IP_NF_TARGET_TTL=m
IP_NF_TARGET_ULOG=m
IP_PIMSM_V1=n
IP_PIMSM_V2=n
IP_PNP=n
IP_ROUTE_MULTIPATH=y
IP_ROUTE_VERBOSE=y
IP_SCTP=n
IP_VS=n
IRDA=n
IRQBALANCE=y
ISA=y
ISAPNP=y
ISA_DMA_API=y
ISCSI_TCP=m
ISDN=n
ISI=n
ISO9660_FS=m
IT8712F_WDT=m
ITCO_VENDOR_SUPPORT=y
ITCO_WDT=m
IWL3945=m
IWL3945_DEBUG=n
IWL3945_QOS=y
IWL3945_SPECTRUM_MEASUREMENT=y
IWL4965=m
IWL4965_DEBUG=n
IWL4965_HT=y
IWL4965_QOS=y
IWL4965_SENSITIVITY=y
IWL4965_SPECTRUM_MEASUREMENT=y
IXGB=n
IXGBE=n
JBD=m
JFS_DEBUG=n
JFS_FS=m
JFS_POSIX_ACL=y
JFS_SECURITY=y
JFS_STATISTICS=n
JOLIET=y
K8_NB=y
KALLSYMS=y
KALLSYMS_ALL=n
KALLSYMS_EXTRA_PASS=n
KARMA_PARTITION=n
KEXEC=n
KEYBOARD_ATKBD=y
KEYBOARD_LKKBD=n
KEYBOARD_NEWTON=n
KEYBOARD_STOWAWAY=n
KEYBOARD_SUNKBD=n
KEYBOARD_XTKBD=n
KEYS=n
KMOD=y
KPROBES=n
KTIME_SCALAR=y
KVM=m
KVM_AMD=m
KVM_INTEL=m
LANCE=m
LAPB=n
LATENCYTOP=n
LBD=n
LDM_PARTITION=n
LEGACY_PTYS=n
LGUEST=n
LIBCRC32C=m
LIBERTAS=n
LLC2=n
LLC=m
LOCALVERSION=""
LOCALVERSION_AUTO=n
LOCKD=m
LOCKDEP_SUPPORT=y
LOCKD_V4=y
LOCK_KERNEL=y
LOCK_STAT=n
LOGO=n
LOG_BUF_SHIFT=16
LP486E=n
LP_CONSOLE=n
LSF=n
LXT_PHY=m
LZO_COMPRESS=m
LZO_DECOMPRESS=m
M386=n
M486=n
M586=n
M586MMX=n
M586TSC=n
M686=n
MAC80211=m
MAC80211_DEBUG=n
MAC80211_DEBUG_PACKET_ALIGNMENT=n
MAC80211_RC_DEFAULT="pid"
MAC80211_RC_DEFAULT_NONE=n
MAC80211_RC_DEFAULT_PID=y
MAC80211_RC_DEFAULT_SIMPLE=n
MAC80211_RC_PID=y
MAC80211_RC_SIMPLE=n
MACHZ_WDT=n
MACINTOSH_DRIVERS=n
MACVLAN=m
MAC_PARTITION=n
MAGIC_SYSRQ=y
MARKERS=n
MARVELL_PHY=m
MATH_EMULATION=n
MCA=n
MCORE2=n
MCRUSOE=n
MCYRIXIII=n
MD=y
MDA_CONSOLE=n
MDIO_BITBANG=n
MD_FAULTY=m
MD_LINEAR=m
MD_MULTIPATH=n
MD_RAID0=m
MD_RAID10=m
MD_RAID1=m
MD_RAID456=m
MD_RAID5_RESHAPE=n
MEFFICEON=n
MEGARAID_LEGACY=n
MEGARAID_MAILBOX=m
MEGARAID_MM=m
MEGARAID_NEWGEN=y
MEGARAID_SAS=m
MEMSTICK=n
MFD_SM501=n
MGEODEGX1=n
MGEODE_LX=n
MICROCODE=m
MICROCODE_OLD_INTERFACE=y
MII=m
MINIX_FS=n
MINIX_SUBPARTITION=n
MISC_DEVICES=y
MIXCOMWD=n
MK6=n
MK7=n
MK8=n
MLX4_CORE=n
MMC=n
MMU=y
MODULES=y
MODULE_FORCE_UNLOAD=n
MODULE_SRCVERSION_ALL=n
MODULE_UNLOAD=y
MODVERSIONS=n
MOUSE_APPLETOUCH=n
MOUSE_INPORT=n
MOUSE_LOGIBM=n
MOUSE_PC110PAD=n
MOUSE_PS2=m
MOUSE_PS2_ALPS=y
MOUSE_PS2_LIFEBOOK=y
MOUSE_PS2_LOGIPS2PP=y
MOUSE_PS2_SYNAPTICS=y
MOUSE_PS2_TOUCHKIT=n
MOUSE_PS2_TRACKPOINT=y
MOUSE_SERIAL=m
MOUSE_VSXXXAA=n
MOXA_INTELLIO=n
MOXA_SMARTIO=n
MPENTIUM4=n
MPENTIUMII=n
MPENTIUMIII=y
MPENTIUMM=n
MPSC=n
MSDOS_FS=m
MSDOS_PARTITION=y
MSI_LAPTOP=m
MTD=n
MTRR=y
MVIAC3_2=n
MVIAC7=n
MWAVE=m
MWINCHIP2=n
MWINCHIP3D=n
MWINCHIPC6=n
MYRI10GE=n
NAMESPACES=n
NATSEMI=m
NCP_FS=n
NE2000=m
NE2K_PCI=m
NET=y
NETCONSOLE=m
NETCONSOLE_DYNAMIC=n
NETDEVICES=y
NETDEVICES_MULTIQUEUE=n
NETDEV_10000=y
NETDEV_1000=y
NETFILTER=y
NETFILTER_ADVANCED=y
NETFILTER_DEBUG=n
NETFILTER_NETLINK=m
NETFILTER_NETLINK_LOG=m
NETFILTER_NETLINK_QUEUE=m
NETFILTER_XTABLES=m
NETFILTER_XT_MATCH_COMMENT=m
NETFILTER_XT_MATCH_CONNBYTES=m
NETFILTER_XT_MATCH_CONNLIMIT=m
NETFILTER_XT_MATCH_CONNMARK=m
NETFILTER_XT_MATCH_CONNTRACK=m
NETFILTER_XT_MATCH_DCCP=n
NETFILTER_XT_MATCH_DSCP=n
NETFILTER_XT_MATCH_ESP=m
NETFILTER_XT_MATCH_HASHLIMIT=m
NETFILTER_XT_MATCH_HELPER=m
NETFILTER_XT_MATCH_IPRANGE=m
NETFILTER_XT_MATCH_LENGTH=m
NETFILTER_XT_MATCH_LIMIT=m
NETFILTER_XT_MATCH_MAC=m
NETFILTER_XT_MATCH_MARK=m
NETFILTER_XT_MATCH_MULTIPORT=m
NETFILTER_XT_MATCH_OWNER=m
NETFILTER_XT_MATCH_PHYSDEV=m
NETFILTER_XT_MATCH_PKTTYPE=m
NETFILTER_XT_MATCH_POLICY=m
NETFILTER_XT_MATCH_QUOTA=m
NETFILTER_XT_MATCH_RATEEST=m
NETFILTER_XT_MATCH_REALM=m
NETFILTER_XT_MATCH_SCTP=n
NETFILTER_XT_MATCH_STATE=m
NETFILTER_XT_MATCH_STATISTIC=m
NETFILTER_XT_MATCH_STRING=m
NETFILTER_XT_MATCH_TCPMSS=m
NETFILTER_XT_MATCH_TIME=m
NETFILTER_XT_MATCH_U32=m
NETFILTER_XT_TARGET_CLASSIFY=m
NETFILTER_XT_TARGET_CONNMARK=m
NETFILTER_XT_TARGET_DSCP=n
NETFILTER_XT_TARGET_MARK=m
NETFILTER_XT_TARGET_NFLOG=m
NETFILTER_XT_TARGET_NFQUEUE=m
NETFILTER_XT_TARGET_NOTRACK=m
NETFILTER_XT_TARGET_RATEEST=m
NETFILTER_XT_TARGET_TCPMSS=m
NETFILTER_XT_TARGET_TCPOPTSTRIP=m
NETFILTER_XT_TARGET_TRACE=m
NETPOLL=y
NETPOLL_TRAP=n
NETWORK_FILESYSTEMS=y
NETWORK_SECMARK=n
NETXEN_NIC=n
NET_9P=n
NET_ACT_GACT=m
NET_ACT_IPT=m
NET_ACT_MIRRED=m
NET_ACT_NAT=m
NET_ACT_PEDIT=m
NET_ACT_POLICE=m
NET_ACT_SIMP=n
NET_CLS=y
NET_CLS_ACT=y
NET_CLS_BASIC=n
NET_CLS_FLOW=m
NET_CLS_FW=m
NET_CLS_IND=y
NET_CLS_ROUTE4=m
NET_CLS_ROUTE=y
NET_CLS_RSVP6=m
NET_CLS_RSVP=m
NET_CLS_TCINDEX=m
NET_CLS_U32=m
NET_EMATCH=n
NET_ETHERNET=y
NET_FC=n
NET_IPGRE=m
NET_IPGRE_BROADCAST=n
NET_IPIP=m
NET_ISA=y
NET_KEY=m
NET_KEY_MIGRATE=n
NET_PCI=y
NET_PCMCIA=n
NET_PKTGEN=n
NET_POCKET=y
NET_POLL_CONTROLLER=y
NET_SB1000=n
NET_SCHED=y
NET_SCH_ATM=m
NET_SCH_CBQ=m
NET_SCH_DSMARK=m
NET_SCH_FIFO=y
NET_SCH_GRED=m
NET_SCH_HFSC=m
NET_SCH_HTB=m
NET_SCH_INGRESS=m
NET_SCH_NETEM=m
NET_SCH_PRIO=m
NET_SCH_RED=m
NET_SCH_RR=m
NET_SCH_SFQ=m
NET_SCH_TBF=m
NET_SCH_TEQL=m
NET_TULIP=y
NET_VENDOR_3COM=y
NET_VENDOR_RACAL=n
NET_VENDOR_SMC=y
NEW_LEDS=n
NFSD=m
NFSD_TCP=y
NFSD_V2_ACL=y
NFSD_V3=y
NFSD_V3_ACL=y
NFSD_V4=n
NFS_ACL_SUPPORT=m
NFS_COMMON=y
NFS_DIRECTIO=y
NFS_FS=m
NFS_V3=y
NFS_V3_ACL=y
NFS_V4=n
NF_CONNTRACK=m
NF_CONNTRACK_AMANDA=n
NF_CONNTRACK_EVENTS=n
NF_CONNTRACK_FTP=m
NF_CONNTRACK_H323=m
NF_CONNTRACK_IPV4=m
NF_CONNTRACK_IPV6=m
NF_CONNTRACK_IRC=m
NF_CONNTRACK_MARK=y
NF_CONNTRACK_NETBIOS_NS=m
NF_CONNTRACK_PPTP=m
NF_CONNTRACK_PROC_COMPAT=y
NF_CONNTRACK_SANE=n
NF_CONNTRACK_SIP=m
NF_CONNTRACK_TFTP=m
NF_CT_ACCT=y
NF_CT_NETLINK=m
NF_CT_PROTO_GRE=m
NF_CT_PROTO_SCTP=n
NF_CT_PROTO_UDPLITE=n
NF_NAT=m
NF_NAT_AMANDA=n
NF_NAT_FTP=m
NF_NAT_H323=m
NF_NAT_IRC=m
NF_NAT_NEEDED=y
NF_NAT_PPTP=m
NF_NAT_PROTO_GRE=m
NF_NAT_SIP=m
NF_NAT_SNMP_BASIC=n
NF_NAT_TFTP=m
NIU=n
NL80211=y
NLS=m
NLS_ASCII=n
NLS_CODEPAGE_1250=m
NLS_CODEPAGE_1251=m
NLS_CODEPAGE_437=m
NLS_CODEPAGE_737=n
NLS_CODEPAGE_775=n
NLS_CODEPAGE_850=m
NLS_CODEPAGE_852=m
NLS_CODEPAGE_855=m
NLS_CODEPAGE_857=n
NLS_CODEPAGE_860=n
NLS_CODEPAGE_861=n
NLS_CODEPAGE_862=n
NLS_CODEPAGE_863=n
NLS_CODEPAGE_864=n
NLS_CODEPAGE_865=n
NLS_CODEPAGE_866=m
NLS_CODEPAGE_869=n
NLS_CODEPAGE_874=n
NLS_CODEPAGE_932=n
NLS_CODEPAGE_936=n
NLS_CODEPAGE_949=n
NLS_CODEPAGE_950=n
NLS_DEFAULT="utf8"
NLS_ISO8859_13=m
NLS_ISO8859_14=n
NLS_ISO8859_15=m
NLS_ISO8859_1=m
NLS_ISO8859_2=m
NLS_ISO8859_3=n
NLS_ISO8859_4=n
NLS_ISO8859_5=m
NLS_ISO8859_6=n
NLS_ISO8859_7=n
NLS_ISO8859_8=n
NLS_ISO8859_9=n
NLS_KOI8_R=m
NLS_KOI8_U=m
NLS_UTF8=m
NOHIGHMEM=n
NORTEL_HERMES=m
NOZOMI=n
NO_HZ=y
NR_CPUS=8
NS83820=m
NSC_GPIO=n
NTFS_DEBUG=n
NTFS_FS=m
NTFS_RW=y
NVRAM=m
N_HDLC=n
OCFS2_FS=n
OSF_PARTITION=n
P54_COMMON=n
PACKET=y
PACKET_MMAP=n
PAGE_OFFSET=0xC0000000
PARAVIRT_GUEST=n
PARIDE=n
PARPORT=m
PARPORT_1284=y
PARPORT_AX88796=n
PARPORT_GSC=n
PARPORT_PC=m
PARPORT_PC_FIFO=y
PARPORT_PC_PCMCIA=n
PARPORT_PC_SUPERIO=y
PARPORT_SERIAL=m
PARTITION_ADVANCED=y
PATA_ACPI=m
PATA_ALI=n
PATA_AMD=m
PATA_ARTOP=n
PATA_ATIIXP=m
PATA_CMD640_PCI=n
PATA_CMD64X=m
PATA_CS5520=m
PATA_CS5530=m
PATA_CS5535=m
PATA_CS5536=m
PATA_CYPRESS=n
PATA_EFAR=n
PATA_HPT366=n
PATA_HPT37X=n
PATA_HPT3X2N=n
PATA_HPT3X3=n
PATA_ISAPNP=m
PATA_IT8213=n
PATA_IT821X=n
PATA_JMICRON=n
PATA_LEGACY=m
PATA_MARVELL=n
PATA_MPIIX=m
PATA_NETCELL=n
PATA_NINJA32=n
PATA_NS87410=n
PATA_NS87415=n
PATA_OLDPIIX=m
PATA_OPTI=n
PATA_OPTIDMA=n
PATA_PCMCIA=n
PATA_PDC2027X=n
PATA_PDC_OLD=n
PATA_PLATFORM=n
PATA_QDI=n
PATA_RADISYS=n
PATA_RZ1000=n
PATA_SC1200=n
PATA_SERVERWORKS=m
PATA_SIL680=n
PATA_SIS=m
PATA_TRIFLEX=n
PATA_VIA=m
PATA_WINBOND=n
PATA_WINBOND_VLB=n
PC8736x_GPIO=n
PC87413_WDT=n
PCCARD=m
PCCARD_NONSTATIC=m
PCF8575=m
PCI=y
PCIEAER=y
PCIEPORTBUS=y
PCIPCWATCHDOG=n
PCI_BIOS=y
PCI_DEBUG=n
PCI_DIRECT=y
PCI_DOMAINS=y
PCI_GOANY=y
PCI_GOBIOS=n
PCI_GODIRECT=n
PCI_GOMMCONFIG=n
PCI_HERMES=m
PCI_LEGACY=y
PCI_MMCONFIG=y
PCI_MSI=y
PCMCIA=m
PCMCIA_DEBUG=n
PCMCIA_HERMES=m
PCMCIA_IOCTL=y
PCMCIA_LOAD_CIS=y
PCMCIA_PROBE=y
PCMCIA_RAYCS=n
PCMCIA_SPECTRUM=n
PCMCIA_WL3501=n
PCMCIA_XIRCOM=n
PCNET32=m
PCNET32_NAPI=n
PCWATCHDOG=n
PD6729=n
PDA_POWER=n
PDC_ADMA=n
PHANTOM=n
PHONE=n
PHYLIB=m
PHYSICAL_ALIGN=0x100000
PHYSICAL_START=0x100000
PLIP=n
PLIST=y
PLX_HERMES=m
PM=y
PM_DEBUG=n
PM_LEGACY=n
PM_SLEEP=y
PM_SLEEP_SMP=y
PM_STD_PARTITION=""
PNP=y
PNPACPI=y
PNPBIOS=y
PNPBIOS_PROC_FS=y
PNP_DEBUG=n
POSIX_MQUEUE=n
POWER_SUPPLY=y
POWER_SUPPLY_DEBUG=n
PPDEV=n
PPP=m
PPPOATM=m
PPPOE=m
PPPOL2TP=m
PPP_ASYNC=m
PPP_BSDCOMP=m
PPP_DEFLATE=m
PPP_FILTER=y
PPP_MPPE=m
PPP_MULTILINK=n
PPP_SYNC_TTY=m
PREEMPT=n
PREEMPT_NONE=y
PREEMPT_NOTIFIERS=y
PREEMPT_VOLUNTARY=n
PREVENT_FIRMWARE_BUILD=y
PRINTER=m
PRINTK=y
PRINTK_TIME=n
PRINT_QUOTA_WARNING=y
PRISM54=n
PROC_FS=y
PROC_KCORE=y
PROC_PAGE_MONITOR=y
PROC_SYSCTL=y
PROFILING=n
PROVE_LOCKING=n
PROVIDE_OHCI1394_DMA_INIT=n
QFMT_V1=m
QFMT_V2=m
QLA3XXX=n
QNX4FS_FS=n
QSEMI_PHY=m
QUOTA=y
QUOTACTL=y
QUOTA_NETLINK_INTERFACE=y
R3964=n
R6040=m
R8169=m
R8169_NAPI=n
R8169_VLAN=y
RADIO_ADAPTERS=n
RAID_ATTRS=n
RAW_DRIVER=n
RCU_TORTURE_TEST=n
REALTEK_PHY=m
REISERFS_FS=n
RELAY=n
RELOCATABLE=n
RESOURCES_64BIT=n
RFKILL=m
RFKILL_INPUT=m
RIO=n
RISCOM8=n
ROCKETPORT=m
ROMFS_FS=m
RPCSEC_GSS_KRB5=n
RPCSEC_GSS_SPKM3=n
RT2400PCI=m
RT2400PCI_RFKILL=y
RT2500PCI=m
RT2500PCI_RFKILL=y
RT2500USB=n
RT2X00=m
RT2X00_DEBUG=n
RT2X00_LIB=m
RT2X00_LIB_FIRMWARE=y
RT2X00_LIB_PCI=m
RT2X00_LIB_RFKILL=y
RT61PCI=m
RT61PCI_RFKILL=y
RT73USB=n
RTC=y
RTC_CLASS=n
RTL8180=m
RTL8187=n
RT_GROUP_SCHED=n
RT_MUTEXES=y
RT_MUTEX_TESTER=n
RWSEM_GENERIC_SPINLOCK=n
RWSEM_XCHGADD_ALGORITHM=y
S2IO=n
SAMPLES=n
SATA_AHCI=m
SATA_INIC162X=n
SATA_MV=n
SATA_NV=m
SATA_PROMISE=n
SATA_QSTOR=n
SATA_SIL24=m
SATA_SIL=m
SATA_SIS=m
SATA_SVW=n
SATA_SX4=n
SATA_ULI=n
SATA_VIA=m
SATA_VITESSE=n
SBC7240_WDT=n
SBC8360_WDT=n
SBC_EPX_C3_WATCHDOG=n
SC1200_WDT=n
SC520_WDT=n
SC92031=n
SCHEDSTATS=n
SCHED_DEBUG=y
SCHED_HRTICK=y
SCHED_MC=y
SCHED_NO_NO_OMIT_FRAME_POINTER=y
SCHED_SMT=y
SCSI=m
SCSI_3W_9XXX=n
SCSI_7000FASST=n
SCSI_AACRAID=m
SCSI_ACARD=n
SCSI_ADVANSYS=n
SCSI_AHA152X=n
SCSI_AHA1542=n
SCSI_AIC79XX=m
SCSI_AIC7XXX=m
SCSI_AIC7XXX_OLD=n
SCSI_AIC94XX=n
SCSI_ARCMSR=n
SCSI_BUSLOGIC=m
SCSI_CONSTANTS=y
SCSI_DC390T=n
SCSI_DC395x=n
SCSI_DEBUG=n
SCSI_DMA=y
SCSI_DMX3191D=n
SCSI_DPT_I2O=m
SCSI_DTC3280=n
SCSI_EATA=n
SCSI_FC_ATTRS=n
SCSI_FUTURE_DOMAIN=n
SCSI_GDTH=n
SCSI_GENERIC_NCR5380=n
SCSI_GENERIC_NCR5380_MMIO=n
SCSI_HPTIOP=n
SCSI_IMM=n
SCSI_IN2000=n
SCSI_INIA100=n
SCSI_INITIO=n
SCSI_IPR=n
SCSI_IPS=m
SCSI_ISCSI_ATTRS=m
SCSI_LOGGING=y
SCSI_LOWLEVEL=y
SCSI_LOWLEVEL_PCMCIA=n
SCSI_LPFC=n
SCSI_MULTI_LUN=y
SCSI_MVSAS=m
SCSI_NCR53C406A=n
SCSI_NETLINK=n
SCSI_NSP32=n
SCSI_OMIT_FLASHPOINT=y
SCSI_PAS16=n
SCSI_PPA=n
SCSI_PROC_FS=y
SCSI_QLA_FC=n
SCSI_QLA_ISCSI=m
SCSI_QLOGIC_1280=n
SCSI_QLOGIC_FAS=n
SCSI_SAS_ATA=y
SCSI_SAS_ATTRS=m
SCSI_SAS_HOST_SMP=y
SCSI_SAS_LIBSAS=m
SCSI_SAS_LIBSAS_DEBUG=y
SCSI_SCAN_ASYNC=n
SCSI_SPI_ATTRS=m
SCSI_SRP=n
SCSI_SRP_ATTRS=n
SCSI_STEX=n
SCSI_SYM53C416=n
SCSI_SYM53C8XX_2=m
SCSI_SYM53C8XX_DEFAULT_TAGS=16
SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
SCSI_SYM53C8XX_MAX_TAGS=64
SCSI_SYM53C8XX_MMIO=y
SCSI_T128=n
SCSI_TGT=m
SCSI_U14_34F=n
SCSI_ULTRASTOR=n
SCSI_WAIT_SCAN=m
SCx200=n
SCx200_ACB=n
SECCOMP=n
SECURITY=n
SECURITY_FILE_CAPABILITIES=n
SEEQ8005=n
SELECT_MEMORY_MODEL=y
SEMAPHORE_SLEEPERS=y
SENSORS_ABITUGURU3=m
SENSORS_ABITUGURU=m
SENSORS_AD7418=m
SENSORS_ADM1021=m
SENSORS_ADM1025=m
SENSORS_ADM1026=m
SENSORS_ADM1029=m
SENSORS_ADM1031=m
SENSORS_ADM9240=m
SENSORS_ADS7828=m
SENSORS_ADT7470=m
SENSORS_ADT7473=m
SENSORS_APPLESMC=n
SENSORS_ASB100=m
SENSORS_ATXP1=m
SENSORS_CORETEMP=m
SENSORS_DME1737=m
SENSORS_DS1621=m
SENSORS_EEPROM=m
SENSORS_F71805F=m
SENSORS_F71882FG=m
SENSORS_F75375S=m
SENSORS_FSCHER=m
SENSORS_FSCHMD=m
SENSORS_FSCPOS=m
SENSORS_GL518SM=m
SENSORS_GL520SM=m
SENSORS_HDAPS=n
SENSORS_I5K_AMB=m
SENSORS_IBMPEX=m
SENSORS_IT87=m
SENSORS_K8TEMP=m
SENSORS_LM63=m
SENSORS_LM75=m
SENSORS_LM77=m
SENSORS_LM78=m
SENSORS_LM80=m
SENSORS_LM83=m
SENSORS_LM85=m
SENSORS_LM87=m
SENSORS_LM90=m
SENSORS_LM92=m
SENSORS_LM93=m
SENSORS_MAX1619=m
SENSORS_MAX6650=m
SENSORS_MAX6875=m
SENSORS_PC87360=m
SENSORS_PC87427=m
SENSORS_PCF8574=m
SENSORS_PCF8591=m
SENSORS_SIS5595=m
SENSORS_SMSC47B397=m
SENSORS_SMSC47M192=m
SENSORS_SMSC47M1=m
SENSORS_THMC50=m
SENSORS_TSL2550=n
SENSORS_VIA686A=m
SENSORS_VT1211=m
SENSORS_VT8231=m
SENSORS_W83627EHF=m
SENSORS_W83627HF=m
SENSORS_W83781D=m
SENSORS_W83791D=m
SENSORS_W83792D=m
SENSORS_W83793=m
SENSORS_W83L785TS=m
SENSORS_W83L786NG=m
SERIAL_8250=m
SERIAL_8250_CS=n
SERIAL_8250_DETECT_IRQ=n
SERIAL_8250_EXTENDED=y
SERIAL_8250_MANY_PORTS=n
SERIAL_8250_NR_UARTS=16
SERIAL_8250_PCI=m
SERIAL_8250_PNP=m
SERIAL_8250_RSA=n
SERIAL_8250_RUNTIME_UARTS=4
SERIAL_8250_SHARE_IRQ=y
SERIAL_CORE=m
SERIAL_JSM=n
SERIAL_NONSTANDARD=y
SERIO=y
SERIO_CT82C710=n
SERIO_I8042=y
SERIO_LIBPS2=y
SERIO_PARKBD=n
SERIO_PCIPS2=n
SERIO_RAW=n
SERIO_SERPORT=m
SGI_IOC4=n
SGI_PARTITION=n
SHMEM=y
SIGNALFD=y
SIS190=m
SIS900=m
SK98LIN=n
SKGE=m
SKY2=m
SLAB=n
SLABINFO=y
SLHC=m
SLIP=n
SLOB=n
SLUB=y
SLUB_DEBUG=y
SLUB_DEBUG_ON=n
SLUB_STATS=n
SMB_FS=n
SMC9194=n
SMP=y
SMSC37B787_WDT=n
SMSC_PHY=m
SND=m
SND_AC97_CODEC=m
SND_AC97_POWER_SAVE=y
SND_AC97_POWER_SAVE_DEFAULT=0
SND_AD1816A=n
SND_AD1848=n
SND_AD1889=m
SND_ADLIB=n
SND_ALI5451=m
SND_ALS100=n
SND_ALS300=n
SND_ALS4000=m
SND_ATIIXP=m
SND_ATIIXP_MODEM=n
SND_AU8810=m
SND_AU8820=m
SND_AU8830=m
SND_AZT2320=n
SND_AZT3328=m
SND_BT87X=m
SND_BT87X_OVERCLOCK=n
SND_CA0106=m
SND_CMI8330=n
SND_CMIPCI=m
SND_CS4231=n
SND_CS4232=n
SND_CS4236=n
SND_CS4281=m
SND_CS46XX=m
SND_CS46XX_NEW_DSP=y
SND_CS5530=n
SND_CS5535AUDIO=n
SND_DARLA20=n
SND_DARLA24=n
SND_DEBUG=n
SND_DT019X=n
SND_DUMMY=m
SND_DYNAMIC_MINORS=n
SND_ECHO3G=n
SND_EMU10K1=m
SND_EMU10K1X=m
SND_ENS1370=m
SND_ENS1371=m
SND_ES1688=n
SND_ES18XX=n
SND_ES1938=m
SND_ES1968=m
SND_ES968=n
SND_FM801=n
SND_GINA20=n
SND_GINA24=n
SND_GUSCLASSIC=n
SND_GUSEXTREME=n
SND_GUSMAX=n
SND_HDA_CODEC_ANALOG=y
SND_HDA_CODEC_ATIHDMI=y
SND_HDA_CODEC_CMEDIA=y
SND_HDA_CODEC_CONEXANT=y
SND_HDA_CODEC_REALTEK=y
SND_HDA_CODEC_SI3054=y
SND_HDA_CODEC_SIGMATEL=y
SND_HDA_CODEC_VIA=y
SND_HDA_GENERIC=y
SND_HDA_HWDEP=n
SND_HDA_INTEL=m
SND_HDA_POWER_SAVE=n
SND_HDSP=m
SND_HDSPM=m
SND_HIFIER=n
SND_HWDEP=m
SND_ICE1712=m
SND_ICE1724=m
SND_INDIGO=n
SND_INDIGODJ=n
SND_INDIGOIO=n
SND_INTEL8X0=m
SND_INTEL8X0M=m
SND_INTERWAVE=n
SND_INTERWAVE_STB=n
SND_KORG1212=m
SND_KORG1212_FIRMWARE_IN_KERNEL=y
SND_LAYLA20=n
SND_LAYLA24=n
SND_MAESTRO3=m
SND_MAESTRO3_FIRMWARE_IN_KERNEL=y
SND_MIA=n
SND_MIRO=n
SND_MIXART=m
SND_MIXER_OSS=m
SND_MONA=n
SND_MPU401=m
SND_MPU401_UART=m
SND_MTPAV=m
SND_MTS64=n
SND_NM256=m
SND_OPL3SA2=n
SND_OPL3_LIB=m
SND_OPTI92X_AD1848=n
SND_OPTI92X_CS4231=n
SND_OPTI93X=n
SND_OSSEMUL=y
SND_OXYGEN=n
SND_PCM=m
SND_PCM_OSS=m
SND_PCM_OSS_PLUGINS=y
SND_PCXHR=n
SND_PDAUDIOCF=n
SND_PORTMAN2X4=n
SND_RAWMIDI=m
SND_RIPTIDE=n
SND_RME32=m
SND_RME9652=m
SND_RME96=m
SND_RTCTIMER=m
SND_SB16=n
SND_SB8=n
SND_SBAWE=n
SND_SB_COMMON=m
SND_SC6000=n
SND_SEQUENCER=m
SND_SEQUENCER_OSS=y
SND_SEQ_DUMMY=m
SND_SEQ_RTCTIMER_DEFAULT=y
SND_SERIAL_U16550=m
SND_SGALAXY=n
SND_SIS7019=n
SND_SOC=n
SND_SONICVIBES=m
SND_SSCAPE=n
SND_SUPPORT_OLD_API=y
SND_TIMER=m
SND_TRIDENT=m
SND_USB_AUDIO=m
SND_USB_CAIAQ=n
SND_USB_USX2Y=n
SND_VERBOSE_PRINTK=n
SND_VERBOSE_PROCFS=y
SND_VIA82XX=m
SND_VIA82XX_MODEM=n
SND_VIRMIDI=m
SND_VIRTUOSO=n
SND_VX222=m
SND_VXPOCKET=n
SND_VX_LIB=m
SND_WAVEFRONT=n
SND_YMFPCI=m
SND_YMFPCI_FIRMWARE_IN_KERNEL=y
SOFT_WATCHDOG=m
SOLARIS_X86_PARTITION=n
SONYPI=n
SONYPI_COMPAT=n
SONY_LAPTOP=m
SOUND=m
SOUND_PRIME=n
SPARSEMEM_MANUAL=n
SPARSEMEM_STATIC=y
SPARSEMEM_VMEMMAP_ENABLE=n
SPECIALIX=n
SPI=n
SPI_MASTER=n
SPLIT_PTLOCK_CPUS=4
SQUASHFS=m
SQUASHFS_FRAGMENT_CACHE_SIZE=3
SSB=m
SSB_B43_PCI_BRIDGE=y
SSB_DEBUG=n
SSB_DRIVER_PCICORE=y
SSB_DRIVER_PCICORE_POSSIBLE=y
SSB_PCIHOST=y
SSB_PCIHOST_POSSIBLE=y
SSB_PCMCIAHOST=n
SSB_PCMCIAHOST_POSSIBLE=y
SSB_POSSIBLE=y
SSB_SILENT=n
STACKTRACE_SUPPORT=y
STALDRV=n
STANDALONE=y
STOP_MACHINE=y
SUNDANCE=m
SUNDANCE_MMIO=n
SUNGEM=n
SUNRPC=m
SUNRPC_BIND34=y
SUN_PARTITION=n
SUSPEND=y
SUSPEND_FREEZER=y
SWAP=y
SX=n
SYNCLINK=n
SYNCLINKMP=n
SYNCLINK_CS=n
SYNCLINK_GT=n
SYN_COOKIES=y
SYSCTL=y
SYSCTL_SYSCALL=n
SYSFS=y
SYSFS_DEPRECATED_V2=n
SYSV68_PARTITION=n
SYSVIPC=y
SYSVIPC_SYSCTL=y
SYSV_FS=n
SYS_HYPERVISOR=n
TASKSTATS=n
TC1100_WMI=m
TCG_TPM=n
TCIC=n
TCP_CONG_ADVANCED=n
TCP_CONG_CUBIC=y
TCP_MD5SIG=n
TEHUTI=n
TELCLOCK=n
TEXTSEARCH=y
TEXTSEARCH_BM=m
TEXTSEARCH_FSM=m
TEXTSEARCH_KMP=m
THERMAL=y
THINKPAD_ACPI=m
THINKPAD_ACPI_BAY=n
THINKPAD_ACPI_DEBUG=n
THINKPAD_ACPI_HOTKEY_POLL=y
THINKPAD_ACPI_VIDEO=y
TICK_ONESHOT=y
TIFM_CORE=n
TIGON3=m
TIMERFD=y
TIMER_STATS=n
TINY_SHMEM=n
TIPC=n
TLAN=m
TMD_HERMES=m
TMPFS=y
TMPFS_POSIX_ACL=n
TOSHIBA=m
TPS65010=m
TR=n
TRACE_IRQFLAGS_SUPPORT=y
TULIP=m
TULIP_MMIO=y
TULIP_MWI=y
TULIP_NAPI=y
TULIP_NAPI_HW_MITIGATION=y
TUN=m
TUNER_MT20XX=m
TUNER_SIMPLE=m
TUNER_TDA8290=m
TUNER_TDA9887=m
TUNER_TEA5761=m
TUNER_TEA5767=m
TUNER_XC2028=m
TYPHOON=m
UDF_FS=m
UDF_NLS=y
UEVENT_HELPER_PATH="/sbin/hotplug"
UFS_FS=n
UID16=y
UIO=n
ULI526X=n
ULTRA=m
ULTRIX_PARTITION=n
UNION_FS=m
UNION_FS_DEBUG=n
UNION_FS_XATTR=n
UNIX98_PTYS=y
UNIX=y
UNIXWARE_DISKLABEL=n
UNUSED_SYMBOLS=n
USB=m
USBPCWATCHDOG=n
USB_ACM=m
USB_ADUTUX=n
USB_ANNOUNCE_NEW_DEVICES=y
USB_APPLEDISPLAY=n
USB_ARCH_HAS_EHCI=y
USB_ARCH_HAS_HCD=y
USB_ARCH_HAS_OHCI=y
USB_ATM=n
USB_AUERSWALD=n
USB_BERRY_CHARGE=n
USB_CATC=n
USB_CYPRESS_CY7C63=n
USB_CYTHERM=n
USB_DEBUG=n
USB_DEVICEFS=y
USB_DEVICE_CLASS=y
USB_DYNAMIC_MINORS=n
USB_EHCI_HCD=m
USB_EHCI_ROOT_HUB_TT=y
USB_EHCI_TT_NEWSCHED=n
USB_EMI26=n
USB_EMI62=n
USB_FTDI_ELAN=n
USB_GADGET=n
USB_HID=m
USB_HIDDEV=y
USB_HIDINPUT_POWERBOOK=n
USB_IDMOUSE=n
USB_IOWARRIOR=n
USB_ISP116X_HCD=m
USB_KAWETH=n
USB_KBD=n
USB_LCD=n
USB_LD=n
USB_LED=n
USB_LEGOTOWER=n
USB_LIBUSUAL=n
USB_MDC800=n
USB_MICROTEK=n
USB_MON=n
USB_MOUSE=n
USB_NET_RNDIS_WLAN=n
USB_OHCI_BIG_ENDIAN_DESC=n
USB_OHCI_BIG_ENDIAN_MMIO=n
USB_OHCI_HCD=m
USB_OHCI_HCD_SSB=n
USB_OHCI_LITTLE_ENDIAN=y
USB_OTG=n
USB_PEGASUS=n
USB_PERSIST=n
USB_PHIDGET=n
USB_PRINTER=m
USB_R8A66597_HCD=n
USB_RIO500=n
USB_RTL8150=n
USB_SERIAL=n
USB_SISUSBVGA=n
USB_SL811_HCD=n
USB_STORAGE=m
USB_STORAGE_ALAUDA=y
USB_STORAGE_DATAFAB=y
USB_STORAGE_DEBUG=n
USB_STORAGE_DPCM=y
USB_STORAGE_FREECOM=y
USB_STORAGE_ISD200=y
USB_STORAGE_JUMPSHOT=y
USB_STORAGE_KARMA=y
USB_STORAGE_SDDR09=y
USB_STORAGE_SDDR55=y
USB_STORAGE_USBAT=y
USB_SUPPORT=y
USB_SUSPEND=n
USB_TEST=n
USB_TRANCEVIBRATOR=n
USB_UHCI_HCD=m
USB_USBNET=n
USB_USS720=n
USB_ZD1201=n
USER_SCHED=y
V4L_USB_DRIVERS=n
VETH=m
VFAT_FS=m
VGACON_SOFT_SCROLLBACK=n
VGASTATE=m
VGA_CONSOLE=y
VIA_RHINE=m
VIA_RHINE_MMIO=y
VIA_RHINE_NAPI=y
VIA_VELOCITY=m
VIDEOBUF_DMA_SG=m
VIDEOBUF_GEN=m
VIDEO_ADV_DEBUG=n
VIDEO_BT848=n
VIDEO_CAFE_CCIC=n
VIDEO_CAPTURE_DRIVERS=y
VIDEO_CX88=n
VIDEO_DEV=m
VIDEO_HELPER_CHIPS_AUTO=y
VIDEO_HEXIUM_GEMINI=n
VIDEO_HEXIUM_ORION=n
VIDEO_IR=m
VIDEO_IR_I2C=m
VIDEO_OUTPUT_CONTROL=m
VIDEO_SAA5246A=n
VIDEO_SAA5249=n
VIDEO_SAA7134=m
VIDEO_SAA7134_ALSA=m
VIDEO_SELECT=y
VIDEO_TUNER=m
VIDEO_TUNER_CUSTOMIZE=y
VIDEO_TVEEPROM=m
VIDEO_V4L1=n
VIDEO_V4L1_COMPAT=y
VIDEO_V4L2=y
VIDEO_V4L2_COMMON=m
VIDEO_VIVI=n
VIRTIO=m
VIRTIO_BALLOON=m
VIRTIO_BLK=m
VIRTIO_NET=m
VIRTIO_PCI=m
VIRTIO_RING=m
VIRTUALIZATION=y
VIRT_TO_BUS=y
VITESSE_PHY=m
VLAN_8021Q=m
VM86=y
VMSPLIT_1G=n
VMSPLIT_2G=n
VMSPLIT_2G_OPT=n
VMSPLIT_3G=y
VMSPLIT_3G_OPT=n
VM_EVENT_COUNTERS=y
VORTEX=m
VT=y
VT_CONSOLE=y
VT_HW_CONSOLE_BINDING=n
VXFS_FS=n
W1=n
W83627HF_WDT=n
W83697HF_WDT=n
W83877F_WDT=n
W83977F_WDT=n
WAFER_WDT=n
WAN=n
WAN_ROUTER=n
WATCHDOG=y
WATCHDOG_NOWAYOUT=n
WD80x3=n
WDT=n
WDTPCI=n
WINBOND_840=m
WIRELESS_EXT=y
WLAN_80211=y
WLAN_PRE80211=n
X25=n
X86=y
X86_32=y
X86_32_SMP=y
X86_64=n
X86_ACPI_CPUFREQ=m
X86_ACPI_CPUFREQ_PROC_INTF=n
X86_BIGSMP=n
X86_BIOS_REBOOT=y
X86_BSWAP=y
X86_CMOV=y
X86_CMPXCHG=y
X86_CPUFREQ_NFORCE2=m
X86_CPUID=m
X86_DEBUGCTLMSR=y
X86_ELAN=n
X86_ES7000=n
X86_E_POWERSAVER=m
X86_FIND_SMP_CONFIG=y
X86_GENERIC=y
X86_GENERICARCH=n
X86_GOOD_APIC=y
X86_GX_SUSPMOD=n
X86_HT=y
X86_INTEL_USERCOPY=y
X86_INVLPG=y
X86_IO_APIC=y
X86_L1_CACHE_SHIFT=7
X86_LOCAL_APIC=y
X86_LONGHAUL=m
X86_LONGRUN=n
X86_MCE=y
X86_MCE_NONFATAL=y
X86_MCE_P4THERMAL=y
X86_MINIMUM_CPU_FAMILY=4
X86_MPPARSE=y
X86_MSR=m
X86_NUMAQ=n
X86_P4_CLOCKMOD=m
X86_PC=y
X86_PM_TIMER=n
X86_POPAD_OK=y
X86_POWERNOW_K6=m
X86_POWERNOW_K7=m
X86_POWERNOW_K7_ACPI=y
X86_POWERNOW_K8=m
X86_POWERNOW_K8_ACPI=y
X86_RDC321X=n
X86_REBOOTFIXUPS=n
X86_SMP=y
X86_SPEEDSTEP_CENTRINO=n
X86_SPEEDSTEP_ICH=m
X86_SPEEDSTEP_LIB=m
X86_SPEEDSTEP_RELAXED_CAP_CHECK=n
X86_SPEEDSTEP_SMI=m
X86_SUMMIT=n
X86_TRAMPOLINE=y
X86_TSC=y
X86_USE_PPRO_CHECKSUM=y
X86_VISWS=n
X86_VOYAGER=n
X86_VSMP=n
X86_WP_WORKS_OK=y
X86_XADD=y
XFRM=y
XFRM_MIGRATE=n
XFRM_STATISTICS=n
XFRM_SUB_POLICY=n
XFRM_USER=m
XFS_FS=m
XFS_POSIX_ACL=y
XFS_QUOTA=y
XFS_RT=n
XFS_SECURITY=n
XOR_BLOCKS=m
YELLOWFIN=m
YENTA=m
YENTA_ENE_TUNE=y
YENTA_O2=y
YENTA_RICOH=y
YENTA_TI=y
YENTA_TOSHIBA=y
ZD1211RW=n
ZISOFS=y
ZLIB_DEFLATE=m
ZLIB_INFLATE=m
ZNET=n
ZONE_DMA32=n
ZONE_DMA=y
ZONE_DMA_FLAG=1

Re: 2.6.25: random stalls on certain hardware - regression?

[Oliver Neukum]

Am Dienstag 08 Juli 2008 23:56:12 schrieb Michael Tokarev:

> Any idea how to start welcome.  Meanwhile I'll try current 2.6.26pre,
> in a hope it will not destroy all our data.. ;)

Can you test whether select() triggers the lockup?

	Regards
		Oliver

Re: 2.6.25: random stalls on certain hardware - regression?

[Michael Tokarev]

Oliver Neukum wrote:
> Am Dienstag 08 Juli 2008 23:56:12 schrieb Michael Tokarev:
> 
>> Any idea how to start welcome.  Meanwhile I'll try current 2.6.26pre,
>> in a hope it will not destroy all our data.. ;)
> 
> Can you test whether select() triggers the lockup?

Hmm.  Sure I can, and it definitely uses select() here and there -
samba & Oracle8 & cronyd & vtund & syslogd - that's services
running on it all using select(2).  Can you elaborate a bit
more on this please?  I mean, should it lock up on first select()
(so that it's not what I see here) or on some random one (still
may be the situation I see), or not so-random which is more-or-less
possible to trigger?

Thank you for the reply!

/mjt

Re: 2.6.25: random stalls on certain hardware - regression?

[Oliver Neukum]

Am Mittwoch 09 Juli 2008 11:22:08 schrieb Michael Tokarev:
> Oliver Neukum wrote:
> > Am Dienstag 08 Juli 2008 23:56:12 schrieb Michael Tokarev:
> > 
> >> Any idea how to start welcome.  Meanwhile I'll try current 2.6.26pre,
> >> in a hope it will not destroy all our data.. ;)
> > 
> > Can you test whether select() triggers the lockup?
> 
> Hmm.  Sure I can, and it definitely uses select() here and there -
> samba & Oracle8 & cronyd & vtund & syslogd - that's services
> running on it all using select(2).  Can you elaborate a bit
> more on this please?  I mean, should it lock up on first select()
> (so that it's not what I see here) or on some random one (still

select() but not necessary the first call. I've got a report which seems
to indicate a rogue pointer while building the poll table. You might
strace a test programm and see whether it hangs in select() if it manages
to trigger the lockup.

	Regards
		Oliver

Re: 2.6.25: random stalls on certain hardware - regression?

[Marcin Slusarz]

On Wed, Jul 09, 2008 at 01:56:12AM +0400, Michael Tokarev wrote:
> No hope to resolve this but still - maybe someone has an idea...
>
> Hardware is -
>  AMD Athlon x2-64 system
>  Asus M2N-SLI DELUXE motherboard (nvidia MCP55)
>  AMD BE-2400 CPU
>  2x 2Gb mem
>  Adaptec 3950B Ultra2 SCSI adapter (*)
>  4x 36Gb Seagate SCSI disks
>  2x nvidia GbE ethernet
>  rtl8139 nic as 3rd one
>
> When booting 2.6.25 - either 32 or 64 bits - the system freezes/hangs
> at some random point.  So far there was about 10 hangs, some after
> several minutes after boot, some are within hours.
>
> The "hang" is a complete system freeze - on the console there's
> still "... login:" prompt, but nothing works - keyboard is stuck
> (numlock doesn't work), and server is not responding over network.
> Sometimes ping to the server itself works, but definitely not routing.
> (it's a server so no fancy stuff is loaded - X isn't even installed).
>
> The kernel is vanilla 2.6.25 - tried several - .5, .8, .10 now -
> the effect is the same.
>
> 2.6.24 and before worked without any glitch (2.6.24 is currently
> running).  We tried different versions of BIOS (was 12something
> before, tried 1304 and - currently - 1405, since 1502 is still
> beta) - no difference at all.
>
> The problem is that it is a production machine, and quite some
> people depend on it (it's a remote office with only one server),
> so I've very limited ability to try something.  Unfortunately not
> git bisect, -- or at least I'm afraid to try it, both because of
> possibility to have many reboots AND new freezes, and because
> unstable kernel (2.6.25pre stuff) with possibility to break something.

Definitely try 2.6.26-rc9 (which will soon become 2.6.26, so people will
probably ask you to test patches on top of this kernel) and if it will still 
lock up, take a look at Documentation/nmi_watchdog.txt and maybe
Documentation/networking/netconsole.txt in kernel sources.

Marcin

Re: 2.6.25: random stalls on certain hardware - regression?

[Michael Tokarev]

Marcin Slusarz wrote:
> On Wed, Jul 09, 2008 at 01:56:12AM +0400, Michael Tokarev wrote:

[random complete hangs on 2.6.25 but not 2.6.24]

> Definitely try 2.6.26-rc9 (which will soon become 2.6.26, so people will

Ok, tried 2.6.26-rc9 today night - after disabling some drivers which
were not compilable and fixing some places in core code.  It does the
same thing - i.e. it locks up the same way as 2.6.25 did, and locks up
hard as before (no keyboard and network working).

> probably ask you to test patches on top of this kernel) and if it will still 
> lock up, take a look at Documentation/nmi_watchdog.txt and maybe
> Documentation/networking/netconsole.txt in kernel sources.

Aha!.  That's very useful information, let me try just that... later
today, after work hours.  Added nmi_watchdog=1 and vga=0x5032 (in a
hope to see the whole thing on the screen) to 2.6.26pre lilo line...

Thanks!

Re: 2.6.25: random stalls on certain hardware - regression?

[Michael Tokarev]

Oliver Neukum wrote:

[hard hangs in 2.6.25+ may be related to select()]
> 
> select() but not necessary the first call. I've got a report which seems
> to indicate a rogue pointer while building the poll table. You might
> strace a test programm and see whether it hangs in select() if it manages
> to trigger the lockup.

I wrote a small program that opens/closes tcp and unix sockets at random,
does listen() on them and select()s them with zero timeout, with short
sleep()s in between - so that most of the time it will be in sleep(),
and only for a short time in select() and other syscalls.  So if strace
will show it hanged in sleep() (most "active") we can't prove anything,
and if it will hand in select().... well... we can't prove anything
either, really... ;)  I think.  Today night it hanged again, now with
2.6.26-pre9 kernel, but I wasn't able to see the strace output unfortunately.
Will re-try again tonight.

The question here is if it all really worth the effort.  The thing is that
we cant prove anything either way, because if it were hanged in sleep() it
may be due to some OTHER program were in "bad" select() at that time, or
if strace showed it hanged in select(), it may be that some other part of
the system hanged at that time (but this is less likely still, as select()
timeframe is very small compared with sleep() timeframe).

Thanks!

/mjt

Re: 2.6.25: random stalls on certain hardware - regression?

[Michael Tokarev]

Michael Tokarev wrote at Thu, 10 Jul 2008 11:04:55 +0400:

[almost a month-ago email/thread...]

> Marcin Slusarz wrote:
>> On Wed, Jul 09, 2008 at 01:56:12AM +0400, Michael Tokarev wrote:
> 
> [random complete hangs on 2.6.25 but not 2.6.24]
> 
>> Definitely try 2.6.26-rc9 (which will soon become 2.6.26, so people will
> 
> Ok, tried 2.6.26-rc9 today night - after disabling some drivers which
> were not compilable and fixing some places in core code.  It does the
> same thing - i.e. it locks up the same way as 2.6.25 did, and locks up
> hard as before (no keyboard and network working).

Ok, I finally tried current 2.6.26 [.2] on this same machine, -- it's
up and running since last Friday, so almost 4 days already, and it works
so far, no lock-ups for now.  2.6.26-pre definitely didn't work - max
uptime was ~12 hours.  I can't say the problem has been fixed for sure,
maybe it's just a {bad|good} luck, will see how it will work in next days.

And btw, nmi-watchdog didn't help, -- with it enabled, the machine locked
up the same way as before -- completely silent, nothing's on the screen etc.
(And no, it's not a hardware problem, since all kernels up to 2.6.24 works
just fine, under any loads.  Or should not be hardware, anyway)

/mjt