* Network performance degradation from 2.6.11.12 to 2.6.16.20
@ 2006-06-16 16:01 Harry Edmon
2006-06-17 22:35 ` Andrew Morton
0 siblings, 1 reply; 59+ messages in thread
From: Harry Edmon @ 2006-06-16 16:01 UTC (permalink / raw)
To: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]
I have a system with a strange network performance degradation from
2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
The system is has Dual single core Xeons with hyperthreading on. The
application is the LDM system from UCAR/Unidata
(http://www.unidata.ucar.edu/software/ldm). This system requests
weather data from a variety of systems using RPC calls over a reserved
TCP port (388), puts them into a memory mapped queue file, and then
sends the data out to a variety of downstream requesting systems, again
using RPC calls. When the load is heavy, the 2.6.16.20 kernel falls way
behind with the data ingestion. The 2.6.11.12 kernel does not. I have
tried an experiment with a 2.6.17-rc6 system where it just does the
ingestion, and not the downstream distribution, and it is able to keep
up. I would really appreciate any pointers as to where the problem may
be and how to diagnose it. I have attached the config files from both
kernels and the sysctl.conf file I am using. I have also included the
output from "netstat -s" on the 2.6.16.20 system during a time when it
was having problems.
[-- Attachment #2: config-2.6.11.12 --]
[-- Type: text/plain, Size: 36012 bytes --]
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.11.12
# Wed Oct 19 11:16:32 2005
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_LOCK_KERNEL=y
#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=14
CONFIG_HOTPLUG=y
CONFIG_KOBJECT_UEVENT=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
# CONFIG_TINY_SHMEM is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y
#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
CONFIG_X86_GENERIC=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_HPET_TIMER=y
CONFIG_SMP=y
CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_TSC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=m
# CONFIG_X86_MCE_P4THERMAL is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
#
# Firmware Drivers
#
CONFIG_EDD=m
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_HIGHMEM=y
CONFIG_HIGHPTE=y
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
CONFIG_IRQBALANCE=y
CONFIG_HAVE_DEC_LOCK=y
# CONFIG_REGPARM is not set
#
# Power management options (ACPI, APM)
#
# CONFIG_PM is not set
#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_THERMAL=m
CONFIG_ACPI_ASUS=m
# CONFIG_ACPI_IBM is not set
CONFIG_ACPI_TOSHIBA=m
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCIEPORTBUS is not set
CONFIG_PCI_MSI=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set
#
# PC-card bridges
#
#
# PCI Hotplug Support
#
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_COMPAQ=m
CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y
CONFIG_HOTPLUG_PCI_IBM=m
CONFIG_HOTPLUG_PCI_ACPI=m
# CONFIG_HOTPLUG_PCI_ACPI_IBM is not set
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=m
# CONFIG_HOTPLUG_PCI_SHPC_POLL_EVENT_MODE is not set
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
# CONFIG_PARPORT is not set
#
# Plug and Play support
#
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set
#
# Protocols
#
CONFIG_PNPACPI=y
#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=8192
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_LBD=y
# CONFIG_CDROM_PKTCDVD is not set
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_ATA_OVER_ETH is not set
#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=m
CONFIG_BLK_DEV_IDE=m
#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=m
# CONFIG_IDEDISK_MULTI_MODE is not set
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDETAPE=m
CONFIG_BLK_DEV_IDEFLOPPY=m
CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set
#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
CONFIG_BLK_DEV_CMD640=y
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
# CONFIG_BLK_DEV_IDEPNP is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=m
CONFIG_BLK_DEV_OPTI621=m
CONFIG_BLK_DEV_RZ1000=m
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_AEC62XX=m
CONFIG_BLK_DEV_ALI15X3=m
# CONFIG_WDC_ALI15X3 is not set
CONFIG_BLK_DEV_AMD74XX=m
CONFIG_BLK_DEV_ATIIXP=m
CONFIG_BLK_DEV_CMD64X=m
CONFIG_BLK_DEV_TRIFLEX=m
CONFIG_BLK_DEV_CY82C693=m
CONFIG_BLK_DEV_CS5520=m
CONFIG_BLK_DEV_CS5530=m
CONFIG_BLK_DEV_HPT34X=m
# CONFIG_HPT34X_AUTODMA is not set
CONFIG_BLK_DEV_HPT366=m
CONFIG_BLK_DEV_SC1200=m
CONFIG_BLK_DEV_PIIX=m
CONFIG_BLK_DEV_NS87415=m
CONFIG_BLK_DEV_PDC202XX_OLD=m
CONFIG_PDC202XX_BURST=y
CONFIG_BLK_DEV_PDC202XX_NEW=m
# CONFIG_PDC202XX_FORCE is not set
CONFIG_BLK_DEV_SVWKS=m
CONFIG_BLK_DEV_SIIMAGE=m
CONFIG_BLK_DEV_SIS5513=m
CONFIG_BLK_DEV_SLC90E66=m
CONFIG_BLK_DEV_TRM290=m
CONFIG_BLK_DEV_VIA82CXXX=m
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#
CONFIG_SCSI=m
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
CONFIG_CHR_DEV_OSST=m
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=m
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
# CONFIG_SCSI_ISCSI_ATTRS is not set
#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=m
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=8
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
CONFIG_AIC7XXX_DEBUG_ENABLE=y
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC7XXX_OLD=m
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=15000
CONFIG_AIC79XX_ENABLE_RD_STRM=y
CONFIG_AIC79XX_DEBUG_ENABLE=y
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
CONFIG_SCSI_DPT_I2O=m
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_AHCI is not set
CONFIG_SCSI_SATA_SVW=m
CONFIG_SCSI_ATA_PIIX=m
CONFIG_SCSI_SATA_NV=m
CONFIG_SCSI_SATA_PROMISE=m
# CONFIG_SCSI_SATA_QSTOR is not set
CONFIG_SCSI_SATA_SX4=m
CONFIG_SCSI_SATA_SIL=m
CONFIG_SCSI_SATA_SIS=m
# CONFIG_SCSI_SATA_ULI is not set
CONFIG_SCSI_SATA_VIA=m
CONFIG_SCSI_SATA_VITESSE=m
CONFIG_SCSI_BUSLOGIC=m
# CONFIG_SCSI_OMIT_FLASHPOINT is not set
CONFIG_SCSI_DMX3191D=m
CONFIG_SCSI_EATA=m
CONFIG_SCSI_EATA_TAGGED_QUEUE=y
CONFIG_SCSI_EATA_LINKED_COMMANDS=y
CONFIG_SCSI_EATA_MAX_TAGS=16
CONFIG_SCSI_EATA_PIO=m
CONFIG_SCSI_FUTURE_DOMAIN=m
CONFIG_SCSI_GDTH=m
CONFIG_SCSI_IPS=m
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
CONFIG_SCSI_IPR=m
# CONFIG_SCSI_IPR_TRACE is not set
# CONFIG_SCSI_IPR_DUMP is not set
CONFIG_SCSI_QLOGIC_ISP=m
CONFIG_SCSI_QLOGIC_FC=m
CONFIG_SCSI_QLOGIC_FC_FIRMWARE=y
CONFIG_SCSI_QLOGIC_1280=m
# CONFIG_SCSI_QLOGIC_1280_1040 is not set
CONFIG_SCSI_QLA2XXX=m
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
CONFIG_SCSI_DC395x=m
CONFIG_SCSI_DC390T=m
CONFIG_SCSI_NSP32=m
CONFIG_SCSI_DEBUG=m
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
# CONFIG_MD_RAID10 is not set
CONFIG_MD_RAID5=m
CONFIG_MD_RAID6=m
CONFIG_MD_MULTIPATH=m
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
CONFIG_I2O=m
CONFIG_I2O_CONFIG=m
CONFIG_I2O_BLOCK=m
CONFIG_I2O_SCSI=m
CONFIG_I2O_PROC=m
#
# Networking support
#
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=m
CONFIG_PACKET_MMAP=y
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=m
CONFIG_NET_KEY=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_FWMARK=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m
CONFIG_IP_TCPDIAG=y
# CONFIG_IP_TCPDIAG_IPV6 is not set
#
# IP: Virtual Server Configuration
#
CONFIG_IP_VS=m
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12
#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IPV6=m
CONFIG_IPV6_PRIVACY=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
CONFIG_INET6_TUNNEL=m
CONFIG_IPV6_TUNNEL=m
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
# CONFIG_IP_NF_CT_ACCT is not set
# CONFIG_IP_NF_CONNTRACK_MARK is not set
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_REALM=m
# CONFIG_IP_NF_MATCH_SCTP is not set
# CONFIG_IP_NF_MATCH_COMMENT is not set
# CONFIG_IP_NF_MATCH_HASHLIMIT is not set
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_NAT_TFTP=m
CONFIG_IP_NF_NAT_AMANDA=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_TARGET_CLASSIFY=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_TARGET_NOTRACK=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
#
# IPv6: Netfilter Configuration
#
CONFIG_IP6_NF_QUEUE=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_LIMIT=m
CONFIG_IP6_NF_MATCH_MAC=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_MULTIPORT=m
CONFIG_IP6_NF_MATCH_OWNER=m
CONFIG_IP6_NF_MATCH_MARK=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_AHESP=m
CONFIG_IP6_NF_MATCH_LENGTH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_TARGET_MARK=m
CONFIG_IP6_NF_RAW=m
CONFIG_XFRM=y
CONFIG_XFRM_USER=m
#
# SCTP Configuration (EXPERIMENTAL)
#
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
# CONFIG_NET_SCH_CLK_CPU is not set
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
# CONFIG_NET_CLS_IND is not set
# CONFIG_CLS_U32_MARK is not set
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
# CONFIG_NET_CLS_ACT is not set
CONFIG_NET_CLS_POLICE=y
#
# Network testing
#
CONFIG_NET_PKTGEN=m
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_NET_SB1000 is not set
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
#
# Tulip family network device support
#
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_HP100=m
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
# CONFIG_AMD8111E_NAPI is not set
CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_ADAPTEC_STARFIRE_NAPI is not set
CONFIG_B44=m
CONFIG_FORCEDETH=m
# CONFIG_DGRS is not set
CONFIG_EEPRO100=m
CONFIG_E100=m
# CONFIG_E100_NAPI is not set
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
CONFIG_8139TOO_PIO=y
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_TLAN=m
CONFIG_VIA_RHINE=m
# CONFIG_VIA_RHINE_MMIO is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
CONFIG_DL2K=m
CONFIG_E1000=m
# CONFIG_E1000_NAPI is not set
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=m
# CONFIG_R8169_NAPI is not set
CONFIG_SK98LIN=m
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=m
#
# Ethernet (10000 Mbit)
#
CONFIG_IXGB=m
# CONFIG_IXGB_NAPI is not set
CONFIG_S2IO=m
# CONFIG_S2IO_NAPI is not set
# CONFIG_2BUFF_MODE is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
CONFIG_SHAPER=m
CONFIG_NETCONSOLE=m
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_TSDEV=m
CONFIG_INPUT_TSDEV_SCREEN_X=240
CONFIG_INPUT_TSDEV_SCREEN_Y=320
CONFIG_INPUT_EVDEV=m
CONFIG_INPUT_EVBUG=m
#
# Input I/O drivers
#
CONFIG_GAMEPORT=m
CONFIG_SOUND_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_VORTEX=m
CONFIG_GAMEPORT_FM801=m
# CONFIG_GAMEPORT_CS461X is not set
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_VSXXXAA=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=y
CONFIG_JOYSTICK_IFORCE_232=y
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
# CONFIG_JOYSTICK_TWIDDLER is not set
# CONFIG_JOYSTICK_JOYDUMP is not set
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_GUNZE=m
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
CONFIG_INPUT_UINPUT=m
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_ROCKETPORT=m
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
CONFIG_MOXA_SMARTIO=m
# CONFIG_ISI is not set
CONFIG_SYNCLINK=m
CONFIG_SYNCLINKMP=m
CONFIG_N_HDLC=m
CONFIG_STALDRV=y
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
# CONFIG_SERIAL_8250_ACPI is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_MULTIPORT=y
CONFIG_SERIAL_8250_RSA=y
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
#
# IPMI
#
CONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
#
# Watchdog Cards
#
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_ACQUIRE_WDT=m
CONFIG_ADVANTECH_WDT=m
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
CONFIG_SC520_WDT=m
CONFIG_EUROTECH_WDT=m
CONFIG_IB700_WDT=m
CONFIG_WAFER_WDT=m
CONFIG_I8XX_TCO=m
CONFIG_SC1200_WDT=m
CONFIG_SCx200_WDT=m
CONFIG_60XX_WDT=m
CONFIG_CPU5_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_MACHZ_WDT=m
#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m
CONFIG_WDT_501_PCI=y
#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_HW_RANDOM=m
CONFIG_NVRAM=m
CONFIG_RTC=m
CONFIG_GEN_RTC=m
CONFIG_GEN_RTC_X=y
CONFIG_DTLK=m
CONFIG_R3964=m
CONFIG_APPLICOM=m
CONFIG_SONYPI=m
#
# Ftape, the floppy tape device driver
#
CONFIG_AGP=m
CONFIG_AGP_ALI=m
CONFIG_AGP_ATI=m
CONFIG_AGP_AMD=m
CONFIG_AGP_AMD64=m
CONFIG_AGP_INTEL=m
# CONFIG_AGP_INTEL_MCH is not set
CONFIG_AGP_NVIDIA=m
CONFIG_AGP_SIS=m
CONFIG_AGP_SWORKS=m
CONFIG_AGP_VIA=m
CONFIG_AGP_EFFICEON=m
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
CONFIG_DRM_I810=m
# CONFIG_DRM_I830 is not set
# CONFIG_DRM_I915 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
# CONFIG_HPET_RTC_IRQ is not set
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=m
#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_CHARDEV=m
#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m
# CONFIG_I2C_ALGOPCA is not set
#
# I2C Hardware Bus support
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
# CONFIG_I2C_AMD756_S4882 is not set
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_I810=m
CONFIG_I2C_ISA=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_PROSAVAGE=m
CONFIG_I2C_SAVAGE4=m
CONFIG_SCx200_ACB=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
# CONFIG_I2C_STUB is not set
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
CONFIG_I2C_VOODOO3=m
# CONFIG_I2C_PCA_ISA is not set
#
# Hardware Sensors Chip support
#
CONFIG_I2C_SENSOR=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
# CONFIG_SENSORS_ADM1026 is not set
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_FSCHER=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_IT87=m
# CONFIG_SENSORS_LM63 is not set
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
# CONFIG_SENSORS_LM87 is not set
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_MAX1619=m
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83627HF=m
#
# Other I2C Chip support
#
CONFIG_SENSORS_EEPROM=m
CONFIG_SENSORS_PCF8574=m
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_RTC8564=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
CONFIG_FB_CIRRUS=m
CONFIG_FB_PM2=m
CONFIG_FB_PM2_FIFO_DISCONNECT=y
CONFIG_FB_CYBER2000=m
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
CONFIG_FB_VGA16=m
# CONFIG_FB_VESA is not set
CONFIG_VIDEO_SELECT=y
CONFIG_FB_HGA=m
# CONFIG_FB_HGA_ACCEL is not set
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
CONFIG_FB_RIVA_DEBUG=y
CONFIG_FB_I810=m
# CONFIG_FB_I810_GTF is not set
# CONFIG_FB_INTEL is not set
CONFIG_FB_MATROX=m
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
# CONFIG_FB_MATROX_G is not set
CONFIG_FB_MATROX_I2C=m
CONFIG_FB_MATROX_MULTIHEAD=y
CONFIG_FB_RADEON_OLD=m
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
# CONFIG_FB_RADEON_DEBUG is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY=m
CONFIG_FB_ATY_CT=y
# CONFIG_FB_ATY_GENERIC_LCD is not set
CONFIG_FB_ATY_XL_INIT=y
CONFIG_FB_ATY_GX=y
# CONFIG_FB_SAVAGE is not set
CONFIG_FB_SIS=m
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
CONFIG_FB_3DFX=m
# CONFIG_FB_3DFX_ACCEL is not set
CONFIG_FB_VOODOO1=m
CONFIG_FB_TRIDENT=m
# CONFIG_FB_TRIDENT_ACCEL is not set
CONFIG_FB_VIRTUAL=m
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=m
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
#
# Logo configuration
#
# CONFIG_LOGO is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
#
# Sound
#
# CONFIG_SOUND is not set
#
# USB support
#
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_BANDWIDTH=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG is not set
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_SPLIT_ISO=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_OHCI_HCD=m
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_SL811_HCD is not set
#
# USB Device Class drivers
#
# CONFIG_USB_BLUETOOTH_TTY is not set
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support' may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_RW_DETECT is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
# CONFIG_USB_STORAGE_HP8200e is not set
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
# CONFIG_HID_FF is not set
CONFIG_USB_HIDDEV=y
#
# USB HID Boot Protocol drivers
#
CONFIG_USB_KBD=m
CONFIG_USB_MOUSE=m
CONFIG_USB_AIPTEK=m
CONFIG_USB_WACOM=m
CONFIG_USB_KBTAB=m
CONFIG_USB_POWERMATE=m
CONFIG_USB_MTOUCH=m
CONFIG_USB_EGALAX=m
CONFIG_USB_XPAD=m
CONFIG_USB_ATI_REMOTE=m
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
#
# Video4Linux support is needed for USB Multimedia device support
#
#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
#
# USB Host-to-Host Cables
#
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_GENESYS=y
CONFIG_USB_NET1080=y
CONFIG_USB_PL2301=y
CONFIG_USB_KC2190=y
#
# Intelligent USB Devices/Gadgets
#
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_ZAURUS=y
CONFIG_USB_CDCETHER=y
#
# USB Network Adapters
#
CONFIG_USB_AX8817X=y
#
# USB port drivers
#
#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
# CONFIG_USB_SERIAL_KEYSPAN_MPR is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA18X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QW is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QI is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49WLC is not set
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_SAFE=m
# CONFIG_USB_SERIAL_SAFE_PADDED is not set
# CONFIG_USB_SERIAL_TI is not set
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_EZUSB=y
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
CONFIG_USB_AUERSWALD=m
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_LED=m
CONFIG_USB_CYTHERM=m
# CONFIG_USB_PHIDGETKIT is not set
CONFIG_USB_PHIDGETSERVO=m
# CONFIG_USB_IDMOUSE is not set
CONFIG_USB_TEST=m
#
# USB ATM/DSL drivers
#
#
# USB Gadget Support
#
CONFIG_USB_GADGET=m
# CONFIG_USB_GADGET_DEBUG_FILES is not set
CONFIG_USB_GADGET_NET2280=y
CONFIG_USB_NET2280=m
# CONFIG_USB_GADGET_PXA2XX is not set
# CONFIG_USB_GADGET_GOKU is not set
# CONFIG_USB_GADGET_SA1100 is not set
# CONFIG_USB_GADGET_LH7A40X is not set
# CONFIG_USB_GADGET_DUMMY_HCD is not set
# CONFIG_USB_GADGET_OMAP is not set
CONFIG_USB_GADGET_DUALSPEED=y
CONFIG_USB_ZERO=m
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
CONFIG_USB_GADGETFS=m
CONFIG_USB_FILE_STORAGE=m
# CONFIG_USB_FILE_STORAGE_TEST is not set
CONFIG_USB_G_SERIAL=m
#
# MMC/SD Card support
#
# CONFIG_MMC is not set
#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set
#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_REISERFS_FS_XATTR is not set
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
# CONFIG_JFS_SECURITY is not set
# CONFIG_JFS_DEBUG is not set
CONFIG_JFS_STATISTICS=y
CONFIG_FS_POSIX_ACL=y
#
# XFS support
#
CONFIG_XFS_FS=m
CONFIG_XFS_EXPORT=y
# CONFIG_XFS_RT is not set
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_SECURITY is not set
# CONFIG_XFS_POSIX_ACL is not set
CONFIG_MINIX_FS=m
CONFIG_ROMFS_FS=m
CONFIG_QUOTA=y
CONFIG_QFMT_V1=m
CONFIG_QFMT_V2=m
CONFIG_QUOTACTL=y
CONFIG_DNOTIFY=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=m
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=m
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_DEVFS_FS=y
# CONFIG_DEVFS_MOUNT is not set
# CONFIG_DEVFS_DEBUG is not set
CONFIG_DEVPTS_FS_XATTR=y
CONFIG_DEVPTS_FS_SECURITY=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_XATTR is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
#
# Miscellaneous filesystems
#
CONFIG_ADFS_FS=m
# CONFIG_ADFS_FS_RW is not set
CONFIG_AFFS_FS=m
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BEFS_FS=m
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=m
CONFIG_EFS_FS=m
CONFIG_CRAMFS=y
CONFIG_VXFS_FS=m
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_RPCSEC_GSS_KRB5=y
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
# CONFIG_SMB_NLS_DEFAULT is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
CONFIG_NCP_FS=m
CONFIG_NCPFS_PACKET_SIGNING=y
CONFIG_NCPFS_IOCTL_LOCKING=y
CONFIG_NCPFS_STRONG=y
CONFIG_NCPFS_NFS_NS=y
CONFIG_NCPFS_OS2_NS=y
# CONFIG_NCPFS_SMALLDOS is not set
CONFIG_NCPFS_NLS=y
CONFIG_NCPFS_EXTRAS=y
CONFIG_CODA_FS=m
# CONFIG_CODA_FS_OLD_API is not set
CONFIG_AFS_FS=m
CONFIG_RXRPC=m
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
CONFIG_ACORN_PARTITION_CUMANA=y
# CONFIG_ACORN_PARTITION_EESOX is not set
CONFIG_ACORN_PARTITION_ICS=y
# CONFIG_ACORN_PARTITION_ADFS is not set
# CONFIG_ACORN_PARTITION_POWERTEC is not set
CONFIG_ACORN_PARTITION_RISCIX=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
# CONFIG_EFI_PARTITION is not set
#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="cp437"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
#
# Profiling support
#
CONFIG_PROFILING=y
CONFIG_OPROFILE=m
#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_HIGHMEM is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_KPROBES is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_4KSTACKS is not set
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
#
# Security options
#
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
# CONFIG_SECURITY_NETWORK is not set
CONFIG_SECURITY_CAPABILITIES=m
CONFIG_SECURITY_ROOTPLUG=m
# CONFIG_SECURITY_SECLVL is not set
# CONFIG_SECURITY_SELINUX is not set
#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
# CONFIG_CRYPTO_WP512 is not set
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_TEST=m
#
# Hardware crypto devices
#
# CONFIG_CRYPTO_DEV_PADLOCK is not set
#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=m
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_PC=y
[-- Attachment #3: config-2.6.16.20 --]
[-- Type: text/plain, Size: 37194 bytes --]
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.16.20
# Thu Jun 15 15:47:21 2006
#
CONFIG_X86_32=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
# CONFIG_IKCONFIG is not set
# CONFIG_CPUSETS is not set
CONFIG_INITRAMFS_SOURCE=""
CONFIG_UID16=y
CONFIG_VM86=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
CONFIG_SLAB=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
# CONFIG_SLOB is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y
#
# Block layer
#
CONFIG_LBD=y
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_HPET_TIMER=y
CONFIG_SMP=y
CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
# CONFIG_PREEMPT_BKL is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=m
CONFIG_X86_MCE_P4THERMAL=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_X86_REBOOTFIXUPS is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
#
# Firmware Drivers
#
CONFIG_EDD=m
# CONFIG_DELL_RBU is not set
CONFIG_DCDBAS=m
# CONFIG_NOHIGHMEM is not set
# CONFIG_HIGHMEM4G is not set
CONFIG_HIGHMEM64G=y
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
CONFIG_X86_PAE=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_HIGHPTE=y
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
CONFIG_IRQBALANCE=y
# CONFIG_REGPARM is not set
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x100000
# CONFIG_HOTPLUG_CPU is not set
CONFIG_DOUBLEFAULT=y
#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
CONFIG_PM_DEBUG=y
#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_AC=m
# CONFIG_ACPI_BATTERY is not set
CONFIG_ACPI_BUTTON=m
# CONFIG_ACPI_VIDEO is not set
CONFIG_ACPI_HOTKEY=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_THERMAL=m
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_IBM is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
#
# APM (Advanced Power Management) BIOS Support
#
# CONFIG_APM is not set
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCIEPORTBUS is not set
CONFIG_PCI_MSI=y
# CONFIG_PCI_LEGACY_PROC is not set
# CONFIG_PCI_DEBUG is not set
CONFIG_ISA_DMA_API=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set
#
# PCI Hotplug Support
#
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_COMPAQ=m
CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y
CONFIG_HOTPLUG_PCI_IBM=m
# CONFIG_HOTPLUG_PCI_ACPI is not set
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=m
# CONFIG_HOTPLUG_PCI_SHPC_POLL_EVENT_MODE is not set
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m
#
# Networking
#
CONFIG_NET=y
#
# Networking options
#
# CONFIG_NETDEBUG is not set
CONFIG_PACKET=m
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=m
CONFIG_XFRM=y
CONFIG_XFRM_USER=m
CONFIG_NET_KEY=m
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_ADVANCED=y
#
# TCP congestion control
#
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=m
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_SCALABLE=m
# CONFIG_IPV6 is not set
# CONFIG_NETFILTER is not set
#
# DCCP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
#
# TIPC Configuration (EXPERIMENTAL)
#
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_IEEE80211 is not set
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set
#
# Connector - unified userspace <-> kernelspace linker
#
# CONFIG_CONNECTOR is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
# CONFIG_PARPORT is not set
#
# Plug and Play support
#
# CONFIG_PNP is not set
#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=8192
CONFIG_BLK_DEV_INITRD=y
# CONFIG_CDROM_PKTCDVD is not set
# CONFIG_ATA_OVER_ETH is not set
#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=m
CONFIG_BLK_DEV_IDE=m
#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=m
# CONFIG_IDEDISK_MULTI_MODE is not set
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDETAPE=m
CONFIG_BLK_DEV_IDEFLOPPY=m
CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set
#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
CONFIG_BLK_DEV_CMD640=y
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=m
CONFIG_BLK_DEV_OPTI621=m
CONFIG_BLK_DEV_RZ1000=m
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_AEC62XX=m
CONFIG_BLK_DEV_ALI15X3=m
# CONFIG_WDC_ALI15X3 is not set
CONFIG_BLK_DEV_AMD74XX=m
CONFIG_BLK_DEV_ATIIXP=m
CONFIG_BLK_DEV_CMD64X=m
CONFIG_BLK_DEV_TRIFLEX=m
CONFIG_BLK_DEV_CY82C693=m
CONFIG_BLK_DEV_CS5520=m
CONFIG_BLK_DEV_CS5530=m
# CONFIG_BLK_DEV_CS5535 is not set
CONFIG_BLK_DEV_HPT34X=m
# CONFIG_HPT34X_AUTODMA is not set
CONFIG_BLK_DEV_HPT366=m
CONFIG_BLK_DEV_SC1200=m
CONFIG_BLK_DEV_PIIX=m
# CONFIG_BLK_DEV_IT821X is not set
CONFIG_BLK_DEV_NS87415=m
CONFIG_BLK_DEV_PDC202XX_OLD=m
CONFIG_PDC202XX_BURST=y
CONFIG_BLK_DEV_PDC202XX_NEW=m
CONFIG_BLK_DEV_SVWKS=m
CONFIG_BLK_DEV_SIIMAGE=m
CONFIG_BLK_DEV_SIS5513=m
CONFIG_BLK_DEV_SLC90E66=m
CONFIG_BLK_DEV_TRM290=m
CONFIG_BLK_DEV_VIA82CXXX=m
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=m
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
CONFIG_CHR_DEV_OSST=m
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=m
# CONFIG_CHR_DEV_SCH is not set
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
#
# SCSI low-level drivers
#
# CONFIG_ISCSI_TCP is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=8
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
CONFIG_AIC7XXX_DEBUG_ENABLE=y
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC7XXX_OLD=m
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=15000
CONFIG_AIC79XX_ENABLE_RD_STRM=y
CONFIG_AIC79XX_DEBUG_ENABLE=y
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
CONFIG_SCSI_DPT_I2O=m
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
CONFIG_SCSI_SATA=m
# CONFIG_SCSI_SATA_AHCI is not set
CONFIG_SCSI_SATA_SVW=m
CONFIG_SCSI_ATA_PIIX=m
# CONFIG_SCSI_SATA_MV is not set
CONFIG_SCSI_SATA_NV=m
# CONFIG_SCSI_PDC_ADMA is not set
# CONFIG_SCSI_SATA_QSTOR is not set
CONFIG_SCSI_SATA_PROMISE=m
CONFIG_SCSI_SATA_SX4=m
CONFIG_SCSI_SATA_SIL=m
# CONFIG_SCSI_SATA_SIL24 is not set
CONFIG_SCSI_SATA_SIS=m
# CONFIG_SCSI_SATA_ULI is not set
CONFIG_SCSI_SATA_VIA=m
CONFIG_SCSI_SATA_VITESSE=m
CONFIG_SCSI_BUSLOGIC=m
# CONFIG_SCSI_OMIT_FLASHPOINT is not set
CONFIG_SCSI_DMX3191D=m
CONFIG_SCSI_EATA=m
CONFIG_SCSI_EATA_TAGGED_QUEUE=y
CONFIG_SCSI_EATA_LINKED_COMMANDS=y
CONFIG_SCSI_EATA_MAX_TAGS=16
CONFIG_SCSI_FUTURE_DOMAIN=m
CONFIG_SCSI_GDTH=m
CONFIG_SCSI_IPS=m
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
CONFIG_SCSI_IPR=m
# CONFIG_SCSI_IPR_TRACE is not set
# CONFIG_SCSI_IPR_DUMP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
CONFIG_SCSI_QLOGIC_1280=m
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_LPFC is not set
CONFIG_SCSI_DC395x=m
CONFIG_SCSI_DC390T=m
CONFIG_SCSI_NSP32=m
CONFIG_SCSI_DEBUG=m
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
# CONFIG_MD_RAID10 is not set
CONFIG_MD_RAID5=m
CONFIG_MD_RAID6=m
CONFIG_MD_MULTIPATH=m
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m
# CONFIG_DM_MULTIPATH is not set
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_SPI is not set
# CONFIG_FUSION_FC is not set
# CONFIG_FUSION_SAS is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
CONFIG_I2O=m
CONFIG_I2O_LCT_NOTIFY_ON_CHANGES=y
CONFIG_I2O_EXT_ADAPTEC=y
CONFIG_I2O_EXT_ADAPTEC_DMA64=y
CONFIG_I2O_CONFIG=m
CONFIG_I2O_CONFIG_OLD_IOCTL=y
# CONFIG_I2O_BUS is not set
CONFIG_I2O_BLOCK=m
CONFIG_I2O_SCSI=m
CONFIG_I2O_PROC=m
#
# Network device support
#
CONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# PHY device support
#
# CONFIG_PHYLIB is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
# CONFIG_CASSINI is not set
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
#
# Tulip family network device support
#
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
# CONFIG_ULI526X is not set
CONFIG_HP100=m
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
# CONFIG_AMD8111E_NAPI is not set
CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_ADAPTEC_STARFIRE_NAPI is not set
CONFIG_B44=m
CONFIG_FORCEDETH=m
# CONFIG_DGRS is not set
CONFIG_EEPRO100=m
CONFIG_E100=m
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
CONFIG_8139TOO_PIO=y
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_TLAN=m
CONFIG_VIA_RHINE=m
# CONFIG_VIA_RHINE_MMIO is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
CONFIG_DL2K=m
CONFIG_E1000=m
CONFIG_E1000_NAPI=y
# CONFIG_E1000_DISABLE_PACKET_SPLIT is not set
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=m
# CONFIG_R8169_NAPI is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
CONFIG_SK98LIN=m
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=m
# CONFIG_BNX2 is not set
#
# Ethernet (10000 Mbit)
#
# CONFIG_CHELSIO_T1 is not set
CONFIG_IXGB=m
# CONFIG_IXGB_NAPI is not set
CONFIG_S2IO=m
# CONFIG_S2IO_NAPI is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
CONFIG_SHAPER=m
CONFIG_NETCONSOLE=m
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_TSDEV=m
CONFIG_INPUT_TSDEV_SCREEN_X=240
CONFIG_INPUT_TSDEV_SCREEN_Y=320
CONFIG_INPUT_EVDEV=m
CONFIG_INPUT_EVBUG=m
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_VSXXXAA=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=y
CONFIG_JOYSTICK_IFORCE_232=y
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
# CONFIG_JOYSTICK_TWIDJOY is not set
# CONFIG_JOYSTICK_JOYDUMP is not set
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_GUNZE=m
# CONFIG_TOUCHSCREEN_ELO is not set
# CONFIG_TOUCHSCREEN_MTOUCH is not set
# CONFIG_TOUCHSCREEN_MK712 is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_WISTRON_BTNS is not set
CONFIG_INPUT_UINPUT=m
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_NONSTANDARD=y
# CONFIG_COMPUTONE is not set
CONFIG_ROCKETPORT=m
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
# CONFIG_DIGIEPCA is not set
# CONFIG_MOXA_INTELLIO is not set
CONFIG_MOXA_SMARTIO=m
# CONFIG_ISI is not set
CONFIG_SYNCLINK=m
CONFIG_SYNCLINKMP=m
# CONFIG_SYNCLINK_GT is not set
CONFIG_N_HDLC=m
# CONFIG_SPECIALIX is not set
# CONFIG_SX is not set
CONFIG_STALDRV=y
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
# CONFIG_SERIAL_8250_ACPI is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_RSA=y
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
#
# IPMI
#
CONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
#
# Watchdog Cards
#
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_ACQUIRE_WDT=m
CONFIG_ADVANTECH_WDT=m
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
CONFIG_SC520_WDT=m
CONFIG_EUROTECH_WDT=m
CONFIG_IB700_WDT=m
# CONFIG_IBMASR is not set
CONFIG_WAFER_WDT=m
# CONFIG_I6300ESB_WDT is not set
CONFIG_I8XX_TCO=m
CONFIG_SC1200_WDT=m
CONFIG_60XX_WDT=m
# CONFIG_SBC8360_WDT is not set
CONFIG_CPU5_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83877F_WDT=m
# CONFIG_W83977F_WDT is not set
CONFIG_MACHZ_WDT=m
# CONFIG_SBC_EPX_C3_WATCHDOG is not set
#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m
CONFIG_WDT_501_PCI=y
#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_HW_RANDOM=m
CONFIG_NVRAM=m
CONFIG_RTC=m
CONFIG_GEN_RTC=m
CONFIG_GEN_RTC_X=y
CONFIG_DTLK=m
CONFIG_R3964=m
CONFIG_APPLICOM=m
CONFIG_SONYPI=m
#
# Ftape, the floppy tape device driver
#
CONFIG_AGP=m
CONFIG_AGP_ALI=m
CONFIG_AGP_ATI=m
CONFIG_AGP_AMD=m
CONFIG_AGP_AMD64=m
CONFIG_AGP_INTEL=m
CONFIG_AGP_NVIDIA=m
CONFIG_AGP_SIS=m
CONFIG_AGP_SWORKS=m
CONFIG_AGP_VIA=m
CONFIG_AGP_EFFICEON=m
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
CONFIG_DRM_I810=m
# CONFIG_DRM_I830 is not set
# CONFIG_DRM_I915 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_MWAVE is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
CONFIG_HANGCHECK_TIMER=m
#
# TPM devices
#
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_CHARDEV=m
#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m
# CONFIG_I2C_ALGOPCA is not set
#
# I2C Hardware Bus support
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
# CONFIG_I2C_AMD756_S4882 is not set
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_I810=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_ISA=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_PROSAVAGE=m
CONFIG_I2C_SAVAGE4=m
CONFIG_SCx200_ACB=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
# CONFIG_I2C_STUB is not set
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
CONFIG_I2C_VOODOO3=m
# CONFIG_I2C_PCA_ISA is not set
#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
CONFIG_SENSORS_EEPROM=m
CONFIG_SENSORS_PCF8574=m
# CONFIG_SENSORS_PCA9539 is not set
CONFIG_SENSORS_PCF8591=m
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_RTC_X1205_I2C is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set
#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
#
# Hardware Monitoring support
#
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
# CONFIG_SENSORS_ADM1026 is not set
CONFIG_SENSORS_ADM1031=m
# CONFIG_SENSORS_ADM9240 is not set
CONFIG_SENSORS_ASB100=m
# CONFIG_SENSORS_ATXP1 is not set
CONFIG_SENSORS_DS1621=m
# CONFIG_SENSORS_F71805F is not set
CONFIG_SENSORS_FSCHER=m
# CONFIG_SENSORS_FSCPOS is not set
CONFIG_SENSORS_GL518SM=m
# CONFIG_SENSORS_GL520SM is not set
CONFIG_SENSORS_IT87=m
# CONFIG_SENSORS_LM63 is not set
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
# CONFIG_SENSORS_LM87 is not set
CONFIG_SENSORS_LM90=m
# CONFIG_SENSORS_LM92 is not set
CONFIG_SENSORS_MAX1619=m
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
CONFIG_SENSORS_VIA686A=m
# CONFIG_SENSORS_VT8231 is not set
CONFIG_SENSORS_W83781D=m
# CONFIG_SENSORS_W83792D is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia Capabilities Port drivers
#
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_CFB_FILLRECT=m
CONFIG_FB_CFB_COPYAREA=m
CONFIG_FB_CFB_IMAGEBLIT=m
# CONFIG_FB_MACMODES is not set
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y
CONFIG_FB_CIRRUS=m
CONFIG_FB_PM2=m
CONFIG_FB_PM2_FIFO_DISCONNECT=y
CONFIG_FB_CYBER2000=m
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
CONFIG_FB_VGA16=m
# CONFIG_FB_VESA is not set
CONFIG_VIDEO_SELECT=y
CONFIG_FB_HGA=m
# CONFIG_FB_HGA_ACCEL is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
CONFIG_FB_RIVA_DEBUG=y
CONFIG_FB_I810=m
# CONFIG_FB_I810_GTF is not set
# CONFIG_FB_INTEL is not set
CONFIG_FB_MATROX=m
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
# CONFIG_FB_MATROX_G is not set
CONFIG_FB_MATROX_I2C=m
CONFIG_FB_MATROX_MULTIHEAD=y
# CONFIG_FB_RADEON_OLD is not set
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
# CONFIG_FB_RADEON_DEBUG is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY=m
CONFIG_FB_ATY_CT=y
# CONFIG_FB_ATY_GENERIC_LCD is not set
CONFIG_FB_ATY_GX=y
# CONFIG_FB_SAVAGE is not set
CONFIG_FB_SIS=m
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
CONFIG_FB_3DFX=m
# CONFIG_FB_3DFX_ACCEL is not set
CONFIG_FB_VOODOO1=m
# CONFIG_FB_CYBLA is not set
CONFIG_FB_TRIDENT=m
# CONFIG_FB_TRIDENT_ACCEL is not set
# CONFIG_FB_GEODE is not set
CONFIG_FB_VIRTUAL=m
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=m
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
#
# Logo configuration
#
# CONFIG_LOGO is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
#
# Sound
#
# CONFIG_SOUND is not set
#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_BANDWIDTH=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set
#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_SPLIT_ISO=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
# CONFIG_USB_ISP116X_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_BIG_ENDIAN is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_SL811_HCD is not set
#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#
#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
# CONFIG_USB_STORAGE_USBAT is not set
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_LIBUSUAL is not set
#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
# CONFIG_USB_HIDINPUT_POWERBOOK is not set
# CONFIG_HID_FF is not set
CONFIG_USB_HIDDEV=y
#
# USB HID Boot Protocol drivers
#
CONFIG_USB_KBD=m
CONFIG_USB_MOUSE=m
CONFIG_USB_AIPTEK=m
CONFIG_USB_WACOM=m
# CONFIG_USB_ACECAD is not set
CONFIG_USB_KBTAB=m
CONFIG_USB_POWERMATE=m
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_ITMTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_YEALINK is not set
CONFIG_USB_XPAD=m
CONFIG_USB_ATI_REMOTE=m
# CONFIG_USB_ATI_REMOTE2 is not set
# CONFIG_USB_KEYSPAN_REMOTE is not set
# CONFIG_USB_APPLETOUCH is not set
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
#
# Video4Linux support is needed for USB Multimedia device support
#
#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=m
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_RNDIS_HOST is not set
# CONFIG_USB_NET_CDC_SUBSET is not set
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_MON=y
#
# USB port drivers
#
#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRPRIME is not set
# CONFIG_USB_SERIAL_ANYDATA is not set
CONFIG_USB_SERIAL_BELKIN=m
# CONFIG_USB_SERIAL_WHITEHEAT is not set
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
# CONFIG_USB_SERIAL_CP2101 is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
# CONFIG_USB_SERIAL_KEYSPAN_MPR is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA18X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QW is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QI is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49WLC is not set
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_PL2303=m
# CONFIG_USB_SERIAL_HP4X is not set
CONFIG_USB_SERIAL_SAFE=m
# CONFIG_USB_SERIAL_SAFE_PADDED is not set
# CONFIG_USB_SERIAL_TI is not set
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_EZUSB=y
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
CONFIG_USB_AUERSWALD=m
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_LED=m
CONFIG_USB_CYTHERM=m
# CONFIG_USB_PHIDGETKIT is not set
CONFIG_USB_PHIDGETSERVO=m
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
CONFIG_USB_TEST=m
#
# USB DSL modem support
#
#
# USB Gadget Support
#
CONFIG_USB_GADGET=m
# CONFIG_USB_GADGET_DEBUG_FILES is not set
CONFIG_USB_GADGET_SELECTED=y
CONFIG_USB_GADGET_NET2280=y
CONFIG_USB_NET2280=m
# CONFIG_USB_GADGET_PXA2XX is not set
# CONFIG_USB_GADGET_GOKU is not set
# CONFIG_USB_GADGET_LH7A40X is not set
# CONFIG_USB_GADGET_OMAP is not set
# CONFIG_USB_GADGET_DUMMY_HCD is not set
CONFIG_USB_GADGET_DUALSPEED=y
CONFIG_USB_ZERO=m
CONFIG_USB_ETH=m
CONFIG_USB_ETH_RNDIS=y
CONFIG_USB_GADGETFS=m
CONFIG_USB_FILE_STORAGE=m
# CONFIG_USB_FILE_STORAGE_TEST is not set
CONFIG_USB_G_SERIAL=m
#
# MMC/SD Card support
#
# CONFIG_MMC is not set
#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set
#
# EDAC - error detection and reporting (RAS) (EXPERIMENTAL)
#
# CONFIG_EDAC is not set
#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_REISERFS_FS_XATTR is not set
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
# CONFIG_JFS_SECURITY is not set
# CONFIG_JFS_DEBUG is not set
CONFIG_JFS_STATISTICS=y
CONFIG_FS_POSIX_ACL=y
CONFIG_XFS_FS=m
CONFIG_XFS_EXPORT=y
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_SECURITY is not set
# CONFIG_XFS_POSIX_ACL is not set
# CONFIG_XFS_RT is not set
# CONFIG_OCFS2_FS is not set
CONFIG_MINIX_FS=m
CONFIG_ROMFS_FS=m
CONFIG_INOTIFY=y
CONFIG_QUOTA=y
CONFIG_QFMT_V1=m
CONFIG_QFMT_V2=m
CONFIG_QUOTACTL=y
CONFIG_DNOTIFY=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=m
# CONFIG_FUSE_FS is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=m
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
# CONFIG_RELAYFS_FS is not set
# CONFIG_CONFIGFS_FS is not set
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
CONFIG_CRAMFS=y
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V3_ACL is not set
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_RPCSEC_GSS_KRB5=y
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
# CONFIG_SMB_NLS_DEFAULT is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
CONFIG_NCP_FS=m
CONFIG_NCPFS_PACKET_SIGNING=y
CONFIG_NCPFS_IOCTL_LOCKING=y
CONFIG_NCPFS_STRONG=y
CONFIG_NCPFS_NFS_NS=y
CONFIG_NCPFS_OS2_NS=y
# CONFIG_NCPFS_SMALLDOS is not set
CONFIG_NCPFS_NLS=y
CONFIG_NCPFS_EXTRAS=y
CONFIG_CODA_FS=m
# CONFIG_CODA_FS_OLD_API is not set
CONFIG_AFS_FS=m
CONFIG_RXRPC=m
# CONFIG_9P_FS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
CONFIG_ACORN_PARTITION_CUMANA=y
# CONFIG_ACORN_PARTITION_EESOX is not set
CONFIG_ACORN_PARTITION_ICS=y
# CONFIG_ACORN_PARTITION_ADFS is not set
# CONFIG_ACORN_PARTITION_POWERTEC is not set
CONFIG_ACORN_PARTITION_RISCIX=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
# CONFIG_KARMA_PARTITION is not set
# CONFIG_EFI_PARTITION is not set
#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="cp437"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
#
# Instrumentation Support
#
CONFIG_PROFILING=y
CONFIG_OPROFILE=m
# CONFIG_KPROBES is not set
#
# Kernel hacking
#
# CONFIG_PRINTK_TIME is not set
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_KERNEL=y
CONFIG_LOG_BUF_SHIFT=14
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_DEBUG_SLAB is not set
CONFIG_DEBUG_MUTEXES=y
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_HIGHMEM is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_DEBUG_VM is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_FORCED_INLINING=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_DEBUG_RODATA is not set
# CONFIG_4KSTACKS is not set
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
#
# Security options
#
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
# CONFIG_SECURITY_NETWORK is not set
CONFIG_SECURITY_CAPABILITIES=m
CONFIG_SECURITY_ROOTPLUG=m
# CONFIG_SECURITY_SECLVL is not set
#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_TGR192 is not set
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
# CONFIG_CRYPTO_AES is not set
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_TEST=m
#
# Hardware crypto devices
#
# CONFIG_CRYPTO_DEV_PADLOCK is not set
#
# Library routines
#
CONFIG_CRC_CCITT=m
# CONFIG_CRC16 is not set
CONFIG_CRC32=m
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_X86_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_KTIME_SCALAR=y
[-- Attachment #4: sysctl.conf --]
[-- Type: text/plain, Size: 495 bytes --]
#
# /etc/sysctl.conf - Configuration file for setting system variables
# See sysctl.conf (5) for information.
#
#kernel.domainname = example.com
#net/ipv4/icmp_echo_ignore_broadcasts=1
kernel/shmmax=536870912
kernel.core_pattern=core_%e.%p
net/core/wmem_max=16777216
net/core/rmem_max=16777216
net/ipv4/tcp_rmem=4096 87380 16777216
net/ipv4/tcp_wmem=4096 65536 16777216
net.ipv4.tcp_no_metrics_save = 1
net/ipv4/tcp_fin_timeout = 20
net.core.netdev_max_backlog = 2500
vm.min_free_kbytes = 25000
[-- Attachment #5: netstat.out --]
[-- Type: text/plain, Size: 2938 bytes --]
Ip:
76182271 total packets received
51800 with invalid addresses
0 forwarded
2011 with unknown protocol
0 incoming packets discarded
76128460 incoming packets delivered
51947861 requests sent out
Icmp:
15439 ICMP messages received
7604 input ICMP message failed.
ICMP input histogram:
destination unreachable: 15431
echo requests: 8
15632 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 15624
echo replies: 8
Tcp:
45535 active connections openings
51487 passive connection openings
7819 failed connection attempts
8576 connection resets received
83 connections established
75757261 segments received
51600510 segments send out
33330 segments retransmited
0 bad segments received.
6823 resets sent
Udp:
332009 packets received
251 packets to unknown port received.
0 packet receive errors
331719 packets sent
TcpExt:
2 resets received for embryonic SYN_RECV sockets
26971 packets pruned from receive queue because of socket buffer overrun
33592 TCP sockets finished time wait in fast timer
9 time wait sockets recycled by time stamp
146 packets rejects in established connections because of timestamp
4956489 delayed acks sent
1332 delayed acks further delayed because of locked socket
Quick ack mode was activated 24409 times
7 times the listen queue of a socket overflowed
7 SYNs to LISTEN sockets ignored
105137 packets directly queued to recvmsg prequeue.
57762555 of bytes directly received from backlog
25177512 of bytes directly received from prequeue
42421102 packet headers predicted
94081 packets header predicted and directly queued to user
3055918 acknowledgments not containing data received
45986456 predicted acknowledgments
67 times recovered from packet loss due to fast retransmit
7789 times recovered from packet loss due to SACK data
Detected reordering 6 times using FACK
Detected reordering 27 times using SACK
Detected reordering 1100 times using time stamp
4781 congestion windows fully recovered
26640 congestion windows partially recovered using Hoe heuristic
TCPDSACKUndo: 14
3680 congestion windows recovered after partial ack
8806 TCP data loss events
36 timeouts after SACK recovery
1 timeouts in loss state
9453 fast retransmits
8727 forward retransmits
438 retransmits in slow start
13538 other TCP timeouts
TCPRenoRecoveryFail: 12
60 sack retransmits failed
6459303 packets collapsed in receive queue due to low socket buffer
18284 DSACKs sent for old packets
12579 DSACKs received
92 DSACKs for out of order packets received
1099 connections reset due to unexpected data
50 connections reset due to early user close
242 connections aborted due to timeout
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-16 16:01 Network performance degradation from 2.6.11.12 to 2.6.16.20 Harry Edmon
@ 2006-06-17 22:35 ` Andrew Morton
2006-06-17 23:23 ` Harry Edmon
0 siblings, 1 reply; 59+ messages in thread
From: Andrew Morton @ 2006-06-17 22:35 UTC (permalink / raw)
To: Harry Edmon; +Cc: linux-kernel, netdev
On Fri, 16 Jun 2006 09:01:23 -0700
Harry Edmon <harry@atmos.washington.edu> wrote:
> I have a system with a strange network performance degradation from
> 2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
> The system is has Dual single core Xeons with hyperthreading on. The
> application is the LDM system from UCAR/Unidata
> (http://www.unidata.ucar.edu/software/ldm). This system requests
> weather data from a variety of systems using RPC calls over a reserved
> TCP port (388), puts them into a memory mapped queue file, and then
> sends the data out to a variety of downstream requesting systems, again
> using RPC calls. When the load is heavy, the 2.6.16.20 kernel falls way
> behind with the data ingestion. The 2.6.11.12 kernel does not. I have
> tried an experiment with a 2.6.17-rc6 system where it just does the
> ingestion, and not the downstream distribution, and it is able to keep
> up. I would really appreciate any pointers as to where the problem may
> be and how to diagnose it. I have attached the config files from both
> kernels and the sysctl.conf file I am using. I have also included the
> output from "netstat -s" on the 2.6.16.20 system during a time when it
> was having problems.
>
(added netdev)
A quick grep indicates that it isn't using TCP_NODELAY - we've had problems
with that in the past.
Perhaps a tcpdump of the net traffic will help to determine what's going on.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-17 22:35 ` Andrew Morton
@ 2006-06-17 23:23 ` Harry Edmon
2006-06-17 23:56 ` Andrew Morton
2006-06-19 14:47 ` Jesper Dangaard Brouer
0 siblings, 2 replies; 59+ messages in thread
From: Harry Edmon @ 2006-06-17 23:23 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, netdev
I assume you are talking about using TCP_NODELAY as a socket option within the
LDM software. I could give that a try.
There is a lot of traffic on this node, on the order of 2000 packets in and out
per second, so the tcpdump output will grow pretty fast. How long a tcpdump
would be useful, and what options would you suggest?
I should also note that my network interfaces are Intel, using the latest e1000
driver.
Andrew Morton wrote:
> On Fri, 16 Jun 2006 09:01:23 -0700
> Harry Edmon <harry@atmos.washington.edu> wrote:
>
>> I have a system with a strange network performance degradation from
>> 2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
>> The system is has Dual single core Xeons with hyperthreading on. The
>> application is the LDM system from UCAR/Unidata
>> (http://www.unidata.ucar.edu/software/ldm). This system requests
>> weather data from a variety of systems using RPC calls over a reserved
>> TCP port (388), puts them into a memory mapped queue file, and then
>> sends the data out to a variety of downstream requesting systems, again
>> using RPC calls. When the load is heavy, the 2.6.16.20 kernel falls way
>> behind with the data ingestion. The 2.6.11.12 kernel does not. I have
>> tried an experiment with a 2.6.17-rc6 system where it just does the
>> ingestion, and not the downstream distribution, and it is able to keep
>> up. I would really appreciate any pointers as to where the problem may
>> be and how to diagnose it. I have attached the config files from both
>> kernels and the sysctl.conf file I am using. I have also included the
>> output from "netstat -s" on the 2.6.16.20 system during a time when it
>> was having problems.
>>
>
> (added netdev)
>
> A quick grep indicates that it isn't using TCP_NODELAY - we've had problems
> with that in the past.
>
> Perhaps a tcpdump of the net traffic will help to determine what's going on.
--
Dr. Harry Edmon E-MAIL: harry@atmos.washington.edu
206-543-0547 harry@u.washington.edu
Dept of Atmospheric Sciences FAX: 206-543-0308
University of Washington, Box 351640, Seattle, WA 98195-1640
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-17 23:23 ` Harry Edmon
@ 2006-06-17 23:56 ` Andrew Morton
2006-06-18 3:16 ` Stephen Hemminger
2006-06-19 14:47 ` Jesper Dangaard Brouer
1 sibling, 1 reply; 59+ messages in thread
From: Andrew Morton @ 2006-06-17 23:56 UTC (permalink / raw)
To: Harry Edmon; +Cc: linux-kernel, netdev
On Sat, 17 Jun 2006 16:23:34 -0700
Harry Edmon <harry@atmos.washington.edu> wrote:
> Andrew Morton wrote:
> > On Fri, 16 Jun 2006 09:01:23 -0700
> > Harry Edmon <harry@atmos.washington.edu> wrote:
> >
> >> I have a system with a strange network performance degradation from
> >> 2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
> >> The system is has Dual single core Xeons with hyperthreading on. The
> >> application is the LDM system from UCAR/Unidata
> >> (http://www.unidata.ucar.edu/software/ldm). This system requests
> >> weather data from a variety of systems using RPC calls over a reserved
> >> TCP port (388), puts them into a memory mapped queue file, and then
> >> sends the data out to a variety of downstream requesting systems, again
> >> using RPC calls. When the load is heavy, the 2.6.16.20 kernel falls way
> >> behind with the data ingestion. The 2.6.11.12 kernel does not. I have
> >> tried an experiment with a 2.6.17-rc6 system where it just does the
> >> ingestion, and not the downstream distribution, and it is able to keep
> >> up. I would really appreciate any pointers as to where the problem may
> >> be and how to diagnose it. I have attached the config files from both
> >> kernels and the sysctl.conf file I am using. I have also included the
> >> output from "netstat -s" on the 2.6.16.20 system during a time when it
> >> was having problems.
> >>
> >
> > (added netdev)
> >
> > A quick grep indicates that it isn't using TCP_NODELAY - we've had problems
> > with that in the past.
> >
> > Perhaps a tcpdump of the net traffic will help to determine what's going on.
>
[ edit, edit - please don't top-post ]
> I assume you are talking about using TCP_NODELAY as a socket option within the
> LDM software. I could give that a try.
The use of TCP_NODELAY caused problems with the JVM debugger. I'm not
suggesting that enabling it will fix anything here.
>
> There is a lot of traffic on this node, on the order of 2000 packets in and out
> per second, so the tcpdump output will grow pretty fast. How long a tcpdump
> would be useful, and what options would you suggest?
I don't know, frankly - first one needs to develop some sort of theory,
then use the diagnostic tools to prove or disprove that theory. And I
don't have a theory.
I guess a simple one-second bare `tcpdump -i eth0' would be a starting
point. Perhaps compare the output of that with the output from a
correctly-operating kernel, see if anything suggests itself. That might
also give us something which the networking developers can use.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-17 23:56 ` Andrew Morton
@ 2006-06-18 3:16 ` Stephen Hemminger
2006-06-18 23:23 ` Harry Edmon
2006-06-19 13:54 ` Harry Edmon
0 siblings, 2 replies; 59+ messages in thread
From: Stephen Hemminger @ 2006-06-18 3:16 UTC (permalink / raw)
To: Andrew Morton; +Cc: Harry Edmon, linux-kernel, netdev
Andrew Morton wrote:
> On Sat, 17 Jun 2006 16:23:34 -0700
> Harry Edmon <harry@atmos.washington.edu> wrote:
>
>
>> Andrew Morton wrote:
>>
>>> On Fri, 16 Jun 2006 09:01:23 -0700
>>> Harry Edmon <harry@atmos.washington.edu> wrote:
>>>
>>>
>>>> I have a system with a strange network performance degradation from
>>>> 2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
>>>> The system is has Dual single core Xeons with hyperthreading on. The
>>>> application is the LDM system from UCAR/Unidata
>>>> (http://www.unidata.ucar.edu/software/ldm). This system requests
>>>> weather data from a variety of systems using RPC calls over a reserved
>>>> TCP port (388), puts them into a memory mapped queue file, and then
>>>> sends the data out to a variety of downstream requesting systems, again
>>>> using RPC calls. When the load is heavy, the 2.6.16.20 kernel falls way
>>>> behind with the data ingestion. The 2.6.11.12 kernel does not. I have
>>>> tried an experiment with a 2.6.17-rc6 system where it just does the
>>>> ingestion, and not the downstream distribution, and it is able to keep
>>>> up. I would really appreciate any pointers as to where the problem may
>>>> be and how to diagnose it. I have attached the config files from both
>>>> kernels and the sysctl.conf file I am using. I have also included the
>>>> output from "netstat -s" on the 2.6.16.20 system during a time when it
>>>> was having problems.
>>>>
>>>>
>>> (added netdev)
>>>
>>> A quick grep indicates that it isn't using TCP_NODELAY - we've had problems
>>> with that in the past.
>>>
>>> Perhaps a tcpdump of the net traffic will help to determine what's going on.
>>>
>
> [ edit, edit - please don't top-post ]
>
>
>> I assume you are talking about using TCP_NODELAY as a socket option within the
>> LDM software. I could give that a try.
>>
>
> The use of TCP_NODELAY caused problems with the JVM debugger. I'm not
> suggesting that enabling it will fix anything here.
>
>
>> There is a lot of traffic on this node, on the order of 2000 packets in and out
>> per second, so the tcpdump output will grow pretty fast. How long a tcpdump
>> would be useful, and what options would you suggest?
>>
>
> I don't know, frankly - first one needs to develop some sort of theory,
> then use the diagnostic tools to prove or disprove that theory. And I
> don't have a theory.
>
> I guess a simple one-second bare `tcpdump -i eth0' would be a starting
> point. Perhaps compare the output of that with the output from a
> correctly-operating kernel, see if anything suggests itself. That might
> also give us something which the networking developers can use.
>
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Does this fix it?
# sysctl -w net.ipv4.tcp_abc=0
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-18 3:16 ` Stephen Hemminger
@ 2006-06-18 23:23 ` Harry Edmon
2006-06-19 13:54 ` Harry Edmon
1 sibling, 0 replies; 59+ messages in thread
From: Harry Edmon @ 2006-06-18 23:23 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Andrew Morton, linux-kernel, netdev
Stephen Hemminger wrote:
>>
> Does this fix it?
> # sysctl -w net.ipv4.tcp_abc=0
Thanks for the suggestion. I will give it a try later tonight. Also Andrew -
sorry for the incorrect placement of my follow-up comments. I do appreciate
everyone's help in figuring this out.
--
Dr. Harry Edmon E-MAIL: harry@atmos.washington.edu
206-543-0547 harry@u.washington.edu
Dept of Atmospheric Sciences FAX: 206-543-0308
University of Washington, Box 351640, Seattle, WA 98195-1640
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-18 3:16 ` Stephen Hemminger
2006-06-18 23:23 ` Harry Edmon
@ 2006-06-19 13:54 ` Harry Edmon
2006-06-20 2:11 ` Herbert Xu
1 sibling, 1 reply; 59+ messages in thread
From: Harry Edmon @ 2006-06-19 13:54 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Andrew Morton, linux-kernel, netdev
Stephen Hemminger wrote:
> Does this fix it?
> # sysctl -w net.ipv4.tcp_abc=0
That did not help. I have 1 minute outputs from tcpdump under both 2.6.11.12
and 2.6.16.20. You will see a large size difference between the files. Since
the 2.6.11.12 one is 2 MBytes, I thought I would post them via the web instead
of via attachments. Look at:
http://www.atmos.washington.edu/~harry/linux/2.6.11.12.out.1min
http://www.atmos.washington.edu/~harry/linux/2.6.16.20.out.1min
And again, thank to all of you for looking into this.
--
Dr. Harry Edmon E-MAIL: harry@atmos.washington.edu
206-543-0547 harry@u.washington.edu
Dept of Atmospheric Sciences FAX: 206-543-0308
University of Washington, Box 351640, Seattle, WA 98195-1640
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-17 23:23 ` Harry Edmon
2006-06-17 23:56 ` Andrew Morton
@ 2006-06-19 14:47 ` Jesper Dangaard Brouer
2006-06-19 15:24 ` Andi Kleen
2006-06-19 16:40 ` Harry Edmon
1 sibling, 2 replies; 59+ messages in thread
From: Jesper Dangaard Brouer @ 2006-06-19 14:47 UTC (permalink / raw)
To: Harry Edmon; +Cc: linux-kernel, netdev
>> Harry Edmon <harry@atmos.washington.edu> wrote:
>>
>>> I have a system with a strange network performance degradation from
>>> 2.6.11.12 to most recent kernels including 2.6.16.20 and 2.6.17-rc6.
>>> The system is has Dual single core Xeons with hyperthreading on.
<cut>
Hi Harry
Can you check which "high-res timesource" you are using?
In the kernel log look for:
kernel: Using tsc for high-res timesource
kernel: Using pmtmr for high-res timesource
I have experinced some network performance degradation when using the
"pmtmr" timesource, on a Opteron AMD system. It seems that the default
timesource change between 2.6.15 to 2.6.16.
If you use "pmtmr" try to reboot with kernel option "clock=tsc".
On my Opteron AMD system i normally can route 400 kpps, but with
timesource "pmtmr" i could only route around 83 kpps. (I found the timer
to be the issue by using oprofile).
Cheers,
Jesper Brouer
--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 14:47 ` Jesper Dangaard Brouer
@ 2006-06-19 15:24 ` Andi Kleen
2006-06-19 17:34 ` Chris Friesen
` (3 more replies)
2006-06-19 16:40 ` Harry Edmon
1 sibling, 4 replies; 59+ messages in thread
From: Andi Kleen @ 2006-06-19 15:24 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: Harry Edmon, linux-kernel, netdev
> If you use "pmtmr" try to reboot with kernel option "clock=tsc".
That's dangerous advice - when the system choses not to use
TSC it often has a reason.
>
> On my Opteron AMD system i normally can route 400 kpps, but with
> timesource "pmtmr" i could only route around 83 kpps. (I found the timer
> to be the issue by using oprofile).
Unless you're using packet sniffing or any other application
that requests time stamps on a socket then the timer shouldn't
make much difference. Incoming packets are only time stamped
when someone asks for the timestamps.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 15:24 ` Andi Kleen
@ 2006-06-19 17:34 ` Chris Friesen
2006-06-19 20:39 ` Andi Kleen
2006-06-19 18:24 ` Jesper Dangaard Brouer
` (2 subsequent siblings)
3 siblings, 1 reply; 59+ messages in thread
From: Chris Friesen @ 2006-06-19 17:34 UTC (permalink / raw)
To: Andi Kleen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
Andi Kleen wrote:
> Incoming packets are only time stamped
> when someone asks for the timestamps.
Doesn't that add scheduling latency to the timestamps? Or is is a flag
that gets set to trigger timestamping at packet arrival?
Chris
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 17:34 ` Chris Friesen
@ 2006-06-19 20:39 ` Andi Kleen
0 siblings, 0 replies; 59+ messages in thread
From: Andi Kleen @ 2006-06-19 20:39 UTC (permalink / raw)
To: Chris Friesen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
On Monday 19 June 2006 19:34, Chris Friesen wrote:
> Andi Kleen wrote:
> > Incoming packets are only time stamped
> > when someone asks for the timestamps.
>
> Doesn't that add scheduling latency to the timestamps? Or is is a flag
> that gets set to trigger timestamping at packet arrival?
It's a flag (or more precise a global counter)
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 15:24 ` Andi Kleen
2006-06-19 17:34 ` Chris Friesen
@ 2006-06-19 18:24 ` Jesper Dangaard Brouer
2006-06-25 21:51 ` Harry Edmon
2006-06-25 22:22 ` Willy Tarreau
2006-09-16 12:08 ` Vladimir B. Savkin
3 siblings, 1 reply; 59+ messages in thread
From: Jesper Dangaard Brouer @ 2006-06-19 18:24 UTC (permalink / raw)
To: Andi Kleen; +Cc: Harry Edmon, linux-kernel, netdev
On Mon, 19 Jun 2006, Andi Kleen wrote:
>> If you use "pmtmr" try to reboot with kernel option "clock=tsc".
>
> That's dangerous advice - when the system choses not to use
> TSC it often has a reason.
Sorry, it was not a general advice, just something to try out. It really
solved my network performance issue...
>> On my Opteron AMD system i normally can route 400 kpps, but with
>> timesource "pmtmr" i could only route around 83 kpps. (I found the timer
>> to be the issue by using oprofile).
>
> Unless you're using packet sniffing or any other application
> that requests time stamps on a socket then the timer shouldn't
> make much difference. Incoming packets are only time stamped
> when someone asks for the timestamps.
I do not know what caused the issue on my machine, but I can look into it
if you like to know?
I do have VLAN interfaces on the machine and it seems that eth1 runs in
PROMISC mode (eth1.xxx does not). Could it be caused by that?
Hilsen
Jesper Brouer
--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 18:24 ` Jesper Dangaard Brouer
@ 2006-06-25 21:51 ` Harry Edmon
2006-06-26 4:20 ` Bill Fink
0 siblings, 1 reply; 59+ messages in thread
From: Harry Edmon @ 2006-06-25 21:51 UTC (permalink / raw)
To: linux-kernel, netdev
I understand the saying "beggars can't be choosers", but I have heard nothing on
this issue since June 19th. Does anyone have any ideas on what is going on? Is
there more information I can collect that would help diagnose this problem? And
again, thanks for any and all help!
--
Dr. Harry Edmon E-MAIL: harry@atmos.washington.edu
206-543-0547 harry@u.washington.edu
Dept of Atmospheric Sciences FAX: 206-543-0308
University of Washington, Box 351640, Seattle, WA 98195-1640
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-25 21:51 ` Harry Edmon
@ 2006-06-26 4:20 ` Bill Fink
0 siblings, 0 replies; 59+ messages in thread
From: Bill Fink @ 2006-06-26 4:20 UTC (permalink / raw)
To: Harry Edmon; +Cc: linux-kernel, netdev
On Sun, 25 Jun 2006, Harry Edmon wrote:
> I understand the saying "beggars can't be choosers", but I have heard nothing on
> this issue since June 19th. Does anyone have any ideas on what is going on? Is
> there more information I can collect that would help diagnose this problem? And
> again, thanks for any and all help!
Harry,
I'd suggest checking all the ethtool configuration settings
(ethtool -a, -c, -g, -k) and statistics (ethtool -S) for both
the working and problematic kernels, and then comparing them
to see if anything jumps out at you. Also compare ifconfig
settings and dmesg output. Check /proc/interrupts to see if
there is any difference with the interrupt routing. Check
sysctl.conf and rc.local for any special system configuration
or device settings that might differ between the systems.
The one thing that has caused me a lot of network performance
issues on e1000 is having TSO enabled, so if that is enabled
(check with ethtool -k), then I'd try disabling it to see if
that helps.
-Hope this helps
-Bill
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 15:24 ` Andi Kleen
2006-06-19 17:34 ` Chris Friesen
2006-06-19 18:24 ` Jesper Dangaard Brouer
@ 2006-06-25 22:22 ` Willy Tarreau
2006-06-26 5:23 ` Andi Kleen
2006-09-16 12:08 ` Vladimir B. Savkin
3 siblings, 1 reply; 59+ messages in thread
From: Willy Tarreau @ 2006-06-25 22:22 UTC (permalink / raw)
To: Andi Kleen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
Hi Andi,
On Mon, Jun 19, 2006 at 05:24:31PM +0200, Andi Kleen wrote:
>
> > If you use "pmtmr" try to reboot with kernel option "clock=tsc".
>
> That's dangerous advice - when the system choses not to use
> TSC it often has a reason.
>
> >
> > On my Opteron AMD system i normally can route 400 kpps, but with
> > timesource "pmtmr" i could only route around 83 kpps. (I found the timer
> > to be the issue by using oprofile).
>
> Unless you're using packet sniffing or any other application
> that requests time stamps on a socket then the timer shouldn't
> make much difference. Incoming packets are only time stamped
> when someone asks for the timestamps.
I encountered the same problem on a dual core opteron equipped with a
broadcom NIC (tg3) under 2.4. It could receive 1 Mpps when using TSC
as the clock source, but the time jumped back and forth, so I changed
it to 'notsc', then the performance dropped dramatically to around the
same value as above with one CPU saturated. I suspect that the clock
precision is needed by the tg3 driver to correctly decide to switch to
polling mode, but unfortunately, the performance drop rendered the
solution so much unusable that I finally decided to use it only in
uniprocessor with TSC enabled.
> -Andi
Regards,
Willy
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-25 22:22 ` Willy Tarreau
@ 2006-06-26 5:23 ` Andi Kleen
2006-07-04 11:41 ` Jesper Dangaard Brouer
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-06-26 5:23 UTC (permalink / raw)
To: Willy Tarreau; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
> I encountered the same problem on a dual core opteron equipped with a
> broadcom NIC (tg3) under 2.4. It could receive 1 Mpps when using TSC
> as the clock source, but the time jumped back and forth, so I changed
> it to 'notsc', then the performance dropped dramatically to around the
> same value as above with one CPU saturated. I suspect that the clock
> precision is needed by the tg3 driver to correctly decide to switch to
> polling mode, but unfortunately, the performance drop rendered the
> solution so much unusable that I finally decided to use it only in
> uniprocessor with TSC enabled.
2.6 is more clever at this than 2.4. In particular it does the timestamp
for each packet only when actually needed, which is relativelt rare.
Old experiences do not always apply to new kernels.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-26 5:23 ` Andi Kleen
@ 2006-07-04 11:41 ` Jesper Dangaard Brouer
2006-07-04 11:54 ` Andi Kleen
0 siblings, 1 reply; 59+ messages in thread
From: Jesper Dangaard Brouer @ 2006-07-04 11:41 UTC (permalink / raw)
To: Andi Kleen; +Cc: Willy Tarreau, Harry Edmon, linux-kernel, netdev
On Mon, 26 Jun 2006, Andi Kleen wrote:
>> I encountered the same problem on a dual core opteron equipped with a
>> broadcom NIC (tg3) under 2.4. It could receive 1 Mpps when using TSC
>> as the clock source, but the time jumped back and forth, so I changed
>> it to 'notsc', then the performance dropped dramatically to around the
>> same value as above with one CPU saturated. I suspect that the clock
>> precision is needed by the tg3 driver to correctly decide to switch to
>> polling mode, but unfortunately, the performance drop rendered the
>> solution so much unusable that I finally decided to use it only in
>> uniprocessor with TSC enabled.
>
> 2.6 is more clever at this than 2.4. In particular it does the timestamp
> for each packet only when actually needed, which is relativelt rare.
>
> Old experiences do not always apply to new kernels.
Note, that I experinced this problem on 2.6.
Actually the change happens between kernel version 2.6.15 and 2.6.16. And
is a result of Andi's changes to arch/x86_64/Kconfig and
drivers/acpi/Kconfig, which "allows/activates" the use of the timer on
x86_64.
Cheers,
Jesper Brouer
--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-07-04 11:41 ` Jesper Dangaard Brouer
@ 2006-07-04 11:54 ` Andi Kleen
2006-07-10 10:55 ` Jesper Dangaard Brouer
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-07-04 11:54 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: Willy Tarreau, Harry Edmon, linux-kernel, netdev
On Tuesday 04 July 2006 13:41, Jesper Dangaard Brouer wrote:
>
> On Mon, 26 Jun 2006, Andi Kleen wrote:
>
> >> I encountered the same problem on a dual core opteron equipped with a
> >> broadcom NIC (tg3) under 2.4. It could receive 1 Mpps when using TSC
> >> as the clock source, but the time jumped back and forth, so I changed
> >> it to 'notsc', then the performance dropped dramatically to around the
> >> same value as above with one CPU saturated. I suspect that the clock
> >> precision is needed by the tg3 driver to correctly decide to switch to
> >> polling mode, but unfortunately, the performance drop rendered the
> >> solution so much unusable that I finally decided to use it only in
> >> uniprocessor with TSC enabled.
> >
> > 2.6 is more clever at this than 2.4. In particular it does the timestamp
> > for each packet only when actually needed, which is relativelt rare.
> >
> > Old experiences do not always apply to new kernels.
>
> Note, that I experinced this problem on 2.6.
>
> Actually the change happens between kernel version 2.6.15 and 2.6.16.
The timestamp optimizations are older. Don't remember the exact release,
but earlier 2.6.
> And
> is a result of Andi's changes to arch/x86_64/Kconfig and
> drivers/acpi/Kconfig, which "allows/activates" the use of the timer on
> x86_64.
Not sure what you mean here?
2.6.18 will likely be more aggressive at using the TSC on i386 on
Intel systems where possible, but x86-64 did this already for a long time.
When x86-64 uses non TSC then it's because using the TSC is not safe.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-07-04 11:54 ` Andi Kleen
@ 2006-07-10 10:55 ` Jesper Dangaard Brouer
0 siblings, 0 replies; 59+ messages in thread
From: Jesper Dangaard Brouer @ 2006-07-10 10:55 UTC (permalink / raw)
To: Andi Kleen; +Cc: linux-kernel, netdev
On Tue, 4 Jul 2006, Andi Kleen wrote:
> On Tuesday 04 July 2006 13:41, Jesper Dangaard Brouer wrote:
>>
>> Actually the change happens between kernel version 2.6.15 and 2.6.16.
>
> The timestamp optimizations are older. Don't remember the exact release,
> but earlier 2.6.
What I'm saying is that, with the same Config file, some Kconfig option
changed between 2.6.15 and 2.6.16, that made my system use pmtmr for high-res
timesource instead of TSC.
>> And
>> is a result of Andi's changes to arch/x86_64/Kconfig and
>> drivers/acpi/Kconfig, which "allows/activates" the use of the timer on
>> x86_64.
>
> Not sure what you mean here?
I think, that the changes you made to the files "arch/x86_64/Kconfig" and
"drivers/acpi/Kconfig", caused this change...
commit: e78256b8f3e2850ad55c2d69e1429e6c2607afd3
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.17.y.git;a=commitdiff;h=e78256b8f3e2850ad55c2d69e1429e6c2607afd3
and maybe
commit: 2eb1bdbad89b19c99f8ac1de1492cdabbff6b3d3
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.17.y.git;a=commitdiff;h=2eb1bdbad89b19c99f8ac1de1492cdabbff6b3d3
Hilsen
Jesper Brouer
--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 15:24 ` Andi Kleen
` (2 preceding siblings ...)
2006-06-25 22:22 ` Willy Tarreau
@ 2006-09-16 12:08 ` Vladimir B. Savkin
2006-09-18 8:35 ` Andi Kleen
3 siblings, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-16 12:08 UTC (permalink / raw)
To: Andi Kleen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
On Mon, Jun 19, 2006 at 05:24:31PM +0200, Andi Kleen wrote:
>
> > If you use "pmtmr" try to reboot with kernel option "clock=tsc".
>
> That's dangerous advice - when the system choses not to use
> TSC it often has a reason.
I just found out that TSC clocksource is not implemented on x86-64.
Kernel version 2.6.18-rc7, is it true?
I've also had experience of unsychronized TSC on dual-core Athlon,
but it was cured by idle=poll.
>
> >
> > On my Opteron AMD system i normally can route 400 kpps, but with
> > timesource "pmtmr" i could only route around 83 kpps. (I found the timer
> > to be the issue by using oprofile).
>
> Unless you're using packet sniffing or any other application
> that requests time stamps on a socket then the timer shouldn't
> make much difference. Incoming packets are only time stamped
> when someone asks for the timestamps.
>
It seems that dhcpd3 makes the box timestamping incoming packets,
killing the performance. I think that combining router and DHCP server
on a same box is a legitimate situation, isn't it?
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-16 12:08 ` Vladimir B. Savkin
@ 2006-09-18 8:35 ` Andi Kleen
2006-09-18 9:03 ` Vladimir B. Savkin
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 8:35 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
"Vladimir B. Savkin" <master@sectorb.msk.ru> writes:
> On Mon, Jun 19, 2006 at 05:24:31PM +0200, Andi Kleen wrote:
> >
> > > If you use "pmtmr" try to reboot with kernel option "clock=tsc".
> >
> > That's dangerous advice - when the system choses not to use
> > TSC it often has a reason.
>
> I just found out that TSC clocksource is not implemented on x86-64.
> Kernel version 2.6.18-rc7, is it true?
The x86-64 timer subsystems currently doesn't have clocksources
at all, but it supports TSC and some other timers.
>
> I've also had experience of unsychronized TSC on dual-core Athlon,
> but it was cured by idle=poll.
You can use that, but it will make your system run quite hot
and cost you a lot of powe^wmoney.
> It seems that dhcpd3 makes the box timestamping incoming packets,
> killing the performance. I think that combining router and DHCP server
> on a same box is a legitimate situation, isn't it?
Yes. Good point. DHCP is broken and needs to be fixed. Can you
send a bug report to the DHCP maintainers?
iirc the problem used to be that RAW sockets didn't do something
they need them to do. Maybe we can fix that now.
If that's not possible we can probably add a ioctl or similar
to disable time stamping for packet sockets (DHCP shouldn't really
need a fine grained time stamp). dhcpcd would need to use that then.
Keep me updated what they say.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 8:35 ` Andi Kleen
@ 2006-09-18 9:03 ` Vladimir B. Savkin
2006-09-18 9:58 ` Andi Kleen
0 siblings, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-18 9:03 UTC (permalink / raw)
To: Andi Kleen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
On Mon, Sep 18, 2006 at 10:35:38AM +0200, Andi Kleen wrote:
> > I just found out that TSC clocksource is not implemented on x86-64.
> > Kernel version 2.6.18-rc7, is it true?
>
> The x86-64 timer subsystems currently doesn't have clocksources
> at all, but it supports TSC and some other timers.
Hm. On my box, TSC did not work, until I hacked arch/i386/kernel/tsc.c
in it.
Neither clock=tsc nor clocksource=tsc didn't have any effect.
> > I've also had experience of unsychronized TSC on dual-core Athlon,
> > but it was cured by idle=poll.
>
> You can use that, but it will make your system run quite hot
> and cost you a lot of powe^wmoney.
Here in Russia electric power is cheap compared with hardware upgrade.
> > It seems that dhcpd3 makes the box timestamping incoming packets,
> > killing the performance. I think that combining router and DHCP server
> > on a same box is a legitimate situation, isn't it?
>
> Yes. Good point. DHCP is broken and needs to be fixed. Can you
> send a bug report to the DHCP maintainers?
>
> iirc the problem used to be that RAW sockets didn't do something
> they need them to do. Maybe we can fix that now.
Will try some days later.
Oh, and pppoe-server uses some kind of packet socket too, doesn't it?
>
> If that's not possible we can probably add a ioctl or similar
> to disable time stamping for packet sockets (DHCP shouldn't really
> need a fine grained time stamp). dhcpcd would need to use that then.
I would like some sysctl very much, too. Let tcpdump show imprecise
timestamps when forwarding performance is more important.
After all, Ciscos don't have any tcpdump analog at all, and they are
very popular :)
>
> Keep me updated what they say.
>
> -Andi
>
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 9:03 ` Vladimir B. Savkin
@ 2006-09-18 9:58 ` Andi Kleen
2006-09-18 10:29 ` Vladimir B. Savkin
2006-09-18 14:09 ` David Miller
0 siblings, 2 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 9:58 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
"Vladimir B. Savkin" <master@sectorb.msk.ru> writes:
> On Mon, Sep 18, 2006 at 10:35:38AM +0200, Andi Kleen wrote:
> > > I just found out that TSC clocksource is not implemented on x86-64.
> > > Kernel version 2.6.18-rc7, is it true?
> >
> > The x86-64 timer subsystems currently doesn't have clocksources
> > at all, but it supports TSC and some other timers.
>
> until I hacked arch/i386/kernel/tsc.c
Then you don't use x86-64.
>
> > > I've also had experience of unsychronized TSC on dual-core Athlon,
> > > but it was cured by idle=poll.
> >
> > You can use that, but it will make your system run quite hot
> > and cost you a lot of powe^wmoney.
>
> Here in Russia electric power is cheap compared with hardware upgrade.
It's not just electrical power - the hardware is more stressed and will
likely fail earlier too. As a rule of thumb the hotter your hardware runs
the earlier it will fail.
>
> > > It seems that dhcpd3 makes the box timestamping incoming packets,
> > > killing the performance. I think that combining router and DHCP server
> > > on a same box is a legitimate situation, isn't it?
> >
> > Yes. Good point. DHCP is broken and needs to be fixed. Can you
> > send a bug report to the DHCP maintainers?
> >
> > iirc the problem used to be that RAW sockets didn't do something
> > they need them to do. Maybe we can fix that now.
>
> Will try some days later.
>
> Oh, and pppoe-server uses some kind of packet socket too, doesn't it?
The problem is not really using a packet socket, but using the SIOCGSTAMP
ioctl on it. As soon as someone issues it the system will take accurate
time stamps for each incoming packet until the respective socket is closed.
Quick fix is to change user space to use gettimeofday() when it reads
the packet instead.
For netdev: I'm more and more thinking we should just avoid the problem
completely and switch to "true end2end" timestamps. This means don't
time stamp when a packet is received, but only when it is delivered
to a socket. The timestamp at receiving is a lie anyways because
the network hardware can add an arbitary long delay before the driver interrupt
handler runs. Then the problem above would completely disappear.
Comments? Opinions?
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 9:58 ` Andi Kleen
@ 2006-09-18 10:29 ` Vladimir B. Savkin
2006-09-18 11:27 ` Andi Kleen
2006-09-18 14:09 ` David Miller
1 sibling, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-18 10:29 UTC (permalink / raw)
To: Andi Kleen; +Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
On Mon, Sep 18, 2006 at 11:58:21AM +0200, Andi Kleen wrote:
> > > The x86-64 timer subsystems currently doesn't have clocksources
> > > at all, but it supports TSC and some other timers.
> >
>
> > until I hacked arch/i386/kernel/tsc.c
>
> Then you don't use x86-64.
>
Oh. I mean I made arch/i386/kernel/tsc.c compile on x86-64
by hacking some Makefiles and headers.
But the question is, why stock 2.6.18-rc7 could not use TSC on its own?
> > > > I've also had experience of unsychronized TSC on dual-core Athlon,
> > > > but it was cured by idle=poll.
> > >
> > > You can use that, but it will make your system run quite hot
> > > and cost you a lot of powe^wmoney.
> >
> > Here in Russia electric power is cheap compared with hardware upgrade.
>
> It's not just electrical power - the hardware is more stressed and will
> likely fail earlier too. As a rule of thumb the hotter your hardware runs
> the earlier it will fail.
What hardware exactly. Doesn't it affect only CPU? And they are not
know to fail before any other components.
> >
> > > > It seems that dhcpd3 makes the box timestamping incoming packets,
> > > > killing the performance. I think that combining router and DHCP server
> > > > on a same box is a legitimate situation, isn't it?
> > >
> > > Yes. Good point. DHCP is broken and needs to be fixed. Can you
> > > send a bug report to the DHCP maintainers?
> > >
> > > iirc the problem used to be that RAW sockets didn't do something
> > > they need them to do. Maybe we can fix that now.
> >
> > Will try some days later.
> >
> > Oh, and pppoe-server uses some kind of packet socket too, doesn't it?
>
> The problem is not really using a packet socket, but using the SIOCGSTAMP
> ioctl on it. As soon as someone issues it the system will take accurate
> time stamps for each incoming packet until the respective socket is closed.
>
> Quick fix is to change user space to use gettimeofday() when it reads
> the packet instead.
Ok, thank you, I now understand.
>
> For netdev: I'm more and more thinking we should just avoid the problem
> completely and switch to "true end2end" timestamps. This means don't
> time stamp when a packet is received, but only when it is delivered
> to a socket. The timestamp at receiving is a lie anyways because
> the network hardware can add an arbitary long delay before the driver interrupt
> handler runs. Then the problem above would completely disappear.
> Comments? Opinions?
>
> -Andi
>
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 10:29 ` Vladimir B. Savkin
@ 2006-09-18 11:27 ` Andi Kleen
2006-09-18 15:38 ` Alexey Kuznetsov
2006-09-18 21:08 ` Network performance degradation from 2.6.11.12 to 2.6.16.20 Vladimir B. Savkin
0 siblings, 2 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 11:27 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev
"Vladimir B. Savkin" <master@sectorb.msk.ru> writes:
[you seem to send your emails in a strange way that doesn't keep me in cc.
Please stop doing that.]
> On Mon, Sep 18, 2006 at 11:58:21AM +0200, Andi Kleen wrote:
> > > > The x86-64 timer subsystems currently doesn't have clocksources
> > > > at all, but it supports TSC and some other timers.
> > >
> >
> > > until I hacked arch/i386/kernel/tsc.c
> >
> > Then you don't use x86-64.
> >
> Oh. I mean I made arch/i386/kernel/tsc.c compile on x86-64
> by hacking some Makefiles and headers.
The codebase for timing (and lots of other things) is quite different
between 32bit and 64bit. You're really surprised it doesn't work if you do such things?
> But the question is, why stock 2.6.18-rc7 could not use TSC on its own?
x86-64 doesn't use the TSC when it deems it to not be reliable, which
is the case on your system.
> > > > > I've also had experience of unsychronized TSC on dual-core Athlon,
> > > > > but it was cured by idle=poll.
> > > >
> > > > You can use that, but it will make your system run quite hot
> > > > and cost you a lot of powe^wmoney.
> > >
> > > Here in Russia electric power is cheap compared with hardware upgrade.
> >
> > It's not just electrical power - the hardware is more stressed and will
> > likely fail earlier too. As a rule of thumb the hotter your hardware runs
> > the earlier it will fail.
>
> What hardware exactly. Doesn't it affect only CPU? And they are not
> know to fail before any other components.
All hardware. It's basic physics.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 11:27 ` Andi Kleen
@ 2006-09-18 15:38 ` Alexey Kuznetsov
2006-09-18 15:54 ` Andi Kleen
2006-09-18 21:08 ` Network performance degradation from 2.6.11.12 to 2.6.16.20 Vladimir B. Savkin
1 sibling, 1 reply; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-18 15:38 UTC (permalink / raw)
To: Andi Kleen
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
Hello!
> For netdev: I'm more and more thinking we should just avoid the problem
> completely and switch to "true end2end" timestamps. This means don't
> time stamp when a packet is received, but only when it is delivered
> to a socket.
This will work.
>From viewpoint of existing uses of timestamp by packet socket
this time is not worse. The only danger is violation of casuality
(when forwarded packet or reply packet gets timestamp earlier than
original packet). This pathology was main reason why timestamp
is recorded early, before packet is demultiplexed in netif_receive_skb().
But it is not a practical problem: delivery to packet/raw sockets
is occasionally placed _before_ delivery to real protocol handlers.
> handler runs. Then the problem above would completely disappear.
Well, not completely. Too slow clock source remains too slow clock source.
If it is so slow, that it results in "performance degradation", it just
should not be used at all, even such pariah as tcpdump wants to be fast.
Actually, I have a question. Why the subject is
"Network performance degradation from 2.6.11.12 to 2.6.16.20"?
I do not see beginning of the thread and cannot guess
why clock source degraded. :-)
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 15:38 ` Alexey Kuznetsov
@ 2006-09-18 15:54 ` Andi Kleen
2006-09-18 16:28 ` Alexey Kuznetsov
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 15:54 UTC (permalink / raw)
To: Alexey Kuznetsov
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
On Monday 18 September 2006 17:38, Alexey Kuznetsov wrote:
> Hello!
>
> > For netdev: I'm more and more thinking we should just avoid the problem
> > completely and switch to "true end2end" timestamps. This means don't
> > time stamp when a packet is received, but only when it is delivered
> > to a socket.
>
> This will work.
>
> From viewpoint of existing uses of timestamp by packet socket
> this time is not worse. The only danger is violation of casuality
> (when forwarded packet or reply packet gets timestamp earlier than
> original packet).
Hmm, not sure how that could happen. Also is it a real problem
even if it could?
> > handler runs. Then the problem above would completely disappear.
>
> Well, not completely. Too slow clock source remains too slow clock source.
> If it is so slow, that it results in "performance degradation", it just
> should not be used at all, even such pariah as tcpdump wants to be fast.
>
> Actually, I have a question. Why the subject is
> "Network performance degradation from 2.6.11.12 to 2.6.16.20"?
> I do not see beginning of the thread and cannot guess
> why clock source degraded. :-)
It's a long and sad story.
Old kernels didn't disable the TSC on those boxes (multi core K8) and assumed
they were synchronized for timing purposes.
This initially mostly worked if you don't use cpufreq,
but over a longer uptime the TSCs would drift against each other and timing
would jump more and more between CPUs.
On older versions of K8 this drift happened much slower (more
aggressive power saving in HLT in newer steppings made it worse; that is why
idle=poll helps) and could be often ignored. But technically it was still a
bug there because it would could break timing after long uptimes.
New multi socket K8 boxes are generally
totally unusable with TSC because they use cpufreq and the TSCs can run
at completely differently frequencies, which obviously doesn't give very
good timing information if you assume the TSC is globally synchronized.
That is why later kernels default to TSC off. The original plan
was to use HPET then, which is slower than TSC, but still not that bad.
But while most modern systems have a HPET timer somewhere in the chipset
nearly all BIOS vendors "forgot" to describe it in the BIOS because Windows
didn't use it and Linux can't find it because of that.
Then it has to use the ACPI pmtmr which is really really slow.
The overhead of that thing is so large that you can clearly see it in
the network benchmark.
The real fix long term is to change the timer subsystem to keep all TSC
state per CPU, then it'll work on the K8s too. Unfortunately it's a moderately
hard problem to make the result still fully monotonic. But people are working
on it.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 15:54 ` Andi Kleen
@ 2006-09-18 16:28 ` Alexey Kuznetsov
2006-09-18 16:50 ` Andi Kleen
0 siblings, 1 reply; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-18 16:28 UTC (permalink / raw)
To: Andi Kleen
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
Hello!
> Hmm, not sure how that could happen. Also is it a real problem
> even if it could?
As I said, the problem is _occasionally_ theoretical.
This would happen f.e. if packet socket handler was installed
after IP handler. Then tcpdump would get packet after it is processed
(acked/replied/forwarded). This would be disasterous, the results
are unparsable.
I recall, the issue was discussed, and that time it looked more
reasonable to solve problems of this kind taking timestamp once
before it is seen by all the rest of stack. Who could expect that
PIT nightmare is going to return? :-)
> Then it has to use the ACPI pmtmr which is really really slow.
> The overhead of that thing is so large that you can clearly see it in
> the network benchmark.
I see. Thank you.
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 16:28 ` Alexey Kuznetsov
@ 2006-09-18 16:50 ` Andi Kleen
2006-09-18 21:03 ` Alexey Kuznetsov
2006-09-18 21:18 ` Vladimir B. Savkin
0 siblings, 2 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 16:50 UTC (permalink / raw)
To: Alexey Kuznetsov
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
On Monday 18 September 2006 18:28, Alexey Kuznetsov wrote:
> Hello!
>
> > Hmm, not sure how that could happen. Also is it a real problem
> > even if it could?
>
> As I said, the problem is _occasionally_ theoretical.
>
> This would happen f.e. if packet socket handler was installed
> after IP handler. Then tcpdump would get packet after it is processed
> (acked/replied/forwarded). This would be disasterous, the results
> are unparsable.
But that never happens right?
And do you have some other prefered way to solve this? Even if the timer
was fast it would be still good to avoid it in the fast path when DHCPD
is running.
I suppose in the worst case a sysctl like Vladimir asked for could be added,
but it would seem somewhat lame.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 16:50 ` Andi Kleen
@ 2006-09-18 21:03 ` Alexey Kuznetsov
2006-09-18 21:22 ` David Miller
2006-09-19 5:52 ` Andi Kleen
2006-09-18 21:18 ` Vladimir B. Savkin
1 sibling, 2 replies; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-18 21:03 UTC (permalink / raw)
To: Andi Kleen
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
Hello!
> But that never happens right?
Right.
Well, not right. It happens. Simply because you get packet
with newer timestamp after previous handler saw this packet
and did some actions. I just do not see any bad consequences.
> And do you have some other prefered way to solve this? Even if the timer
> was fast it would be still good to avoid it in the fast path when DHCPD
> is running.
No. The way, which you suggested, seems to be the best.
1. It even does not disable possibility to record timestamp inside
driver, which Alan was afraid of. The sequence is:
if (!skb->tstamp.off_sec)
net_timestamp(skb);
2. Maybe, netif_rx() should continue to get timestamp in netif_rx().
3. NAPI already introduced almost the same inaccuracy. And it is really
silly to waste time getting timestamp in netif_receive_skb() a few
moments before the packet is delivered to a socket.
4. ...but clock source, which takes one of top lines in profiles
must be repaired yet. :-)
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:03 ` Alexey Kuznetsov
@ 2006-09-18 21:22 ` David Miller
2006-09-18 21:46 ` Alexey Kuznetsov
` (2 more replies)
2006-09-19 5:52 ` Andi Kleen
1 sibling, 3 replies; 59+ messages in thread
From: David Miller @ 2006-09-18 21:22 UTC (permalink / raw)
To: kuznet; +Cc: ak, master, hawk, harry, linux-kernel, netdev
From: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
Date: Tue, 19 Sep 2006 01:03:21 +0400
> 1. It even does not disable possibility to record timestamp inside
> driver, which Alan was afraid of. The sequence is:
>
> if (!skb->tstamp.off_sec)
> net_timestamp(skb);
>
> 2. Maybe, netif_rx() should continue to get timestamp in netif_rx().
>
> 3. NAPI already introduced almost the same inaccuracy. And it is really
> silly to waste time getting timestamp in netif_receive_skb() a few
> moments before the packet is delivered to a socket.
>
> 4. ...but clock source, which takes one of top lines in profiles
> must be repaired yet. :-)
Ok, ok, but don't we have queueing disciplines that need the timestamp
even on ingress?
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:22 ` David Miller
@ 2006-09-18 21:46 ` Alexey Kuznetsov
2006-09-19 5:55 ` Andi Kleen
2006-09-19 20:31 ` Thomas Graf
2 siblings, 0 replies; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-18 21:46 UTC (permalink / raw)
To: David Miller; +Cc: ak, master, hawk, harry, linux-kernel, netdev
Hello!
> Ok, ok, but don't we have queueing disciplines that need the timestamp
> even on ingress?
I cannot find.
ip_queue does. But it is just another user, not different of sockets.
BTW in any case, any user of timestamp who sees 0, because skb was received
before timestamping was enabled, has to calculate timestamp itself right
in the place where Andi suggested. Seems, preparation to the change
makes sense even without the change. :-)
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:22 ` David Miller
2006-09-18 21:46 ` Alexey Kuznetsov
@ 2006-09-19 5:55 ` Andi Kleen
2006-09-19 20:31 ` Thomas Graf
2 siblings, 0 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-19 5:55 UTC (permalink / raw)
To: David Miller; +Cc: kuznet, master, hawk, harry, linux-kernel, netdev
On Monday 18 September 2006 23:22, David Miller wrote:
> Ok, ok, but don't we have queueing disciplines that need the timestamp
> even on ingress?
I grepped and I can't find any. The only non SIOCGTSTAMP users of the
time stamp seem to be sunrpc and conntrack and I bet both can be converted
over to jiffies without trouble.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:22 ` David Miller
2006-09-18 21:46 ` Alexey Kuznetsov
2006-09-19 5:55 ` Andi Kleen
@ 2006-09-19 20:31 ` Thomas Graf
2006-09-19 20:43 ` Andi Kleen
2 siblings, 1 reply; 59+ messages in thread
From: Thomas Graf @ 2006-09-19 20:31 UTC (permalink / raw)
To: David Miller; +Cc: kuznet, ak, master, hawk, harry, linux-kernel, netdev
* David Miller <davem@davemloft.net> 2006-09-18 14:22
> From: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
> Date: Tue, 19 Sep 2006 01:03:21 +0400
>
> > 1. It even does not disable possibility to record timestamp inside
> > driver, which Alan was afraid of. The sequence is:
> >
> > if (!skb->tstamp.off_sec)
> > net_timestamp(skb);
> >
> > 2. Maybe, netif_rx() should continue to get timestamp in netif_rx().
> >
> > 3. NAPI already introduced almost the same inaccuracy. And it is really
> > silly to waste time getting timestamp in netif_receive_skb() a few
> > moments before the packet is delivered to a socket.
> >
> > 4. ...but clock source, which takes one of top lines in profiles
> > must be repaired yet. :-)
>
> Ok, ok, but don't we have queueing disciplines that need the timestamp
> even on ingress?
Queueing disciplines generally only care about the time delta
between two packets, using the receive stamp would lead to
wrong results as soon as a packet is queued more than once.
However, since we recently introcued ingress queueing we
must update the stamp to make up for the delay caused by the
queue. Updating the stamp at socket enqueue time would solve
this automatically.
It seems only natural to me that the real problem is the slow
clock source which needs to be resolved regardless of the
outcome of this discussion. I believe that updating the stamp
at socket enqueue time is the right thing to do but it shouldn't
be considered as a solution to the performance problem.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-19 20:31 ` Thomas Graf
@ 2006-09-19 20:43 ` Andi Kleen
0 siblings, 0 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-19 20:43 UTC (permalink / raw)
To: Thomas Graf
Cc: David Miller, kuznet, master, hawk, harry, linux-kernel, netdev
> It seems only natural to me that the real problem is the slow
> clock source which needs to be resolved regardless of the
> outcome of this discussion. I believe that updating the stamp
> at socket enqueue time is the right thing to do but it shouldn't
> be considered as a solution to the performance problem.
While I agree it would be nice to fix that particular issue
(it's unfortunately hard) slow clock sources in general won't go
away. They are also in lots of other platforms.
And even if you have a fast clock source not using it when you
don't need to is better. For example some x86s can be quite
slow even reading TSCs. It's much better than pmtmr
it's still is a expensive operations that is best avoided.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:03 ` Alexey Kuznetsov
2006-09-18 21:22 ` David Miller
@ 2006-09-19 5:52 ` Andi Kleen
1 sibling, 0 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-19 5:52 UTC (permalink / raw)
To: Alexey Kuznetsov
Cc: Vladimir B. Savkin, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev
On Monday 18 September 2006 23:03, Alexey Kuznetsov wrote:
>
> > And do you have some other prefered way to solve this? Even if the timer
> > was fast it would be still good to avoid it in the fast path when DHCPD
> > is running.
>
> No. The way, which you suggested, seems to be the best.
Ok. I also checked my desktop and for some reason I got a timestamp counter
of 7 (and it doesn't even run client dhcp). Haven't investigated why yet, and I am
still hoping it's not a leak.
But that hints that trying to fix all of user space to not use the ioctl
would have been probably too much work.
> 1. It even does not disable possibility to record timestamp inside
> driver, which Alan was afraid of. The sequence is:
>
> if (!skb->tstamp.off_sec)
> net_timestamp(skb);
>
> 2. Maybe, netif_rx() should continue to get timestamp in netif_rx().
Hmm, there are still quite a lot users and even with netif_rx() you
can have long delays from interrupt mitigation etc.
% grep -rw netif_rx drivers/net/* | wc -l
253
> 3. NAPI already introduced almost the same inaccuracy. And it is really
> silly to waste time getting timestamp in netif_receive_skb() a few
> moments before the packet is delivered to a socket.
>
> 4. ...but clock source, which takes one of top lines in profiles
> must be repaired yet. :-)
It's being worked on, but it'll take some time. But even when TSC
can be used it's still a good idea to not call gtod unnecessarily
because it can be still relatively slow (e.g. on P4 RDTSC takes
hundreds of cycles because it synchronizes the CPU). Also on some
other non x86 platforms it is also relatively slow because they have
to reach out to the chipset and every time you do that things get slow.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 16:50 ` Andi Kleen
2006-09-18 21:03 ` Alexey Kuznetsov
@ 2006-09-18 21:18 ` Vladimir B. Savkin
2006-09-18 22:00 ` Alexey Kuznetsov
1 sibling, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-18 21:18 UTC (permalink / raw)
To: Andi Kleen
Cc: Alexey Kuznetsov, Jesper Dangaard Brouer, Harry Edmon,
linux-kernel, netdev, Andi Kleen
On Mon, Sep 18, 2006 at 06:50:22PM +0200, Andi Kleen wrote:
>
> I suppose in the worst case a sysctl like Vladimir asked for could be added,
> but it would seem somewhat lame.
>
Please think about it this way:
suppose you haave a heavily loaded router and some network problem is to
be diagnosed. You run tcpdump and suddenly router becomes overloaded (by
switching to timestamp-it-all mode), drops OSPF adjancecies etc. Users
are angry, and you can't diagnose anything. But with impresise
timestamps and maybe even with reordered packets you still have some
traces to analyze.
So, in this particular corner case it's not that lame.
Or maybe patching tcpdump will do better?
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:18 ` Vladimir B. Savkin
@ 2006-09-18 22:00 ` Alexey Kuznetsov
2006-09-18 21:57 ` David Lang
` (2 more replies)
0 siblings, 3 replies; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-18 22:00 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: Andi Kleen, Jesper Dangaard Brouer, Harry Edmon, linux-kernel,
netdev
Hello!
> Please think about it this way:
> suppose you haave a heavily loaded router and some network problem is to
> be diagnosed. You run tcpdump and suddenly router becomes overloaded (by
> switching to timestamp-it-all mode
I am sorry. I cannot think that way. :-)
Instead of attempts to scare, better resend original report,
where you said how much performance degraded, I cannot find it.
* I do see get_offset_pmtmr() in top lines of profile. That's scary enough.
* I do not undestand what the hell dhcp needs timestamps for.
* I do not listen any suggestions to screw up tcpdump with a sysctl.
Kernel already implements much better thing then a sysctl.
Do not want timestamps? Fix tcpdump, add an options, submit the
patch to tcpdump maintainers. Not a big deal.
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 22:00 ` Alexey Kuznetsov
@ 2006-09-18 21:57 ` David Lang
2006-09-19 19:40 ` David Miller
2006-09-18 22:03 ` Vladimir B. Savkin
2006-09-19 19:47 ` David Miller
2 siblings, 1 reply; 59+ messages in thread
From: David Lang @ 2006-09-18 21:57 UTC (permalink / raw)
To: Alexey Kuznetsov
Cc: Vladimir B. Savkin, Andi Kleen, Jesper Dangaard Brouer,
Harry Edmon, linux-kernel, netdev
On Tue, 19 Sep 2006, Alexey Kuznetsov wrote:
> Hello!
>
>> Please think about it this way:
>> suppose you haave a heavily loaded router and some network problem is to
>> be diagnosed. You run tcpdump and suddenly router becomes overloaded (by
>> switching to timestamp-it-all mode
>
> I am sorry. I cannot think that way. :-)
>
> Instead of attempts to scare, better resend original report,
> where you said how much performance degraded, I cannot find it.
>
> * I do see get_offset_pmtmr() in top lines of profile. That's scary enough.
> * I do not undestand what the hell dhcp needs timestamps for.
> * I do not listen any suggestions to screw up tcpdump with a sysctl.
> Kernel already implements much better thing then a sysctl.
> Do not want timestamps? Fix tcpdump, add an options, submit the
> patch to tcpdump maintainers. Not a big deal.
if fireing up one program (however minor) can cause network performance to drop
by >50% (based on the numbers reported earlier in this thread) that is a
significant problem for sysadmins.
yes tcpdump may be wrong in requesting timestamps (in most cases it probably is,
but in some cases it's doing exactly what the sysadmin wants it to do), but I
don't think that many sysadmins would expect this much of a performance hit.
there should be some way to tell the system to ignore requests for timestamps so
that a badly behaved program cannot cripple the system this way (and preferably
something that doesn't require a full SELinux/capabilities implementation)
David Lang
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 21:57 ` David Lang
@ 2006-09-19 19:40 ` David Miller
2006-09-19 19:44 ` Stephen Hemminger
0 siblings, 1 reply; 59+ messages in thread
From: David Miller @ 2006-09-19 19:40 UTC (permalink / raw)
To: dlang; +Cc: kuznet, master, ak, hawk, harry, linux-kernel, netdev
From: David Lang <dlang@digitalinsight.com>
Date: Mon, 18 Sep 2006 14:57:04 -0700 (PDT)
> yes tcpdump may be wrong in requesting timestamps (in most cases it
> probably is, but in some cases it's doing exactly what the sysadmin
> wants it to do), but I don't think that many sysadmins would expect
> this much of a performance hit. there should be some way to tell
> the system to ignore requests for timestamps so that a badly behaved
> program cannot cripple the system this way (and preferably something
> that doesn't require a full SELinux/capabilities implementation)
tcpdump is not wrong in requesting timestamps, and there are
many legitimate userland programs that call gettimeofday()
for internal timestamping _A LOT_. Such as X11 clients.
The real fact of the matter is that these x86_64 systems are using the
slowest possible time-of-day implementation, simply because it's "too
hard" currently to properly probe the most efficient mechanism which
is present in the system.
If getting the time of day is at the top of the profiles in the packet
input path, and we're only capturing a timestamp once per packet,
something is _VERY VERY_ wrong with the timestamp implementation
because think of all of the other seriously expensive things that
happen on a per-packet basis which should absolutely dwarf
timestamping in terms of cost.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 22:00 ` Alexey Kuznetsov
2006-09-18 21:57 ` David Lang
@ 2006-09-18 22:03 ` Vladimir B. Savkin
2006-09-19 19:41 ` David Miller
2006-09-19 19:47 ` David Miller
2 siblings, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-18 22:03 UTC (permalink / raw)
To: Alexey Kuznetsov
Cc: Andi Kleen, Jesper Dangaard Brouer, Harry Edmon, linux-kernel,
netdev
On Tue, Sep 19, 2006 at 02:00:38AM +0400, Alexey Kuznetsov wrote:
> Hello!
>
> > Please think about it this way:
> > suppose you haave a heavily loaded router and some network problem is to
> > be diagnosed. You run tcpdump and suddenly router becomes overloaded (by
> > switching to timestamp-it-all mode
>
> I am sorry. I cannot think that way. :-)
>
> Instead of attempts to scare, better resend original report,
> where you said how much performance degraded, I cannot find it.
>
> * I do see get_offset_pmtmr() in top lines of profile. That's scary enough.
I had it at the very top line.
> * I do not undestand what the hell dhcp needs timestamps for.
> * I do not listen any suggestions to screw up tcpdump with a sysctl.
> Kernel already implements much better thing then a sysctl.
> Do not want timestamps? Fix tcpdump, add an options, submit the
> patch to tcpdump maintainers. Not a big deal.
OK, point taken.
It's better to patch tcpdump.
>
> Alexey
>
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 22:03 ` Vladimir B. Savkin
@ 2006-09-19 19:41 ` David Miller
0 siblings, 0 replies; 59+ messages in thread
From: David Miller @ 2006-09-19 19:41 UTC (permalink / raw)
To: master; +Cc: kuznet, ak, hawk, harry, linux-kernel, netdev
From: "Vladimir B. Savkin" <master@sectorb.msk.ru>
Date: Tue, 19 Sep 2006 02:03:31 +0400
> On Tue, Sep 19, 2006 at 02:00:38AM +0400, Alexey Kuznetsov wrote:
> > * I do see get_offset_pmtmr() in top lines of profile. That's scary enough.
>
> I had it at the very top line.
That is just rediculious.
You can "fix" the networking by making it timestamp less but what
about things like just normal X11 clients that call gettimeofday()
at very high rates?
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 22:00 ` Alexey Kuznetsov
2006-09-18 21:57 ` David Lang
2006-09-18 22:03 ` Vladimir B. Savkin
@ 2006-09-19 19:47 ` David Miller
2006-09-22 15:35 ` Alexey Kuznetsov
2 siblings, 1 reply; 59+ messages in thread
From: David Miller @ 2006-09-19 19:47 UTC (permalink / raw)
To: kuznet; +Cc: master, ak, hawk, harry, linux-kernel, netdev
From: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
Date: Tue, 19 Sep 2006 02:00:38 +0400
> * I do not undestand what the hell dhcp needs timestamps for.
I can't even find a reference to SIOCGSTAMP in the
dhcp-2.0pl5 or dhcp3-3.0.3 sources shipped in Ubuntu.
But I will note that tpacket_rcv() expects to always get
valid timestamps in the SKB, it does a:
if (skb->tstamp.off_sec == 0) {
__net_timestamp(skb);
sock_enable_timestamp(sk);
}
so that it can fill in the h->tp_sec and h->tp_usec
fields.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-19 19:47 ` David Miller
@ 2006-09-22 15:35 ` Alexey Kuznetsov
2006-09-22 15:43 ` Andi Kleen
0 siblings, 1 reply; 59+ messages in thread
From: Alexey Kuznetsov @ 2006-09-22 15:35 UTC (permalink / raw)
To: David Miller; +Cc: master, ak, hawk, harry, linux-kernel, netdev
Hello!
> I can't even find a reference to SIOCGSTAMP in the
> dhcp-2.0pl5 or dhcp3-3.0.3 sources shipped in Ubuntu.
>
> But I will note that tpacket_rcv() expects to always get
> valid timestamps in the SKB, it does a:
It is equally unlikely it uses mmapped packet socket (tpacket_rcv).
I even installed that dhcp on x86_64. And I do not see anything,
netstamp_needed remains zero when running both server and client.
It looks like dhcp was defamed without a guilt. :-)
Seems, Andi saw some leakage in netstamp_needed (value of 7),
but I do not see this too.
In any case, the issue is obviously more serious than just behaviour
of some applications. On my notebook one gettimeofday() takes:
0.2 us with tsc
4.6 us with pm (AND THIS CRAP IS DEFAULT!!)
9.4 us with pit (kinda expected)
It is ridiculous. Obviosuly, nobody (not only tcpdump, but everything
else) does not need such clock. Taking timestamp takes time comparable
with processing the whole tcp frame. :-) I have no idea what is possible
to do without breaking everything, but it is not something to ignore.
This timer must be shot. :-)
Alexey
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-22 15:35 ` Alexey Kuznetsov
@ 2006-09-22 15:43 ` Andi Kleen
2006-09-22 16:51 ` Rick Jones
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-09-22 15:43 UTC (permalink / raw)
To: Alexey Kuznetsov; +Cc: David Miller, master, hawk, harry, linux-kernel, netdev
On Friday 22 September 2006 17:35, Alexey Kuznetsov wrote:
> Hello!
>
> > I can't even find a reference to SIOCGSTAMP in the
> > dhcp-2.0pl5 or dhcp3-3.0.3 sources shipped in Ubuntu.
> >
> > But I will note that tpacket_rcv() expects to always get
> > valid timestamps in the SKB, it does a:
>
> It is equally unlikely it uses mmapped packet socket (tpacket_rcv).
>
> I even installed that dhcp on x86_64. And I do not see anything,
> netstamp_needed remains zero when running both server and client.
> It looks like dhcp was defamed without a guilt. :-)
>
> Seems, Andi saw some leakage in netstamp_needed (value of 7),
> but I do not see this too.
That came from named. It opens lots of sockets with SIOCGSTAMP.
No idea what it needs that many for.
I suspect it was either dhcpd (server) or that ppp user space daemon
the original reporter was running.
Maybe it would be a good idea to add a printk by default?
> In any case, the issue is obviously more serious than just behaviour
> of some applications. On my notebook one gettimeofday() takes:
>
> 0.2 us with tsc
> 4.6 us with pm (AND THIS CRAP IS DEFAULT!!)
This is actually quite fast. I've seen much worse ratios.
Also on some i386 kernels the pmtimer reads the register three
times to work around some buggy implementation that doesn't latch the counter
properly.
> 9.4 us with pit (kinda expected)
>
> It is ridiculous. Obviosuly, nobody (not only tcpdump, but everything
> else) does not need such clock. Taking timestamp takes time comparable
> with processing the whole tcp frame. :-) I have no idea what is possible
> to do without breaking everything, but it is not something to ignore.
> This timer must be shot. :-)
If it's a reasonably new notebook it might be actually possible to change.
The default choices are quite conservative there because in the past
there were lots of problems with notebooks changing frequency behind
the kernel's back etc. and screwing up TSC. But that shouldn't happen anymore.
If you had a 64bit laptop the kernel would likely do the right choice :)
Notebooks are easy because they are only single socket, so the only thing
needed is to keep track of the frequency (or not if you have a Core+)
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-22 15:43 ` Andi Kleen
@ 2006-09-22 16:51 ` Rick Jones
2007-03-06 13:25 ` Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20) Vladimir B. Savkin
0 siblings, 1 reply; 59+ messages in thread
From: Rick Jones @ 2006-09-22 16:51 UTC (permalink / raw)
To: Andi Kleen
Cc: Alexey Kuznetsov, David Miller, master, hawk, harry, linux-kernel,
netdev
> That came from named. It opens lots of sockets with SIOCGSTAMP.
> No idea what it needs that many for.
IIRC ISC BIND named opens a socket for each IP it finds on the system.
Presumeably in this way it "knows" implicitly the destination IP without
using platform-specific recvfrom/whatever extensions and gets some
additional parallelism in the stack on SMP systems.
Why it needs/wants the timestamps I've no idea, I don't think it gets
them that way on all platforms. I suppose the next time I do some named
benchmarking I can try to take a closer look in the source.
rick jones
^ permalink raw reply [flat|nested] 59+ messages in thread
* Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20)
2006-09-22 16:51 ` Rick Jones
@ 2007-03-06 13:25 ` Vladimir B. Savkin
2007-03-06 14:38 ` Eric Dumazet
0 siblings, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2007-03-06 13:25 UTC (permalink / raw)
To: netdev
Cc: Andi Kleen, Alexey Kuznetsov, David Miller, hawk, harry,
linux-kernel, Rick Jones
On Fri, Sep 22, 2006 at 09:51:09AM -0700, Rick Jones wrote:
> >That came from named. It opens lots of sockets with SIOCGSTAMP.
> >No idea what it needs that many for.
>
> IIRC ISC BIND named opens a socket for each IP it finds on the system.
> Presumeably in this way it "knows" implicitly the destination IP without
> using platform-specific recvfrom/whatever extensions and gets some
> additional parallelism in the stack on SMP systems.
>
> Why it needs/wants the timestamps I've no idea, I don't think it gets
> them that way on all platforms. I suppose the next time I do some named
> benchmarking I can try to take a closer look in the source.
>
Returning to the discussion about packet timestamps, I just
use the following patch now:
diff -ur ../linux-2.6.20.1/include/linux/sysctl.h linux-2.6.20.1-ts/include/linux/sysctl.h
--- ../linux-2.6.20.1/include/linux/sysctl.h 2007-02-20 09:34:32.000000000 +0300
+++ linux-2.6.20.1-ts/include/linux/sysctl.h 2007-03-04 19:10:36.000000000 +0300
@@ -280,6 +280,7 @@
NET_CORE_BUDGET=19,
NET_CORE_AEVENT_ETIME=20,
NET_CORE_AEVENT_RSEQTH=21,
+ NET_CORE_ACCURATE_TIMESTAMPS=99,
};
/* /proc/sys/net/ethernet */
diff -ur ../linux-2.6.20.1/net/core/dev.c linux-2.6.20.1-ts/net/core/dev.c
--- ../linux-2.6.20.1/net/core/dev.c 2007-02-20 09:34:32.000000000 +0300
+++ linux-2.6.20.1-ts/net/core/dev.c 2007-03-04 19:09:44.000000000 +0300
@@ -1043,9 +1043,11 @@
}
EXPORT_SYMBOL(__net_timestamp);
+int sysctl_accurate_timestamps = 1;
+
static inline void net_timestamp(struct sk_buff *skb)
{
- if (atomic_read(&netstamp_needed))
+ if (sysctl_accurate_timestamps && atomic_read(&netstamp_needed))
__net_timestamp(skb);
else {
skb->tstamp.off_sec = 0;
diff -ur ../linux-2.6.20.1/net/core/sysctl_net_core.c linux-2.6.20.1-ts/net/core/sysctl_net_core.c
--- ../linux-2.6.20.1/net/core/sysctl_net_core.c 2007-02-20 09:34:32.000000000 +0300
+++ linux-2.6.20.1-ts/net/core/sysctl_net_core.c 2007-03-04 19:05:11.000000000 +0300
@@ -21,6 +21,8 @@
extern int sysctl_core_destroy_delay;
+extern int sysctl_accurate_timestamps;
+
#ifdef CONFIG_XFRM
extern u32 sysctl_xfrm_aevent_etime;
extern u32 sysctl_xfrm_aevent_rseqth;
@@ -136,6 +138,14 @@
.mode = 0644,
.proc_handler = &proc_dointvec
},
+ {
+ .ctl_name = NET_CORE_ACCURATE_TIMESTAMPS,
+ .procname = "accurate_timestamps",
+ .data = &sysctl_accurate_timestamps,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = &proc_dointvec
+ },
{ .ctl_name = 0 }
};
May I ask about integrating this or a similar solution for those
like me who values routing performance (with bind9 running) over
minor convinience of having tcpdump always display accurate
timestamps?
And why current kernel (2.6.20.1) still ignores parameter
clocksource=tsc ? I think with idle=poll TSC is safe to use on my setup,
it had ran with TSC for many months without a problem.
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20)
2007-03-06 13:25 ` Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20) Vladimir B. Savkin
@ 2007-03-06 14:38 ` Eric Dumazet
2007-03-06 14:43 ` Vladimir B. Savkin
0 siblings, 1 reply; 59+ messages in thread
From: Eric Dumazet @ 2007-03-06 14:38 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: netdev, Andi Kleen, Alexey Kuznetsov, David Miller, hawk, harry,
linux-kernel, Rick Jones
On Tuesday 06 March 2007 14:25, Vladimir B. Savkin wrote:
> },
> + {
> + .ctl_name = NET_CORE_ACCURATE_TIMESTAMPS,
> + .procname = "accurate_timestamps",
> + .data = &sysctl_accurate_timestamps,
> + .maxlen = sizeof(int),
> + .mode = 0644,
> + .proc_handler = &proc_dointvec
> + },
> { .ctl_name = 0 }
> };
>
>
> May I ask about integrating this or a similar solution for those
> like me who values routing performance (with bind9 running) over
> minor convinience of having tcpdump always display accurate
> timestamps?
>
Quite frankly I dont like this patch :
1) Fix applications, do not bloat kernel.
2) "accurate_timestamps" is misleading.
Should be "disable_timestamps"
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20)
2007-03-06 14:38 ` Eric Dumazet
@ 2007-03-06 14:43 ` Vladimir B. Savkin
2007-03-06 15:16 ` Eric Dumazet
0 siblings, 1 reply; 59+ messages in thread
From: Vladimir B. Savkin @ 2007-03-06 14:43 UTC (permalink / raw)
To: Eric Dumazet
Cc: netdev, Andi Kleen, Alexey Kuznetsov, David Miller, hawk, harry,
linux-kernel, Rick Jones
On Tue, Mar 06, 2007 at 03:38:44PM +0100, Eric Dumazet wrote:
> 2) "accurate_timestamps" is misleading.
> Should be "disable_timestamps"
Not, if default is 1, as in my patch.
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20)
2007-03-06 14:43 ` Vladimir B. Savkin
@ 2007-03-06 15:16 ` Eric Dumazet
2007-03-06 18:15 ` Vladimir B. Savkin
0 siblings, 1 reply; 59+ messages in thread
From: Eric Dumazet @ 2007-03-06 15:16 UTC (permalink / raw)
To: Vladimir B. Savkin
Cc: netdev, Andi Kleen, Alexey Kuznetsov, David Miller, hawk, harry,
linux-kernel, Rick Jones
On Tuesday 06 March 2007 15:43, Vladimir B. Savkin wrote:
> On Tue, Mar 06, 2007 at 03:38:44PM +0100, Eric Dumazet wrote:
> > 2) "accurate_timestamps" is misleading.
> > Should be "disable_timestamps"
>
> Not, if default is 1, as in my patch.
Yes I saw this. I should write more words next time :)
Full explanation:
----------------------
If your tunable is named "accurate_timestamps" then a 0 value would mean :
Use a low precision timestamp (based on xtime for example) instead of a full
resolution...
This is not what your patch does (while it could do that, but beware that
net-2.6.22 includes now a ktime_t timestamping)
So :
------
It would be better to name the tunable "disable_timestamps", default 0 of
course....
It would better describe what your patch is actually doing : Even if a tcpdump
is running (so asking for timestamps), it wont have them because the sysctl
disabled them.
Thank you
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20)
2007-03-06 15:16 ` Eric Dumazet
@ 2007-03-06 18:15 ` Vladimir B. Savkin
0 siblings, 0 replies; 59+ messages in thread
From: Vladimir B. Savkin @ 2007-03-06 18:15 UTC (permalink / raw)
To: Eric Dumazet
Cc: netdev, Andi Kleen, Alexey Kuznetsov, David Miller, hawk, harry,
linux-kernel, Rick Jones
On Tue, Mar 06, 2007 at 04:16:24PM +0100, Eric Dumazet wrote:
>
> It would be better to name the tunable "disable_timestamps", default 0 of
> course....
I agree.
If networking maintainers are interested, I surely can prepare a patch.
But IMO some way to force TSC usage on x86_64 will be even better.
> It would better describe what your patch is actually doing : Even if a tcpdump
> is running (so asking for timestamps), it wont have them because the sysctl
> disabled them.
Well, tcpdump will have timestamps, but taken at wrong moment.
But some other applications (that use ip_queue, ulog etc.) will not,
as I understand.
>
> Thank you
>
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 11:27 ` Andi Kleen
2006-09-18 15:38 ` Alexey Kuznetsov
@ 2006-09-18 21:08 ` Vladimir B. Savkin
1 sibling, 0 replies; 59+ messages in thread
From: Vladimir B. Savkin @ 2006-09-18 21:08 UTC (permalink / raw)
To: Andi Kleen
Cc: Jesper Dangaard Brouer, Harry Edmon, linux-kernel, netdev,
Andi Kleen
On Mon, Sep 18, 2006 at 01:27:57PM +0200, Andi Kleen wrote:
> The codebase for timing (and lots of other things) is quite different
> between 32bit and 64bit. You're really surprised it doesn't work if you do such things?
>
It works, and after your remark above, I'm surprised.
Dunno about slow TSC drift though, there was not enough time passed to
detect it, and I hope we will have this problem soved in a better way
before the drift becomes visible :)
> > But the question is, why stock 2.6.18-rc7 could not use TSC on its own?
>
> x86-64 doesn't use the TSC when it deems it to not be reliable, which
> is the case on your system.
>
Could it at least print something so that I know that using TSC was
considered, but rejected?
> > What hardware exactly. Doesn't it affect only CPU? And they are not
> > know to fail before any other components.
>
> All hardware. It's basic physics.
Hm, what other hardware is affected by idle=poll? Does this option ear
out HDDs?
~
:wq
With best regards,
Vladimir Savkin.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 9:58 ` Andi Kleen
2006-09-18 10:29 ` Vladimir B. Savkin
@ 2006-09-18 14:09 ` David Miller
2006-09-18 14:29 ` Andi Kleen
1 sibling, 1 reply; 59+ messages in thread
From: David Miller @ 2006-09-18 14:09 UTC (permalink / raw)
To: ak; +Cc: master, hawk, harry, linux-kernel, netdev
From: Andi Kleen <ak@suse.de>
Date: 18 Sep 2006 11:58:21 +0200
> For netdev: I'm more and more thinking we should just avoid the
> problem completely and switch to "true end2end" timestamps. This
> means don't time stamp when a packet is received, but only when it
> is delivered to a socket. The timestamp at receiving is a lie
> anyways because the network hardware can add an arbitary long delay
> before the driver interrupt handler runs. Then the problem above
> would completely disappear.
I don't think this is wise.
People who run tcpdump want "wire" timestamps as close as possible.
Yes, things get delayed with the IRQ path, DMA delays, IRQ
mitigation and whatnot, but it's an order of magnitude worse if
you delay to user read() since that introduces also the delay of
the packet copies to userspace which are significantly larger than
these hardware level delays. If tcpdump gets swapped out, the
timestamp delay can be on the order of several seconds making it
totally useless.
Andi, you will need to find another solution to this problem :-)
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 14:09 ` David Miller
@ 2006-09-18 14:29 ` Andi Kleen
2006-09-18 15:19 ` Alan Cox
0 siblings, 1 reply; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 14:29 UTC (permalink / raw)
To: David Miller; +Cc: master, hawk, harry, linux-kernel, netdev
>
> People who run tcpdump want "wire" timestamps as close as possible.
> Yes, things get delayed with the IRQ path, DMA delays, IRQ
> mitigation and whatnot, but it's an order of magnitude worse if
> you delay to user read() since that introduces also the delay of
> the packet copies to userspace which are significantly larger than
> these hardware level delays. If tcpdump gets swapped out, the
> timestamp delay can be on the order of several seconds making it
> totally useless.
My proposal wasn't to delay to user read, just to do the time stamp in socket
context. This means as soon as packet or RAW/UDP have looked up the socket and can
check a per socket flag do the time stamp.
The only delay this would add would be the queueing time from the NIC
to the softirq. Do you really think that is that bad?
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 14:29 ` Andi Kleen
@ 2006-09-18 15:19 ` Alan Cox
2006-09-18 15:19 ` Andi Kleen
0 siblings, 1 reply; 59+ messages in thread
From: Alan Cox @ 2006-09-18 15:19 UTC (permalink / raw)
To: Andi Kleen; +Cc: David Miller, master, hawk, harry, linux-kernel, netdev
Ar Llu, 2006-09-18 am 16:29 +0200, ysgrifennodd Andi Kleen:
> The only delay this would add would be the queueing time from the NIC
> to the softirq. Do you really think that is that bad?
If you are trying to do things like network record/playback then you
want the minimal delay. There's a reason the original timestamp code
supported the hardware setting the timestamp itself - we actually had a
separare set of logic on a board that was doing the timestamping by
watching the IRQ line of the NIC chip.
Alan
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-09-18 15:19 ` Alan Cox
@ 2006-09-18 15:19 ` Andi Kleen
0 siblings, 0 replies; 59+ messages in thread
From: Andi Kleen @ 2006-09-18 15:19 UTC (permalink / raw)
To: Alan Cox; +Cc: David Miller, master, hawk, harry, linux-kernel, netdev
On Monday 18 September 2006 17:19, Alan Cox wrote:
> Ar Llu, 2006-09-18 am 16:29 +0200, ysgrifennodd Andi Kleen:
> > The only delay this would add would be the queueing time from the NIC
> > to the softirq. Do you really think that is that bad?
>
> If you are trying to do things like network record/playback then you
> want the minimal delay.
But it's not minimal. Maybe it was long ago when the code was designed
on a 3c509 but not with modern hardware: Think interrupt mitigation and NAPI.
And with NAPI we tend to process the packets directly after they
are fetched out of the RX queue, so there is practically no delay
between driver seeing the packet and softirq seeing it. All the queuing
is done either at hardware level or later at socket level.
> There's a reason the original timestamp code
> supported the hardware setting the timestamp itself - we actually had a
> separare set of logic on a board that was doing the timestamping by
> watching the IRQ line of the NIC chip.
That would be fine too (because it will be likely fast), but unfortunately
I don't know of any driver that does that.
-Andi
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: Network performance degradation from 2.6.11.12 to 2.6.16.20
2006-06-19 14:47 ` Jesper Dangaard Brouer
2006-06-19 15:24 ` Andi Kleen
@ 2006-06-19 16:40 ` Harry Edmon
1 sibling, 0 replies; 59+ messages in thread
From: Harry Edmon @ 2006-06-19 16:40 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: linux-kernel, netdev
Jesper Dangaard Brouer wrote:
>
>>> Harry Edmon <harry@atmos.washington.edu> wrote:
>>>
>>>> I have a system with a strange network performance degradation from
>>>> 2.6.11.12 to most recent kernels including 2.6.16.20 and
>>>> 2.6.17-rc6. The system is has Dual single core Xeons with
>>>> hyperthreading on.
> <cut>
>
> Hi Harry
>
> Can you check which "high-res timesource" you are using?
>
> In the kernel log look for:
> kernel: Using tsc for high-res timesource
> kernel: Using pmtmr for high-res timesource
>
> I have experinced some network performance degradation when using the
> "pmtmr" timesource, on a Opteron AMD system. It seems that the
> default timesource change between 2.6.15 to 2.6.16.
>
> If you use "pmtmr" try to reboot with kernel option "clock=tsc".
>
> On my Opteron AMD system i normally can route 400 kpps, but with
> timesource "pmtmr" i could only route around 83 kpps. (I found the
> timer to be the issue by using oprofile).
>
>
We have CONFIG_HPET_TIMER=y, so we do not see these messages.
^ permalink raw reply [flat|nested] 59+ messages in thread
end of thread, other threads:[~2007-03-06 18:15 UTC | newest]
Thread overview: 59+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-06-16 16:01 Network performance degradation from 2.6.11.12 to 2.6.16.20 Harry Edmon
2006-06-17 22:35 ` Andrew Morton
2006-06-17 23:23 ` Harry Edmon
2006-06-17 23:56 ` Andrew Morton
2006-06-18 3:16 ` Stephen Hemminger
2006-06-18 23:23 ` Harry Edmon
2006-06-19 13:54 ` Harry Edmon
2006-06-20 2:11 ` Herbert Xu
2006-06-19 14:47 ` Jesper Dangaard Brouer
2006-06-19 15:24 ` Andi Kleen
2006-06-19 17:34 ` Chris Friesen
2006-06-19 20:39 ` Andi Kleen
2006-06-19 18:24 ` Jesper Dangaard Brouer
2006-06-25 21:51 ` Harry Edmon
2006-06-26 4:20 ` Bill Fink
2006-06-25 22:22 ` Willy Tarreau
2006-06-26 5:23 ` Andi Kleen
2006-07-04 11:41 ` Jesper Dangaard Brouer
2006-07-04 11:54 ` Andi Kleen
2006-07-10 10:55 ` Jesper Dangaard Brouer
2006-09-16 12:08 ` Vladimir B. Savkin
2006-09-18 8:35 ` Andi Kleen
2006-09-18 9:03 ` Vladimir B. Savkin
2006-09-18 9:58 ` Andi Kleen
2006-09-18 10:29 ` Vladimir B. Savkin
2006-09-18 11:27 ` Andi Kleen
2006-09-18 15:38 ` Alexey Kuznetsov
2006-09-18 15:54 ` Andi Kleen
2006-09-18 16:28 ` Alexey Kuznetsov
2006-09-18 16:50 ` Andi Kleen
2006-09-18 21:03 ` Alexey Kuznetsov
2006-09-18 21:22 ` David Miller
2006-09-18 21:46 ` Alexey Kuznetsov
2006-09-19 5:55 ` Andi Kleen
2006-09-19 20:31 ` Thomas Graf
2006-09-19 20:43 ` Andi Kleen
2006-09-19 5:52 ` Andi Kleen
2006-09-18 21:18 ` Vladimir B. Savkin
2006-09-18 22:00 ` Alexey Kuznetsov
2006-09-18 21:57 ` David Lang
2006-09-19 19:40 ` David Miller
2006-09-19 19:44 ` Stephen Hemminger
2006-09-18 22:03 ` Vladimir B. Savkin
2006-09-19 19:41 ` David Miller
2006-09-19 19:47 ` David Miller
2006-09-22 15:35 ` Alexey Kuznetsov
2006-09-22 15:43 ` Andi Kleen
2006-09-22 16:51 ` Rick Jones
2007-03-06 13:25 ` Packet timestamps (was: Re: Network performance degradation from 2.6.11.12 to 2.6.16.20) Vladimir B. Savkin
2007-03-06 14:38 ` Eric Dumazet
2007-03-06 14:43 ` Vladimir B. Savkin
2007-03-06 15:16 ` Eric Dumazet
2007-03-06 18:15 ` Vladimir B. Savkin
2006-09-18 21:08 ` Network performance degradation from 2.6.11.12 to 2.6.16.20 Vladimir B. Savkin
2006-09-18 14:09 ` David Miller
2006-09-18 14:29 ` Andi Kleen
2006-09-18 15:19 ` Alan Cox
2006-09-18 15:19 ` Andi Kleen
2006-06-19 16:40 ` Harry Edmon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox