* 2.6.0-test4 parallel seek & read problem
@ 2003-08-26 17:32 Magnus Andersson
2003-09-29 14:55 ` Magnus Andersson
0 siblings, 1 reply; 2+ messages in thread
From: Magnus Andersson @ 2003-08-26 17:32 UTC (permalink / raw)
To: linux-kernel
Hello everyone!
I have run into some strange behavior with kernel 2.6.0-test4.
I'm doing a lot of parallel random seeks and reads in a 128 Gigabyte
file. The file is located on a JFS partition on a 3ware Storage
Controller containing four IDE disks with an average seek time of
8.5 ms. The 3ware controller is configured as RAID0 with stripe size 1MB.
The operating system is Debian GNU/Linux, version testing.
Running on this system is a program listening for connections. When a
connection is accepted, a thread is created. This thread will get a
free file descriptor from a pool of already opened file desriptors.
The thread will do a random seek on the fd, and a small read, around 3 Kb.
This data is then sent to the client and the connection is closed.
My program is linked agains libpthread.a from the libc6-dev package,
version 2.3.1-16, in Debian.
If I'm running kernel 2.4.21 or 2.4.22, I can connect to the program
500 times per second. Using iostat-4.1.5 to monitor the system I see
500 reads/second issued to the disk. So far so fine.
If I change to 2.6.0-test4 I can only connect to the program around 75
times per second. During this time iostat reports around 120
reads/second issued to the disk.
If I change the elevator to 'deadline' or 'noop' I can connect to the
program around 300 times/second. Iostat reports around 520
reads/second issued to the disk.
I also have a modified version of my program using nonblocking network
code, and aio. This time I'm using the same fd for all aio
requests. The file is opened with the flag O_DIRECT, bypassing the
buffer cache. This time I'm able to reach close to 500 requests/second.
2.6.0-test1-ac3, and 2.6.0-test3-mm1 are behaving the same as 2.6.0-test4.
Someone have any idea what is going on? Seems like something is not
what it should be. My hope is to be able to reach the same amounts of
connections/second as with the 2.4 kernels.
I can't provide the source to the program, because it is owned by
someone else, but I hope someone can point me in the right direction
for further investigation of this problem.
Below are the output of dmesg and the kernel config file for 2.6.0-test4.
Best regards
/Magnus
-------------------------START OF DMESG---------------------------------------
Linux version 2.6.0-test4 (magan@duck) (gcc version 3.3.1 20030626 (Debian prerelease)) #1 Mon Aug 25 15:54:42 CEST 2003
Video mode to be used for restore is 9
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000001ffeb000 (usable)
BIOS-e820: 000000001ffeb000 - 000000001ffef000 (ACPI data)
BIOS-e820: 000000001ffef000 - 000000001ffff000 (reserved)
BIOS-e820: 000000001ffff000 - 0000000020000000 (ACPI NVS)
BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
511MB LOWMEM available.
On node 0 totalpages: 131051
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 126955 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
DMI 2.3 present.
ACPI: RSDP (v000 ASUS ) @ 0x000f7850
ACPI: RSDT (v001 ASUS CUSL2-C 0x30303031 MSFT 0x31313031) @ 0x1ffeb000
ACPI: FADT (v001 ASUS CUSL2-C 0x30303031 MSFT 0x31313031) @ 0x1ffeb080
ACPI: BOOT (v001 ASUS CUSL2-C 0x30303031 MSFT 0x31313031) @ 0x1ffeb040
ACPI: DSDT (v001 ASUS CUSL2-C 0x00001000 MSFT 0x0100000b) @ 0x00000000
Building zonelist for node : 0
Kernel command line: auto BOOT_IMAGE=2.6.0-test4 ro root=302
Initializing CPU#0
PID hash table entries: 2048 (order 11: 16384 bytes)
Detected 1005.174 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 1990.65 BogoMIPS
Memory: 514888k/524204k available (2309k kernel code, 8520k reserved, 726k data, 120k init, 0k highmem)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
-> /dev
-> /dev/console
-> /root
CPU: After generic identify, caps: 0383f9ff 00000000 00000000 00000000
CPU: After vendor identify, caps: 0383f9ff 00000000 00000000 00000000
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 256K
CPU: After all inits, caps: 0383f9ff 00000000 00000000 00000040
CPU: Intel Pentium III (Coppermine) stepping 06
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
Initializing RT netlink socket
PCI: PCI BIOS revision 2.10 entry at 0xf0df0, last bus=2
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
BIO: pool of 256 setup, 14Kb (56 bytes/bio)
biovec pool[0]: 1 bvecs: 256 entries (12 bytes)
biovec pool[1]: 4 bvecs: 256 entries (48 bytes)
biovec pool[2]: 16 bvecs: 256 entries (192 bytes)
biovec pool[3]: 64 bvecs: 256 entries (768 bytes)
biovec pool[4]: 128 bvecs: 256 entries (1536 bytes)
biovec pool[5]: 256 bvecs: 256 entries (3072 bytes)
Linux Plug and Play Support v0.97 (c) Adam Belay
PnPBIOS: Scanning system for PnP BIOS support...
PnPBIOS: Found PnP BIOS installation structure at 0xc00fbd80
PnPBIOS: PnP BIOS version 1.0, entry 0xf0000:0xbdb0, dseg 0xf0000
PnPBIOS: 12 nodes reported by PnP BIOS; 12 recorded by driver
SCSI subsystem initialized
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
Transparent bridge - 0000:00:1e.0
PCI: Using IRQ router PIIX [8086/2440] at 0000:00:1f.0
pty: 256 Unix98 ptys configured
SBF: Simple Boot Flag extension found and enabled.
SBF: Setting boot flags 0x1
Journalled Block Device driver loaded
SGI XFS for Linux with no debug enabled
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Real Time Clock Driver v1.11a
Using anticipatory scheduling elevator
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
loop: loaded (max 8 devices)
PCI: Found IRQ 12 for device 0000:02:0d.0
PCI: Sharing IRQ 12 with 0000:02:09.0
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
0000:02:0d.0: 3Com PCI 3c905C Tornado at 0xd000. Vers LK1.1.19
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ICH2: IDE controller at PCI slot 0000:00:1f.1
ICH2: chipset revision 2
ICH2: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xb800-0xb807, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0xb808-0xb80f, BIOS settings: hdc:pio, hdd:pio
hda: IC35L020AVER07-0, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: max request size: 128KiB
hda: 40188960 sectors (20576 MB) w/1916KiB Cache, CHS=39870/16/63, UDMA(100)
hda: hda1 hda2
3ware Storage Controller device driver for Linux v1.02.00.032.
PCI: Found IRQ 4 for device 0000:02:0a.0
scsi0 : Found a 3ware Storage Controller at 0xd400, IRQ: 4, P-chip: 1.3
scsi0 : 3ware Storage Controller
Vendor: 3ware Model: Logical Disk 0 Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 00
Vendor: 3ware Model: Logical Disk 4 Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 00
Vendor: 3ware Model: Logical Disk 5 Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 00
Vendor: 3ware Model: Logical Disk 6 Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sda: 600530944 512-byte hdwr sectors (307472 MB)
SCSI device sda: drive cache: write back
sda: sda1
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sdb: 150136560 512-byte hdwr sectors (76870 MB)
SCSI device sdb: drive cache: write through
sdb: unknown partition table
Attached scsi disk sdb at scsi0, channel 0, id 4, lun 0
SCSI device sdc: 150136560 512-byte hdwr sectors (76870 MB)
SCSI device sdc: drive cache: write through
sdc: unknown partition table
Attached scsi disk sdc at scsi0, channel 0, id 5, lun 0
SCSI device sdd: 150136560 512-byte hdwr sectors (76870 MB)
SCSI device sdd: drive cache: write through
sdd: unknown partition table
Attached scsi disk sdd at scsi0, channel 0, id 6, lun 0
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
input: AT Set 2 keyboard on isa0060/serio0
serio: i8042 KBD port at 0x60,0x64 irq 1
md: linear personality registered as nr 1
md: raid0 personality registered as nr 2
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
NET4: Linux TCP/IP 1.0 for NET4.0
IP: routing cache hash table of 4096 buckets, 32Kbytes
TCP: Hash tables configured (established 32768 bind 65536)
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 120k freed
Adding 1951856k swap on /dev/hda1. Priority:-1 extents:1
md: could not lock unknown-block(22,0).
md: could not import unknown-block(22,0)!
md: autostart unknown-block(22,0) failed!
--------------------END OF DMESG-------------------------------------------
--------------------KERNEL CONFIG-------------------------------------------
#
# Automatically generated make config: don't edit
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
# CONFIG_BROKEN is not set
#
# General setup
#
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
CONFIG_LOG_BUF_SHIFT=14
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
# CONFIG_MODVERSIONS is not set
CONFIG_KMOD=y
#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
CONFIG_MPENTIUMIII=y
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MELAN is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_SMP is not set
# CONFIG_PREEMPT is not set
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_TSC=y
# CONFIG_X86_MCE is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
#
# Power management options (ACPI, APM)
#
# CONFIG_PM is not set
#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI_HT=y
# CONFIG_ACPI is not set
CONFIG_ACPI_BOOT=y
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
CONFIG_HOTPLUG=y
#
# PCMCIA/CardBus support
#
# CONFIG_PCMCIA is not set
CONFIG_PCMCIA_PROBE=y
#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
#
# Executable file formats
#
CONFIG_KCORE_ELF=y
# CONFIG_KCORE_AOUT is not set
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_AOUT is not set
# CONFIG_BINFMT_MISC is not set
#
# Generic Driver Options
#
# CONFIG_FW_LOADER is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
# CONFIG_PARPORT is not set
#
# Plug and Play support
#
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set
#
# Protocols
#
CONFIG_ISAPNP=y
CONFIG_PNPBIOS=y
#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_BLK_DEV_XD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_RAM is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_LBD is not set
#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
# CONFIG_IDEDISK_MULTI_MODE is not set
# CONFIG_IDEDISK_STROKE is not set
# CONFIG_BLK_DEV_IDECD is not set
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set
CONFIG_IDE_TASKFILE_IO=y
#
# IDE chipset support/bugfixes
#
# CONFIG_BLK_DEV_CMD640 is not set
# CONFIG_BLK_DEV_IDEPNP is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
# CONFIG_BLK_DEV_GENERIC is not set
# CONFIG_BLK_DEV_OPTI621 is not set
CONFIG_BLK_DEV_RZ1000=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDE_TCQ is not set
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_IDEDMA_PCI_WIP is not set
CONFIG_BLK_DEV_ADMA=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_DMA_NONPCI is not set
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#
CONFIG_SCSI=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
# CONFIG_CHR_DEV_SG is not set
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
CONFIG_SCSI_REPORT_LUNS=y
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=y
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_IN2000 is not set
# CONFIG_SCSI_AM53C974 is not set
# CONFIG_SCSI_MEGARAID is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_CPQFCTS is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_SYM53C8XX is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_PCI2000 is not set
# CONFIG_SCSI_PCI2220I is not set
# CONFIG_SCSI_PSI240I is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_SEAGATE is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set
#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=y
CONFIG_MD_RAID0=y
# CONFIG_MD_RAID1 is not set
# CONFIG_MD_RAID5 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_BLK_DEV_DM is not set
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support (EXPERIMENTAL)
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
# CONFIG_I2O is not set
#
# Networking support
#
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
# CONFIG_INET_ECN is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_IPV6 is not set
# CONFIG_DECNET is not set
# CONFIG_BRIDGE is not set
# CONFIG_NETFILTER is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
CONFIG_IPV6_SCTP__=y
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_LLC is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_FASTROUTE is not set
# CONFIG_NET_HW_FLOWCONTROL is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
CONFIG_NETDEVICES=y
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
CONFIG_DUMMY=y
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_ETHERTAP is not set
# CONFIG_NET_SB1000 is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
# CONFIG_MII is not set
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
CONFIG_NET_VENDOR_3COM=y
# CONFIG_EL1 is not set
# CONFIG_EL2 is not set
# CONFIG_ELPLUS is not set
# CONFIG_EL16 is not set
# CONFIG_EL3 is not set
# CONFIG_3C515 is not set
CONFIG_VORTEX=y
# CONFIG_TYPHOON is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set
#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
# CONFIG_NET_PCI is not set
# CONFIG_NET_POCKET is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SK98LIN is not set
CONFIG_TIGON3=y
#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Token Ring devices (depends on LLC=y)
#
# CONFIG_NET_FC is not set
# CONFIG_RCPCI is not set
# CONFIG_SHAPER is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
#
# Amateur Radio support
#
# CONFIG_HAMRADIO is not set
#
# IrDA (infrared) support
#
# CONFIG_IRDA is not set
#
# ISDN subsystem
#
# CONFIG_ISDN_BOOL is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
#
# Non-8250 serial port support
#
CONFIG_UNIX98_PTYS=y
CONFIG_UNIX98_PTY_COUNT=256
#
# I2C support
#
# CONFIG_I2C is not set
#
# I2C Hardware Sensors Mainboard support
#
#
# I2C Hardware Sensors Chip support
#
# CONFIG_I2C_SENSOR is not set
#
# Mice
#
# CONFIG_BUSMOUSE is not set
# CONFIG_QIC02_TAPE is not set
#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set
#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
# CONFIG_AGP is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HANGCHECK_TIMER is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_JFS_FS=y
# CONFIG_JFS_POSIX_ACL is not set
# CONFIG_JFS_DEBUG is not set
CONFIG_JFS_STATISTICS=y
CONFIG_XFS_FS=y
# CONFIG_XFS_RT is not set
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_POSIX_ACL is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set
#
# DOS/FAT/NT Filesystems
#
# CONFIG_FAT_FS is not set
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_DEVFS_FS is not set
CONFIG_DEVPTS_FS=y
# CONFIG_DEVPTS_FS_XATTR is not set
# CONFIG_TMPFS is not set
CONFIG_RAMFS=y
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V4 is not set
# CONFIG_NFSD is not set
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
# CONFIG_EXPORTFS is not set
CONFIG_SUNRPC=y
# CONFIG_SUNRPC_GSS is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_INTERMEZZO_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_NLS=y
#
# Native Language Support
#
CONFIG_NLS_DEFAULT="iso8859-1"
# CONFIG_NLS_CODEPAGE_437 is not set
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
# CONFIG_NLS_ISO8859_1 is not set
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set
#
# Graphics support
#
# CONFIG_FB is not set
# CONFIG_VIDEO_SELECT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
#
# Sound
#
# CONFIG_SOUND is not set
#
# USB support
#
# CONFIG_USB is not set
# CONFIG_USB_GADGET is not set
#
# Bluetooth support
#
# CONFIG_BT is not set
#
# Profiling support
#
# CONFIG_PROFILING is not set
#
# Kernel hacking
#
# CONFIG_DEBUG_KERNEL is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_FRAME_POINTER is not set
#
# Security options
#
# CONFIG_SECURITY is not set
#
# Cryptographic options
#
# CONFIG_CRYPTO is not set
#
# Library routines
#
# CONFIG_CRC32 is not set
CONFIG_X86_BIOS_REBOOT=y
--------------------END OF KERNEL CONFIG-------------------------------------
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: 2.6.0-test4 parallel seek & read problem
2003-08-26 17:32 2.6.0-test4 parallel seek & read problem Magnus Andersson
@ 2003-09-29 14:55 ` Magnus Andersson
0 siblings, 0 replies; 2+ messages in thread
From: Magnus Andersson @ 2003-09-29 14:55 UTC (permalink / raw)
To: linux-kernel; +Cc: Andrew Morton, Nick Piggin
[-- Attachment #1: Type: text/plain, Size: 6006 bytes --]
Hello again!
I have some more information for you all regarding this issue.
The problem with my server only able to handle 75 connections/sec was
solved by the 'as-initial-io-noantic' patch from Nick Piggin,
attached to this email if you want to try it out.
Looks like 2.6.0-test6 contains a modified version of this patch,
this modified version doesn't work well, the first patch was better.
With the original patch applied to -test4 or -test5, my server is
only able to handle half the amount of connections/sec as it should
be able to handle.
It doesn't matter which elevator I run. as-, deadline-, and noop-
all have the same problem.
The reason for this performance loss is that my read requests get
splitted by the kernel into those 4096 bytes chunks. Later, these
chunks are never merged by the kernel. In other words: 1 seek&read
issued by me, goes to the disk as 2 separate seek&read. Between these
2 seek&read are a lot of other seeks&reads issued by other threads.
I put this printk() after the call to elv_next_request(),
inside scsi_request_fn(), in scsi_lib.c, to see what is going on.
if(req) // 2 lines added by me
printk("SEC %d\n", req->sector);
The output from dmesg is attached to this email, in the file
printk.output. There you can see that the requests were splitted and
not merged. The splitted chunks are not even sent following each
other, there are a lot of other requests in between, resulting in a
lot of extra head movements.
I tried to disable readahead with 'blockdev --setra 0 /dev/sda',
but still the same problem, and I tried to raise readahead, but
no luck there too.
If the file I read from is opened with the flag O_DIRECT, everything
is good. All 3 elevators performs at a maximum rate.
Attached to this email is also a test program, called thread_seek,
for you all to try out.
The test program keeps nthreads (64 default) running at the same time.
The threads do 1 seek(), a READSIZE (4096 default) bytes read(), and
exits. The file it is operating on is "./bigfile"
Important is the choise of filesystem, choose one with extends. JFS
and XFS is two I know about.
I don't know if the raid system is part of the problem, but I don't
think so. Feel free to try on a different kind of disk setup.
Make sure the bigfile is much bigger than your memory to avoid cache hits.
Do a umount & mount between tests to clear the memory cache if you
feel that it is needed.
compiling: gcc -o thread_seek thread_seek.c -lpthread
running: ./thread_seek [nthreads]
Here are the commands I use to see what is going on.
2.4.22:
'sar -c 1 0'
'sar -b 1 0'
2.6.0-test4:
'sar -c 1 0'
'iostat -x 1'
What you should see on 2.4 is that the output from the two sar should
be quite equal. With kernel 2.4 for example, I see 500
processes/second created, and 500 reads/second going to the disk.
With 2.6 I see 260 processes/second created, and 500 reads/second
going to the disk. The values you will see depends on your setup. But
the ratio between processes/second & reads/second should be 1. On 2.4
it is 1, and on 2.6 it is 0.52, wich is not good. Should be 1 here
too.
Setting READSIZE to 1 byte gives no performance loss. The server on 2.6
is able to handle 500 proc/second & 500 reads/second.
I hope someone is able to reproduce this behavior. Feel free
to ask for more information if I forgot to say something
you need to know.
I also found strange stuff inside elevator_noop_add_request()
list_add_tail(&rq->queuelist, &q->queue_head);
should probably be changed to:
list_add_tail(&rq->queuelist, insert);
/Magnus
On Tue, Aug 26, 2003 at 07:32:41PM +0200, Magnus Andersson wrote:
> Hello everyone!
>
> I have run into some strange behavior with kernel 2.6.0-test4.
>
> I'm doing a lot of parallel random seeks and reads in a 128 Gigabyte
> file. The file is located on a JFS partition on a 3ware Storage
> Controller containing four IDE disks with an average seek time of
> 8.5 ms. The 3ware controller is configured as RAID0 with stripe size 1MB.
>
> The operating system is Debian GNU/Linux, version testing.
>
> Running on this system is a program listening for connections. When a
> connection is accepted, a thread is created. This thread will get a
> free file descriptor from a pool of already opened file desriptors.
> The thread will do a random seek on the fd, and a small read, around 3 Kb.
> This data is then sent to the client and the connection is closed.
>
> My program is linked agains libpthread.a from the libc6-dev package,
> version 2.3.1-16, in Debian.
>
> If I'm running kernel 2.4.21 or 2.4.22, I can connect to the program
> 500 times per second. Using iostat-4.1.5 to monitor the system I see
> 500 reads/second issued to the disk. So far so fine.
>
> If I change to 2.6.0-test4 I can only connect to the program around 75
> times per second. During this time iostat reports around 120
> reads/second issued to the disk.
>
> If I change the elevator to 'deadline' or 'noop' I can connect to the
> program around 300 times/second. Iostat reports around 520
> reads/second issued to the disk.
>
> I also have a modified version of my program using nonblocking network
> code, and aio. This time I'm using the same fd for all aio
> requests. The file is opened with the flag O_DIRECT, bypassing the
> buffer cache. This time I'm able to reach close to 500 requests/second.
>
> 2.6.0-test1-ac3, and 2.6.0-test3-mm1 are behaving the same as 2.6.0-test4.
>
> Someone have any idea what is going on? Seems like something is not
> what it should be. My hope is to be able to reach the same amounts of
> connections/second as with the 2.4 kernels.
>
> I can't provide the source to the program, because it is owned by
> someone else, but I hope someone can point me in the right direction
> for further investigation of this problem.
>
> Below are the output of dmesg and the kernel config file for 2.6.0-test4.
>
> Best regards
>
> /Magnus
LocalWords: nthreads READSIZE
[-- Attachment #2: as-initial-io-noantic --]
[-- Type: text/plain, Size: 538 bytes --]
--- linux-2.6/drivers/block/as-iosched.c.orig 2003-08-27 19:15:24.000000000 +1000
+++ linux-2.6/drivers/block/as-iosched.c 2003-08-27 19:18:13.000000000 +1000
@@ -709,6 +709,14 @@ static int as_can_break_anticipation(str
return 1;
}
+ if (aic->seek_samples == 0 || aic->ttime_samples == 0) {
+ /*
+ * Process has just started IO so default to not anticipate.
+ * Maybe should be smarter.
+ */
+ return 1;
+ }
+
if (aic->ttime_mean > ad->antic_expire) {
/* the process thinks too much between requests */
return 1;
[-- Attachment #3: printk.output --]
[-- Type: text/plain, Size: 13426 bytes --]
SEC 172051559
SEC 13813847
SEC 51237615
SEC 257744159
SEC 53686847
SEC 137177695
SEC 257744167
SEC 83426135
SEC 205555351
SEC 135267503
SEC 23309303
SEC 83426143
SEC 169104455
SEC 61499679
SEC 148370231
SEC 92443095
SEC 131900239
SEC 82034119
SEC 272031151
SEC 230777247
SEC 100578959
SEC 233801887
SEC 82034127
SEC 66901983
SEC 256493039
SEC 22051087
SEC 272031159
SEC 273658927
SEC 105083095
SEC 94483935
SEC 263437351
SEC 68318911
SEC 144796511
SEC 22051095
SEC 222388159
SEC 270802071
SEC 103099303
SEC 76072159
SEC 263437359
SEC 239927103
SEC 246605751
SEC 44445911
SEC 147363839
SEC 120025063
SEC 94075375
SEC 222388167
SEC 20386855
SEC 127551663
SEC 270802079
SEC 119170007
SEC 135267511
SEC 196171671
SEC 13913719
SEC 216117327
SEC 66066511
SEC 104923519
SEC 240880119
SEC 173821935
SEC 41347055
SEC 239927111
SEC 207128775
SEC 58956727
SEC 60025927
SEC 94075383
SEC 119170015
SEC 184290999
SEC 230666983
SEC 66066519
SEC 138100855
SEC 255878895
SEC 234213607
SEC 131900247
SEC 58956735
SEC 126403215
SEC 15054079
SEC 24817471
SEC 204953343
SEC 236541455
SEC 43407967
SEC 180961311
SEC 54968479
SEC 120829735
SEC 43407975
SEC 9785215
SEC 254416295
SEC 40044159
SEC 232982767
SEC 260532927
SEC 246605759
SEC 242248263
SEC 14644367
SEC 255878903
SEC 95238295
SEC 15054087
SEC 270508199
SEC 20386863
SEC 247261103
SEC 95238303
SEC 161708527
SEC 235662823
SEC 41347063
SEC 59500591
SEC 161708535
SEC 147308535
SEC 9785223
SEC 230391735
SEC 54968487
SEC 182184839
SEC 147308543
SEC 104923527
SEC 183982231
SEC 107056799
SEC 13486631
SEC 13913727
SEC 230391743
SEC 174778119
SEC 44445919
SEC 183982239
SEC 97193383
SEC 204899303
SEC 126403223
SEC 246319231
SEC 24817479
SEC 271034303
SEC 60025935
SEC 243920143
SEC 61499687
SEC 138100863
SEC 66901991
SEC 180961319
SEC 91056623
SEC 68318919
SEC 40044167
SEC 13327079
SEC 15026039
SEC 183837607
SEC 209808183
SEC 94483943
SEC 13327087
SEC 221087743
SEC 137177703
SEC 195626039
SEC 72230487
SEC 144796519
SEC 164200583
SEC 228672047
SEC 147363847
SEC 172051567
SEC 170040023
SEC 73980887
SEC 191188727
SEC 148370239
SEC 8945447
SEC 169104463
SEC 84774735
SEC 170040031
SEC 240748527
SEC 174479871
SEC 207128783
SEC 73980895
SEC 204953351
SEC 164200591
SEC 8945455
SEC 108952063
SEC 189060039
SEC 174778127
SEC 144553111
SEC 233801895
SEC 174479879
SEC 218810311
SEC 107952551
SEC 240880127
SEC 100385047
SEC 183837615
SEC 108952071
SEC 151077735
SEC 54418487
SEC 209965591
SEC 107952559
SEC 217406503
SEC 195626047
SEC 11099095
SEC 260532935
SEC 204899311
SEC 202259247
SEC 54418495
SEC 126854047
SEC 218810319
SEC 48845551
SEC 273658935
SEC 217406511
SEC 13486639
SEC 247189359
SEC 211302215
SEC 15038279
SEC 228672055
SEC 11099103
SEC 238555119
SEC 38489039
SEC 240748535
SEC 59500599
SEC 119050591
SEC 119448951
SEC 202259255
SEC 188781391
SEC 86228223
SEC 239900503
SEC 65671023
SEC 169764335
SEC 195576463
SEC 145282799
SEC 247261111
SEC 86228231
SEC 175043159
SEC 93345735
SEC 84774743
SEC 210166407
SEC 91056631
SEC 108048151
SEC 270508207
SEC 97193391
SEC 107056807
SEC 271034311
SEC 120829743
SEC 144553119
SEC 143163055
SEC 242758527
SEC 15026047
SEC 60004495
SEC 190250447
SEC 143163063
SEC 97230359
SEC 189060047
SEC 121631215
SEC 60004503
SEC 72230495
SEC 182184847
SEC 134023007
SEC 209965599
SEC 191188735
SEC 190250455
SEC 259782111
SEC 100385055
SEC 144318647
SEC 138620911
SEC 97230367
SEC 49217207
SEC 57249663
SEC 192911959
SEC 209808191
SEC 49217215
SEC 221087751
SEC 119448959
SEC 15038287
SEC 126854055
SEC 192911967
SEC 166672791
SEC 165928935
SEC 217907079
SEC 48845559
SEC 151077743
SEC 166672799
SEC 235662831
SEC 165928943
SEC 93345743
SEC 24636871
SEC 76173815
SEC 119050599
SEC 217907087
SEC 252238983
SEC 126251575
SEC 157247391
SEC 243920151
SEC 252238991
SEC 53107823
SEC 117011503
SEC 169764343
SEC 246319239
SEC 121631223
SEC 247189367
SEC 93171375
SEC 53107831
SEC 271004903
SEC 190849287
SEC 222187279
SEC 64662383
SEC 145282807
SEC 195576471
SEC 105185631
SEC 175043167
SEC 139463215
SEC 173512887
SEC 188781399
SEC 24636879
SEC 165636287
SEC 210166415
SEC 173512895
SEC 61488783
SEC 71309455
SEC 38489047
SEC 110441303
SEC 211302223
SEC 269598351
SEC 57249671
SEC 61488791
SEC 225573535
SEC 161166039
SEC 239900511
SEC 64662391
SEC 242758535
SEC 71309463
SEC 43103079
SEC 65671031
SEC 259782119
SEC 124095087
SEC 269598359
SEC 262065111
SEC 154509991
SEC 271004911
SEC 225573543
SEC 269534607
SEC 38520175
SEC 35937271
SEC 239653151
SEC 93171383
SEC 133366335
SEC 109896895
SEC 35937279
SEC 67189407
SEC 110789831
SEC 67189415
SEC 158897095
SEC 39137751
SEC 102212719
SEC 243041263
SEC 108048159
SEC 40070927
SEC 138620919
SEC 43103087
SEC 247942095
SEC 139463223
SEC 189677711
SEC 120256959
SEC 105185639
SEC 133415759
SEC 76173823
SEC 120256967
SEC 167801095
SEC 109896903
SEC 125278847
SEC 110789839
SEC 125278855
SEC 81100167
SEC 142417903
SEC 59901551
SEC 164916039
SEC 222187287
SEC 249394783
SEC 81100175
SEC 110441311
SEC 117011511
SEC 133366343
SEC 142417911
SEC 140703607
SEC 220075063
SEC 82654663
SEC 176750447
SEC 134023015
SEC 176750455
SEC 69511239
SEC 213379951
SEC 238555127
SEC 156818855
SEC 126251583
SEC 144318655
SEC 242775487
SEC 12593943
SEC 156818863
SEC 248105967
SEC 135122143
SEC 158897103
SEC 267063343
SEC 154509999
SEC 161166047
SEC 167801103
SEC 130798751
SEC 157247399
SEC 275665679
SEC 269397391
SEC 209151967
SEC 65434487
SEC 190849295
SEC 40070935
SEC 60524287
SEC 217823271
SEC 55138759
SEC 255785615
SEC 239653159
SEC 75091975
SEC 195973959
SEC 235601903
SEC 217823279
SEC 269534615
SEC 45535367
SEC 69511247
SEC 212791751
SEC 60561175
SEC 55138767
SEC 38520183
SEC 60404503
SEC 128105607
SEC 59901559
SEC 124095095
SEC 255785623
SEC 60524295
SEC 39137759
SEC 135122151
SEC 235601911
SEC 267204439
SEC 194999175
SEC 219305271
SEC 102212727
SEC 231063359
SEC 45535375
SEC 188652527
SEC 128105615
SEC 20506983
SEC 268983959
SEC 114516943
SEC 143827151
SEC 188652535
SEC 60404511
SEC 251567439
SEC 60561183
SEC 164916047
SEC 65434495
SEC 231063367
SEC 35044399
SEC 165636295
SEC 98201191
SEC 133415767
SEC 268983967
SEC 67384351
SEC 75091983
SEC 48976863
SEC 213379959
SEC 138640623
SEC 251567447
SEC 82654671
SEC 260765967
SEC 176792423
SEC 35044407
SEC 129848655
SEC 232184407
SEC 101497207
SEC 65834431
SEC 138640631
SEC 258446559
SEC 186815119
SEC 108658575
SEC 260765975
SEC 199440007
SEC 27162639
SEC 242758551
SEC 247942103
SEC 248105975
SEC 130798759
SEC 189677719
SEC 101497215
SEC 140703615
SEC 262065119
SEC 76138895
SEC 271538815
SEC 269397399
SEC 209151975
SEC 152811639
SEC 232184415
SEC 172743607
SEC 243041271
SEC 194999183
SEC 12593951
SEC 63671687
SEC 195973967
SEC 108658583
SEC 108601319
SEC 20506991
SEC 212791759
SEC 175397975
SEC 144056399
SEC 152811647
SEC 172743615
SEC 87102575
SEC 256861831
SEC 219305279
SEC 251200503
SEC 220075071
SEC 65834439
SEC 271538823
SEC 48186863
SEC 26160711
SEC 49759319
SEC 27162647
SEC 75929615
SEC 76138903
SEC 239738239
SEC 242775495
SEC 108601327
SEC 249394791
SEC 114516951
SEC 98201199
SEC 49202247
SEC 135146735
SEC 242742111
SEC 205215527
SEC 175397983
SEC 165023359
SEC 267063351
SEC 89558175
SEC 143827159
SEC 267204447
SEC 75929623
SEC 170588167
SEC 202746255
SEC 236218031
SEC 144056407
SEC 275665687
SEC 115031175
SEC 239738247
SEC 81488087
SEC 176951839
SEC 190974631
SEC 176792431
SEC 115031183
SEC 23128919
SEC 121435983
SEC 170588175
SEC 194908175
SEC 17597087
SEC 234808871
SEC 153930847
SEC 82738591
SEC 265632535
SEC 253193887
SEC 276456703
SEC 48976871
SEC 205702735
SEC 82738599
SEC 251200511
SEC 93751279
SEC 53040463
SEC 272470279
SEC 256861839
SEC 140850623
SEC 265632543
SEC 58910407
SEC 250529055
SEC 211170719
SEC 67384359
SEC 17597095
SEC 208947071
SEC 48186871
SEC 113554967
SEC 95508247
SEC 49202255
SEC 200871647
SEC 87102583
SEC 49759327
SEC 23128927
SEC 58910415
SEC 129848663
SEC 26160719
SEC 184452663
SEC 93751287
SEC 63671695
SEC 216894655
SEC 232399095
SEC 71789735
SEC 186815127
SEC 58973287
SEC 88509967
SEC 202937135
SEC 216745727
SEC 84655239
SEC 58973295
SEC 81488095
SEC 199440015
SEC 89558183
SEC 84655247
SEC 101585639
SEC 45626551
SEC 106101439
SEC 147925407
SEC 205215535
SEC 106101447
SEC 210120855
SEC 184452671
SEC 271466231
SEC 135146743
SEC 205702743
SEC 153930855
SEC 201810199
SEC 271466239
SEC 120949551
SEC 141212623
SEC 208947079
SEC 242742119
SEC 165023367
SEC 141212631
SEC 241730447
SEC 216745735
SEC 242758559
SEC 190974639
SEC 121124703
SEC 200871655
SEC 232399103
SEC 202746263
SEC 234808879
SEC 236505823
SEC 258446567
SEC 216894663
SEC 236218039
SEC 19118735
SEC 253193895
SEC 250529063
SEC 90794815
SEC 19118743
SEC 104516335
SEC 178977367
SEC 104452679
SEC 112676663
SEC 45626559
SEC 259307255
SEC 178977375
SEC 136701799
SEC 93659759
SEC 123873487
SEC 276456711
SEC 53040471
SEC 71789743
SEC 181178239
SEC 56431959
SEC 114157287
SEC 51378815
SEC 274711519
SEC 95508255
SEC 88509975
SEC 51378823
SEC 220852079
SEC 52807527
SEC 101585647
SEC 119318751
SEC 85109583
SEC 113554975
SEC 90794823
SEC 225582591
SEC 104516343
SEC 121124711
SEC 93659767
SEC 121435991
SEC 225582599
SEC 112676671
SEC 34691327
SEC 106607655
SEC 196873975
SEC 119318759
SEC 114157295
SEC 34691335
SEC 275217983
SEC 120949559
SEC 140850631
SEC 123873495
SEC 136701807
SEC 236907215
SEC 176951847
SEC 110377903
SEC 275217991
SEC 77873071
SEC 110377911
SEC 256461911
SEC 147925415
SEC 201810207
SEC 194908183
SEC 77873079
SEC 58790607
SEC 203247767
SEC 202126631
SEC 229539327
SEC 255926639
SEC 196582031
SEC 171528399
SEC 229539335
SEC 202937143
SEC 46584079
SEC 181178247
SEC 196582039
SEC 81709495
SEC 16203895
SEC 250269335
SEC 167129847
SEC 206498959
SEC 157103487
SEC 210120863
SEC 33941655
SEC 61832119
SEC 211170727
SEC 51080359
SEC 20052727
SEC 241730455
SEC 61832127
SEC 205165335
SEC 214632423
SEC 122824343
SEC 20052735
SEC 140191023
SEC 72637647
SEC 81807879
SEC 220852087
SEC 274711527
SEC 72637655
SEC 128485151
SEC 272470287
SEC 236505831
SEC 26748951
SEC 65279047
SEC 117772527
SEC 243127655
SEC 33941663
SEC 16203903
SEC 46584087
SEC 52807535
SEC 51080367
SEC 63747023
SEC 9727031
SEC 154125591
SEC 56431967
SEC 248621495
SEC 81709503
SEC 58790615
SEC 122824351
SEC 81807887
SEC 85109591
SEC 128485159
SEC 28894271
SEC 212288703
SEC 244553023
SEC 240170887
SEC 104452687
SEC 20195519
SEC 140191031
SEC 106607663
SEC 87504743
SEC 60974015
SEC 154125599
SEC 70560551
SEC 106535679
SEC 243848543
SEC 183553015
SEC 106535687
SEC 84613951
SEC 214632431
SEC 20081295
SEC 230841647
SEC 84613959
SEC 224588031
SEC 214736271
SEC 185091447
SEC 157103495
SEC 230841655
SEC 86633039
SEC 214736279
SEC 165450671
SEC 255926647
SEC 62035695
SEC 167129855
SEC 155598055
SEC 88750831
SEC 171528407
SEC 145660055
SEC 155598063
SEC 196873983
SEC 80230447
SEC 88750839
SEC 157307751
SEC 206498967
SEC 70322287
SEC 132340807
SEC 28894279
SEC 145660063
SEC 108340719
SEC 37795543
SEC 188785751
SEC 202126639
SEC 148319719
SEC 80230455
SEC 31851487
SEC 236907223
SEC 203247775
SEC 60974023
SEC 205165343
SEC 250269343
SEC 132340815
SEC 233859647
SEC 270652639
SEC 225418159
SEC 35692287
SEC 76047119
SEC 256461919
SEC 109379583
SEC 59689919
SEC 9727039
SEC 225418167
SEC 149748295
SEC 259307263
SEC 20081303
SEC 103313047
SEC 20195527
SEC 82232647
SEC 26748959
SEC 35692295
SEC 150942231
SEC 185091455
SEC 37795551
SEC 254427015
SEC 119379759
SEC 188785759
SEC 49133999
SEC 149748303
SEC 176409007
SEC 212288711
SEC 59689927
SEC 108707647
SEC 53238151
SEC 70322295
SEC 254427023
SEC 70560559
SEC 61420463
SEC 41722847
SEC 257685167
SEC 62035703
SEC 49134007
SEC 270024007
SEC 63747031
SEC 11128215
SEC 150493111
SEC 53238159
SEC 148918551
SEC 86633047
SEC 65279055
SEC 31851495
SEC 108707655
SEC 79049759
SEC 18028527
SEC 197659967
SEC 87504751
SEC 148319727
SEC 102963759
SEC 231653911
SEC 270024015
SEC 178557551
SEC 95239991
SEC 76047127
SEC 183553023
SEC 61784455
SEC 117772535
SEC 102963767
SEC 190999695
SEC 82232655
SEC 25147095
SEC 95239999
SEC 8835991
SEC 22143575
SEC 252103959
SEC 233859655
SEC 108340727
SEC 157307759
SEC 141336751
SEC 109379591
SEC 165450679
SEC 248621503
SEC 173172303
SEC 176583135
SEC 37392095
SEC 162253743
SEC 22143583
SEC 251391223
SEC 223742871
SEC 193250671
SEC 134560935
SEC 209745527
SEC 30787143
SEC 41722855
SEC 134560943
SEC 108068639
SEC 61784463
SEC 11128223
SEC 224588039
SEC 108068647
SEC 28391807
SEC 200923263
SEC 270504303
SEC 71219791
SEC 240170895
SEC 185298527
SEC 28391815
SEC 209693343
SEC 243127663
SEC 200923271
SEC 37392103
SEC 243848551
SEC 86238191
SEC 267178911
SEC 71219799
SEC 157982879
SEC 21306431
SEC 245068151
SEC 222179039
SEC 244553031
SEC 96866095
SEC 83571471
SEC 153955399
SEC 49938543
SEC 141336759
SEC 21306439
SEC 261117143
SEC 103313055
SEC 216100911
SEC 270652647
SEC 173172311
SEC 111395999
SEC 162253751
SEC 193250679
SEC 176409015
SEC 8835999
SEC 182721087
SEC 112948215
SEC 18028535
SEC 185298535
SEC 182721095
SEC 92534743
SEC 231653919
SEC 25147103
SEC 190999703
SEC 237512511
SEC 197659975
SEC 45690991
SEC 61420471
SEC 195375095
SEC 30787151
SEC 209693351
SEC 16479719
SEC 223742879
SEC 78461455
SEC 59330903
SEC 264401663
SEC 91499119
SEC 49938551
SEC 79049767
SEC 78461463
SEC 18776463
SEC 47245743
SEC 83571479
SEC 251391231
SEC 86238199
SEC 59756255
SEC 252103967
SEC 50773639
SEC 213929823
SEC 119379767
SEC 157982887
SEC 50773647
SEC 170143759
SEC 215533183
SEC 148918559
SEC 124077087
SEC 216210495
SEC 150493119
SEC 18776471
SEC 216210503
SEC 87799663
SEC 47245751
SEC 150942239
SEC 219983815
SEC 59756263
SEC 103870319
[-- Attachment #4: thread_seek.c --]
[-- Type: text/plain, Size: 2626 bytes --]
#define _FILE_OFFSET_BITS 64
#define _LARGEFILE_SOURCE
#define _LARGEFILE64_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdint.h>
#include <error.h>
#include <errno.h>
#include <assert.h>
#include <pthread.h>
#include <semaphore.h>
#define ERR(args...) fprintf(stderr, args);
#define READSIZE 4096
#define FILENAME "./bigfile"
int nthreads = 64;
extern int errno;
int fds[1024];
pthread_mutex_t locks[1024];
uint64_t filesize;
sem_t sem;
void * thread(void *arg)
{
char buf[READSIZE];
off_t offset;
int index, fd, ret;
fd = 0;
do {
for(index = 0; index < nthreads; index++) {
if(pthread_mutex_trylock(&locks[index]) == 0) {
// Found a free file descriptor
fd = fds[index];
break;
}
}
} while(fd == 0);
offset = (((uint64_t)random() << 32) | random()) % (filesize - READSIZE);
assert(offset < filesize - READSIZE);
if(lseek(fd, offset, SEEK_SET) == -1) {
ERR("thread(): lseek(): %s\n", strerror(errno));
goto done;
}
if((ret = read(fd, buf, READSIZE)) != READSIZE) {
if(ret == -1) {
ERR("thread(): read(): %s\n", strerror(errno));
} else if(ret == 0) {
ERR("thread(): read(): EOF\n");
} else {
ERR("thread(): read(): Only read %d of %d bytes\n", ret, READSIZE);
}
}
done:
pthread_mutex_unlock(&locks[index]);
sem_post(&sem);
}
int main(int argc, char **argv)
{
pthread_attr_t attr;
pthread_t child;
struct stat fstat;
int i;
if(argc == 2) {
nthreads = atoi(argv[1]);
assert(nthreads < 1024);
}
if(pthread_attr_init(&attr) != 0) {
ERR("main(): pthread_attr_init(): failed\n");
exit(1);
}
if(pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED) != 0) {
ERR("main(): pthread_attr_setdetachstate(): failed\n");
exit(1);
}
if(stat(FILENAME, &fstat) == -1) {
ERR("main(): stat(): %s\n", strerror(errno));
exit(1);
}
filesize = fstat.st_size;
for(i = 0; i < nthreads; i++) {
if((fds[i] = open(FILENAME, O_RDONLY | O_LARGEFILE)) == -1) {
ERR("main(): open(): %s\n", strerror(errno));
exit(1);
}
if(pthread_mutex_init(&locks[i], NULL) != 0) {
ERR("main(): pthread_mutex_init(): %s\n", strerror(errno));
exit(1);
}
}
if(sem_init(&sem, 0, nthreads) == -1) {
ERR("main(): sem_init(): %s\n", strerror(errno));
exit(1);
}
while(1) {
if(sem_wait(&sem) == 0) {
if(pthread_create(&child, &attr, thread, NULL) != 0) {
ERR("main(): pthread_create(): failed\n");
exit(1);
}
}
}
}
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2003-09-29 14:56 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-08-26 17:32 2.6.0-test4 parallel seek & read problem Magnus Andersson
2003-09-29 14:55 ` Magnus Andersson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox