2.6.{26.2,27-rc} oops on virtualbox

public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed

* 2.6.{26.2,27-rc} oops on virtualbox
@ 2008-08-20 19:29 Luiz Fernando N. Capitulino
  2008-08-21 21:34 ` H. Peter Anvin
  0 siblings, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-20 19:29 UTC (permalink / raw)
  To: linux-kernel; +Cc: mathieu.desnoyers, mingo

 Hi there,

 Users of different Linux distros (includes Ubuntu, Mandriva, ArchLinux and
possibly Fedora) are reporting that kernel 2.6.26.2 is OOPSing in the
Virtual Box emulator[1].

 It is not clear if this is a kernel or Virtual Box bug, but as the kernel
is also OOPsing in QEMU (although with different behaivor) I have decided
to post my debug results here in case someone is interested in debugging
the kernel part further.

 I have done a bisection by hand among kernel versions and found that
the commit which triggers the oops in _Virtual Box_ was introduced in
2.6.26-rc1 and the problem also happens with latest Linus tree.

 By using git bisect I found that the commit is this:

"""
commit e587cadd8f47e202a30712e2906a65a0606d5865
Author: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Date:   Thu Mar 6 08:48:49 2008 -0500

    x86: enhance DEBUG_RODATA support - alternatives

[...]

"""

 By reverting this commit I don't get the OOPS anymore. I have
tested with 2.6.26-rc1 and latest Linus tree (2.6.27-rc3).

 What puzzles me though is that a similar problem happens with
QEMU, but it also OOPSes with kernels before 2.6.26-rc1,
reverting the patch above makes no difference and it works with
current Linus tree.

 Does this look like a kernel bug?

 All my tests have been done with vanilla kernels, but I have
built .iso installation images with them and I'm not sure of
what the build script does.

[1] http://en.wikipedia.org/wiki/Virtual_box

 Thanks for reading this.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-20 19:29 2.6.{26.2,27-rc} oops on virtualbox Luiz Fernando N. Capitulino
@ 2008-08-21 21:34 ` H. Peter Anvin
  2008-08-22  6:42   ` H. Peter Anvin
  2008-08-22 14:28   ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-21 21:34 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: linux-kernel, mathieu.desnoyers, mingo

[-- Attachment #1: Type: text/plain, Size: 141 bytes --]

> 
>  Does this look like a kernel bug?
> 

No, it looks like a very common virtualizer bug.  Does the attached 
patch work for you?

	-hpa


[-- Attachment #2: diff --]
[-- Type: text/plain, Size: 386 bytes --]

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 2763cb3..33193fe 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -157,8 +157,6 @@ static const struct nop {
 } noptypes[] = {
 	{ X86_FEATURE_K8, k8_nops },
 	{ X86_FEATURE_K7, k7_nops },
-	{ X86_FEATURE_P4, p6_nops },
-	{ X86_FEATURE_P3, p6_nops },
 	{ -1, NULL }
 };
 

^ permalink raw reply related	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-21 21:34 ` H. Peter Anvin
@ 2008-08-22  6:42   ` H. Peter Anvin
  2008-08-22  6:50     ` Ingo Molnar
  2008-08-22 14:28   ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22  6:42 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: linux-kernel, mathieu.desnoyers, mingo

H. Peter Anvin wrote:
>>
>>  Does this look like a kernel bug?
>>
> 
> No, it looks like a very common virtualizer bug.  Does the attached 
> patch work for you?
> 

Also, in addition to this, please try tip:master.  There is a patch in 
tip:master which I hope should fix this problem, but the details are 
important.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22  6:42   ` H. Peter Anvin
@ 2008-08-22  6:50     ` Ingo Molnar
  2008-08-22 14:39       ` Luiz Fernando N. Capitulino
  0 siblings, 1 reply; 52+ messages in thread
From: Ingo Molnar @ 2008-08-22  6:50 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Luiz Fernando N. Capitulino, linux-kernel, mathieu.desnoyers


* H. Peter Anvin <hpa@zytor.com> wrote:

> H. Peter Anvin wrote:
>>>
>>>  Does this look like a kernel bug?
>>>
>>
>> No, it looks like a very common virtualizer bug.  Does the attached  
>> patch work for you?
>>
>
> Also, in addition to this, please try tip:master.  There is a patch in 
> tip:master which I hope should fix this problem, but the details are 
> important.

access coordinates would be at:

  http://people.redhat.com/mingo/tip.git/README

	Ingo

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22  6:50     ` Ingo Molnar
@ 2008-08-22 14:39       ` Luiz Fernando N. Capitulino
  2008-08-22 15:34         ` Mathieu Desnoyers
  0 siblings, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 14:39 UTC (permalink / raw)
  To: Ingo Molnar; +Cc: H. Peter Anvin, linux-kernel, mathieu.desnoyers

[-- Attachment #1: Type: text/plain, Size: 2706 bytes --]

Em Fri, 22 Aug 2008 08:50:12 +0200
Ingo Molnar <mingo@elte.hu> escreveu:

| 
| * H. Peter Anvin <hpa@zytor.com> wrote:
| 
| > H. Peter Anvin wrote:
| >>>
| >>>  Does this look like a kernel bug?
| >>>
| >>
| >> No, it looks like a very common virtualizer bug.  Does the attached  
| >> patch work for you?
| >>
| >
| > Also, in addition to this, please try tip:master.  There is a patch in 
| > tip:master which I hope should fix this problem, but the details are 
| > important.
| 
| access coordinates would be at:
| 
|   http://people.redhat.com/mingo/tip.git/README

 As I already have Linus tree downloaded I have cloned it in
the usual way.

 Got the same results: OOPS in virtualbox but it works on QEMU.

 The OOPS's output follows and I have attached the .config I'm using
to reproduce the problem.

"""
BUG: unable to handle kernel NULL pointer dereference at 00000246
IP: [<c01310f1>] vprintk+0x181/0x440
*pde = 00000000 
Oops: 0002 [#1] SMP 
Modules linked in:

Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test24-tip #3)
EIP: 0060:[<c01310f1>] EFLAGS: 00010246 CPU: 0
EIP is at vprintk+0x181/0x440
EAX: 00000246 EBX: 00000000 ECX: c0130ca9 EDX: 0000dedd
ESI: c0474ae3 EDI: c04cf6bc EBP: c7435f24 ESP: c7435eb0
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
Stack: 0000dedd c0130ca9 c7435f40 00000000 a026104f a026106c c7434000 c7435ee6 
       00000006 00000246 00000000 a0260cf3 0000001c c7434000 00000282 00000046 
       c11a85a0 c7435efc c0135c6f c7435f14 c0115fcb a0296e91 c0104c2c 00000000 
Call Trace:
 [<c0130ca9>] ? release_console_sem+0x199/0x1e0
 [<c0135c6f>] ? irq_exit+0x3f/0x90
 [<c0115fcb>] ? smp_apic_timer_interrupt+0x5b/0x90
 [<c0104c2c>] ? apic_timer_interrupt+0x28/0x30
 [<c0474ae3>] ? net_ns_init+0x0/0x1ad
 [<c0474ae3>] ? net_ns_init+0x0/0x1ad
 [<c0346ed9>] ? printk+0x18/0x1f
 [<c0474b00>] ? net_ns_init+0x1d/0x1ad
 [<c0474ae3>] ? net_ns_init+0x0/0x1ad
 [<c0101116>] ? do_one_initcall+0x26/0x170
 [<c0128f66>] ? try_to_wake_up+0xc6/0x240
 [<c012910f>] ? wake_up_process+0xf/0x20
 [<c014192d>] ? start_workqueue_thread+0x1d/0x20
 [<c0141d4b>] ? __create_workqueue_key+0x1eb/0x240
 [<c0141820>] ? worker_thread+0x0/0xf0
 [<c044b387>] ? kernel_init+0x141/0x214
 [<c044b246>] ? kernel_init+0x0/0x214
 [<c0104dc7>] ? kernel_thread_helper+0x7/0x10
 =======================
Code: c0 0f 84 0b 01 00 00 b8 50 f1 41 c0 c7 05 ec f1 41 c0 ff ff ff ff e8 cf 8b 21 00 e8 ea 04 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 ec f1 41 c0 e8 
EIP: [<c01310f1>] vprintk+0x181/0x440 SS:ESP 0069:c7435eb0
---[ end trace 4eaa2a86a8e2da22 ]---
"""

-- 
Luiz Fernando N. Capitulino

[-- Attachment #2: config --]
[-- Type: application/octet-stream, Size: 56933 bytes --]

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.27-rc4-test24
# Fri Aug 22 10:57:41 2008
#
# CONFIG_64BIT is not set
CONFIG_X86_32=y
# CONFIG_X86_64 is not set
CONFIG_X86=y
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
# CONFIG_GENERIC_LOCKBREAK is not set
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_FAST_CMPXCHG_LOCAL=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
# CONFIG_GENERIC_GPIO is not set
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
# CONFIG_ARCH_HAS_ILOG2_U32 is not set
# CONFIG_ARCH_HAS_ILOG2_U64 is not set
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
# CONFIG_GENERIC_TIME_VSYSCALL is not set
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
# CONFIG_HAVE_CPUMASK_OF_CPU_MAP is not set
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
# CONFIG_ZONE_DMA32 is not set
CONFIG_ARCH_POPULATES_NODE_MAP=y
# CONFIG_AUDIT_ARCH is not set
CONFIG_ARCH_SUPPORTS_AOUT=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_X86_SMP=y
CONFIG_X86_32_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_KTIME_SCALAR=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_AUDIT_TREE=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=17
# CONFIG_CGROUPS is not set
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
# CONFIG_GROUP_SCHED is not set
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y
CONFIG_RELAY=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
# CONFIG_FASTBOOT is not set
CONFIG_SYSCTL=y
CONFIG_EMBEDDED=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
# CONFIG_PCSPKR_PLATFORM is not set
# CONFIG_COMPAT_BRK is not set
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_ANON_INODES=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_SLUB_DEBUG=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
# CONFIG_MARKERS is not set
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
# CONFIG_HAVE_DMA_ATTRS is not set
CONFIG_USE_GENERIC_SMP_HELPERS=y
# CONFIG_HAVE_CLK is not set
CONFIG_HAVE_DYN_ARRAY=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_HAVE_GENERIC_DMA_COHERENT=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_LBD=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_LSF=y
CONFIG_BLK_DEV_BSG=y
# CONFIG_BLK_DEV_INTEGRITY is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_CLASSIC_RCU=y

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
# CONFIG_HAVE_SPARSE_IRQ is not set
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_VSMP is not set
# CONFIG_X86_RDC321X is not set
CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER=y
CONFIG_PARAVIRT_GUEST=y
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_VMI=y
# CONFIG_KVM_CLOCK is not set
# CONFIG_KVM_GUEST is not set
CONFIG_LGUEST_GUEST=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_CLOCK is not set
# CONFIG_PARAVIRT_DEBUG is not set
# CONFIG_MEMTEST is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
CONFIG_M686=y
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_GENERIC=y
CONFIG_X86_CPU=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_X86_XADD=y
CONFIG_X86_PPRO_FENCE=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=4
CONFIG_X86_DEBUGCTLMSR=y
# CONFIG_X86_DS is not set
# CONFIG_PROCESSOR_SELECT is not set
CONFIG_CPU_SUP_INTEL_32=y
CONFIG_CPU_SUP_CYRIX_32=y
CONFIG_CPU_SUP_AMD_32=y
CONFIG_CPU_SUP_CENTAUR_32=y
CONFIG_CPU_SUP_TRANSMETA_32=y
CONFIG_CPU_SUP_UMC_32=y
CONFIG_HPET_TIMER=y
CONFIG_DMI=y
# CONFIG_IOMMU_HELPER is not set
CONFIG_NR_CPUS=32
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=m
CONFIG_X86_MCE_P4THERMAL=y
CONFIG_VM86=y
CONFIG_TOSHIBA=m
CONFIG_I8K=m
CONFIG_X86_REBOOTFIXUPS=y
CONFIG_MICROCODE=m
# CONFIG_MICROCODE_INTEL is not set
# CONFIG_MICROCODE_AMD is not set
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_VMSPLIT_3G=y
# CONFIG_VMSPLIT_3G_OPT is not set
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_2G_OPT is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
# CONFIG_SPARSEMEM_VMEMMAP_ENABLE is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_RESOURCES_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
# CONFIG_HIGHPTE is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
# CONFIG_X86_PAT is not set
# CONFIG_EFI is not set
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
CONFIG_CRASH_DUMP=y
# CONFIG_KEXEC_JUMP is not set
CONFIG_PHYSICAL_START=0x100000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x100000
CONFIG_HOTPLUG_CPU=y
CONFIG_COMPAT_VDSO=y
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management options
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_SLEEP=y
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_HIBERNATION=y
CONFIG_PM_STD_PARTITION=""
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROCFS_POWER=y
CONFIG_ACPI_SYSFS_POWER=y
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_DOCK=m
CONFIG_ACPI_BAY=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_THERMAL=m
CONFIG_ACPI_WMI=m
CONFIG_ACPI_ASUS=m
CONFIG_ACPI_TOSHIBA=m
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=m
CONFIG_ACPI_SBS=m
CONFIG_X86_APM_BOOT=y
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
CONFIG_APM_CPU_IDLE=y
# CONFIG_APM_DISPLAY_BLANK is not set
# CONFIG_APM_ALLOW_INTS is not set
# CONFIG_APM_REAL_MODE_POWER_OFF is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=m
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=m
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=m
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m

#
# CPUFreq processor drivers
#
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_POWERNOW_K6=m
CONFIG_X86_POWERNOW_K7=m
CONFIG_X86_POWERNOW_K7_ACPI=y
CONFIG_X86_POWERNOW_K8=m
CONFIG_X86_POWERNOW_K8_ACPI=y
CONFIG_X86_GX_SUSPMOD=m
CONFIG_X86_SPEEDSTEP_CENTRINO=m
CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
CONFIG_X86_SPEEDSTEP_ICH=m
CONFIG_X86_SPEEDSTEP_SMI=m
CONFIG_X86_P4_CLOCKMOD=m
CONFIG_X86_CPUFREQ_NFORCE2=m
CONFIG_X86_LONGRUN=m
CONFIG_X86_LONGHAUL=m
CONFIG_X86_E_POWERSAVER=m

#
# shared options
#
# CONFIG_X86_ACPI_CPUFREQ_PROC_INTF is not set
CONFIG_X86_SPEEDSTEP_LIB=m
CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK=y
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
# CONFIG_PCI_GOOLPC is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
CONFIG_PCIEAER=y
# CONFIG_PCIEASPM is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_LEGACY=y
# CONFIG_PCI_DEBUG is not set
CONFIG_HT_IRQ=y
CONFIG_ISA_DMA_API=y
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
CONFIG_SCx200=m
CONFIG_SCx200HR_TIMER=m
# CONFIG_OLPC is not set
CONFIG_K8_NB=y
CONFIG_PCCARD=m
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_PCMCIA=m
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_PCMCIA_IOCTL=y
CONFIG_CARDBUS=y

#
# PC-card bridges
#
CONFIG_YENTA=m
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
CONFIG_PD6729=m
CONFIG_I82092=m
CONFIG_I82365=m
CONFIG_TCIC=m
CONFIG_PCMCIA_PROBE=y
CONFIG_PCCARD_NONSTATIC=m
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_COMPAQ=m
# CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM is not set
CONFIG_HOTPLUG_PCI_IBM=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=m

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=m
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=m
CONFIG_XFRM_SUB_POLICY=y
CONFIG_XFRM_MIGRATE=y
# CONFIG_XFRM_STATISTICS is not set
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
CONFIG_NET_KEY_MIGRATE=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_LRO=m
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_NETLABEL is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
CONFIG_SCTP_HMAC_NONE=y
# CONFIG_SCTP_HMAC_SHA1 is not set
# CONFIG_SCTP_HMAC_MD5 is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_SCHED is not set

#
# Network testing
#
CONFIG_NET_PKTGEN=m
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y

#
# Wireless
#
CONFIG_CFG80211=m
# CONFIG_NL80211 is not set
CONFIG_WIRELESS_EXT=y
# CONFIG_WIRELESS_EXT_SYSFS is not set
# CONFIG_MAC80211 is not set
CONFIG_IEEE80211=m
# CONFIG_IEEE80211_DEBUG is not set
CONFIG_IEEE80211_CRYPT_WEP=m
# CONFIG_IEEE80211_CRYPT_CCMP is not set
# CONFIG_IEEE80211_CRYPT_TKIP is not set
CONFIG_RFKILL=m
# CONFIG_RFKILL_INPUT is not set
CONFIG_RFKILL_LEDS=y
# CONFIG_NET_9P is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_STANDALONE=y
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=y
# CONFIG_FIRMWARE_IN_KERNEL is not set
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_CONNECTOR=m
# CONFIG_MTD is not set
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_SERIAL=m
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
CONFIG_PARPORT_PC_PCMCIA=m
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_AX88796=m
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_ISAPNP=y
CONFIG_PNPBIOS=y
CONFIG_PNPBIOS_PROC_FS=y
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_FD=m
CONFIG_BLK_DEV_XD=m
CONFIG_PARIDE=m

#
# Parallel IDE high-level drivers
#
CONFIG_PARIDE_PD=m
CONFIG_PARIDE_PCD=m
CONFIG_PARIDE_PF=m
CONFIG_PARIDE_PT=m
CONFIG_PARIDE_PG=m

#
# Parallel IDE protocol modules
#
CONFIG_PARIDE_ATEN=m
CONFIG_PARIDE_BPCK=m
CONFIG_PARIDE_BPCK6=m
CONFIG_PARIDE_COMM=m
CONFIG_PARIDE_DSTR=m
CONFIG_PARIDE_FIT2=m
CONFIG_PARIDE_FIT3=m
CONFIG_PARIDE_EPAT=m
CONFIG_PARIDE_EPATC8=y
CONFIG_PARIDE_EPIA=m
CONFIG_PARIDE_FRIQ=m
CONFIG_PARIDE_FRPW=m
CONFIG_PARIDE_KBIC=m
CONFIG_PARIDE_KTTI=m
CONFIG_PARIDE_ON20=m
CONFIG_PARIDE_ON26=m
CONFIG_BLK_CPQ_DA=m
CONFIG_BLK_CPQ_CISS_DA=m
CONFIG_CISS_SCSI_TAPE=y
CONFIG_BLK_DEV_DAC960=m
CONFIG_BLK_DEV_UMEM=m
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_SX8=m
CONFIG_BLK_DEV_UB=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=32000
CONFIG_BLK_DEV_XIP=y
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
CONFIG_ATA_OVER_ETH=m
CONFIG_VIRTIO_BLK=m
# CONFIG_BLK_DEV_HD is not set
CONFIG_MISC_DEVICES=y
CONFIG_IBM_ASM=m
CONFIG_PHANTOM=m
CONFIG_EEPROM_93CX6=m
CONFIG_SGI_IOC4=m
CONFIG_TIFM_CORE=m
CONFIG_TIFM_7XX1=m
CONFIG_ACER_WMI=m
CONFIG_ASUS_LAPTOP=m
CONFIG_FUJITSU_LAPTOP=m
# CONFIG_FUJITSU_LAPTOP_DEBUG is not set
CONFIG_TC1100_WMI=m
# CONFIG_HP_WMI is not set
CONFIG_MSI_LAPTOP=m
# CONFIG_COMPAL_LAPTOP is not set
CONFIG_SONY_LAPTOP=m
# CONFIG_SONYPI_COMPAT is not set
CONFIG_THINKPAD_ACPI=m
# CONFIG_THINKPAD_ACPI_DEBUG is not set
CONFIG_THINKPAD_ACPI_BAY=y
CONFIG_THINKPAD_ACPI_VIDEO=y
CONFIG_THINKPAD_ACPI_HOTKEY_POLL=y
CONFIG_INTEL_MENLOW=m
# CONFIG_EEEPC_LAPTOP is not set
CONFIG_ENCLOSURE_SERVICES=m
# CONFIG_HP_ILO is not set
CONFIG_HAVE_IDE=y
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=m

#
# Please see Documentation/ide/ide.txt for help/info on IDE drives
#
CONFIG_IDE_TIMINGS=y
CONFIG_IDE_ATAPI=y
# CONFIG_BLK_DEV_IDE_SATA is not set
CONFIG_BLK_DEV_IDEDISK=m
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECS=m
CONFIG_BLK_DEV_DELKIN=m
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDECD_VERBOSE_ERRORS=y
CONFIG_BLK_DEV_IDETAPE=m
CONFIG_BLK_DEV_IDEFLOPPY=m
CONFIG_BLK_DEV_IDESCSI=m
CONFIG_BLK_DEV_IDEACPI=y
CONFIG_IDE_TASK_IOCTL=y
CONFIG_IDE_PROC_FS=y

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
CONFIG_BLK_DEV_PLATFORM=m
CONFIG_BLK_DEV_CMD640=m
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
# CONFIG_BLK_DEV_IDEPNP is not set
CONFIG_BLK_DEV_IDEDMA_SFF=y

#
# PCI IDE chipsets support
#
CONFIG_BLK_DEV_IDEPCI=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=m
CONFIG_BLK_DEV_OPTI621=m
CONFIG_BLK_DEV_RZ1000=m
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_BLK_DEV_AEC62XX=m
CONFIG_BLK_DEV_ALI15X3=m
CONFIG_BLK_DEV_AMD74XX=m
CONFIG_BLK_DEV_ATIIXP=m
CONFIG_BLK_DEV_CMD64X=m
CONFIG_BLK_DEV_TRIFLEX=m
CONFIG_BLK_DEV_CS5520=m
CONFIG_BLK_DEV_CS5530=m
CONFIG_BLK_DEV_CS5535=m
CONFIG_BLK_DEV_HPT366=m
CONFIG_BLK_DEV_JMICRON=m
CONFIG_BLK_DEV_SC1200=m
CONFIG_BLK_DEV_PIIX=m
CONFIG_BLK_DEV_IT8213=m
CONFIG_BLK_DEV_IT821X=m
CONFIG_BLK_DEV_NS87415=m
CONFIG_BLK_DEV_PDC202XX_OLD=m
CONFIG_BLK_DEV_PDC202XX_NEW=m
CONFIG_BLK_DEV_SVWKS=m
CONFIG_BLK_DEV_SIIMAGE=m
CONFIG_BLK_DEV_SIS5513=m
CONFIG_BLK_DEV_SLC90E66=m
CONFIG_BLK_DEV_TRM290=m
CONFIG_BLK_DEV_VIA82CXXX=m
CONFIG_BLK_DEV_TC86C001=m

#
# Other IDE chipsets support
#

#
# Note: most of these also require special kernel boot parameters
#
# CONFIG_BLK_DEV_4DRIVES is not set
CONFIG_BLK_DEV_ALI14XX=m
CONFIG_BLK_DEV_DTC2278=m
CONFIG_BLK_DEV_HT6560B=m
CONFIG_BLK_DEV_QD65XX=m
CONFIG_BLK_DEV_UMC8672=m
CONFIG_BLK_DEV_IDEDMA=y

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=m
CONFIG_SCSI_DMA=y
CONFIG_SCSI_TGT=m
# CONFIG_SCSI_NETLINK is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
# CONFIG_BLK_DEV_SD is not set
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
# CONFIG_CHR_DEV_SG is not set
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_ENCLOSURE is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
# CONFIG_SCSI_DH is not set
CONFIG_ATA=m
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y
CONFIG_SATA_AHCI=m
CONFIG_SATA_SIL24=m
# CONFIG_ATA_SFF is not set
# CONFIG_MD is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#

#
# Enable only one of the two stacks, unless you know what you are doing
#
# CONFIG_FIREWIRE is not set
# CONFIG_IEEE1394 is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_BONDING=m
CONFIG_MACVLAN=m
CONFIG_EQUALIZER=m
CONFIG_TUN=m
CONFIG_VETH=m
CONFIG_NET_SB1000=m
# CONFIG_ARCNET is not set
CONFIG_PHYLIB=m

#
# MII PHY device drivers
#
CONFIG_MARVELL_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_QSEMI_PHY=m
CONFIG_LXT_PHY=m
CONFIG_CICADA_PHY=m
CONFIG_VITESSE_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_BROADCOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_MDIO_BITBANG=m
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_EL1=m
CONFIG_EL2=m
CONFIG_ELPLUS=m
CONFIG_EL16=m
CONFIG_EL3=m
CONFIG_3C515=m
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
CONFIG_LANCE=m
CONFIG_NET_VENDOR_SMC=y
CONFIG_ULTRA=m
CONFIG_SMC9194=m
CONFIG_NET_VENDOR_RACAL=y
CONFIG_NI52=m
CONFIG_NI65=m
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_PCMCIA_XIRCOM=m
CONFIG_AT1700=m
CONFIG_DEPCA=m
CONFIG_HP100=m
CONFIG_NET_ISA=y
CONFIG_E2100=m
CONFIG_EWRK3=m
CONFIG_EEXPRESS=m
CONFIG_EEXPRESS_PRO=m
CONFIG_HPLAN=m
CONFIG_LP486E=m
CONFIG_ETH16I=m
CONFIG_NE2000=m
CONFIG_ZNET=m
CONFIG_SEEQ8005=m
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
CONFIG_ADAPTEC_STARFIRE=m
CONFIG_AC3200=m
CONFIG_APRICOT=m
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
CONFIG_FORCEDETH=m
CONFIG_FORCEDETH_NAPI=y
CONFIG_CS89x0=m
CONFIG_EEPRO100=m
CONFIG_E100=m
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_R6040=m
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_TLAN=m
CONFIG_VIA_RHINE=m
# CONFIG_VIA_RHINE_MMIO is not set
CONFIG_SC92031=m
CONFIG_NET_POCKET=y
CONFIG_ATP=m
CONFIG_DE600=m
CONFIG_DE620=m
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
CONFIG_DL2K=m
CONFIG_E1000=m
# CONFIG_E1000_DISABLE_PACKET_SPLIT is not set
CONFIG_E1000E=m
CONFIG_IP1000=m
CONFIG_IGB=m
# CONFIG_IGB_LRO is not set
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=m
CONFIG_SIS190=m
CONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=m
CONFIG_BNX2=m
CONFIG_QLA3XXX=m
CONFIG_ATL1=m
# CONFIG_ATL1E is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set

#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
# CONFIG_IWLWIFI_LEDS is not set

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
# CONFIG_USB_HSO is not set
# CONFIG_NET_PCMCIA is not set
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
# CONFIG_PPP_MULTILINK is not set
# CONFIG_PPP_FILTER is not set
# CONFIG_PPP_ASYNC is not set
# CONFIG_PPP_SYNC_TTY is not set
# CONFIG_PPP_DEFLATE is not set
# CONFIG_PPP_BSDCOMP is not set
# CONFIG_PPP_MPPE is not set
# CONFIG_PPPOE is not set
# CONFIG_PPPOL2TP is not set
# CONFIG_SLIP is not set
CONFIG_SLHC=m
# CONFIG_NET_FC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_VIRTIO_NET is not set
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=m
CONFIG_INPUT_POLLDEV=m

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_EVDEV=m
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_KEYBOARD_STOWAWAY=m
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_TOUCHKIT=y
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_APPLETOUCH=m
# CONFIG_MOUSE_BCM5974 is not set
CONFIG_MOUSE_INPORT=m
CONFIG_MOUSE_ATIXL=y
CONFIG_MOUSE_LOGIBM=m
CONFIG_MOUSE_PC110PAD=m
CONFIG_MOUSE_VSXXXAA=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=y
CONFIG_JOYSTICK_IFORCE_232=y
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
CONFIG_JOYSTICK_TWIDJOY=m
# CONFIG_JOYSTICK_ZHENHUA is not set
CONFIG_JOYSTICK_DB9=m
CONFIG_JOYSTICK_GAMECON=m
CONFIG_JOYSTICK_TURBOGRAFX=m
CONFIG_JOYSTICK_JOYDUMP=m
CONFIG_JOYSTICK_XPAD=m
CONFIG_JOYSTICK_XPAD_FF=y
CONFIG_JOYSTICK_XPAD_LEDS=y
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
CONFIG_TABLET_USB_GTCO=m
CONFIG_TABLET_USB_KBTAB=m
CONFIG_TABLET_USB_WACOM=m
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_FUJITSU=m
CONFIG_TOUCHSCREEN_GUNZE=m
CONFIG_TOUCHSCREEN_ELO=m
CONFIG_TOUCHSCREEN_MTOUCH=m
# CONFIG_TOUCHSCREEN_INEXIO is not set
CONFIG_TOUCHSCREEN_MK712=m
# CONFIG_TOUCHSCREEN_HTCPEN is not set
CONFIG_TOUCHSCREEN_PENMOUNT=m
CONFIG_TOUCHSCREEN_TOUCHRIGHT=m
CONFIG_TOUCHSCREEN_TOUCHWIN=m
CONFIG_TOUCHSCREEN_UCB1400=m
# CONFIG_TOUCHSCREEN_WM97XX is not set
CONFIG_TOUCHSCREEN_USB_COMPOSITE=m
CONFIG_TOUCHSCREEN_USB_EGALAX=y
CONFIG_TOUCHSCREEN_USB_PANJIT=y
CONFIG_TOUCHSCREEN_USB_3M=y
CONFIG_TOUCHSCREEN_USB_ITM=y
CONFIG_TOUCHSCREEN_USB_ETURBO=y
CONFIG_TOUCHSCREEN_USB_GUNZE=y
CONFIG_TOUCHSCREEN_USB_DMC_TSC10=y
CONFIG_TOUCHSCREEN_USB_IRTOUCH=y
CONFIG_TOUCHSCREEN_USB_IDEALTEK=y
CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y
CONFIG_TOUCHSCREEN_USB_GOTOP=y
# CONFIG_TOUCHSCREEN_TOUCHIT213 is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_APANEL=m
CONFIG_INPUT_WISTRON_BTNS=m
CONFIG_INPUT_ATLAS_BTNS=m
CONFIG_INPUT_ATI_REMOTE=m
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
CONFIG_INPUT_POWERMATE=m
CONFIG_INPUT_YEALINK=m
CONFIG_INPUT_UINPUT=m

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PARKBD=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_DEVKMEM is not set
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_COMPUTONE=m
CONFIG_ROCKETPORT=m
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
CONFIG_DIGIEPCA=m
CONFIG_ESPSERIAL=m
CONFIG_MOXA_INTELLIO=m
CONFIG_MOXA_SMARTIO=m
CONFIG_ISI=m
CONFIG_SYNCLINK=m
CONFIG_SYNCLINKMP=m
CONFIG_SYNCLINK_GT=m
CONFIG_N_HDLC=m
CONFIG_RISCOM8=m
CONFIG_SPECIALIX=m
CONFIG_SX=m
CONFIG_RIO=m
# CONFIG_RIO_OLDPCI is not set
CONFIG_STALDRV=y
CONFIG_NOZOMI=m

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
# CONFIG_SERIAL_8250_MANY_PORTS is not set
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_JSM=m
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_PRINTER=m
CONFIG_LP_CONSOLE=y
CONFIG_PPDEV=m
CONFIG_HVC_DRIVER=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_PANIC_EVENT=y
CONFIG_IPMI_PANIC_STRING=y
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
CONFIG_HW_RANDOM=m
CONFIG_HW_RANDOM_INTEL=m
CONFIG_HW_RANDOM_AMD=m
CONFIG_HW_RANDOM_GEODE=m
CONFIG_HW_RANDOM_VIA=m
# CONFIG_HW_RANDOM_VIRTIO is not set
CONFIG_NVRAM=m
# CONFIG_RTC is not set
# CONFIG_GEN_RTC is not set
CONFIG_DTLK=m
CONFIG_R3964=m
CONFIG_APPLICOM=m
CONFIG_SONYPI=m

#
# PCMCIA character devices
#
CONFIG_SYNCLINK_CS=m
CONFIG_CARDMAN_4000=m
CONFIG_CARDMAN_4040=m
CONFIG_IPWIRELESS=m
CONFIG_MWAVE=m
CONFIG_SCx200_GPIO=m
CONFIG_PC8736x_GPIO=m
CONFIG_NSC_GPIO=m
CONFIG_CS5535_GPIO=m
CONFIG_RAW_DRIVER=m
CONFIG_MAX_RAW_DEVS=256
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=m
CONFIG_TCG_TPM=m
CONFIG_TCG_TIS=m
CONFIG_TCG_NSC=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
CONFIG_TELCLOCK=m
CONFIG_DEVPORT=y
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_CHARDEV=m
# CONFIG_I2C_HELPER_AUTO is not set

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
# CONFIG_I2C_ISCH is not set
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
CONFIG_I2C_OCORES=m
CONFIG_I2C_SIMTEC=m

#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_PARPORT=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_TAOS_EVM=m
CONFIG_I2C_TINY_USB=m

#
# Graphics adapter I2C/DDC channel drivers
#
CONFIG_I2C_VOODOO3=m

#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_PCA_ISA=m
# CONFIG_I2C_PCA_PLATFORM is not set
CONFIG_I2C_STUB=m
# CONFIG_SCx200_I2C is not set
CONFIG_SCx200_ACB=m

#
# Miscellaneous I2C Chip support
#
CONFIG_DS1682=m
# CONFIG_AT24 is not set
CONFIG_SENSORS_EEPROM=m
CONFIG_SENSORS_PCF8574=m
CONFIG_PCF8575=m
# CONFIG_SENSORS_PCA9539 is not set
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_MAX6875=m
CONFIG_SENSORS_TSL2550=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
# CONFIG_SPI is not set
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
CONFIG_W1=m
CONFIG_W1_CON=y

#
# 1-wire Bus Masters
#
CONFIG_W1_MASTER_MATROX=m
CONFIG_W1_MASTER_DS2490=m
CONFIG_W1_MASTER_DS2482=m

#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=m
CONFIG_W1_SLAVE_SMEM=m
CONFIG_W1_SLAVE_DS2433=m
# CONFIG_W1_SLAVE_DS2433_CRC is not set
CONFIG_W1_SLAVE_DS2760=m
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
CONFIG_PDA_POWER=m
CONFIG_BATTERY_DS2760=m
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
# CONFIG_SENSORS_AD7414 is not set
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7473=m
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHER=m
CONFIG_SENSORS_FSCPOS=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_CORETEMP=m
# CONFIG_SENSORS_IBMAEM is not set
CONFIG_SENSORS_IBMPEX=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_HDAPS=m
CONFIG_SENSORS_APPLESMC=m
# CONFIG_HWMON_DEBUG_CHIP is not set
CONFIG_THERMAL=y
# CONFIG_THERMAL_HWMON is not set
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_NOWAYOUT=y

#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_ACQUIRE_WDT=m
CONFIG_ADVANTECH_WDT=m
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
CONFIG_SC520_WDT=m
CONFIG_EUROTECH_WDT=m
CONFIG_IB700_WDT=m
CONFIG_IBMASR=m
CONFIG_WAFER_WDT=m
CONFIG_I6300ESB_WDT=m
CONFIG_ITCO_WDT=m
CONFIG_ITCO_VENDOR_SUPPORT=y
CONFIG_IT8712F_WDT=m
CONFIG_HP_WATCHDOG=m
CONFIG_SC1200_WDT=m
CONFIG_SCx200_WDT=m
CONFIG_PC87413_WDT=m
CONFIG_60XX_WDT=m
CONFIG_SBC8360_WDT=m
CONFIG_SBC7240_WDT=m
CONFIG_CPU5_WDT=m
CONFIG_SMSC37B787_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83697HF_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_W83977F_WDT=m
CONFIG_MACHZ_WDT=m
CONFIG_SBC_EPX_C3_WATCHDOG=m

#
# ISA-based Watchdog Cards
#
CONFIG_PCWATCHDOG=m
CONFIG_MIXCOMWD=m
CONFIG_WDT=m
CONFIG_WDT_501=y

#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m
CONFIG_WDT_501_PCI=y

#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m

#
# Sonics Silicon Backplane
#
CONFIG_SSB_POSSIBLE=y
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
# CONFIG_SSB_B43_PCI_BRIDGE is not set
CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
CONFIG_SSB_PCMCIAHOST=y
# CONFIG_SSB_SILENT is not set
# CONFIG_SSB_DEBUG is not set
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y

#
# Multifunction device drivers
#
# CONFIG_MFD_CORE is not set
CONFIG_MFD_SM501=m
# CONFIG_HTC_PASIC3 is not set
# CONFIG_MFD_TMIO is not set

#
# Multimedia devices
#

#
# Multimedia core support
#
CONFIG_VIDEO_DEV=m
CONFIG_VIDEO_V4L2_COMMON=m
# CONFIG_VIDEO_ALLOW_V4L1 is not set
CONFIG_VIDEO_V4L1_COMPAT=y
CONFIG_DVB_CORE=m
CONFIG_VIDEO_MEDIA=m

#
# Multimedia drivers
#
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
# CONFIG_MEDIA_ATTACH is not set
CONFIG_MEDIA_TUNER=m
# CONFIG_MEDIA_TUNER_CUSTOMIZE is not set
CONFIG_MEDIA_TUNER_SIMPLE=m
CONFIG_MEDIA_TUNER_TDA8290=m
CONFIG_MEDIA_TUNER_TDA827X=m
CONFIG_MEDIA_TUNER_TDA18271=m
CONFIG_MEDIA_TUNER_TDA9887=m
CONFIG_MEDIA_TUNER_TEA5761=m
CONFIG_MEDIA_TUNER_TEA5767=m
CONFIG_MEDIA_TUNER_MT20XX=m
CONFIG_MEDIA_TUNER_MT2060=m
CONFIG_MEDIA_TUNER_MT2266=m
CONFIG_MEDIA_TUNER_MT2131=m
CONFIG_MEDIA_TUNER_QT1010=m
CONFIG_MEDIA_TUNER_XC2028=m
CONFIG_MEDIA_TUNER_XC5000=m
CONFIG_MEDIA_TUNER_MXL5005S=m
CONFIG_VIDEO_V4L2=m
CONFIG_VIDEOBUF_GEN=m
CONFIG_VIDEOBUF_DMA_SG=m
CONFIG_VIDEOBUF_VMALLOC=m
CONFIG_VIDEOBUF_DVB=m
CONFIG_VIDEO_BTCX=m
CONFIG_VIDEO_IR=m
CONFIG_VIDEO_TVEEPROM=m
CONFIG_VIDEO_TUNER=m
CONFIG_VIDEO_CAPTURE_DRIVERS=y
# CONFIG_VIDEO_ADV_DEBUG is not set
CONFIG_VIDEO_HELPER_CHIPS_AUTO=y
CONFIG_VIDEO_IR_I2C=m
CONFIG_VIDEO_TVAUDIO=m
CONFIG_VIDEO_TDA7432=m
CONFIG_VIDEO_TDA9875=m
CONFIG_VIDEO_MSP3400=m
CONFIG_VIDEO_CS53L32A=m
CONFIG_VIDEO_WM8775=m
CONFIG_VIDEO_OV7670=m
CONFIG_VIDEO_SAA711X=m
CONFIG_VIDEO_TVP5150=m
CONFIG_VIDEO_CX25840=m
CONFIG_VIDEO_CX2341X=m
# CONFIG_VIDEO_VIVI is not set
CONFIG_VIDEO_BT848=m
CONFIG_VIDEO_BT848_DVB=y
CONFIG_VIDEO_SAA6588=m
CONFIG_VIDEO_SAA5246A=m
CONFIG_VIDEO_SAA5249=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_SAA7134_DVB=m
CONFIG_VIDEO_HEXIUM_ORION=m
CONFIG_VIDEO_HEXIUM_GEMINI=m
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_CX88_BLACKBIRD=m
CONFIG_VIDEO_CX88_DVB=m
CONFIG_VIDEO_CX88_VP3054=m
CONFIG_VIDEO_CX23885=m
# CONFIG_VIDEO_AU0828 is not set
# CONFIG_VIDEO_CX18 is not set
CONFIG_VIDEO_CAFE_CCIC=m
CONFIG_V4L_USB_DRIVERS=y
# CONFIG_USB_VIDEO_CLASS is not set
# CONFIG_USB_GSPCA is not set
CONFIG_VIDEO_PVRUSB2=m
CONFIG_VIDEO_PVRUSB2_SYSFS=y
# CONFIG_VIDEO_PVRUSB2_DVB is not set
# CONFIG_VIDEO_PVRUSB2_DEBUGIFC is not set
CONFIG_VIDEO_EM28XX=m
# CONFIG_VIDEO_EM28XX_DVB is not set
CONFIG_VIDEO_USBVISION=m
CONFIG_USB_ET61X251=m
CONFIG_USB_SN9C102=m
CONFIG_USB_ZC0301=m
CONFIG_USB_ZR364XX=m
CONFIG_USB_STKWEBCAM=m
# CONFIG_USB_S2255 is not set
# CONFIG_SOC_CAMERA is not set
# CONFIG_VIDEO_SH_MOBILE_CEU is not set
CONFIG_RADIO_ADAPTERS=y
CONFIG_RADIO_CADET=m
CONFIG_RADIO_RTRACK=m
CONFIG_RADIO_RTRACK2=m
CONFIG_RADIO_AZTECH=m
CONFIG_RADIO_GEMTEK=m
CONFIG_RADIO_GEMTEK_PCI=m
CONFIG_RADIO_MAXIRADIO=m
CONFIG_RADIO_MAESTRO=m
CONFIG_RADIO_SF16FMI=m
CONFIG_RADIO_SF16FMR2=m
CONFIG_RADIO_TERRATEC=m
CONFIG_RADIO_TRUST=m
CONFIG_RADIO_TYPHOON=m
CONFIG_RADIO_TYPHOON_PROC_FS=y
CONFIG_RADIO_ZOLTRIX=m
CONFIG_USB_DSBR=m
CONFIG_USB_SI470X=m
CONFIG_DVB_CAPTURE_DRIVERS=y

#
# Supported SAA7146 based PCI Adapters
#
CONFIG_TTPCI_EEPROM=m
CONFIG_DVB_AV7110=m
CONFIG_DVB_AV7110_OSD=y
CONFIG_DVB_BUDGET_CORE=m
CONFIG_DVB_BUDGET=m
CONFIG_DVB_BUDGET_CI=m
CONFIG_DVB_BUDGET_AV=m
CONFIG_DVB_BUDGET_PATCH=m

#
# Supported USB Adapters
#
CONFIG_DVB_USB=m
# CONFIG_DVB_USB_DEBUG is not set
CONFIG_DVB_USB_A800=m
CONFIG_DVB_USB_DIBUSB_MB=m
# CONFIG_DVB_USB_DIBUSB_MB_FAULTY is not set
CONFIG_DVB_USB_DIBUSB_MC=m
CONFIG_DVB_USB_DIB0700=m
CONFIG_DVB_USB_UMT_010=m
CONFIG_DVB_USB_CXUSB=m
CONFIG_DVB_USB_M920X=m
CONFIG_DVB_USB_GL861=m
CONFIG_DVB_USB_AU6610=m
CONFIG_DVB_USB_DIGITV=m
CONFIG_DVB_USB_VP7045=m
CONFIG_DVB_USB_VP702X=m
CONFIG_DVB_USB_GP8PSK=m
CONFIG_DVB_USB_NOVA_T_USB2=m
CONFIG_DVB_USB_TTUSB2=m
CONFIG_DVB_USB_DTT200U=m
CONFIG_DVB_USB_OPERA1=m
# CONFIG_DVB_USB_DW2102 is not set
# CONFIG_DVB_USB_ANYSEE is not set
CONFIG_DVB_TTUSB_BUDGET=m
CONFIG_DVB_TTUSB_DEC=m
CONFIG_DVB_CINERGYT2=m
# CONFIG_DVB_CINERGYT2_TUNING is not set
# CONFIG_DVB_SIANO_SMS1XXX is not set

#
# Supported FlexCopII (B2C2) Adapters
#
CONFIG_DVB_B2C2_FLEXCOP=m
CONFIG_DVB_B2C2_FLEXCOP_PCI=m
CONFIG_DVB_B2C2_FLEXCOP_USB=m
# CONFIG_DVB_B2C2_FLEXCOP_DEBUG is not set

#
# Supported BT878 Adapters
#
CONFIG_DVB_BT8XX=m

#
# Supported Pluto2 Adapters
#
CONFIG_DVB_PLUTO2=m

#
# Supported DVB Frontends
#

#
# Customise DVB Frontends
#
# CONFIG_DVB_FE_CUSTOMISE is not set

#
# DVB-S (satellite) frontends
#
CONFIG_DVB_CX24110=m
CONFIG_DVB_CX24123=m
CONFIG_DVB_MT312=m
CONFIG_DVB_S5H1420=m
CONFIG_DVB_STV0299=m
CONFIG_DVB_TDA8083=m
CONFIG_DVB_TDA10086=m
CONFIG_DVB_VES1X93=m
CONFIG_DVB_TUNER_ITD1000=m
CONFIG_DVB_TDA826X=m
CONFIG_DVB_TUA6100=m

#
# DVB-T (terrestrial) frontends
#
CONFIG_DVB_SP8870=m
CONFIG_DVB_SP887X=m
CONFIG_DVB_CX22700=m
CONFIG_DVB_CX22702=m
# CONFIG_DVB_DRX397XD is not set
CONFIG_DVB_L64781=m
CONFIG_DVB_TDA1004X=m
CONFIG_DVB_NXT6000=m
CONFIG_DVB_MT352=m
CONFIG_DVB_ZL10353=m
CONFIG_DVB_DIB3000MB=m
CONFIG_DVB_DIB3000MC=m
CONFIG_DVB_DIB7000M=m
CONFIG_DVB_DIB7000P=m
CONFIG_DVB_TDA10048=m

#
# DVB-C (cable) frontends
#
CONFIG_DVB_VES1820=m
CONFIG_DVB_TDA10021=m
CONFIG_DVB_TDA10023=m
CONFIG_DVB_STV0297=m

#
# ATSC (North American/Korean Terrestrial/Cable DTV) frontends
#
CONFIG_DVB_NXT200X=m
CONFIG_DVB_OR51211=m
CONFIG_DVB_OR51132=m
CONFIG_DVB_BCM3510=m
CONFIG_DVB_LGDT330X=m
CONFIG_DVB_S5H1409=m
# CONFIG_DVB_AU8522 is not set
CONFIG_DVB_S5H1411=m

#
# Digital terrestrial only tuners/PLL
#
CONFIG_DVB_PLL=m
CONFIG_DVB_TUNER_DIB0070=m

#
# SEC control devices for DVB-S
#
CONFIG_DVB_LNBP21=m
# CONFIG_DVB_ISL6405 is not set
CONFIG_DVB_ISL6421=m
CONFIG_DAB=y
CONFIG_USB_DABUSB=m

#
# Graphics support
#
CONFIG_AGP=m
CONFIG_AGP_ALI=m
CONFIG_AGP_ATI=m
CONFIG_AGP_AMD=m
CONFIG_AGP_AMD64=m
CONFIG_AGP_INTEL=m
CONFIG_AGP_NVIDIA=m
CONFIG_AGP_SIS=m
CONFIG_AGP_SWORKS=m
CONFIG_AGP_VIA=m
CONFIG_AGP_EFFICEON=m
CONFIG_DRM=m
CONFIG_DRM_TDFX=m
CONFIG_DRM_R128=m
CONFIG_DRM_RADEON=m
CONFIG_DRM_I810=m
CONFIG_DRM_I830=m
CONFIG_DRM_I915=m
CONFIG_DRM_MGA=m
CONFIG_DRM_SIS=m
CONFIG_DRM_VIA=m
CONFIG_DRM_SAVAGE=m
CONFIG_VGASTATE=m
CONFIG_VIDEO_OUTPUT_CONTROL=m
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_DDC=m
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=m
CONFIG_FB_SVGALIB=m
# CONFIG_FB_MACMODES is not set
CONFIG_FB_BACKLIGHT=y
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y

#
# Frame buffer hardware drivers
#
CONFIG_FB_CIRRUS=m
CONFIG_FB_PM2=m
# CONFIG_FB_PM2_FIFO_DISCONNECT is not set
CONFIG_FB_CYBER2000=m
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
CONFIG_FB_VGA16=m
CONFIG_FB_UVESA=m
CONFIG_FB_VESA=y
CONFIG_FB_EFI=y
# CONFIG_FB_N411 is not set
CONFIG_FB_HGA=m
# CONFIG_FB_HGA_ACCEL is not set
CONFIG_FB_S1D13XXX=m
CONFIG_FB_NVIDIA=m
CONFIG_FB_NVIDIA_I2C=y
# CONFIG_FB_NVIDIA_DEBUG is not set
CONFIG_FB_NVIDIA_BACKLIGHT=y
CONFIG_FB_RIVA=m
# CONFIG_FB_RIVA_I2C is not set
# CONFIG_FB_RIVA_DEBUG is not set
CONFIG_FB_RIVA_BACKLIGHT=y
CONFIG_FB_I810=m
# CONFIG_FB_I810_GTF is not set
CONFIG_FB_LE80578=m
CONFIG_FB_CARILLO_RANCH=m
CONFIG_FB_INTEL=m
# CONFIG_FB_INTEL_DEBUG is not set
CONFIG_FB_INTEL_I2C=y
CONFIG_FB_MATROX=m
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
CONFIG_FB_MATROX_G=y
CONFIG_FB_MATROX_I2C=m
# CONFIG_FB_MATROX_MAVEN is not set
CONFIG_FB_MATROX_MULTIHEAD=y
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
CONFIG_FB_RADEON_BACKLIGHT=y
# CONFIG_FB_RADEON_DEBUG is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY128_BACKLIGHT=y
CONFIG_FB_ATY=m
CONFIG_FB_ATY_CT=y
CONFIG_FB_ATY_GENERIC_LCD=y
CONFIG_FB_ATY_GX=y
CONFIG_FB_ATY_BACKLIGHT=y
CONFIG_FB_S3=m
CONFIG_FB_SAVAGE=m
CONFIG_FB_SAVAGE_I2C=y
CONFIG_FB_SAVAGE_ACCEL=y
CONFIG_FB_SIS=m
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
CONFIG_FB_3DFX=m
# CONFIG_FB_3DFX_ACCEL is not set
CONFIG_FB_VOODOO1=m
CONFIG_FB_VT8623=m
# CONFIG_FB_CYBLA is not set
CONFIG_FB_TRIDENT=m
# CONFIG_FB_TRIDENT_ACCEL is not set
CONFIG_FB_ARK=m
CONFIG_FB_PM3=m
# CONFIG_FB_CARMINE is not set
CONFIG_FB_GEODE=y
CONFIG_FB_GEODE_LX=m
CONFIG_FB_GEODE_GX=m
CONFIG_FB_GEODE_GX1=m
CONFIG_FB_SM501=m
CONFIG_FB_VIRTUAL=m
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_LCD_CLASS_DEVICE=m
# CONFIG_LCD_ILI9320 is not set
# CONFIG_LCD_PLATFORM is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_CORGI=m
CONFIG_BACKLIGHT_PROGEAR=m
CONFIG_BACKLIGHT_CARILLO_RANCH=m
# CONFIG_BACKLIGHT_MBP_NVIDIA is not set

#
# Display device support
#
CONFIG_DISPLAY_SUPPORT=m

#
# Display hardware drivers
#

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
CONFIG_VIDEO_SELECT=y
CONFIG_MDA_CONSOLE=m
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
CONFIG_FONTS=y
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
# CONFIG_FONT_6x11 is not set
# CONFIG_FONT_7x14 is not set
# CONFIG_FONT_PEARL_8x8 is not set
# CONFIG_FONT_ACORN_8x8 is not set
# CONFIG_FONT_MINI_4x6 is not set
# CONFIG_FONT_SUN8x16 is not set
# CONFIG_FONT_SUN12x22 is not set
# CONFIG_FONT_10x18 is not set
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_SOUND is not set
CONFIG_AC97_BUS=m
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_HID_DEBUG=y
CONFIG_HIDRAW=y

#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT_POWERBOOK=y
CONFIG_HID_FF=y
CONFIG_HID_PID=y
CONFIG_LOGITECH_FF=y
# CONFIG_LOGIRUMBLEPAD2_FF is not set
CONFIG_PANTHERLORD_FF=y
CONFIG_THRUSTMASTER_FF=y
CONFIG_ZEROPLUS_FF=y
CONFIG_USB_HIDDEV=y

#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
CONFIG_USB_MON=y

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_ISP116X_HCD=m
# CONFIG_USB_ISP1760_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_HCD_SSB is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
CONFIG_USB_U132_HCD=m
CONFIG_USB_SL811_HCD=m
CONFIG_USB_SL811_CS=m
CONFIG_USB_R8A66597_HCD=m

#
# Enable Host or Gadget support to see Inventra options
#

#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
# CONFIG_USB_WDM is not set

#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#

#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
CONFIG_USB_STORAGE_USBAT=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
CONFIG_USB_STORAGE_ALAUDA=y
# CONFIG_USB_STORAGE_ONETOUCH is not set
CONFIG_USB_STORAGE_KARMA=y
# CONFIG_USB_STORAGE_SIERRA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_LIBUSUAL is not set

#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m

#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=m
CONFIG_USB_EZUSB=y
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP2101=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_FUNSOFT=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_IUU=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_MOS7720=m
CONFIG_USB_SERIAL_MOS7840=m
# CONFIG_USB_SERIAL_MOTOROLA is not set
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
# CONFIG_USB_SERIAL_SPCP8X5 is not set
CONFIG_USB_SERIAL_HP4X=m
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_SERIAL_DEBUG=m

#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
# CONFIG_USB_RIO500 is not set
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_BERRY_CHARGE=m
CONFIG_USB_LED=m
CONFIG_USB_CYPRESS_CY7C63=m
CONFIG_USB_CYTHERM=m
CONFIG_USB_PHIDGET=m
CONFIG_USB_PHIDGETKIT=m
CONFIG_USB_PHIDGETMOTORCONTROL=m
CONFIG_USB_PHIDGETSERVO=m
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
CONFIG_USB_SISUSBVGA=m
CONFIG_USB_SISUSBVGA_CON=y
CONFIG_USB_LD=m
CONFIG_USB_TRANCEVIBRATOR=m
CONFIG_USB_IOWARRIOR=m
CONFIG_USB_TEST=m
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=m

#
# LED drivers
#
CONFIG_LEDS_NET48XX=m
CONFIG_LEDS_WRAP=m
# CONFIG_LEDS_PCA9532 is not set
CONFIG_LEDS_CLEVO_MAIL=m
# CONFIG_LEDS_PCA955X is not set

#
# LED Triggers
#
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_IDE_DISK=y
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
# CONFIG_LEDS_TRIGGER_DEFAULT_ON is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
# CONFIG_EDAC is not set
# CONFIG_RTC_CLASS is not set
CONFIG_DMADEVICES=y

#
# DMA Devices
#
CONFIG_INTEL_IOATDMA=m
CONFIG_DMA_ENGINE=y

#
# DMA Clients
#
CONFIG_NET_DMA=y
# CONFIG_DMATEST is not set
CONFIG_DCA=m
CONFIG_AUXDISPLAY=y
CONFIG_KS0108=m
CONFIG_KS0108_PORT=0x378
CONFIG_KS0108_DELAY=2
CONFIG_CFAG12864B=m
CONFIG_CFAG12864B_RATE=20
# CONFIG_UIO is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_FIRMWARE_MEMMAP is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set
# CONFIG_ISCSI_IBFT_FIND is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT2_FS_XIP=y
CONFIG_FS_XIP=y
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
# CONFIG_EXT4DEV_FS is not set
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
CONFIG_DNOTIFY=y
CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
CONFIG_PRINT_QUOTA_WARNING=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=m
CONFIG_FUSE_FS=m
CONFIG_GENERIC_ACL=y

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_VMCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=m

#
# Miscellaneous filesystems
#
CONFIG_ADFS_FS=m
# CONFIG_ADFS_FS_RW is not set
CONFIG_AFFS_FS=m
CONFIG_ECRYPT_FS=m
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BEFS_FS=m
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=m
CONFIG_EFS_FS=m
CONFIG_CRAMFS=y
CONFIG_VXFS_FS=m
CONFIG_MINIX_FS=m
# CONFIG_OMFS_FS is not set
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=m
CONFIG_ROMFS_FS=m
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
# CONFIG_NETWORK_FILESYSTEMS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
# CONFIG_DLM_DEBUG is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
CONFIG_MAGIC_SYSRQ=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
# CONFIG_DETECT_SOFTLOCKUP is not set
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
CONFIG_SLUB_DEBUG_ON=y
# CONFIG_SLUB_STATS is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
# CONFIG_PROVE_LOCKING is not set
CONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_HIGHMEM is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
CONFIG_DEBUG_VM=y
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not set
# CONFIG_DEBUG_MEMORY_INIT is not set
# CONFIG_DEBUG_LIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
CONFIG_FRAME_POINTER=y
CONFIG_BOOT_PRINTK_DELAY=y
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_CPU_STALL is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not set
# CONFIG_SYSCTL_SYSCALL_CHECK is not set
CONFIG_HAVE_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE is not set
# CONFIG_IRQSOFF_TRACER is not set
# CONFIG_SYSPROF_TRACER is not set
# CONFIG_SCHED_TRACER is not set
# CONFIG_CONTEXT_SWITCH_TRACER is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
# CONFIG_STRICT_DEVMEM is not set
CONFIG_X86_VERBOSE_BOOTUP=y
# CONFIG_EARLY_PRINTK is not set
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACK_USAGE=y
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_X86_PTDUMP is not set
# CONFIG_DEBUG_RODATA is not set
# CONFIG_DEBUG_NX_TEST is not set
# CONFIG_4KSTACKS is not set
CONFIG_DOUBLEFAULT=y
# CONFIG_MMIOTRACE is not set
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_KMEMCHECK is not set
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set

#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_NETWORK_XFRM=y
CONFIG_SECURITY_FILE_CAPABILITIES=y
CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR=0
# CONFIG_SECURITY_SELINUX is not set
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_AEAD=m
CONFIG_CRYPTO_BLKCIPHER=m
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_GF128MUL=m
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_AUTHENC=m
CONFIG_CRYPTO_TEST=m

#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
CONFIG_CRYPTO_SEQIV=m

#
# Block modes
#
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_CTR=m
# CONFIG_CRYPTO_CTS is not set
CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_XTS=m

#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_XCBC=m

#
# Digest
#
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
# CONFIG_CRYPTO_RMD128 is not set
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m

#
# Ciphers
#
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_CAMELLIA=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_SALSA20=m
CONFIG_CRYPTO_SALSA20_586=m
CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
CONFIG_CRYPTO_TWOFISH_586=m

#
# Compression
#
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_HW=y
CONFIG_CRYPTO_DEV_PADLOCK=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m
CONFIG_CRYPTO_DEV_PADLOCK_SHA=m
CONFIG_CRYPTO_DEV_GEODE=m
CONFIG_CRYPTO_DEV_HIFN_795X=m
# CONFIG_CRYPTO_DEV_HIFN_795X_RNG is not set
CONFIG_HAVE_KVM=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_KVM_AMD=m
CONFIG_LGUEST=m
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_BALLOON=m

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_CRC_CCITT=m
CONFIG_CRC16=m
# CONFIG_CRC_T10DIF is not set
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
CONFIG_CRC7=m
CONFIG_LIBCRC32C=m
CONFIG_AUDIT_GENERIC=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=m
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CHECK_SIGNATURE=y

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 14:39       ` Luiz Fernando N. Capitulino
@ 2008-08-22 15:34         ` Mathieu Desnoyers
  2008-08-22 16:29           ` Luiz Fernando N. Capitulino
  0 siblings, 1 reply; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-22 15:34 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Ingo Molnar, H. Peter Anvin, linux-kernel

* Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> Em Fri, 22 Aug 2008 08:50:12 +0200
> Ingo Molnar <mingo@elte.hu> escreveu:
> 
> | 
> | * H. Peter Anvin <hpa@zytor.com> wrote:
> | 
> | > H. Peter Anvin wrote:
> | >>>
> | >>>  Does this look like a kernel bug?
> | >>>
> | >>
> | >> No, it looks like a very common virtualizer bug.  Does the attached  
> | >> patch work for you?
> | >>
> | >
> | > Also, in addition to this, please try tip:master.  There is a patch in 
> | > tip:master which I hope should fix this problem, but the details are 
> | > important.
> | 
> | access coordinates would be at:
> | 
> |   http://people.redhat.com/mingo/tip.git/README
> 
>  As I already have Linus tree downloaded I have cloned it in
> the usual way.
> 
>  Got the same results: OOPS in virtualbox but it works on QEMU.
> 
>  The OOPS's output follows and I have attached the .config I'm using
> to reproduce the problem.
> 

Can you try booting with the kernel argument :
  debug_alternative 

The dmesg of the kernel bootup up to the oops would be helpful.

My guess is that there may be something wrong with irq disabling which
protects text_poke_early in apply_alternatives().

Mathieu


> """
> BUG: unable to handle kernel NULL pointer dereference at 00000246
> IP: [<c01310f1>] vprintk+0x181/0x440
> *pde = 00000000 
> Oops: 0002 [#1] SMP 
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test24-tip #3)
> EIP: 0060:[<c01310f1>] EFLAGS: 00010246 CPU: 0
> EIP is at vprintk+0x181/0x440
> EAX: 00000246 EBX: 00000000 ECX: c0130ca9 EDX: 0000dedd
> ESI: c0474ae3 EDI: c04cf6bc EBP: c7435f24 ESP: c7435eb0
>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
> Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
> Stack: 0000dedd c0130ca9 c7435f40 00000000 a026104f a026106c c7434000 c7435ee6 
>        00000006 00000246 00000000 a0260cf3 0000001c c7434000 00000282 00000046 
>        c11a85a0 c7435efc c0135c6f c7435f14 c0115fcb a0296e91 c0104c2c 00000000 
> Call Trace:
>  [<c0130ca9>] ? release_console_sem+0x199/0x1e0
>  [<c0135c6f>] ? irq_exit+0x3f/0x90
>  [<c0115fcb>] ? smp_apic_timer_interrupt+0x5b/0x90
>  [<c0104c2c>] ? apic_timer_interrupt+0x28/0x30
>  [<c0474ae3>] ? net_ns_init+0x0/0x1ad
>  [<c0474ae3>] ? net_ns_init+0x0/0x1ad
>  [<c0346ed9>] ? printk+0x18/0x1f
>  [<c0474b00>] ? net_ns_init+0x1d/0x1ad
>  [<c0474ae3>] ? net_ns_init+0x0/0x1ad
>  [<c0101116>] ? do_one_initcall+0x26/0x170
>  [<c0128f66>] ? try_to_wake_up+0xc6/0x240
>  [<c012910f>] ? wake_up_process+0xf/0x20
>  [<c014192d>] ? start_workqueue_thread+0x1d/0x20
>  [<c0141d4b>] ? __create_workqueue_key+0x1eb/0x240
>  [<c0141820>] ? worker_thread+0x0/0xf0
>  [<c044b387>] ? kernel_init+0x141/0x214
>  [<c044b246>] ? kernel_init+0x0/0x214
>  [<c0104dc7>] ? kernel_thread_helper+0x7/0x10
>  =======================
> Code: c0 0f 84 0b 01 00 00 b8 50 f1 41 c0 c7 05 ec f1 41 c0 ff ff ff ff e8 cf 8b 21 00 e8 ea 04 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 ec f1 41 c0 e8 
> EIP: [<c01310f1>] vprintk+0x181/0x440 SS:ESP 0069:c7435eb0
> ---[ end trace 4eaa2a86a8e2da22 ]---
> """
> 
> -- 
> Luiz Fernando N. Capitulino



-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 15:34         ` Mathieu Desnoyers
@ 2008-08-22 16:29           ` Luiz Fernando N. Capitulino
  2008-08-22 16:35             ` Mathieu Desnoyers
  2008-08-22 17:16             ` H. Peter Anvin
  0 siblings, 2 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 16:29 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: Ingo Molnar, H. Peter Anvin, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

Em Fri, 22 Aug 2008 11:34:52 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| > Em Fri, 22 Aug 2008 08:50:12 +0200
| > Ingo Molnar <mingo@elte.hu> escreveu:
| > 
| > | 
| > | * H. Peter Anvin <hpa@zytor.com> wrote:
| > | 
| > | > H. Peter Anvin wrote:
| > | >>>
| > | >>>  Does this look like a kernel bug?
| > | >>>
| > | >>
| > | >> No, it looks like a very common virtualizer bug.  Does the attached  
| > | >> patch work for you?
| > | >>
| > | >
| > | > Also, in addition to this, please try tip:master.  There is a patch in 
| > | > tip:master which I hope should fix this problem, but the details are 
| > | > important.
| > | 
| > | access coordinates would be at:
| > | 
| > |   http://people.redhat.com/mingo/tip.git/README
| > 
| >  As I already have Linus tree downloaded I have cloned it in
| > the usual way.
| > 
| >  Got the same results: OOPS in virtualbox but it works on QEMU.
| > 
| >  The OOPS's output follows and I have attached the .config I'm using
| > to reproduce the problem.
| > 
| 
| Can you try booting with the kernel argument :
|   debug_alternative 
| 
| The dmesg of the kernel bootup up to the oops would be helpful.
| 
| My guess is that there may be something wrong with irq disabling which
| protects text_poke_early in apply_alternatives().

 I have attached two files:

  - normal.txt: normal boot with no debug options
  - debug-alternative.txt ignore_loglevel and debug-alternative boot
    options

 I had to pass ignore_loglevel otherwise it wouldn't print
anything.

-- 
Luiz Fernando N. Capitulino

[-- Attachment #2: normal.txt --]
[-- Type: text/plain, Size: 7273 bytes --]

Linux version 2.6.27-rc4-test25 (lcapitulino@build1.conectiva) (gcc version 4.3.1 20080626 (prerelease) (GCC) ) #89 SMP Fri Aug 22 12:47:34 BRT 2008
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
 BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
last_pfn = 0x7ff0 max_arch_pfn = 0x100000
RAMDISK: 07b9b000 - 07fbf89d
DMI 2.5 present.
ACPI: RSDP 000E0000, 0024 (r2 VBOX  )
ACPI: XSDT 07FF0030, 002C (r1 VBOX   VBOXXSDT        1 ASL        61)
ACPI: FACP 07FF0060, 00F4 (r4 VBOX   VBOXFACP        1 ASL        61)
ACPI: DSDT 07FF01A0, 1064 (r1 VBOX   VBOXBIOS        2 INTL 20080213)
ACPI: FACS 07FF0160, 0040
0MB HIGHMEM available.
127MB LOWMEM available.
  mapped low ram: 0 - 07ff0000
  low ram: 00000000 - 07ff0000
  bootmap 00002000 - 00003000
(9 early reservations) ==> bootmem [0000000000 - 0007ff0000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000100000 - 0000814b10]    TEXT DATA BSS ==> [0000100000 - 0000814b10]
  #4 [0007b9b000 - 0007fbf89d]          RAMDISK ==> [0007b9b000 - 0007fbf89d]
  #5 [0000815000 - 0000819000]    INIT_PG_TABLE ==> [0000815000 - 0000819000]
  #6 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #7 [0000007000 - 0000009000]          PGTABLE ==> [0000007000 - 0000009000]
  #8 [0000002000 - 0000003000]          BOOTMAP ==> [0000002000 - 0000003000]
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  Normal   0x00001000 -> 0x00007ff0
  HighMem  0x00007ff0 -> 0x00007ff0
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x00007ff0
ACPI: PM-Timer IO Port: 0x4008
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Found and enabled local APIC!
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 10000000 (gap: 8000000:f7fc0000)
PERCPU: Allocating 40224 bytes of per cpu data
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32239
Kernel command line: initrd=alt0/all.rdz vga=788 splash=silent BOOT_IMAGE=alt0/vmlinuz vga=0 console=ttyS0,9600 console=tty0
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 512 (order: 9, 2048 bytes)
TSC calibrated against PM_TIMER
Detected 2410.453 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
console [ttyS0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:           4096
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 2335 kB
 per task-struct memory footprint: 1152 bytes
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 117088k/131008k available (2340k kernel code, 13364k reserved, 1027k data, 308k init, 0k highmem)
virtual kernel memory layout:
    fixmap  : 0xffe18000 - 0xfffff000   (1948 kB)
    pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
    vmalloc : 0xc8800000 - 0xff7fe000   ( 879 MB)
    lowmem  : 0xc0000000 - 0xc7ff0000   ( 127 MB)
      .init : 0xc0451000 - 0xc049e000   ( 308 kB)
      .data : 0xc03493b8 - 0xc044a040   (1027 kB)
      .text : 0xc0100000 - 0xc03493b8   (2340 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
SLUB: Genslabs=12, HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Calibrating delay loop (skipped), value calculated using timer frequency.. 4820.90 BogoMIPS (lpj=2410453)
Security Framework initialized
Mount-cache hash table entries: 512
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 10k freed
ACPI: Core revision 20080609
ACPI: setting ELCR to 0200 (from 0c00)
weird, boot CPU (#0) not listedby the BIOS.
SMP motherboard not detected.
SMP disabled
Brought up 1 CPUs
Total of 1 processors activated (4820.90 BogoMIPS).
khelper used greatest stack depth: 7108 bytes left
net_namespace: 384 bytes
Booting paravirtualized kernel on bare hardware
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfadb0, last bus=0
PCI: Using configuration type 1 for base access
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using PIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 9 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 9 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 9 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 9 *10 11)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
ACPI: bus type pnp registered
BUG: unable to handle kernel NULL pointer dereference at 00000246
IP: [<c012fc71>] vprintk+0x181/0x440
*pde = 00000000 
Oops: 0002 [#1] SMP 
Modules linked in:

Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test25 #89)
EIP: 0060:[<c012fc71>] EFLAGS: 00010246 CPU: 0
EIP is at vprintk+0x181/0x440
EAX: 00000246 EBX: 00000000 ECX: c012f8a9 EDX: 00003b3a
ESI: 00000000 EDI: c04d76c1 EBP: c7435f20 ESP: c7435eac
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
Stack: 00003b3a c012f8a9 c7435f3c c02948b1 c7435f18 c02957f6 00000074 c7435ee2 
       00000006 00000246 00000000 00000000 00000021 00000000 00000001 00000000 
       a027c4ab a027c4c8 00000001 00000297 00000246 00000000 00000001 00000000 
Call Trace:
 [<c012f8a9>] ? release_console_sem+0x1c9/0x1e0
 [<c02948b1>] ? put_device+0x11/0x20
 [<c02957f6>] ? device_add+0x26/0x610
 [<c0471c5c>] ? pnpacpi_init+0x0/0x89
 [<c03450f4>] ? printk+0x18/0x1c
 [<c0266f87>] ? register_acpi_bus_type+0x58/0x69
 [<c0471ca5>] ? pnpacpi_init+0x49/0x89
 [<c0101116>] ? do_one_initcall+0x26/0x170
 [<c01e1d14>] ? create_proc_entry+0x54/0xa0
 [<c016ef86>] ? register_irq_proc+0xb6/0xd0
 [<c016efea>] ? init_irq_proc+0x4a/0x60
 [<c045132d>] ? kernel_init+0x10f/0x166
 [<c045121e>] ? kernel_init+0x0/0x166
 [<c0104b67>] ? kernel_thread_helper+0x7/0x10
 =======================
Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8 
EIP: [<c012fc71>] vprintk+0x181/0x440 SS:ESP 0069:c7435eac
---[ end trace 4eaa2a86a8e2da22 ]---
Kernel panic - not syncing: Attempted to kill init!

[-- Attachment #3: debug-alternative.txt --]
[-- Type: text/plain, Size: 6848 bytes --]

Linux version 2.6.27-rc4-test25 (lcapitulino@build1.conectiva) (gcc version 4.3.1 20080626 (prerelease) (GCC) ) #89 SMP Fri Aug 22 12:47:34 BRT 2008
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
 BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
debug: ignoring loglevel setting.
last_pfn = 0x7ff0 max_arch_pfn = 0x100000
kernel direct mapping tables up to 7ff0000 @ 7000-d000
RAMDISK: 07b9b000 - 07fbf89d
DMI 2.5 present.
ACPI: RSDP 000E0000, 0024 (r2 VBOX  )
ACPI: XSDT 07FF0030, 002C (r1 VBOX   VBOXXSDT        1 ASL        61)
ACPI: FACP 07FF0060, 00F4 (r4 VBOX   VBOXFACP        1 ASL        61)
ACPI: DSDT 07FF01A0, 1064 (r1 VBOX   VBOXBIOS        2 INTL 20080213)
ACPI: FACS 07FF0160, 0040
0MB HIGHMEM available.
127MB LOWMEM available.
  mapped low ram: 0 - 07ff0000
  low ram: 00000000 - 07ff0000
  bootmap 00002000 - 00003000
(9 early reservations) ==> bootmem [0000000000 - 0007ff0000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000100000 - 0000814b10]    TEXT DATA BSS ==> [0000100000 - 0000814b10]
  #4 [0007b9b000 - 0007fbf89d]          RAMDISK ==> [0007b9b000 - 0007fbf89d]
  #5 [0000815000 - 0000819000]    INIT_PG_TABLE ==> [0000815000 - 0000819000]
  #6 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #7 [0000007000 - 0000009000]          PGTABLE ==> [0000007000 - 0000009000]
  #8 [0000002000 - 0000003000]          BOOTMAP ==> [0000002000 - 0000003000]
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  Normal   0x00001000 -> 0x00007ff0
  HighMem  0x00007ff0 -> 0x00007ff0
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x00007ff0
On node 0 totalpages: 32655
free_area_init_node: node 0, pgdat c041f600, node_mem_map c1000000
  DMA zone: 3947 pages, LIFO batch:0
  Normal zone: 28292 pages, LIFO batch:7
ACPI: PM-Timer IO Port: 0x4008
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Found and enabled local APIC!
mapped APIC to ffffb000 (fee00000)
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 10000000 (gap: 8000000:f7fc0000)
PERCPU: Allocating 40224 bytes of per cpu data
NR_CPUS: 32, nr_cpu_ids: 1, nr_node_ids 1
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32239
Kernel command line: initrd=alt0/all.rdz vga=788 splash=silent BOOT_IMAGE=alt0/vmlinuz vga=0 console=ttyS0,9600 console=tty0 ignore_loglevel debug-alternative
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 512 (order: 9, 2048 bytes)
TSC calibrated against PM_TIMER
Detected 2410.976 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
console [ttyS0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:           4096
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 2335 kB
 per task-struct memory footprint: 1152 bytes
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 117088k/131008k available (2340k kernel code, 13364k reserved, 1027k data, 308k init, 0k highmem)
virtual kernel memory layout:
    fixmap  : 0xffe18000 - 0xfffff000   (1948 kB)
    pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
    vmalloc : 0xc8800000 - 0xff7fe000   ( 879 MB)
    lowmem  : 0xc0000000 - 0xc7ff0000   ( 127 MB)
      .init : 0xc0451000 - 0xc049e000   ( 308 kB)
      .data : 0xc03493b8 - 0xc044a040   (1027 kB)
      .text : 0xc0100000 - 0xc03493b8   (2340 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
CPA: page pool initialized 1 of 1 pages preallocated
SLUB: Genslabs=12, HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Calibrating delay loop (skipped), value calculated using timer frequency.. 4821.95 BogoMIPS (lpj=2410976)
Security Framework initialized
Mount-cache hash table entries: 512
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
Checking 'hlt' instruction... OK.
apply_alternatives: alt table c048afa4 -> c048f11f
SMP alternatives: switching to UP code
Freeing SMP alternatives: 10k freed
ACPI: Core revision 20080609
ACPI: setting ELCR to 0200 (from 0c00)
weird, boot CPU (#0) not listedby the BIOS.
SMP motherboard not detected.
SMP disabled
Brought up 1 CPUs
Total of 1 processors activated (4821.95 BogoMIPS).
BUG: unable to handle kernel NULL pointer dereference at 00000246
IP: [<c012fc71>] vprintk+0x181/0x440
*pde = 00000000 
Oops: 0002 [#1] SMP 
Modules linked in:

Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test25 #89)
EIP: 0060:[<c012fc71>] EFLAGS: 00010246 CPU: 0
EIP is at vprintk+0x181/0x440
EAX: 00000246 EBX: 00000000 ECX: c012f8a9 EDX: 00009695
ESI: 00000000 EDI: c04d76d7 EBP: c7435f98 ESP: c7435f24
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
Stack: 00009695 c012f8a9 c7435fb4 0000007b c012007b 000000d8 ffffff10 c7435f5a 
       00000006 00000246 00000000 c046902c 00000037 00000246 c03438ac c7435f7a 
       00000006 00000246 00000000 00000000 00000015 c7435f94 c045e900 00000030 
Call Trace:
 [<c012f8a9>] ? release_console_sem+0x1c9/0x1e0
 [<c012007b>] ? resched_task+0x4b/0x70
 [<c046902c>] ? relay_init+0xd/0x11
 [<c03438ac>] ? end_local_APIC_setup+0xb9/0xf2
 [<c045e900>] ? prefill_possible_map+0x7/0x8a
 [<c03450f4>] ? printk+0x18/0x1c
 [<c045eacd>] ? native_smp_cpus_done+0x93/0xe9
 [<c04512f3>] ? kernel_init+0xd5/0x166
 [<c045121e>] ? kernel_init+0x0/0x166
 [<c0104b67>] ? kernel_thread_helper+0x7/0x10
 =======================
Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8 
EIP: [<c012fc71>] vprintk+0x181/0x440 SS:ESP 0069:c7435f24
---[ end trace 4eaa2a86a8e2da22 ]---
Kernel panic - not syncing: Attempted to kill init!

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 16:29           ` Luiz Fernando N. Capitulino
@ 2008-08-22 16:35             ` Mathieu Desnoyers
  2008-08-22 17:20               ` Luiz Fernando N. Capitulino
  2008-08-22 17:16             ` H. Peter Anvin
  1 sibling, 1 reply; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-22 16:35 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Ingo Molnar, H. Peter Anvin, linux-kernel

* Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> Em Fri, 22 Aug 2008 11:34:52 -0400
> Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
> 
> | * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> | > Em Fri, 22 Aug 2008 08:50:12 +0200
> | > Ingo Molnar <mingo@elte.hu> escreveu:
> | > 
> | > | 
> | > | * H. Peter Anvin <hpa@zytor.com> wrote:
> | > | 
> | > | > H. Peter Anvin wrote:
> | > | >>>
> | > | >>>  Does this look like a kernel bug?
> | > | >>>
> | > | >>
> | > | >> No, it looks like a very common virtualizer bug.  Does the attached  
> | > | >> patch work for you?
> | > | >>
> | > | >
> | > | > Also, in addition to this, please try tip:master.  There is a patch in 
> | > | > tip:master which I hope should fix this problem, but the details are 
> | > | > important.
> | > | 
> | > | access coordinates would be at:
> | > | 
> | > |   http://people.redhat.com/mingo/tip.git/README
> | > 
> | >  As I already have Linus tree downloaded I have cloned it in
> | > the usual way.
> | > 
> | >  Got the same results: OOPS in virtualbox but it works on QEMU.
> | > 
> | >  The OOPS's output follows and I have attached the .config I'm using
> | > to reproduce the problem.
> | > 
> | 
> | Can you try booting with the kernel argument :
> |   debug_alternative 
> | 
> | The dmesg of the kernel bootup up to the oops would be helpful.
> | 
> | My guess is that there may be something wrong with irq disabling which
> | protects text_poke_early in apply_alternatives().
> 
>  I have attached two files:
> 
>   - normal.txt: normal boot with no debug options
>   - debug-alternative.txt ignore_loglevel and debug-alternative boot
>     options
> 
>  I had to pass ignore_loglevel otherwise it wouldn't print
> anything.
> 

Ok, now can you try booting with either of those args :

noreplace-paravirt
noreplace-smp

And see which one(s) works ?

Thanks,

Mathieu

> -- 
> Luiz Fernando N. Capitulino

> Linux version 2.6.27-rc4-test25 (lcapitulino@build1.conectiva) (gcc version 4.3.1 20080626 (prerelease) (GCC) ) #89 SMP Fri Aug 22 12:47:34 BRT 2008
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
>  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
>  BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
>  BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
> last_pfn = 0x7ff0 max_arch_pfn = 0x100000
> RAMDISK: 07b9b000 - 07fbf89d
> DMI 2.5 present.
> ACPI: RSDP 000E0000, 0024 (r2 VBOX  )
> ACPI: XSDT 07FF0030, 002C (r1 VBOX   VBOXXSDT        1 ASL        61)
> ACPI: FACP 07FF0060, 00F4 (r4 VBOX   VBOXFACP        1 ASL        61)
> ACPI: DSDT 07FF01A0, 1064 (r1 VBOX   VBOXBIOS        2 INTL 20080213)
> ACPI: FACS 07FF0160, 0040
> 0MB HIGHMEM available.
> 127MB LOWMEM available.
>   mapped low ram: 0 - 07ff0000
>   low ram: 00000000 - 07ff0000
>   bootmap 00002000 - 00003000
> (9 early reservations) ==> bootmem [0000000000 - 0007ff0000]
>   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
>   #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
>   #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
>   #3 [0000100000 - 0000814b10]    TEXT DATA BSS ==> [0000100000 - 0000814b10]
>   #4 [0007b9b000 - 0007fbf89d]          RAMDISK ==> [0007b9b000 - 0007fbf89d]
>   #5 [0000815000 - 0000819000]    INIT_PG_TABLE ==> [0000815000 - 0000819000]
>   #6 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
>   #7 [0000007000 - 0000009000]          PGTABLE ==> [0000007000 - 0000009000]
>   #8 [0000002000 - 0000003000]          BOOTMAP ==> [0000002000 - 0000003000]
> Zone PFN ranges:
>   DMA      0x00000000 -> 0x00001000
>   Normal   0x00001000 -> 0x00007ff0
>   HighMem  0x00007ff0 -> 0x00007ff0
> Movable zone start PFN for each node
> early_node_map[2] active PFN ranges
>     0: 0x00000000 -> 0x0000009f
>     0: 0x00000100 -> 0x00007ff0
> ACPI: PM-Timer IO Port: 0x4008
> SMP: Allowing 1 CPUs, 0 hotplug CPUs
> Found and enabled local APIC!
> PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
> PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
> PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
> Allocating PCI resources starting at 10000000 (gap: 8000000:f7fc0000)
> PERCPU: Allocating 40224 bytes of per cpu data
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32239
> Kernel command line: initrd=alt0/all.rdz vga=788 splash=silent BOOT_IMAGE=alt0/vmlinuz vga=0 console=ttyS0,9600 console=tty0
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Initializing CPU#0
> PID hash table entries: 512 (order: 9, 2048 bytes)
> TSC calibrated against PM_TIMER
> Detected 2410.453 MHz processor.
> Console: colour VGA+ 80x25
> console [tty0] enabled
> console [ttyS0] enabled
> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> ... MAX_LOCKDEP_SUBCLASSES:    8
> ... MAX_LOCK_DEPTH:          48
> ... MAX_LOCKDEP_KEYS:        8191
> ... CLASSHASH_SIZE:           4096
> ... MAX_LOCKDEP_ENTRIES:     8192
> ... MAX_LOCKDEP_CHAINS:      16384
> ... CHAINHASH_SIZE:          8192
>  memory used by lock dependency info: 2335 kB
>  per task-struct memory footprint: 1152 bytes
> Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
> Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
> Memory: 117088k/131008k available (2340k kernel code, 13364k reserved, 1027k data, 308k init, 0k highmem)
> virtual kernel memory layout:
>     fixmap  : 0xffe18000 - 0xfffff000   (1948 kB)
>     pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
>     vmalloc : 0xc8800000 - 0xff7fe000   ( 879 MB)
>     lowmem  : 0xc0000000 - 0xc7ff0000   ( 127 MB)
>       .init : 0xc0451000 - 0xc049e000   ( 308 kB)
>       .data : 0xc03493b8 - 0xc044a040   (1027 kB)
>       .text : 0xc0100000 - 0xc03493b8   (2340 kB)
> Checking if this processor honours the WP bit even in supervisor mode...Ok.
> SLUB: Genslabs=12, HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
> Calibrating delay loop (skipped), value calculated using timer frequency.. 4820.90 BogoMIPS (lpj=2410453)
> Security Framework initialized
> Mount-cache hash table entries: 512
> CPU: Trace cache: 12K uops, L1 D cache: 16K
> CPU: L2 cache: 1024K
> Intel machine check architecture supported.
> Intel machine check reporting enabled on CPU#0.
> Checking 'hlt' instruction... OK.
> SMP alternatives: switching to UP code
> Freeing SMP alternatives: 10k freed
> ACPI: Core revision 20080609
> ACPI: setting ELCR to 0200 (from 0c00)
> weird, boot CPU (#0) not listedby the BIOS.
> SMP motherboard not detected.
> SMP disabled
> Brought up 1 CPUs
> Total of 1 processors activated (4820.90 BogoMIPS).
> khelper used greatest stack depth: 7108 bytes left
> net_namespace: 384 bytes
> Booting paravirtualized kernel on bare hardware
> NET: Registered protocol family 16
> ACPI: bus type pci registered
> PCI: PCI BIOS revision 2.10 entry at 0xfadb0, last bus=0
> PCI: Using configuration type 1 for base access
> ACPI: Interpreter enabled
> ACPI: (supports S0 S5)
> ACPI: Using PIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> ACPI: PCI Interrupt Link [LNKA] (IRQs 5 9 10 11) *0, disabled.
> ACPI: PCI Interrupt Link [LNKB] (IRQs 5 9 10 11) *0, disabled.
> ACPI: PCI Interrupt Link [LNKC] (IRQs 5 9 10 *11)
> ACPI: PCI Interrupt Link [LNKD] (IRQs 5 9 *10 11)
> Linux Plug and Play Support v0.97 (c) Adam Belay
> pnp: PnP ACPI init
> ACPI: bus type pnp registered
> BUG: unable to handle kernel NULL pointer dereference at 00000246
> IP: [<c012fc71>] vprintk+0x181/0x440
> *pde = 00000000 
> Oops: 0002 [#1] SMP 
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test25 #89)
> EIP: 0060:[<c012fc71>] EFLAGS: 00010246 CPU: 0
> EIP is at vprintk+0x181/0x440
> EAX: 00000246 EBX: 00000000 ECX: c012f8a9 EDX: 00003b3a
> ESI: 00000000 EDI: c04d76c1 EBP: c7435f20 ESP: c7435eac
>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
> Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
> Stack: 00003b3a c012f8a9 c7435f3c c02948b1 c7435f18 c02957f6 00000074 c7435ee2 
>        00000006 00000246 00000000 00000000 00000021 00000000 00000001 00000000 
>        a027c4ab a027c4c8 00000001 00000297 00000246 00000000 00000001 00000000 
> Call Trace:
>  [<c012f8a9>] ? release_console_sem+0x1c9/0x1e0
>  [<c02948b1>] ? put_device+0x11/0x20
>  [<c02957f6>] ? device_add+0x26/0x610
>  [<c0471c5c>] ? pnpacpi_init+0x0/0x89
>  [<c03450f4>] ? printk+0x18/0x1c
>  [<c0266f87>] ? register_acpi_bus_type+0x58/0x69
>  [<c0471ca5>] ? pnpacpi_init+0x49/0x89
>  [<c0101116>] ? do_one_initcall+0x26/0x170
>  [<c01e1d14>] ? create_proc_entry+0x54/0xa0
>  [<c016ef86>] ? register_irq_proc+0xb6/0xd0
>  [<c016efea>] ? init_irq_proc+0x4a/0x60
>  [<c045132d>] ? kernel_init+0x10f/0x166
>  [<c045121e>] ? kernel_init+0x0/0x166
>  [<c0104b67>] ? kernel_thread_helper+0x7/0x10
>  =======================
> Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8 
> EIP: [<c012fc71>] vprintk+0x181/0x440 SS:ESP 0069:c7435eac
> ---[ end trace 4eaa2a86a8e2da22 ]---
> Kernel panic - not syncing: Attempted to kill init!

> Linux version 2.6.27-rc4-test25 (lcapitulino@build1.conectiva) (gcc version 4.3.1 20080626 (prerelease) (GCC) ) #89 SMP Fri Aug 22 12:47:34 BRT 2008
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
>  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
>  BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
>  BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
> debug: ignoring loglevel setting.
> last_pfn = 0x7ff0 max_arch_pfn = 0x100000
> kernel direct mapping tables up to 7ff0000 @ 7000-d000
> RAMDISK: 07b9b000 - 07fbf89d
> DMI 2.5 present.
> ACPI: RSDP 000E0000, 0024 (r2 VBOX  )
> ACPI: XSDT 07FF0030, 002C (r1 VBOX   VBOXXSDT        1 ASL        61)
> ACPI: FACP 07FF0060, 00F4 (r4 VBOX   VBOXFACP        1 ASL        61)
> ACPI: DSDT 07FF01A0, 1064 (r1 VBOX   VBOXBIOS        2 INTL 20080213)
> ACPI: FACS 07FF0160, 0040
> 0MB HIGHMEM available.
> 127MB LOWMEM available.
>   mapped low ram: 0 - 07ff0000
>   low ram: 00000000 - 07ff0000
>   bootmap 00002000 - 00003000
> (9 early reservations) ==> bootmem [0000000000 - 0007ff0000]
>   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
>   #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
>   #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
>   #3 [0000100000 - 0000814b10]    TEXT DATA BSS ==> [0000100000 - 0000814b10]
>   #4 [0007b9b000 - 0007fbf89d]          RAMDISK ==> [0007b9b000 - 0007fbf89d]
>   #5 [0000815000 - 0000819000]    INIT_PG_TABLE ==> [0000815000 - 0000819000]
>   #6 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
>   #7 [0000007000 - 0000009000]          PGTABLE ==> [0000007000 - 0000009000]
>   #8 [0000002000 - 0000003000]          BOOTMAP ==> [0000002000 - 0000003000]
> Zone PFN ranges:
>   DMA      0x00000000 -> 0x00001000
>   Normal   0x00001000 -> 0x00007ff0
>   HighMem  0x00007ff0 -> 0x00007ff0
> Movable zone start PFN for each node
> early_node_map[2] active PFN ranges
>     0: 0x00000000 -> 0x0000009f
>     0: 0x00000100 -> 0x00007ff0
> On node 0 totalpages: 32655
> free_area_init_node: node 0, pgdat c041f600, node_mem_map c1000000
>   DMA zone: 3947 pages, LIFO batch:0
>   Normal zone: 28292 pages, LIFO batch:7
> ACPI: PM-Timer IO Port: 0x4008
> SMP: Allowing 1 CPUs, 0 hotplug CPUs
> Found and enabled local APIC!
> mapped APIC to ffffb000 (fee00000)
> PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
> PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
> PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
> Allocating PCI resources starting at 10000000 (gap: 8000000:f7fc0000)
> PERCPU: Allocating 40224 bytes of per cpu data
> NR_CPUS: 32, nr_cpu_ids: 1, nr_node_ids 1
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32239
> Kernel command line: initrd=alt0/all.rdz vga=788 splash=silent BOOT_IMAGE=alt0/vmlinuz vga=0 console=ttyS0,9600 console=tty0 ignore_loglevel debug-alternative
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Initializing CPU#0
> PID hash table entries: 512 (order: 9, 2048 bytes)
> TSC calibrated against PM_TIMER
> Detected 2410.976 MHz processor.
> Console: colour VGA+ 80x25
> console [tty0] enabled
> console [ttyS0] enabled
> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> ... MAX_LOCKDEP_SUBCLASSES:    8
> ... MAX_LOCK_DEPTH:          48
> ... MAX_LOCKDEP_KEYS:        8191
> ... CLASSHASH_SIZE:           4096
> ... MAX_LOCKDEP_ENTRIES:     8192
> ... MAX_LOCKDEP_CHAINS:      16384
> ... CHAINHASH_SIZE:          8192
>  memory used by lock dependency info: 2335 kB
>  per task-struct memory footprint: 1152 bytes
> Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
> Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
> Memory: 117088k/131008k available (2340k kernel code, 13364k reserved, 1027k data, 308k init, 0k highmem)
> virtual kernel memory layout:
>     fixmap  : 0xffe18000 - 0xfffff000   (1948 kB)
>     pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
>     vmalloc : 0xc8800000 - 0xff7fe000   ( 879 MB)
>     lowmem  : 0xc0000000 - 0xc7ff0000   ( 127 MB)
>       .init : 0xc0451000 - 0xc049e000   ( 308 kB)
>       .data : 0xc03493b8 - 0xc044a040   (1027 kB)
>       .text : 0xc0100000 - 0xc03493b8   (2340 kB)
> Checking if this processor honours the WP bit even in supervisor mode...Ok.
> CPA: page pool initialized 1 of 1 pages preallocated
> SLUB: Genslabs=12, HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
> Calibrating delay loop (skipped), value calculated using timer frequency.. 4821.95 BogoMIPS (lpj=2410976)
> Security Framework initialized
> Mount-cache hash table entries: 512
> CPU: Trace cache: 12K uops, L1 D cache: 16K
> CPU: L2 cache: 1024K
> Intel machine check architecture supported.
> Intel machine check reporting enabled on CPU#0.
> Checking 'hlt' instruction... OK.
> apply_alternatives: alt table c048afa4 -> c048f11f
> SMP alternatives: switching to UP code
> Freeing SMP alternatives: 10k freed
> ACPI: Core revision 20080609
> ACPI: setting ELCR to 0200 (from 0c00)
> weird, boot CPU (#0) not listedby the BIOS.
> SMP motherboard not detected.
> SMP disabled
> Brought up 1 CPUs
> Total of 1 processors activated (4821.95 BogoMIPS).
> BUG: unable to handle kernel NULL pointer dereference at 00000246
> IP: [<c012fc71>] vprintk+0x181/0x440
> *pde = 00000000 
> Oops: 0002 [#1] SMP 
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted (2.6.27-rc4-test25 #89)
> EIP: 0060:[<c012fc71>] EFLAGS: 00010246 CPU: 0
> EIP is at vprintk+0x181/0x440
> EAX: 00000246 EBX: 00000000 ECX: c012f8a9 EDX: 00009695
> ESI: 00000000 EDI: c04d76d7 EBP: c7435f98 ESP: c7435f24
>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
> Process swapper (pid: 1, ti=c7434000 task=c7438000 task.ti=c7434000)
> Stack: 00009695 c012f8a9 c7435fb4 0000007b c012007b 000000d8 ffffff10 c7435f5a 
>        00000006 00000246 00000000 c046902c 00000037 00000246 c03438ac c7435f7a 
>        00000006 00000246 00000000 00000000 00000015 c7435f94 c045e900 00000030 
> Call Trace:
>  [<c012f8a9>] ? release_console_sem+0x1c9/0x1e0
>  [<c012007b>] ? resched_task+0x4b/0x70
>  [<c046902c>] ? relay_init+0xd/0x11
>  [<c03438ac>] ? end_local_APIC_setup+0xb9/0xf2
>  [<c045e900>] ? prefill_possible_map+0x7/0x8a
>  [<c03450f4>] ? printk+0x18/0x1c
>  [<c045eacd>] ? native_smp_cpus_done+0x93/0xe9
>  [<c04512f3>] ? kernel_init+0xd5/0x166
>  [<c045121e>] ? kernel_init+0x0/0x166
>  [<c0104b67>] ? kernel_thread_helper+0x7/0x10
>  =======================
> Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8 
> EIP: [<c012fc71>] vprintk+0x181/0x440 SS:ESP 0069:c7435f24
> ---[ end trace 4eaa2a86a8e2da22 ]---
> Kernel panic - not syncing: Attempted to kill init!


-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 16:35             ` Mathieu Desnoyers
@ 2008-08-22 17:20               ` Luiz Fernando N. Capitulino
  2008-08-22 18:11                 ` H. Peter Anvin
  2008-08-22 20:57                 ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 17:20 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: Ingo Molnar, H. Peter Anvin, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 2282 bytes --]

Em Fri, 22 Aug 2008 12:35:20 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| > Em Fri, 22 Aug 2008 11:34:52 -0400
| > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
| > 
| > | * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| > | > Em Fri, 22 Aug 2008 08:50:12 +0200
| > | > Ingo Molnar <mingo@elte.hu> escreveu:
| > | > 
| > | > | 
| > | > | * H. Peter Anvin <hpa@zytor.com> wrote:
| > | > | 
| > | > | > H. Peter Anvin wrote:
| > | > | >>>
| > | > | >>>  Does this look like a kernel bug?
| > | > | >>>
| > | > | >>
| > | > | >> No, it looks like a very common virtualizer bug.  Does the attached  
| > | > | >> patch work for you?
| > | > | >>
| > | > | >
| > | > | > Also, in addition to this, please try tip:master.  There is a patch in 
| > | > | > tip:master which I hope should fix this problem, but the details are 
| > | > | > important.
| > | > | 
| > | > | access coordinates would be at:
| > | > | 
| > | > |   http://people.redhat.com/mingo/tip.git/README
| > | > 
| > | >  As I already have Linus tree downloaded I have cloned it in
| > | > the usual way.
| > | > 
| > | >  Got the same results: OOPS in virtualbox but it works on QEMU.
| > | > 
| > | >  The OOPS's output follows and I have attached the .config I'm using
| > | > to reproduce the problem.
| > | > 
| > | 
| > | Can you try booting with the kernel argument :
| > |   debug_alternative 
| > | 
| > | The dmesg of the kernel bootup up to the oops would be helpful.
| > | 
| > | My guess is that there may be something wrong with irq disabling which
| > | protects text_poke_early in apply_alternatives().
| > 
| >  I have attached two files:
| > 
| >   - normal.txt: normal boot with no debug options
| >   - debug-alternative.txt ignore_loglevel and debug-alternative boot
| >     options
| > 
| >  I had to pass ignore_loglevel otherwise it wouldn't print
| > anything.
| > 
| 
| Ok, now can you try booting with either of those args :
| 
| noreplace-paravirt
| noreplace-smp
| 
| And see which one(s) works ?

 noreplace-paravirt works, the other one causes no change.

 You will find the full boot log (with debug-alternative enabled)
attached.

-- 
Luiz Fernando N. Capitulino

[-- Attachment #2: working.txt --]
[-- Type: text/plain, Size: 8310 bytes --]

Linux version 2.6.27-rc4-test25 (lcapitulino@build1.conectiva) (gcc version 4.3.1 20080626 (prerelease) (GCC) ) #89 SMP Fri Aug 22 12:47:34 BRT 2008
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
 BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
debug: ignoring loglevel setting.
last_pfn = 0x7ff0 max_arch_pfn = 0x100000
kernel direct mapping tables up to 7ff0000 @ 7000-d000
RAMDISK: 07b9b000 - 07fbf89d
DMI 2.5 present.
ACPI: RSDP 000E0000, 0024 (r2 VBOX  )
ACPI: XSDT 07FF0030, 002C (r1 VBOX   VBOXXSDT        1 ASL        61)
ACPI: FACP 07FF0060, 00F4 (r4 VBOX   VBOXFACP        1 ASL        61)
ACPI: DSDT 07FF01A0, 1064 (r1 VBOX   VBOXBIOS        2 INTL 20080213)
ACPI: FACS 07FF0160, 0040
0MB HIGHMEM available.
127MB LOWMEM available.
  mapped low ram: 0 - 07ff0000
  low ram: 00000000 - 07ff0000
  bootmap 00002000 - 00003000
(9 early reservations) ==> bootmem [0000000000 - 0007ff0000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000100000 - 0000814b10]    TEXT DATA BSS ==> [0000100000 - 0000814b10]
  #4 [0007b9b000 - 0007fbf89d]          RAMDISK ==> [0007b9b000 - 0007fbf89d]
  #5 [0000815000 - 0000819000]    INIT_PG_TABLE ==> [0000815000 - 0000819000]
  #6 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #7 [0000007000 - 0000009000]          PGTABLE ==> [0000007000 - 0000009000]
  #8 [0000002000 - 0000003000]          BOOTMAP ==> [0000002000 - 0000003000]
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  Normal   0x00001000 -> 0x00007ff0
  HighMem  0x00007ff0 -> 0x00007ff0
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x00007ff0
On node 0 totalpages: 32655
free_area_init_node: node 0, pgdat c041f600, node_mem_map c1000000
  DMA zone: 3947 pages, LIFO batch:0
  Normal zone: 28292 pages, LIFO batch:7
ACPI: PM-Timer IO Port: 0x4008
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Found and enabled local APIC!
mapped APIC to ffffb000 (fee00000)
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 10000000 (gap: 8000000:f7fc0000)
PERCPU: Allocating 40224 bytes of per cpu data
NR_CPUS: 32, nr_cpu_ids: 1, nr_node_ids 1
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32239
Kernel command line: initrd=alt0/all.rdz vga=788 splash=silent BOOT_IMAGE=alt0/vmlinuz vga=0 console=ttyS0,9600 console=tty ignore_loglevel debug-alternative noreplace-paravirt
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 512 (order: 9, 2048 bytes)
TSC calibrated against PM_TIMER
Detected 2410.580 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
console [ttyS0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:           4096
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 2335 kB
 per task-struct memory footprint: 1152 bytes
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 117088k/131008k available (2340k kernel code, 13364k reserved, 1027k data, 308k init, 0k highmem)
virtual kernel memory layout:
    fixmap  : 0xffe18000 - 0xfffff000   (1948 kB)
    pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
    vmalloc : 0xc8800000 - 0xff7fe000   ( 879 MB)
    lowmem  : 0xc0000000 - 0xc7ff0000   ( 127 MB)
      .init : 0xc0451000 - 0xc049e000   ( 308 kB)
      .data : 0xc03493b8 - 0xc044a040   (1027 kB)
      .text : 0xc0100000 - 0xc03493b8   (2340 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
CPA: page pool initialized 1 of 1 pages preallocated
SLUB: Genslabs=12, HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Calibrating delay loop (skipped), value calculated using timer frequency.. 4821.16 BogoMIPS (lpj=2410580)
Security Framework initialized
Mount-cache hash table entries: 512
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
Checking 'hlt' instruction... OK.
apply_alternatives: alt table c048afa4 -> c048f11f
SMP alternatives: switching to UP code
Freeing SMP alternatives: 10k freed
ACPI: Core revision 20080609
ACPI: setting ELCR to 0200 (from 0c00)
weird, boot CPU (#0) not listedby the BIOS.
SMP motherboard not detected.
SMP disabled
Brought up 1 CPUs
Total of 1 processors activated (4821.16 BogoMIPS).
CPU0 attaching sched-domain:
 domain 0: span 0 level CPU
  groups: 0
khelper used greatest stack depth: 6824 bytes left
net_namespace: 384 bytes
Booting paravirtualized kernel on bare hardware
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfadb0, last bus=0
PCI: Using configuration type 1 for base access
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using PIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
bus 00 -> node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 9 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 9 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 9 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 9 *10 11)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 6 devices
ACPI: ACPI bus type pnp unregistered
PnPBIOS: Disabled by ACPI PNP
PCI: Using ACPI for IRQ routing
bus: 00 index 0 io port: [0, ffff]
bus: 00 index 1 mmio: [0, ffffffffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs...<7>Switched to high resolution mode on CPU 0
 it is
Freeing initrd memory: 4242k freed
khelper used greatest stack depth: 6640 bytes left
apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
apm: overridden by ACPI.
audit: initializing netlink socket (disabled)
type=2000 audit(1219414726.056:1): initialized
HugeTLB registered 4 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
msgmni has been set to 237
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: Activating ISA DMA hang workarounds
pci 0000:00:02.0: Boot video device
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Serial: 8250/16550 driver4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16450
brd: module loaded
PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard as /class/input/input0
cpuidle: using governor ladder
cpuidle: using governor menu
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
TCP cubic registered
Using IPI No-Shortcut mode
registered taskstats version 1
Freeing unused kernel memory: 308k freed

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 17:20               ` Luiz Fernando N. Capitulino
@ 2008-08-22 18:11                 ` H. Peter Anvin
  2008-08-22 19:40                   ` Luiz Fernando N. Capitulino
  2008-08-22 20:57                 ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 18:11 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 138 bytes --]

Hi Luiz, two more tests:

1. a small program to run in userspace and tell us what you get;
2. a patch against -linus for testing.

	-hpa


[-- Attachment #2: nopl.c --]
[-- Type: text/x-csrc, Size: 263 bytes --]

#include <stdio.h>

int main(void)
{
  unsigned int eax, edx;

  asm(".byte 0x0f,0x1f,0x84,0x00,0x98,0x98,0x98,0x98,0x92,0x90,0x90,0x90"
      : "=a" (eax), "=d" (edx)
      : "a" (0x88776655), "d" (0x44332211));

  printf("%08x:%08x\n", edx, eax);
  return 0;
}

[-- Attachment #3: diff --]
[-- Type: text/plain, Size: 386 bytes --]

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 2763cb3..33193fe 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -157,8 +157,6 @@ static const struct nop {
 } noptypes[] = {
 	{ X86_FEATURE_K8, k8_nops },
 	{ X86_FEATURE_K7, k7_nops },
-	{ X86_FEATURE_P4, p6_nops },
-	{ X86_FEATURE_P3, p6_nops },
 	{ -1, NULL }
 };
 

^ permalink raw reply related	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 18:11                 ` H. Peter Anvin
@ 2008-08-22 19:40                   ` Luiz Fernando N. Capitulino
  2008-08-22 20:31                     ` H. Peter Anvin
  0 siblings, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 19:40 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Em Fri, 22 Aug 2008 11:11:25 -0700
"H. Peter Anvin" <hpa@zytor.com> escreveu:

| Hi Luiz, two more tests:
| 
| 1. a small program to run in userspace and tell us what you get;

 88776655:44332211

 It is the same output in the virtualized system and the host
system.

| 2. a patch against -linus for testing.

 I have tried this patch with Linus tree early today, should I try
it with Ingo's tree too?

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 19:40                   ` Luiz Fernando N. Capitulino
@ 2008-08-22 20:31                     ` H. Peter Anvin
  2008-08-22 20:55                       ` Luiz Fernando N. Capitulino
  0 siblings, 1 reply; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 20:31 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Luiz Fernando N. Capitulino wrote:
> 
> | 2. a patch against -linus for testing.
> 
>  I have tried this patch with Linus tree early today, should I try
> it with Ingo's tree too?
> 

It doesn't apply to tip.  This did not fix the problem?

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 20:31                     ` H. Peter Anvin
@ 2008-08-22 20:55                       ` Luiz Fernando N. Capitulino
  0 siblings, 0 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 20:55 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Em Fri, 22 Aug 2008 13:31:49 -0700
"H. Peter Anvin" <hpa@zytor.com> escreveu:

| Luiz Fernando N. Capitulino wrote:
| > 
| > | 2. a patch against -linus for testing.
| > 
| >  I have tried this patch with Linus tree early today, should I try
| > it with Ingo's tree too?
| > 
| 
| It doesn't apply to tip.  This did not fix the problem?

 No, it did not. :(

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 17:20               ` Luiz Fernando N. Capitulino
  2008-08-22 18:11                 ` H. Peter Anvin
@ 2008-08-22 20:57                 ` Luiz Fernando N. Capitulino
  2008-08-22 21:08                   ` H. Peter Anvin
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 20:57 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Mathieu Desnoyers, Ingo Molnar, H. Peter Anvin, linux-kernel

Em Fri, 22 Aug 2008 14:20:54 -0300
"Luiz Fernando N. Capitulino" <lcapitulino@mandriva.com.br> escreveu:

| Em Fri, 22 Aug 2008 12:35:20 -0400
| Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
| 
| | * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| | > Em Fri, 22 Aug 2008 11:34:52 -0400
| | > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
| | > 
| | > | * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| | > | > Em Fri, 22 Aug 2008 08:50:12 +0200
| | > | > Ingo Molnar <mingo@elte.hu> escreveu:
| | > | > 
| | > | > | 
| | > | > | * H. Peter Anvin <hpa@zytor.com> wrote:
| | > | > | 
| | > | > | > H. Peter Anvin wrote:
| | > | > | >>>
| | > | > | >>>  Does this look like a kernel bug?
| | > | > | >>>
| | > | > | >>
| | > | > | >> No, it looks like a very common virtualizer bug.  Does the attached  
| | > | > | >> patch work for you?
| | > | > | >>
| | > | > | >
| | > | > | > Also, in addition to this, please try tip:master.  There is a patch in 
| | > | > | > tip:master which I hope should fix this problem, but the details are 
| | > | > | > important.
| | > | > | 
| | > | > | access coordinates would be at:
| | > | > | 
| | > | > |   http://people.redhat.com/mingo/tip.git/README
| | > | > 
| | > | >  As I already have Linus tree downloaded I have cloned it in
| | > | > the usual way.
| | > | > 
| | > | >  Got the same results: OOPS in virtualbox but it works on QEMU.
| | > | > 
| | > | >  The OOPS's output follows and I have attached the .config I'm using
| | > | > to reproduce the problem.
| | > | > 
| | > | 
| | > | Can you try booting with the kernel argument :
| | > |   debug_alternative 
| | > | 
| | > | The dmesg of the kernel bootup up to the oops would be helpful.
| | > | 
| | > | My guess is that there may be something wrong with irq disabling which
| | > | protects text_poke_early in apply_alternatives().
| | > 
| | >  I have attached two files:
| | > 
| | >   - normal.txt: normal boot with no debug options
| | >   - debug-alternative.txt ignore_loglevel and debug-alternative boot
| | >     options
| | > 
| | >  I had to pass ignore_loglevel otherwise it wouldn't print
| | > anything.
| | > 
| | 
| | Ok, now can you try booting with either of those args :
| | 
| | noreplace-paravirt
| | noreplace-smp
| | 
| | And see which one(s) works ?
| 
|  noreplace-paravirt works, the other one causes no change.

 I have asked Mandriva and Ubuntu users to test this and all of
them so far are saying that noreplace-paravirt works.

 It makes the system slower, but it works.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 20:57                 ` Luiz Fernando N. Capitulino
@ 2008-08-22 21:08                   ` H. Peter Anvin
  2008-08-26 14:18                     ` Gerhard Brauer
  0 siblings, 1 reply; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 21:08 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Luiz Fernando N. Capitulino wrote:
> 
>  I have asked Mandriva and Ubuntu users to test this and all of
> them so far are saying that noreplace-paravirt works.
> 
>  It makes the system slower, but it works.
> 

Yes, the big issue is exactly what VirtualBox screws up in this matter, 
how to detect it, and how to work around it.

It's pretty clear it's a VirtualBox f*ckup at this point, but the 
failure mechanism isn't at all obvious and so far the workaround is elusive.

I'm strongly suspect this is a VirtualBox tcache management failure, but 
that doesn't help the situation without knowing how it happens.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 21:08                   ` H. Peter Anvin
@ 2008-08-26 14:18                     ` Gerhard Brauer
  2008-08-26 14:53                       ` Mathieu Desnoyers
  2008-08-26 16:02                       ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-26 14:18 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Luiz Fernando N. Capitulino, Mathieu Desnoyers, Ingo Molnar,
	linux-kernel

On Fri, Aug 22, 2008 at 02:08:13PM -0700, H. Peter Anvin wrote:
> Luiz Fernando N. Capitulino wrote:
>>
>>  I have asked Mandriva and Ubuntu users to test this and all of
>> them so far are saying that noreplace-paravirt works.
>>
>>  It makes the system slower, but it works.
>>
>
> Yes, the big issue is exactly what VirtualBox screws up in this matter,  
> how to detect it, and how to work around it.
>
> It's pretty clear it's a VirtualBox f*ckup at this point, but the failure 
> mechanism isn't at all obvious and so far the workaround is elusive.
>
> I'm strongly suspect this is a VirtualBox tcache management failure, but  
> that doesn't help the situation without knowing how it happens.

On Archlinux we have the same problem. We have a bugreport here:
http://bugs.archlinux.org/task/11141

Myself test it with a LiveCD/Install-ISO which has 2.6.26 as install
kernel. We have the guest oops on virtualbox-ose, virtualbox-sun and both on
i686 or x86_64 hosts.

Some things i noticed:
- The system boots always when i either enable VT-x in guest settings or
  disable acpi and run the guest with acpi=off.
- The oops occurs always on (disk)-io, no matter which file system i
  use.
- When the oops has occured and the guest has to close and restart then,
  if i don't use VT-x or acpi=off, i always get an oops directly when
  initrd/kernel is starting. Last screen message before the oops then is
  "Freeing SMP alternatives".

Here is also an archive with guest dmesg and messages.log from such an
oops when heavy disk io leads to the oops:
http://bugs.archlinux.org/task/11141?getfile=2445


> 	-hpa

Gerhard

-- 
Standards sind eine tolle Sache.
Ich finde, jeder sollte einen haben.

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 14:18                     ` Gerhard Brauer
@ 2008-08-26 14:53                       ` Mathieu Desnoyers
  2008-08-26 16:09                         ` Luiz Fernando N. Capitulino
                                           ` (2 more replies)
  2008-08-26 16:02                       ` Luiz Fernando N. Capitulino
  1 sibling, 3 replies; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-26 14:53 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: H. Peter Anvin, Luiz Fernando N. Capitulino, Ingo Molnar,
	linux-kernel

* Gerhard Brauer (gerhard.brauer@web.de) wrote:
> On Fri, Aug 22, 2008 at 02:08:13PM -0700, H. Peter Anvin wrote:
> > Luiz Fernando N. Capitulino wrote:
> >>
> >>  I have asked Mandriva and Ubuntu users to test this and all of
> >> them so far are saying that noreplace-paravirt works.
> >>
> >>  It makes the system slower, but it works.
> >>
> >
> > Yes, the big issue is exactly what VirtualBox screws up in this matter,  
> > how to detect it, and how to work around it.
> >
> > It's pretty clear it's a VirtualBox f*ckup at this point, but the failure 
> > mechanism isn't at all obvious and so far the workaround is elusive.
> >
> > I'm strongly suspect this is a VirtualBox tcache management failure, but  
> > that doesn't help the situation without knowing how it happens.
> 
> On Archlinux we have the same problem. We have a bugreport here:
> http://bugs.archlinux.org/task/11141
> 
> Myself test it with a LiveCD/Install-ISO which has 2.6.26 as install
> kernel. We have the guest oops on virtualbox-ose, virtualbox-sun and both on
> i686 or x86_64 hosts.
> 
> Some things i noticed:
> - The system boots always when i either enable VT-x in guest settings or
>   disable acpi and run the guest with acpi=off.
> - The oops occurs always on (disk)-io, no matter which file system i
>   use.
> - When the oops has occured and the guest has to close and restart then,
>   if i don't use VT-x or acpi=off, i always get an oops directly when
>   initrd/kernel is starting. Last screen message before the oops then is
>   "Freeing SMP alternatives".
> 
> Here is also an archive with guest dmesg and messages.log from such an
> oops when heavy disk io leads to the oops:
> http://bugs.archlinux.org/task/11141?getfile=2445
> 

Hrm, can you try this ?

1 - Make sure you kernel is not CONFIG_DEBUG_RODATA

2 - Change the whole text_poke implementation in
arch/x86/kernel/alternative.c to this :

void *__kprobes text_poke(void *addr, const void *opcode, size_t len)
{
  return text_poke_early(addr, opcode, len);
}

If this works, I suspect that the problem comes from a vmap/vunmap
problem. If it still fails, the problem would likely come from a race
with interrupt disabling probably due to missing data/instruction cache
flush.

Then, after having tested (2), try this on top of it :

In arch/x86/kernel/alternative.c, alternatives_smp_switch()

Add   unsigned long flags;
Change 
spin_lock -> spin_lock_irqsave(&smp_alt, flags);
spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);

This will help testing if there is a problem with interrupts coming
shortly after the modification. If it fixes the problem, my guess is
that we should flush the instruction cache (and maybe the data cache ?)
in text_poke and text_poke early when interrupts are off.

Mathieu


> 
> > 	-hpa
> 
> Gerhard
> 
> -- 
> Standards sind eine tolle Sache.
> Ich finde, jeder sollte einen haben.

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 14:53                       ` Mathieu Desnoyers
@ 2008-08-26 16:09                         ` Luiz Fernando N. Capitulino
  2008-08-26 16:13                         ` Luiz Fernando N. Capitulino
  2008-08-26 19:27                         ` Gerhard Brauer
  2 siblings, 0 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-26 16:09 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Tue, 26 Aug 2008 10:53:38 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| * Gerhard Brauer (gerhard.brauer@web.de) wrote:
| > On Fri, Aug 22, 2008 at 02:08:13PM -0700, H. Peter Anvin wrote:
| > > Luiz Fernando N. Capitulino wrote:
| > >>
| > >>  I have asked Mandriva and Ubuntu users to test this and all of
| > >> them so far are saying that noreplace-paravirt works.
| > >>
| > >>  It makes the system slower, but it works.
| > >>
| > >
| > > Yes, the big issue is exactly what VirtualBox screws up in this matter,  
| > > how to detect it, and how to work around it.
| > >
| > > It's pretty clear it's a VirtualBox f*ckup at this point, but the failure 
| > > mechanism isn't at all obvious and so far the workaround is elusive.
| > >
| > > I'm strongly suspect this is a VirtualBox tcache management failure, but  
| > > that doesn't help the situation without knowing how it happens.
| > 
| > On Archlinux we have the same problem. We have a bugreport here:
| > http://bugs.archlinux.org/task/11141
| > 
| > Myself test it with a LiveCD/Install-ISO which has 2.6.26 as install
| > kernel. We have the guest oops on virtualbox-ose, virtualbox-sun and both on
| > i686 or x86_64 hosts.
| > 
| > Some things i noticed:
| > - The system boots always when i either enable VT-x in guest settings or
| >   disable acpi and run the guest with acpi=off.
| > - The oops occurs always on (disk)-io, no matter which file system i
| >   use.
| > - When the oops has occured and the guest has to close and restart then,
| >   if i don't use VT-x or acpi=off, i always get an oops directly when
| >   initrd/kernel is starting. Last screen message before the oops then is
| >   "Freeing SMP alternatives".
| > 
| > Here is also an archive with guest dmesg and messages.log from such an
| > oops when heavy disk io leads to the oops:
| > http://bugs.archlinux.org/task/11141?getfile=2445
| > 
| 
| Hrm, can you try this ?
| 
| 1 - Make sure you kernel is not CONFIG_DEBUG_RODATA

"""
$ grep CONFIG_DEBUG_RODATA .config
# CONFIG_DEBUG_RODATA is not set
$
"""

| 2 - Change the whole text_poke implementation in
| arch/x86/kernel/alternative.c to this :
| 
| void *__kprobes text_poke(void *addr, const void *opcode, size_t len)
| {
|   return text_poke_early(addr, opcode, len);
| }
| 
| If this works, I suspect that the problem comes from a vmap/vunmap
| problem. If it still fails, the problem would likely come from a race
| with interrupt disabling probably due to missing data/instruction cache
| flush.

 I still get the oops with this change. :((

| Then, after having tested (2), try this on top of it :
| 
| In arch/x86/kernel/alternative.c, alternatives_smp_switch()
| 
| Add   unsigned long flags;
| Change 
| spin_lock -> spin_lock_irqsave(&smp_alt, flags);
| spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);
| 
| This will help testing if there is a problem with interrupts coming
| shortly after the modification. If it fixes the problem, my guess is
| that we should flush the instruction cache (and maybe the data cache ?)
| in text_poke and text_poke early when interrupts are off.

 By 'on top of it' you mean I should make these changes with the
text_poke() version above right?

 By the way, I have added a comment in the virtualbox's bugzilla
pointing out this thread but no feedback from them so far.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 14:53                       ` Mathieu Desnoyers
  2008-08-26 16:09                         ` Luiz Fernando N. Capitulino
@ 2008-08-26 16:13                         ` Luiz Fernando N. Capitulino
  2008-08-26 17:18                           ` Mathieu Desnoyers
  2008-08-26 19:27                         ` Gerhard Brauer
  2 siblings, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-26 16:13 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Tue, 26 Aug 2008 10:53:38 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| Then, after having tested (2), try this on top of it :
| 
| In arch/x86/kernel/alternative.c, alternatives_smp_switch()
| 
| Add   unsigned long flags;
| Change 
| spin_lock -> spin_lock_irqsave(&smp_alt, flags);
| spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);

 Hmm, I can't find spin_lock functions in alternatives_smp_switch()
looks like the current implementation is now using mutexes.

 What tree are you referring to?

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 16:13                         ` Luiz Fernando N. Capitulino
@ 2008-08-26 17:18                           ` Mathieu Desnoyers
  2008-08-26 17:32                             ` H. Peter Anvin
  2008-08-26 18:02                             ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-26 17:18 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

* Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> Em Tue, 26 Aug 2008 10:53:38 -0400
> Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
> 
> | Then, after having tested (2), try this on top of it :
> | 
> | In arch/x86/kernel/alternative.c, alternatives_smp_switch()
> | 
> | Add   unsigned long flags;
> | Change 
> | spin_lock -> spin_lock_irqsave(&smp_alt, flags);
> | spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);
> 
>  Hmm, I can't find spin_lock functions in alternatives_smp_switch()
> looks like the current implementation is now using mutexes.
> 

Sorry, I was looking directly at the commit which caused the problem.
Yes, these modif should go on top of the text_poke -> text_poke_early.

So in current mainline, change, in alternatives_smp_switch() :

mutex_lock(&smp_alt);
...

mutex_unlock(&smp_alt);

to

mutex_lock(&smp_alt);
local_irq_save(flags);
...

local_irq_restore(flags);
mutex_unlock(&smp_alt);

Thanks,

Mathieu

>  What tree are you referring to?
> 
> -- 
> Luiz Fernando N. Capitulino

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 17:18                           ` Mathieu Desnoyers
@ 2008-08-26 17:32                             ` H. Peter Anvin
  2008-08-26 18:02                             ` Luiz Fernando N. Capitulino
  1 sibling, 0 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-26 17:32 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, Gerhard Brauer, Ingo Molnar,
	linux-kernel, Colin Ian King

I have been unable to replicate this on my own hardware mostly because 
my testing machine decided to blow its DVD drive in some very strange 
way, but I did pick apart the data from Luiz, and found it very interesting:

The code sequence before patching looks like:

c012fc69:       51                      push   %ecx
c012fc6a:       52                      push   %edx
c012fc6b:       ff 15 40 b9 41 c0       call   *0xc041b940
c012fc71:       5a                      pop    %edx
c012fc72:       59                      pop    %ecx

After patching:

50 9d 0f 1f 84 00 00 00 <00> 00

... which disassembles to (in Intel notation):

C012FC69  50                push eax
C012FC6A  9D                popfd
C012FC6B  0F1F840000000000  nop dword [eax+eax+0x0]

We do, indeed have a return point that falls in the *middle* of a 
patched instruction, and if the patching happens in the middle of the 
instruction call, then, well, bad things happen.

Furthermore, why on Earth is %ecx/%edx pushed and popped in-line here? 
Surely it should be the responsibility of the PV call to present a 
no-clobber interface (using an assembly wrapper if necessary[*]), rather 
than bloating every callsite like this?

	-hpa

[*] One can compile gcc code with -fcall-saved-* to use nonstandard 
register conventions.  Unfortunately stock gcc only lets you do this 
with a file parameter, and doesn't support doing this with attributes.

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 17:18                           ` Mathieu Desnoyers
  2008-08-26 17:32                             ` H. Peter Anvin
@ 2008-08-26 18:02                             ` Luiz Fernando N. Capitulino
  2008-08-26 18:15                               ` Mathieu Desnoyers
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-26 18:02 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Tue, 26 Aug 2008 13:18:22 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| > Em Tue, 26 Aug 2008 10:53:38 -0400
| > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
| > 
| > | Then, after having tested (2), try this on top of it :
| > | 
| > | In arch/x86/kernel/alternative.c, alternatives_smp_switch()
| > | 
| > | Add   unsigned long flags;
| > | Change 
| > | spin_lock -> spin_lock_irqsave(&smp_alt, flags);
| > | spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);
| > 
| >  Hmm, I can't find spin_lock functions in alternatives_smp_switch()
| > looks like the current implementation is now using mutexes.
| > 
| 
| Sorry, I was looking directly at the commit which caused the problem.
| Yes, these modif should go on top of the text_poke -> text_poke_early.
| 
| So in current mainline, change, in alternatives_smp_switch() :
| 
| mutex_lock(&smp_alt);
| ...
| 
| mutex_unlock(&smp_alt);
| 
| to
| 
| mutex_lock(&smp_alt);
| local_irq_save(flags);
| ...
| 
| local_irq_restore(flags);
| mutex_unlock(&smp_alt);

 Did not help, same oops here.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 18:02                             ` Luiz Fernando N. Capitulino
@ 2008-08-26 18:15                               ` Mathieu Desnoyers
  2008-08-26 19:52                                 ` H. Peter Anvin
  2008-08-26 20:34                                 ` Gerhard Brauer
  0 siblings, 2 replies; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-26 18:15 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

* Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> Em Tue, 26 Aug 2008 13:18:22 -0400
> Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
> 
> | * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> | > Em Tue, 26 Aug 2008 10:53:38 -0400
> | > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
> | > 
> | > | Then, after having tested (2), try this on top of it :
> | > | 
> | > | In arch/x86/kernel/alternative.c, alternatives_smp_switch()
> | > | 
> | > | Add   unsigned long flags;
> | > | Change 
> | > | spin_lock -> spin_lock_irqsave(&smp_alt, flags);
> | > | spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);
> | > 
> | >  Hmm, I can't find spin_lock functions in alternatives_smp_switch()
> | > looks like the current implementation is now using mutexes.
> | > 
> | 
> | Sorry, I was looking directly at the commit which caused the problem.
> | Yes, these modif should go on top of the text_poke -> text_poke_early.
> | 
> | So in current mainline, change, in alternatives_smp_switch() :
> | 
> | mutex_lock(&smp_alt);
> | ...
> | 
> | mutex_unlock(&smp_alt);
> | 
> | to
> | 
> | mutex_lock(&smp_alt);
> | local_irq_save(flags);
> | ...
> | 
> | local_irq_restore(flags);
> | mutex_unlock(&smp_alt);
> 
>  Did not help, same oops here.
> 

Ok, it might still be caused by paravirt and alternatives instruction
patching. What if you also do :

alternative_instructions()

+        unsigned long flags;
        /* The patching is not fully atomic, so try to avoid local interruptions
           that might execute the to be patched code.
           Other CPUs are not running. */
        stop_nmi();
#ifdef CONFIG_X86_MCE
        stop_mce();
#endif
+        local_irq_save(flags);


...
+        local_irq_restore(flags);
        restart_nmi();
#ifdef CONFIG_X86_MCE
        restart_mce();
#endif

?

Hrm,

Since those local_irq_save/restore occur _before_ the paravirt patching
is done, I wonder if there would be a race in the way cli/sti traps are
handled by Virtualbox wrt incoming interrupt ?

Thanks,

Mathieu

> -- 
> Luiz Fernando N. Capitulino

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 18:15                               ` Mathieu Desnoyers
@ 2008-08-26 19:52                                 ` H. Peter Anvin
  2008-08-26 20:34                                 ` Gerhard Brauer
  1 sibling, 0 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-26 19:52 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, Gerhard Brauer, Ingo Molnar,
	linux-kernel

One thing that I think really needs to be considered is that the current 
PV stubs are (a) large, and (b) non-atomic.

In the case at hand we have:

c012fc69:       51                      push   %ecx
c012fc6a:       52                      push   %edx
c012fc6b:       ff 15 40 b9 41 c0       call   *0xc041b940
c012fc71:       5a                      pop    %edx
c012fc72:       59                      pop    %ecx

Ten bytes replacing a two-byte native sequence.

If this was done as a call to an out-of-line stub, it would be only five 
bytes, which would reduce native icache overhead from 400% to 150%, but 
perhaps more importantly, it would not be subject to returns inside the 
sequence itself (since the out-of-line stub would still exist.)  As an 
optional bonus, at least on 32 bits the indirect call could be replaced 
with a direct call in the out-of-line stub.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 18:15                               ` Mathieu Desnoyers
  2008-08-26 19:52                                 ` H. Peter Anvin
@ 2008-08-26 20:34                                 ` Gerhard Brauer
  2008-08-26 20:48                                   ` Mathieu Desnoyers
  2008-08-27 19:13                                   ` Luiz Fernando N. Capitulino
  1 sibling, 2 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-26 20:34 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

On Tue, Aug 26, 2008 at 02:15:58PM -0400, Mathieu Desnoyers wrote:
> 
> Ok, it might still be caused by paravirt and alternatives instruction
> patching. What if you also do :
> 
> alternative_instructions()
> 
> +        unsigned long flags;
>         /* The patching is not fully atomic, so try to avoid local interruptions
>            that might execute the to be patched code.
>            Other CPUs are not running. */
>         stop_nmi();
> #ifdef CONFIG_X86_MCE
>         stop_mce();
> #endif
> +        local_irq_save(flags);
> 
> 
> ...
> +        local_irq_restore(flags);
>         restart_nmi();
> #ifdef CONFIG_X86_MCE
>         restart_mce();
> #endif
> 
> ?

Hej! This last changes (in addition to the others you mentioned) seems
to be a good shot. I could reboot 8 times the guest, compile several
packages (something which always leeds to the oops) and currently i
build two big packages simultan. So this is heavy IO.

I will try tomorrow more heavy build tests (to gain the good feeling to
the vbox+guest kernel again like it was with 2.6.25), but i think your
changes goes in the right direction.

Here is the diff what i've changed on your hints:

,----[ arch/x86/kernel/alternative.c ]
| --- alternative.c.org	2008-07-13 23:51:29.000000000 +0200
| +++ alternative.c	2008-08-26 21:35:20.000000000 +0200
| @@ -343,6 +343,7 @@
|  void alternatives_smp_switch(int smp)
|  {
|  	struct smp_alt_module *mod;
| +	unsigned long flags;
|  
|  #ifdef CONFIG_LOCKDEP
|  	/*
| @@ -359,7 +360,7 @@
|  		return;
|  	BUG_ON(!smp && (num_online_cpus() > 1));
|  
| -	spin_lock(&smp_alt);
| +	spin_lock_irqsave(&smp_alt, flags);
|  
|  	/*
|  	 * Avoid unnecessary switches because it forces JIT based VMs to
| @@ -383,7 +384,7 @@
|  						mod->text, mod->text_end);
|  	}
|  	smp_mode = smp;
| -	spin_unlock(&smp_alt);
| +	spin_unlock_irqrestore(&smp_alt, flags);
|  }
|  
|  #endif
| @@ -420,6 +421,7 @@
|  
|  void __init alternative_instructions(void)
|  {
| +	unsigned long flags;
|  	/* The patching is not fully atomic, so try to avoid local interruptions
|  	   that might execute the to be patched code.
|  	   Other CPUs are not running. */
| @@ -427,6 +429,7 @@
|  #ifdef CONFIG_X86_MCE
|  	stop_mce();
|  #endif
| +	local_irq_save(flags);
|  
|  	apply_alternatives(__alt_instructions, __alt_instructions_end);
|  
| @@ -465,6 +468,7 @@
|  				(unsigned long)__smp_locks,
|  				(unsigned long)__smp_locks_end);
|  
| +	local_irq_restore(flags);
|  	restart_nmi();
|  #ifdef CONFIG_X86_MCE
|  	restart_mce();
| @@ -508,33 +512,5 @@
|   */
|  void *__kprobes text_poke(void *addr, const void *opcode, size_t len)
|  {
| -	unsigned long flags;
| -	char *vaddr;
| -	int nr_pages = 2;
| -	struct page *pages[2];
| -	int i;
| -
| -	if (!core_kernel_text((unsigned long)addr)) {
| -		pages[0] = vmalloc_to_page(addr);
| -		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
| -	} else {
| -		pages[0] = virt_to_page(addr);
| -		WARN_ON(!PageReserved(pages[0]));
| -		pages[1] = virt_to_page(addr + PAGE_SIZE);
| -	}
| -	BUG_ON(!pages[0]);
| -	if (!pages[1])
| -		nr_pages = 1;
| -	vaddr = vmap(pages, nr_pages, VM_MAP, PAGE_KERNEL);
| -	BUG_ON(!vaddr);
| -	local_irq_save(flags);
| -	memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len);
| -	local_irq_restore(flags);
| -	vunmap(vaddr);
| -	sync_core();
| -	/* Could also do a CLFLUSH here to speed up CPU recovery; but
| -	   that causes hangs on some VIA CPUs. */
| -	for (i = 0; i < len; i++)
| -		BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]);
| -	return addr;
| +	return text_poke_early(addr, opcode, len);
|  }
`----

So if Luiz and others could also try all 3 mentioned changes, maybe we
have a solution. I also will build tomorrow a new LiveCD/Install ISO
with these patches to see if the error there is also gone.

> Thanks,
> 
> Mathieu

Gerhard


-- 
Was wir wissen, ist ein Tropfen.
Was wir nicht wissen, ein Ozean (Newton)

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 20:34                                 ` Gerhard Brauer
@ 2008-08-26 20:48                                   ` Mathieu Desnoyers
  2008-08-26 21:25                                     ` Gerhard Brauer
  2008-08-27 19:13                                   ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-26 20:48 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

* Gerhard Brauer (gerhard.brauer@web.de) wrote:
> On Tue, Aug 26, 2008 at 02:15:58PM -0400, Mathieu Desnoyers wrote:
> > 
> > Ok, it might still be caused by paravirt and alternatives instruction
> > patching. What if you also do :
> > 
> > alternative_instructions()
> > 
> > +        unsigned long flags;
> >         /* The patching is not fully atomic, so try to avoid local interruptions
> >            that might execute the to be patched code.
> >            Other CPUs are not running. */
> >         stop_nmi();
> > #ifdef CONFIG_X86_MCE
> >         stop_mce();
> > #endif
> > +        local_irq_save(flags);
> > 
> > 
> > ...
> > +        local_irq_restore(flags);
> >         restart_nmi();
> > #ifdef CONFIG_X86_MCE
> >         restart_mce();
> > #endif
> > 
> > ?
> 
> Hej! This last changes (in addition to the others you mentioned) seems
> to be a good shot. I could reboot 8 times the guest, compile several
> packages (something which always leeds to the oops) and currently i
> build two big packages simultan. So this is heavy IO.
> 
> I will try tomorrow more heavy build tests (to gain the good feeling to
> the vbox+guest kernel again like it was with 2.6.25), but i think your
> changes goes in the right direction.
> 

OK, so we have a problem with interrupts coming while we are doing the
alternatives patching.

First thing, I wonder if Virtualbox expects the OS to patch all its
paravirt instructions in one go ?

Also, could you then try to :
- to revert all those changes
- Do this to text_poke_early and text_poke :

- put the sync_core() within the irq off critical section
(test)
- add a wbinvd();  just after the sync_core() in both functions
(test).

Thanks,

Mathieu


> Here is the diff what i've changed on your hints:
> 
> ,----[ arch/x86/kernel/alternative.c ]
> | --- alternative.c.org	2008-07-13 23:51:29.000000000 +0200
> | +++ alternative.c	2008-08-26 21:35:20.000000000 +0200
> | @@ -343,6 +343,7 @@
> |  void alternatives_smp_switch(int smp)
> |  {
> |  	struct smp_alt_module *mod;
> | +	unsigned long flags;
> |  
> |  #ifdef CONFIG_LOCKDEP
> |  	/*
> | @@ -359,7 +360,7 @@
> |  		return;
> |  	BUG_ON(!smp && (num_online_cpus() > 1));
> |  
> | -	spin_lock(&smp_alt);
> | +	spin_lock_irqsave(&smp_alt, flags);
> |  
> |  	/*
> |  	 * Avoid unnecessary switches because it forces JIT based VMs to
> | @@ -383,7 +384,7 @@
> |  						mod->text, mod->text_end);
> |  	}
> |  	smp_mode = smp;
> | -	spin_unlock(&smp_alt);
> | +	spin_unlock_irqrestore(&smp_alt, flags);
> |  }
> |  
> |  #endif
> | @@ -420,6 +421,7 @@
> |  
> |  void __init alternative_instructions(void)
> |  {
> | +	unsigned long flags;
> |  	/* The patching is not fully atomic, so try to avoid local interruptions
> |  	   that might execute the to be patched code.
> |  	   Other CPUs are not running. */
> | @@ -427,6 +429,7 @@
> |  #ifdef CONFIG_X86_MCE
> |  	stop_mce();
> |  #endif
> | +	local_irq_save(flags);
> |  
> |  	apply_alternatives(__alt_instructions, __alt_instructions_end);
> |  
> | @@ -465,6 +468,7 @@
> |  				(unsigned long)__smp_locks,
> |  				(unsigned long)__smp_locks_end);
> |  
> | +	local_irq_restore(flags);
> |  	restart_nmi();
> |  #ifdef CONFIG_X86_MCE
> |  	restart_mce();
> | @@ -508,33 +512,5 @@
> |   */
> |  void *__kprobes text_poke(void *addr, const void *opcode, size_t len)
> |  {
> | -	unsigned long flags;
> | -	char *vaddr;
> | -	int nr_pages = 2;
> | -	struct page *pages[2];
> | -	int i;
> | -
> | -	if (!core_kernel_text((unsigned long)addr)) {
> | -		pages[0] = vmalloc_to_page(addr);
> | -		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
> | -	} else {
> | -		pages[0] = virt_to_page(addr);
> | -		WARN_ON(!PageReserved(pages[0]));
> | -		pages[1] = virt_to_page(addr + PAGE_SIZE);
> | -	}
> | -	BUG_ON(!pages[0]);
> | -	if (!pages[1])
> | -		nr_pages = 1;
> | -	vaddr = vmap(pages, nr_pages, VM_MAP, PAGE_KERNEL);
> | -	BUG_ON(!vaddr);
> | -	local_irq_save(flags);
> | -	memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len);
> | -	local_irq_restore(flags);
> | -	vunmap(vaddr);
> | -	sync_core();
> | -	/* Could also do a CLFLUSH here to speed up CPU recovery; but
> | -	   that causes hangs on some VIA CPUs. */
> | -	for (i = 0; i < len; i++)
> | -		BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]);
> | -	return addr;
> | +	return text_poke_early(addr, opcode, len);
> |  }
> `----
> 
> So if Luiz and others could also try all 3 mentioned changes, maybe we
> have a solution. I also will build tomorrow a new LiveCD/Install ISO
> with these patches to see if the error there is also gone.
> 
> > Thanks,
> > 
> > Mathieu
> 
> Gerhard
> 
> 
> -- 
> Was wir wissen, ist ein Tropfen.
> Was wir nicht wissen, ein Ozean (Newton)

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 20:48                                   ` Mathieu Desnoyers
@ 2008-08-26 21:25                                     ` Gerhard Brauer
  2008-08-26 21:35                                       ` Mathieu Desnoyers
  0 siblings, 1 reply; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-26 21:25 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

On Tue, Aug 26, 2008 at 04:48:14PM -0400, Mathieu Desnoyers wrote:
> 
> OK, so we have a problem with interrupts coming while we are doing the
> alternatives patching.
> 
> First thing, I wonder if Virtualbox expects the OS to patch all its
> paravirt instructions in one go ?
> 
> Also, could you then try to :
> - to revert all those changes
> - Do this to text_poke_early and text_poke :
> 
> - put the sync_core() within the irq off critical section
> (test)

Could you please explain more what to change? I don't see where to put
sync_core(), i not found this section in both functions. (I'm not a developer)

> - add a wbinvd();  just after the sync_core() in both functions
> (test).

Also verbose please...

> Thanks,
> 
> Mathieu

Thank you
	Gerhard


^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 21:25                                     ` Gerhard Brauer
@ 2008-08-26 21:35                                       ` Mathieu Desnoyers
  2008-08-26 21:51                                         ` H. Peter Anvin
  2008-08-27  0:13                                         ` Gerhard Brauer
  0 siblings, 2 replies; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-26 21:35 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

* Gerhard Brauer (gerhard.brauer@web.de) wrote:
> On Tue, Aug 26, 2008 at 04:48:14PM -0400, Mathieu Desnoyers wrote:
> > 
> > OK, so we have a problem with interrupts coming while we are doing the
> > alternatives patching.
> > 
> > First thing, I wonder if Virtualbox expects the OS to patch all its
> > paravirt instructions in one go ?
> > 
> > Also, could you then try to :
> > - to revert all those changes
> > - Do this to text_poke_early and text_poke :
> > 
> > - put the sync_core() within the irq off critical section
> > (test)
> 
> Could you please explain more what to change? I don't see where to put
> sync_core(), i not found this section in both functions. (I'm not a developer)
> 

Sure,

First patch to test :

x86 alternative text_poke move sync_core

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 arch/x86/kernel/alternative.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/arch/x86/kernel/alternative.c
===================================================================
--- linux-2.6-lttng.orig/arch/x86/kernel/alternative.c	2008-08-26 17:26:41.000000000 -0400
+++ linux-2.6-lttng/arch/x86/kernel/alternative.c	2008-08-26 17:26:58.000000000 -0400
@@ -488,8 +488,8 @@ void *text_poke_early(void *addr, const 
 	unsigned long flags;
 	local_irq_save(flags);
 	memcpy(addr, opcode, len);
-	local_irq_restore(flags);
 	sync_core();
+	local_irq_restore(flags);
 	/* Could also do a CLFLUSH here to speed up CPU recovery; but
 	   that causes hangs on some VIA CPUs. */
 	return addr;
@@ -529,9 +529,9 @@ void *__kprobes text_poke(void *addr, co
 	BUG_ON(!vaddr);
 	local_irq_save(flags);
 	memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len);
+	sync_core();
 	local_irq_restore(flags);
 	vunmap(vaddr);
-	sync_core();
 	/* Could also do a CLFLUSH here to speed up CPU recovery; but
 	   that causes hangs on some VIA CPUs. */
 	for (i = 0; i < len; i++)


> > - add a wbinvd();  just after the sync_core() in both functions
> > (test).
> 
> Also verbose please...
> 

Second patch to apply on top of the first one :


x86 alternative text_poke add wbinvd

Add a cache flush instruction before reenabling interrupts in text_poke.

If this works, we could use clflush() (which is sadly buggy on some archs) which
is faster since it only clear a cacheline instead of the entire cache.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 arch/x86/kernel/alternative.c |    2 ++
 1 file changed, 2 insertions(+)

Index: linux-2.6-lttng/arch/x86/kernel/alternative.c
===================================================================
--- linux-2.6-lttng.orig/arch/x86/kernel/alternative.c	2008-08-26 17:27:33.000000000 -0400
+++ linux-2.6-lttng/arch/x86/kernel/alternative.c	2008-08-26 17:27:53.000000000 -0400
@@ -489,6 +489,7 @@ void *text_poke_early(void *addr, const 
 	local_irq_save(flags);
 	memcpy(addr, opcode, len);
 	sync_core();
+	wbinvd();
 	local_irq_restore(flags);
 	/* Could also do a CLFLUSH here to speed up CPU recovery; but
 	   that causes hangs on some VIA CPUs. */
@@ -530,6 +531,7 @@ void *__kprobes text_poke(void *addr, co
 	local_irq_save(flags);
 	memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len);
 	sync_core();
+	wbinvd();
 	local_irq_restore(flags);
 	vunmap(vaddr);
 	/* Could also do a CLFLUSH here to speed up CPU recovery; but



Thanks,

Mathieu

> > Thanks,
> > 
> > Mathieu
> 
> Thank you
> 	Gerhard
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 21:35                                       ` Mathieu Desnoyers
@ 2008-08-26 21:51                                         ` H. Peter Anvin
  2008-08-27  0:13                                         ` Gerhard Brauer
  1 sibling, 0 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-26 21:51 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Gerhard Brauer, Luiz Fernando N. Capitulino, Ingo Molnar,
	linux-kernel

Mathieu Desnoyers wrote:
> 
> x86 alternative text_poke add wbinvd
> 
> Add a cache flush instruction before reenabling interrupts in text_poke.
> 
> If this works, we could use clflush() (which is sadly buggy on some archs) which
> is faster since it only clear a cacheline instead of the entire cache.
> 

Well, in this case it's VirtualBox we're talking about, a virtual 
architecture.  It's hard to know what it will do under *any* circumstances.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 21:35                                       ` Mathieu Desnoyers
  2008-08-26 21:51                                         ` H. Peter Anvin
@ 2008-08-27  0:13                                         ` Gerhard Brauer
  1 sibling, 0 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-27  0:13 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

On Tue, Aug 26, 2008 at 05:35:59PM -0400, Mathieu Desnoyers wrote:
> * Gerhard Brauer (gerhard.brauer@web.de) wrote:
> 
> First patch to test :
> 
> x86 alternative text_poke move sync_core

With this got the oops again when compiling in guest. Reboot afterwards
leads to the early oops.

> Second patch to apply on top of the first one :
>  
> x86 alternative text_poke add wbinvd

With second patch i get the early oops after "freeing smp". Seems no way
to get the guest bootet normaly (ony with replace-paravirt).

So the changes from the other mail took more effect IMHO.
I could test more tomorrow if you have more ideas.

> Mathieu

Gerhard


^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 20:34                                 ` Gerhard Brauer
  2008-08-26 20:48                                   ` Mathieu Desnoyers
@ 2008-08-27 19:13                                   ` Luiz Fernando N. Capitulino
  2008-08-27 23:33                                     ` Mathieu Desnoyers
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-27 19:13 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Mathieu Desnoyers, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Tue, 26 Aug 2008 22:34:49 +0200
Gerhard Brauer <gerhard.brauer@web.de> escreveu:

| On Tue, Aug 26, 2008 at 02:15:58PM -0400, Mathieu Desnoyers wrote:
| > 
| > Ok, it might still be caused by paravirt and alternatives instruction
| > patching. What if you also do :
| > 
| > alternative_instructions()
| > 
| > +        unsigned long flags;
| >         /* The patching is not fully atomic, so try to avoid local interruptions
| >            that might execute the to be patched code.
| >            Other CPUs are not running. */
| >         stop_nmi();
| > #ifdef CONFIG_X86_MCE
| >         stop_mce();
| > #endif
| > +        local_irq_save(flags);
| > 
| > 
| > ...
| > +        local_irq_restore(flags);
| >         restart_nmi();
| > #ifdef CONFIG_X86_MCE
| >         restart_mce();
| > #endif
| > 
| > ?
| 
| Hej! This last changes (in addition to the others you mentioned) seems
| to be a good shot. I could reboot 8 times the guest, compile several
| packages (something which always leeds to the oops) and currently i
| build two big packages simultan. So this is heavy IO.

 Yeah, it works for me too and it's good to know that you are doing
additional tests. I'm doing only boot tests... I was testing lots of
kernels and doing additional tests would take a lot of time.

 Now, what does this mean? Is VirtualBox issuing interrupts when it
shouldn't or should this section of the code be better protected?

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-27 19:13                                   ` Luiz Fernando N. Capitulino
@ 2008-08-27 23:33                                     ` Mathieu Desnoyers
  2008-08-28 13:30                                       ` Luiz Fernando N. Capitulino
  2008-08-28 13:50                                       ` Gerhard Brauer
  0 siblings, 2 replies; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-27 23:33 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

* Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
> Em Tue, 26 Aug 2008 22:34:49 +0200
> Gerhard Brauer <gerhard.brauer@web.de> escreveu:
> 
> | On Tue, Aug 26, 2008 at 02:15:58PM -0400, Mathieu Desnoyers wrote:
> | > 
> | > Ok, it might still be caused by paravirt and alternatives instruction
> | > patching. What if you also do :
> | > 
> | > alternative_instructions()
> | > 
> | > +        unsigned long flags;
> | >         /* The patching is not fully atomic, so try to avoid local interruptions
> | >            that might execute the to be patched code.
> | >            Other CPUs are not running. */
> | >         stop_nmi();
> | > #ifdef CONFIG_X86_MCE
> | >         stop_mce();
> | > #endif
> | > +        local_irq_save(flags);
> | > 
> | > 
> | > ...
> | > +        local_irq_restore(flags);
> | >         restart_nmi();
> | > #ifdef CONFIG_X86_MCE
> | >         restart_mce();
> | > #endif
> | > 
> | > ?
> | 
> | Hej! This last changes (in addition to the others you mentioned) seems
> | to be a good shot. I could reboot 8 times the guest, compile several
> | packages (something which always leeds to the oops) and currently i
> | build two big packages simultan. So this is heavy IO.
> 
>  Yeah, it works for me too and it's good to know that you are doing
> additional tests. I'm doing only boot tests... I was testing lots of
> kernels and doing additional tests would take a lot of time.
> 
>  Now, what does this mean? Is VirtualBox issuing interrupts when it
> shouldn't or should this section of the code be better protected?
> 

Since this problem appears while we are using a simple memcpy (the
text_poke_early version), but disappears when we disable interrupts for
a longer period of this, I suspect a problem with irq disabling in
Virtualbox.

We could try to add some nsleep() or msleep() calls within text_poke and
text_poke_early before and after the code modificatoin to see if the
problem disappears. If it does, then that would somewhat confirm the
racy irq disable thesis.

Mathieu

> -- 
> Luiz Fernando N. Capitulino

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-27 23:33                                     ` Mathieu Desnoyers
@ 2008-08-28 13:30                                       ` Luiz Fernando N. Capitulino
  2008-08-31  9:29                                         ` Gerhard Brauer
  2008-08-28 13:50                                       ` Gerhard Brauer
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-28 13:30 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Gerhard Brauer, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Wed, 27 Aug 2008 19:33:28 -0400
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:

| * Luiz Fernando N. Capitulino (lcapitulino@mandriva.com.br) wrote:
| > Em Tue, 26 Aug 2008 22:34:49 +0200
| > Gerhard Brauer <gerhard.brauer@web.de> escreveu:
| > 
| > | On Tue, Aug 26, 2008 at 02:15:58PM -0400, Mathieu Desnoyers wrote:
| > | > 
| > | > Ok, it might still be caused by paravirt and alternatives instruction
| > | > patching. What if you also do :
| > | > 
| > | > alternative_instructions()
| > | > 
| > | > +        unsigned long flags;
| > | >         /* The patching is not fully atomic, so try to avoid local interruptions
| > | >            that might execute the to be patched code.
| > | >            Other CPUs are not running. */
| > | >         stop_nmi();
| > | > #ifdef CONFIG_X86_MCE
| > | >         stop_mce();
| > | > #endif
| > | > +        local_irq_save(flags);
| > | > 
| > | > 
| > | > ...
| > | > +        local_irq_restore(flags);
| > | >         restart_nmi();
| > | > #ifdef CONFIG_X86_MCE
| > | >         restart_mce();
| > | > #endif
| > | > 
| > | > ?
| > | 
| > | Hej! This last changes (in addition to the others you mentioned) seems
| > | to be a good shot. I could reboot 8 times the guest, compile several
| > | packages (something which always leeds to the oops) and currently i
| > | build two big packages simultan. So this is heavy IO.
| > 
| >  Yeah, it works for me too and it's good to know that you are doing
| > additional tests. I'm doing only boot tests... I was testing lots of
| > kernels and doing additional tests would take a lot of time.
| > 
| >  Now, what does this mean? Is VirtualBox issuing interrupts when it
| > shouldn't or should this section of the code be better protected?
| > 
| 
| Since this problem appears while we are using a simple memcpy (the
| text_poke_early version), but disappears when we disable interrupts for
| a longer period of this, I suspect a problem with irq disabling in
| Virtualbox.
| 
| We could try to add some nsleep() or msleep() calls within text_poke and
| text_poke_early before and after the code modificatoin to see if the
| problem disappears. If it does, then that would somewhat confirm the
| racy irq disable thesis.

 Well, a Ubuntu kernel guy has reported in the virtualbox's ticket[1]
that the oops doesn't happen if he puts a printk() in the crash site.

 The funny thing is that someone (who might be a virtualbox developer)
used the same race argument to say that this is a bug in the kernel.

 What concerns me though is that how can virtualbox be worth using
in the Linux community if it's probably not working for various distros
(currently Fedora, Ubuntu, Mandriva and ArchLinux).

 Thanks for the effort, guys.

[1] http://www.virtualbox.org/ticket/1875

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-28 13:30                                       ` Luiz Fernando N. Capitulino
@ 2008-08-31  9:29                                         ` Gerhard Brauer
  2008-08-31 13:28                                           ` Stefan Lippers-Hollmann
  2008-08-31 14:09                                           ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-31  9:29 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Mathieu Desnoyers, H. Peter Anvin, Ingo Molnar, linux-kernel

On Thu, Aug 28, 2008 at 10:30:13AM -0300, Luiz Fernando N. Capitulino wrote:
> Em Wed, 27 Aug 2008 19:33:28 -0400
> Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
> | 
> | Since this problem appears while we are using a simple memcpy (the
> | text_poke_early version), but disappears when we disable interrupts for
> | a longer period of this, I suspect a problem with irq disabling in
> | Virtualbox.
> | 
> | We could try to add some nsleep() or msleep() calls within text_poke and
> | text_poke_early before and after the code modificatoin to see if the
> | problem disappears. If it does, then that would somewhat confirm the
> | racy irq disable thesis.
> 
>  Well, a Ubuntu kernel guy has reported in the virtualbox's ticket[1]
> that the oops doesn't happen if he puts a printk() in the crash site.
> 
>  The funny thing is that someone (who might be a virtualbox developer)
> used the same race argument to say that this is a bug in the kernel.
> 
>  What concerns me though is that how can virtualbox be worth using
> in the Linux community if it's probably not working for various distros
> (currently Fedora, Ubuntu, Mandriva and ArchLinux).
> 
>  Thanks for the effort, guys.
> 
> [1] http://www.virtualbox.org/ticket/1875

Ok, some news from archlinux side:
Our distribution kernel was upgraded from 2.6.26.2 to 2.6.26.3. With
this upgrade to patchlevel .3 the "early oops"(freeing smp...) has gone.
My virtual machines boots always fine with this, and i have one
confirmation from a user about this.

Kernel upgrade does not solve the kernel panic during work with the VM,
when there is heavy disk IO. I test and could reproduce this by untar 2
big files in seperate dirs: bsdtar -x -f VirtualBox-1.6.2-OSE.tar.bz2.
Doing this simultan crashed the VM always.
SreenShot:
http://users.archlinux.de/~gerbra/tmp/2008-08-31-110449_724x456_scrot.png

This heavy IO oops does not occur under 2.6.26.2 when using the
"3-changes-patch" against alternatives.c, which we have tested in the
other mails. There must be something irq related which fix this
3-changes-patch, and what was not fixed in 2.6.26.3
On the other hand: I never have stressed a VM like this before
researching for this problem. So it could also be that the heavy-IO
problem way a total seperate problem from that we're talking about here.
Doing my "normal" work now in VM (it's my devel VM for compiling and
testing), until now i don't have had this IO oops.

We use a mostly unpatched kernel as distribution kernel.

So short summary from my side:
a) With "3-changes-patch" i got a rock solide VM
b) 2.6.26.2 have the early oops on boot and IO oops when sometimes
   bootet.
c) 2.6.26.3 have only the heavy-IO oops

I'll try a fresh VM, where i will test:
a) Using sata controller emulation as bus (now i have ide(piix3))
b) Using different filesystems (With 2.6.26.2 early oops and heavy-io
   oops could be reproduced with any filesystem).

Regards
	Gerhard

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-31  9:29                                         ` Gerhard Brauer
@ 2008-08-31 13:28                                           ` Stefan Lippers-Hollmann
  2008-08-31 14:03                                             ` Gerhard Brauer
  2008-08-31 14:09                                           ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: Stefan Lippers-Hollmann @ 2008-08-31 13:28 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Luiz Fernando N. Capitulino, Mathieu Desnoyers, H. Peter Anvin,
	Ingo Molnar, linux-kernel

Hi

On Sonntag, 31. August 2008, Gerhard Brauer wrote:
[...]
> Ok, some news from archlinux side:
> Our distribution kernel was upgraded from 2.6.26.2 to 2.6.26.3. With
> this upgrade to patchlevel .3 the "early oops"(freeing smp...) has gone.
> My virtual machines boots always fine with this, and i have one
> confirmation from a user about this.

Sorry, I can't confirm this here on Debian unstable (with virtualbox-ose 
1.6.2 or 1.6.4), are you sure that other configuration options didn't 
change between the different kernel versions? Preemption and paravirt can
influence the probability of the early boot panic seriously, without really
avoiding it alltogether.

Actually I still get the same issues with implanting
ftp://ftp5.gwdg.de/pub/linux/archlinux/core/os/i686/kernel26-2.6.26.3-1-i686.pkg.tar.gz
into the test vm using virtualbox-ose 1.6.4.

> Kernel upgrade does not solve the kernel panic during work with the VM,
> when there is heavy disk IO. I test and could reproduce this by untar 2
> big files in seperate dirs: bsdtar -x -f VirtualBox-1.6.2-OSE.tar.bz2.
> Doing this simultan crashed the VM always.
> SreenShot:
> http://users.archlinux.de/~gerbra/tmp/2008-08-31-110449_724x456_scrot.png
> 
> This heavy IO oops does not occur under 2.6.26.2 when using the
> "3-changes-patch" against alternatives.c, which we have tested in the
> other mails. There must be something irq related which fix this
> 3-changes-patch, and what was not fixed in 2.6.26.3
> On the other hand: I never have stressed a VM like this before
> researching for this problem. So it could also be that the heavy-IO
> problem way a total seperate problem from that we're talking about here.
> Doing my "normal" work now in VM (it's my devel VM for compiling and
> testing), until now i don't have had this IO oops.
> 
> We use a mostly unpatched kernel as distribution kernel.
> 
> So short summary from my side:
> a) With "3-changes-patch" i got a rock solide VM
> b) 2.6.26.2 have the early oops on boot and IO oops when sometimes
>    bootet.
> c) 2.6.26.3 have only the heavy-IO oops
> 
> I'll try a fresh VM, where i will test:
> a) Using sata controller emulation as bus (now i have ide(piix3))
> b) Using different filesystems (With 2.6.26.2 early oops and heavy-io
>    oops could be reproduced with any filesystem).
> 
> 
> Regards
> 	Gerhard

Regards
	Stefan Lippers-Hollmann

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-31 13:28                                           ` Stefan Lippers-Hollmann
@ 2008-08-31 14:03                                             ` Gerhard Brauer
  0 siblings, 0 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-31 14:03 UTC (permalink / raw)
  To: Stefan Lippers-Hollmann
  Cc: Luiz Fernando N. Capitulino, Mathieu Desnoyers, H. Peter Anvin,
	Ingo Molnar, linux-kernel

On Sun, Aug 31, 2008 at 03:28:22PM +0200, Stefan Lippers-Hollmann wrote:
> Hi
> 
> On Sonntag, 31. August 2008, Gerhard Brauer wrote:
> [...]
> > Ok, some news from archlinux side:
> > Our distribution kernel was upgraded from 2.6.26.2 to 2.6.26.3. With
> > this upgrade to patchlevel .3 the "early oops"(freeing smp...) has gone.
> > My virtual machines boots always fine with this, and i have one
> > confirmation from a user about this.
> 
> Sorry, I can't confirm this here on Debian unstable (with virtualbox-ose 
> 1.6.2 or 1.6.4), are you sure that other configuration options didn't 
> change between the different kernel versions? Preemption and paravirt can
> influence the probability of the early boot panic seriously, without really
> avoiding it alltogether.

Only changes between our 2.6.26.2-1 and 2.6.26.3-1 are some minor
framebuffer changes in config. If i have a look at the different
patchsets between the two versions i don't see something which could be
the reason between work and not work.

> Actually I still get the same issues with implanting
> ftp://ftp5.gwdg.de/pub/linux/archlinux/core/os/i686/kernel26-2.6.26.3-1-i686.pkg.tar.gz
> into the test vm using virtualbox-ose 1.6.4.

Hmm, one user also reports that he have no problem when using a vanilla
2.6.26 as guest kernel. But there must be some reasons when different
distributions notice a major problem between 2.6.25 and 2.6.26 with
their stock kernels. Although i don't even know if our few reports here
are very representavive...

> Regards
> 	Stefan Lippers-Hollmann

Gerhard

-- 
Heute ist das Morgen wovor du gestern Angst hattest...

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-31  9:29                                         ` Gerhard Brauer
  2008-08-31 13:28                                           ` Stefan Lippers-Hollmann
@ 2008-08-31 14:09                                           ` Luiz Fernando N. Capitulino
  2008-09-21 13:41                                             ` Gerhard Brauer
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-31 14:09 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Mathieu Desnoyers, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Sun, 31 Aug 2008 11:29:23 +0200
Gerhard Brauer <gerhard.brauer@web.de> escreveu:

| On Thu, Aug 28, 2008 at 10:30:13AM -0300, Luiz Fernando N. Capitulino wrote:
| > Em Wed, 27 Aug 2008 19:33:28 -0400
| > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> escreveu:
| > | 
| > | Since this problem appears while we are using a simple memcpy (the
| > | text_poke_early version), but disappears when we disable interrupts for
| > | a longer period of this, I suspect a problem with irq disabling in
| > | Virtualbox.
| > | 
| > | We could try to add some nsleep() or msleep() calls within text_poke and
| > | text_poke_early before and after the code modificatoin to see if the
| > | problem disappears. If it does, then that would somewhat confirm the
| > | racy irq disable thesis.
| > 
| >  Well, a Ubuntu kernel guy has reported in the virtualbox's ticket[1]
| > that the oops doesn't happen if he puts a printk() in the crash site.
| > 
| >  The funny thing is that someone (who might be a virtualbox developer)
| > used the same race argument to say that this is a bug in the kernel.
| > 
| >  What concerns me though is that how can virtualbox be worth using
| > in the Linux community if it's probably not working for various distros
| > (currently Fedora, Ubuntu, Mandriva and ArchLinux).
| > 
| >  Thanks for the effort, guys.
| > 
| > [1] http://www.virtualbox.org/ticket/1875
| 
| Ok, some news from archlinux side:
| Our distribution kernel was upgraded from 2.6.26.2 to 2.6.26.3. With
| this upgrade to patchlevel .3 the "early oops"(freeing smp...) has gone.
| My virtual machines boots always fine with this, and i have one
| confirmation from a user about this.
| 
| Kernel upgrade does not solve the kernel panic during work with the VM,
| when there is heavy disk IO. I test and could reproduce this by untar 2
| big files in seperate dirs: bsdtar -x -f VirtualBox-1.6.2-OSE.tar.bz2.
| Doing this simultan crashed the VM always.
| SreenShot:
| http://users.archlinux.de/~gerbra/tmp/2008-08-31-110449_724x456_scrot.png
| 
| This heavy IO oops does not occur under 2.6.26.2 when using the
| "3-changes-patch" against alternatives.c, which we have tested in the
| other mails. There must be something irq related which fix this
| 3-changes-patch, and what was not fixed in 2.6.26.3
| On the other hand: I never have stressed a VM like this before
| researching for this problem. So it could also be that the heavy-IO
| problem way a total seperate problem from that we're talking about here.
| Doing my "normal" work now in VM (it's my devel VM for compiling and
| testing), until now i don't have had this IO oops.

 Mandriva kernel was 2.6.26.3 based at the time I started testing
this and all my last tests have been done on 2.6.27-rc4. I think it's
very unusual to have a change in a -stable kernel not present in the
latest -rc.

 Also note that CPU settings in the VM has a big influency in the
problem, so I'm pretty sure 2.6.26.3 doesn't fix the problem.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-31 14:09                                           ` Luiz Fernando N. Capitulino
@ 2008-09-21 13:41                                             ` Gerhard Brauer
  2008-09-22  9:51                                               ` Ingo Molnar
  2008-09-24 13:24                                               ` Luiz Fernando N. Capitulino
  0 siblings, 2 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-09-21 13:41 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: Mathieu Desnoyers, H. Peter Anvin, Ingo Molnar, linux-kernel

Am Sonntag, den 31.08.2008, 11:09 -0300 schrieb Luiz Fernando N.
Capitulino:

>  Mandriva kernel was 2.6.26.3 based at the time I started testing
> this and all my last tests have been done on 2.6.27-rc4. I think it's
> very unusual to have a change in a -stable kernel not present in the
> latest -rc.

I was away the last days, but i notice that the virtualbox update to
2.0.2 solve all problems i mentioned. We also have same responses from
other arch users.
I never saw the "early boot oops" (this have had gone still with our
kernel update, mystical....). But also the "heavy IO oops" has gone. So
it was a virtualbox problem, they fixed it in:
http://www.virtualbox.org/ticket/1875

So from my side this is solved.

Gerhard


^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-09-21 13:41                                             ` Gerhard Brauer
@ 2008-09-22  9:51                                               ` Ingo Molnar
  2008-09-24 13:24                                               ` Luiz Fernando N. Capitulino
  1 sibling, 0 replies; 52+ messages in thread
From: Ingo Molnar @ 2008-09-22  9:51 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Luiz Fernando N. Capitulino, Mathieu Desnoyers, H. Peter Anvin,
	linux-kernel


* Gerhard Brauer <gerhard.brauer@web.de> wrote:

> Am Sonntag, den 31.08.2008, 11:09 -0300 schrieb Luiz Fernando N.
> Capitulino:
> 
> >  Mandriva kernel was 2.6.26.3 based at the time I started testing
> > this and all my last tests have been done on 2.6.27-rc4. I think it's
> > very unusual to have a change in a -stable kernel not present in the
> > latest -rc.
> 
> I was away the last days, but i notice that the virtualbox update to
> 2.0.2 solve all problems i mentioned. We also have same responses from
> other arch users.
> I never saw the "early boot oops" (this have had gone still with our
> kernel update, mystical....). But also the "heavy IO oops" has gone. So
> it was a virtualbox problem, they fixed it in:
> http://www.virtualbox.org/ticket/1875
> 
> So from my side this is solved.

great - the VirtualBox recompiler didnt notice the paravirt code 
modification sequence. Any Linux kernel side NOP issue (which caused 
that early oops) should be solved in 2.6.27-rc7 as well.

so the combo of 2.0.2 and later VirtualBox plus v2.6.27-rc7 and later 
should have no known bugs.

	Ingo

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-09-21 13:41                                             ` Gerhard Brauer
  2008-09-22  9:51                                               ` Ingo Molnar
@ 2008-09-24 13:24                                               ` Luiz Fernando N. Capitulino
  1 sibling, 0 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-09-24 13:24 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: Mathieu Desnoyers, H. Peter Anvin, Ingo Molnar, linux-kernel

Em Sun, 21 Sep 2008 15:41:39 +0200
Gerhard Brauer <gerhard.brauer@web.de> escreveu:

| Am Sonntag, den 31.08.2008, 11:09 -0300 schrieb Luiz Fernando N.
| Capitulino:
| 
| >  Mandriva kernel was 2.6.26.3 based at the time I started testing
| > this and all my last tests have been done on 2.6.27-rc4. I think it's
| > very unusual to have a change in a -stable kernel not present in the
| > latest -rc.
| 
| I was away the last days, but i notice that the virtualbox update to
| 2.0.2 solve all problems i mentioned. We also have same responses from
| other arch users.
| I never saw the "early boot oops" (this have had gone still with our
| kernel update, mystical....). But also the "heavy IO oops" has gone. So
| it was a virtualbox problem, they fixed it in:
| http://www.virtualbox.org/ticket/1875
| 
| So from my side this is solved.

 Yeah, we have ran some tests here as well and it is solved.

 Thanks a lot for the people involved in debugging this problem.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-27 23:33                                     ` Mathieu Desnoyers
  2008-08-28 13:30                                       ` Luiz Fernando N. Capitulino
@ 2008-08-28 13:50                                       ` Gerhard Brauer
  1 sibling, 0 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-28 13:50 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: Luiz Fernando N. Capitulino, H. Peter Anvin, Ingo Molnar,
	linux-kernel

On Wed, Aug 27, 2008 at 07:33:28PM -0400, Mathieu Desnoyers wrote:
> 
> We could try to add some nsleep() or msleep() calls within text_poke and
> text_poke_early before and after the code modificatoin to see if the
> problem disappears. If it does, then that would somewhat confirm the
> racy irq disable thesis.

nsleep isn't known here as a function, only references i found is maybe
in posix-timers.c.

msleep() is known, but each time i add for ex.
msleep(100);
in any place in text_poke and/or text_poke_early it get a kernel panic
on boot. Here's a screenie:
http://users.archlinux.de/~gerbra/tmp/2008-08-28-132337_724x456_scrot.png

I also tried to work with the isolated changes we have last made, but it
seems that only the 3 changes together work.
Also i tried to went back to older versions of alternatives.c referenced
in:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.26.y.git;a=history;f=arch/x86/kernel/alternative.c;h=65c7857a90ddfc6ff084c6817baba045ced0ad71;hb=v2.6.26

But with my few knowledges i ran in too many errors.

So, have you any further ideas, code that i/we could test?
Or - i'm naive - are the "3 changes" we made ready to go in the kernel
without to harm something real important than virtualbox?

> Mathieu

Gerhard

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 14:53                       ` Mathieu Desnoyers
  2008-08-26 16:09                         ` Luiz Fernando N. Capitulino
  2008-08-26 16:13                         ` Luiz Fernando N. Capitulino
@ 2008-08-26 19:27                         ` Gerhard Brauer
  2 siblings, 0 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-26 19:27 UTC (permalink / raw)
  To: Mathieu Desnoyers
  Cc: H. Peter Anvin, Luiz Fernando N. Capitulino, Ingo Molnar,
	linux-kernel

On Tue, Aug 26, 2008 at 10:53:38AM -0400, Mathieu Desnoyers wrote:
> * Gerhard Brauer (gerhard.brauer@web.de) wrote:
> > 
> > Here is also an archive with guest dmesg and messages.log from such an
> > oops when heavy disk io leads to the oops:
> > http://bugs.archlinux.org/task/11141?getfile=2445
> > 
> 
> Hrm, can you try this ?

Sorry for the delay but i need to build a complete distribution kernel
and my machine is not the fastest.

My host:
archlinux 2.6.26 P4 2Ghz
VirtualBox: Sun xVM 1.6.4
gcc 4.4.1-3

My guest:
archlinux 2.6.26

My "tests":
I could sometimes boot the guest with the "tricks" (VT-x enabled, acpi
off,...). But i always get an oops if i compile something bigger on this
guest (ex. virtualbox-modules where the tarball must be untarrt with
bsdtar -> disk io)
If this happens the next reboot leads always to the early oops (Freeing
smp....). Each reboot do this. Then i close virtualbox application,
unload/reload vboxdrv from host and start vbox again. Then i could
mostimes boot the guest again. But next heavy disk IO leads again to the
oops.
If i could boot without oops, and reboot or halt the guest, then the
next boots are clean.

> 1 - Make sure you kernel is not CONFIG_DEBUG_RODATA
Not set.

> 2 - Change the whole text_poke implementation in
> arch/x86/kernel/alternative.c to this :
With this changes i also get the oops, in all above mentioned tests.

> Then, after having tested (2), try this on top of it :
> 
> In arch/x86/kernel/alternative.c, alternatives_smp_switch()
> 
> Add   unsigned long flags;
> Change 
> spin_lock -> spin_lock_irqsave(&smp_alt, flags);
> spin_unlock(&smp_alt); -> spin_unlock_irqrestore(&smp_alt, flags);

With our distribution kernel i could change these spin_lock/unlock in
alternatives.c. Fist thought was that there was a slightly better
behavior (first boot goes on, i could compile something, but next
package i build thee opps (heavy io opps) comes again. And then also
after reboot the early oops (freeing smp...)
Here is a screenie from oops when building something:
http://users.archlinux.de/~gerbra/tmp/2008-08-26-210724_724x456_scrot.png

Sometimes (could not be reproduced) the virtualbox app also traps with
an error dialog (Guru message), which offers a log from the VM and a
scren shot. Maybe this could be helpfull. Log and screenie could be
found here:
http://users.archlinux.de/~gerbra/tmp/vbox-guru/

> 
> This will help testing if there is a problem with interrupts coming
> shortly after the modification. If it fixes the problem, my guess is
> that we should flush the instruction cache (and maybe the data cache ?)
> in text_poke and text_poke early when interrupts are off.

>From my side i would say: both changes would not solve the oops.

> Mathieu

Regards
	Gerhard

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 14:18                     ` Gerhard Brauer
  2008-08-26 14:53                       ` Mathieu Desnoyers
@ 2008-08-26 16:02                       ` Luiz Fernando N. Capitulino
  2008-08-26 16:40                         ` Gerhard Brauer
  1 sibling, 1 reply; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-26 16:02 UTC (permalink / raw)
  To: Gerhard Brauer
  Cc: H. Peter Anvin, Mathieu Desnoyers, Ingo Molnar, linux-kernel

Em Tue, 26 Aug 2008 16:18:51 +0200
Gerhard Brauer <gerhard.brauer@web.de> escreveu:

| On Fri, Aug 22, 2008 at 02:08:13PM -0700, H. Peter Anvin wrote:
| > Luiz Fernando N. Capitulino wrote:
| >>
| >>  I have asked Mandriva and Ubuntu users to test this and all of
| >> them so far are saying that noreplace-paravirt works.
| >>
| >>  It makes the system slower, but it works.
| >>
| >
| > Yes, the big issue is exactly what VirtualBox screws up in this matter,  
| > how to detect it, and how to work around it.
| >
| > It's pretty clear it's a VirtualBox f*ckup at this point, but the failure 
| > mechanism isn't at all obvious and so far the workaround is elusive.
| >
| > I'm strongly suspect this is a VirtualBox tcache management failure, but  
| > that doesn't help the situation without knowing how it happens.
| 
| On Archlinux we have the same problem. We have a bugreport here:
| http://bugs.archlinux.org/task/11141
| 
| Myself test it with a LiveCD/Install-ISO which has 2.6.26 as install
| kernel. We have the guest oops on virtualbox-ose, virtualbox-sun and both on
| i686 or x86_64 hosts.
| 
| Some things i noticed:
| - The system boots always when i either enable VT-x in guest settings or
|   disable acpi and run the guest with acpi=off.

 Yes, lots of ubuntu users have reported the same but another "lots"
of them have reported that the trick didn't work.

 Thanks for joining!

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-26 16:02                       ` Luiz Fernando N. Capitulino
@ 2008-08-26 16:40                         ` Gerhard Brauer
  0 siblings, 0 replies; 52+ messages in thread
From: Gerhard Brauer @ 2008-08-26 16:40 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: H. Peter Anvin, Mathieu Desnoyers, Ingo Molnar, linux-kernel

On Tue, Aug 26, 2008 at 01:02:16PM -0300, Luiz Fernando N. Capitulino wrote:
> Em Tue, 26 Aug 2008 16:18:51 +0200
> Gerhard Brauer <gerhard.brauer@web.de> escreveu:
> 
> | Some things i noticed:
> | - The system boots always when i either enable VT-x in guest settings or
> |   disable acpi and run the guest with acpi=off.
> 
>  Yes, lots of ubuntu users have reported the same but another "lots"
> of them have reported that the trick didn't work.

I must relativate above: i have two test enviroments, one is our
LiveCD/Install-ISO with 2.6.26 which we made special for a linux
conference last weekend (our official iso comes still with 2.6.25). With
this iso the "trick" with VT-x or noacpi works.
But on an installed archlinux (with distribution kernel 2.6.26) this
does'nt work. Sometimes it works when i restart the virtualbox
application, but mostly not. So on this installed guest system the only
working solution seems to add noreplace-paravirt as kernel parameter.
But this makes the system terrible slow (mostly on udev things).

I try Mathieu's hints currently by building a new distribution kernel
with the changes.
But i think the biggest problem to maybe solve this from the sight of
kernel devs is that we all have different "test" enviroments (vbox
versions, architectures, distribution kernels,...) where the oops (i
think) not appears for all on the same place.
On the other hand, the more we we try such patches in different
enviroments there is a better chance to get a real fix - from kernel dev
side or from virtualbox/Sun side...

>  Thanks for joining!
> Luiz Fernando N. Capitulino

Gerhard

-- 
www,archlinux.de

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 16:29           ` Luiz Fernando N. Capitulino
  2008-08-22 16:35             ` Mathieu Desnoyers
@ 2008-08-22 17:16             ` H. Peter Anvin
  2008-08-22 17:45               ` Mathieu Desnoyers
  2008-08-22 19:10               ` Luiz Fernando N. Capitulino
  1 sibling, 2 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 17:16 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Was looking at the code stream, and noticed this:

Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff 
e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 
8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8

Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff 
e8 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 
8b 45 bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8

The EIP is in the *MIDDLE* of a NOPL instruction:

C012FC46  C00F84            ror byte [edi],0x84
C012FC49  0B01              or eax,[ecx]
C012FC4B  0000              add [eax],al
C012FC4D  B8D0BF41C0        mov eax,0xc041bfd0
C012FC52  C7056CC041C0FFFF  mov dword [dword 0xc041c06c],0xffffffff
          -FFFF
C012FC5C  E87F822100        call dword 0xc0347ee0
C012FC61  E81A030200        call dword 0xc014ff80
C012FC66  8B45B0            mov eax,[ebp-0x50]
C012FC69  50                push eax
C012FC6A  9D                popfd
C012FC6B  0F1F840000000000  nop dword [eax+eax+0x0]
C012FC73  8B45BC            mov eax,[ebp-0x44]
C012FC76  83C460            add esp,byte +0x60
C012FC79  5B                pop ebx
C012FC7A  5E                pop esi
C012FC7B  5F                pop edi
C012FC7C  5D                pop ebp
C012FC7D  C3                ret
C012FC7E  6690              xchg ax,ax
C012FC80  A16CC041C0        mov eax,[0xc041c06c]

There are two possibilities: VirtualBox mis-executes (not merely traps, 
which is what tip:master looks for) the NOPL instruction, or something 
is jumping into the middle of the sequence that is then replaced by the 
NOPL.

So, Luiz: the DEBUG_INFO version of vmlinux would be helpful.  It would 
also help to know the exact version of VirtualBox you're running, what 
source you got it from, and what your host system looks like.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 17:16             ` H. Peter Anvin
@ 2008-08-22 17:45               ` Mathieu Desnoyers
  2008-08-22 17:57                 ` H. Peter Anvin
  2008-08-22 19:10               ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: Mathieu Desnoyers @ 2008-08-22 17:45 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Luiz Fernando N. Capitulino, Ingo Molnar, linux-kernel

* H. Peter Anvin (hpa@zytor.com) wrote:
> Was looking at the code stream, and noticed this:
>
> Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 
> 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 
> bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8
>
> Code: c0 0f 84 0b 01 00 00 b8 d0 bf 41 c0 c7 05 6c c0 41 c0 ff ff ff ff e8 
> 7f 82 21 00 e8 1a 03 02 00 8b 45 b0 50 9d 0f 1f 84 00 00 00 <00> 00 8b 45 
> bc 83 c4 60 5b 5e 5f 5d c3 66 90 a1 6c c0 41 c0 e8
>
> The EIP is in the *MIDDLE* of a NOPL instruction:
>
> C012FC46  C00F84            ror byte [edi],0x84
> C012FC49  0B01              or eax,[ecx]
> C012FC4B  0000              add [eax],al
> C012FC4D  B8D0BF41C0        mov eax,0xc041bfd0
> C012FC52  C7056CC041C0FFFF  mov dword [dword 0xc041c06c],0xffffffff
>          -FFFF
> C012FC5C  E87F822100        call dword 0xc0347ee0
> C012FC61  E81A030200        call dword 0xc014ff80
> C012FC66  8B45B0            mov eax,[ebp-0x50]
> C012FC69  50                push eax
> C012FC6A  9D                popfd
> C012FC6B  0F1F840000000000  nop dword [eax+eax+0x0]
> C012FC73  8B45BC            mov eax,[ebp-0x44]
> C012FC76  83C460            add esp,byte +0x60
> C012FC79  5B                pop ebx
> C012FC7A  5E                pop esi
> C012FC7B  5F                pop edi
> C012FC7C  5D                pop ebp
> C012FC7D  C3                ret
> C012FC7E  6690              xchg ax,ax
> C012FC80  A16CC041C0        mov eax,[0xc041c06c]
>
> There are two possibilities: VirtualBox mis-executes (not merely traps, 
> which is what tip:master looks for) the NOPL instruction, or something is 
> jumping into the middle of the sequence that is then replaced by the NOPL.
>
> So, Luiz: the DEBUG_INFO version of vmlinux would be helpful.  It would 
> also help to know the exact version of VirtualBox you're running, what 
> source you got it from, and what your host system looks like.
>
> 	-hpa

The patch which turns on this bug this this important change to the
apply paravirt : it disables interrupts _near_ the code patching,
_within_ the loop. Before, interrupts were disabled outside of the loop.
It needs to disable interrupts within the loop to be able to use vmap in
text_poke().

So I bet VirtualBox has a race in the way it handles interrupt
disabling.

Mathieu

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 17:45               ` Mathieu Desnoyers
@ 2008-08-22 17:57                 ` H. Peter Anvin
  0 siblings, 0 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 17:57 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: Luiz Fernando N. Capitulino, Ingo Molnar, linux-kernel

Mathieu Desnoyers wrote:
> 
> The patch which turns on this bug this this important change to the
> apply paravirt : it disables interrupts _near_ the code patching,
> _within_ the loop. Before, interrupts were disabled outside of the loop.
> It needs to disable interrupts within the loop to be able to use vmap in
> text_poke().
> 
> So I bet VirtualBox has a race in the way it handles interrupt
> disabling.
> 

That seems a bit far-fetched.  The fault is in an initcall, and there 
are no interrupts involved.  Perhaps VirtualBox doesn't manage its 
tcache correctly, but I don't see this as being interrupt-related.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 17:16             ` H. Peter Anvin
  2008-08-22 17:45               ` Mathieu Desnoyers
@ 2008-08-22 19:10               ` Luiz Fernando N. Capitulino
  2008-08-22 19:14                 ` H. Peter Anvin
  2008-08-22 19:18                 ` H. Peter Anvin
  1 sibling, 2 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 19:10 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Em Fri, 22 Aug 2008 10:16:07 -0700
"H. Peter Anvin" <hpa@zytor.com> escreveu:

| So, Luiz: the DEBUG_INFO version of vmlinux would be helpful.  It would 
| also help to know the exact version of VirtualBox you're running, what 
| source you got it from, and what your host system looks like.

 You will find vmlinux with DEBUG_INFO enabled at:

http://users.mandriva.com.br/~lcapitulino/virtualbox-oops/

 I'm running Mandriva's VirtualBox 1.6.4 OSE, my host kernel is 2.6.26-3mnb
(patched).

 I could try with upstream's VirtualBox just to be sure it's not
something else, but I don't think it is since there are reports for
ArchLinux and Ubuntu as well:

https://bugs.launchpad.net/ubuntu/intrepid/+source/linux/+bug/246067

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 19:10               ` Luiz Fernando N. Capitulino
@ 2008-08-22 19:14                 ` H. Peter Anvin
  2008-08-22 19:18                 ` H. Peter Anvin
  1 sibling, 0 replies; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 19:14 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Luiz Fernando N. Capitulino wrote:
> Em Fri, 22 Aug 2008 10:16:07 -0700
> "H. Peter Anvin" <hpa@zytor.com> escreveu:
> 
> | So, Luiz: the DEBUG_INFO version of vmlinux would be helpful.  It would 
> | also help to know the exact version of VirtualBox you're running, what 
> | source you got it from, and what your host system looks like.
> 
>  You will find vmlinux with DEBUG_INFO enabled at:
> 
> http://users.mandriva.com.br/~lcapitulino/virtualbox-oops/
> 
>  I'm running Mandriva's VirtualBox 1.6.4 OSE, my host kernel is 2.6.26-3mnb
> (patched).
> 
>  I could try with upstream's VirtualBox just to be sure it's not
> something else, but I don't think it is since there are reports for
> ArchLinux and Ubuntu as well:
> 
> https://bugs.launchpad.net/ubuntu/intrepid/+source/linux/+bug/246067
> 

Not necessary, but I wanted to get the information so I can try to 
reproduce locally.

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 19:10               ` Luiz Fernando N. Capitulino
  2008-08-22 19:14                 ` H. Peter Anvin
@ 2008-08-22 19:18                 ` H. Peter Anvin
  2008-08-22 19:42                   ` Luiz Fernando N. Capitulino
  1 sibling, 1 reply; 52+ messages in thread
From: H. Peter Anvin @ 2008-08-22 19:18 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Luiz Fernando N. Capitulino wrote:
> 
>  I'm running Mandriva's VirtualBox 1.6.4 OSE, my host kernel is 2.6.26-3mnb
> (patched).
> 

What is your host *system* like -- CPU especially, and is your host 
kernel 32 or 64 bits?

	-hpa

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-22 19:18                 ` H. Peter Anvin
@ 2008-08-22 19:42                   ` Luiz Fernando N. Capitulino
  0 siblings, 0 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 19:42 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Mathieu Desnoyers, Ingo Molnar, linux-kernel

Em Fri, 22 Aug 2008 12:18:21 -0700
"H. Peter Anvin" <hpa@zytor.com> escreveu:

| Luiz Fernando N. Capitulino wrote:
| > 
| >  I'm running Mandriva's VirtualBox 1.6.4 OSE, my host kernel is 2.6.26-3mnb
| > (patched).
| > 
| 
| What is your host *system* like -- CPU especially, and is your host 
| kernel 32 or 64 bits?

 32 bits, /proc/cpuinfo output:

"""
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	: Intel(R) Pentium(R) 4 CPU 2.40GHz
stepping	: 1
cpu MHz		: 2410.462
cache size	: 1024 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe constant_tsc pebs bts pni monitor ds_cpl cid xtpr
bogomips	: 4825.33
clflush size	: 64
power management:
"""

 I have 1G of RAM and a VIA mobo.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 2.6.{26.2,27-rc} oops on virtualbox
  2008-08-21 21:34 ` H. Peter Anvin
  2008-08-22  6:42   ` H. Peter Anvin
@ 2008-08-22 14:28   ` Luiz Fernando N. Capitulino
  1 sibling, 0 replies; 52+ messages in thread
From: Luiz Fernando N. Capitulino @ 2008-08-22 14:28 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: linux-kernel, mathieu.desnoyers, mingo

Em Thu, 21 Aug 2008 14:34:07 -0700
"H. Peter Anvin" <hpa@zytor.com> escreveu:

| > 
| >  Does this look like a kernel bug?
| > 
| 
| No, it looks like a very common virtualizer bug.  Does the attached 
| patch work for you?

 Unfortunately it does not.

-- 
Luiz Fernando N. Capitulino

^ permalink raw reply	[flat|nested] 52+ messages in thread

end of thread, other threads:[~2008-09-24 13:24 UTC | newest]

Thread overview: 52+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-08-20 19:29 2.6.{26.2,27-rc} oops on virtualbox Luiz Fernando N. Capitulino
2008-08-21 21:34 ` H. Peter Anvin
2008-08-22  6:42   ` H. Peter Anvin
2008-08-22  6:50     ` Ingo Molnar
2008-08-22 14:39       ` Luiz Fernando N. Capitulino
2008-08-22 15:34         ` Mathieu Desnoyers
2008-08-22 16:29           ` Luiz Fernando N. Capitulino
2008-08-22 16:35             ` Mathieu Desnoyers
2008-08-22 17:20               ` Luiz Fernando N. Capitulino
2008-08-22 18:11                 ` H. Peter Anvin
2008-08-22 19:40                   ` Luiz Fernando N. Capitulino
2008-08-22 20:31                     ` H. Peter Anvin
2008-08-22 20:55                       ` Luiz Fernando N. Capitulino
2008-08-22 20:57                 ` Luiz Fernando N. Capitulino
2008-08-22 21:08                   ` H. Peter Anvin
2008-08-26 14:18                     ` Gerhard Brauer
2008-08-26 14:53                       ` Mathieu Desnoyers
2008-08-26 16:09                         ` Luiz Fernando N. Capitulino
2008-08-26 16:13                         ` Luiz Fernando N. Capitulino
2008-08-26 17:18                           ` Mathieu Desnoyers
2008-08-26 17:32                             ` H. Peter Anvin
2008-08-26 18:02                             ` Luiz Fernando N. Capitulino
2008-08-26 18:15                               ` Mathieu Desnoyers
2008-08-26 19:52                                 ` H. Peter Anvin
2008-08-26 20:34                                 ` Gerhard Brauer
2008-08-26 20:48                                   ` Mathieu Desnoyers
2008-08-26 21:25                                     ` Gerhard Brauer
2008-08-26 21:35                                       ` Mathieu Desnoyers
2008-08-26 21:51                                         ` H. Peter Anvin
2008-08-27  0:13                                         ` Gerhard Brauer
2008-08-27 19:13                                   ` Luiz Fernando N. Capitulino
2008-08-27 23:33                                     ` Mathieu Desnoyers
2008-08-28 13:30                                       ` Luiz Fernando N. Capitulino
2008-08-31  9:29                                         ` Gerhard Brauer
2008-08-31 13:28                                           ` Stefan Lippers-Hollmann
2008-08-31 14:03                                             ` Gerhard Brauer
2008-08-31 14:09                                           ` Luiz Fernando N. Capitulino
2008-09-21 13:41                                             ` Gerhard Brauer
2008-09-22  9:51                                               ` Ingo Molnar
2008-09-24 13:24                                               ` Luiz Fernando N. Capitulino
2008-08-28 13:50                                       ` Gerhard Brauer
2008-08-26 19:27                         ` Gerhard Brauer
2008-08-26 16:02                       ` Luiz Fernando N. Capitulino
2008-08-26 16:40                         ` Gerhard Brauer
2008-08-22 17:16             ` H. Peter Anvin
2008-08-22 17:45               ` Mathieu Desnoyers
2008-08-22 17:57                 ` H. Peter Anvin
2008-08-22 19:10               ` Luiz Fernando N. Capitulino
2008-08-22 19:14                 ` H. Peter Anvin
2008-08-22 19:18                 ` H. Peter Anvin
2008-08-22 19:42                   ` Luiz Fernando N. Capitulino
2008-08-22 14:28   ` Luiz Fernando N. Capitulino

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox