FYI, we noticed the below changes on

git://bee.sh.intel.com/git/ydu19/linux rewrite-v1.1-on-3.16-rc3
commit 69b181fe34a030a624fc05e7c06f576d20a5b2ce ("sched: Remove update_rq_runnable_avg")

+------------------------------------------------------+-----------+------------+
|                                                      | v3.16-rc3 | 69b181fe34 |
+------------------------------------------------------+-----------+------------+
| boot_successes                                       | 3430      | 20         |
| boot_failures                                        | 366       | 6          |
| BUG:kernel_boot_hang                                 | 113       | 2          |
| BUG:kernel_early_hang_without_any_printk_output      | 93        |            |
| BUG:kernel_test_hang                                 | 9         |            |
| Out_of_memory:Kill_process                           | 15        |            |
| backtrace:vfs_write                                  | 1         | 4          |
| backtrace:SyS_write                                  | 1         | 4          |
| page_allocation_failure:order:,mode                  | 3         |            |
| backtrace:do_fork                                    | 7         |            |
| backtrace:SyS_clone                                  | 7         |            |
| backtrace:do_execve_common                           | 1         |            |
| backtrace:SyS_execve                                 | 1         |            |
| backtrace:vm_mmap_pgoff                              | 1         |            |
| backtrace:SyS_mmap_pgoff                             | 1         |            |
| backtrace:SyS_mmap                                   | 1         |            |
| BUG:kernel_test_crashed                              | 3         |            |
| BUG:kernel_boot_crashed                              | 133       |            |
| BUG:unable_to_handle_kernel_NULL_pointer_dereference | 0         | 4          |
| Oops                                                 | 0         | 4          |
| RIP:print_cfs_group_stats                            | 0         | 4          |
| kernel_BUG_at_arch/x86/mm/pageattr.c                 | 0         | 4          |
| invalid_opcode                                       | 0         | 4          |
| RIP:change_page_attr_set_clr                         | 0         | 4          |
| Kernel_panic-not_syncing:Fatal_exception             | 0         | 4          |
+------------------------------------------------------+-----------+------------+


[18195.617056]   .tg->cfs_bandwidth.timer_active: 0
[18195.617056]   .throttled                     : 0
[18195.617056]   .throttle_count                : 0
[18195.617056] BUG: unable to handle kernel NULL pointer dereference at 0000000000000040
[18195.617056] IP: [<ffffffff810b706b>] print_cfs_group_stats.isra.0+0x6db/0xad0
[18195.617056] PGD 7f464067 PUD 7f0ef067 PMD 0 
[18195.617056] Oops: 0000 [#1] SMP 
[18195.617056] Modules linked in: ipmi_watchdog ipmi_msghandler dm_mod fuse sg sr_mod cdrom ata_generic pata_acpi parport_pc parport floppy snd_pcm snd_timer snd soundcore ata_piix pcspkr cirrus libata syscopyarea sysfillrect sysimgblt ttm i2c_piix4 drm_kms_helper drm
[18195.617056] CPU: 2 PID: 1522 Comm: run-lkp Not tainted 3.16.0-rc3-00329-gec43f5a #1
[18195.617056] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[18195.617056] task: ffff88007f5dd850 ti: ffff88007f6dc000 task.ti: ffff88007f6dc000
[18195.617056] RIP: 0010:[<ffffffff810b706b>]  [<ffffffff810b706b>] print_cfs_group_stats.isra.0+0x6db/0xad0
[18195.617056] RSP: 0018:ffff88007f6dfd00  EFLAGS: 00010046
[18195.617056] RAX: ffffffff81fe5240 RBX: 0000000000000000 RCX: 0000000000007250
[18195.617056] RDX: ffff880076808000 RSI: 0000000000000000 RDI: 0000000000000000
[18195.617056] RBP: ffff88007f6dfd10 R08: 0000000000000092 R09: 000000000000037a
[18195.617056] R10: 0000000000000080 R11: 0000000000000002 R12: 0000000000000000
[18195.617056] R13: ffff880076e14880 R14: 0000027f6567ae6d R15: 0000000000000000
[18195.617056] FS:  00007f3479d82700(0000) GS:ffff880076f00000(0000) knlGS:0000000000000000
[18195.617056] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18195.617056] CR2: 0000000000000040 CR3: 000000007e0c4000 CR4: 00000000000006e0
[18195.617056] Stack:
[18195.617056]  0000000000000000 ffff880076e14900 ffff88007f6dfd78 ffffffff810b8589
[18195.617056]  0000027f6567ae6d 0000000000000086 ffffffffffffffff ffffffffffffffff
[18195.617056]  ffffffffffffffff ffffffffffffffff ffff880076e14900 0000000000000000
[18195.617056] Call Trace:
[18195.617056]  [<ffffffff810b8589>] print_cfs_rq+0x3c9/0x850
[18195.617056]  [<ffffffff810af43e>] print_cfs_stats+0x6e/0xc0
[18195.617056]  [<ffffffff810b7800>] print_cpu+0x350/0xce0
[18195.617056]  [<ffffffff810b6851>] ? sched_debug_header+0x5a1/0x600
[18195.617056]  [<ffffffff810b8cd1>] sysrq_sched_debug_show+0x31/0x60
[18195.617056]  [<ffffffff810a0d95>] show_state_filter+0xa5/0xc0
[18195.617056]  [<ffffffff814a45c3>] sysrq_handle_showstate_blocked+0x13/0x20
[18195.617056]  [<ffffffff814a4d67>] __handle_sysrq+0x117/0x170
[18195.617056]  [<ffffffff814a521a>] write_sysrq_trigger+0x4a/0x50
[18195.617056]  [<ffffffff8122b5ad>] proc_reg_write+0x3d/0x80
[18195.617056]  [<ffffffff811c62c7>] vfs_write+0xb7/0x1f0
[18195.617056]  [<ffffffff811c6e66>] SyS_write+0x46/0xb0
[18195.617056]  [<ffffffff81832ba9>] system_call_fastpath+0x16/0x1b
[18195.617056] Code: d7 82 de 1b 43 4c 89 c0 48 f7 e1 48 89 d0 48 89 d1 48 c1 e8 12 48 c1 e9 12 48 69 c0 40 42 0f 00 48 f7 d9 49 29 c0 e9 c8 f9 ff ff <49> 8b 44 24 40 48 85 c0 0f 88 c3 02 00 00 be 40 42 0f 00 31 d2 
[18195.617056] RIP  [<ffffffff810b706b>] print_cfs_group_stats.isra.0+0x6db/0xad0
[18195.617056]  RSP <ffff88007f6dfd00>
[18195.617056] CR2: 0000000000000040
[18195.617056] ------------[ cut here ]------------
[18195.617056] kernel BUG at arch/x86/mm/pageattr.c:216!
[18195.617056] invalid opcode: 0000 [#2] SMP 
[18195.617056] Modules linked in: ipmi_watchdog ipmi_msghandler dm_mod fuse sg sr_mod cdrom ata_generic pata_acpi parport_pc parport floppy snd_pcm snd_timer snd soundcore ata_piix pcspkr cirrus libata syscopyarea sysfillrect sysimgblt ttm i2c_piix4 drm_kms_helper drm
[18195.617056] CPU: 2 PID: 1522 Comm: run-lkp Not tainted 3.16.0-rc3-00329-gec43f5a #1
[18195.617056] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[18195.617056] task: ffff88007f5dd850 ti: ffff88007f6dc000 task.ti: ffff88007f6dc000
[18195.617056] RIP: 0010:[<ffffffff8105bbd3>]  [<ffffffff8105bbd3>] change_page_attr_set_clr+0x433/0x440
[18195.617056] RSP: 0018:ffff88007f6df068  EFLAGS: 00010046
[18195.617056] RAX: 0000000000000046 RBX: 0000000000000000 RCX: 0000000000000010
[18195.617056] RDX: 0000000000002200 RSI: 0000000000000000 RDI: 0000000080000000
[18195.617056] RBP: ffff88007f6df0f8 R08: 8000000040e8d163 R09: 0000000000040e8d
[18195.617056] R10: ffffea0001fc0200 R11: ffffffff813d9029 R12: 0000000000000000
[18195.617056] R13: 0000000000000010 R14: 0000000000000004 R15: 0000000000000005
[18195.617056] FS:  00007f3479d82700(0000) GS:ffff880076f00000(0000) knlGS:0000000000000000
[18195.617056] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18195.617056] CR2: 0000000000000040 CR3: 000000007e0c4000 CR4: 00000000000006e0
[18195.617056] Stack:
[18195.617056]  ffff88007f6df0a8 0000000000000000 0000000000000000 0000000000000200
[18195.617056]  0000000000000008 0000000000000000 0000000000000000 0000000000000010
[18195.617056]  0000000000000000 0000000500000001 0000000000040e8d 0000020000000000
[18195.617056] Call Trace:
[18195.617056]  [<ffffffff8105bf26>] _set_pages_array+0xe6/0x130
[18195.617056]  [<ffffffff8105bfa3>] set_pages_array_wc+0x13/0x20
[18195.617056]  [<ffffffffa008302f>] ttm_set_pages_caching+0x2f/0x70 [ttm]
[18195.617056]  [<ffffffffa0083174>] ttm_alloc_new_pages.isra.6+0xb4/0x180 [ttm]
[18195.617056]  [<ffffffffa0083a63>] ttm_pool_populate+0x3c3/0x4d0 [ttm]
[18195.617056]  [<ffffffffa00e520e>] cirrus_ttm_tt_populate+0xe/0x10 [cirrus]
[18195.617056]  [<ffffffffa0080571>] ttm_bo_move_memcpy+0x5d1/0x680 [ttm]
[18195.617056]  [<ffffffff8118c59e>] ? map_vm_area+0x2e/0x40
[18195.617056]  [<ffffffffa007c289>] ? ttm_tt_init+0x69/0xb0 [ttm]
[18195.617056]  [<ffffffffa00e51b8>] cirrus_bo_move+0x18/0x20 [cirrus]
[18195.617056]  [<ffffffffa007dd45>] ttm_bo_handle_move_mem+0x265/0x5b0 [ttm]
[18195.617056]  [<ffffffffa007e6a6>] ? ttm_bo_mem_space+0x116/0x340 [ttm]
[18195.617056]  [<ffffffffa007ed6f>] ttm_bo_validate+0x21f/0x230 [ttm]
[18195.617056]  [<ffffffffa00e59a2>] cirrus_bo_push_sysram+0x82/0xe0 [cirrus]
[18195.617056]  [<ffffffffa00e3bf5>] cirrus_crtc_do_set_base.isra.8.constprop.10+0x75/0x400 [cirrus]
[18195.617056]  [<ffffffffa00e43c9>] cirrus_crtc_mode_set+0x449/0x4d0 [cirrus]
[18195.617056]  [<ffffffffa006a8e9>] drm_crtc_helper_set_mode+0x2b9/0x4f0 [drm_kms_helper]
[18195.617056]  [<ffffffffa006b66f>] drm_crtc_helper_set_config+0x87f/0xaa0 [drm_kms_helper]
[18195.617056]  [<ffffffff818305de>] ? __ww_mutex_lock+0x2e/0xaa
[18195.617056]  [<ffffffffa000f2a1>] drm_mode_set_config_internal+0x61/0xe0 [drm]
[18195.617056]  [<ffffffffa006e324>] drm_fb_helper_pan_display+0x94/0xf0 [drm_kms_helper]
[18195.617056]  [<ffffffff8143d499>] fb_pan_display+0xc9/0x190
[18195.617056]  [<ffffffff81437650>] bit_update_start+0x20/0x50
[18195.617056]  [<ffffffff81437102>] fbcon_switch+0x3a2/0x550
[18195.617056]  [<ffffffff814aec19>] redraw_screen+0x189/0x240
[18195.617056]  [<ffffffff8143d80e>] ? fb_blank+0x9e/0xc0
[18195.617056]  [<ffffffff8143469a>] fbcon_blank+0x20a/0x2d0
[18195.617056]  [<ffffffff810ba9b2>] ? up+0x32/0x50
[18195.617056]  [<ffffffff810c79d8>] ? console_unlock+0x1f8/0x440
[18195.617056]  [<ffffffff81079133>] ? __internal_add_timer+0x113/0x130
[18195.617056]  [<ffffffff8107917f>] ? internal_add_timer+0x2f/0x70
[18195.617056]  [<ffffffff8107b1b2>] ? mod_timer+0x142/0x1f0
[18195.617056]  [<ffffffff814af738>] do_unblank_screen+0xb8/0x200
[18195.617056]  [<ffffffff814af890>] unblank_screen+0x10/0x20
[18195.617056]  [<ffffffff813e7d89>] bust_spinlocks+0x19/0x40
[18195.617056]  [<ffffffff81017718>] oops_end+0x38/0x150
[18195.617056]  [<ffffffff81821839>] no_context+0x2b3/0x2c0
[18195.617056]  [<ffffffff81433061>] ? fbcon_clear+0x1b1/0x1f0
[18195.617056]  [<ffffffff818218b9>] __bad_area_nosemaphore+0x73/0x1ca
[18195.617056]  [<ffffffff81821a23>] bad_area_nosemaphore+0x13/0x15
[18195.617056]  [<ffffffff81058c90>] __do_page_fault+0x90/0x550
[18195.617056]  [<ffffffff814baddb>] ? wait_for_xmitr+0x3b/0xa0
[18195.617056]  [<ffffffff814bafe7>] ? serial8250_console_putchar+0x27/0x30
[18195.617056]  [<ffffffff814bafc0>] ? serial8250_console_write+0x180/0x180
[18195.617056]  [<ffffffff814baddb>] ? wait_for_xmitr+0x3b/0xa0
[18195.617056]  [<ffffffff814baf32>] ? serial8250_console_write+0xf2/0x180
[18195.617056]  [<ffffffff81059201>] trace_do_page_fault+0x41/0x130
[18195.617056]  [<ffffffff810536b9>] do_async_page_fault+0x29/0xe0
[18195.617056]  [<ffffffff81834b88>] async_page_fault+0x28/0x30
[18195.617056]  [<ffffffff810b706b>] ? print_cfs_group_stats.isra.0+0x6db/0xad0
[18195.617056]  [<ffffffff810b8589>] print_cfs_rq+0x3c9/0x850
[18195.617056]  [<ffffffff810af43e>] print_cfs_stats+0x6e/0xc0
[18195.617056]  [<ffffffff810b7800>] print_cpu+0x350/0xce0
[18195.617056]  [<ffffffff810b6851>] ? sched_debug_header+0x5a1/0x600
[18195.617056]  [<ffffffff810b8cd1>] sysrq_sched_debug_show+0x31/0x60
[18195.617056]  [<ffffffff810a0d95>] show_state_filter+0xa5/0xc0
[18195.617056]  [<ffffffff814a45c3>] sysrq_handle_showstate_blocked+0x13/0x20
[18195.617056]  [<ffffffff814a4d67>] __handle_sysrq+0x117/0x170
[18195.617056]  [<ffffffff814a521a>] write_sysrq_trigger+0x4a/0x50
[18195.617056]  [<ffffffff8122b5ad>] proc_reg_write+0x3d/0x80
[18195.617056]  [<ffffffff811c62c7>] vfs_write+0xb7/0x1f0
[18195.617056]  [<ffffffff811c6e66>] SyS_write+0x46/0xb0
[18195.617056]  [<ffffffff81832ba9>] system_call_fastpath+0x16/0x1b
[18195.617056] Code: ff ff 48 8b 4d 80 e9 9f fc ff ff 0f 0b 0f 0b be ba 00 00 00 48 c7 c7 b8 a0 ae 81 89 4d 80 e8 d5 15 01 00 8b 4d 80 e9 04 ff ff ff <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 
[18195.617056] RIP  [<ffffffff8105bbd3>] change_page_attr_set_clr+0x433/0x440
[18195.617056]  RSP <ffff88007f6df068>
[18195.617056] ---[ end trace f2721a175d3d926b ]---
[18195.617056] Kernel panic - not syncing: Fatal exception



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.

Thanks,
Aaron