Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.

public inbox for linux-scsi@vger.kernel.org
 help / color / mirror / Atom feed

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
       [not found] ` <20170306102733.GC14932@quack2.suse.cz>
@ 2017-03-06 12:01   ` Jan Kara
  2017-03-06 14:35     ` James Bottomley
  0 siblings, 1 reply; 10+ messages in thread
From: Jan Kara @ 2017-03-06 12:01 UTC (permalink / raw)
  To: Fengguang Wu
  Cc: Jan Kara, Jens Axboe, linux-block, linux-kernel, LKP,
	James E.J. Bottomley, Martin K. Petersen, linux-scsi

[-- Attachment #1: Type: text/plain, Size: 152499 bytes --]

On Mon 06-03-17 11:27:33, Jan Kara wrote:
> Hi,
> 
> On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > FYI next-20170303 is good while mainline is bad with this error.
> > The attached reproduce-* may help reproduce the issue.
> 
> Thanks for report! So from the stacktrace we are in the path testing
> removal of a device immediately after it has been probed and for some
> reason bdi_unregister() hangs - likely waiting for cgroup-writeback
> references to drop. Given how early this happens my guess is we fail to
> initialize something but for now I don't see how my patch could make a
> difference. I'm trying to reproduce this to be able to debug more...

OK, so after some debugging I think this is yet another problem in SCSI
initialization / destruction code which my patch only makes visible (added
relevant maintainers).

I can reproduce the problem reliably with enabling:

CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
CONFIG_SCSI_DEBUG=m
CONFIG_BLK_CGROUP=y
CONFIG_MEMCG=y
(and thus CONFIG_CGROUP_WRITEBACK=y)

then 'modprobe scsi_debug' is all it takes to reproduce hang. Relevant
kernel messages with some of my debugging added (attached is a patch that
adds those debug messages):

[   58.721765] scsi host0: scsi_debug: version 1.86 [20160430]
[   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1, statistics=0
[   58.728946] CGWB init ffff88007fbb2000
[   58.730095] Created sdev ffff880078e1a000
[   58.731611] scsi 0:0:0:0: Direct-Access     Linux    scsi_debug 0186 PQ : 0 ANSI: 7
[   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical blocks: (8.39 MB/8.00 MiB)
[   58.789687] sd 0:0:0:0: [sda] Write Protect is off
[   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
[   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
[   58.896808] Unreg1
[   58.897960] Unreg2
[   58.898637] Unreg3
[   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
[   58.900004] Unreg4
[   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[   58.908362] Reusing sdev ffff880078e1a000
[   58.916194] sd 0:0:0:0: [sdb] 16384 512-byte logical blocks: (8.39 MB/8.00 MiB)
[   58.923513] sd 0:0:0:0: [sdb] Write Protect is off
[   58.925387] sd 0:0:0:0: [sdb] Mode Sense: 73 00 10 08
[   58.935263] sd 0:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   58.983663] sd 0:0:0:0: [sdb] Attached SCSI disk
[   58.985413] Unreg1
[   58.986022] Unreg2
[   58.986776] Unreg3
[   58.987443] CGWB ffff88007fbb2000 usage_cnt: 4294967295

In the above, you can note that 'sdb' reused LUN 0 from the SCSI host. That
is also confirmed by the 'Reusing sdev XY' debug message which I've added
to scsi_probe_and_add_lun(). And IMO that's a bug because you reuse the
scsi_device structure, request_queue & backing_dev_info are already torn down
at that moment and you just start to use them again. Moreover it seems you
drop the reference to found scsi_device so that will get freed under your
hands.

My patch just made this problem visible as the second call to
bdi_unregister() on the same bdi resulted in underflow of usage_cnt and a
hang...

James, Martin?

								Honza

> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
> > 
> > commit 165a5e22fafb127ecb5914e12e8c32a1f0d3f820
> > Author:     Jan Kara <jack@suse.cz>
> > AuthorDate: Wed Feb 8 08:05:56 2017 +0100
> > Commit:     Jens Axboe <axboe@fb.com>
> > CommitDate: Thu Mar 2 16:08:35 2017 -0700
> > 
> >      block: Move bdi_unregister() to del_gendisk()
> >      
> >      Commit 6cd18e711dd8 "block: destroy bdi before blockdev is
> >      unregistered." moved bdi unregistration (at that time through
> >      bdi_destroy()) from blk_release_queue() to blk_cleanup_queue() because
> >      it needs to happen before blk_unregister_region() call in del_gendisk()
> >      for MD. SCSI though will free up the device number from sd_remove()
> >      called through a maze of callbacks from device_del() in
> >      __scsi_remove_device() before blk_cleanup_queue() and thus similar races
> >      as described in 6cd18e711dd8 can happen for SCSI as well as reported by
> >      Omar [1].
> >      
> >      Moving bdi_unregister() to del_gendisk() works for MD and fixes the
> >      problem for SCSI since del_gendisk() gets called from sd_remove() before
> >      freeing the device number.
> >      
> >      This also makes device_add_disk() (calling bdi_register_owner()) more
> >      symmetric with del_gendisk().
> >      
> >      [1] http://marc.info/?l=linux-block&m=148554717109098&w=2
> >      
> >      Tested-by: Lekshmi Pillai <lekshmicpillai@in.ibm.com>
> >      Acked-by: Tejun Heo <tj@kernel.org>
> >      Signed-off-by: Jan Kara <jack@suse.cz>
> >      Tested-by: Omar Sandoval <osandov@fb.com>
> >      Signed-off-by: Jens Axboe <axboe@fb.com>
> > 
> > 113285b473  blk-mq: ensure that bd->last is always set correctly
> > 165a5e22fa  block: Move bdi_unregister() to del_gendisk()
> > +--------------------------------------------------+------------+------------+
> > |                                                  | 113285b473 | 165a5e22fa |
> > +--------------------------------------------------+------------+------------+
> > | boot_successes                                   | 35         | 0          |
> > | boot_failures                                    | 0          | 15         |
> > | INFO:task_blocked_for_more_than#seconds          | 0          | 15         |
> > | Kernel_panic-not_syncing:hung_task:blocked_tasks | 0          | 15         |
> > +--------------------------------------------------+------------+------------+
> > 
> > [  187.365491] rcu_preempt: wait state: 1 ->state: 0x1
> > [  187.366039] rcu_sched: wait state: 1 ->state: 0x1
> > [  187.366039] rcu_sched: wait state: 1 ->state: 0x1
> > [  187.366564] rcu_bh: wait state: 1 ->state: 0x1
> > [  187.366564] rcu_bh: wait state: 1 ->state: 0x1
> > [  243.680303] INFO: task swapper:1 blocked for more than 120 seconds.
> > [  243.680303] INFO: task swapper:1 blocked for more than 120 seconds.
> > [  243.682035]       Not tainted 4.10.0-11089-g165a5e2 #2
> > [  243.682035]       Not tainted 4.10.0-11089-g165a5e2 #2
> > [  243.683000] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > [  243.683000] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > [  243.684306] swapper         D    0     1      0 0x00000000
> > [  243.684306] swapper         D    0     1      0 0x00000000
> > [  243.685246] Call Trace:
> > [  243.685246] Call Trace:
> > [  243.685650]  __schedule+0x31a/0xc00
> > [  243.685650]  __schedule+0x31a/0xc00
> > [  243.686236]  schedule+0x7b/0x170
> > [  243.686236]  schedule+0x7b/0x170
> > [  243.686803]  bdi_unregister+0x3a3/0x430
> > [  243.686803]  bdi_unregister+0x3a3/0x430
> > [  243.687447]  ? woken_wake_function+0x40/0x40
> > [  243.687447]  ? woken_wake_function+0x40/0x40
> > [  243.688150]  del_gendisk+0x232/0x370
> > [  243.688150]  del_gendisk+0x232/0x370
> > [  243.688721]  sd_remove+0x97/0x140
> > [  243.688721]  sd_remove+0x97/0x140
> > [  243.689260]  device_release_driver_internal+0x4b8/0x4d0
> > [  243.689260]  device_release_driver_internal+0x4b8/0x4d0
> > [  243.690065]  device_release_driver+0x1a/0x30
> > [  243.690065]  device_release_driver+0x1a/0x30
> > [  243.690928]  bus_remove_device+0x15a/0x1f0
> > [  243.690928]  bus_remove_device+0x15a/0x1f0
> > [  243.691558]  device_del+0x34d/0x5a0
> > [  243.691558]  device_del+0x34d/0x5a0
> > [  243.692155]  ? kobject_put+0xf6/0x1d0
> > [  243.692155]  ? kobject_put+0xf6/0x1d0
> > [  243.692700]  __scsi_remove_device+0x1a3/0x1c0
> > [  243.692700]  __scsi_remove_device+0x1a3/0x1c0
> > [  243.693665]  scsi_forget_host+0xb7/0x120
> > [  243.693665]  scsi_forget_host+0xb7/0x120
> > [  243.694301]  scsi_remove_host+0xf1/0x300
> > [  243.694301]  scsi_remove_host+0xf1/0x300
> > [  243.694973]  sdebug_driver_remove+0x34/0x120
> > [  243.694973]  sdebug_driver_remove+0x34/0x120
> > [  243.695621]  ? devices_kset_move_last+0x9d/0x100
> > [  243.695621]  ? devices_kset_move_last+0x9d/0x100
> > [  243.696430]  driver_probe_device+0x172/0x6b0
> > [  243.696430]  driver_probe_device+0x172/0x6b0
> > [  243.697413]  __device_attach_driver+0x15d/0x1d0
> > [  243.697413]  __device_attach_driver+0x15d/0x1d0
> > [  243.698460]  ? __driver_attach+0x1b0/0x1b0
> > [  243.698460]  ? __driver_attach+0x1b0/0x1b0
> > [  243.699111]  bus_for_each_drv+0xb1/0x110
> > [  243.699111]  bus_for_each_drv+0xb1/0x110
> > [  243.699856]  __device_attach+0x14d/0x210
> > [  243.699856]  __device_attach+0x14d/0x210
> > [  243.700643]  ? __driver_attach+0x1b0/0x1b0
> > [  243.700643]  ? __driver_attach+0x1b0/0x1b0
> > [  243.701362]  device_initial_probe+0x1b/0x30
> > [  243.701362]  device_initial_probe+0x1b/0x30
> > [  243.701886]  bus_probe_device+0xe5/0x100
> > [  243.701886]  bus_probe_device+0xe5/0x100
> > [  243.702447]  device_add+0x822/0xc20
> > [  243.702447]  device_add+0x822/0xc20
> > [  243.702879]  ? pm_runtime_init+0x125/0x140
> > [  243.702879]  ? pm_runtime_init+0x125/0x140
> > [  243.703366]  device_register+0x2e/0x40
> > [  243.703366]  device_register+0x2e/0x40
> > [  243.703823]  sdebug_add_adapter+0x144/0x2e0
> > [  243.703823]  sdebug_add_adapter+0x144/0x2e0
> > [  243.704381]  ? kobject_uevent+0x18/0x30
> > [  243.704381]  ? kobject_uevent+0x18/0x30
> > [  243.704881]  scsi_debug_init+0x955/0xb9d
> > [  243.704881]  scsi_debug_init+0x955/0xb9d
> > [  243.705372]  ? init_sg+0x1a9/0x212
> > [  243.705372]  ? init_sg+0x1a9/0x212
> > [  243.705763]  ? init_sg+0x212/0x212
> > [  243.705763]  ? init_sg+0x212/0x212
> > [  243.706166]  do_one_initcall+0x5d/0x2b0
> > [  243.706166]  do_one_initcall+0x5d/0x2b0
> > [  243.706634]  ? parse_args+0x44e/0x710
> > [  243.706634]  ? parse_args+0x44e/0x710
> > [  243.707059]  ? kernel_init_freeable+0x20b/0x3ab
> > [  243.707059]  ? kernel_init_freeable+0x20b/0x3ab
> > [  243.707569]  kernel_init_freeable+0x23c/0x3ab
> > [  243.707569]  kernel_init_freeable+0x23c/0x3ab
> > [  243.708076]  ? rest_init+0x270/0x270
> > [  243.708076]  ? rest_init+0x270/0x270
> > [  243.708543]  kernel_init+0x16/0x250
> > [  243.708543]  kernel_init+0x16/0x250
> > [  243.708978]  ret_from_fork+0x21/0x30
> > [  243.708978]  ret_from_fork+0x21/0x30
> > [  243.709399] 
> > [  243.709399] Showing all locks held in the system:
> > [  243.709399] 
> > [  243.709399] Showing all locks held in the system:
> > [  243.710265] 3 locks held by swapper/1:
> > [  243.710265] 3 locks held by swapper/1:
> > [  243.710785]  #0:  (&dev->mutex){......}, at: [<987efeb8>] __device_attach+0x28/0x210
> > [  243.710785]  #0:  (&dev->mutex){......}, at: [<987efeb8>] __device_attach+0x28/0x210
> > [  243.711670]  #1:  (&shost->scan_mutex){+.+.+.}, at: [<9883cac1>] scsi_remove_host+0x21/0x300
> > [  243.711670]  #1:  (&shost->scan_mutex){+.+.+.}, at: [<9883cac1>] scsi_remove_host+0x21/0x300
> > [  243.712674]  #2:  (&dev->mutex){......}, at: [<987f0d94>] device_release_driver_internal+0x54/0x4d0
> > [  243.712674]  #2:  (&dev->mutex){......}, at: [<987f0d94>] device_release_driver_internal+0x54/0x4d0
> > [  243.713807] 2 locks held by khungtaskd/13:
> > [  243.713807] 2 locks held by khungtaskd/13:
> > [  243.714297]  #0:  (rcu_read_lock){......}, at: [<97f5f626>] watchdog+0x126/0x8c0
> > [  243.714297]  #0:  (rcu_read_lock){......}, at: [<97f5f626>] watchdog+0x126/0x8c0
> > [  243.715168]  #1:  (tasklist_lock){.+.+..}, at: [<97ee6d62>] debug_show_all_locks+0x52/0x320
> > [  243.715168]  #1:  (tasklist_lock){.+.+..}, at: [<97ee6d62>] debug_show_all_locks+0x52/0x320
> > [  243.716128] 
> > [  243.716128] 
> > [  243.716372] =============================================
> > [  243.716372] 
> > [  243.716372] =============================================
> > [  243.716372] 
> > [  243.717191] Kernel panic - not syncing: hung_task: blocked tasks
> > [  243.717191] Kernel panic - not syncing: hung_task: blocked tasks
> > [  243.717889] CPU: 0 PID: 13 Comm: khungtaskd Not tainted 4.10.0-11089-g165a5e2 #2
> > [  243.717889] CPU: 0 PID: 13 Comm: khungtaskd Not tainted 4.10.0-11089-g165a5e2 #2
> > [  243.718742] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
> > [  243.718742] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
> > [  243.719871] Call Trace:
> > [  243.719871] Call Trace:
> > [  243.720185]  dump_stack+0x32/0x43
> > [  243.720185]  dump_stack+0x32/0x43
> > [  243.720185]  panic+0x135/0x408
> > [  243.720185]  panic+0x135/0x408
> > [  243.720185]  watchdog+0x6d3/0x8c0
> > [  243.720185]  watchdog+0x6d3/0x8c0
> > [  243.720185]  kthread+0x151/0x1d0
> > [  243.720185]  kthread+0x151/0x1d0
> > [  243.720185]  ? hung_task_panic+0x20/0x20
> > [  243.720185]  ? hung_task_panic+0x20/0x20
> > [  243.720185]  ? kthread_worker_fn+0x220/0x220
> > [  243.720185]  ? kthread_worker_fn+0x220/0x220
> > [  243.720185]  ret_from_fork+0x21/0x30
> > [  243.720185]  ret_from_fork+0x21/0x30
> > [  243.720185] Kernel Offset: 0x16e00000 from 0x81000000 (relocation range: 0x80000000-0xa07dffff)
> > [  243.720185] Kernel Offset: 0x16e00000 from 0x81000000 (relocation range: 0x80000000-0xa07dffff)
> > 
> > 
> >                                                           # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
> > git bisect start 0724f5d8ca3a89134a8c248aa4c311ce43d53512 c470abd4fde40ea6a0846a2beab642a578c0b8cd --
> > git bisect  bad 304f9c7ff9aabc9a3cec12bf88f29d24139ac370  # 14:12  B      0     4   15   0  Merge 'linux-review/Roger-Quadros/Revert-usb-gadget-uvc-Add-missing-call-for-additional-setup-data/20170303-232925' into devel-spot-201703040603
> > git bisect  bad a28ca518ffc36daf89550978066ad20094c19e51  # 14:33  B      0     8   22   3  Merge 'linux-review/Arushi-Singhal/staging-speakup-Alignment-match-open-parenthesis/20170304-030809' into devel-spot-201703040603
> > git bisect  bad 93354ede964ded2791d3b573635d538b14590545  # 14:55  B      0     4   15   0  Merge 'linux-review/Georgios-Emmanouil/Staging-wilc1000-wilc_sdio-Modified-comment-style-to-preferred-kernel-comment-style/20170304-050826' into devel-spot-201703040603
> > git bisect  bad 402decdec94a2e4e2635ef662e57f6217a305d81  # 15:36  B      0    11   22   0  Merge 'linux-review/Heiner-Kallweit/staging-fbtft-improve-a-syslog-message-in-fbtft_register_framebuffer/20170304-054107' into devel-spot-201703040603
> > git bisect  bad 1735b51c4dda1ca7f2c239c4b790c3c638281f54  # 15:53  B      0     6   17   0  Merge 'block/for-linus' into devel-spot-201703040603
> > git bisect good cf5f16969d36dfa64e92cf19594b94de0c381519  # 16:05  G     11     0    0   0  0day base guard for 'devel-spot-201703040603'
> > git bisect good caa59428971d5ad81d19512365c9ba580d83268c  # 16:23  G     11     0    0   0  Merge tag 'staging-4.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
> > git bisect good d5500a074741b78b7f778b4ab3415d5ecdcda0a7  # 16:48  G     11     0    0   0  Merge tag 'usercopy-v4.11-rc1.fix' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
> > git bisect good 7067739df23ffd641ca99c967830e0ed2ba39eab  # 17:13  G     11     0    0   0  Merge branch 'i2c/for-4.11' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
> > git bisect good b2deee2dc06db7cdf99b84346e69bdb9db9baa85  # 17:47  G     11     0    0   0  Merge tag 'ceph-for-4.11-rc1' of git://github.com/ceph/ceph-client
> > git bisect good 010426079ec1228a7f980d2eef766a84c0f9241a  # 18:03  G     11     0    0   0  sched/headers: Prepare for new header dependencies before moving more code to <linux/sched/mm.h>
> > git bisect good 3f80dd67c367878aaad16e458eebc3c8980bb841  # 18:28  G     11     0    0   0  Merge tag 'acpi-extra-4.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
> > git bisect good e26512fea5bcd6602dbf02a551ed073cd4529449  # 18:42  G     10     0    0   0  sched/headers: Remove <linux/cred.h> inclusion from <linux/sched.h>
> > git bisect good 78769912f680fc0a79a67e798a0ae76f07e63a7b  # 19:08  G     11     0    0   0  Merge tag 'linux-kselftest-4.11-rc1-urgent_fix' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
> > git bisect good 1827adb11ad26b2290dc9fe2aaf54976b2439865  # 19:21  G     11     0    0   0  Merge branch 'WIP.sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
> > git bisect  bad 165a5e22fafb127ecb5914e12e8c32a1f0d3f820  # 19:39  B      0    11   22   0  block: Move bdi_unregister() to del_gendisk()
> > git bisect good 6a8a21546507a3ec88e81c2ec927a3fb63efa8ff  # 19:55  G     11     0    0   0  nbd: stop leaking sockets
> > git bisect good e02898b423802b1f3a3aaa7f16e896da069ba8f7  # 20:36  G     11     0    0   0  loop: fix LO_FLAGS_PARTSCAN hang
> > git bisect good 7b36a7189fc320f0b783dd51bd1f541db56cfbdd  # 20:53  G     11     0    0   0  block: don't call ioc_exit_icq() with the queue lock held for blk-mq
> > git bisect good 113285b473824922498d07d7f82459507b9792eb  # 21:37  G     11     0    0   0  blk-mq: ensure that bd->last is always set correctly
> > # first bad commit: [165a5e22fafb127ecb5914e12e8c32a1f0d3f820] block: Move bdi_unregister() to del_gendisk()
> > git bisect good 113285b473824922498d07d7f82459507b9792eb  # 21:41  G     31     0    0   0  blk-mq: ensure that bd->last is always set correctly
> > # extra tests with CONFIG_DEBUG_INFO_REDUCED
> > git bisect  bad 165a5e22fafb127ecb5914e12e8c32a1f0d3f820  # 22:16  B      0    10   22   1  block: Move bdi_unregister() to del_gendisk()
> > # extra tests on HEAD of linux-devel/devel-spot-201703040603
> > git bisect  bad 0724f5d8ca3a89134a8c248aa4c311ce43d53512  # 22:16  B      0    39   55   2  0day head guard for 'devel-spot-201703040603'
> > # extra tests on tree/branch linus/master
> > git bisect  bad 0710f3ff91ecc4a715db6e4d0690472b13c4dac6  # 22:42  B      0     7   18   0  Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
> > # extra tests with first bad commit reverted
> > git bisect good 8b222187187cd9372d0d1ee036f1966e142bbd42  # 23:09  G     10     0    0   0  Revert "block: Move bdi_unregister() to del_gendisk()"
> > # extra tests on tree/branch linux-next/master
> > git bisect good c0b7b2b33bd17f7155956d0338ce92615da686c9  # 23:28  G     11     0    0   0  Add linux-next specific files for 20170303
> > 
> > ---
> > 0-DAY kernel test infrastructure                Open Source Technology Center
> > https://lists.01.org/pipermail/lkp                          Intel Corporation
> 
> 
> > #!/bin/bash
> > 
> > kernel=$1
> > initrd=quantal-core-i386.cgz
> > 
> > wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd
> > 
> > kvm=(
> > 	qemu-system-x86_64
> > 	-enable-kvm
> > 	-cpu kvm64
> > 	-kernel $kernel
> > 	-initrd $initrd
> > 	-m 512
> > 	-smp 2
> > 	-device e1000,netdev=net0
> > 	-netdev user,id=net0
> > 	-boot order=nc
> > 	-no-reboot
> > 	-watchdog i6300esb
> > 	-watchdog-action debug
> > 	-rtc base=localtime
> > 	-serial stdio
> > 	-display none
> > 	-monitor null
> > )
> > 
> > append=(
> > 	root=/dev/ram0
> > 	hung_task_panic=1
> > 	debug
> > 	apic=debug
> > 	sysrq_always_enabled
> > 	rcupdate.rcu_cpu_stall_timeout=100
> > 	net.ifnames=0
> > 	printk.devkmsg=on
> > 	panic=-1
> > 	softlockup_panic=1
> > 	nmi_watchdog=panic
> > 	oops=panic
> > 	load_ramdisk=2
> > 	prompt_ramdisk=0
> > 	drbd.minor_count=8
> > 	systemd.log_level=err
> > 	ignore_loglevel
> > 	earlyprintk=ttyS0,115200
> > 	console=ttyS0,115200
> > 	console=tty0
> > 	vga=normal
> > 	rw
> > 	drbd.minor_count=8
> > )
> > 
> > "${kvm[@]}" -append "${append[*]}"
> 
> > #
> > # Automatically generated file; DO NOT EDIT.
> > # Linux/i386 4.10.0 Kernel Configuration
> > #
> > # CONFIG_64BIT is not set
> > CONFIG_X86_32=y
> > CONFIG_X86=y
> > CONFIG_INSTRUCTION_DECODER=y
> > CONFIG_OUTPUT_FORMAT="elf32-i386"
> > CONFIG_ARCH_DEFCONFIG="arch/x86/configs/i386_defconfig"
> > CONFIG_LOCKDEP_SUPPORT=y
> > CONFIG_STACKTRACE_SUPPORT=y
> > CONFIG_MMU=y
> > CONFIG_ARCH_MMAP_RND_BITS_MIN=8
> > CONFIG_ARCH_MMAP_RND_BITS_MAX=16
> > CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
> > CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
> > CONFIG_NEED_DMA_MAP_STATE=y
> > CONFIG_NEED_SG_DMA_LENGTH=y
> > CONFIG_GENERIC_ISA_DMA=y
> > CONFIG_GENERIC_BUG=y
> > CONFIG_GENERIC_HWEIGHT=y
> > CONFIG_ARCH_MAY_HAVE_PC_FDC=y
> > CONFIG_RWSEM_XCHGADD_ALGORITHM=y
> > CONFIG_GENERIC_CALIBRATE_DELAY=y
> > CONFIG_ARCH_HAS_CPU_RELAX=y
> > CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
> > CONFIG_HAVE_SETUP_PER_CPU_AREA=y
> > CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
> > CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
> > CONFIG_ARCH_HIBERNATION_POSSIBLE=y
> > CONFIG_ARCH_SUSPEND_POSSIBLE=y
> > CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
> > CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
> > CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
> > CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
> > CONFIG_X86_32_LAZY_GS=y
> > CONFIG_ARCH_SUPPORTS_UPROBES=y
> > CONFIG_FIX_EARLYCON_MEM=y
> > CONFIG_PGTABLE_LEVELS=3
> > CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
> > CONFIG_CONSTRUCTORS=y
> > CONFIG_IRQ_WORK=y
> > CONFIG_BUILDTIME_EXTABLE_SORT=y
> > CONFIG_THREAD_INFO_IN_TASK=y
> > 
> > #
> > # General setup
> > #
> > CONFIG_BROKEN_ON_SMP=y
> > CONFIG_INIT_ENV_ARG_LIMIT=32
> > CONFIG_CROSS_COMPILE=""
> > # CONFIG_COMPILE_TEST is not set
> > CONFIG_LOCALVERSION=""
> > CONFIG_LOCALVERSION_AUTO=y
> > CONFIG_HAVE_KERNEL_GZIP=y
> > CONFIG_HAVE_KERNEL_BZIP2=y
> > CONFIG_HAVE_KERNEL_LZMA=y
> > CONFIG_HAVE_KERNEL_XZ=y
> > CONFIG_HAVE_KERNEL_LZO=y
> > CONFIG_HAVE_KERNEL_LZ4=y
> > # CONFIG_KERNEL_GZIP is not set
> > # CONFIG_KERNEL_BZIP2 is not set
> > # CONFIG_KERNEL_LZMA is not set
> > # CONFIG_KERNEL_XZ is not set
> > CONFIG_KERNEL_LZO=y
> > # CONFIG_KERNEL_LZ4 is not set
> > CONFIG_DEFAULT_HOSTNAME="(none)"
> > # CONFIG_SWAP is not set
> > # CONFIG_SYSVIPC is not set
> > # CONFIG_POSIX_MQUEUE is not set
> > CONFIG_CROSS_MEMORY_ATTACH=y
> > CONFIG_FHANDLE=y
> > # CONFIG_USELIB is not set
> > # CONFIG_AUDIT is not set
> > CONFIG_HAVE_ARCH_AUDITSYSCALL=y
> > 
> > #
> > # IRQ subsystem
> > #
> > CONFIG_GENERIC_IRQ_PROBE=y
> > CONFIG_GENERIC_IRQ_SHOW=y
> > CONFIG_GENERIC_IRQ_CHIP=y
> > CONFIG_IRQ_DOMAIN=y
> > # CONFIG_IRQ_DOMAIN_DEBUG is not set
> > CONFIG_IRQ_FORCED_THREADING=y
> > CONFIG_SPARSE_IRQ=y
> > CONFIG_CLOCKSOURCE_WATCHDOG=y
> > CONFIG_ARCH_CLOCKSOURCE_DATA=y
> > CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
> > CONFIG_GENERIC_TIME_VSYSCALL=y
> > CONFIG_GENERIC_CLOCKEVENTS=y
> > CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
> > CONFIG_GENERIC_CMOS_UPDATE=y
> > 
> > #
> > # Timers subsystem
> > #
> > CONFIG_TICK_ONESHOT=y
> > CONFIG_HZ_PERIODIC=y
> > # CONFIG_NO_HZ_IDLE is not set
> > # CONFIG_NO_HZ is not set
> > CONFIG_HIGH_RES_TIMERS=y
> > 
> > #
> > # CPU/Task time and stats accounting
> > #
> > CONFIG_TICK_CPU_ACCOUNTING=y
> > # CONFIG_IRQ_TIME_ACCOUNTING is not set
> > # CONFIG_BSD_PROCESS_ACCT is not set
> > # CONFIG_TASKSTATS is not set
> > 
> > #
> > # RCU Subsystem
> > #
> > CONFIG_PREEMPT_RCU=y
> > CONFIG_RCU_EXPERT=y
> > CONFIG_SRCU=y
> > CONFIG_TASKS_RCU=y
> > CONFIG_RCU_STALL_COMMON=y
> > CONFIG_RCU_FANOUT=32
> > CONFIG_RCU_FANOUT_LEAF=16
> > CONFIG_TREE_RCU_TRACE=y
> > # CONFIG_RCU_BOOST is not set
> > CONFIG_RCU_KTHREAD_PRIO=0
> > # CONFIG_RCU_NOCB_CPU is not set
> > CONFIG_BUILD_BIN2C=y
> > CONFIG_IKCONFIG=y
> > CONFIG_IKCONFIG_PROC=y
> > CONFIG_LOG_BUF_SHIFT=17
> > CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
> > CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
> > CONFIG_CGROUPS=y
> > CONFIG_PAGE_COUNTER=y
> > CONFIG_MEMCG=y
> > CONFIG_BLK_CGROUP=y
> > # CONFIG_DEBUG_BLK_CGROUP is not set
> > CONFIG_CGROUP_WRITEBACK=y
> > CONFIG_CGROUP_SCHED=y
> > CONFIG_FAIR_GROUP_SCHED=y
> > CONFIG_CFS_BANDWIDTH=y
> > CONFIG_RT_GROUP_SCHED=y
> > CONFIG_CGROUP_PIDS=y
> > # CONFIG_CGROUP_RDMA is not set
> > # CONFIG_CGROUP_FREEZER is not set
> > # CONFIG_CPUSETS is not set
> > CONFIG_CGROUP_DEVICE=y
> > # CONFIG_CGROUP_CPUACCT is not set
> > CONFIG_CGROUP_PERF=y
> > # CONFIG_CGROUP_BPF is not set
> > CONFIG_CGROUP_DEBUG=y
> > CONFIG_SOCK_CGROUP_DATA=y
> > # CONFIG_CHECKPOINT_RESTORE is not set
> > # CONFIG_NAMESPACES is not set
> > CONFIG_SCHED_AUTOGROUP=y
> > # CONFIG_SYSFS_DEPRECATED is not set
> > # CONFIG_RELAY is not set
> > CONFIG_BLK_DEV_INITRD=y
> > CONFIG_INITRAMFS_SOURCE=""
> > CONFIG_RD_GZIP=y
> > # CONFIG_RD_BZIP2 is not set
> > CONFIG_RD_LZMA=y
> > # CONFIG_RD_XZ is not set
> > # CONFIG_RD_LZO is not set
> > # CONFIG_RD_LZ4 is not set
> > CONFIG_INITRAMFS_COMPRESSION=".gz"
> > CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
> > # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
> > CONFIG_SYSCTL=y
> > CONFIG_ANON_INODES=y
> > CONFIG_HAVE_UID16=y
> > CONFIG_SYSCTL_EXCEPTION_TRACE=y
> > CONFIG_HAVE_PCSPKR_PLATFORM=y
> > CONFIG_BPF=y
> > CONFIG_EXPERT=y
> > CONFIG_UID16=y
> > CONFIG_MULTIUSER=y
> > CONFIG_SGETMASK_SYSCALL=y
> > CONFIG_SYSFS_SYSCALL=y
> > # CONFIG_SYSCTL_SYSCALL is not set
> > # CONFIG_POSIX_TIMERS is not set
> > CONFIG_KALLSYMS=y
> > CONFIG_KALLSYMS_ALL=y
> > # CONFIG_KALLSYMS_ABSOLUTE_PERCPU is not set
> > CONFIG_KALLSYMS_BASE_RELATIVE=y
> > CONFIG_PRINTK=y
> > CONFIG_PRINTK_NMI=y
> > CONFIG_BUG=y
> > CONFIG_PCSPKR_PLATFORM=y
> > # CONFIG_BASE_FULL is not set
> > CONFIG_FUTEX=y
> > CONFIG_EPOLL=y
> > CONFIG_SIGNALFD=y
> > CONFIG_TIMERFD=y
> > CONFIG_EVENTFD=y
> > CONFIG_BPF_SYSCALL=y
> > CONFIG_SHMEM=y
> > # CONFIG_AIO is not set
> > # CONFIG_ADVISE_SYSCALLS is not set
> > # CONFIG_USERFAULTFD is not set
> > CONFIG_PCI_QUIRKS=y
> > # CONFIG_MEMBARRIER is not set
> > CONFIG_EMBEDDED=y
> > CONFIG_HAVE_PERF_EVENTS=y
> > CONFIG_PERF_USE_VMALLOC=y
> > # CONFIG_PC104 is not set
> > 
> > #
> > # Kernel Performance Events And Counters
> > #
> > CONFIG_PERF_EVENTS=y
> > CONFIG_DEBUG_PERF_USE_VMALLOC=y
> > # CONFIG_VM_EVENT_COUNTERS is not set
> > # CONFIG_COMPAT_BRK is not set
> > # CONFIG_SLAB is not set
> > # CONFIG_SLUB is not set
> > CONFIG_SLOB=y
> > # CONFIG_SYSTEM_DATA_VERIFICATION is not set
> > # CONFIG_PROFILING is not set
> > CONFIG_KEXEC_CORE=y
> > CONFIG_HAVE_OPROFILE=y
> > CONFIG_OPROFILE_NMI_TIMER=y
> > CONFIG_KPROBES=y
> > # CONFIG_JUMP_LABEL is not set
> > # CONFIG_UPROBES is not set
> > # CONFIG_HAVE_64BIT_ALIGNED_ACCESS is not set
> > CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
> > CONFIG_ARCH_USE_BUILTIN_BSWAP=y
> > CONFIG_KRETPROBES=y
> > CONFIG_HAVE_IOREMAP_PROT=y
> > CONFIG_HAVE_KPROBES=y
> > CONFIG_HAVE_KRETPROBES=y
> > CONFIG_HAVE_OPTPROBES=y
> > CONFIG_HAVE_KPROBES_ON_FTRACE=y
> > CONFIG_HAVE_NMI=y
> > CONFIG_HAVE_ARCH_TRACEHOOK=y
> > CONFIG_HAVE_DMA_CONTIGUOUS=y
> > CONFIG_GENERIC_SMP_IDLE_THREAD=y
> > CONFIG_ARCH_HAS_SET_MEMORY=y
> > CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
> > CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
> > CONFIG_HAVE_CLK=y
> > CONFIG_HAVE_DMA_API_DEBUG=y
> > CONFIG_HAVE_HW_BREAKPOINT=y
> > CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
> > CONFIG_HAVE_USER_RETURN_NOTIFIER=y
> > CONFIG_HAVE_PERF_EVENTS_NMI=y
> > CONFIG_HAVE_PERF_REGS=y
> > CONFIG_HAVE_PERF_USER_STACK_DUMP=y
> > CONFIG_HAVE_ARCH_JUMP_LABEL=y
> > CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
> > CONFIG_HAVE_CMPXCHG_LOCAL=y
> > CONFIG_HAVE_CMPXCHG_DOUBLE=y
> > CONFIG_ARCH_WANT_IPC_PARSE_VERSION=y
> > CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
> > CONFIG_HAVE_GCC_PLUGINS=y
> > # CONFIG_GCC_PLUGINS is not set
> > CONFIG_HAVE_CC_STACKPROTECTOR=y
> > # CONFIG_CC_STACKPROTECTOR is not set
> > CONFIG_CC_STACKPROTECTOR_NONE=y
> > # CONFIG_CC_STACKPROTECTOR_REGULAR is not set
> > # CONFIG_CC_STACKPROTECTOR_STRONG is not set
> > CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
> > CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
> > CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
> > CONFIG_HAVE_ARCH_HUGE_VMAP=y
> > CONFIG_MODULES_USE_ELF_REL=y
> > CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
> > CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
> > CONFIG_HAVE_EXIT_THREAD=y
> > CONFIG_ARCH_MMAP_RND_BITS=8
> > CONFIG_HAVE_COPY_THREAD_TLS=y
> > # CONFIG_HAVE_ARCH_HASH is not set
> > CONFIG_ISA_BUS_API=y
> > CONFIG_CLONE_BACKWARDS=y
> > CONFIG_OLD_SIGSUSPEND3=y
> > CONFIG_OLD_SIGACTION=y
> > # CONFIG_CPU_NO_EFFICIENT_FFS is not set
> > # CONFIG_HAVE_ARCH_VMAP_STACK is not set
> > # CONFIG_ARCH_OPTIONAL_KERNEL_RWX is not set
> > # CONFIG_ARCH_OPTIONAL_KERNEL_RWX_DEFAULT is not set
> > CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
> > CONFIG_STRICT_KERNEL_RWX=y
> > CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
> > CONFIG_STRICT_MODULE_RWX=y
> > 
> > #
> > # GCOV-based kernel profiling
> > #
> > CONFIG_GCOV_KERNEL=y
> > CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
> > CONFIG_GCOV_PROFILE_ALL=y
> > CONFIG_GCOV_FORMAT_AUTODETECT=y
> > # CONFIG_GCOV_FORMAT_3_4 is not set
> > # CONFIG_GCOV_FORMAT_4_7 is not set
> > CONFIG_HAVE_GENERIC_DMA_COHERENT=y
> > CONFIG_RT_MUTEXES=y
> > CONFIG_BASE_SMALL=1
> > CONFIG_MODULES=y
> > # CONFIG_MODULE_FORCE_LOAD is not set
> > # CONFIG_MODULE_UNLOAD is not set
> > # CONFIG_MODVERSIONS is not set
> > # CONFIG_MODULE_SRCVERSION_ALL is not set
> > # CONFIG_MODULE_SIG is not set
> > CONFIG_MODULE_COMPRESS=y
> > CONFIG_MODULE_COMPRESS_GZIP=y
> > # CONFIG_MODULE_COMPRESS_XZ is not set
> > CONFIG_TRIM_UNUSED_KSYMS=y
> > CONFIG_MODULES_TREE_LOOKUP=y
> > CONFIG_BLOCK=y
> > # CONFIG_LBDAF is not set
> > CONFIG_BLK_SCSI_REQUEST=y
> > CONFIG_BLK_DEV_BSG=y
> > CONFIG_BLK_DEV_BSGLIB=y
> > CONFIG_BLK_DEV_INTEGRITY=y
> > # CONFIG_BLK_DEV_ZONED is not set
> > # CONFIG_BLK_DEV_THROTTLING is not set
> > CONFIG_BLK_CMDLINE_PARSER=y
> > CONFIG_BLK_WBT=y
> > # CONFIG_BLK_WBT_SQ is not set
> > # CONFIG_BLK_WBT_MQ is not set
> > CONFIG_BLK_DEBUG_FS=y
> > # CONFIG_BLK_SED_OPAL is not set
> > 
> > #
> > # Partition Types
> > #
> > # CONFIG_PARTITION_ADVANCED is not set
> > CONFIG_MSDOS_PARTITION=y
> > CONFIG_EFI_PARTITION=y
> > CONFIG_BLK_MQ_PCI=y
> > 
> > #
> > # IO Schedulers
> > #
> > CONFIG_IOSCHED_NOOP=y
> > CONFIG_IOSCHED_DEADLINE=m
> > CONFIG_IOSCHED_CFQ=m
> > CONFIG_CFQ_GROUP_IOSCHED=y
> > CONFIG_DEFAULT_NOOP=y
> > CONFIG_DEFAULT_IOSCHED="noop"
> > CONFIG_MQ_IOSCHED_DEADLINE=y
> > CONFIG_ASN1=y
> > CONFIG_UNINLINE_SPIN_UNLOCK=y
> > CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
> > CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
> > CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
> > # CONFIG_FREEZER is not set
> > 
> > #
> > # Processor type and features
> > #
> > CONFIG_ZONE_DMA=y
> > # CONFIG_SMP is not set
> > CONFIG_X86_FEATURE_NAMES=y
> > # CONFIG_X86_FAST_FEATURE_TESTS is not set
> > # CONFIG_GOLDFISH is not set
> > CONFIG_INTEL_RDT_A=y
> > CONFIG_X86_EXTENDED_PLATFORM=y
> > # CONFIG_X86_GOLDFISH is not set
> > # CONFIG_X86_INTEL_LPSS is not set
> > # CONFIG_X86_AMD_PLATFORM_DEVICE is not set
> > CONFIG_IOSF_MBI=m
> > CONFIG_IOSF_MBI_DEBUG=y
> > # CONFIG_X86_RDC321X is not set
> > # CONFIG_X86_32_IRIS is not set
> > # CONFIG_SCHED_OMIT_FRAME_POINTER is not set
> > CONFIG_HYPERVISOR_GUEST=y
> > CONFIG_PARAVIRT=y
> > # CONFIG_PARAVIRT_DEBUG is not set
> > CONFIG_KVM_GUEST=y
> > # CONFIG_KVM_DEBUG_FS is not set
> > # CONFIG_LGUEST_GUEST is not set
> > # CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
> > CONFIG_PARAVIRT_CLOCK=y
> > CONFIG_NO_BOOTMEM=y
> > # CONFIG_M486 is not set
> > # CONFIG_M586 is not set
> > # CONFIG_M586TSC is not set
> > CONFIG_M586MMX=y
> > # CONFIG_M686 is not set
> > # CONFIG_MPENTIUMII is not set
> > # CONFIG_MPENTIUMIII is not set
> > # CONFIG_MPENTIUMM is not set
> > # CONFIG_MPENTIUM4 is not set
> > # CONFIG_MK6 is not set
> > # CONFIG_MK7 is not set
> > # CONFIG_MK8 is not set
> > # CONFIG_MCRUSOE is not set
> > # CONFIG_MEFFICEON is not set
> > # CONFIG_MWINCHIPC6 is not set
> > # CONFIG_MWINCHIP3D is not set
> > # CONFIG_MELAN is not set
> > # CONFIG_MGEODEGX1 is not set
> > # CONFIG_MGEODE_LX is not set
> > # CONFIG_MCYRIXIII is not set
> > # CONFIG_MVIAC3_2 is not set
> > # CONFIG_MVIAC7 is not set
> > # CONFIG_MCORE2 is not set
> > # CONFIG_MATOM is not set
> > CONFIG_X86_GENERIC=y
> > CONFIG_X86_INTERNODE_CACHE_SHIFT=6
> > CONFIG_X86_L1_CACHE_SHIFT=6
> > CONFIG_X86_PPRO_FENCE=y
> > CONFIG_X86_F00F_BUG=y
> > CONFIG_X86_ALIGNMENT_16=y
> > CONFIG_X86_INTEL_USERCOPY=y
> > CONFIG_X86_TSC=y
> > CONFIG_X86_CMPXCHG64=y
> > CONFIG_X86_MINIMUM_CPU_FAMILY=5
> > # CONFIG_PROCESSOR_SELECT is not set
> > CONFIG_CPU_SUP_INTEL=y
> > CONFIG_CPU_SUP_CYRIX_32=y
> > CONFIG_CPU_SUP_AMD=y
> > CONFIG_CPU_SUP_CENTAUR=y
> > CONFIG_CPU_SUP_TRANSMETA_32=y
> > CONFIG_CPU_SUP_UMC_32=y
> > CONFIG_HPET_TIMER=y
> > CONFIG_DMI=y
> > CONFIG_SWIOTLB=y
> > CONFIG_IOMMU_HELPER=y
> > CONFIG_NR_CPUS=1
> > # CONFIG_PREEMPT_NONE is not set
> > # CONFIG_PREEMPT_VOLUNTARY is not set
> > CONFIG_PREEMPT=y
> > CONFIG_PREEMPT_COUNT=y
> > # CONFIG_X86_UP_APIC is not set
> > # CONFIG_X86_MCE is not set
> > 
> > #
> > # Performance monitoring
> > #
> > CONFIG_PERF_EVENTS_INTEL_UNCORE=y
> > CONFIG_PERF_EVENTS_INTEL_RAPL=y
> > CONFIG_PERF_EVENTS_INTEL_CSTATE=y
> > CONFIG_PERF_EVENTS_AMD_POWER=y
> > # CONFIG_X86_LEGACY_VM86 is not set
> > # CONFIG_VM86 is not set
> > CONFIG_X86_16BIT=y
> > CONFIG_X86_ESPFIX32=y
> > CONFIG_TOSHIBA=m
> > CONFIG_I8K=m
> > # CONFIG_X86_REBOOTFIXUPS is not set
> > CONFIG_MICROCODE=y
> > # CONFIG_MICROCODE_INTEL is not set
> > # CONFIG_MICROCODE_AMD is not set
> > CONFIG_MICROCODE_OLD_INTERFACE=y
> > CONFIG_X86_MSR=m
> > CONFIG_X86_CPUID=m
> > # CONFIG_NOHIGHMEM is not set
> > # CONFIG_HIGHMEM4G is not set
> > CONFIG_HIGHMEM64G=y
> > # CONFIG_VMSPLIT_3G is not set
> > CONFIG_VMSPLIT_2G=y
> > # CONFIG_VMSPLIT_1G is not set
> > CONFIG_PAGE_OFFSET=0x80000000
> > CONFIG_HIGHMEM=y
> > CONFIG_X86_PAE=y
> > CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
> > CONFIG_ARCH_DMA_ADDR_T_64BIT=y
> > CONFIG_ARCH_FLATMEM_ENABLE=y
> > CONFIG_ARCH_SPARSEMEM_ENABLE=y
> > CONFIG_ARCH_SELECT_MEMORY_MODEL=y
> > CONFIG_ILLEGAL_POINTER_VALUE=0
> > CONFIG_SELECT_MEMORY_MODEL=y
> > CONFIG_FLATMEM_MANUAL=y
> > # CONFIG_SPARSEMEM_MANUAL is not set
> > CONFIG_FLATMEM=y
> > CONFIG_FLAT_NODE_MEM_MAP=y
> > CONFIG_SPARSEMEM_STATIC=y
> > CONFIG_HAVE_MEMBLOCK=y
> > CONFIG_HAVE_MEMBLOCK_NODE_MAP=y
> > CONFIG_ARCH_DISCARD_MEMBLOCK=y
> > # CONFIG_HAVE_BOOTMEM_INFO_NODE is not set
> > CONFIG_SPLIT_PTLOCK_CPUS=4
> > CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
> > CONFIG_MEMORY_BALLOON=y
> > # CONFIG_BALLOON_COMPACTION is not set
> > CONFIG_COMPACTION=y
> > CONFIG_MIGRATION=y
> > CONFIG_PHYS_ADDR_T_64BIT=y
> > # CONFIG_BOUNCE is not set
> > CONFIG_VIRT_TO_BUS=y
> > # CONFIG_KSM is not set
> > CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
> > CONFIG_TRANSPARENT_HUGEPAGE=y
> > # CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
> > CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
> > CONFIG_TRANSPARENT_HUGE_PAGECACHE=y
> > CONFIG_NEED_PER_CPU_KM=y
> > # CONFIG_CLEANCACHE is not set
> > # CONFIG_CMA is not set
> > # CONFIG_ZPOOL is not set
> > CONFIG_ZBUD=y
> > CONFIG_ZSMALLOC=y
> > CONFIG_PGTABLE_MAPPING=y
> > # CONFIG_ZSMALLOC_STAT is not set
> > CONFIG_GENERIC_EARLY_IOREMAP=y
> > CONFIG_ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT=y
> > CONFIG_IDLE_PAGE_TRACKING=y
> > # CONFIG_X86_PMEM_LEGACY is not set
> > CONFIG_HIGHPTE=y
> > # CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
> > CONFIG_X86_RESERVE_LOW=64
> > # CONFIG_MATH_EMULATION is not set
> > # CONFIG_MTRR is not set
> > # CONFIG_ARCH_RANDOM is not set
> > # CONFIG_X86_SMAP is not set
> > CONFIG_X86_INTEL_MPX=y
> > # CONFIG_EFI is not set
> > # CONFIG_SECCOMP is not set
> > CONFIG_HZ_100=y
> > # CONFIG_HZ_250 is not set
> > # CONFIG_HZ_300 is not set
> > # CONFIG_HZ_1000 is not set
> > CONFIG_HZ=100
> > CONFIG_SCHED_HRTICK=y
> > CONFIG_KEXEC=y
> > # CONFIG_CRASH_DUMP is not set
> > CONFIG_PHYSICAL_START=0x1000000
> > CONFIG_RELOCATABLE=y
> > CONFIG_RANDOMIZE_BASE=y
> > CONFIG_X86_NEED_RELOCS=y
> > CONFIG_PHYSICAL_ALIGN=0x200000
> > # CONFIG_COMPAT_VDSO is not set
> > # CONFIG_CMDLINE_BOOL is not set
> > CONFIG_MODIFY_LDT_SYSCALL=y
> > CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
> > 
> > #
> > # Power management and ACPI options
> > #
> > # CONFIG_SUSPEND is not set
> > CONFIG_PM=y
> > CONFIG_PM_DEBUG=y
> > # CONFIG_PM_ADVANCED_DEBUG is not set
> > CONFIG_PM_CLK=y
> > CONFIG_WQ_POWER_EFFICIENT_DEFAULT=y
> > CONFIG_ACPI=y
> > CONFIG_ACPI_LEGACY_TABLES_LOOKUP=y
> > CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC=y
> > CONFIG_ACPI_SYSTEM_POWER_STATES_SUPPORT=y
> > # CONFIG_ACPI_DEBUGGER is not set
> > # CONFIG_ACPI_PROCFS_POWER is not set
> > CONFIG_ACPI_REV_OVERRIDE_POSSIBLE=y
> > # CONFIG_ACPI_EC_DEBUGFS is not set
> > CONFIG_ACPI_AC=y
> > CONFIG_ACPI_BATTERY=y
> > CONFIG_ACPI_BUTTON=y
> > # CONFIG_ACPI_VIDEO is not set
> > CONFIG_ACPI_FAN=y
> > # CONFIG_ACPI_DOCK is not set
> > CONFIG_ACPI_CPU_FREQ_PSS=y
> > CONFIG_ACPI_PROCESSOR_CSTATE=y
> > CONFIG_ACPI_PROCESSOR_IDLE=y
> > CONFIG_ACPI_PROCESSOR=y
> > # CONFIG_ACPI_PROCESSOR_AGGREGATOR is not set
> > CONFIG_ACPI_THERMAL=y
> > CONFIG_ACPI_CUSTOM_DSDT_FILE=""
> > # CONFIG_ACPI_CUSTOM_DSDT is not set
> > CONFIG_ARCH_HAS_ACPI_TABLE_UPGRADE=y
> > CONFIG_ACPI_TABLE_UPGRADE=y
> > # CONFIG_ACPI_DEBUG is not set
> > # CONFIG_ACPI_PCI_SLOT is not set
> > CONFIG_X86_PM_TIMER=y
> > # CONFIG_ACPI_CONTAINER is not set
> > # CONFIG_ACPI_SBS is not set
> > # CONFIG_ACPI_HED is not set
> > # CONFIG_ACPI_CUSTOM_METHOD is not set
> > # CONFIG_ACPI_REDUCED_HARDWARE_ONLY is not set
> > # CONFIG_ACPI_NFIT is not set
> > CONFIG_HAVE_ACPI_APEI=y
> > CONFIG_HAVE_ACPI_APEI_NMI=y
> > # CONFIG_ACPI_APEI is not set
> > # CONFIG_DPTF_POWER is not set
> > # CONFIG_PMIC_OPREGION is not set
> > # CONFIG_ACPI_CONFIGFS is not set
> > # CONFIG_SFI is not set
> > 
> > #
> > # CPU Frequency scaling
> > #
> > # CONFIG_CPU_FREQ is not set
> > 
> > #
> > # CPU Idle
> > #
> > CONFIG_CPU_IDLE=y
> > CONFIG_CPU_IDLE_GOV_LADDER=y
> > # CONFIG_CPU_IDLE_GOV_MENU is not set
> > # CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED is not set
> > CONFIG_INTEL_IDLE=y
> > 
> > #
> > # Bus options (PCI etc.)
> > #
> > CONFIG_PCI=y
> > # CONFIG_PCI_GOBIOS is not set
> > # CONFIG_PCI_GOMMCONFIG is not set
> > # CONFIG_PCI_GODIRECT is not set
> > CONFIG_PCI_GOANY=y
> > CONFIG_PCI_BIOS=y
> > CONFIG_PCI_DIRECT=y
> > CONFIG_PCI_MMCONFIG=y
> > CONFIG_PCI_DOMAINS=y
> > # CONFIG_PCI_CNB20LE_QUIRK is not set
> > # CONFIG_PCIEPORTBUS is not set
> > CONFIG_PCI_BUS_ADDR_T_64BIT=y
> > # CONFIG_PCI_MSI is not set
> > # CONFIG_PCI_DEBUG is not set
> > # CONFIG_PCI_REALLOC_ENABLE_AUTO is not set
> > # CONFIG_PCI_STUB is not set
> > # CONFIG_PCI_IOV is not set
> > # CONFIG_PCI_PRI is not set
> > # CONFIG_PCI_PASID is not set
> > CONFIG_PCI_LABEL=y
> > # CONFIG_HOTPLUG_PCI is not set
> > 
> > #
> > # DesignWare PCI Core Support
> > #
> > 
> > #
> > # PCI host controller drivers
> > #
> > CONFIG_ISA_BUS=y
> > CONFIG_ISA_DMA_API=y
> > CONFIG_ISA=y
> > # CONFIG_EISA is not set
> > # CONFIG_SCx200 is not set
> > CONFIG_ALIX=y
> > CONFIG_NET5501=y
> > # CONFIG_GEOS is not set
> > CONFIG_AMD_NB=y
> > CONFIG_PCCARD=y
> > CONFIG_PCMCIA=m
> > # CONFIG_PCMCIA_LOAD_CIS is not set
> > CONFIG_CARDBUS=y
> > 
> > #
> > # PC-card bridges
> > #
> > # CONFIG_YENTA is not set
> > # CONFIG_PD6729 is not set
> > # CONFIG_I82092 is not set
> > CONFIG_I82365=m
> > CONFIG_TCIC=m
> > CONFIG_PCMCIA_PROBE=y
> > CONFIG_PCCARD_NONSTATIC=y
> > # CONFIG_RAPIDIO is not set
> > CONFIG_X86_SYSFB=y
> > 
> > #
> > # Executable file formats / Emulations
> > #
> > CONFIG_BINFMT_ELF=y
> > CONFIG_ELFCORE=y
> > CONFIG_BINFMT_SCRIPT=y
> > CONFIG_HAVE_AOUT=y
> > # CONFIG_BINFMT_AOUT is not set
> > CONFIG_BINFMT_MISC=m
> > # CONFIG_COREDUMP is not set
> > CONFIG_COMPAT_32=y
> > CONFIG_HAVE_ATOMIC_IOMAP=y
> > CONFIG_NET=y
> > CONFIG_NET_INGRESS=y
> > 
> > #
> > # Networking options
> > #
> > CONFIG_PACKET=m
> > CONFIG_PACKET_DIAG=m
> > CONFIG_UNIX=y
> > # CONFIG_UNIX_DIAG is not set
> > CONFIG_XFRM=y
> > CONFIG_XFRM_ALGO=y
> > CONFIG_XFRM_USER=y
> > CONFIG_XFRM_SUB_POLICY=y
> > CONFIG_XFRM_MIGRATE=y
> > # CONFIG_XFRM_STATISTICS is not set
> > CONFIG_XFRM_IPCOMP=m
> > CONFIG_NET_KEY=y
> > # CONFIG_NET_KEY_MIGRATE is not set
> > CONFIG_INET=y
> > CONFIG_IP_MULTICAST=y
> > # CONFIG_IP_ADVANCED_ROUTER is not set
> > CONFIG_IP_PNP=y
> > CONFIG_IP_PNP_DHCP=y
> > # CONFIG_IP_PNP_BOOTP is not set
> > # CONFIG_IP_PNP_RARP is not set
> > # CONFIG_NET_IPIP is not set
> > CONFIG_NET_IPGRE_DEMUX=y
> > CONFIG_NET_IP_TUNNEL=y
> > CONFIG_NET_IPGRE=m
> > # CONFIG_NET_IPGRE_BROADCAST is not set
> > # CONFIG_IP_MROUTE is not set
> > # CONFIG_SYN_COOKIES is not set
> > CONFIG_NET_UDP_TUNNEL=y
> > CONFIG_NET_FOU=m
> > CONFIG_NET_FOU_IP_TUNNELS=y
> > # CONFIG_INET_AH is not set
> > CONFIG_INET_ESP=y
> > # CONFIG_INET_ESP_OFFLOAD is not set
> > CONFIG_INET_IPCOMP=m
> > CONFIG_INET_XFRM_TUNNEL=m
> > CONFIG_INET_TUNNEL=m
> > # CONFIG_INET_XFRM_MODE_TRANSPORT is not set
> > # CONFIG_INET_XFRM_MODE_TUNNEL is not set
> > CONFIG_INET_XFRM_MODE_BEET=m
> > CONFIG_INET_DIAG=m
> > CONFIG_INET_TCP_DIAG=m
> > CONFIG_INET_UDP_DIAG=m
> > CONFIG_INET_RAW_DIAG=m
> > CONFIG_INET_DIAG_DESTROY=y
> > CONFIG_TCP_CONG_ADVANCED=y
> > CONFIG_TCP_CONG_BIC=y
> > CONFIG_TCP_CONG_CUBIC=y
> > # CONFIG_TCP_CONG_WESTWOOD is not set
> > # CONFIG_TCP_CONG_HTCP is not set
> > # CONFIG_TCP_CONG_HSTCP is not set
> > CONFIG_TCP_CONG_HYBLA=m
> > CONFIG_TCP_CONG_VEGAS=y
> > CONFIG_TCP_CONG_NV=m
> > CONFIG_TCP_CONG_SCALABLE=m
> > CONFIG_TCP_CONG_LP=y
> > # CONFIG_TCP_CONG_VENO is not set
> > CONFIG_TCP_CONG_YEAH=y
> > CONFIG_TCP_CONG_ILLINOIS=y
> > # CONFIG_TCP_CONG_DCTCP is not set
> > CONFIG_TCP_CONG_CDG=y
> > # CONFIG_TCP_CONG_BBR is not set
> > # CONFIG_DEFAULT_BIC is not set
> > # CONFIG_DEFAULT_CUBIC is not set
> > CONFIG_DEFAULT_VEGAS=y
> > # CONFIG_DEFAULT_CDG is not set
> > # CONFIG_DEFAULT_RENO is not set
> > CONFIG_DEFAULT_TCP_CONG="vegas"
> > # CONFIG_TCP_MD5SIG is not set
> > CONFIG_IPV6=y
> > # CONFIG_IPV6_ROUTER_PREF is not set
> > # CONFIG_IPV6_OPTIMISTIC_DAD is not set
> > CONFIG_INET6_AH=y
> > CONFIG_INET6_ESP=y
> > # CONFIG_INET6_ESP_OFFLOAD is not set
> > # CONFIG_INET6_IPCOMP is not set
> > # CONFIG_IPV6_MIP6 is not set
> > # CONFIG_IPV6_ILA is not set
> > # CONFIG_INET6_XFRM_TUNNEL is not set
> > CONFIG_INET6_TUNNEL=y
> > # CONFIG_INET6_XFRM_MODE_TRANSPORT is not set
> > CONFIG_INET6_XFRM_MODE_TUNNEL=m
> > CONFIG_INET6_XFRM_MODE_BEET=m
> > CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION=m
> > CONFIG_IPV6_VTI=m
> > # CONFIG_IPV6_SIT is not set
> > CONFIG_IPV6_TUNNEL=y
> > CONFIG_IPV6_GRE=m
> > CONFIG_IPV6_FOU=m
> > CONFIG_IPV6_FOU_TUNNEL=m
> > CONFIG_IPV6_MULTIPLE_TABLES=y
> > CONFIG_IPV6_SUBTREES=y
> > # CONFIG_IPV6_MROUTE is not set
> > CONFIG_IPV6_SEG6_LWTUNNEL=y
> > # CONFIG_IPV6_SEG6_INLINE is not set
> > CONFIG_IPV6_SEG6_HMAC=y
> > CONFIG_NETWORK_SECMARK=y
> > CONFIG_NET_PTP_CLASSIFY=y
> > CONFIG_NETWORK_PHY_TIMESTAMPING=y
> > CONFIG_NETFILTER=y
> > CONFIG_NETFILTER_DEBUG=y
> > # CONFIG_NETFILTER_ADVANCED is not set
> > 
> > #
> > # Core Netfilter Configuration
> > #
> > CONFIG_NETFILTER_INGRESS=y
> > CONFIG_NETFILTER_NETLINK=y
> > CONFIG_NETFILTER_NETLINK_LOG=y
> > CONFIG_NF_CONNTRACK=m
> > CONFIG_NF_LOG_COMMON=y
> > # CONFIG_NF_LOG_NETDEV is not set
> > # CONFIG_NF_CONNTRACK_SECMARK is not set
> > CONFIG_NF_CONNTRACK_PROCFS=y
> > # CONFIG_NF_CONNTRACK_FTP is not set
> > CONFIG_NF_CONNTRACK_IRC=m
> > CONFIG_NF_CONNTRACK_BROADCAST=m
> > CONFIG_NF_CONNTRACK_NETBIOS_NS=m
> > # CONFIG_NF_CONNTRACK_SIP is not set
> > CONFIG_NF_CT_NETLINK=m
> > CONFIG_NETFILTER_NETLINK_GLUE_CT=y
> > CONFIG_NF_TABLES=m
> > CONFIG_NF_TABLES_INET=m
> > CONFIG_NF_TABLES_NETDEV=m
> > CONFIG_NFT_EXTHDR=m
> > CONFIG_NFT_META=m
> > CONFIG_NFT_RT=m
> > CONFIG_NFT_NUMGEN=m
> > CONFIG_NFT_CT=m
> > # CONFIG_NFT_SET_RBTREE is not set
> > CONFIG_NFT_SET_HASH=m
> > # CONFIG_NFT_SET_BITMAP is not set
> > CONFIG_NFT_COUNTER=m
> > CONFIG_NFT_LOG=m
> > CONFIG_NFT_LIMIT=m
> > # CONFIG_NFT_NAT is not set
> > CONFIG_NFT_OBJREF=m
> > CONFIG_NFT_QUOTA=m
> > CONFIG_NFT_REJECT=m
> > CONFIG_NFT_REJECT_INET=m
> > # CONFIG_NFT_COMPAT is not set
> > CONFIG_NFT_HASH=m
> > CONFIG_NFT_FIB=m
> > CONFIG_NFT_FIB_INET=m
> > CONFIG_NF_DUP_NETDEV=m
> > CONFIG_NFT_DUP_NETDEV=m
> > CONFIG_NFT_FWD_NETDEV=m
> > CONFIG_NETFILTER_XTABLES=m
> > 
> > #
> > # Xtables combined modules
> > #
> > # CONFIG_NETFILTER_XT_MARK is not set
> > 
> > #
> > # Xtables targets
> > #
> > CONFIG_NETFILTER_XT_TARGET_LOG=m
> > CONFIG_NETFILTER_XT_TARGET_NFLOG=m
> > CONFIG_NETFILTER_XT_TARGET_SECMARK=m
> > # CONFIG_NETFILTER_XT_TARGET_TCPMSS is not set
> > 
> > #
> > # Xtables matches
> > #
> > # CONFIG_NETFILTER_XT_MATCH_ADDRTYPE is not set
> > CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
> > CONFIG_NETFILTER_XT_MATCH_POLICY=m
> > CONFIG_NETFILTER_XT_MATCH_STATE=m
> > CONFIG_IP_SET=y
> > CONFIG_IP_SET_MAX=256
> > CONFIG_IP_SET_BITMAP_IP=m
> > CONFIG_IP_SET_BITMAP_IPMAC=m
> > CONFIG_IP_SET_BITMAP_PORT=m
> > CONFIG_IP_SET_HASH_IP=y
> > CONFIG_IP_SET_HASH_IPMARK=m
> > CONFIG_IP_SET_HASH_IPPORT=m
> > CONFIG_IP_SET_HASH_IPPORTIP=y
> > CONFIG_IP_SET_HASH_IPPORTNET=m
> > CONFIG_IP_SET_HASH_IPMAC=y
> > # CONFIG_IP_SET_HASH_MAC is not set
> > # CONFIG_IP_SET_HASH_NETPORTNET is not set
> > CONFIG_IP_SET_HASH_NET=y
> > CONFIG_IP_SET_HASH_NETNET=y
> > CONFIG_IP_SET_HASH_NETPORT=m
> > CONFIG_IP_SET_HASH_NETIFACE=y
> > # CONFIG_IP_SET_LIST_SET is not set
> > CONFIG_IP_VS=m
> > CONFIG_IP_VS_IPV6=y
> > CONFIG_IP_VS_DEBUG=y
> > CONFIG_IP_VS_TAB_BITS=12
> > 
> > #
> > # IPVS transport protocol load balancing support
> > #
> > CONFIG_IP_VS_PROTO_TCP=y
> > # CONFIG_IP_VS_PROTO_UDP is not set
> > # CONFIG_IP_VS_PROTO_AH_ESP is not set
> > # CONFIG_IP_VS_PROTO_ESP is not set
> > # CONFIG_IP_VS_PROTO_AH is not set
> > # CONFIG_IP_VS_PROTO_SCTP is not set
> > 
> > #
> > # IPVS scheduler
> > #
> > # CONFIG_IP_VS_RR is not set
> > CONFIG_IP_VS_WRR=m
> > CONFIG_IP_VS_LC=m
> > CONFIG_IP_VS_WLC=m
> > CONFIG_IP_VS_FO=m
> > CONFIG_IP_VS_OVF=m
> > # CONFIG_IP_VS_LBLC is not set
> > CONFIG_IP_VS_LBLCR=m
> > # CONFIG_IP_VS_DH is not set
> > CONFIG_IP_VS_SH=m
> > CONFIG_IP_VS_SED=m
> > # CONFIG_IP_VS_NQ is not set
> > 
> > #
> > # IPVS SH scheduler
> > #
> > CONFIG_IP_VS_SH_TAB_BITS=8
> > 
> > #
> > # IPVS application helper
> > #
> > CONFIG_IP_VS_NFCT=y
> > 
> > #
> > # IP: Netfilter Configuration
> > #
> > # CONFIG_NF_DEFRAG_IPV4 is not set
> > # CONFIG_NF_CONNTRACK_IPV4 is not set
> > CONFIG_NF_SOCKET_IPV4=y
> > CONFIG_NF_TABLES_IPV4=m
> > # CONFIG_NFT_CHAIN_ROUTE_IPV4 is not set
> > CONFIG_NFT_REJECT_IPV4=m
> > CONFIG_NFT_DUP_IPV4=m
> > CONFIG_NFT_FIB_IPV4=m
> > CONFIG_NF_TABLES_ARP=m
> > CONFIG_NF_DUP_IPV4=m
> > CONFIG_NF_LOG_ARP=y
> > CONFIG_NF_LOG_IPV4=m
> > CONFIG_NF_REJECT_IPV4=m
> > # CONFIG_IP_NF_IPTABLES is not set
> > 
> > #
> > # IPv6: Netfilter Configuration
> > #
> > # CONFIG_NF_DEFRAG_IPV6 is not set
> > # CONFIG_NF_CONNTRACK_IPV6 is not set
> > CONFIG_NF_SOCKET_IPV6=m
> > CONFIG_NF_TABLES_IPV6=m
> > CONFIG_NFT_CHAIN_ROUTE_IPV6=m
> > CONFIG_NFT_REJECT_IPV6=m
> > CONFIG_NFT_DUP_IPV6=m
> > CONFIG_NFT_FIB_IPV6=m
> > CONFIG_NF_DUP_IPV6=m
> > CONFIG_NF_REJECT_IPV6=m
> > CONFIG_NF_LOG_IPV6=m
> > CONFIG_IP6_NF_IPTABLES=m
> > # CONFIG_IP6_NF_MATCH_IPV6HEADER is not set
> > CONFIG_IP6_NF_FILTER=m
> > # CONFIG_IP6_NF_TARGET_REJECT is not set
> > CONFIG_IP6_NF_MANGLE=m
> > CONFIG_IP6_NF_RAW=m
> > CONFIG_IP_DCCP=y
> > CONFIG_INET_DCCP_DIAG=m
> > 
> > #
> > # DCCP CCIDs Configuration
> > #
> > # CONFIG_IP_DCCP_CCID2_DEBUG is not set
> > # CONFIG_IP_DCCP_CCID3 is not set
> > 
> > #
> > # DCCP Kernel Hacking
> > #
> > CONFIG_IP_DCCP_DEBUG=y
> > # CONFIG_NET_DCCPPROBE is not set
> > CONFIG_IP_SCTP=y
> > # CONFIG_NET_SCTPPROBE is not set
> > # CONFIG_SCTP_DBG_OBJCNT is not set
> > # CONFIG_SCTP_DEFAULT_COOKIE_HMAC_MD5 is not set
> > # CONFIG_SCTP_DEFAULT_COOKIE_HMAC_SHA1 is not set
> > CONFIG_SCTP_DEFAULT_COOKIE_HMAC_NONE=y
> > CONFIG_SCTP_COOKIE_HMAC_MD5=y
> > # CONFIG_SCTP_COOKIE_HMAC_SHA1 is not set
> > CONFIG_INET_SCTP_DIAG=m
> > # CONFIG_RDS is not set
> > # CONFIG_TIPC is not set
> > # CONFIG_ATM is not set
> > CONFIG_L2TP=y
> > # CONFIG_L2TP_DEBUGFS is not set
> > CONFIG_L2TP_V3=y
> > CONFIG_L2TP_IP=y
> > CONFIG_L2TP_ETH=y
> > CONFIG_MRP=y
> > # CONFIG_BRIDGE is not set
> > CONFIG_VLAN_8021Q=y
> > # CONFIG_VLAN_8021Q_GVRP is not set
> > CONFIG_VLAN_8021Q_MVRP=y
> > CONFIG_DECNET=y
> > # CONFIG_DECNET_ROUTER is not set
> > CONFIG_LLC=y
> > CONFIG_LLC2=m
> > CONFIG_IPX=y
> > # CONFIG_IPX_INTERN is not set
> > # CONFIG_ATALK is not set
> > # CONFIG_X25 is not set
> > CONFIG_LAPB=y
> > CONFIG_PHONET=m
> > CONFIG_6LOWPAN=m
> > CONFIG_6LOWPAN_DEBUGFS=y
> > # CONFIG_6LOWPAN_NHC is not set
> > CONFIG_IEEE802154=y
> > CONFIG_IEEE802154_NL802154_EXPERIMENTAL=y
> > # CONFIG_IEEE802154_SOCKET is not set
> > CONFIG_IEEE802154_6LOWPAN=m
> > # CONFIG_MAC802154 is not set
> > CONFIG_NET_SCHED=y
> > 
> > #
> > # Queueing/Scheduling
> > #
> > CONFIG_NET_SCH_CBQ=y
> > CONFIG_NET_SCH_HTB=m
> > CONFIG_NET_SCH_HFSC=m
> > CONFIG_NET_SCH_PRIO=m
> > CONFIG_NET_SCH_MULTIQ=m
> > CONFIG_NET_SCH_RED=y
> > CONFIG_NET_SCH_SFB=y
> > CONFIG_NET_SCH_SFQ=m
> > # CONFIG_NET_SCH_TEQL is not set
> > # CONFIG_NET_SCH_TBF is not set
> > # CONFIG_NET_SCH_GRED is not set
> > CONFIG_NET_SCH_DSMARK=m
> > CONFIG_NET_SCH_NETEM=y
> > # CONFIG_NET_SCH_DRR is not set
> > # CONFIG_NET_SCH_MQPRIO is not set
> > CONFIG_NET_SCH_CHOKE=m
> > CONFIG_NET_SCH_QFQ=y
> > CONFIG_NET_SCH_CODEL=y
> > CONFIG_NET_SCH_FQ_CODEL=m
> > CONFIG_NET_SCH_FQ=y
> > CONFIG_NET_SCH_HHF=y
> > CONFIG_NET_SCH_PIE=y
> > CONFIG_NET_SCH_PLUG=y
> > 
> > #
> > # Classification
> > #
> > CONFIG_NET_CLS=y
> > CONFIG_NET_CLS_BASIC=y
> > CONFIG_NET_CLS_TCINDEX=m
> > # CONFIG_NET_CLS_ROUTE4 is not set
> > # CONFIG_NET_CLS_FW is not set
> > # CONFIG_NET_CLS_U32 is not set
> > CONFIG_NET_CLS_RSVP=y
> > CONFIG_NET_CLS_RSVP6=m
> > CONFIG_NET_CLS_FLOW=y
> > CONFIG_NET_CLS_CGROUP=y
> > CONFIG_NET_CLS_BPF=y
> > CONFIG_NET_CLS_FLOWER=y
> > CONFIG_NET_CLS_MATCHALL=y
> > # CONFIG_NET_EMATCH is not set
> > # CONFIG_NET_CLS_ACT is not set
> > CONFIG_NET_SCH_FIFO=y
> > CONFIG_DCB=y
> > CONFIG_DNS_RESOLVER=y
> > # CONFIG_BATMAN_ADV is not set
> > # CONFIG_OPENVSWITCH is not set
> > CONFIG_VSOCKETS=m
> > CONFIG_VIRTIO_VSOCKETS=m
> > CONFIG_VIRTIO_VSOCKETS_COMMON=m
> > CONFIG_NETLINK_DIAG=m
> > CONFIG_MPLS=y
> > # CONFIG_NET_MPLS_GSO is not set
> > CONFIG_MPLS_ROUTING=m
> > CONFIG_MPLS_IPTUNNEL=m
> > # CONFIG_HSR is not set
> > CONFIG_NET_SWITCHDEV=y
> > # CONFIG_NET_L3_MASTER_DEV is not set
> > # CONFIG_NET_NCSI is not set
> > CONFIG_CGROUP_NET_PRIO=y
> > CONFIG_CGROUP_NET_CLASSID=y
> > CONFIG_NET_RX_BUSY_POLL=y
> > CONFIG_BQL=y
> > 
> > #
> > # Network testing
> > #
> > # CONFIG_NET_PKTGEN is not set
> > # CONFIG_NET_TCPPROBE is not set
> > CONFIG_HAMRADIO=y
> > 
> > #
> > # Packet Radio protocols
> > #
> > CONFIG_AX25=m
> > # CONFIG_AX25_DAMA_SLAVE is not set
> > CONFIG_NETROM=m
> > # CONFIG_ROSE is not set
> > 
> > #
> > # AX.25 network device drivers
> > #
> > # CONFIG_MKISS is not set
> > # CONFIG_6PACK is not set
> > # CONFIG_BPQETHER is not set
> > CONFIG_DMASCC=m
> > CONFIG_SCC=m
> > # CONFIG_SCC_DELAY is not set
> > CONFIG_SCC_TRXECHO=y
> > # CONFIG_BAYCOM_SER_FDX is not set
> > # CONFIG_BAYCOM_SER_HDX is not set
> > CONFIG_BAYCOM_PAR=m
> > CONFIG_BAYCOM_EPP=m
> > # CONFIG_YAM is not set
> > CONFIG_CAN=y
> > CONFIG_CAN_RAW=m
> > CONFIG_CAN_BCM=m
> > # CONFIG_CAN_GW is not set
> > 
> > #
> > # CAN Device Drivers
> > #
> > CONFIG_CAN_VCAN=y
> > # CONFIG_CAN_SLCAN is not set
> > # CONFIG_CAN_DEV is not set
> > # CONFIG_CAN_DEBUG_DEVICES is not set
> > CONFIG_IRDA=y
> > 
> > #
> > # IrDA protocols
> > #
> > CONFIG_IRLAN=m
> > # CONFIG_IRCOMM is not set
> > CONFIG_IRDA_ULTRA=y
> > 
> > #
> > # IrDA options
> > #
> > # CONFIG_IRDA_CACHE_LAST_LSAP is not set
> > # CONFIG_IRDA_FAST_RR is not set
> > CONFIG_IRDA_DEBUG=y
> > 
> > #
> > # Infrared-port device drivers
> > #
> > 
> > #
> > # SIR device drivers
> > #
> > # CONFIG_IRTTY_SIR is not set
> > 
> > #
> > # Dongle support
> > #
> > CONFIG_KINGSUN_DONGLE=m
> > CONFIG_KSDAZZLE_DONGLE=y
> > CONFIG_KS959_DONGLE=y
> > 
> > #
> > # FIR device drivers
> > #
> > CONFIG_USB_IRDA=y
> > # CONFIG_SIGMATEL_FIR is not set
> > CONFIG_NSC_FIR=y
> > CONFIG_WINBOND_FIR=y
> > # CONFIG_TOSHIBA_FIR is not set
> > CONFIG_SMC_IRCC_FIR=y
> > CONFIG_ALI_FIR=y
> > # CONFIG_VLSI_FIR is not set
> > CONFIG_VIA_FIR=y
> > CONFIG_MCS_FIR=y
> > CONFIG_BT=y
> > # CONFIG_BT_BREDR is not set
> > # CONFIG_BT_LE is not set
> > CONFIG_BT_LEDS=y
> > # CONFIG_BT_SELFTEST is not set
> > CONFIG_BT_DEBUGFS=y
> > 
> > #
> > # Bluetooth device drivers
> > #
> > # CONFIG_BT_HCIBTUSB is not set
> > # CONFIG_BT_HCIBTSDIO is not set
> > # CONFIG_BT_HCIUART is not set
> > # CONFIG_BT_HCIBCM203X is not set
> > # CONFIG_BT_HCIBFUSB is not set
> > CONFIG_BT_HCIDTL1=m
> > CONFIG_BT_HCIBT3C=m
> > CONFIG_BT_HCIBLUECARD=m
> > CONFIG_BT_HCIBTUART=m
> > # CONFIG_BT_HCIVHCI is not set
> > CONFIG_BT_MRVL=m
> > CONFIG_BT_MRVL_SDIO=m
> > CONFIG_AF_RXRPC=m
> > CONFIG_AF_RXRPC_IPV6=y
> > # CONFIG_AF_RXRPC_INJECT_LOSS is not set
> > CONFIG_AF_RXRPC_DEBUG=y
> > # CONFIG_RXKAD is not set
> > # CONFIG_AF_KCM is not set
> > # CONFIG_STREAM_PARSER is not set
> > CONFIG_FIB_RULES=y
> > # CONFIG_WIRELESS is not set
> > CONFIG_WIMAX=m
> > CONFIG_WIMAX_DEBUG_LEVEL=8
> > CONFIG_RFKILL=y
> > CONFIG_RFKILL_LEDS=y
> > # CONFIG_RFKILL_INPUT is not set
> > # CONFIG_RFKILL_GPIO is not set
> > CONFIG_NET_9P=m
> > # CONFIG_NET_9P_VIRTIO is not set
> > CONFIG_NET_9P_DEBUG=y
> > CONFIG_CAIF=m
> > CONFIG_CAIF_DEBUG=y
> > CONFIG_CAIF_NETDEV=m
> > CONFIG_CAIF_USB=m
> > CONFIG_CEPH_LIB=y
> > # CONFIG_CEPH_LIB_PRETTYDEBUG is not set
> > CONFIG_CEPH_LIB_USE_DNS_RESOLVER=y
> > # CONFIG_NFC is not set
> > # CONFIG_PSAMPLE is not set
> > # CONFIG_NET_IFE is not set
> > CONFIG_LWTUNNEL=y
> > # CONFIG_LWTUNNEL_BPF is not set
> > CONFIG_DST_CACHE=y
> > CONFIG_GRO_CELLS=y
> > # CONFIG_NET_DEVLINK is not set
> > CONFIG_MAY_USE_DEVLINK=y
> > 
> > #
> > # Device Drivers
> > #
> > 
> > #
> > # Generic Driver Options
> > #
> > # CONFIG_UEVENT_HELPER is not set
> > CONFIG_DEVTMPFS=y
> > # CONFIG_DEVTMPFS_MOUNT is not set
> > # CONFIG_STANDALONE is not set
> > # CONFIG_PREVENT_FIRMWARE_BUILD is not set
> > CONFIG_FW_LOADER=y
> > # CONFIG_FIRMWARE_IN_KERNEL is not set
> > CONFIG_EXTRA_FIRMWARE=""
> > CONFIG_FW_LOADER_USER_HELPER=y
> > # CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set
> > CONFIG_WANT_DEV_COREDUMP=y
> > CONFIG_ALLOW_DEV_COREDUMP=y
> > CONFIG_DEV_COREDUMP=y
> > # CONFIG_DEBUG_DRIVER is not set
> > CONFIG_DEBUG_DEVRES=y
> > CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> > # CONFIG_TEST_ASYNC_DRIVER_PROBE is not set
> > # CONFIG_SYS_HYPERVISOR is not set
> > # CONFIG_GENERIC_CPU_DEVICES is not set
> > CONFIG_GENERIC_CPU_AUTOPROBE=y
> > CONFIG_REGMAP=y
> > CONFIG_REGMAP_I2C=y
> > CONFIG_REGMAP_SPMI=m
> > CONFIG_REGMAP_MMIO=y
> > CONFIG_REGMAP_IRQ=y
> > CONFIG_DMA_SHARED_BUFFER=y
> > # CONFIG_DMA_FENCE_TRACE is not set
> > 
> > #
> > # Bus devices
> > #
> > CONFIG_CONNECTOR=y
> > CONFIG_PROC_EVENTS=y
> > CONFIG_MTD=m
> > # CONFIG_MTD_TESTS is not set
> > CONFIG_MTD_REDBOOT_PARTS=m
> > CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-1
> > CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED=y
> > CONFIG_MTD_REDBOOT_PARTS_READONLY=y
> > CONFIG_MTD_CMDLINE_PARTS=m
> > CONFIG_MTD_AR7_PARTS=m
> > 
> > #
> > # User Modules And Translation Layers
> > #
> > CONFIG_MTD_BLKDEVS=m
> > CONFIG_MTD_BLOCK=m
> > CONFIG_MTD_BLOCK_RO=m
> > # CONFIG_FTL is not set
> > CONFIG_NFTL=m
> > # CONFIG_NFTL_RW is not set
> > CONFIG_INFTL=m
> > # CONFIG_RFD_FTL is not set
> > CONFIG_SSFDC=m
> > CONFIG_SM_FTL=m
> > # CONFIG_MTD_OOPS is not set
> > CONFIG_MTD_PARTITIONED_MASTER=y
> > 
> > #
> > # RAM/ROM/Flash chip drivers
> > #
> > CONFIG_MTD_CFI=m
> > CONFIG_MTD_JEDECPROBE=m
> > CONFIG_MTD_GEN_PROBE=m
> > CONFIG_MTD_CFI_ADV_OPTIONS=y
> > CONFIG_MTD_CFI_NOSWAP=y
> > # CONFIG_MTD_CFI_BE_BYTE_SWAP is not set
> > # CONFIG_MTD_CFI_LE_BYTE_SWAP is not set
> > # CONFIG_MTD_CFI_GEOMETRY is not set
> > CONFIG_MTD_MAP_BANK_WIDTH_1=y
> > CONFIG_MTD_MAP_BANK_WIDTH_2=y
> > CONFIG_MTD_MAP_BANK_WIDTH_4=y
> > # CONFIG_MTD_MAP_BANK_WIDTH_8 is not set
> > # CONFIG_MTD_MAP_BANK_WIDTH_16 is not set
> > # CONFIG_MTD_MAP_BANK_WIDTH_32 is not set
> > CONFIG_MTD_CFI_I1=y
> > CONFIG_MTD_CFI_I2=y
> > # CONFIG_MTD_CFI_I4 is not set
> > # CONFIG_MTD_CFI_I8 is not set
> > CONFIG_MTD_OTP=y
> > # CONFIG_MTD_CFI_INTELEXT is not set
> > # CONFIG_MTD_CFI_AMDSTD is not set
> > CONFIG_MTD_CFI_STAA=m
> > CONFIG_MTD_CFI_UTIL=m
> > CONFIG_MTD_RAM=m
> > CONFIG_MTD_ROM=m
> > # CONFIG_MTD_ABSENT is not set
> > 
> > #
> > # Mapping drivers for chip access
> > #
> > # CONFIG_MTD_COMPLEX_MAPPINGS is not set
> > # CONFIG_MTD_PHYSMAP is not set
> > CONFIG_MTD_AMD76XROM=m
> > CONFIG_MTD_ICHXROM=m
> > # CONFIG_MTD_ESB2ROM is not set
> > # CONFIG_MTD_CK804XROM is not set
> > # CONFIG_MTD_SCB2_FLASH is not set
> > # CONFIG_MTD_NETtel is not set
> > CONFIG_MTD_L440GX=m
> > # CONFIG_MTD_INTEL_VR_NOR is not set
> > # CONFIG_MTD_PLATRAM is not set
> > 
> > #
> > # Self-contained MTD device drivers
> > #
> > # CONFIG_MTD_PMC551 is not set
> > CONFIG_MTD_SLRAM=m
> > CONFIG_MTD_PHRAM=m
> > CONFIG_MTD_MTDRAM=m
> > CONFIG_MTDRAM_TOTAL_SIZE=4096
> > CONFIG_MTDRAM_ERASE_SIZE=128
> > CONFIG_MTD_BLOCK2MTD=m
> > 
> > #
> > # Disk-On-Chip Device Drivers
> > #
> > # CONFIG_MTD_DOCG3 is not set
> > CONFIG_MTD_NAND_ECC=m
> > # CONFIG_MTD_NAND_ECC_SMC is not set
> > # CONFIG_MTD_NAND is not set
> > CONFIG_MTD_ONENAND=m
> > CONFIG_MTD_ONENAND_VERIFY_WRITE=y
> > # CONFIG_MTD_ONENAND_GENERIC is not set
> > # CONFIG_MTD_ONENAND_OTP is not set
> > CONFIG_MTD_ONENAND_2X_PROGRAM=y
> > 
> > #
> > # LPDDR & LPDDR2 PCM memory drivers
> > #
> > CONFIG_MTD_LPDDR=m
> > CONFIG_MTD_QINFO_PROBE=m
> > # CONFIG_MTD_SPI_NOR is not set
> > # CONFIG_MTD_UBI is not set
> > # CONFIG_OF is not set
> > CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
> > CONFIG_PARPORT=m
> > # CONFIG_PARPORT_PC is not set
> > # CONFIG_PARPORT_GSC is not set
> > CONFIG_PARPORT_AX88796=m
> > CONFIG_PARPORT_1284=y
> > CONFIG_PARPORT_NOT_PC=y
> > CONFIG_PNP=y
> > # CONFIG_PNP_DEBUG_MESSAGES is not set
> > 
> > #
> > # Protocols
> > #
> > # CONFIG_ISAPNP is not set
> > # CONFIG_PNPBIOS is not set
> > CONFIG_PNPACPI=y
> > CONFIG_BLK_DEV=y
> > # CONFIG_BLK_DEV_NULL_BLK is not set
> > # CONFIG_BLK_DEV_FD is not set
> > # CONFIG_BLK_DEV_PCIESSD_MTIP32XX is not set
> > # CONFIG_ZRAM is not set
> > # CONFIG_BLK_CPQ_CISS_DA is not set
> > # CONFIG_BLK_DEV_DAC960 is not set
> > # CONFIG_BLK_DEV_UMEM is not set
> > # CONFIG_BLK_DEV_COW_COMMON is not set
> > # CONFIG_BLK_DEV_LOOP is not set
> > # CONFIG_BLK_DEV_DRBD is not set
> > # CONFIG_BLK_DEV_NBD is not set
> > # CONFIG_BLK_DEV_SX8 is not set
> > # CONFIG_BLK_DEV_RAM is not set
> > # CONFIG_CDROM_PKTCDVD is not set
> > # CONFIG_ATA_OVER_ETH is not set
> > # CONFIG_VIRTIO_BLK is not set
> > # CONFIG_BLK_DEV_HD is not set
> > # CONFIG_BLK_DEV_RBD is not set
> > # CONFIG_BLK_DEV_RSXX is not set
> > CONFIG_NVME_CORE=y
> > # CONFIG_BLK_DEV_NVME is not set
> > CONFIG_BLK_DEV_NVME_SCSI=y
> > CONFIG_NVME_FABRICS=y
> > CONFIG_NVME_FC=y
> > CONFIG_NVME_TARGET=m
> > CONFIG_NVME_TARGET_LOOP=m
> > # CONFIG_NVME_TARGET_FC is not set
> > 
> > #
> > # Misc devices
> > #
> > # CONFIG_SENSORS_LIS3LV02D is not set
> > CONFIG_AD525X_DPOT=m
> > CONFIG_AD525X_DPOT_I2C=m
> > CONFIG_DUMMY_IRQ=m
> > # CONFIG_IBM_ASM is not set
> > # CONFIG_PHANTOM is not set
> > # CONFIG_SGI_IOC4 is not set
> > # CONFIG_TIFM_CORE is not set
> > # CONFIG_ICS932S401 is not set
> > # CONFIG_ENCLOSURE_SERVICES is not set
> > # CONFIG_HP_ILO is not set
> > CONFIG_APDS9802ALS=m
> > CONFIG_ISL29003=m
> > CONFIG_ISL29020=m
> > CONFIG_SENSORS_TSL2550=m
> > CONFIG_SENSORS_BH1770=m
> > CONFIG_SENSORS_APDS990X=y
> > # CONFIG_HMC6352 is not set
> > # CONFIG_DS1682 is not set
> > # CONFIG_PCH_PHUB is not set
> > # CONFIG_USB_SWITCH_FSA9480 is not set
> > CONFIG_SRAM=y
> > CONFIG_PANEL=m
> > CONFIG_PANEL_PARPORT=0
> > CONFIG_PANEL_PROFILE=5
> > CONFIG_PANEL_CHANGE_MESSAGE=y
> > CONFIG_PANEL_BOOT_MESSAGE=""
> > CONFIG_C2PORT=m
> > # CONFIG_C2PORT_DURAMAR_2150 is not set
> > 
> > #
> > # EEPROM support
> > #
> > CONFIG_EEPROM_AT24=y
> > # CONFIG_EEPROM_LEGACY is not set
> > CONFIG_EEPROM_MAX6875=m
> > # CONFIG_EEPROM_93CX6 is not set
> > # CONFIG_EEPROM_IDT_89HPESX is not set
> > # CONFIG_CB710_CORE is not set
> > 
> > #
> > # Texas Instruments shared transport line discipline
> > #
> > # CONFIG_TI_ST is not set
> > # CONFIG_SENSORS_LIS3_I2C is not set
> > 
> > #
> > # Altera FPGA firmware download module
> > #
> > # CONFIG_ALTERA_STAPL is not set
> > # CONFIG_INTEL_MEI is not set
> > # CONFIG_INTEL_MEI_ME is not set
> > # CONFIG_INTEL_MEI_TXE is not set
> > # CONFIG_VMWARE_VMCI is not set
> > 
> > #
> > # Intel MIC Bus Driver
> > #
> > 
> > #
> > # SCIF Bus Driver
> > #
> > 
> > #
> > # VOP Bus Driver
> > #
> > 
> > #
> > # Intel MIC Host Driver
> > #
> > 
> > #
> > # Intel MIC Card Driver
> > #
> > 
> > #
> > # SCIF Driver
> > #
> > 
> > #
> > # Intel MIC Coprocessor State Management (COSM) Drivers
> > #
> > 
> > #
> > # VOP Driver
> > #
> > CONFIG_ECHO=m
> > # CONFIG_CXL_BASE is not set
> > # CONFIG_CXL_AFU_DRIVER_OPS is not set
> > CONFIG_HAVE_IDE=y
> > # CONFIG_IDE is not set
> > 
> > #
> > # SCSI device support
> > #
> > CONFIG_SCSI_MOD=y
> > CONFIG_RAID_ATTRS=y
> > CONFIG_SCSI=y
> > CONFIG_SCSI_DMA=y
> > CONFIG_SCSI_NETLINK=y
> > # CONFIG_SCSI_MQ_DEFAULT is not set
> > CONFIG_SCSI_PROC_FS=y
> > 
> > #
> > # SCSI support type (disk, tape, CD-ROM)
> > #
> > CONFIG_BLK_DEV_SD=y
> > # CONFIG_CHR_DEV_ST is not set
> > CONFIG_CHR_DEV_OSST=y
> > # CONFIG_BLK_DEV_SR is not set
> > CONFIG_CHR_DEV_SG=y
> > CONFIG_CHR_DEV_SCH=m
> > # CONFIG_SCSI_CONSTANTS is not set
> > # CONFIG_SCSI_LOGGING is not set
> > # CONFIG_SCSI_SCAN_ASYNC is not set
> > 
> > #
> > # SCSI Transports
> > #
> > CONFIG_SCSI_SPI_ATTRS=y
> > CONFIG_SCSI_FC_ATTRS=y
> > # CONFIG_SCSI_ISCSI_ATTRS is not set
> > CONFIG_SCSI_SAS_ATTRS=y
> > CONFIG_SCSI_SAS_LIBSAS=m
> > CONFIG_SCSI_SAS_ATA=y
> > CONFIG_SCSI_SAS_HOST_SMP=y
> > # CONFIG_SCSI_SRP_ATTRS is not set
> > CONFIG_SCSI_LOWLEVEL=y
> > # CONFIG_ISCSI_TCP is not set
> > CONFIG_ISCSI_BOOT_SYSFS=m
> > # CONFIG_SCSI_CXGB3_ISCSI is not set
> > # CONFIG_SCSI_CXGB4_ISCSI is not set
> > # CONFIG_SCSI_BNX2_ISCSI is not set
> > # CONFIG_SCSI_BNX2X_FCOE is not set
> > # CONFIG_BE2ISCSI is not set
> > # CONFIG_BLK_DEV_3W_XXXX_RAID is not set
> > # CONFIG_SCSI_HPSA is not set
> > # CONFIG_SCSI_3W_9XXX is not set
> > # CONFIG_SCSI_3W_SAS is not set
> > # CONFIG_SCSI_ACARD is not set
> > CONFIG_SCSI_AHA152X=y
> > # CONFIG_SCSI_AHA1542 is not set
> > # CONFIG_SCSI_AACRAID is not set
> > # CONFIG_SCSI_AIC7XXX is not set
> > # CONFIG_SCSI_AIC79XX is not set
> > # CONFIG_SCSI_AIC94XX is not set
> > # CONFIG_SCSI_MVSAS is not set
> > # CONFIG_SCSI_MVUMI is not set
> > # CONFIG_SCSI_DPT_I2O is not set
> > CONFIG_SCSI_ADVANSYS=y
> > # CONFIG_SCSI_ARCMSR is not set
> > # CONFIG_SCSI_ESAS2R is not set
> > # CONFIG_MEGARAID_NEWGEN is not set
> > # CONFIG_MEGARAID_LEGACY is not set
> > # CONFIG_MEGARAID_SAS is not set
> > # CONFIG_SCSI_MPT3SAS is not set
> > # CONFIG_SCSI_MPT2SAS is not set
> > # CONFIG_SCSI_SMARTPQI is not set
> > CONFIG_SCSI_UFSHCD=y
> > # CONFIG_SCSI_UFSHCD_PCI is not set
> > CONFIG_SCSI_UFSHCD_PLATFORM=m
> > CONFIG_SCSI_UFS_DWC_TC_PLATFORM=m
> > # CONFIG_SCSI_HPTIOP is not set
> > CONFIG_SCSI_BUSLOGIC=m
> > # CONFIG_SCSI_FLASHPOINT is not set
> > # CONFIG_VMWARE_PVSCSI is not set
> > CONFIG_LIBFC=m
> > CONFIG_LIBFCOE=m
> > # CONFIG_FCOE is not set
> > # CONFIG_FCOE_FNIC is not set
> > # CONFIG_SCSI_SNIC is not set
> > # CONFIG_SCSI_DMX3191D is not set
> > CONFIG_SCSI_EATA=m
> > CONFIG_SCSI_EATA_TAGGED_QUEUE=y
> > # CONFIG_SCSI_EATA_LINKED_COMMANDS is not set
> > CONFIG_SCSI_EATA_MAX_TAGS=16
> > CONFIG_SCSI_FUTURE_DOMAIN=m
> > CONFIG_SCSI_GDTH=m
> > # CONFIG_SCSI_ISCI is not set
> > CONFIG_SCSI_GENERIC_NCR5380=y
> > # CONFIG_SCSI_IPS is not set
> > # CONFIG_SCSI_INITIO is not set
> > # CONFIG_SCSI_INIA100 is not set
> > # CONFIG_SCSI_NCR53C406A is not set
> > # CONFIG_SCSI_STEX is not set
> > # CONFIG_SCSI_SYM53C8XX_2 is not set
> > # CONFIG_SCSI_IPR is not set
> > CONFIG_SCSI_QLOGIC_FAS=m
> > # CONFIG_SCSI_QLOGIC_1280 is not set
> > # CONFIG_SCSI_QLA_FC is not set
> > # CONFIG_SCSI_QLA_ISCSI is not set
> > # CONFIG_SCSI_LPFC is not set
> > CONFIG_SCSI_SYM53C416=y
> > # CONFIG_SCSI_DC395x is not set
> > # CONFIG_SCSI_AM53C974 is not set
> > # CONFIG_SCSI_NSP32 is not set
> > # CONFIG_SCSI_WD719X is not set
> > CONFIG_SCSI_DEBUG=y
> > # CONFIG_SCSI_PMCRAID is not set
> > # CONFIG_SCSI_PM8001 is not set
> > # CONFIG_SCSI_BFA_FC is not set
> > CONFIG_SCSI_VIRTIO=m
> > # CONFIG_SCSI_CHELSIO_FCOE is not set
> > # CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
> > CONFIG_SCSI_DH=y
> > # CONFIG_SCSI_DH_RDAC is not set
> > # CONFIG_SCSI_DH_HP_SW is not set
> > CONFIG_SCSI_DH_EMC=m
> > # CONFIG_SCSI_DH_ALUA is not set
> > # CONFIG_SCSI_OSD_INITIATOR is not set
> > CONFIG_ATA=m
> > # CONFIG_ATA_NONSTANDARD is not set
> > # CONFIG_ATA_VERBOSE_ERROR is not set
> > CONFIG_ATA_ACPI=y
> > # CONFIG_SATA_ZPODD is not set
> > # CONFIG_SATA_PMP is not set
> > 
> > #
> > # Controllers with non-SFF native interface
> > #
> > # CONFIG_SATA_AHCI is not set
> > # CONFIG_SATA_AHCI_PLATFORM is not set
> > # CONFIG_SATA_INIC162X is not set
> > # CONFIG_SATA_ACARD_AHCI is not set
> > # CONFIG_SATA_SIL24 is not set
> > # CONFIG_ATA_SFF is not set
> > CONFIG_MD=y
> > CONFIG_BLK_DEV_MD=y
> > # CONFIG_MD_AUTODETECT is not set
> > # CONFIG_MD_LINEAR is not set
> > # CONFIG_MD_RAID0 is not set
> > CONFIG_MD_RAID1=y
> > CONFIG_MD_RAID10=y
> > CONFIG_MD_RAID456=y
> > CONFIG_MD_MULTIPATH=y
> > CONFIG_MD_FAULTY=m
> > # CONFIG_MD_CLUSTER is not set
> > # CONFIG_BCACHE is not set
> > CONFIG_BLK_DEV_DM_BUILTIN=y
> > CONFIG_BLK_DEV_DM=y
> > # CONFIG_DM_MQ_DEFAULT is not set
> > # CONFIG_DM_DEBUG is not set
> > CONFIG_DM_BUFIO=y
> > # CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set
> > CONFIG_DM_BIO_PRISON=m
> > CONFIG_DM_PERSISTENT_DATA=m
> > CONFIG_DM_CRYPT=y
> > CONFIG_DM_SNAPSHOT=y
> > # CONFIG_DM_THIN_PROVISIONING is not set
> > # CONFIG_DM_CACHE is not set
> > CONFIG_DM_ERA=m
> > CONFIG_DM_MIRROR=y
> > CONFIG_DM_LOG_USERSPACE=y
> > CONFIG_DM_RAID=y
> > CONFIG_DM_ZERO=m
> > CONFIG_DM_MULTIPATH=m
> > CONFIG_DM_MULTIPATH_QL=m
> > CONFIG_DM_MULTIPATH_ST=m
> > CONFIG_DM_DELAY=m
> > CONFIG_DM_UEVENT=y
> > CONFIG_DM_FLAKEY=m
> > CONFIG_DM_VERITY=m
> > # CONFIG_DM_VERITY_FEC is not set
> > CONFIG_DM_SWITCH=y
> > # CONFIG_DM_LOG_WRITES is not set
> > CONFIG_TARGET_CORE=m
> > CONFIG_TCM_IBLOCK=m
> > # CONFIG_TCM_FILEIO is not set
> > CONFIG_TCM_PSCSI=m
> > CONFIG_TCM_USER2=m
> > CONFIG_LOOPBACK_TARGET=m
> > CONFIG_TCM_FC=m
> > # CONFIG_ISCSI_TARGET is not set
> > # CONFIG_FUSION is not set
> > 
> > #
> > # IEEE 1394 (FireWire) support
> > #
> > # CONFIG_FIREWIRE is not set
> > # CONFIG_FIREWIRE_NOSY is not set
> > CONFIG_MACINTOSH_DRIVERS=y
> > # CONFIG_MAC_EMUMOUSEBTN is not set
> > # CONFIG_NETDEVICES is not set
> > CONFIG_NVM=y
> > # CONFIG_NVM_DEBUG is not set
> > # CONFIG_NVM_RRPC is not set
> > 
> > #
> > # Input device support
> > #
> > CONFIG_INPUT=y
> > CONFIG_INPUT_LEDS=m
> > # CONFIG_INPUT_FF_MEMLESS is not set
> > # CONFIG_INPUT_POLLDEV is not set
> > # CONFIG_INPUT_SPARSEKMAP is not set
> > # CONFIG_INPUT_MATRIXKMAP is not set
> > 
> > #
> > # Userland interfaces
> > #
> > CONFIG_INPUT_MOUSEDEV=y
> > CONFIG_INPUT_MOUSEDEV_PSAUX=y
> > CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
> > CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
> > # CONFIG_INPUT_JOYDEV is not set
> > # CONFIG_INPUT_EVDEV is not set
> > # CONFIG_INPUT_EVBUG is not set
> > 
> > #
> > # Input Device Drivers
> > #
> > CONFIG_INPUT_KEYBOARD=y
> > # CONFIG_KEYBOARD_ADC is not set
> > # CONFIG_KEYBOARD_ADP5520 is not set
> > # CONFIG_KEYBOARD_ADP5588 is not set
> > # CONFIG_KEYBOARD_ADP5589 is not set
> > CONFIG_KEYBOARD_ATKBD=y
> > # CONFIG_KEYBOARD_QT1070 is not set
> > # CONFIG_KEYBOARD_QT2160 is not set
> > # CONFIG_KEYBOARD_LKKBD is not set
> > # CONFIG_KEYBOARD_GPIO is not set
> > # CONFIG_KEYBOARD_GPIO_POLLED is not set
> > # CONFIG_KEYBOARD_TCA6416 is not set
> > # CONFIG_KEYBOARD_TCA8418 is not set
> > # CONFIG_KEYBOARD_MATRIX is not set
> > # CONFIG_KEYBOARD_LM8323 is not set
> > # CONFIG_KEYBOARD_LM8333 is not set
> > # CONFIG_KEYBOARD_MAX7359 is not set
> > # CONFIG_KEYBOARD_MCS is not set
> > # CONFIG_KEYBOARD_MPR121 is not set
> > # CONFIG_KEYBOARD_NEWTON is not set
> > # CONFIG_KEYBOARD_OPENCORES is not set
> > # CONFIG_KEYBOARD_SAMSUNG is not set
> > # CONFIG_KEYBOARD_STOWAWAY is not set
> > # CONFIG_KEYBOARD_SUNKBD is not set
> > # CONFIG_KEYBOARD_TM2_TOUCHKEY is not set
> > # CONFIG_KEYBOARD_XTKBD is not set
> > # CONFIG_KEYBOARD_CROS_EC is not set
> > CONFIG_INPUT_MOUSE=y
> > CONFIG_MOUSE_PS2=y
> > CONFIG_MOUSE_PS2_ALPS=y
> > CONFIG_MOUSE_PS2_BYD=y
> > CONFIG_MOUSE_PS2_LOGIPS2PP=y
> > CONFIG_MOUSE_PS2_SYNAPTICS=y
> > CONFIG_MOUSE_PS2_CYPRESS=y
> > CONFIG_MOUSE_PS2_LIFEBOOK=y
> > CONFIG_MOUSE_PS2_TRACKPOINT=y
> > # CONFIG_MOUSE_PS2_ELANTECH is not set
> > # CONFIG_MOUSE_PS2_SENTELIC is not set
> > # CONFIG_MOUSE_PS2_TOUCHKIT is not set
> > CONFIG_MOUSE_PS2_FOCALTECH=y
> > # CONFIG_MOUSE_PS2_VMMOUSE is not set
> > # CONFIG_MOUSE_SERIAL is not set
> > # CONFIG_MOUSE_APPLETOUCH is not set
> > # CONFIG_MOUSE_BCM5974 is not set
> > # CONFIG_MOUSE_CYAPA is not set
> > # CONFIG_MOUSE_ELAN_I2C is not set
> > # CONFIG_MOUSE_INPORT is not set
> > # CONFIG_MOUSE_LOGIBM is not set
> > # CONFIG_MOUSE_PC110PAD is not set
> > # CONFIG_MOUSE_VSXXXAA is not set
> > # CONFIG_MOUSE_GPIO is not set
> > # CONFIG_MOUSE_SYNAPTICS_I2C is not set
> > # CONFIG_MOUSE_SYNAPTICS_USB is not set
> > # CONFIG_INPUT_JOYSTICK is not set
> > # CONFIG_INPUT_TABLET is not set
> > # CONFIG_INPUT_TOUCHSCREEN is not set
> > # CONFIG_INPUT_MISC is not set
> > # CONFIG_RMI4_CORE is not set
> > 
> > #
> > # Hardware I/O ports
> > #
> > CONFIG_SERIO=y
> > CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
> > CONFIG_SERIO_I8042=y
> > CONFIG_SERIO_SERPORT=y
> > CONFIG_SERIO_CT82C710=y
> > # CONFIG_SERIO_PARKBD is not set
> > # CONFIG_SERIO_PCIPS2 is not set
> > CONFIG_SERIO_LIBPS2=y
> > CONFIG_SERIO_RAW=y
> > CONFIG_SERIO_ALTERA_PS2=m
> > CONFIG_SERIO_PS2MULT=m
> > # CONFIG_SERIO_ARC_PS2 is not set
> > CONFIG_USERIO=y
> > CONFIG_GAMEPORT=y
> > CONFIG_GAMEPORT_NS558=m
> > # CONFIG_GAMEPORT_L4 is not set
> > # CONFIG_GAMEPORT_EMU10K1 is not set
> > # CONFIG_GAMEPORT_FM801 is not set
> > 
> > #
> > # Character devices
> > #
> > CONFIG_TTY=y
> > # CONFIG_VT is not set
> > CONFIG_UNIX98_PTYS=y
> > CONFIG_LEGACY_PTYS=y
> > CONFIG_LEGACY_PTY_COUNT=256
> > # CONFIG_SERIAL_NONSTANDARD is not set
> > # CONFIG_NOZOMI is not set
> > # CONFIG_N_GSM is not set
> > # CONFIG_TRACE_SINK is not set
> > CONFIG_DEVMEM=y
> > # CONFIG_DEVKMEM is not set
> > 
> > #
> > # Serial drivers
> > #
> > CONFIG_SERIAL_EARLYCON=y
> > CONFIG_SERIAL_8250=y
> > CONFIG_SERIAL_8250_DEPRECATED_OPTIONS=y
> > CONFIG_SERIAL_8250_PNP=y
> > # CONFIG_SERIAL_8250_FINTEK is not set
> > CONFIG_SERIAL_8250_CONSOLE=y
> > CONFIG_SERIAL_8250_DMA=y
> > CONFIG_SERIAL_8250_PCI=y
> > CONFIG_SERIAL_8250_EXAR=y
> > # CONFIG_SERIAL_8250_CS is not set
> > CONFIG_SERIAL_8250_NR_UARTS=4
> > CONFIG_SERIAL_8250_RUNTIME_UARTS=4
> > # CONFIG_SERIAL_8250_EXTENDED is not set
> > # CONFIG_SERIAL_8250_FSL is not set
> > # CONFIG_SERIAL_8250_DW is not set
> > # CONFIG_SERIAL_8250_RT288X is not set
> > CONFIG_SERIAL_8250_LPSS=y
> > CONFIG_SERIAL_8250_MID=y
> > # CONFIG_SERIAL_8250_MOXA is not set
> > 
> > #
> > # Non-8250 serial port support
> > #
> > # CONFIG_SERIAL_UARTLITE is not set
> > CONFIG_SERIAL_CORE=y
> > CONFIG_SERIAL_CORE_CONSOLE=y
> > # CONFIG_SERIAL_JSM is not set
> > # CONFIG_SERIAL_SCCNXP is not set
> > # CONFIG_SERIAL_SC16IS7XX is not set
> > # CONFIG_SERIAL_TIMBERDALE is not set
> > # CONFIG_SERIAL_ALTERA_JTAGUART is not set
> > # CONFIG_SERIAL_ALTERA_UART is not set
> > # CONFIG_SERIAL_PCH_UART is not set
> > # CONFIG_SERIAL_ARC is not set
> > # CONFIG_SERIAL_RP2 is not set
> > # CONFIG_SERIAL_FSL_LPUART is not set
> > # CONFIG_SERIAL_DEV_BUS is not set
> > # CONFIG_TTY_PRINTK is not set
> > # CONFIG_PRINTER is not set
> > # CONFIG_PPDEV is not set
> > # CONFIG_VIRTIO_CONSOLE is not set
> > # CONFIG_IPMI_HANDLER is not set
> > CONFIG_HW_RANDOM=m
> > CONFIG_HW_RANDOM_TIMERIOMEM=m
> > CONFIG_HW_RANDOM_INTEL=m
> > CONFIG_HW_RANDOM_AMD=m
> > CONFIG_HW_RANDOM_GEODE=m
> > CONFIG_HW_RANDOM_VIA=m
> > CONFIG_HW_RANDOM_VIRTIO=m
> > # CONFIG_HW_RANDOM_TPM is not set
> > CONFIG_NVRAM=y
> > CONFIG_DTLK=y
> > # CONFIG_R3964 is not set
> > # CONFIG_APPLICOM is not set
> > # CONFIG_SONYPI is not set
> > 
> > #
> > # PCMCIA character devices
> > #
> > # CONFIG_SYNCLINK_CS is not set
> > # CONFIG_CARDMAN_4000 is not set
> > # CONFIG_CARDMAN_4040 is not set
> > CONFIG_SCR24X=m
> > # CONFIG_MWAVE is not set
> > CONFIG_PC8736x_GPIO=m
> > CONFIG_NSC_GPIO=m
> > CONFIG_RAW_DRIVER=y
> > CONFIG_MAX_RAW_DEVS=256
> > # CONFIG_HPET is not set
> > # CONFIG_HANGCHECK_TIMER is not set
> > CONFIG_TCG_TPM=m
> > # CONFIG_TCG_TIS is not set
> > CONFIG_TCG_TIS_I2C_ATMEL=m
> > CONFIG_TCG_TIS_I2C_INFINEON=m
> > # CONFIG_TCG_TIS_I2C_NUVOTON is not set
> > CONFIG_TCG_NSC=m
> > # CONFIG_TCG_ATMEL is not set
> > CONFIG_TCG_INFINEON=m
> > # CONFIG_TCG_CRB is not set
> > # CONFIG_TCG_VTPM_PROXY is not set
> > CONFIG_TCG_TIS_ST33ZP24=m
> > CONFIG_TCG_TIS_ST33ZP24_I2C=m
> > CONFIG_TELCLOCK=y
> > CONFIG_DEVPORT=y
> > # CONFIG_XILLYBUS is not set
> > 
> > #
> > # I2C support
> > #
> > CONFIG_I2C=y
> > CONFIG_ACPI_I2C_OPREGION=y
> > CONFIG_I2C_BOARDINFO=y
> > # CONFIG_I2C_COMPAT is not set
> > # CONFIG_I2C_CHARDEV is not set
> > CONFIG_I2C_MUX=y
> > 
> > #
> > # Multiplexer I2C Chip support
> > #
> > CONFIG_I2C_MUX_GPIO=y
> > CONFIG_I2C_MUX_PCA9541=m
> > # CONFIG_I2C_MUX_PCA954x is not set
> > CONFIG_I2C_MUX_REG=m
> > CONFIG_I2C_MUX_MLXCPLD=m
> > # CONFIG_I2C_HELPER_AUTO is not set
> > CONFIG_I2C_SMBUS=m
> > 
> > #
> > # I2C Algorithms
> > #
> > CONFIG_I2C_ALGOBIT=y
> > CONFIG_I2C_ALGOPCF=m
> > CONFIG_I2C_ALGOPCA=m
> > 
> > #
> > # I2C Hardware Bus support
> > #
> > 
> > #
> > # PC SMBus host controller drivers
> > #
> > # CONFIG_I2C_ALI1535 is not set
> > # CONFIG_I2C_ALI1563 is not set
> > # CONFIG_I2C_ALI15X3 is not set
> > # CONFIG_I2C_AMD756 is not set
> > # CONFIG_I2C_AMD8111 is not set
> > # CONFIG_I2C_I801 is not set
> > # CONFIG_I2C_ISCH is not set
> > # CONFIG_I2C_ISMT is not set
> > # CONFIG_I2C_PIIX4 is not set
> > # CONFIG_I2C_NFORCE2 is not set
> > # CONFIG_I2C_SIS5595 is not set
> > # CONFIG_I2C_SIS630 is not set
> > # CONFIG_I2C_SIS96X is not set
> > # CONFIG_I2C_VIA is not set
> > # CONFIG_I2C_VIAPRO is not set
> > 
> > #
> > # ACPI drivers
> > #
> > # CONFIG_I2C_SCMI is not set
> > 
> > #
> > # I2C system bus drivers (mostly embedded / system-on-chip)
> > #
> > CONFIG_I2C_CBUS_GPIO=m
> > # CONFIG_I2C_DESIGNWARE_PLATFORM is not set
> > # CONFIG_I2C_DESIGNWARE_PCI is not set
> > # CONFIG_I2C_EG20T is not set
> > # CONFIG_I2C_EMEV2 is not set
> > CONFIG_I2C_GPIO=m
> > # CONFIG_I2C_KEMPLD is not set
> > CONFIG_I2C_OCORES=m
> > CONFIG_I2C_PCA_PLATFORM=m
> > # CONFIG_I2C_PXA_PCI is not set
> > CONFIG_I2C_SIMTEC=m
> > CONFIG_I2C_XILINX=y
> > 
> > #
> > # External I2C/SMBus adapter drivers
> > #
> > # CONFIG_I2C_DIOLAN_U2C is not set
> > CONFIG_I2C_DLN2=y
> > CONFIG_I2C_PARPORT=m
> > # CONFIG_I2C_PARPORT_LIGHT is not set
> > CONFIG_I2C_ROBOTFUZZ_OSIF=y
> > # CONFIG_I2C_TAOS_EVM is not set
> > CONFIG_I2C_TINY_USB=m
> > 
> > #
> > # Other I2C/SMBus bus drivers
> > #
> > # CONFIG_I2C_ELEKTOR is not set
> > # CONFIG_I2C_PCA_ISA is not set
> > # CONFIG_I2C_CROS_EC_TUNNEL is not set
> > # CONFIG_SCx200_ACB is not set
> > CONFIG_I2C_STUB=m
> > CONFIG_I2C_SLAVE=y
> > CONFIG_I2C_SLAVE_EEPROM=y
> > # CONFIG_I2C_DEBUG_CORE is not set
> > # CONFIG_I2C_DEBUG_ALGO is not set
> > # CONFIG_I2C_DEBUG_BUS is not set
> > # CONFIG_SPI is not set
> > CONFIG_SPMI=y
> > # CONFIG_HSI is not set
> > 
> > #
> > # PPS support
> > #
> > CONFIG_PPS=y
> > # CONFIG_PPS_DEBUG is not set
> > CONFIG_NTP_PPS=y
> > 
> > #
> > # PPS clients support
> > #
> > # CONFIG_PPS_CLIENT_KTIMER is not set
> > # CONFIG_PPS_CLIENT_LDISC is not set
> > CONFIG_PPS_CLIENT_PARPORT=m
> > # CONFIG_PPS_CLIENT_GPIO is not set
> > 
> > #
> > # PPS generators support
> > #
> > 
> > #
> > # PTP clock support
> > #
> > 
> > #
> > # Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
> > #
> > # CONFIG_PTP_1588_CLOCK_PCH is not set
> > CONFIG_GPIOLIB=y
> > CONFIG_GPIO_ACPI=y
> > CONFIG_GPIOLIB_IRQCHIP=y
> > CONFIG_DEBUG_GPIO=y
> > CONFIG_GPIO_SYSFS=y
> > CONFIG_GPIO_GENERIC=y
> > CONFIG_GPIO_MAX730X=y
> > 
> > #
> > # Memory mapped GPIO drivers
> > #
> > # CONFIG_GPIO_AMDPT is not set
> > # CONFIG_GPIO_AXP209 is not set
> > CONFIG_GPIO_DWAPB=m
> > # CONFIG_GPIO_EXAR is not set
> > CONFIG_GPIO_GENERIC_PLATFORM=y
> > # CONFIG_GPIO_ICH is not set
> > # CONFIG_GPIO_LYNXPOINT is not set
> > # CONFIG_GPIO_MOCKUP is not set
> > # CONFIG_GPIO_VX855 is not set
> > 
> > #
> > # Port-mapped I/O GPIO drivers
> > #
> > CONFIG_GPIO_104_DIO_48E=m
> > CONFIG_GPIO_104_IDIO_16=m
> > CONFIG_GPIO_104_IDI_48=y
> > # CONFIG_GPIO_F7188X is not set
> > CONFIG_GPIO_GPIO_MM=m
> > CONFIG_GPIO_IT87=m
> > # CONFIG_GPIO_SCH is not set
> > # CONFIG_GPIO_SCH311X is not set
> > CONFIG_GPIO_WS16C48=m
> > 
> > #
> > # I2C GPIO expanders
> > #
> > CONFIG_GPIO_ADP5588=y
> > # CONFIG_GPIO_ADP5588_IRQ is not set
> > CONFIG_GPIO_MAX7300=y
> > CONFIG_GPIO_MAX732X=y
> > CONFIG_GPIO_MAX732X_IRQ=y
> > CONFIG_GPIO_PCA953X=y
> > CONFIG_GPIO_PCA953X_IRQ=y
> > CONFIG_GPIO_PCF857X=m
> > CONFIG_GPIO_TPIC2810=m
> > 
> > #
> > # MFD GPIO expanders
> > #
> > # CONFIG_GPIO_ADP5520 is not set
> > CONFIG_GPIO_DA9055=y
> > # CONFIG_GPIO_DLN2 is not set
> > CONFIG_GPIO_KEMPLD=y
> > CONFIG_GPIO_LP3943=m
> > CONFIG_GPIO_LP873X=m
> > CONFIG_GPIO_PALMAS=y
> > # CONFIG_GPIO_RC5T583 is not set
> > CONFIG_GPIO_TPS6586X=y
> > # CONFIG_GPIO_TPS65910 is not set
> > # CONFIG_GPIO_TWL6040 is not set
> > # CONFIG_GPIO_UCB1400 is not set
> > CONFIG_GPIO_WM8994=m
> > 
> > #
> > # PCI GPIO expanders
> > #
> > # CONFIG_GPIO_AMD8111 is not set
> > # CONFIG_GPIO_BT8XX is not set
> > # CONFIG_GPIO_ML_IOH is not set
> > # CONFIG_GPIO_PCH is not set
> > # CONFIG_GPIO_PCI_IDIO_16 is not set
> > # CONFIG_GPIO_RDC321X is not set
> > 
> > #
> > # SPI or I2C GPIO expanders
> > #
> > 
> > #
> > # USB GPIO expanders
> > #
> > CONFIG_W1=y
> > CONFIG_W1_CON=y
> > 
> > #
> > # 1-wire Bus Masters
> > #
> > # CONFIG_W1_MASTER_MATROX is not set
> > # CONFIG_W1_MASTER_DS2490 is not set
> > # CONFIG_W1_MASTER_DS2482 is not set
> > CONFIG_W1_MASTER_DS1WM=y
> > CONFIG_W1_MASTER_GPIO=y
> > 
> > #
> > # 1-wire Slaves
> > #
> > # CONFIG_W1_SLAVE_THERM is not set
> > CONFIG_W1_SLAVE_SMEM=y
> > # CONFIG_W1_SLAVE_DS2405 is not set
> > CONFIG_W1_SLAVE_DS2408=y
> > # CONFIG_W1_SLAVE_DS2408_READBACK is not set
> > CONFIG_W1_SLAVE_DS2413=m
> > # CONFIG_W1_SLAVE_DS2406 is not set
> > # CONFIG_W1_SLAVE_DS2423 is not set
> > # CONFIG_W1_SLAVE_DS2431 is not set
> > # CONFIG_W1_SLAVE_DS2433 is not set
> > CONFIG_W1_SLAVE_DS2760=y
> > # CONFIG_W1_SLAVE_DS2780 is not set
> > CONFIG_W1_SLAVE_DS2781=y
> > CONFIG_W1_SLAVE_DS28E04=y
> > CONFIG_W1_SLAVE_BQ27000=y
> > # CONFIG_POWER_AVS is not set
> > # CONFIG_POWER_RESET is not set
> > CONFIG_POWER_SUPPLY=y
> > # CONFIG_POWER_SUPPLY_DEBUG is not set
> > # CONFIG_PDA_POWER is not set
> > # CONFIG_GENERIC_ADC_BATTERY is not set
> > # CONFIG_TEST_POWER is not set
> > # CONFIG_BATTERY_DS2760 is not set
> > # CONFIG_BATTERY_DS2780 is not set
> > # CONFIG_BATTERY_DS2781 is not set
> > # CONFIG_BATTERY_DS2782 is not set
> > # CONFIG_BATTERY_SBS is not set
> > # CONFIG_CHARGER_SBS is not set
> > # CONFIG_BATTERY_BQ27XXX is not set
> > # CONFIG_AXP288_FUEL_GAUGE is not set
> > # CONFIG_BATTERY_MAX17040 is not set
> > # CONFIG_BATTERY_MAX17042 is not set
> > # CONFIG_CHARGER_PCF50633 is not set
> > # CONFIG_CHARGER_ISP1704 is not set
> > # CONFIG_CHARGER_MAX8903 is not set
> > # CONFIG_CHARGER_LP8727 is not set
> > # CONFIG_CHARGER_LP8788 is not set
> > # CONFIG_CHARGER_GPIO is not set
> > # CONFIG_CHARGER_MAX77693 is not set
> > # CONFIG_CHARGER_BQ2415X is not set
> > # CONFIG_CHARGER_BQ24190 is not set
> > # CONFIG_CHARGER_BQ24257 is not set
> > # CONFIG_CHARGER_BQ24735 is not set
> > # CONFIG_CHARGER_BQ25890 is not set
> > # CONFIG_CHARGER_SMB347 is not set
> > # CONFIG_CHARGER_TPS65090 is not set
> > # CONFIG_CHARGER_TPS65217 is not set
> > # CONFIG_BATTERY_GAUGE_LTC2941 is not set
> > # CONFIG_BATTERY_RT5033 is not set
> > # CONFIG_CHARGER_RT9455 is not set
> > # CONFIG_AXP20X_POWER is not set
> > CONFIG_HWMON=m
> > CONFIG_HWMON_VID=m
> > # CONFIG_HWMON_DEBUG_CHIP is not set
> > 
> > #
> > # Native drivers
> > #
> > # CONFIG_SENSORS_ABITUGURU is not set
> > # CONFIG_SENSORS_ABITUGURU3 is not set
> > # CONFIG_SENSORS_AD7414 is not set
> > # CONFIG_SENSORS_AD7418 is not set
> > # CONFIG_SENSORS_ADM1021 is not set
> > # CONFIG_SENSORS_ADM1025 is not set
> > CONFIG_SENSORS_ADM1026=m
> > CONFIG_SENSORS_ADM1029=m
> > CONFIG_SENSORS_ADM1031=m
> > CONFIG_SENSORS_ADM9240=m
> > # CONFIG_SENSORS_ADT7410 is not set
> > CONFIG_SENSORS_ADT7411=m
> > # CONFIG_SENSORS_ADT7462 is not set
> > # CONFIG_SENSORS_ADT7470 is not set
> > # CONFIG_SENSORS_ADT7475 is not set
> > CONFIG_SENSORS_ASC7621=m
> > # CONFIG_SENSORS_K8TEMP is not set
> > # CONFIG_SENSORS_K10TEMP is not set
> > # CONFIG_SENSORS_FAM15H_POWER is not set
> > # CONFIG_SENSORS_APPLESMC is not set
> > CONFIG_SENSORS_ASB100=m
> > # CONFIG_SENSORS_ATXP1 is not set
> > CONFIG_SENSORS_DS620=m
> > CONFIG_SENSORS_DS1621=m
> > CONFIG_SENSORS_DELL_SMM=m
> > # CONFIG_SENSORS_DA9055 is not set
> > # CONFIG_SENSORS_I5K_AMB is not set
> > CONFIG_SENSORS_F71805F=m
> > # CONFIG_SENSORS_F71882FG is not set
> > CONFIG_SENSORS_F75375S=m
> > # CONFIG_SENSORS_FSCHMD is not set
> > CONFIG_SENSORS_FTSTEUTATES=m
> > CONFIG_SENSORS_GL518SM=m
> > # CONFIG_SENSORS_GL520SM is not set
> > # CONFIG_SENSORS_G760A is not set
> > # CONFIG_SENSORS_G762 is not set
> > CONFIG_SENSORS_GPIO_FAN=m
> > # CONFIG_SENSORS_HIH6130 is not set
> > CONFIG_SENSORS_IIO_HWMON=m
> > # CONFIG_SENSORS_I5500 is not set
> > CONFIG_SENSORS_CORETEMP=m
> > CONFIG_SENSORS_IT87=m
> > # CONFIG_SENSORS_JC42 is not set
> > # CONFIG_SENSORS_POWR1220 is not set
> > CONFIG_SENSORS_LINEAGE=m
> > CONFIG_SENSORS_LTC2945=m
> > CONFIG_SENSORS_LTC2990=m
> > CONFIG_SENSORS_LTC4151=m
> > CONFIG_SENSORS_LTC4215=m
> > CONFIG_SENSORS_LTC4222=m
> > CONFIG_SENSORS_LTC4245=m
> > CONFIG_SENSORS_LTC4260=m
> > CONFIG_SENSORS_LTC4261=m
> > CONFIG_SENSORS_MAX16065=m
> > CONFIG_SENSORS_MAX1619=m
> > # CONFIG_SENSORS_MAX1668 is not set
> > CONFIG_SENSORS_MAX197=m
> > CONFIG_SENSORS_MAX6639=m
> > CONFIG_SENSORS_MAX6642=m
> > CONFIG_SENSORS_MAX6650=m
> > # CONFIG_SENSORS_MAX6697 is not set
> > CONFIG_SENSORS_MAX31790=m
> > CONFIG_SENSORS_MCP3021=m
> > CONFIG_SENSORS_TC654=m
> > CONFIG_SENSORS_MENF21BMC_HWMON=m
> > # CONFIG_SENSORS_LM63 is not set
> > CONFIG_SENSORS_LM73=m
> > # CONFIG_SENSORS_LM75 is not set
> > CONFIG_SENSORS_LM77=m
> > CONFIG_SENSORS_LM78=m
> > # CONFIG_SENSORS_LM80 is not set
> > CONFIG_SENSORS_LM83=m
> > # CONFIG_SENSORS_LM85 is not set
> > # CONFIG_SENSORS_LM87 is not set
> > CONFIG_SENSORS_LM90=m
> > CONFIG_SENSORS_LM92=m
> > # CONFIG_SENSORS_LM93 is not set
> > CONFIG_SENSORS_LM95234=m
> > CONFIG_SENSORS_LM95241=m
> > # CONFIG_SENSORS_LM95245 is not set
> > CONFIG_SENSORS_PC87360=m
> > # CONFIG_SENSORS_PC87427 is not set
> > CONFIG_SENSORS_NTC_THERMISTOR=m
> > # CONFIG_SENSORS_NCT6683 is not set
> > CONFIG_SENSORS_NCT6775=m
> > CONFIG_SENSORS_NCT7802=m
> > CONFIG_SENSORS_NCT7904=m
> > CONFIG_SENSORS_PCF8591=m
> > CONFIG_PMBUS=m
> > CONFIG_SENSORS_PMBUS=m
> > CONFIG_SENSORS_ADM1275=m
> > CONFIG_SENSORS_LM25066=m
> > CONFIG_SENSORS_LTC2978=m
> > CONFIG_SENSORS_LTC3815=m
> > CONFIG_SENSORS_MAX16064=m
> > CONFIG_SENSORS_MAX20751=m
> > CONFIG_SENSORS_MAX34440=m
> > CONFIG_SENSORS_MAX8688=m
> > CONFIG_SENSORS_TPS40422=m
> > # CONFIG_SENSORS_UCD9000 is not set
> > CONFIG_SENSORS_UCD9200=m
> > CONFIG_SENSORS_ZL6100=m
> > # CONFIG_SENSORS_SHT15 is not set
> > CONFIG_SENSORS_SHT21=m
> > CONFIG_SENSORS_SHT3x=m
> > # CONFIG_SENSORS_SHTC1 is not set
> > # CONFIG_SENSORS_SIS5595 is not set
> > CONFIG_SENSORS_DME1737=m
> > # CONFIG_SENSORS_EMC1403 is not set
> > # CONFIG_SENSORS_EMC2103 is not set
> > CONFIG_SENSORS_EMC6W201=m
> > CONFIG_SENSORS_SMSC47M1=m
> > CONFIG_SENSORS_SMSC47M192=m
> > CONFIG_SENSORS_SMSC47B397=m
> > CONFIG_SENSORS_SCH56XX_COMMON=m
> > CONFIG_SENSORS_SCH5627=m
> > CONFIG_SENSORS_SCH5636=m
> > # CONFIG_SENSORS_STTS751 is not set
> > CONFIG_SENSORS_SMM665=m
> > # CONFIG_SENSORS_ADC128D818 is not set
> > CONFIG_SENSORS_ADS1015=m
> > CONFIG_SENSORS_ADS7828=m
> > # CONFIG_SENSORS_AMC6821 is not set
> > # CONFIG_SENSORS_INA209 is not set
> > # CONFIG_SENSORS_INA2XX is not set
> > CONFIG_SENSORS_INA3221=m
> > # CONFIG_SENSORS_TC74 is not set
> > CONFIG_SENSORS_THMC50=m
> > CONFIG_SENSORS_TMP102=m
> > CONFIG_SENSORS_TMP103=m
> > CONFIG_SENSORS_TMP108=m
> > CONFIG_SENSORS_TMP401=m
> > CONFIG_SENSORS_TMP421=m
> > CONFIG_SENSORS_VIA_CPUTEMP=m
> > # CONFIG_SENSORS_VIA686A is not set
> > CONFIG_SENSORS_VT1211=m
> > # CONFIG_SENSORS_VT8231 is not set
> > CONFIG_SENSORS_W83781D=m
> > CONFIG_SENSORS_W83791D=m
> > # CONFIG_SENSORS_W83792D is not set
> > # CONFIG_SENSORS_W83793 is not set
> > CONFIG_SENSORS_W83795=m
> > # CONFIG_SENSORS_W83795_FANCTRL is not set
> > # CONFIG_SENSORS_W83L785TS is not set
> > CONFIG_SENSORS_W83L786NG=m
> > # CONFIG_SENSORS_W83627HF is not set
> > # CONFIG_SENSORS_W83627EHF is not set
> > 
> > #
> > # ACPI drivers
> > #
> > # CONFIG_SENSORS_ACPI_POWER is not set
> > # CONFIG_SENSORS_ATK0110 is not set
> > CONFIG_THERMAL=y
> > CONFIG_THERMAL_WRITABLE_TRIPS=y
> > # CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE is not set
> > CONFIG_THERMAL_DEFAULT_GOV_FAIR_SHARE=y
> > # CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE is not set
> > # CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR is not set
> > CONFIG_THERMAL_GOV_FAIR_SHARE=y
> > CONFIG_THERMAL_GOV_STEP_WISE=y
> > CONFIG_THERMAL_GOV_BANG_BANG=y
> > CONFIG_THERMAL_GOV_USER_SPACE=y
> > CONFIG_THERMAL_GOV_POWER_ALLOCATOR=y
> > # CONFIG_THERMAL_EMULATION is not set
> > CONFIG_INTEL_POWERCLAMP=m
> > CONFIG_INTEL_SOC_DTS_IOSF_CORE=m
> > CONFIG_INTEL_SOC_DTS_THERMAL=m
> > 
> > #
> > # ACPI INT340X thermal drivers
> > #
> > # CONFIG_INT340X_THERMAL is not set
> > # CONFIG_INTEL_PCH_THERMAL is not set
> > CONFIG_GENERIC_ADC_THERMAL=m
> > CONFIG_WATCHDOG=y
> > CONFIG_WATCHDOG_CORE=y
> > CONFIG_WATCHDOG_NOWAYOUT=y
> > CONFIG_WATCHDOG_SYSFS=y
> > 
> > #
> > # Watchdog Device Drivers
> > #
> > # CONFIG_SOFT_WATCHDOG is not set
> > CONFIG_DA9055_WATCHDOG=m
> > CONFIG_DA9063_WATCHDOG=m
> > CONFIG_DA9062_WATCHDOG=m
> > # CONFIG_MENF21BMC_WATCHDOG is not set
> > # CONFIG_WDAT_WDT is not set
> > CONFIG_XILINX_WATCHDOG=m
> > CONFIG_ZIIRAVE_WATCHDOG=m
> > # CONFIG_CADENCE_WATCHDOG is not set
> > # CONFIG_DW_WATCHDOG is not set
> > CONFIG_MAX63XX_WATCHDOG=y
> > CONFIG_ACQUIRE_WDT=y
> > CONFIG_ADVANTECH_WDT=m
> > # CONFIG_ALIM1535_WDT is not set
> > # CONFIG_ALIM7101_WDT is not set
> > # CONFIG_EBC_C384_WDT is not set
> > # CONFIG_F71808E_WDT is not set
> > # CONFIG_SP5100_TCO is not set
> > CONFIG_SBC_FITPC2_WATCHDOG=m
> > # CONFIG_EUROTECH_WDT is not set
> > CONFIG_IB700_WDT=m
> > # CONFIG_IBMASR is not set
> > CONFIG_WAFER_WDT=m
> > # CONFIG_I6300ESB_WDT is not set
> > # CONFIG_IE6XX_WDT is not set
> > # CONFIG_ITCO_WDT is not set
> > CONFIG_IT8712F_WDT=y
> > CONFIG_IT87_WDT=y
> > # CONFIG_HP_WATCHDOG is not set
> > CONFIG_KEMPLD_WDT=y
> > CONFIG_SC1200_WDT=m
> > CONFIG_PC87413_WDT=y
> > # CONFIG_NV_TCO is not set
> > # CONFIG_60XX_WDT is not set
> > # CONFIG_SBC8360_WDT is not set
> > # CONFIG_SBC7240_WDT is not set
> > CONFIG_CPU5_WDT=y
> > CONFIG_SMSC_SCH311X_WDT=y
> > CONFIG_SMSC37B787_WDT=y
> > # CONFIG_VIA_WDT is not set
> > # CONFIG_W83627HF_WDT is not set
> > CONFIG_W83877F_WDT=m
> > # CONFIG_W83977F_WDT is not set
> > # CONFIG_MACHZ_WDT is not set
> > CONFIG_SBC_EPX_C3_WATCHDOG=y
> > # CONFIG_NI903X_WDT is not set
> > # CONFIG_NIC7018_WDT is not set
> > CONFIG_MEN_A21_WDT=m
> > 
> > #
> > # ISA-based Watchdog Cards
> > #
> > # CONFIG_PCWATCHDOG is not set
> > CONFIG_MIXCOMWD=y
> > # CONFIG_WDT is not set
> > 
> > #
> > # PCI-based Watchdog Cards
> > #
> > # CONFIG_PCIPCWATCHDOG is not set
> > # CONFIG_WDTPCI is not set
> > 
> > #
> > # USB-based Watchdog Cards
> > #
> > CONFIG_USBPCWATCHDOG=m
> > 
> > #
> > # Watchdog Pretimeout Governors
> > #
> > CONFIG_WATCHDOG_PRETIMEOUT_GOV=y
> > # CONFIG_WATCHDOG_PRETIMEOUT_DEFAULT_GOV_NOOP is not set
> > CONFIG_WATCHDOG_PRETIMEOUT_DEFAULT_GOV_PANIC=y
> > CONFIG_WATCHDOG_PRETIMEOUT_GOV_NOOP=y
> > CONFIG_WATCHDOG_PRETIMEOUT_GOV_PANIC=y
> > CONFIG_SSB_POSSIBLE=y
> > 
> > #
> > # Sonics Silicon Backplane
> > #
> > # CONFIG_SSB is not set
> > CONFIG_BCMA_POSSIBLE=y
> > 
> > #
> > # Broadcom specific AMBA
> > #
> > CONFIG_BCMA=m
> > CONFIG_BCMA_HOST_PCI_POSSIBLE=y
> > CONFIG_BCMA_HOST_PCI=y
> > CONFIG_BCMA_HOST_SOC=y
> > CONFIG_BCMA_DRIVER_PCI=y
> > CONFIG_BCMA_SFLASH=y
> > # CONFIG_BCMA_DRIVER_GMAC_CMN is not set
> > # CONFIG_BCMA_DRIVER_GPIO is not set
> > CONFIG_BCMA_DEBUG=y
> > 
> > #
> > # Multifunction device drivers
> > #
> > CONFIG_MFD_CORE=y
> > # CONFIG_MFD_CS5535 is not set
> > CONFIG_MFD_AS3711=y
> > CONFIG_PMIC_ADP5520=y
> > # CONFIG_MFD_AAT2870_CORE is not set
> > # CONFIG_MFD_BCM590XX is not set
> > CONFIG_MFD_AXP20X=m
> > CONFIG_MFD_AXP20X_I2C=m
> > CONFIG_MFD_CROS_EC=m
> > CONFIG_MFD_CROS_EC_I2C=m
> > # CONFIG_PMIC_DA903X is not set
> > # CONFIG_MFD_DA9052_I2C is not set
> > CONFIG_MFD_DA9055=y
> > CONFIG_MFD_DA9062=y
> > CONFIG_MFD_DA9063=m
> > # CONFIG_MFD_DA9150 is not set
> > CONFIG_MFD_DLN2=y
> > # CONFIG_MFD_MC13XXX_I2C is not set
> > # CONFIG_HTC_PASIC3 is not set
> > # CONFIG_HTC_I2CPLD is not set
> > # CONFIG_MFD_INTEL_QUARK_I2C_GPIO is not set
> > # CONFIG_LPC_ICH is not set
> > # CONFIG_LPC_SCH is not set
> > # CONFIG_INTEL_SOC_PMIC is not set
> > # CONFIG_MFD_INTEL_LPSS_ACPI is not set
> > # CONFIG_MFD_INTEL_LPSS_PCI is not set
> > # CONFIG_MFD_JANZ_CMODIO is not set
> > CONFIG_MFD_KEMPLD=y
> > CONFIG_MFD_88PM800=m
> > CONFIG_MFD_88PM805=y
> > # CONFIG_MFD_88PM860X is not set
> > # CONFIG_MFD_MAX14577 is not set
> > CONFIG_MFD_MAX77693=m
> > CONFIG_MFD_MAX77843=y
> > # CONFIG_MFD_MAX8907 is not set
> > # CONFIG_MFD_MAX8925 is not set
> > # CONFIG_MFD_MAX8997 is not set
> > CONFIG_MFD_MAX8998=y
> > CONFIG_MFD_MT6397=m
> > CONFIG_MFD_MENF21BMC=m
> > # CONFIG_MFD_VIPERBOARD is not set
> > # CONFIG_MFD_RETU is not set
> > CONFIG_MFD_PCF50633=y
> > CONFIG_PCF50633_ADC=m
> > CONFIG_PCF50633_GPIO=y
> > CONFIG_UCB1400_CORE=m
> > # CONFIG_MFD_RDC321X is not set
> > # CONFIG_MFD_RTSX_PCI is not set
> > CONFIG_MFD_RT5033=y
> > CONFIG_MFD_RTSX_USB=m
> > CONFIG_MFD_RC5T583=y
> > CONFIG_MFD_SEC_CORE=y
> > # CONFIG_MFD_SI476X_CORE is not set
> > CONFIG_MFD_SM501=m
> > # CONFIG_MFD_SM501_GPIO is not set
> > CONFIG_MFD_SKY81452=y
> > # CONFIG_MFD_SMSC is not set
> > CONFIG_ABX500_CORE=y
> > # CONFIG_AB3100_CORE is not set
> > CONFIG_MFD_SYSCON=y
> > CONFIG_MFD_TI_AM335X_TSCADC=m
> > CONFIG_MFD_LP3943=m
> > CONFIG_MFD_LP8788=y
> > CONFIG_MFD_PALMAS=y
> > # CONFIG_TPS6105X is not set
> > # CONFIG_TPS65010 is not set
> > CONFIG_TPS6507X=y
> > # CONFIG_MFD_TPS65086 is not set
> > CONFIG_MFD_TPS65090=y
> > CONFIG_MFD_TPS65217=m
> > CONFIG_MFD_TI_LP873X=y
> > # CONFIG_MFD_TPS65218 is not set
> > CONFIG_MFD_TPS6586X=y
> > CONFIG_MFD_TPS65910=y
> > # CONFIG_MFD_TPS65912_I2C is not set
> > CONFIG_MFD_TPS80031=y
> > # CONFIG_TWL4030_CORE is not set
> > CONFIG_TWL6040_CORE=y
> > CONFIG_MFD_WL1273_CORE=y
> > CONFIG_MFD_LM3533=y
> > # CONFIG_MFD_TIMBERDALE is not set
> > # CONFIG_MFD_TMIO is not set
> > # CONFIG_MFD_VX855 is not set
> > # CONFIG_MFD_ARIZONA_I2C is not set
> > CONFIG_MFD_WM8400=y
> > # CONFIG_MFD_WM831X_I2C is not set
> > # CONFIG_MFD_WM8350_I2C is not set
> > CONFIG_MFD_WM8994=m
> > # CONFIG_REGULATOR is not set
> > CONFIG_MEDIA_SUPPORT=y
> > 
> > #
> > # Multimedia core support
> > #
> > # CONFIG_MEDIA_CAMERA_SUPPORT is not set
> > # CONFIG_MEDIA_ANALOG_TV_SUPPORT is not set
> > CONFIG_MEDIA_DIGITAL_TV_SUPPORT=y
> > CONFIG_MEDIA_RADIO_SUPPORT=y
> > CONFIG_MEDIA_SDR_SUPPORT=y
> > # CONFIG_MEDIA_RC_SUPPORT is not set
> > CONFIG_MEDIA_CEC_SUPPORT=y
> > CONFIG_MEDIA_CEC_DEBUG=y
> > CONFIG_MEDIA_CEC_EDID=y
> > CONFIG_MEDIA_CONTROLLER=y
> > # CONFIG_MEDIA_CONTROLLER_DVB is not set
> > CONFIG_VIDEO_DEV=y
> > CONFIG_VIDEO_V4L2_SUBDEV_API=y
> > CONFIG_VIDEO_V4L2=y
> > CONFIG_VIDEO_ADV_DEBUG=y
> > # CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
> > CONFIG_V4L2_FLASH_LED_CLASS=m
> > CONFIG_DVB_CORE=y
> > # CONFIG_DVB_NET is not set
> > # CONFIG_TTPCI_EEPROM is not set
> > CONFIG_DVB_MAX_ADAPTERS=16
> > # CONFIG_DVB_DYNAMIC_MINORS is not set
> > # CONFIG_DVB_DEMUX_SECTION_LOSS_LOG is not set
> > 
> > #
> > # Media drivers
> > #
> > # CONFIG_MEDIA_USB_SUPPORT is not set
> > # CONFIG_MEDIA_PCI_SUPPORT is not set
> > # CONFIG_DVB_PLATFORM_DRIVERS is not set
> > 
> > #
> > # Supported MMC/SDIO adapters
> > #
> > CONFIG_SMS_SDIO_DRV=y
> > CONFIG_RADIO_ADAPTERS=y
> > CONFIG_RADIO_SI470X=y
> > CONFIG_USB_SI470X=y
> > # CONFIG_RADIO_SI4713 is not set
> > # CONFIG_USB_MR800 is not set
> > # CONFIG_USB_DSBR is not set
> > # CONFIG_RADIO_MAXIRADIO is not set
> > # CONFIG_RADIO_SHARK is not set
> > # CONFIG_RADIO_SHARK2 is not set
> > CONFIG_USB_KEENE=y
> > # CONFIG_USB_RAREMONO is not set
> > CONFIG_USB_MA901=y
> > # CONFIG_RADIO_TEA5764 is not set
> > CONFIG_RADIO_SAA7706H=y
> > CONFIG_RADIO_TEF6862=y
> > CONFIG_RADIO_WL1273=y
> > 
> > #
> > # Texas Instruments WL128x FM driver (ST based)
> > #
> > # CONFIG_V4L_RADIO_ISA_DRIVERS is not set
> > CONFIG_MEDIA_COMMON_OPTIONS=y
> > 
> > #
> > # common driver options
> > #
> > CONFIG_CYPRESS_FIRMWARE=m
> > CONFIG_SMS_SIANO_MDTV=y
> > 
> > #
> > # Media ancillary drivers (tuners, sensors, i2c, spi, frontends)
> > #
> > CONFIG_MEDIA_SUBDRV_AUTOSELECT=y
> > CONFIG_MEDIA_ATTACH=y
> > 
> > #
> > # Audio decoders, processors and mixers
> > #
> > 
> > #
> > # RDS decoders
> > #
> > 
> > #
> > # Video decoders
> > #
> > 
> > #
> > # Video and audio decoders
> > #
> > 
> > #
> > # Video encoders
> > #
> > 
> > #
> > # Camera sensor devices
> > #
> > 
> > #
> > # Flash devices
> > #
> > 
> > #
> > # Video improvement chips
> > #
> > 
> > #
> > # Audio/Video compression chips
> > #
> > 
> > #
> > # Miscellaneous helper chips
> > #
> > 
> > #
> > # Sensors used on soc_camera driver
> > #
> > CONFIG_MEDIA_TUNER=y
> > CONFIG_MEDIA_TUNER_SIMPLE=y
> > CONFIG_MEDIA_TUNER_TDA8290=y
> > CONFIG_MEDIA_TUNER_TDA827X=y
> > CONFIG_MEDIA_TUNER_TDA18271=y
> > CONFIG_MEDIA_TUNER_TDA9887=y
> > CONFIG_MEDIA_TUNER_TEA5761=y
> > CONFIG_MEDIA_TUNER_TEA5767=y
> > CONFIG_MEDIA_TUNER_MT20XX=y
> > CONFIG_MEDIA_TUNER_XC2028=y
> > CONFIG_MEDIA_TUNER_XC5000=y
> > CONFIG_MEDIA_TUNER_XC4000=y
> > CONFIG_MEDIA_TUNER_MC44S803=y
> > 
> > #
> > # Multistandard (satellite) frontends
> > #
> > 
> > #
> > # Multistandard (cable + terrestrial) frontends
> > #
> > 
> > #
> > # DVB-S (satellite) frontends
> > #
> > 
> > #
> > # DVB-T (terrestrial) frontends
> > #
> > # CONFIG_DVB_AS102_FE is not set
> > # CONFIG_DVB_GP8PSK_FE is not set
> > 
> > #
> > # DVB-C (cable) frontends
> > #
> > 
> > #
> > # ATSC (North American/Korean Terrestrial/Cable DTV) frontends
> > #
> > 
> > #
> > # ISDB-T (terrestrial) frontends
> > #
> > 
> > #
> > # ISDB-S (satellite) & ISDB-T (terrestrial) frontends
> > #
> > 
> > #
> > # Digital terrestrial only tuners/PLL
> > #
> > 
> > #
> > # SEC control devices for DVB-S
> > #
> > 
> > #
> > # Tools to develop new frontends
> > #
> > # CONFIG_DVB_DUMMY_FE is not set
> > 
> > #
> > # Graphics support
> > #
> > # CONFIG_AGP is not set
> > CONFIG_VGA_ARB=y
> > CONFIG_VGA_ARB_MAX_GPUS=16
> > # CONFIG_VGA_SWITCHEROO is not set
> > # CONFIG_DRM is not set
> > 
> > #
> > # ACP (Audio CoProcessor) Configuration
> > #
> > # CONFIG_DRM_LIB_RANDOM is not set
> > 
> > #
> > # Frame buffer Devices
> > #
> > # CONFIG_FB is not set
> > CONFIG_BACKLIGHT_LCD_SUPPORT=y
> > CONFIG_LCD_CLASS_DEVICE=m
> > CONFIG_LCD_PLATFORM=m
> > CONFIG_BACKLIGHT_CLASS_DEVICE=m
> > CONFIG_BACKLIGHT_GENERIC=m
> > CONFIG_BACKLIGHT_LM3533=m
> > # CONFIG_BACKLIGHT_APPLE is not set
> > CONFIG_BACKLIGHT_PM8941_WLED=m
> > CONFIG_BACKLIGHT_SAHARA=m
> > CONFIG_BACKLIGHT_ADP5520=m
> > CONFIG_BACKLIGHT_ADP8860=m
> > CONFIG_BACKLIGHT_ADP8870=m
> > CONFIG_BACKLIGHT_PCF50633=m
> > CONFIG_BACKLIGHT_LM3639=m
> > CONFIG_BACKLIGHT_SKY81452=m
> > # CONFIG_BACKLIGHT_TPS65217 is not set
> > CONFIG_BACKLIGHT_AS3711=m
> > CONFIG_BACKLIGHT_GPIO=m
> > # CONFIG_BACKLIGHT_LV5207LP is not set
> > CONFIG_BACKLIGHT_BD6107=m
> > # CONFIG_VGASTATE is not set
> > CONFIG_SOUND=m
> > CONFIG_SOUND_OSS_CORE=y
> > CONFIG_SOUND_OSS_CORE_PRECLAIM=y
> > CONFIG_SND=m
> > CONFIG_SND_TIMER=m
> > CONFIG_SND_PCM=m
> > CONFIG_SND_DMAENGINE_PCM=m
> > CONFIG_SND_HWDEP=m
> > CONFIG_SND_RAWMIDI=m
> > CONFIG_SND_JACK=y
> > CONFIG_SND_JACK_INPUT_DEV=y
> > CONFIG_SND_SEQUENCER=m
> > CONFIG_SND_SEQ_DUMMY=m
> > CONFIG_SND_OSSEMUL=y
> > CONFIG_SND_MIXER_OSS=m
> > CONFIG_SND_PCM_OSS=m
> > CONFIG_SND_PCM_OSS_PLUGINS=y
> > # CONFIG_SND_PCM_TIMER is not set
> > # CONFIG_SND_SEQUENCER_OSS is not set
> > # CONFIG_SND_HRTIMER is not set
> > CONFIG_SND_DYNAMIC_MINORS=y
> > CONFIG_SND_MAX_CARDS=32
> > # CONFIG_SND_SUPPORT_OLD_API is not set
> > CONFIG_SND_PROC_FS=y
> > CONFIG_SND_VERBOSE_PROCFS=y
> > # CONFIG_SND_VERBOSE_PRINTK is not set
> > # CONFIG_SND_DEBUG is not set
> > CONFIG_SND_VMASTER=y
> > CONFIG_SND_DMA_SGBUF=y
> > CONFIG_SND_RAWMIDI_SEQ=m
> > # CONFIG_SND_OPL3_LIB_SEQ is not set
> > # CONFIG_SND_OPL4_LIB_SEQ is not set
> > # CONFIG_SND_SBAWE_SEQ is not set
> > # CONFIG_SND_EMU10K1_SEQ is not set
> > CONFIG_SND_MPU401_UART=m
> > CONFIG_SND_VX_LIB=m
> > CONFIG_SND_AC97_CODEC=m
> > CONFIG_SND_DRIVERS=y
> > # CONFIG_SND_PCSP is not set
> > # CONFIG_SND_DUMMY is not set
> > CONFIG_SND_ALOOP=m
> > CONFIG_SND_VIRMIDI=m
> > # CONFIG_SND_MTPAV is not set
> > CONFIG_SND_MTS64=m
> > CONFIG_SND_SERIAL_U16550=m
> > CONFIG_SND_MPU401=m
> > CONFIG_SND_PORTMAN2X4=m
> > # CONFIG_SND_AC97_POWER_SAVE is not set
> > # CONFIG_SND_ISA is not set
> > CONFIG_SND_PCI=y
> > # CONFIG_SND_AD1889 is not set
> > # CONFIG_SND_ALS300 is not set
> > # CONFIG_SND_ALS4000 is not set
> > # CONFIG_SND_ALI5451 is not set
> > # CONFIG_SND_ASIHPI is not set
> > # CONFIG_SND_ATIIXP is not set
> > # CONFIG_SND_ATIIXP_MODEM is not set
> > # CONFIG_SND_AU8810 is not set
> > # CONFIG_SND_AU8820 is not set
> > # CONFIG_SND_AU8830 is not set
> > # CONFIG_SND_AW2 is not set
> > # CONFIG_SND_AZT3328 is not set
> > # CONFIG_SND_BT87X is not set
> > # CONFIG_SND_CA0106 is not set
> > # CONFIG_SND_CMIPCI is not set
> > # CONFIG_SND_OXYGEN is not set
> > # CONFIG_SND_CS4281 is not set
> > # CONFIG_SND_CS46XX is not set
> > # CONFIG_SND_CS5530 is not set
> > # CONFIG_SND_CS5535AUDIO is not set
> > # CONFIG_SND_CTXFI is not set
> > # CONFIG_SND_DARLA20 is not set
> > # CONFIG_SND_GINA20 is not set
> > # CONFIG_SND_LAYLA20 is not set
> > # CONFIG_SND_DARLA24 is not set
> > # CONFIG_SND_GINA24 is not set
> > # CONFIG_SND_LAYLA24 is not set
> > # CONFIG_SND_MONA is not set
> > # CONFIG_SND_MIA is not set
> > # CONFIG_SND_ECHO3G is not set
> > # CONFIG_SND_INDIGO is not set
> > # CONFIG_SND_INDIGOIO is not set
> > # CONFIG_SND_INDIGODJ is not set
> > # CONFIG_SND_INDIGOIOX is not set
> > # CONFIG_SND_INDIGODJX is not set
> > # CONFIG_SND_EMU10K1 is not set
> > # CONFIG_SND_EMU10K1X is not set
> > # CONFIG_SND_ENS1370 is not set
> > # CONFIG_SND_ENS1371 is not set
> > # CONFIG_SND_ES1938 is not set
> > # CONFIG_SND_ES1968 is not set
> > # CONFIG_SND_FM801 is not set
> > # CONFIG_SND_HDSP is not set
> > # CONFIG_SND_HDSPM is not set
> > # CONFIG_SND_ICE1712 is not set
> > # CONFIG_SND_ICE1724 is not set
> > # CONFIG_SND_INTEL8X0 is not set
> > # CONFIG_SND_INTEL8X0M is not set
> > # CONFIG_SND_KORG1212 is not set
> > # CONFIG_SND_LOLA is not set
> > # CONFIG_SND_LX6464ES is not set
> > # CONFIG_SND_MAESTRO3 is not set
> > # CONFIG_SND_MIXART is not set
> > # CONFIG_SND_NM256 is not set
> > # CONFIG_SND_PCXHR is not set
> > # CONFIG_SND_RIPTIDE is not set
> > # CONFIG_SND_RME32 is not set
> > # CONFIG_SND_RME96 is not set
> > # CONFIG_SND_RME9652 is not set
> > # CONFIG_SND_SE6X is not set
> > # CONFIG_SND_SIS7019 is not set
> > # CONFIG_SND_SONICVIBES is not set
> > # CONFIG_SND_TRIDENT is not set
> > # CONFIG_SND_VIA82XX is not set
> > # CONFIG_SND_VIA82XX_MODEM is not set
> > # CONFIG_SND_VIRTUOSO is not set
> > # CONFIG_SND_VX222 is not set
> > # CONFIG_SND_YMFPCI is not set
> > 
> > #
> > # HD-Audio
> > #
> > # CONFIG_SND_HDA_INTEL is not set
> > CONFIG_SND_HDA_PREALLOC_SIZE=64
> > # CONFIG_SND_USB is not set
> > CONFIG_SND_PCMCIA=y
> > CONFIG_SND_VXPOCKET=m
> > # CONFIG_SND_PDAUDIOCF is not set
> > CONFIG_SND_SOC=m
> > CONFIG_SND_SOC_AC97_BUS=y
> > CONFIG_SND_SOC_GENERIC_DMAENGINE_PCM=y
> > CONFIG_SND_SOC_AMD_ACP=m
> > CONFIG_SND_ATMEL_SOC=m
> > # CONFIG_SND_DESIGNWARE_I2S is not set
> > 
> > #
> > # SoC Audio for Freescale CPUs
> > #
> > 
> > #
> > # Common SoC Audio options for Freescale CPUs:
> > #
> > CONFIG_SND_SOC_FSL_ASRC=m
> > CONFIG_SND_SOC_FSL_SAI=m
> > CONFIG_SND_SOC_FSL_SSI=m
> > # CONFIG_SND_SOC_FSL_SPDIF is not set
> > CONFIG_SND_SOC_FSL_ESAI=m
> > # CONFIG_SND_SOC_IMX_AUDMUX is not set
> > # CONFIG_SND_SOC_IMG is not set
> > # CONFIG_SND_SOC_INTEL_BXT_DA7219_MAX98357A_MACH is not set
> > # CONFIG_SND_SOC_INTEL_BXT_RT298_MACH is not set
> > # CONFIG_SND_SOC_INTEL_BYTCR_RT5640_MACH is not set
> > # CONFIG_SND_SOC_INTEL_BYTCR_RT5651_MACH is not set
> > # CONFIG_SND_SOC_INTEL_SKL_RT286_MACH is not set
> > CONFIG_SND_SOC_XTFPGA_I2S=m
> > CONFIG_SND_SOC_I2C_AND_SPI=m
> > 
> > #
> > # CODEC drivers
> > #
> > CONFIG_SND_SOC_AC97_CODEC=m
> > # CONFIG_SND_SOC_ADAU1701 is not set
> > # CONFIG_SND_SOC_ADAU7002 is not set
> > CONFIG_SND_SOC_AK4554=m
> > CONFIG_SND_SOC_AK4613=m
> > CONFIG_SND_SOC_AK4642=m
> > # CONFIG_SND_SOC_AK5386 is not set
> > CONFIG_SND_SOC_ALC5623=m
> > CONFIG_SND_SOC_BT_SCO=m
> > # CONFIG_SND_SOC_CS35L32 is not set
> > CONFIG_SND_SOC_CS35L33=m
> > CONFIG_SND_SOC_CS35L34=m
> > CONFIG_SND_SOC_CS42L42=m
> > CONFIG_SND_SOC_CS42L51=m
> > CONFIG_SND_SOC_CS42L51_I2C=m
> > # CONFIG_SND_SOC_CS42L52 is not set
> > # CONFIG_SND_SOC_CS42L56 is not set
> > CONFIG_SND_SOC_CS42L73=m
> > CONFIG_SND_SOC_CS4265=m
> > # CONFIG_SND_SOC_CS4270 is not set
> > CONFIG_SND_SOC_CS4271=m
> > CONFIG_SND_SOC_CS4271_I2C=m
> > CONFIG_SND_SOC_CS42XX8=m
> > CONFIG_SND_SOC_CS42XX8_I2C=m
> > # CONFIG_SND_SOC_CS4349 is not set
> > # CONFIG_SND_SOC_CS53L30 is not set
> > # CONFIG_SND_SOC_ES8328_I2C is not set
> > CONFIG_SND_SOC_GTM601=m
> > # CONFIG_SND_SOC_INNO_RK3036 is not set
> > CONFIG_SND_SOC_MAX98504=m
> > CONFIG_SND_SOC_MAX9860=m
> > CONFIG_SND_SOC_MSM8916_WCD_ANALOG=m
> > CONFIG_SND_SOC_MSM8916_WCD_DIGITAL=m
> > CONFIG_SND_SOC_PCM1681=m
> > CONFIG_SND_SOC_PCM179X=m
> > CONFIG_SND_SOC_PCM179X_I2C=m
> > CONFIG_SND_SOC_PCM3168A=m
> > CONFIG_SND_SOC_PCM3168A_I2C=m
> > # CONFIG_SND_SOC_PCM512x_I2C is not set
> > # CONFIG_SND_SOC_RT5616 is not set
> > CONFIG_SND_SOC_RT5631=m
> > # CONFIG_SND_SOC_RT5677_SPI is not set
> > # CONFIG_SND_SOC_SGTL5000 is not set
> > CONFIG_SND_SOC_SIRF_AUDIO_CODEC=m
> > CONFIG_SND_SOC_SPDIF=m
> > CONFIG_SND_SOC_SSM2602=m
> > CONFIG_SND_SOC_SSM2602_I2C=m
> > CONFIG_SND_SOC_SSM4567=m
> > CONFIG_SND_SOC_STA32X=m
> > CONFIG_SND_SOC_STA350=m
> > CONFIG_SND_SOC_STI_SAS=m
> > # CONFIG_SND_SOC_TAS2552 is not set
> > CONFIG_SND_SOC_TAS5086=m
> > CONFIG_SND_SOC_TAS571X=m
> > CONFIG_SND_SOC_TAS5720=m
> > CONFIG_SND_SOC_TFA9879=m
> > CONFIG_SND_SOC_TLV320AIC23=m
> > CONFIG_SND_SOC_TLV320AIC23_I2C=m
> > # CONFIG_SND_SOC_TLV320AIC31XX is not set
> > # CONFIG_SND_SOC_TLV320AIC3X is not set
> > # CONFIG_SND_SOC_TS3A227E is not set
> > CONFIG_SND_SOC_WM8510=m
> > # CONFIG_SND_SOC_WM8523 is not set
> > # CONFIG_SND_SOC_WM8580 is not set
> > CONFIG_SND_SOC_WM8711=m
> > # CONFIG_SND_SOC_WM8728 is not set
> > CONFIG_SND_SOC_WM8731=m
> > # CONFIG_SND_SOC_WM8737 is not set
> > # CONFIG_SND_SOC_WM8741 is not set
> > CONFIG_SND_SOC_WM8750=m
> > CONFIG_SND_SOC_WM8753=m
> > # CONFIG_SND_SOC_WM8776 is not set
> > # CONFIG_SND_SOC_WM8804_I2C is not set
> > CONFIG_SND_SOC_WM8903=m
> > CONFIG_SND_SOC_WM8960=m
> > # CONFIG_SND_SOC_WM8962 is not set
> > # CONFIG_SND_SOC_WM8974 is not set
> > CONFIG_SND_SOC_WM8978=m
> > CONFIG_SND_SOC_WM8985=m
> > # CONFIG_SND_SOC_NAU8540 is not set
> > CONFIG_SND_SOC_NAU8810=m
> > # CONFIG_SND_SOC_TPA6130A2 is not set
> > CONFIG_SND_SIMPLE_CARD_UTILS=m
> > CONFIG_SND_SIMPLE_CARD=m
> > # CONFIG_SND_X86 is not set
> > CONFIG_SOUND_PRIME=m
> > # CONFIG_SOUND_MSNDCLAS is not set
> > # CONFIG_SOUND_MSNDPIN is not set
> > CONFIG_SOUND_OSS=m
> > # CONFIG_SOUND_TRACEINIT is not set
> > # CONFIG_SOUND_DMAP is not set
> > # CONFIG_SOUND_VMIDI is not set
> > CONFIG_SOUND_TRIX=m
> > # CONFIG_SOUND_MSS is not set
> > CONFIG_SOUND_MPU401=m
> > CONFIG_SOUND_PAS=m
> > CONFIG_SOUND_PSS=m
> > CONFIG_PSS_MIXER=y
> > # CONFIG_PSS_HAVE_BOOT is not set
> > CONFIG_SOUND_SB=m
> > # CONFIG_SOUND_YM3812 is not set
> > # CONFIG_SOUND_UART6850 is not set
> > CONFIG_SOUND_AEDSP16=m
> > CONFIG_SC6600=y
> > CONFIG_SC6600_JOY=y
> > CONFIG_SC6600_CDROM=4
> > CONFIG_SC6600_CDROMBASE=0
> > CONFIG_SOUND_KAHLUA=m
> > CONFIG_AC97_BUS=m
> > 
> > #
> > # HID support
> > #
> > CONFIG_HID=y
> > # CONFIG_HID_BATTERY_STRENGTH is not set
> > # CONFIG_HIDRAW is not set
> > # CONFIG_UHID is not set
> > CONFIG_HID_GENERIC=y
> > 
> > #
> > # Special HID drivers
> > #
> > # CONFIG_HID_A4TECH is not set
> > # CONFIG_HID_ACRUX is not set
> > # CONFIG_HID_APPLE is not set
> > # CONFIG_HID_APPLEIR is not set
> > # CONFIG_HID_AUREAL is not set
> > # CONFIG_HID_BELKIN is not set
> > # CONFIG_HID_BETOP_FF is not set
> > # CONFIG_HID_CHERRY is not set
> > # CONFIG_HID_CHICONY is not set
> > # CONFIG_HID_CORSAIR is not set
> > # CONFIG_HID_PRODIKEYS is not set
> > # CONFIG_HID_CMEDIA is not set
> > # CONFIG_HID_CP2112 is not set
> > # CONFIG_HID_CYPRESS is not set
> > # CONFIG_HID_DRAGONRISE is not set
> > # CONFIG_HID_EMS_FF is not set
> > # CONFIG_HID_ELECOM is not set
> > # CONFIG_HID_ELO is not set
> > # CONFIG_HID_EZKEY is not set
> > # CONFIG_HID_GEMBIRD is not set
> > # CONFIG_HID_GFRM is not set
> > # CONFIG_HID_HOLTEK is not set
> > # CONFIG_HID_GT683R is not set
> > # CONFIG_HID_KEYTOUCH is not set
> > # CONFIG_HID_KYE is not set
> > # CONFIG_HID_UCLOGIC is not set
> > # CONFIG_HID_WALTOP is not set
> > # CONFIG_HID_GYRATION is not set
> > # CONFIG_HID_ICADE is not set
> > # CONFIG_HID_TWINHAN is not set
> > # CONFIG_HID_KENSINGTON is not set
> > # CONFIG_HID_LCPOWER is not set
> > # CONFIG_HID_LED is not set
> > # CONFIG_HID_LENOVO is not set
> > # CONFIG_HID_LOGITECH is not set
> > # CONFIG_HID_MAGICMOUSE is not set
> > # CONFIG_HID_MAYFLASH is not set
> > # CONFIG_HID_MICROSOFT is not set
> > # CONFIG_HID_MONTEREY is not set
> > # CONFIG_HID_MULTITOUCH is not set
> > # CONFIG_HID_NTRIG is not set
> > # CONFIG_HID_ORTEK is not set
> > # CONFIG_HID_PANTHERLORD is not set
> > # CONFIG_HID_PENMOUNT is not set
> > # CONFIG_HID_PETALYNX is not set
> > # CONFIG_HID_PICOLCD is not set
> > # CONFIG_HID_PLANTRONICS is not set
> > # CONFIG_HID_PRIMAX is not set
> > # CONFIG_HID_ROCCAT is not set
> > # CONFIG_HID_SAITEK is not set
> > # CONFIG_HID_SAMSUNG is not set
> > # CONFIG_HID_SONY is not set
> > # CONFIG_HID_SPEEDLINK is not set
> > # CONFIG_HID_STEELSERIES is not set
> > # CONFIG_HID_SUNPLUS is not set
> > # CONFIG_HID_RMI is not set
> > # CONFIG_HID_GREENASIA is not set
> > # CONFIG_HID_SMARTJOYPLUS is not set
> > # CONFIG_HID_TIVO is not set
> > # CONFIG_HID_TOPSEED is not set
> > # CONFIG_HID_THINGM is not set
> > # CONFIG_HID_THRUSTMASTER is not set
> > # CONFIG_HID_UDRAW_PS3 is not set
> > # CONFIG_HID_WACOM is not set
> > # CONFIG_HID_WIIMOTE is not set
> > # CONFIG_HID_XINMO is not set
> > # CONFIG_HID_ZEROPLUS is not set
> > # CONFIG_HID_ZYDACRON is not set
> > # CONFIG_HID_SENSOR_HUB is not set
> > # CONFIG_HID_ALPS is not set
> > 
> > #
> > # USB HID support
> > #
> > CONFIG_USB_HID=y
> > # CONFIG_HID_PID is not set
> > # CONFIG_USB_HIDDEV is not set
> > 
> > #
> > # I2C HID support
> > #
> > # CONFIG_I2C_HID is not set
> > CONFIG_USB_OHCI_LITTLE_ENDIAN=y
> > CONFIG_USB_SUPPORT=y
> > CONFIG_USB_COMMON=y
> > CONFIG_USB_ARCH_HAS_HCD=y
> > CONFIG_USB=y
> > # CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set
> > 
> > #
> > # Miscellaneous USB options
> > #
> > # CONFIG_USB_DEFAULT_PERSIST is not set
> > # CONFIG_USB_DYNAMIC_MINORS is not set
> > CONFIG_USB_OTG=y
> > # CONFIG_USB_OTG_WHITELIST is not set
> > CONFIG_USB_OTG_BLACKLIST_HUB=y
> > CONFIG_USB_OTG_FSM=y
> > CONFIG_USB_LEDS_TRIGGER_USBPORT=m
> > CONFIG_USB_MON=y
> > # CONFIG_USB_WUSB_CBAF is not set
> > 
> > #
> > # USB Host Controller Drivers
> > #
> > CONFIG_USB_C67X00_HCD=y
> > CONFIG_USB_XHCI_HCD=y
> > CONFIG_USB_XHCI_PCI=y
> > CONFIG_USB_XHCI_PLATFORM=y
> > CONFIG_USB_EHCI_HCD=m
> > CONFIG_USB_EHCI_ROOT_HUB_TT=y
> > CONFIG_USB_EHCI_TT_NEWSCHED=y
> > CONFIG_USB_EHCI_PCI=m
> > CONFIG_USB_EHCI_HCD_PLATFORM=m
> > # CONFIG_USB_OXU210HP_HCD is not set
> > CONFIG_USB_ISP116X_HCD=m
> > CONFIG_USB_ISP1362_HCD=m
> > # CONFIG_USB_FOTG210_HCD is not set
> > # CONFIG_USB_OHCI_HCD is not set
> > # CONFIG_USB_UHCI_HCD is not set
> > CONFIG_USB_SL811_HCD=y
> > CONFIG_USB_SL811_HCD_ISO=y
> > # CONFIG_USB_SL811_CS is not set
> > CONFIG_USB_R8A66597_HCD=m
> > CONFIG_USB_HCD_BCMA=m
> > # CONFIG_USB_HCD_TEST_MODE is not set
> > 
> > #
> > # USB Device Class drivers
> > #
> > # CONFIG_USB_ACM is not set
> > CONFIG_USB_PRINTER=m
> > # CONFIG_USB_WDM is not set
> > CONFIG_USB_TMC=y
> > 
> > #
> > # NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
> > #
> > 
> > #
> > # also be needed; see USB_STORAGE Help for more info
> > #
> > # CONFIG_USB_STORAGE is not set
> > 
> > #
> > # USB Imaging devices
> > #
> > CONFIG_USB_MDC800=m
> > CONFIG_USB_MICROTEK=y
> > CONFIG_USBIP_CORE=m
> > CONFIG_USBIP_VHCI_HCD=m
> > CONFIG_USBIP_VHCI_HC_PORTS=8
> > CONFIG_USBIP_VHCI_NR_HCS=1
> > CONFIG_USBIP_HOST=m
> > CONFIG_USBIP_VUDC=m
> > CONFIG_USBIP_DEBUG=y
> > CONFIG_USB_MUSB_HDRC=m
> > # CONFIG_USB_MUSB_HOST is not set
> > # CONFIG_USB_MUSB_GADGET is not set
> > CONFIG_USB_MUSB_DUAL_ROLE=y
> > 
> > #
> > # Platform Glue Layer
> > #
> > 
> > #
> > # MUSB DMA mode
> > #
> > # CONFIG_MUSB_PIO_ONLY is not set
> > CONFIG_USB_DWC3=y
> > CONFIG_USB_DWC3_HOST=y
> > 
> > #
> > # Platform Glue Driver Support
> > #
> > CONFIG_USB_DWC3_PCI=y
> > # CONFIG_USB_DWC2 is not set
> > CONFIG_USB_CHIPIDEA=m
> > CONFIG_USB_CHIPIDEA_PCI=m
> > # CONFIG_USB_CHIPIDEA_UDC is not set
> > CONFIG_USB_CHIPIDEA_HOST=y
> > # CONFIG_USB_CHIPIDEA_ULPI is not set
> > CONFIG_USB_ISP1760=m
> > CONFIG_USB_ISP1761_UDC=y
> > # CONFIG_USB_ISP1760_HOST_ROLE is not set
> > CONFIG_USB_ISP1760_GADGET_ROLE=y
> > # CONFIG_USB_ISP1760_DUAL_ROLE is not set
> > 
> > #
> > # USB port drivers
> > #
> > CONFIG_USB_USS720=m
> > # CONFIG_USB_SERIAL is not set
> > 
> > #
> > # USB Miscellaneous drivers
> > #
> > # CONFIG_USB_EMI62 is not set
> > # CONFIG_USB_EMI26 is not set
> > # CONFIG_USB_ADUTUX is not set
> > CONFIG_USB_SEVSEG=y
> > CONFIG_USB_RIO500=m
> > # CONFIG_USB_LEGOTOWER is not set
> > CONFIG_USB_LCD=m
> > CONFIG_USB_CYPRESS_CY7C63=m
> > CONFIG_USB_CYTHERM=y
> > CONFIG_USB_IDMOUSE=m
> > # CONFIG_USB_FTDI_ELAN is not set
> > CONFIG_USB_APPLEDISPLAY=m
> > # CONFIG_USB_SISUSBVGA is not set
> > CONFIG_USB_LD=y
> > CONFIG_USB_TRANCEVIBRATOR=m
> > CONFIG_USB_IOWARRIOR=y
> > CONFIG_USB_TEST=m
> > CONFIG_USB_EHSET_TEST_FIXTURE=y
> > CONFIG_USB_ISIGHTFW=m
> > # CONFIG_USB_YUREX is not set
> > CONFIG_USB_EZUSB_FX2=m
> > # CONFIG_USB_HUB_USB251XB is not set
> > # CONFIG_USB_HSIC_USB3503 is not set
> > CONFIG_USB_HSIC_USB4604=y
> > CONFIG_USB_LINK_LAYER_TEST=y
> > # CONFIG_USB_CHAOSKEY is not set
> > # CONFIG_UCSI is not set
> > 
> > #
> > # USB Physical Layer drivers
> > #
> > CONFIG_USB_PHY=y
> > CONFIG_NOP_USB_XCEIV=m
> > # CONFIG_USB_GPIO_VBUS is not set
> > CONFIG_USB_ISP1301=y
> > CONFIG_USB_GADGET=m
> > CONFIG_USB_GADGET_DEBUG=y
> > # CONFIG_USB_GADGET_VERBOSE is not set
> > # CONFIG_USB_GADGET_DEBUG_FILES is not set
> > # CONFIG_USB_GADGET_DEBUG_FS is not set
> > CONFIG_USB_GADGET_VBUS_DRAW=2
> > CONFIG_USB_GADGET_STORAGE_NUM_BUFFERS=2
> > 
> > #
> > # USB Peripheral Controller
> > #
> > # CONFIG_USB_FOTG210_UDC is not set
> > CONFIG_USB_GR_UDC=m
> > # CONFIG_USB_R8A66597 is not set
> > # CONFIG_USB_PXA27X is not set
> > CONFIG_USB_MV_UDC=m
> > CONFIG_USB_MV_U3D=m
> > # CONFIG_USB_M66592 is not set
> > CONFIG_USB_BDC_UDC=m
> > 
> > #
> > # Platform Support
> > #
> > CONFIG_USB_BDC_PCI=m
> > # CONFIG_USB_AMD5536UDC is not set
> > CONFIG_USB_NET2272=m
> > CONFIG_USB_NET2272_DMA=y
> > # CONFIG_USB_NET2280 is not set
> > # CONFIG_USB_GOKU is not set
> > # CONFIG_USB_EG20T is not set
> > CONFIG_USB_DUMMY_HCD=m
> > CONFIG_USB_LIBCOMPOSITE=m
> > CONFIG_USB_F_SS_LB=m
> > CONFIG_USB_U_ETHER=m
> > CONFIG_USB_F_ECM=m
> > CONFIG_USB_F_SUBSET=m
> > CONFIG_USB_F_MASS_STORAGE=m
> > CONFIG_USB_F_FS=m
> > CONFIG_USB_F_UAC2=m
> > CONFIG_USB_F_MIDI=m
> > CONFIG_USB_F_HID=m
> > CONFIG_USB_F_PRINTER=m
> > CONFIG_USB_F_TCM=m
> > # CONFIG_USB_CONFIGFS is not set
> > CONFIG_USB_ZERO=m
> > CONFIG_USB_ZERO_HNPTEST=y
> > CONFIG_USB_AUDIO=m
> > # CONFIG_GADGET_UAC1 is not set
> > CONFIG_USB_ETH=m
> > # CONFIG_USB_ETH_RNDIS is not set
> > # CONFIG_USB_ETH_EEM is not set
> > # CONFIG_USB_G_NCM is not set
> > # CONFIG_USB_GADGETFS is not set
> > CONFIG_USB_FUNCTIONFS=m
> > # CONFIG_USB_FUNCTIONFS_ETH is not set
> > # CONFIG_USB_FUNCTIONFS_RNDIS is not set
> > CONFIG_USB_FUNCTIONFS_GENERIC=y
> > CONFIG_USB_MASS_STORAGE=m
> > CONFIG_USB_GADGET_TARGET=m
> > # CONFIG_USB_G_SERIAL is not set
> > CONFIG_USB_MIDI_GADGET=m
> > CONFIG_USB_G_PRINTER=m
> > # CONFIG_USB_CDC_COMPOSITE is not set
> > # CONFIG_USB_G_NOKIA is not set
> > # CONFIG_USB_G_ACM_MS is not set
> > # CONFIG_USB_G_MULTI is not set
> > CONFIG_USB_G_HID=m
> > # CONFIG_USB_G_DBGP is not set
> > # CONFIG_USB_G_WEBCAM is not set
> > # CONFIG_USB_LED_TRIG is not set
> > CONFIG_USB_ULPI_BUS=m
> > # CONFIG_UWB is not set
> > CONFIG_MMC=y
> > CONFIG_MMC_DEBUG=y
> > CONFIG_MMC_BLOCK=m
> > CONFIG_MMC_BLOCK_MINORS=8
> > # CONFIG_MMC_BLOCK_BOUNCE is not set
> > # CONFIG_SDIO_UART is not set
> > CONFIG_MMC_TEST=y
> > 
> > #
> > # MMC/SD/SDIO Host Controller Drivers
> > #
> > CONFIG_MMC_SDHCI=m
> > # CONFIG_MMC_SDHCI_PCI is not set
> > # CONFIG_MMC_SDHCI_ACPI is not set
> > # CONFIG_MMC_SDHCI_PLTFM is not set
> > CONFIG_MMC_WBSD=y
> > # CONFIG_MMC_TIFM_SD is not set
> > # CONFIG_MMC_SDRICOH_CS is not set
> > # CONFIG_MMC_CB710 is not set
> > # CONFIG_MMC_VIA_SDMMC is not set
> > # CONFIG_MMC_VUB300 is not set
> > CONFIG_MMC_USHC=m
> > # CONFIG_MMC_USDHI6ROL0 is not set
> > CONFIG_MMC_REALTEK_USB=m
> > # CONFIG_MMC_TOSHIBA_PCI is not set
> > CONFIG_MMC_MTK=y
> > CONFIG_MEMSTICK=m
> > # CONFIG_MEMSTICK_DEBUG is not set
> > 
> > #
> > # MemoryStick drivers
> > #
> > CONFIG_MEMSTICK_UNSAFE_RESUME=y
> > CONFIG_MSPRO_BLOCK=m
> > CONFIG_MS_BLOCK=m
> > 
> > #
> > # MemoryStick Host Controller Drivers
> > #
> > # CONFIG_MEMSTICK_TIFM_MS is not set
> > # CONFIG_MEMSTICK_JMICRON_38X is not set
> > # CONFIG_MEMSTICK_R592 is not set
> > # CONFIG_MEMSTICK_REALTEK_USB is not set
> > CONFIG_NEW_LEDS=y
> > CONFIG_LEDS_CLASS=m
> > CONFIG_LEDS_CLASS_FLASH=m
> > # CONFIG_LEDS_BRIGHTNESS_HW_CHANGED is not set
> > 
> > #
> > # LED drivers
> > #
> > # CONFIG_LEDS_LM3530 is not set
> > CONFIG_LEDS_LM3533=m
> > CONFIG_LEDS_LM3642=m
> > # CONFIG_LEDS_PCA9532 is not set
> > # CONFIG_LEDS_GPIO is not set
> > CONFIG_LEDS_LP3944=m
> > # CONFIG_LEDS_LP3952 is not set
> > CONFIG_LEDS_LP55XX_COMMON=m
> > CONFIG_LEDS_LP5521=m
> > CONFIG_LEDS_LP5523=m
> > # CONFIG_LEDS_LP5562 is not set
> > CONFIG_LEDS_LP8501=m
> > CONFIG_LEDS_LP8788=m
> > CONFIG_LEDS_LP8860=m
> > # CONFIG_LEDS_CLEVO_MAIL is not set
> > CONFIG_LEDS_PCA955X=m
> > CONFIG_LEDS_PCA963X=m
> > CONFIG_LEDS_BD2802=m
> > # CONFIG_LEDS_INTEL_SS4200 is not set
> > # CONFIG_LEDS_LT3593 is not set
> > CONFIG_LEDS_ADP5520=m
> > # CONFIG_LEDS_TCA6507 is not set
> > CONFIG_LEDS_TLC591XX=m
> > CONFIG_LEDS_LM355x=m
> > CONFIG_LEDS_OT200=m
> > CONFIG_LEDS_MENF21BMC=m
> > 
> > #
> > # LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
> > #
> > CONFIG_LEDS_BLINKM=m
> > CONFIG_LEDS_USER=m
> > # CONFIG_LEDS_NIC78BX is not set
> > 
> > #
> > # LED Triggers
> > #
> > CONFIG_LEDS_TRIGGERS=y
> > CONFIG_LEDS_TRIGGER_TIMER=y
> > # CONFIG_LEDS_TRIGGER_ONESHOT is not set
> > # CONFIG_LEDS_TRIGGER_DISK is not set
> > CONFIG_LEDS_TRIGGER_MTD=y
> > # CONFIG_LEDS_TRIGGER_HEARTBEAT is not set
> > # CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
> > # CONFIG_LEDS_TRIGGER_CPU is not set
> > CONFIG_LEDS_TRIGGER_GPIO=m
> > CONFIG_LEDS_TRIGGER_DEFAULT_ON=m
> > 
> > #
> > # iptables trigger is under Netfilter config (LED target)
> > #
> > CONFIG_LEDS_TRIGGER_TRANSIENT=m
> > CONFIG_LEDS_TRIGGER_CAMERA=m
> > CONFIG_LEDS_TRIGGER_PANIC=y
> > CONFIG_ACCESSIBILITY=y
> > # CONFIG_INFINIBAND is not set
> > CONFIG_EDAC_ATOMIC_SCRUB=y
> > CONFIG_EDAC_SUPPORT=y
> > # CONFIG_EDAC is not set
> > CONFIG_RTC_LIB=y
> > CONFIG_RTC_MC146818_LIB=y
> > CONFIG_RTC_CLASS=y
> > # CONFIG_RTC_HCTOSYS is not set
> > CONFIG_RTC_SYSTOHC=y
> > CONFIG_RTC_SYSTOHC_DEVICE="rtc0"
> > # CONFIG_RTC_DEBUG is not set
> > 
> > #
> > # RTC interfaces
> > #
> > # CONFIG_RTC_INTF_SYSFS is not set
> > CONFIG_RTC_INTF_PROC=y
> > # CONFIG_RTC_INTF_DEV is not set
> > CONFIG_RTC_DRV_TEST=m
> > 
> > #
> > # I2C RTC drivers
> > #
> > # CONFIG_RTC_DRV_88PM80X is not set
> > CONFIG_RTC_DRV_ABB5ZES3=y
> > # CONFIG_RTC_DRV_ABX80X is not set
> > CONFIG_RTC_DRV_DS1307=m
> > CONFIG_RTC_DRV_DS1307_HWMON=y
> > # CONFIG_RTC_DRV_DS1307_CENTURY is not set
> > CONFIG_RTC_DRV_DS1374=m
> > # CONFIG_RTC_DRV_DS1374_WDT is not set
> > CONFIG_RTC_DRV_DS1672=y
> > # CONFIG_RTC_DRV_LP8788 is not set
> > CONFIG_RTC_DRV_MAX6900=y
> > # CONFIG_RTC_DRV_MAX8998 is not set
> > CONFIG_RTC_DRV_RS5C372=m
> > CONFIG_RTC_DRV_ISL1208=m
> > # CONFIG_RTC_DRV_ISL12022 is not set
> > CONFIG_RTC_DRV_X1205=y
> > # CONFIG_RTC_DRV_PCF8523 is not set
> > CONFIG_RTC_DRV_PCF85063=m
> > CONFIG_RTC_DRV_PCF8563=m
> > CONFIG_RTC_DRV_PCF8583=m
> > # CONFIG_RTC_DRV_M41T80 is not set
> > CONFIG_RTC_DRV_BQ32K=m
> > # CONFIG_RTC_DRV_PALMAS is not set
> > CONFIG_RTC_DRV_TPS6586X=m
> > CONFIG_RTC_DRV_TPS65910=y
> > CONFIG_RTC_DRV_TPS80031=m
> > CONFIG_RTC_DRV_RC5T583=y
> > # CONFIG_RTC_DRV_S35390A is not set
> > CONFIG_RTC_DRV_FM3130=y
> > # CONFIG_RTC_DRV_RX8010 is not set
> > CONFIG_RTC_DRV_RX8581=y
> > CONFIG_RTC_DRV_RX8025=y
> > CONFIG_RTC_DRV_EM3027=m
> > # CONFIG_RTC_DRV_RV8803 is not set
> > CONFIG_RTC_DRV_S5M=y
> > 
> > #
> > # SPI RTC drivers
> > #
> > CONFIG_RTC_I2C_AND_SPI=y
> > 
> > #
> > # SPI and I2C RTC drivers
> > #
> > CONFIG_RTC_DRV_DS3232=y
> > CONFIG_RTC_DRV_PCF2127=m
> > CONFIG_RTC_DRV_RV3029C2=m
> > CONFIG_RTC_DRV_RV3029_HWMON=y
> > 
> > #
> > # Platform RTC drivers
> > #
> > # CONFIG_RTC_DRV_CMOS is not set
> > CONFIG_RTC_DRV_DS1286=m
> > # CONFIG_RTC_DRV_DS1511 is not set
> > CONFIG_RTC_DRV_DS1553=m
> > CONFIG_RTC_DRV_DS1685_FAMILY=y
> > # CONFIG_RTC_DRV_DS1685 is not set
> > # CONFIG_RTC_DRV_DS1689 is not set
> > # CONFIG_RTC_DRV_DS17285 is not set
> > # CONFIG_RTC_DRV_DS17485 is not set
> > CONFIG_RTC_DRV_DS17885=y
> > # CONFIG_RTC_DS1685_PROC_REGS is not set
> > CONFIG_RTC_DS1685_SYSFS_REGS=y
> > CONFIG_RTC_DRV_DS1742=m
> > # CONFIG_RTC_DRV_DS2404 is not set
> > CONFIG_RTC_DRV_DA9055=m
> > CONFIG_RTC_DRV_DA9063=y
> > CONFIG_RTC_DRV_STK17TA8=m
> > # CONFIG_RTC_DRV_M48T86 is not set
> > CONFIG_RTC_DRV_M48T35=m
> > CONFIG_RTC_DRV_M48T59=m
> > # CONFIG_RTC_DRV_MSM6242 is not set
> > CONFIG_RTC_DRV_BQ4802=y
> > # CONFIG_RTC_DRV_RP5C01 is not set
> > # CONFIG_RTC_DRV_V3020 is not set
> > CONFIG_RTC_DRV_PCF50633=y
> > 
> > #
> > # on-CPU RTC drivers
> > #
> > # CONFIG_RTC_DRV_MT6397 is not set
> > 
> > #
> > # HID Sensor RTC drivers
> > #
> > # CONFIG_RTC_DRV_HID_SENSOR_TIME is not set
> > CONFIG_DMADEVICES=y
> > # CONFIG_DMADEVICES_DEBUG is not set
> > 
> > #
> > # DMA Devices
> > #
> > CONFIG_DMA_ENGINE=y
> > CONFIG_DMA_VIRTUAL_CHANNELS=y
> > CONFIG_DMA_ACPI=y
> > # CONFIG_INTEL_IDMA64 is not set
> > # CONFIG_PCH_DMA is not set
> > CONFIG_QCOM_HIDMA_MGMT=y
> > CONFIG_QCOM_HIDMA=y
> > CONFIG_DW_DMAC_CORE=y
> > CONFIG_DW_DMAC=y
> > # CONFIG_DW_DMAC_PCI is not set
> > CONFIG_HSU_DMA=y
> > 
> > #
> > # DMA Clients
> > #
> > CONFIG_ASYNC_TX_DMA=y
> > CONFIG_DMATEST=y
> > 
> > #
> > # DMABUF options
> > #
> > CONFIG_SYNC_FILE=y
> > CONFIG_SW_SYNC=y
> > # CONFIG_AUXDISPLAY is not set
> > CONFIG_UIO=y
> > # CONFIG_UIO_CIF is not set
> > # CONFIG_UIO_PDRV_GENIRQ is not set
> > CONFIG_UIO_DMEM_GENIRQ=m
> > # CONFIG_UIO_AEC is not set
> > # CONFIG_UIO_SERCOS3 is not set
> > # CONFIG_UIO_PCI_GENERIC is not set
> > # CONFIG_UIO_NETX is not set
> > CONFIG_UIO_PRUSS=m
> > # CONFIG_UIO_MF624 is not set
> > CONFIG_VIRT_DRIVERS=y
> > CONFIG_VIRTIO=y
> > 
> > #
> > # Virtio drivers
> > #
> > # CONFIG_VIRTIO_PCI is not set
> > CONFIG_VIRTIO_BALLOON=m
> > # CONFIG_VIRTIO_INPUT is not set
> > # CONFIG_VIRTIO_MMIO is not set
> > 
> > #
> > # Microsoft Hyper-V guest support
> > #
> > # CONFIG_STAGING is not set
> > # CONFIG_X86_PLATFORM_DEVICES is not set
> > CONFIG_PMC_ATOM=y
> > CONFIG_CHROME_PLATFORMS=y
> > CONFIG_CHROMEOS_LAPTOP=y
> > CONFIG_CHROMEOS_PSTORE=m
> > CONFIG_CROS_EC_CHARDEV=m
> > CONFIG_CROS_EC_LPC=m
> > CONFIG_CROS_EC_PROTO=y
> > # CONFIG_CROS_KBD_LED_BACKLIGHT is not set
> > CONFIG_CLKDEV_LOOKUP=y
> > CONFIG_HAVE_CLK_PREPARE=y
> > CONFIG_COMMON_CLK=y
> > 
> > #
> > # Common Clock Framework
> > #
> > # CONFIG_COMMON_CLK_SI5351 is not set
> > # CONFIG_COMMON_CLK_CDCE706 is not set
> > # CONFIG_COMMON_CLK_CS2000_CP is not set
> > # CONFIG_COMMON_CLK_S2MPS11 is not set
> > # CONFIG_CLK_TWL6040 is not set
> > # CONFIG_COMMON_CLK_NXP is not set
> > # CONFIG_COMMON_CLK_PALMAS is not set
> > # CONFIG_COMMON_CLK_PXA is not set
> > # CONFIG_COMMON_CLK_PIC32 is not set
> > 
> > #
> > # Hardware Spinlock drivers
> > #
> > 
> > #
> > # Clock Source drivers
> > #
> > CONFIG_CLKSRC_I8253=y
> > CONFIG_CLKEVT_I8253=y
> > CONFIG_I8253_LOCK=y
> > CONFIG_CLKBLD_I8253=y
> > # CONFIG_ATMEL_PIT is not set
> > # CONFIG_SH_TIMER_CMT is not set
> > # CONFIG_SH_TIMER_MTU2 is not set
> > # CONFIG_SH_TIMER_TMU is not set
> > # CONFIG_EM_TIMER_STI is not set
> > # CONFIG_MAILBOX is not set
> > CONFIG_IOMMU_SUPPORT=y
> > 
> > #
> > # Generic IOMMU Pagetable Support
> > #
> > 
> > #
> > # Remoteproc drivers
> > #
> > CONFIG_REMOTEPROC=y
> > 
> > #
> > # Rpmsg drivers
> > #
> > 
> > #
> > # SOC (System On Chip) specific Drivers
> > #
> > 
> > #
> > # Broadcom SoC drivers
> > #
> > # CONFIG_SUNXI_SRAM is not set
> > CONFIG_SOC_TI=y
> > # CONFIG_SOC_ZTE is not set
> > CONFIG_PM_DEVFREQ=y
> > 
> > #
> > # DEVFREQ Governors
> > #
> > CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y
> > # CONFIG_DEVFREQ_GOV_PERFORMANCE is not set
> > CONFIG_DEVFREQ_GOV_POWERSAVE=m
> > CONFIG_DEVFREQ_GOV_USERSPACE=y
> > # CONFIG_DEVFREQ_GOV_PASSIVE is not set
> > 
> > #
> > # DEVFREQ Drivers
> > #
> > # CONFIG_PM_DEVFREQ_EVENT is not set
> > CONFIG_EXTCON=y
> > 
> > #
> > # Extcon Device Drivers
> > #
> > CONFIG_EXTCON_ADC_JACK=y
> > # CONFIG_EXTCON_AXP288 is not set
> > CONFIG_EXTCON_GPIO=y
> > # CONFIG_EXTCON_INTEL_INT3496 is not set
> > CONFIG_EXTCON_MAX3355=m
> > # CONFIG_EXTCON_MAX77693 is not set
> > # CONFIG_EXTCON_MAX77843 is not set
> > # CONFIG_EXTCON_PALMAS is not set
> > CONFIG_EXTCON_QCOM_SPMI_MISC=y
> > CONFIG_EXTCON_RT8973A=y
> > # CONFIG_EXTCON_SM5502 is not set
> > CONFIG_EXTCON_USB_GPIO=m
> > # CONFIG_MEMORY is not set
> > CONFIG_IIO=y
> > CONFIG_IIO_BUFFER=y
> > CONFIG_IIO_BUFFER_CB=y
> > CONFIG_IIO_KFIFO_BUF=y
> > CONFIG_IIO_TRIGGERED_BUFFER=y
> > CONFIG_IIO_CONFIGFS=y
> > CONFIG_IIO_TRIGGER=y
> > CONFIG_IIO_CONSUMERS_PER_TRIGGER=2
> > CONFIG_IIO_SW_DEVICE=y
> > CONFIG_IIO_SW_TRIGGER=y
> > 
> > #
> > # Accelerometers
> > #
> > # CONFIG_BMA180 is not set
> > CONFIG_BMC150_ACCEL=y
> > CONFIG_BMC150_ACCEL_I2C=y
> > CONFIG_DA280=y
> > # CONFIG_DA311 is not set
> > CONFIG_DMARD09=m
> > CONFIG_DMARD10=m
> > CONFIG_IIO_ST_ACCEL_3AXIS=m
> > CONFIG_IIO_ST_ACCEL_I2C_3AXIS=m
> > CONFIG_KXSD9=y
> > # CONFIG_KXSD9_I2C is not set
> > CONFIG_KXCJK1013=y
> > CONFIG_MC3230=y
> > CONFIG_MMA7455=m
> > CONFIG_MMA7455_I2C=m
> > # CONFIG_MMA7660 is not set
> > # CONFIG_MMA8452 is not set
> > CONFIG_MMA9551_CORE=y
> > # CONFIG_MMA9551 is not set
> > CONFIG_MMA9553=y
> > CONFIG_MXC4005=y
> > CONFIG_MXC6255=y
> > # CONFIG_STK8312 is not set
> > # CONFIG_STK8BA50 is not set
> > 
> > #
> > # Analog to digital converters
> > #
> > CONFIG_AD7291=y
> > CONFIG_AD799X=y
> > CONFIG_AXP288_ADC=m
> > # CONFIG_HX711 is not set
> > CONFIG_INA2XX_ADC=m
> > CONFIG_LP8788_ADC=m
> > # CONFIG_LTC2485 is not set
> > # CONFIG_MAX1363 is not set
> > # CONFIG_MCP3422 is not set
> > CONFIG_NAU7802=m
> > CONFIG_PALMAS_GPADC=y
> > CONFIG_QCOM_SPMI_IADC=m
> > # CONFIG_QCOM_SPMI_VADC is not set
> > CONFIG_STX104=y
> > CONFIG_TI_ADC081C=m
> > CONFIG_TI_ADS1015=m
> > CONFIG_TI_AM335X_ADC=m
> > 
> > #
> > # Amplifiers
> > #
> > 
> > #
> > # Chemical Sensors
> > #
> > CONFIG_ATLAS_PH_SENSOR=m
> > CONFIG_IAQCORE=y
> > CONFIG_VZ89X=m
> > CONFIG_IIO_CROS_EC_SENSORS_CORE=m
> > CONFIG_IIO_CROS_EC_SENSORS=m
> > 
> > #
> > # Hid Sensor IIO Common
> > #
> > CONFIG_IIO_MS_SENSORS_I2C=y
> > 
> > #
> > # SSP Sensor Common
> > #
> > CONFIG_IIO_ST_SENSORS_I2C=y
> > CONFIG_IIO_ST_SENSORS_CORE=y
> > 
> > #
> > # Counters
> > #
> > # CONFIG_104_QUAD_8 is not set
> > 
> > #
> > # Digital to analog converters
> > #
> > CONFIG_AD5064=m
> > CONFIG_AD5380=y
> > CONFIG_AD5446=y
> > # CONFIG_AD5593R is not set
> > CONFIG_CIO_DAC=y
> > # CONFIG_M62332 is not set
> > # CONFIG_MAX517 is not set
> > # CONFIG_MCP4725 is not set
> > 
> > #
> > # IIO dummy driver
> > #
> > CONFIG_IIO_SIMPLE_DUMMY=m
> > # CONFIG_IIO_SIMPLE_DUMMY_EVENTS is not set
> > # CONFIG_IIO_SIMPLE_DUMMY_BUFFER is not set
> > 
> > #
> > # Frequency Synthesizers DDS/PLL
> > #
> > 
> > #
> > # Clock Generator/Distribution
> > #
> > 
> > #
> > # Phase-Locked Loop (PLL) frequency synthesizers
> > #
> > 
> > #
> > # Digital gyroscope sensors
> > #
> > CONFIG_BMG160=y
> > CONFIG_BMG160_I2C=y
> > # CONFIG_MPU3050_I2C is not set
> > CONFIG_IIO_ST_GYRO_3AXIS=y
> > CONFIG_IIO_ST_GYRO_I2C_3AXIS=y
> > CONFIG_ITG3200=y
> > 
> > #
> > # Health Sensors
> > #
> > 
> > #
> > # Heart Rate Monitors
> > #
> > # CONFIG_AFE4404 is not set
> > # CONFIG_MAX30100 is not set
> > 
> > #
> > # Humidity sensors
> > #
> > CONFIG_AM2315=y
> > CONFIG_DHT11=m
> > CONFIG_HDC100X=m
> > CONFIG_HTS221=y
> > CONFIG_HTS221_I2C=y
> > CONFIG_HTU21=m
> > CONFIG_SI7005=m
> > CONFIG_SI7020=m
> > 
> > #
> > # Inertial measurement units
> > #
> > CONFIG_BMI160=y
> > CONFIG_BMI160_I2C=y
> > CONFIG_KMX61=m
> > # CONFIG_INV_MPU6050_I2C is not set
> > # CONFIG_IIO_ST_LSM6DSX is not set
> > 
> > #
> > # Light sensors
> > #
> > # CONFIG_ACPI_ALS is not set
> > # CONFIG_ADJD_S311 is not set
> > CONFIG_AL3320A=y
> > # CONFIG_APDS9300 is not set
> > CONFIG_APDS9960=m
> > # CONFIG_BH1750 is not set
> > CONFIG_BH1780=m
> > CONFIG_CM32181=m
> > CONFIG_CM3232=y
> > # CONFIG_CM3323 is not set
> > # CONFIG_CM36651 is not set
> > # CONFIG_GP2AP020A00F is not set
> > # CONFIG_SENSORS_ISL29018 is not set
> > CONFIG_ISL29125=m
> > CONFIG_JSA1212=y
> > CONFIG_RPR0521=m
> > CONFIG_SENSORS_LM3533=m
> > # CONFIG_LTR501 is not set
> > CONFIG_MAX44000=y
> > # CONFIG_OPT3001 is not set
> > CONFIG_PA12203001=y
> > CONFIG_SI1145=y
> > # CONFIG_STK3310 is not set
> > # CONFIG_TCS3414 is not set
> > CONFIG_TCS3472=m
> > CONFIG_SENSORS_TSL2563=y
> > CONFIG_TSL2583=y
> > CONFIG_TSL4531=m
> > CONFIG_US5182D=y
> > CONFIG_VCNL4000=m
> > # CONFIG_VEML6070 is not set
> > 
> > #
> > # Magnetometer sensors
> > #
> > CONFIG_AK8975=m
> > # CONFIG_AK09911 is not set
> > # CONFIG_BMC150_MAGN_I2C is not set
> > # CONFIG_MAG3110 is not set
> > CONFIG_MMC35240=y
> > # CONFIG_IIO_ST_MAGN_3AXIS is not set
> > # CONFIG_SENSORS_HMC5843_I2C is not set
> > 
> > #
> > # Inclinometer sensors
> > #
> > 
> > #
> > # Triggers - standalone
> > #
> > CONFIG_IIO_HRTIMER_TRIGGER=m
> > CONFIG_IIO_INTERRUPT_TRIGGER=y
> > CONFIG_IIO_TIGHTLOOP_TRIGGER=y
> > CONFIG_IIO_SYSFS_TRIGGER=m
> > 
> > #
> > # Digital potentiometers
> > #
> > # CONFIG_DS1803 is not set
> > # CONFIG_MCP4531 is not set
> > # CONFIG_TPL0102 is not set
> > 
> > #
> > # Digital potentiostats
> > #
> > CONFIG_LMP91000=y
> > 
> > #
> > # Pressure sensors
> > #
> > CONFIG_ABP060MG=m
> > CONFIG_BMP280=y
> > CONFIG_BMP280_I2C=y
> > # CONFIG_IIO_CROS_EC_BARO is not set
> > # CONFIG_HP03 is not set
> > # CONFIG_MPL115_I2C is not set
> > # CONFIG_MPL3115 is not set
> > CONFIG_MS5611=m
> > CONFIG_MS5611_I2C=m
> > CONFIG_MS5637=m
> > # CONFIG_IIO_ST_PRESS is not set
> > # CONFIG_T5403 is not set
> > # CONFIG_HP206C is not set
> > CONFIG_ZPA2326=y
> > CONFIG_ZPA2326_I2C=y
> > 
> > #
> > # Lightning sensors
> > #
> > 
> > #
> > # Proximity and distance sensors
> > #
> > # CONFIG_LIDAR_LITE_V2 is not set
> > # CONFIG_SX9500 is not set
> > # CONFIG_SRF08 is not set
> > 
> > #
> > # Temperature sensors
> > #
> > CONFIG_MLX90614=y
> > # CONFIG_TMP006 is not set
> > # CONFIG_TMP007 is not set
> > CONFIG_TSYS01=y
> > # CONFIG_TSYS02D is not set
> > # CONFIG_NTB is not set
> > # CONFIG_VME_BUS is not set
> > # CONFIG_PWM is not set
> > CONFIG_ARM_GIC_MAX_NR=1
> > CONFIG_IPACK_BUS=y
> > # CONFIG_BOARD_TPCI200 is not set
> > # CONFIG_SERIAL_IPOCTAL is not set
> > CONFIG_RESET_CONTROLLER=y
> > # CONFIG_RESET_ATH79 is not set
> > # CONFIG_RESET_BERLIN is not set
> > # CONFIG_RESET_LPC18XX is not set
> > # CONFIG_RESET_MESON is not set
> > # CONFIG_RESET_PISTACHIO is not set
> > # CONFIG_RESET_SOCFPGA is not set
> > # CONFIG_RESET_STM32 is not set
> > # CONFIG_RESET_SUNXI is not set
> > CONFIG_TI_SYSCON_RESET=m
> > # CONFIG_RESET_ZYNQ is not set
> > # CONFIG_RESET_TEGRA_BPMP is not set
> > CONFIG_FMC=m
> > CONFIG_FMC_FAKEDEV=m
> > CONFIG_FMC_TRIVIAL=m
> > CONFIG_FMC_WRITE_EEPROM=m
> > CONFIG_FMC_CHARDEV=m
> > 
> > #
> > # PHY Subsystem
> > #
> > CONFIG_GENERIC_PHY=y
> > # CONFIG_PHY_PXA_28NM_HSIC is not set
> > # CONFIG_PHY_PXA_28NM_USB2 is not set
> > CONFIG_BCM_KONA_USB2_PHY=m
> > # CONFIG_PHY_QCOM_USB_HS is not set
> > # CONFIG_PHY_QCOM_USB_HSIC is not set
> > CONFIG_PHY_TUSB1210=m
> > # CONFIG_POWERCAP is not set
> > # CONFIG_MCB is not set
> > 
> > #
> > # Performance monitor support
> > #
> > CONFIG_RAS=y
> > # CONFIG_THUNDERBOLT is not set
> > 
> > #
> > # Android
> > #
> > # CONFIG_ANDROID is not set
> > # CONFIG_LIBNVDIMM is not set
> > CONFIG_DEV_DAX=y
> > CONFIG_NR_DEV_DAX=32768
> > CONFIG_NVMEM=y
> > CONFIG_STM=y
> > # CONFIG_STM_DUMMY is not set
> > CONFIG_STM_SOURCE_CONSOLE=y
> > # CONFIG_STM_SOURCE_HEARTBEAT is not set
> > CONFIG_INTEL_TH=m
> > # CONFIG_INTEL_TH_PCI is not set
> > # CONFIG_INTEL_TH_GTH is not set
> > # CONFIG_INTEL_TH_STH is not set
> > # CONFIG_INTEL_TH_MSU is not set
> > CONFIG_INTEL_TH_PTI=m
> > # CONFIG_INTEL_TH_DEBUG is not set
> > 
> > #
> > # FPGA Configuration Support
> > #
> > CONFIG_FPGA=y
> > 
> > #
> > # FSI support
> > #
> > # CONFIG_FSI is not set
> > 
> > #
> > # Firmware Drivers
> > #
> > CONFIG_EDD=m
> > # CONFIG_EDD_OFF is not set
> > CONFIG_FIRMWARE_MEMMAP=y
> > # CONFIG_DELL_RBU is not set
> > CONFIG_DCDBAS=y
> > CONFIG_DMIID=y
> > CONFIG_DMI_SYSFS=m
> > CONFIG_DMI_SCAN_MACHINE_NON_EFI_FALLBACK=y
> > # CONFIG_ISCSI_IBFT_FIND is not set
> > # CONFIG_FW_CFG_SYSFS is not set
> > # CONFIG_GOOGLE_FIRMWARE is not set
> > # CONFIG_EFI_DEV_PATH_PARSER is not set
> > 
> > #
> > # Tegra firmware driver
> > #
> > 
> > #
> > # File systems
> > #
> > CONFIG_DCACHE_WORD_ACCESS=y
> > CONFIG_EXT2_FS=y
> > # CONFIG_EXT2_FS_XATTR is not set
> > CONFIG_EXT3_FS=y
> > CONFIG_EXT3_FS_POSIX_ACL=y
> > CONFIG_EXT3_FS_SECURITY=y
> > CONFIG_EXT4_FS=y
> > CONFIG_EXT4_FS_POSIX_ACL=y
> > CONFIG_EXT4_FS_SECURITY=y
> > CONFIG_EXT4_ENCRYPTION=y
> > CONFIG_EXT4_FS_ENCRYPTION=y
> > CONFIG_EXT4_DEBUG=y
> > CONFIG_JBD2=y
> > CONFIG_JBD2_DEBUG=y
> > CONFIG_FS_MBCACHE=y
> > CONFIG_REISERFS_FS=y
> > # CONFIG_REISERFS_CHECK is not set
> > # CONFIG_REISERFS_PROC_INFO is not set
> > CONFIG_REISERFS_FS_XATTR=y
> > CONFIG_REISERFS_FS_POSIX_ACL=y
> > # CONFIG_REISERFS_FS_SECURITY is not set
> > CONFIG_JFS_FS=m
> > CONFIG_JFS_POSIX_ACL=y
> > CONFIG_JFS_SECURITY=y
> > # CONFIG_JFS_DEBUG is not set
> > # CONFIG_JFS_STATISTICS is not set
> > CONFIG_OCFS2_FS=y
> > CONFIG_OCFS2_FS_O2CB=y
> > CONFIG_OCFS2_FS_USERSPACE_CLUSTER=y
> > # CONFIG_OCFS2_FS_STATS is not set
> > CONFIG_OCFS2_DEBUG_MASKLOG=y
> > CONFIG_OCFS2_DEBUG_FS=y
> > CONFIG_BTRFS_FS=y
> > # CONFIG_BTRFS_FS_POSIX_ACL is not set
> > CONFIG_BTRFS_FS_CHECK_INTEGRITY=y
> > # CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
> > # CONFIG_BTRFS_DEBUG is not set
> > # CONFIG_BTRFS_ASSERT is not set
> > CONFIG_NILFS2_FS=y
> > CONFIG_F2FS_FS=m
> > # CONFIG_F2FS_STAT_FS is not set
> > # CONFIG_F2FS_FS_XATTR is not set
> > CONFIG_F2FS_CHECK_FS=y
> > CONFIG_F2FS_FAULT_INJECTION=y
> > # CONFIG_FS_DAX is not set
> > CONFIG_FS_POSIX_ACL=y
> > CONFIG_EXPORTFS=y
> > CONFIG_EXPORTFS_BLOCK_OPS=y
> > CONFIG_FILE_LOCKING=y
> > # CONFIG_MANDATORY_FILE_LOCKING is not set
> > CONFIG_FS_ENCRYPTION=y
> > CONFIG_FSNOTIFY=y
> > CONFIG_DNOTIFY=y
> > CONFIG_INOTIFY_USER=y
> > # CONFIG_FANOTIFY is not set
> > CONFIG_QUOTA=y
> > CONFIG_QUOTA_NETLINK_INTERFACE=y
> > # CONFIG_PRINT_QUOTA_WARNING is not set
> > CONFIG_QUOTA_DEBUG=y
> > CONFIG_QUOTA_TREE=y
> > # CONFIG_QFMT_V1 is not set
> > # CONFIG_QFMT_V2 is not set
> > CONFIG_QUOTACTL=y
> > CONFIG_AUTOFS4_FS=m
> > CONFIG_FUSE_FS=y
> > # CONFIG_CUSE is not set
> > # CONFIG_OVERLAY_FS is not set
> > 
> > #
> > # Caches
> > #
> > CONFIG_FSCACHE=m
> > # CONFIG_FSCACHE_STATS is not set
> > # CONFIG_FSCACHE_HISTOGRAM is not set
> > CONFIG_FSCACHE_DEBUG=y
> > # CONFIG_FSCACHE_OBJECT_LIST is not set
> > CONFIG_CACHEFILES=m
> > CONFIG_CACHEFILES_DEBUG=y
> > # CONFIG_CACHEFILES_HISTOGRAM is not set
> > 
> > #
> > # CD-ROM/DVD Filesystems
> > #
> > CONFIG_ISO9660_FS=y
> > # CONFIG_JOLIET is not set
> > # CONFIG_ZISOFS is not set
> > # CONFIG_UDF_FS is not set
> > 
> > #
> > # DOS/FAT/NT Filesystems
> > #
> > CONFIG_FAT_FS=y
> > CONFIG_MSDOS_FS=m
> > CONFIG_VFAT_FS=y
> > CONFIG_FAT_DEFAULT_CODEPAGE=437
> > CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
> > CONFIG_FAT_DEFAULT_UTF8=y
> > CONFIG_NTFS_FS=y
> > CONFIG_NTFS_DEBUG=y
> > CONFIG_NTFS_RW=y
> > 
> > #
> > # Pseudo filesystems
> > #
> > CONFIG_PROC_FS=y
> > # CONFIG_PROC_KCORE is not set
> > CONFIG_PROC_SYSCTL=y
> > CONFIG_PROC_PAGE_MONITOR=y
> > CONFIG_PROC_CHILDREN=y
> > CONFIG_KERNFS=y
> > CONFIG_SYSFS=y
> > CONFIG_TMPFS=y
> > # CONFIG_TMPFS_POSIX_ACL is not set
> > # CONFIG_TMPFS_XATTR is not set
> > # CONFIG_HUGETLBFS is not set
> > # CONFIG_HUGETLB_PAGE is not set
> > CONFIG_CONFIGFS_FS=y
> > # CONFIG_MISC_FILESYSTEMS is not set
> > CONFIG_NETWORK_FILESYSTEMS=y
> > CONFIG_NFS_FS=m
> > CONFIG_NFS_V2=m
> > CONFIG_NFS_V3=m
> > # CONFIG_NFS_V3_ACL is not set
> > CONFIG_NFS_V4=m
> > # CONFIG_NFS_SWAP is not set
> > # CONFIG_NFS_V4_1 is not set
> > # CONFIG_NFS_FSCACHE is not set
> > # CONFIG_NFS_USE_LEGACY_DNS is not set
> > CONFIG_NFS_USE_KERNEL_DNS=y
> > # CONFIG_NFSD is not set
> > CONFIG_GRACE_PERIOD=m
> > CONFIG_LOCKD=m
> > CONFIG_LOCKD_V4=y
> > CONFIG_NFS_COMMON=y
> > CONFIG_SUNRPC=m
> > CONFIG_SUNRPC_GSS=m
> > CONFIG_RPCSEC_GSS_KRB5=m
> > # CONFIG_SUNRPC_DEBUG is not set
> > CONFIG_CEPH_FS=y
> > CONFIG_CEPH_FS_POSIX_ACL=y
> > CONFIG_CIFS=m
> > # CONFIG_CIFS_STATS is not set
> > # CONFIG_CIFS_WEAK_PW_HASH is not set
> > CONFIG_CIFS_UPCALL=y
> > # CONFIG_CIFS_XATTR is not set
> > # CONFIG_CIFS_DEBUG is not set
> > # CONFIG_CIFS_DFS_UPCALL is not set
> > CONFIG_CIFS_SMB2=y
> > # CONFIG_CIFS_SMB311 is not set
> > # CONFIG_CIFS_FSCACHE is not set
> > CONFIG_NCP_FS=y
> > # CONFIG_NCPFS_PACKET_SIGNING is not set
> > # CONFIG_NCPFS_IOCTL_LOCKING is not set
> > CONFIG_NCPFS_STRONG=y
> > CONFIG_NCPFS_NFS_NS=y
> > CONFIG_NCPFS_OS2_NS=y
> > # CONFIG_NCPFS_SMALLDOS is not set
> > # CONFIG_NCPFS_NLS is not set
> > # CONFIG_NCPFS_EXTRAS is not set
> > CONFIG_CODA_FS=m
> > # CONFIG_AFS_FS is not set
> > # CONFIG_9P_FS is not set
> > CONFIG_NLS=y
> > CONFIG_NLS_DEFAULT="iso8859-1"
> > CONFIG_NLS_CODEPAGE_437=m
> > CONFIG_NLS_CODEPAGE_737=y
> > CONFIG_NLS_CODEPAGE_775=y
> > # CONFIG_NLS_CODEPAGE_850 is not set
> > # CONFIG_NLS_CODEPAGE_852 is not set
> > CONFIG_NLS_CODEPAGE_855=y
> > CONFIG_NLS_CODEPAGE_857=y
> > CONFIG_NLS_CODEPAGE_860=y
> > # CONFIG_NLS_CODEPAGE_861 is not set
> > # CONFIG_NLS_CODEPAGE_862 is not set
> > CONFIG_NLS_CODEPAGE_863=y
> > CONFIG_NLS_CODEPAGE_864=m
> > CONFIG_NLS_CODEPAGE_865=y
> > # CONFIG_NLS_CODEPAGE_866 is not set
> > CONFIG_NLS_CODEPAGE_869=y
> > CONFIG_NLS_CODEPAGE_936=m
> > CONFIG_NLS_CODEPAGE_950=y
> > # CONFIG_NLS_CODEPAGE_932 is not set
> > CONFIG_NLS_CODEPAGE_949=y
> > CONFIG_NLS_CODEPAGE_874=m
> > CONFIG_NLS_ISO8859_8=m
> > CONFIG_NLS_CODEPAGE_1250=y
> > # CONFIG_NLS_CODEPAGE_1251 is not set
> > CONFIG_NLS_ASCII=m
> > CONFIG_NLS_ISO8859_1=m
> > # CONFIG_NLS_ISO8859_2 is not set
> > CONFIG_NLS_ISO8859_3=y
> > CONFIG_NLS_ISO8859_4=m
> > # CONFIG_NLS_ISO8859_5 is not set
> > # CONFIG_NLS_ISO8859_6 is not set
> > CONFIG_NLS_ISO8859_7=m
> > # CONFIG_NLS_ISO8859_9 is not set
> > CONFIG_NLS_ISO8859_13=m
> > CONFIG_NLS_ISO8859_14=m
> > CONFIG_NLS_ISO8859_15=m
> > CONFIG_NLS_KOI8_R=m
> > # CONFIG_NLS_KOI8_U is not set
> > # CONFIG_NLS_MAC_ROMAN is not set
> > CONFIG_NLS_MAC_CELTIC=m
> > # CONFIG_NLS_MAC_CENTEURO is not set
> > # CONFIG_NLS_MAC_CROATIAN is not set
> > CONFIG_NLS_MAC_CYRILLIC=y
> > # CONFIG_NLS_MAC_GAELIC is not set
> > CONFIG_NLS_MAC_GREEK=m
> > # CONFIG_NLS_MAC_ICELAND is not set
> > CONFIG_NLS_MAC_INUIT=y
> > CONFIG_NLS_MAC_ROMANIAN=m
> > # CONFIG_NLS_MAC_TURKISH is not set
> > # CONFIG_NLS_UTF8 is not set
> > CONFIG_DLM=y
> > CONFIG_DLM_DEBUG=y
> > 
> > #
> > # Kernel hacking
> > #
> > CONFIG_TRACE_IRQFLAGS_SUPPORT=y
> > 
> > #
> > # printk and dmesg options
> > #
> > CONFIG_PRINTK_TIME=y
> > CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
> > CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
> > # CONFIG_BOOT_PRINTK_DELAY is not set
> > # CONFIG_DYNAMIC_DEBUG is not set
> > 
> > #
> > # Compile-time checks and compiler options
> > #
> > # CONFIG_DEBUG_INFO is not set
> > # CONFIG_ENABLE_WARN_DEPRECATED is not set
> > # CONFIG_ENABLE_MUST_CHECK is not set
> > CONFIG_FRAME_WARN=1024
> > CONFIG_STRIP_ASM_SYMS=y
> > # CONFIG_READABLE_ASM is not set
> > # CONFIG_UNUSED_SYMBOLS is not set
> > CONFIG_PAGE_OWNER=y
> > CONFIG_DEBUG_FS=y
> > CONFIG_HEADERS_CHECK=y
> > # CONFIG_DEBUG_SECTION_MISMATCH is not set
> > CONFIG_SECTION_MISMATCH_WARN_ONLY=y
> > CONFIG_ARCH_WANT_FRAME_POINTERS=y
> > CONFIG_FRAME_POINTER=y
> > CONFIG_DEBUG_FORCE_WEAK_PER_CPU=y
> > CONFIG_MAGIC_SYSRQ=y
> > CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
> > CONFIG_MAGIC_SYSRQ_SERIAL=y
> > CONFIG_DEBUG_KERNEL=y
> > 
> > #
> > # Memory Debugging
> > #
> > CONFIG_PAGE_EXTENSION=y
> > # CONFIG_DEBUG_PAGEALLOC is not set
> > CONFIG_PAGE_POISONING=y
> > # CONFIG_PAGE_POISONING_NO_SANITY is not set
> > # CONFIG_PAGE_POISONING_ZERO is not set
> > # CONFIG_DEBUG_RODATA_TEST is not set
> > CONFIG_DEBUG_OBJECTS=y
> > # CONFIG_DEBUG_OBJECTS_SELFTEST is not set
> > # CONFIG_DEBUG_OBJECTS_FREE is not set
> > CONFIG_DEBUG_OBJECTS_TIMERS=y
> > CONFIG_DEBUG_OBJECTS_WORK=y
> > CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
> > CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER=y
> > CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT=1
> > CONFIG_HAVE_DEBUG_KMEMLEAK=y
> > # CONFIG_DEBUG_KMEMLEAK is not set
> > # CONFIG_DEBUG_STACK_USAGE is not set
> > CONFIG_DEBUG_VM=y
> > # CONFIG_DEBUG_VM_VMACACHE is not set
> > CONFIG_DEBUG_VM_RB=y
> > # CONFIG_DEBUG_VM_PGFLAGS is not set
> > CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
> > CONFIG_DEBUG_VIRTUAL=y
> > # CONFIG_DEBUG_MEMORY_INIT is not set
> > CONFIG_DEBUG_HIGHMEM=y
> > CONFIG_HAVE_DEBUG_STACKOVERFLOW=y
> > CONFIG_DEBUG_STACKOVERFLOW=y
> > CONFIG_HAVE_ARCH_KMEMCHECK=y
> > # CONFIG_DEBUG_SHIRQ is not set
> > 
> > #
> > # Debug Lockups and Hangs
> > #
> > # CONFIG_LOCKUP_DETECTOR is not set
> > CONFIG_DETECT_HUNG_TASK=y
> > CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
> > CONFIG_BOOTPARAM_HUNG_TASK_PANIC=y
> > CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=1
> > CONFIG_WQ_WATCHDOG=y
> > CONFIG_PANIC_ON_OOPS=y
> > CONFIG_PANIC_ON_OOPS_VALUE=1
> > CONFIG_PANIC_TIMEOUT=0
> > CONFIG_SCHED_DEBUG=y
> > # CONFIG_SCHED_INFO is not set
> > # CONFIG_SCHEDSTATS is not set
> > # CONFIG_SCHED_STACK_END_CHECK is not set
> > CONFIG_DEBUG_TIMEKEEPING=y
> > CONFIG_DEBUG_PREEMPT=y
> > 
> > #
> > # Lock Debugging (spinlocks, mutexes, etc...)
> > #
> > CONFIG_DEBUG_RT_MUTEXES=y
> > CONFIG_DEBUG_SPINLOCK=y
> > CONFIG_DEBUG_MUTEXES=y
> > CONFIG_DEBUG_WW_MUTEX_SLOWPATH=y
> > CONFIG_DEBUG_LOCK_ALLOC=y
> > CONFIG_PROVE_LOCKING=y
> > CONFIG_LOCKDEP=y
> > # CONFIG_LOCK_STAT is not set
> > # CONFIG_DEBUG_LOCKDEP is not set
> > CONFIG_DEBUG_ATOMIC_SLEEP=y
> > CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
> > CONFIG_LOCK_TORTURE_TEST=m
> > # CONFIG_WW_MUTEX_SELFTEST is not set
> > CONFIG_TRACE_IRQFLAGS=y
> > CONFIG_STACKTRACE=y
> > # CONFIG_DEBUG_KOBJECT is not set
> > # CONFIG_DEBUG_KOBJECT_RELEASE is not set
> > CONFIG_DEBUG_BUGVERBOSE=y
> > CONFIG_DEBUG_LIST=y
> > CONFIG_DEBUG_PI_LIST=y
> > CONFIG_DEBUG_SG=y
> > # CONFIG_DEBUG_NOTIFIERS is not set
> > CONFIG_DEBUG_CREDENTIALS=y
> > 
> > #
> > # RCU Debugging
> > #
> > CONFIG_PROVE_RCU=y
> > CONFIG_PROVE_RCU_REPEATEDLY=y
> > # CONFIG_SPARSE_RCU_POINTER is not set
> > CONFIG_TORTURE_TEST=y
> > # CONFIG_RCU_PERF_TEST is not set
> > CONFIG_RCU_TORTURE_TEST=y
> > # CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT is not set
> > # CONFIG_RCU_TORTURE_TEST_SLOW_INIT is not set
> > CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP=y
> > CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP_DELAY=3
> > CONFIG_RCU_CPU_STALL_TIMEOUT=21
> > CONFIG_RCU_TRACE=y
> > # CONFIG_RCU_EQS_DEBUG is not set
> > CONFIG_DEBUG_WQ_FORCE_RR_CPU=y
> > # CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
> > CONFIG_NOTIFIER_ERROR_INJECTION=y
> > CONFIG_PM_NOTIFIER_ERROR_INJECT=y
> > CONFIG_NETDEV_NOTIFIER_ERROR_INJECT=m
> > CONFIG_FAULT_INJECTION=y
> > CONFIG_FAIL_PAGE_ALLOC=y
> > # CONFIG_FAIL_MAKE_REQUEST is not set
> > CONFIG_FAIL_IO_TIMEOUT=y
> > CONFIG_FAIL_FUTEX=y
> > # CONFIG_FAULT_INJECTION_DEBUG_FS is not set
> > # CONFIG_LATENCYTOP is not set
> > CONFIG_USER_STACKTRACE_SUPPORT=y
> > CONFIG_HAVE_FUNCTION_TRACER=y
> > CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
> > CONFIG_HAVE_DYNAMIC_FTRACE=y
> > CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
> > CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
> > CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
> > CONFIG_HAVE_C_RECORDMCOUNT=y
> > CONFIG_TRACE_CLOCK=y
> > CONFIG_TRACING_SUPPORT=y
> > # CONFIG_FTRACE is not set
> > 
> > #
> > # Runtime Testing
> > #
> > # CONFIG_LKDTM is not set
> > # CONFIG_TEST_LIST_SORT is not set
> > # CONFIG_TEST_SORT is not set
> > # CONFIG_KPROBES_SANITY_TEST is not set
> > # CONFIG_BACKTRACE_SELF_TEST is not set
> > # CONFIG_RBTREE_TEST is not set
> > CONFIG_INTERVAL_TREE_TEST=m
> > CONFIG_PERCPU_TEST=m
> > # CONFIG_ATOMIC64_SELFTEST is not set
> > # CONFIG_ASYNC_RAID6_TEST is not set
> > CONFIG_TEST_HEXDUMP=y
> > CONFIG_TEST_STRING_HELPERS=m
> > # CONFIG_TEST_KSTRTOX is not set
> > CONFIG_TEST_PRINTF=m
> > CONFIG_TEST_BITMAP=m
> > CONFIG_TEST_UUID=m
> > CONFIG_TEST_RHASHTABLE=y
> > CONFIG_TEST_HASH=m
> > # CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
> > # CONFIG_DMA_API_DEBUG is not set
> > # CONFIG_TEST_LKM is not set
> > CONFIG_TEST_USER_COPY=m
> > CONFIG_TEST_BPF=m
> > CONFIG_TEST_FIRMWARE=y
> > # CONFIG_TEST_UDELAY is not set
> > # CONFIG_MEMTEST is not set
> > CONFIG_TEST_STATIC_KEYS=m
> > # CONFIG_BUG_ON_DATA_CORRUPTION is not set
> > # CONFIG_SAMPLES is not set
> > CONFIG_HAVE_ARCH_KGDB=y
> > # CONFIG_KGDB is not set
> > CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
> > # CONFIG_ARCH_WANTS_UBSAN_NO_NULL is not set
> > CONFIG_UBSAN=y
> > # CONFIG_UBSAN_SANITIZE_ALL is not set
> > # CONFIG_UBSAN_ALIGNMENT is not set
> > CONFIG_UBSAN_NULL=y
> > CONFIG_ARCH_HAS_DEVMEM_IS_ALLOWED=y
> > CONFIG_STRICT_DEVMEM=y
> > # CONFIG_IO_STRICT_DEVMEM is not set
> > CONFIG_X86_VERBOSE_BOOTUP=y
> > CONFIG_EARLY_PRINTK=y
> > # CONFIG_EARLY_PRINTK_DBGP is not set
> > CONFIG_X86_PTDUMP_CORE=y
> > CONFIG_X86_PTDUMP=m
> > # CONFIG_DEBUG_WX is not set
> > CONFIG_DOUBLEFAULT=y
> > # CONFIG_DEBUG_TLBFLUSH is not set
> > # CONFIG_IOMMU_STRESS is not set
> > CONFIG_HAVE_MMIOTRACE_SUPPORT=y
> > # CONFIG_X86_DECODER_SELFTEST is not set
> > CONFIG_IO_DELAY_TYPE_0X80=0
> > CONFIG_IO_DELAY_TYPE_0XED=1
> > CONFIG_IO_DELAY_TYPE_UDELAY=2
> > CONFIG_IO_DELAY_TYPE_NONE=3
> > CONFIG_IO_DELAY_0X80=y
> > # CONFIG_IO_DELAY_0XED is not set
> > # CONFIG_IO_DELAY_UDELAY is not set
> > # CONFIG_IO_DELAY_NONE is not set
> > CONFIG_DEFAULT_IO_DELAY_TYPE=0
> > CONFIG_DEBUG_BOOT_PARAMS=y
> > # CONFIG_CPA_DEBUG is not set
> > # CONFIG_OPTIMIZE_INLINING is not set
> > CONFIG_DEBUG_ENTRY=y
> > # CONFIG_X86_DEBUG_FPU is not set
> > # CONFIG_PUNIT_ATOM_DEBUG is not set
> > 
> > #
> > # Security options
> > #
> > CONFIG_KEYS=y
> > # CONFIG_PERSISTENT_KEYRINGS is not set
> > # CONFIG_BIG_KEYS is not set
> > CONFIG_TRUSTED_KEYS=m
> > CONFIG_ENCRYPTED_KEYS=m
> > # CONFIG_KEY_DH_OPERATIONS is not set
> > CONFIG_SECURITY_DMESG_RESTRICT=y
> > # CONFIG_SECURITY is not set
> > CONFIG_SECURITYFS=y
> > CONFIG_HAVE_ARCH_HARDENED_USERCOPY=y
> > # CONFIG_STATIC_USERMODEHELPER is not set
> > CONFIG_DEFAULT_SECURITY_DAC=y
> > CONFIG_DEFAULT_SECURITY=""
> > CONFIG_XOR_BLOCKS=y
> > CONFIG_ASYNC_CORE=y
> > CONFIG_ASYNC_MEMCPY=y
> > CONFIG_ASYNC_XOR=y
> > CONFIG_ASYNC_PQ=y
> > CONFIG_ASYNC_RAID6_RECOV=y
> > CONFIG_CRYPTO=y
> > 
> > #
> > # Crypto core or helper
> > #
> > CONFIG_CRYPTO_ALGAPI=y
> > CONFIG_CRYPTO_ALGAPI2=y
> > CONFIG_CRYPTO_AEAD=y
> > CONFIG_CRYPTO_AEAD2=y
> > CONFIG_CRYPTO_BLKCIPHER=y
> > CONFIG_CRYPTO_BLKCIPHER2=y
> > CONFIG_CRYPTO_HASH=y
> > CONFIG_CRYPTO_HASH2=y
> > CONFIG_CRYPTO_RNG=y
> > CONFIG_CRYPTO_RNG2=y
> > CONFIG_CRYPTO_RNG_DEFAULT=y
> > CONFIG_CRYPTO_AKCIPHER2=y
> > CONFIG_CRYPTO_AKCIPHER=y
> > CONFIG_CRYPTO_KPP2=y
> > CONFIG_CRYPTO_KPP=y
> > CONFIG_CRYPTO_ACOMP2=y
> > CONFIG_CRYPTO_RSA=y
> > CONFIG_CRYPTO_DH=y
> > # CONFIG_CRYPTO_ECDH is not set
> > CONFIG_CRYPTO_MANAGER=y
> > CONFIG_CRYPTO_MANAGER2=y
> > # CONFIG_CRYPTO_USER is not set
> > CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
> > CONFIG_CRYPTO_GF128MUL=y
> > CONFIG_CRYPTO_NULL=y
> > CONFIG_CRYPTO_NULL2=y
> > CONFIG_CRYPTO_WORKQUEUE=y
> > CONFIG_CRYPTO_CRYPTD=y
> > CONFIG_CRYPTO_MCRYPTD=m
> > CONFIG_CRYPTO_AUTHENC=y
> > # CONFIG_CRYPTO_TEST is not set
> > CONFIG_CRYPTO_ABLK_HELPER=y
> > CONFIG_CRYPTO_SIMD=y
> > CONFIG_CRYPTO_GLUE_HELPER_X86=y
> > CONFIG_CRYPTO_ENGINE=m
> > 
> > #
> > # Authenticated Encryption with Associated Data
> > #
> > CONFIG_CRYPTO_CCM=m
> > CONFIG_CRYPTO_GCM=m
> > CONFIG_CRYPTO_CHACHA20POLY1305=y
> > CONFIG_CRYPTO_SEQIV=y
> > CONFIG_CRYPTO_ECHAINIV=y
> > 
> > #
> > # Block modes
> > #
> > CONFIG_CRYPTO_CBC=y
> > CONFIG_CRYPTO_CTR=y
> > CONFIG_CRYPTO_CTS=y
> > CONFIG_CRYPTO_ECB=y
> > CONFIG_CRYPTO_LRW=y
> > CONFIG_CRYPTO_PCBC=m
> > CONFIG_CRYPTO_XTS=y
> > CONFIG_CRYPTO_KEYWRAP=m
> > 
> > #
> > # Hash modes
> > #
> > CONFIG_CRYPTO_CMAC=y
> > CONFIG_CRYPTO_HMAC=y
> > CONFIG_CRYPTO_XCBC=y
> > # CONFIG_CRYPTO_VMAC is not set
> > 
> > #
> > # Digest
> > #
> > CONFIG_CRYPTO_CRC32C=y
> > CONFIG_CRYPTO_CRC32C_INTEL=m
> > CONFIG_CRYPTO_CRC32=m
> > CONFIG_CRYPTO_CRC32_PCLMUL=y
> > CONFIG_CRYPTO_CRCT10DIF=y
> > CONFIG_CRYPTO_GHASH=y
> > CONFIG_CRYPTO_POLY1305=y
> > CONFIG_CRYPTO_MD4=m
> > CONFIG_CRYPTO_MD5=y
> > # CONFIG_CRYPTO_MICHAEL_MIC is not set
> > # CONFIG_CRYPTO_RMD128 is not set
> > CONFIG_CRYPTO_RMD160=y
> > # CONFIG_CRYPTO_RMD256 is not set
> > # CONFIG_CRYPTO_RMD320 is not set
> > CONFIG_CRYPTO_SHA1=y
> > CONFIG_CRYPTO_SHA256=y
> > CONFIG_CRYPTO_SHA512=y
> > # CONFIG_CRYPTO_SHA3 is not set
> > # CONFIG_CRYPTO_TGR192 is not set
> > CONFIG_CRYPTO_WP512=m
> > 
> > #
> > # Ciphers
> > #
> > CONFIG_CRYPTO_AES=y
> > # CONFIG_CRYPTO_AES_TI is not set
> > CONFIG_CRYPTO_AES_586=y
> > CONFIG_CRYPTO_AES_NI_INTEL=y
> > # CONFIG_CRYPTO_ANUBIS is not set
> > CONFIG_CRYPTO_ARC4=y
> > # CONFIG_CRYPTO_BLOWFISH is not set
> > CONFIG_CRYPTO_CAMELLIA=y
> > CONFIG_CRYPTO_CAST_COMMON=m
> > # CONFIG_CRYPTO_CAST5 is not set
> > CONFIG_CRYPTO_CAST6=m
> > CONFIG_CRYPTO_DES=y
> > CONFIG_CRYPTO_FCRYPT=m
> > CONFIG_CRYPTO_KHAZAD=y
> > CONFIG_CRYPTO_SALSA20=y
> > CONFIG_CRYPTO_SALSA20_586=m
> > CONFIG_CRYPTO_CHACHA20=y
> > CONFIG_CRYPTO_SEED=m
> > CONFIG_CRYPTO_SERPENT=y
> > CONFIG_CRYPTO_SERPENT_SSE2_586=y
> > CONFIG_CRYPTO_TEA=y
> > # CONFIG_CRYPTO_TWOFISH is not set
> > CONFIG_CRYPTO_TWOFISH_COMMON=m
> > CONFIG_CRYPTO_TWOFISH_586=m
> > 
> > #
> > # Compression
> > #
> > CONFIG_CRYPTO_DEFLATE=m
> > # CONFIG_CRYPTO_LZO is not set
> > CONFIG_CRYPTO_842=m
> > # CONFIG_CRYPTO_LZ4 is not set
> > CONFIG_CRYPTO_LZ4HC=m
> > 
> > #
> > # Random Number Generation
> > #
> > CONFIG_CRYPTO_ANSI_CPRNG=y
> > CONFIG_CRYPTO_DRBG_MENU=y
> > CONFIG_CRYPTO_DRBG_HMAC=y
> > # CONFIG_CRYPTO_DRBG_HASH is not set
> > CONFIG_CRYPTO_DRBG_CTR=y
> > CONFIG_CRYPTO_DRBG=y
> > CONFIG_CRYPTO_JITTERENTROPY=y
> > CONFIG_CRYPTO_USER_API=m
> > # CONFIG_CRYPTO_USER_API_HASH is not set
> > CONFIG_CRYPTO_USER_API_SKCIPHER=m
> > CONFIG_CRYPTO_USER_API_RNG=m
> > CONFIG_CRYPTO_USER_API_AEAD=m
> > CONFIG_CRYPTO_HASH_INFO=y
> > CONFIG_CRYPTO_HW=y
> > CONFIG_CRYPTO_DEV_PADLOCK=m
> > # CONFIG_CRYPTO_DEV_PADLOCK_AES is not set
> > # CONFIG_CRYPTO_DEV_PADLOCK_SHA is not set
> > # CONFIG_CRYPTO_DEV_GEODE is not set
> > # CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_DESC is not set
> > # CONFIG_CRYPTO_DEV_CCP is not set
> > # CONFIG_CRYPTO_DEV_QAT_DH895xCC is not set
> > # CONFIG_CRYPTO_DEV_QAT_C3XXX is not set
> > # CONFIG_CRYPTO_DEV_QAT_C62X is not set
> > # CONFIG_CRYPTO_DEV_QAT_DH895xCCVF is not set
> > # CONFIG_CRYPTO_DEV_QAT_C3XXXVF is not set
> > # CONFIG_CRYPTO_DEV_QAT_C62XVF is not set
> > CONFIG_CRYPTO_DEV_VIRTIO=m
> > CONFIG_ASYMMETRIC_KEY_TYPE=y
> > CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
> > CONFIG_X509_CERTIFICATE_PARSER=y
> > CONFIG_PKCS7_MESSAGE_PARSER=y
> > 
> > #
> > # Certificates for signature checking
> > #
> > CONFIG_SYSTEM_TRUSTED_KEYRING=y
> > CONFIG_SYSTEM_TRUSTED_KEYS=""
> > CONFIG_SYSTEM_EXTRA_CERTIFICATE=y
> > CONFIG_SYSTEM_EXTRA_CERTIFICATE_SIZE=4096
> > # CONFIG_SECONDARY_TRUSTED_KEYRING is not set
> > CONFIG_HAVE_KVM=y
> > CONFIG_VIRTUALIZATION=y
> > # CONFIG_KVM is not set
> > # CONFIG_VHOST_NET is not set
> > CONFIG_VHOST_SCSI=m
> > CONFIG_VHOST_VSOCK=m
> > CONFIG_VHOST=m
> > CONFIG_VHOST_CROSS_ENDIAN_LEGACY=y
> > # CONFIG_LGUEST is not set
> > # CONFIG_BINARY_PRINTF is not set
> > 
> > #
> > # Library routines
> > #
> > CONFIG_RAID6_PQ=y
> > CONFIG_BITREVERSE=y
> > # CONFIG_HAVE_ARCH_BITREVERSE is not set
> > CONFIG_RATIONAL=y
> > CONFIG_GENERIC_STRNCPY_FROM_USER=y
> > CONFIG_GENERIC_STRNLEN_USER=y
> > CONFIG_GENERIC_NET_UTILS=y
> > CONFIG_GENERIC_FIND_FIRST_BIT=y
> > CONFIG_GENERIC_PCI_IOMAP=y
> > CONFIG_GENERIC_IOMAP=y
> > CONFIG_GENERIC_IO=y
> > CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
> > CONFIG_CRC_CCITT=y
> > CONFIG_CRC16=y
> > CONFIG_CRC_T10DIF=y
> > # CONFIG_CRC_ITU_T is not set
> > CONFIG_CRC32=y
> > # CONFIG_CRC32_SELFTEST is not set
> > CONFIG_CRC32_SLICEBY8=y
> > # CONFIG_CRC32_SLICEBY4 is not set
> > # CONFIG_CRC32_SARWATE is not set
> > # CONFIG_CRC32_BIT is not set
> > CONFIG_CRC7=m
> > CONFIG_LIBCRC32C=y
> > CONFIG_CRC8=m
> > # CONFIG_AUDIT_ARCH_COMPAT_GENERIC is not set
> > CONFIG_RANDOM32_SELFTEST=y
> > CONFIG_842_COMPRESS=m
> > CONFIG_842_DECOMPRESS=m
> > CONFIG_ZLIB_INFLATE=y
> > CONFIG_ZLIB_DEFLATE=y
> > CONFIG_LZO_COMPRESS=y
> > CONFIG_LZO_DECOMPRESS=y
> > CONFIG_LZ4HC_COMPRESS=m
> > CONFIG_LZ4_DECOMPRESS=m
> > # CONFIG_XZ_DEC is not set
> > # CONFIG_XZ_DEC_BCJ is not set
> > CONFIG_DECOMPRESS_GZIP=y
> > CONFIG_DECOMPRESS_LZMA=y
> > CONFIG_GENERIC_ALLOCATOR=y
> > CONFIG_INTERVAL_TREE=y
> > CONFIG_RADIX_TREE_MULTIORDER=y
> > CONFIG_ASSOCIATIVE_ARRAY=y
> > CONFIG_HAS_IOMEM=y
> > CONFIG_HAS_IOPORT_MAP=y
> > CONFIG_HAS_DMA=y
> > # CONFIG_DMA_NOOP_OPS is not set
> > # CONFIG_DMA_VIRT_OPS is not set
> > CONFIG_CHECK_SIGNATURE=y
> > CONFIG_DQL=y
> > CONFIG_GLOB=y
> > CONFIG_GLOB_SELFTEST=y
> > CONFIG_NLATTR=y
> > CONFIG_CLZ_TAB=y
> > # CONFIG_CORDIC is not set
> > CONFIG_DDR=y
> > CONFIG_IRQ_POLL=y
> > CONFIG_MPILIB=y
> > CONFIG_OID_REGISTRY=y
> > # CONFIG_SG_SPLIT is not set
> > CONFIG_SG_POOL=y
> > CONFIG_ARCH_HAS_SG_CHAIN=y
> > CONFIG_ARCH_HAS_MMIO_FLUSH=y
> > CONFIG_STACKDEPOT=y
> > CONFIG_SBITMAP=y
> 
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

[-- Attachment #2: debug.diff --]
[-- Type: text/x-patch, Size: 2555 bytes --]

diff --git a/block/genhd.c b/block/genhd.c
index b26a5ea115d0..d06ba7e731a4 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -662,6 +662,7 @@ void del_gendisk(struct gendisk *disk)
 	struct disk_part_iter piter;
 	struct hd_struct *part;
 
+	WARN_ON(!(disk->flags & GENHD_FL_UP));
 	blk_integrity_del(disk);
 	disk_del_events(disk);
 
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 6f7128f49c30..7d987dcb8fee 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1073,6 +1073,7 @@ static int scsi_probe_and_add_lun(struct scsi_target *starget,
 	 */
 	sdev = scsi_device_lookup_by_target(starget, lun);
 	if (sdev) {
+		printk("Reusing sdev %p\n", sdev);
 		if (rescan != SCSI_SCAN_INITIAL || !scsi_device_created(sdev)) {
 			SCSI_LOG_SCAN_BUS(3, sdev_printk(KERN_INFO, sdev,
 				"scsi scan: device exists on %s\n",
@@ -1089,8 +1090,10 @@ static int scsi_probe_and_add_lun(struct scsi_target *starget,
 			return SCSI_SCAN_LUN_PRESENT;
 		}
 		scsi_device_put(sdev);
-	} else
+	} else {
 		sdev = scsi_alloc_sdev(starget, lun, hostdata);
+		printk("Created sdev %p\n", sdev);
+	}
 	if (!sdev)
 		goto out;
 
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index 6d861d090e9f..6c172099aa23 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -496,6 +496,7 @@ static void cgwb_release_workfn(struct work_struct *work)
 	wb_exit(wb);
 	kfree_rcu(wb, rcu);
 
+	printk("CGWB release! %p\n", bdi);
 	if (atomic_dec_and_test(&bdi->usage_cnt))
 		wake_up_all(&cgwb_release_wait);
 }
@@ -668,6 +669,7 @@ static int cgwb_bdi_init(struct backing_dev_info *bdi)
 {
 	int ret;
 
+	printk("CGWB init %p\n", bdi);
 	INIT_RADIX_TREE(&bdi->cgwb_tree, GFP_ATOMIC);
 	bdi->cgwb_congested_tree = RB_ROOT;
 	atomic_set(&bdi->usage_cnt, 1);
@@ -709,6 +711,7 @@ static void cgwb_bdi_destroy(struct backing_dev_info *bdi)
 	 * all cgwb's and cgwb_congested's ever created on @bdi.
 	 */
 	atomic_dec(&bdi->usage_cnt);
+	printk("CGWB %p usage_cnt: %u\n", bdi, atomic_read(&bdi->usage_cnt));
 	wait_event(cgwb_release_wait, !atomic_read(&bdi->usage_cnt));
 }
 
@@ -877,10 +880,15 @@ static void bdi_remove_from_list(struct backing_dev_info *bdi)
 
 void bdi_unregister(struct backing_dev_info *bdi)
 {
+	WARN_ON(!test_bit(WB_registered, &bdi->wb.state));
 	/* make sure nobody finds us on the bdi_list anymore */
+	printk("Unreg1\n");
 	bdi_remove_from_list(bdi);
+	printk("Unreg2\n");
 	wb_shutdown(&bdi->wb);
+	printk("Unreg3\n");
 	cgwb_bdi_destroy(bdi);
+	printk("Unreg4\n");
 
 	if (bdi->dev) {
 		bdi_debug_unregister(bdi);

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 12:01   ` [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds Jan Kara
@ 2017-03-06 14:35     ` James Bottomley
  2017-03-06 15:14       ` Jan Kara
  0 siblings, 1 reply; 10+ messages in thread
From: James Bottomley @ 2017-03-06 14:35 UTC (permalink / raw)
  To: Jan Kara, Fengguang Wu
  Cc: Jens Axboe, linux-block, linux-kernel, LKP, Martin K. Petersen,
	linux-scsi

On Mon, 2017-03-06 at 13:01 +0100, Jan Kara wrote:
> On Mon 06-03-17 11:27:33, Jan Kara wrote:
> > Hi,
> > 
> > On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > > FYI next-20170303 is good while mainline is bad with this error.
> > > The attached reproduce-* may help reproduce the issue.
> > 
> > Thanks for report! So from the stacktrace we are in the path 
> > testing removal of a device immediately after it has been probed 
> > and for some reason bdi_unregister() hangs - likely waiting for 
> > cgroup-writeback references to drop. Given how early this happens 
> > my guess is we fail to initialize something but for now I don't see 
> > how my patch could make a difference. I'm trying to reproduce this 
> > to be able to debug more...
> 
> OK, so after some debugging I think this is yet another problem in 
> SCSI initialization / destruction code which my patch only makes 
> visible (added relevant maintainers).
> 
> I can reproduce the problem reliably with enabling:
> 
> CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> CONFIG_SCSI_DEBUG=m
> CONFIG_BLK_CGROUP=y
> CONFIG_MEMCG=y
> (and thus CONFIG_CGROUP_WRITEBACK=y)
> 
> then 'modprobe scsi_debug' is all it takes to reproduce hang. 
> Relevant kernel messages with some of my debugging added (attached is 
> a patch that adds those debug messages):

This looks to be precisely the same problem Dan Williams was debugging
for us.

> [   58.721765] scsi host0: scsi_debug: version 1.86 [20160430]
> [   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1,
> statistics=0
> [   58.728946] CGWB init ffff88007fbb2000
> [   58.730095] Created sdev ffff880078e1a000
> [   58.731611] scsi 0:0:0:0: Direct-Access     Linux    scsi_debug
> 0186 PQ : 0 ANSI: 7
> [   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical blocks: (8.39
> MB/8.00 MiB)
> [   58.789687] sd 0:0:0:0: [sda] Write Protect is off
> [   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
> [   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
> enabled, supports DPO and FUA
> [   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
> [   58.896808] Unreg1
> [   58.897960] Unreg2
> [   58.898637] Unreg3
> [   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
> [   58.900004] Unreg4
> [   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache

OK, can you put a WARN_ON trace in sd_shutdown and tell us where this
is coming from.  For the device to be reused after this we have to be
calling sd_shutdown() without going into SDEV_DEL.

Thanks,

James

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 14:35     ` James Bottomley
@ 2017-03-06 15:14       ` Jan Kara
  2017-03-06 15:44         ` James Bottomley
  0 siblings, 1 reply; 10+ messages in thread
From: Jan Kara @ 2017-03-06 15:14 UTC (permalink / raw)
  To: James Bottomley
  Cc: Jan Kara, Fengguang Wu, Jens Axboe, linux-block, linux-kernel,
	LKP, Martin K. Petersen, linux-scsi

On Mon 06-03-17 06:35:21, James Bottomley wrote:
> On Mon, 2017-03-06 at 13:01 +0100, Jan Kara wrote:
> > On Mon 06-03-17 11:27:33, Jan Kara wrote:
> > > Hi,
> > > 
> > > On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > > > FYI next-20170303 is good while mainline is bad with this error.
> > > > The attached reproduce-* may help reproduce the issue.
> > > 
> > > Thanks for report! So from the stacktrace we are in the path 
> > > testing removal of a device immediately after it has been probed 
> > > and for some reason bdi_unregister() hangs - likely waiting for 
> > > cgroup-writeback references to drop. Given how early this happens 
> > > my guess is we fail to initialize something but for now I don't see 
> > > how my patch could make a difference. I'm trying to reproduce this 
> > > to be able to debug more...
> > 
> > OK, so after some debugging I think this is yet another problem in 
> > SCSI initialization / destruction code which my patch only makes 
> > visible (added relevant maintainers).
> > 
> > I can reproduce the problem reliably with enabling:
> > 
> > CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> > CONFIG_SCSI_DEBUG=m
> > CONFIG_BLK_CGROUP=y
> > CONFIG_MEMCG=y
> > (and thus CONFIG_CGROUP_WRITEBACK=y)
> > 
> > then 'modprobe scsi_debug' is all it takes to reproduce hang. 
> > Relevant kernel messages with some of my debugging added (attached is 
> > a patch that adds those debug messages):
> 
> This looks to be precisely the same problem Dan Williams was debugging
> for us.
> 
> > [   58.721765] scsi host0: scsi_debug: version 1.86 [20160430]
> > [   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1,
> > statistics=0
> > [   58.728946] CGWB init ffff88007fbb2000
> > [   58.730095] Created sdev ffff880078e1a000
> > [   58.731611] scsi 0:0:0:0: Direct-Access     Linux    scsi_debug
> > 0186 PQ : 0 ANSI: 7
> > [   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical blocks: (8.39
> > MB/8.00 MiB)
> > [   58.789687] sd 0:0:0:0: [sda] Write Protect is off
> > [   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
> > [   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
> > enabled, supports DPO and FUA
> > [   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
> > [   58.896808] Unreg1
> > [   58.897960] Unreg2
> > [   58.898637] Unreg3
> > [   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
> > [   58.900004] Unreg4
> > [   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> 
> OK, can you put a WARN_ON trace in sd_shutdown and tell us where this
> is coming from.  For the device to be reused after this we have to be
> calling sd_shutdown() without going into SDEV_DEL.

Sure. The call trace is:

[   41.919244] ------------[ cut here ]------------
[   41.919263] WARNING: CPU: 4 PID: 2335 at drivers/scsi/sd.c:3332 sd_shutdown+0x2f/0x100
[   41.919268] Modules linked in: scsi_debug(+) netconsole loop btrfs raid6_pq zlib_deflate lzo_compress xor
[   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 4.11.0-rc1-xen+ #49
[   41.919325] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[   41.919331] Call Trace:
[   41.919343]  dump_stack+0x8e/0xf0
[   41.919354]  __warn+0x116/0x120
[   41.919361]  warn_slowpath_null+0x1d/0x20
[   41.919368]  sd_shutdown+0x2f/0x100
[   41.919374]  sd_remove+0x70/0xd0

*** Here is the unexpected step I guess...

[   41.919383]  driver_probe_device+0xe0/0x4c0
[   41.919395]  ? kobject_uevent_env+0x1bc/0x660
[   41.919403]  __device_attach_driver+0xfa/0x120
[   41.919410]  ? __driver_attach+0x110/0x110
[   41.919417]  bus_for_each_drv+0x68/0x90
[   41.919424]  __device_attach+0xb2/0x110
[   41.919432]  device_initial_probe+0x13/0x20
[   41.919439]  bus_probe_device+0xa8/0xc0
[   41.919446]  device_add+0x297/0x600
[   41.919453]  ? call_rcu_bh+0x20/0x20
[   41.919463]  ? mutex_unlock+0x12/0x20
[   41.919473]  scsi_sysfs_add_sdev+0x69/0x210
[   41.919480]  scsi_probe_and_add_lun+0xcfd/0xd10
[   41.919489]  ? find_next_zero_bit+0x10/0x20
[   41.919496]  __scsi_scan_target+0xd2/0x4c0
[   41.919506]  ? proc_register+0x41/0x130
[   41.919512]  ? proc_register+0x11c/0x130
[   41.919520]  ? trace_hardirqs_on+0xd/0x10
[   41.919527]  ? _raw_spin_unlock_irq+0x30/0x50
[   41.919534]  scsi_scan_channel+0x5f/0xa0
[   41.919541]  scsi_scan_host_selected+0xb9/0x120
[   41.919549]  do_scsi_scan_host+0x81/0x90
[   41.919555]  scsi_scan_host+0x17a/0x1b0
[   41.919568]  ? scsi_add_host_with_dma+0x302/0x310
[   41.919593]  sdebug_driver_probe+0x204/0x320 [scsi_debug]
[   41.919601]  ? driver_sysfs_add+0x90/0xb0
[   41.919608]  driver_probe_device+0xb3/0x4c0
[   41.919615]  ? kobject_uevent_env+0x1bc/0x660
[   41.919623]  __device_attach_driver+0xfa/0x120
[   41.919630]  ? __driver_attach+0x110/0x110
[   41.919636]  bus_for_each_drv+0x68/0x90
[   41.919643]  __device_attach+0xb2/0x110
[   41.919651]  device_initial_probe+0x13/0x20
[   41.919658]  bus_probe_device+0xa8/0xc0
[   41.919664]  device_add+0x297/0x600
[   41.919671]  ? dev_set_name+0x41/0x50
[   41.919681]  device_register+0x1e/0x30
[   41.919696]  sdebug_add_adapter+0xff/0x1d0 [scsi_debug]
[   41.919703]  ? kobject_uevent+0xb/0x10
[   41.919709]  ? 0xffffffffa0172000
[   41.919723]  scsi_debug_init+0x40e/0x1000 [scsi_debug]
[   41.919730]  ? 0xffffffffa0172000
[   41.919739]  do_one_initcall+0xbe/0x200

								Honza

-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 15:14       ` Jan Kara
@ 2017-03-06 15:44         ` James Bottomley
  2017-03-06 16:13           ` Jan Kara
  0 siblings, 1 reply; 10+ messages in thread
From: James Bottomley @ 2017-03-06 15:44 UTC (permalink / raw)
  To: Jan Kara
  Cc: Fengguang Wu, Jens Axboe, linux-block, linux-kernel, LKP,
	Martin K. Petersen, linux-scsi

On Mon, 2017-03-06 at 16:14 +0100, Jan Kara wrote:
> On Mon 06-03-17 06:35:21, James Bottomley wrote:
> > On Mon, 2017-03-06 at 13:01 +0100, Jan Kara wrote:
> > > On Mon 06-03-17 11:27:33, Jan Kara wrote:
> > > > Hi,
> > > > 
> > > > On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > > > > FYI next-20170303 is good while mainline is bad with this
> > > > > error.
> > > > > The attached reproduce-* may help reproduce the issue.
> > > > 
> > > > Thanks for report! So from the stacktrace we are in the path 
> > > > testing removal of a device immediately after it has been
> > > > probed 
> > > > and for some reason bdi_unregister() hangs - likely waiting for
> > > > cgroup-writeback references to drop. Given how early this
> > > > happens 
> > > > my guess is we fail to initialize something but for now I don't
> > > > see 
> > > > how my patch could make a difference. I'm trying to reproduce
> > > > this 
> > > > to be able to debug more...
> > > 
> > > OK, so after some debugging I think this is yet another problem
> > > in 
> > > SCSI initialization / destruction code which my patch only makes 
> > > visible (added relevant maintainers).
> > > 
> > > I can reproduce the problem reliably with enabling:
> > > 
> > > CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> > > CONFIG_SCSI_DEBUG=m
> > > CONFIG_BLK_CGROUP=y
> > > CONFIG_MEMCG=y
> > > (and thus CONFIG_CGROUP_WRITEBACK=y)
> > > 
> > > then 'modprobe scsi_debug' is all it takes to reproduce hang. 
> > > Relevant kernel messages with some of my debugging added
> > > (attached is 
> > > a patch that adds those debug messages):
> > 
> > This looks to be precisely the same problem Dan Williams was
> > debugging
> > for us.
> > 
> > > [   58.721765] scsi host0: scsi_debug: version 1.86 [20160430]
> > > [   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1,
> > > statistics=0
> > > [   58.728946] CGWB init ffff88007fbb2000
> > > [   58.730095] Created sdev ffff880078e1a000
> > > [   58.731611] scsi 0:0:0:0: Direct-Access     Linux   
> > >  scsi_debug
> > > 0186 PQ : 0 ANSI: 7
> > > [   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical blocks:
> > > (8.39
> > > MB/8.00 MiB)
> > > [   58.789687] sd 0:0:0:0: [sda] Write Protect is off
> > > [   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
> > > [   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read
> > > cache:
> > > enabled, supports DPO and FUA
> > > [   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
> > > [   58.896808] Unreg1
> > > [   58.897960] Unreg2
> > > [   58.898637] Unreg3
> > > [   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
> > > [   58.900004] Unreg4
> > > [   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> > 
> > OK, can you put a WARN_ON trace in sd_shutdown and tell us where
> > this
> > is coming from.  For the device to be reused after this we have to
> > be
> > calling sd_shutdown() without going into SDEV_DEL.
> 
> Sure. The call trace is:
> 
> [   41.919244] ------------[ cut here ]------------
> [   41.919263] WARNING: CPU: 4 PID: 2335 at drivers/scsi/sd.c:3332
> sd_shutdown+0x2f/0x100
> [   41.919268] Modules linked in: scsi_debug(+) netconsole loop btrfs
> raid6_pq zlib_deflate lzo_compress xor
> [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 4.11.0-rc1
> -xen+ #49
> [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> [   41.919331] Call Trace:
> [   41.919343]  dump_stack+0x8e/0xf0
> [   41.919354]  __warn+0x116/0x120
> [   41.919361]  warn_slowpath_null+0x1d/0x20
> [   41.919368]  sd_shutdown+0x2f/0x100
> [   41.919374]  sd_remove+0x70/0xd0
> 
> *** Here is the unexpected step I guess...
> 
> [   41.919383]  driver_probe_device+0xe0/0x4c0

Exactly.  It's this, I think

	bool test_remove = IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE) &&
			   !drv->suppress_bind_attrs;

You have that config option set.

So the drivers base layer is calling ->remove after probe and
triggering the destruction of the queue.

What to do about this (apart from nuke such a stupid option) is
somewhat more problematic.

James

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 15:44         ` James Bottomley
@ 2017-03-06 16:13           ` Jan Kara
  2017-03-06 17:25             ` James Bottomley
  0 siblings, 1 reply; 10+ messages in thread
From: Jan Kara @ 2017-03-06 16:13 UTC (permalink / raw)
  To: James Bottomley
  Cc: Jan Kara, Fengguang Wu, Jens Axboe, linux-block, linux-kernel,
	LKP, Martin K. Petersen, linux-scsi

On Mon 06-03-17 07:44:55, James Bottomley wrote:
> On Mon, 2017-03-06 at 16:14 +0100, Jan Kara wrote:
> > On Mon 06-03-17 06:35:21, James Bottomley wrote:
> > > On Mon, 2017-03-06 at 13:01 +0100, Jan Kara wrote:
> > > > On Mon 06-03-17 11:27:33, Jan Kara wrote:
> > > > > Hi,
> > > > > 
> > > > > On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > > > > > FYI next-20170303 is good while mainline is bad with this
> > > > > > error.
> > > > > > The attached reproduce-* may help reproduce the issue.
> > > > > 
> > > > > Thanks for report! So from the stacktrace we are in the path 
> > > > > testing removal of a device immediately after it has been
> > > > > probed 
> > > > > and for some reason bdi_unregister() hangs - likely waiting for
> > > > > cgroup-writeback references to drop. Given how early this
> > > > > happens 
> > > > > my guess is we fail to initialize something but for now I don't
> > > > > see 
> > > > > how my patch could make a difference. I'm trying to reproduce
> > > > > this 
> > > > > to be able to debug more...
> > > > 
> > > > OK, so after some debugging I think this is yet another problem
> > > > in 
> > > > SCSI initialization / destruction code which my patch only makes 
> > > > visible (added relevant maintainers).
> > > > 
> > > > I can reproduce the problem reliably with enabling:
> > > > 
> > > > CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> > > > CONFIG_SCSI_DEBUG=m
> > > > CONFIG_BLK_CGROUP=y
> > > > CONFIG_MEMCG=y
> > > > (and thus CONFIG_CGROUP_WRITEBACK=y)
> > > > 
> > > > then 'modprobe scsi_debug' is all it takes to reproduce hang. 
> > > > Relevant kernel messages with some of my debugging added
> > > > (attached is 
> > > > a patch that adds those debug messages):
> > > 
> > > This looks to be precisely the same problem Dan Williams was
> > > debugging
> > > for us.
> > > 
> > > > [   58.721765] scsi host0: scsi_debug: version 1.86 [20160430]
> > > > [   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1,
> > > > statistics=0
> > > > [   58.728946] CGWB init ffff88007fbb2000
> > > > [   58.730095] Created sdev ffff880078e1a000
> > > > [   58.731611] scsi 0:0:0:0: Direct-Access     Linux   
> > > >  scsi_debug
> > > > 0186 PQ : 0 ANSI: 7
> > > > [   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical blocks:
> > > > (8.39
> > > > MB/8.00 MiB)
> > > > [   58.789687] sd 0:0:0:0: [sda] Write Protect is off
> > > > [   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
> > > > [   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read
> > > > cache:
> > > > enabled, supports DPO and FUA
> > > > [   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
> > > > [   58.896808] Unreg1
> > > > [   58.897960] Unreg2
> > > > [   58.898637] Unreg3
> > > > [   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
> > > > [   58.900004] Unreg4
> > > > [   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> > > 
> > > OK, can you put a WARN_ON trace in sd_shutdown and tell us where
> > > this
> > > is coming from.  For the device to be reused after this we have to
> > > be
> > > calling sd_shutdown() without going into SDEV_DEL.
> > 
> > Sure. The call trace is:
> > 
> > [   41.919244] ------------[ cut here ]------------
> > [   41.919263] WARNING: CPU: 4 PID: 2335 at drivers/scsi/sd.c:3332
> > sd_shutdown+0x2f/0x100
> > [   41.919268] Modules linked in: scsi_debug(+) netconsole loop btrfs
> > raid6_pq zlib_deflate lzo_compress xor
> > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 4.11.0-rc1
> > -xen+ #49
> > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> > [   41.919331] Call Trace:
> > [   41.919343]  dump_stack+0x8e/0xf0
> > [   41.919354]  __warn+0x116/0x120
> > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > [   41.919368]  sd_shutdown+0x2f/0x100
> > [   41.919374]  sd_remove+0x70/0xd0
> > 
> > *** Here is the unexpected step I guess...
> > 
> > [   41.919383]  driver_probe_device+0xe0/0x4c0
> 
> Exactly.  It's this, I think
> 
> 	bool test_remove = IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE) &&
> 			   !drv->suppress_bind_attrs;
> 
> You have that config option set.

Yes - or better said 0-day testing has it set. Maybe that is not a sane
default for 0-day tests? The option is explicitely marked as "unstable"...
Fengguang?

> So the drivers base layer is calling ->remove after probe and
> triggering the destruction of the queue.
> 
> What to do about this (apart from nuke such a stupid option) is
> somewhat more problematic.

I guess this is between you and Greg :).

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 16:13           ` Jan Kara
@ 2017-03-06 17:25             ` James Bottomley
  2017-03-07 14:41               ` Jan Kara
  0 siblings, 1 reply; 10+ messages in thread
From: James Bottomley @ 2017-03-06 17:25 UTC (permalink / raw)
  To: Jan Kara
  Cc: Fengguang Wu, Jens Axboe, linux-block, linux-kernel, LKP,
	Martin K. Petersen, linux-scsi

On Mon, 2017-03-06 at 17:13 +0100, Jan Kara wrote:
> On Mon 06-03-17 07:44:55, James Bottomley wrote:
> > On Mon, 2017-03-06 at 16:14 +0100, Jan Kara wrote:
> > > On Mon 06-03-17 06:35:21, James Bottomley wrote:
> > > > On Mon, 2017-03-06 at 13:01 +0100, Jan Kara wrote:
> > > > > On Mon 06-03-17 11:27:33, Jan Kara wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On Sun 05-03-17 10:21:11, Wu Fengguang wrote:
> > > > > > > FYI next-20170303 is good while mainline is bad with this
> > > > > > > error.
> > > > > > > The attached reproduce-* may help reproduce the issue.
> > > > > > 
> > > > > > Thanks for report! So from the stacktrace we are in the
> > > > > > path 
> > > > > > testing removal of a device immediately after it has been
> > > > > > probed 
> > > > > > and for some reason bdi_unregister() hangs - likely waiting
> > > > > > for
> > > > > > cgroup-writeback references to drop. Given how early this
> > > > > > happens 
> > > > > > my guess is we fail to initialize something but for now I
> > > > > > don't
> > > > > > see 
> > > > > > how my patch could make a difference. I'm trying to
> > > > > > reproduce
> > > > > > this 
> > > > > > to be able to debug more...
> > > > > 
> > > > > OK, so after some debugging I think this is yet another
> > > > > problem
> > > > > in 
> > > > > SCSI initialization / destruction code which my patch only
> > > > > makes 
> > > > > visible (added relevant maintainers).
> > > > > 
> > > > > I can reproduce the problem reliably with enabling:
> > > > > 
> > > > > CONFIG_DEBUG_TEST_DRIVER_REMOVE=y
> > > > > CONFIG_SCSI_DEBUG=m
> > > > > CONFIG_BLK_CGROUP=y
> > > > > CONFIG_MEMCG=y
> > > > > (and thus CONFIG_CGROUP_WRITEBACK=y)
> > > > > 
> > > > > then 'modprobe scsi_debug' is all it takes to reproduce hang.
> > > > > Relevant kernel messages with some of my debugging added
> > > > > (attached is 
> > > > > a patch that adds those debug messages):
> > > > 
> > > > This looks to be precisely the same problem Dan Williams was
> > > > debugging
> > > > for us.
> > > > 
> > > > > [   58.721765] scsi host0: scsi_debug: version 1.86
> > > > > [20160430]
> > > > > [   58.721765]   dev_size_mb=8, opts=0x0, submit_queues=1,
> > > > > statistics=0
> > > > > [   58.728946] CGWB init ffff88007fbb2000
> > > > > [   58.730095] Created sdev ffff880078e1a000
> > > > > [   58.731611] scsi 0:0:0:0: Direct-Access     Linux   
> > > > >  scsi_debug
> > > > > 0186 PQ : 0 ANSI: 7
> > > > > [   58.782246] sd 0:0:0:0: [sda] 16384 512-byte logical
> > > > > blocks:
> > > > > (8.39
> > > > > MB/8.00 MiB)
> > > > > [   58.789687] sd 0:0:0:0: [sda] Write Protect is off
> > > > > [   58.791140] sd 0:0:0:0: [sda] Mode Sense: 73 00 10 08
> > > > > [   58.800879] sd 0:0:0:0: [sda] Write cache: enabled, read
> > > > > cache:
> > > > > enabled, supports DPO and FUA
> > > > > [   58.893738] sd 0:0:0:0: [sda] Attached SCSI disk
> > > > > [   58.896808] Unreg1
> > > > > [   58.897960] Unreg2
> > > > > [   58.898637] Unreg3
> > > > > [   58.899100] CGWB ffff88007fbb2000 usage_cnt: 0
> > > > > [   58.900004] Unreg4
> > > > > [   58.904976] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> > > > 
> > > > OK, can you put a WARN_ON trace in sd_shutdown and tell us
> > > > where
> > > > this
> > > > is coming from.  For the device to be reused after this we have
> > > > to
> > > > be
> > > > calling sd_shutdown() without going into SDEV_DEL.
> > > 
> > > Sure. The call trace is:
> > > 
> > > [   41.919244] ------------[ cut here ]------------
> > > [   41.919263] WARNING: CPU: 4 PID: 2335 at
> > > drivers/scsi/sd.c:3332
> > > sd_shutdown+0x2f/0x100
> > > [   41.919268] Modules linked in: scsi_debug(+) netconsole loop
> > > btrfs
> > > raid6_pq zlib_deflate lzo_compress xor
> > > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 4.11.0
> > > -rc1
> > > -xen+ #49
> > > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> > > [   41.919331] Call Trace:
> > > [   41.919343]  dump_stack+0x8e/0xf0
> > > [   41.919354]  __warn+0x116/0x120
> > > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > > [   41.919368]  sd_shutdown+0x2f/0x100
> > > [   41.919374]  sd_remove+0x70/0xd0
> > > 
> > > *** Here is the unexpected step I guess...
> > > 
> > > [   41.919383]  driver_probe_device+0xe0/0x4c0
> > 
> > Exactly.  It's this, I think
> > 
> > 	bool test_remove = IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE)
> > &&
> > 			   !drv->suppress_bind_attrs;
> > 
> > You have that config option set.
> 
> Yes - or better said 0-day testing has it set. Maybe that is not a 
> sane default for 0-day tests? The option is explicitely marked as
> "unstable"... Fengguang?
> 
> > So the drivers base layer is calling ->remove after probe and
> > triggering the destruction of the queue.
> > 
> > What to do about this (apart from nuke such a stupid option) is
> > somewhat more problematic.
> 
> I guess this is between you and Greg :).

Nice try, sport ... you qualify just behind Dan in the "not my problem"
olympic hurdles event.  I'm annoyed because there's no indication in
the log that the driver add behaviour is radically altered, so I've
been staring at the wrong code for weeks.  However, the unbind/rebind
should work, so the real issue is that your bdi changes (or perhaps
something else in block) have induced a regression in the unbinding of
upper layer drivers.  If you're going to release the bdi in
del_gendisk, you have to have some mechanism for re-acquiring it on re
-probe (likely with the same name) otherwise rebind is broken for every
block driver.

The fact that the second rebind tried with a different name indicates
that sd_devt_release wasn't called, so some vestige of the devt remains
on the subsequent rebind.  Here's the problem: the queue belongs to
SCSI (the lower layer), so it's not going to change because it doesn't
see the release.  The gendisk and its allied stuff belongs to sd so it
gets freed and re-created for the same queue.  Something in block is
very confused when this happens.

James

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-06 17:25             ` James Bottomley
@ 2017-03-07 14:41               ` Jan Kara
  2017-03-07 16:10                 ` James Bottomley
  0 siblings, 1 reply; 10+ messages in thread
From: Jan Kara @ 2017-03-07 14:41 UTC (permalink / raw)
  To: James Bottomley
  Cc: Jan Kara, Fengguang Wu, Jens Axboe, linux-block, linux-kernel,
	LKP, Martin K. Petersen, linux-scsi

On Mon 06-03-17 09:25:42, James Bottomley wrote:
> On Mon, 2017-03-06 at 17:13 +0100, Jan Kara wrote:
> > On Mon 06-03-17 07:44:55, James Bottomley wrote:
...
> > > > Sure. The call trace is:
> > > > 
> > > > [   41.919244] ------------[ cut here ]------------
> > > > [   41.919263] WARNING: CPU: 4 PID: 2335 at
> > > > drivers/scsi/sd.c:3332
> > > > sd_shutdown+0x2f/0x100
> > > > [   41.919268] Modules linked in: scsi_debug(+) netconsole loop
> > > > btrfs
> > > > raid6_pq zlib_deflate lzo_compress xor
> > > > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 4.11.0
> > > > -rc1
> > > > -xen+ #49
> > > > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> > > > [   41.919331] Call Trace:
> > > > [   41.919343]  dump_stack+0x8e/0xf0
> > > > [   41.919354]  __warn+0x116/0x120
> > > > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > > > [   41.919368]  sd_shutdown+0x2f/0x100
> > > > [   41.919374]  sd_remove+0x70/0xd0
> > > > 
> > > > *** Here is the unexpected step I guess...
> > > > 
> > > > [   41.919383]  driver_probe_device+0xe0/0x4c0
> > > 
> > > Exactly.  It's this, I think
> > > 
> > > 	bool test_remove = IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE)
> > > &&
> > > 			   !drv->suppress_bind_attrs;
> > > 
> > > You have that config option set.
> > 
> > Yes - or better said 0-day testing has it set. Maybe that is not a 
> > sane default for 0-day tests? The option is explicitely marked as
> > "unstable"... Fengguang?
> > 
> > > So the drivers base layer is calling ->remove after probe and
> > > triggering the destruction of the queue.
> > > 
> > > What to do about this (apart from nuke such a stupid option) is
> > > somewhat more problematic.
> > 
> > I guess this is between you and Greg :).
> 
> Nice try, sport ... you qualify just behind Dan in the "not my problem"
> olympic hurdles event.  I'm annoyed because there's no indication in
> the log that the driver add behaviour is radically altered, so I've
> been staring at the wrong code for weeks.  However, the unbind/rebind
> should work, so the real issue is that your bdi changes (or perhaps
> something else in block) have induced a regression in the unbinding of
> upper layer drivers.  If you're going to release the bdi in
> del_gendisk, you have to have some mechanism for re-acquiring it on re
> -probe (likely with the same name) otherwise rebind is broken for every
> block driver.

So my patch does not release bdi in del_gendisk(). Bdi has two
initialization / destruction phases (similarly to request queue). You
allocate and initialize bdi through bdi_init(), then you call
bdi_register() to register it (which happens in device_add_disk()). On
shutdown you have to first call bdi_unregister() (used to be called from
blk_cleanup_queue(), my patch moved it to del_gendisk()). After that the
last reference to bdi may be dropped which does final bdi destruction.

So do I understand correctly that SCSI may call device_add_disk(),
del_gendisk() repeatedly for the same request queue? If yes, then indeed I
have a bug to fix... But gendisk seems to get allocated from scratch on
each probe so we don't call device_add_disk(), del_gendisk() more times on
the same disk, right?

> The fact that the second rebind tried with a different name indicates
> that sd_devt_release wasn't called, so some vestige of the devt remains
> on the subsequent rebind.

Yep, I guess that's caused by Dan's patch (commit 0dba1314d4f8 now) which
calls put_disk_devt() only in blk_cleanup_queue() which, if I understood
you correctly, does not get called during unbind-bind cycle? In fact Dan's
patch would end up leaking devt's because of repeated device_add_disk()
calls for the same request queue...

> Here's the problem: the queue belongs to SCSI (the lower layer), so it's
> not going to change because it doesn't see the release.  The gendisk and
> its allied stuff belongs to sd so it gets freed and re-created for the
> same queue.  Something in block is very confused when this happens.

Yep, I think the binding of request queue to different gendisks is
something I or Dan did not expect.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-07 14:41               ` Jan Kara
@ 2017-03-07 16:10                 ` James Bottomley
  2017-03-07 16:26                   ` Jan Kara
  0 siblings, 1 reply; 10+ messages in thread
From: James Bottomley @ 2017-03-07 16:10 UTC (permalink / raw)
  To: Jan Kara
  Cc: Fengguang Wu, Jens Axboe, linux-block, linux-kernel, LKP,
	Martin K. Petersen, linux-scsi

On Tue, 2017-03-07 at 15:41 +0100, Jan Kara wrote:
> On Mon 06-03-17 09:25:42, James Bottomley wrote:
> > On Mon, 2017-03-06 at 17:13 +0100, Jan Kara wrote:
> > > On Mon 06-03-17 07:44:55, James Bottomley wrote:
> ...
> > > > > Sure. The call trace is:
> > > > > 
> > > > > [   41.919244] ------------[ cut here ]------------
> > > > > [   41.919263] WARNING: CPU: 4 PID: 2335 at
> > > > > drivers/scsi/sd.c:3332
> > > > > sd_shutdown+0x2f/0x100
> > > > > [   41.919268] Modules linked in: scsi_debug(+) netconsole 
> > > > > loop btrfs raid6_pq zlib_deflate lzo_compress xor
> > > > > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 
> > > > > 4.11.0-rc1-xen+ #49
> > > > > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs
> > > > > 01/01/2011
> > > > > [   41.919331] Call Trace:
> > > > > [   41.919343]  dump_stack+0x8e/0xf0
> > > > > [   41.919354]  __warn+0x116/0x120
> > > > > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > > > > [   41.919368]  sd_shutdown+0x2f/0x100
> > > > > [   41.919374]  sd_remove+0x70/0xd0
> > > > > 
> > > > > *** Here is the unexpected step I guess...
> > > > > 
> > > > > [   41.919383]  driver_probe_device+0xe0/0x4c0
> > > > 
> > > > Exactly.  It's this, I think
> > > > 
> > > > 	bool test_remove =
> > > > IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE)
> > > > &&
> > > > 			   !drv->suppress_bind_attrs;
> > > > 
> > > > You have that config option set.
> > > 
> > > Yes - or better said 0-day testing has it set. Maybe that is not 
> > > a sane default for 0-day tests? The option is explicitely marked 
> > > as "unstable"... Fengguang?
> > > 
> > > > So the drivers base layer is calling ->remove after probe and
> > > > triggering the destruction of the queue.
> > > > 
> > > > What to do about this (apart from nuke such a stupid option) is
> > > > somewhat more problematic.
> > > 
> > > I guess this is between you and Greg :).
> > 
> > Nice try, sport ... you qualify just behind Dan in the "not my 
> > problem" olympic hurdles event.  I'm annoyed because there's no 
> > indication in the log that the driver add behaviour is radically 
> > altered, so I've been staring at the wrong code for weeks. 
> >  However, the unbind/rebind should work, so the real issue is that 
> > your bdi changes (or perhaps something else in block) have induced 
> > a regression in the unbinding of upper layer drivers.  If you're 
> > going to release the bdi in del_gendisk, you have to have some 
> > mechanism for re-acquiring it on re-probe (likely with the same 
> > name) otherwise rebind is broken for every block driver.
> 
> So my patch does not release bdi in del_gendisk(). Bdi has two
> initialization / destruction phases (similarly to request queue). You
> allocate and initialize bdi through bdi_init(), then you call
> bdi_register() to register it (which happens in device_add_disk()). 
> On shutdown you have to first call bdi_unregister() (used to be 
> called from blk_cleanup_queue(), my patch moved it to del_gendisk()). 
> After that the last reference to bdi may be dropped which does final 
> bdi destruction.
> 
> So do I understand correctly that SCSI may call device_add_disk(),
> del_gendisk() repeatedly for the same request queue?

Yes.  The upper drivers (sd, sr, st and sg) follow a device model. 
 They can thus be bound and unbound many times during the lifetime of a
SCSI device.

>  If yes, then indeed I have a bug to fix... But gendisk seems to get 
> allocated from scratch on each probe so we don't call 
> device_add_disk(), del_gendisk() more times on the same disk, right?

Right, gendisk, being a generic representation of a disk is a property
of the upper layer drivers.  We actually cheat and use it in all of
them (including the apparent character ones, they use alloc_disk,
put_disk but not add_disk).  So it has to be freed when the driver is
unbound and reallocated when it is bound.  It's the fundamental entity
which embeds the SCSI upper layer driver lifetime.

> > The fact that the second rebind tried with a different name 
> > indicates that sd_devt_release wasn't called, so some vestige of 
> > the devt remains on the subsequent rebind.
> 
> Yep, I guess that's caused by Dan's patch (commit 0dba1314d4f8 now) 
> which calls put_disk_devt() only in blk_cleanup_queue() which, if I
> understood you correctly, does not get called during unbind-bind 
> cycle? In fact Dan's patch would end up leaking devt's because of 
> repeated device_add_disk() calls for the same request queue...

That's a bit unfortunate.

> > Here's the problem: the queue belongs to SCSI (the lower layer), so 
> > it's not going to change because it doesn't see the release.  The
> > gendisk and its allied stuff belongs to sd so it gets freed and re
> > -created for the same queue.  Something in block is very confused
> > when this happens.
> 
> Yep, I think the binding of request queue to different gendisks is
> something I or Dan did not expect.

OK, so I think we now understand why this is happening ... the question
is what we do about it?  Is it fixable or do we need to go back to
basics?  The fundamental property which got broken by these patches is
the idea of the separation of the lifecycles of the queue and the
gendisk.  The only requirement is that the queue must exist before the
gendisk is created and must be destroyed after it.  This doesn't forbid
there being many gendisk lifetimes over one queue lifetime and this is
what we rely on for the SCSI driver model.  We rarely get called on it
because unbind/rebind of a ULD is rarely done in production (it's
mostly a devtest thing).

James

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-07 16:10                 ` James Bottomley
@ 2017-03-07 16:26                   ` Jan Kara
  2017-03-08  9:59                     ` Jan Kara
  0 siblings, 1 reply; 10+ messages in thread
From: Jan Kara @ 2017-03-07 16:26 UTC (permalink / raw)
  To: James Bottomley
  Cc: Jan Kara, Fengguang Wu, Jens Axboe, linux-block, linux-kernel,
	LKP, Martin K. Petersen, linux-scsi

On Tue 07-03-17 08:10:29, James Bottomley wrote:
> On Tue, 2017-03-07 at 15:41 +0100, Jan Kara wrote:
> > On Mon 06-03-17 09:25:42, James Bottomley wrote:
> > > On Mon, 2017-03-06 at 17:13 +0100, Jan Kara wrote:
> > > > On Mon 06-03-17 07:44:55, James Bottomley wrote:
> > ...
> > > > > > Sure. The call trace is:
> > > > > > 
> > > > > > [   41.919244] ------------[ cut here ]------------
> > > > > > [   41.919263] WARNING: CPU: 4 PID: 2335 at
> > > > > > drivers/scsi/sd.c:3332
> > > > > > sd_shutdown+0x2f/0x100
> > > > > > [   41.919268] Modules linked in: scsi_debug(+) netconsole 
> > > > > > loop btrfs raid6_pq zlib_deflate lzo_compress xor
> > > > > > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 
> > > > > > 4.11.0-rc1-xen+ #49
> > > > > > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs
> > > > > > 01/01/2011
> > > > > > [   41.919331] Call Trace:
> > > > > > [   41.919343]  dump_stack+0x8e/0xf0
> > > > > > [   41.919354]  __warn+0x116/0x120
> > > > > > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > > > > > [   41.919368]  sd_shutdown+0x2f/0x100
> > > > > > [   41.919374]  sd_remove+0x70/0xd0
> > > > > > 
> > > > > > *** Here is the unexpected step I guess...
> > > > > > 
> > > > > > [   41.919383]  driver_probe_device+0xe0/0x4c0
> > > > > 
> > > > > Exactly.  It's this, I think
> > > > > 
> > > > > 	bool test_remove =
> > > > > IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE)
> > > > > &&
> > > > > 			   !drv->suppress_bind_attrs;
> > > > > 
> > > > > You have that config option set.
> > > > 
> > > > Yes - or better said 0-day testing has it set. Maybe that is not 
> > > > a sane default for 0-day tests? The option is explicitely marked 
> > > > as "unstable"... Fengguang?
> > > > 
> > > > > So the drivers base layer is calling ->remove after probe and
> > > > > triggering the destruction of the queue.
> > > > > 
> > > > > What to do about this (apart from nuke such a stupid option) is
> > > > > somewhat more problematic.
> > > > 
> > > > I guess this is between you and Greg :).
> > > 
> > > Nice try, sport ... you qualify just behind Dan in the "not my 
> > > problem" olympic hurdles event.  I'm annoyed because there's no 
> > > indication in the log that the driver add behaviour is radically 
> > > altered, so I've been staring at the wrong code for weeks. 
> > >  However, the unbind/rebind should work, so the real issue is that 
> > > your bdi changes (or perhaps something else in block) have induced 
> > > a regression in the unbinding of upper layer drivers.  If you're 
> > > going to release the bdi in del_gendisk, you have to have some 
> > > mechanism for re-acquiring it on re-probe (likely with the same 
> > > name) otherwise rebind is broken for every block driver.
> > 
> > So my patch does not release bdi in del_gendisk(). Bdi has two
> > initialization / destruction phases (similarly to request queue). You
> > allocate and initialize bdi through bdi_init(), then you call
> > bdi_register() to register it (which happens in device_add_disk()). 
> > On shutdown you have to first call bdi_unregister() (used to be 
> > called from blk_cleanup_queue(), my patch moved it to del_gendisk()). 
> > After that the last reference to bdi may be dropped which does final 
> > bdi destruction.
> > 
> > So do I understand correctly that SCSI may call device_add_disk(),
> > del_gendisk() repeatedly for the same request queue?
> 
> Yes.  The upper drivers (sd, sr, st and sg) follow a device model. 
>  They can thus be bound and unbound many times during the lifetime of a
> SCSI device.
> 
> >  If yes, then indeed I have a bug to fix... But gendisk seems to get 
> > allocated from scratch on each probe so we don't call 
> > device_add_disk(), del_gendisk() more times on the same disk, right?
> 
> Right, gendisk, being a generic representation of a disk is a property
> of the upper layer drivers.  We actually cheat and use it in all of
> them (including the apparent character ones, they use alloc_disk,
> put_disk but not add_disk).  So it has to be freed when the driver is
> unbound and reallocated when it is bound.  It's the fundamental entity
> which embeds the SCSI upper layer driver lifetime.
> 
> > > The fact that the second rebind tried with a different name 
> > > indicates that sd_devt_release wasn't called, so some vestige of 
> > > the devt remains on the subsequent rebind.
> > 
> > Yep, I guess that's caused by Dan's patch (commit 0dba1314d4f8 now) 
> > which calls put_disk_devt() only in blk_cleanup_queue() which, if I
> > understood you correctly, does not get called during unbind-bind 
> > cycle? In fact Dan's patch would end up leaking devt's because of 
> > repeated device_add_disk() calls for the same request queue...
> 
> That's a bit unfortunate.
> 
> > > Here's the problem: the queue belongs to SCSI (the lower layer), so 
> > > it's not going to change because it doesn't see the release.  The
> > > gendisk and its allied stuff belongs to sd so it gets freed and re
> > > -created for the same queue.  Something in block is very confused
> > > when this happens.
> > 
> > Yep, I think the binding of request queue to different gendisks is
> > something I or Dan did not expect.
> 
> OK, so I think we now understand why this is happening ... the question
> is what we do about it?  Is it fixable or do we need to go back to
> basics?  The fundamental property which got broken by these patches is
> the idea of the separation of the lifecycles of the queue and the
> gendisk.  The only requirement is that the queue must exist before the
> gendisk is created and must be destroyed after it.  This doesn't forbid
> there being many gendisk lifetimes over one queue lifetime and this is
> what we rely on for the SCSI driver model.  We rarely get called on it
> because unbind/rebind of a ULD is rarely done in production (it's
> mostly a devtest thing).

For now I think it's fixable. Let's see what I'll be able to come up with.
Let me also note that before my patch moving bdi_unregister() to
del_gendisk() (commit 165a5e22fafb), the SCSI driver model was buggy in a
different way - because bdi_unregister() was called only from
blk_cleanup_queue(), we were leaking bdi->owner reference to gendisk by
repeated calls to bdi_register_owner() from device_add_disk() and
thus gendisks were never freed after unbind/bind (I've actually verified
this bug exists with debug messages). Also as a result request queue use
count was steadily increasing with each new gendisk and request queue could
never get freed after unbind / rebind.

My patch silently fixed this leakage but introduced a different set of
problems which I'll try to fix.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds.
  2017-03-07 16:26                   ` Jan Kara
@ 2017-03-08  9:59                     ` Jan Kara
  0 siblings, 0 replies; 10+ messages in thread
From: Jan Kara @ 2017-03-08  9:59 UTC (permalink / raw)
  To: James Bottomley
  Cc: Jan Kara, Fengguang Wu, Jens Axboe, linux-block, linux-kernel,
	LKP, Martin K. Petersen, linux-scsi

[-- Attachment #1: Type: text/plain, Size: 7412 bytes --]

On Tue 07-03-17 17:26:56, Jan Kara wrote:
> On Tue 07-03-17 08:10:29, James Bottomley wrote:
> > On Tue, 2017-03-07 at 15:41 +0100, Jan Kara wrote:
> > > On Mon 06-03-17 09:25:42, James Bottomley wrote:
> > > > On Mon, 2017-03-06 at 17:13 +0100, Jan Kara wrote:
> > > > > On Mon 06-03-17 07:44:55, James Bottomley wrote:
> > > ...
> > > > > > > Sure. The call trace is:
> > > > > > > 
> > > > > > > [   41.919244] ------------[ cut here ]------------
> > > > > > > [   41.919263] WARNING: CPU: 4 PID: 2335 at
> > > > > > > drivers/scsi/sd.c:3332
> > > > > > > sd_shutdown+0x2f/0x100
> > > > > > > [   41.919268] Modules linked in: scsi_debug(+) netconsole 
> > > > > > > loop btrfs raid6_pq zlib_deflate lzo_compress xor
> > > > > > > [   41.919319] CPU: 4 PID: 2335 Comm: modprobe Not tainted 
> > > > > > > 4.11.0-rc1-xen+ #49
> > > > > > > [   41.919325] Hardware name: Bochs Bochs, BIOS Bochs
> > > > > > > 01/01/2011
> > > > > > > [   41.919331] Call Trace:
> > > > > > > [   41.919343]  dump_stack+0x8e/0xf0
> > > > > > > [   41.919354]  __warn+0x116/0x120
> > > > > > > [   41.919361]  warn_slowpath_null+0x1d/0x20
> > > > > > > [   41.919368]  sd_shutdown+0x2f/0x100
> > > > > > > [   41.919374]  sd_remove+0x70/0xd0
> > > > > > > 
> > > > > > > *** Here is the unexpected step I guess...
> > > > > > > 
> > > > > > > [   41.919383]  driver_probe_device+0xe0/0x4c0
> > > > > > 
> > > > > > Exactly.  It's this, I think
> > > > > > 
> > > > > > 	bool test_remove =
> > > > > > IS_ENABLED(CONFIG_DEBUG_TEST_DRIVER_REMOVE)
> > > > > > &&
> > > > > > 			   !drv->suppress_bind_attrs;
> > > > > > 
> > > > > > You have that config option set.
> > > > > 
> > > > > Yes - or better said 0-day testing has it set. Maybe that is not 
> > > > > a sane default for 0-day tests? The option is explicitely marked 
> > > > > as "unstable"... Fengguang?
> > > > > 
> > > > > > So the drivers base layer is calling ->remove after probe and
> > > > > > triggering the destruction of the queue.
> > > > > > 
> > > > > > What to do about this (apart from nuke such a stupid option) is
> > > > > > somewhat more problematic.
> > > > > 
> > > > > I guess this is between you and Greg :).
> > > > 
> > > > Nice try, sport ... you qualify just behind Dan in the "not my 
> > > > problem" olympic hurdles event.  I'm annoyed because there's no 
> > > > indication in the log that the driver add behaviour is radically 
> > > > altered, so I've been staring at the wrong code for weeks. 
> > > >  However, the unbind/rebind should work, so the real issue is that 
> > > > your bdi changes (or perhaps something else in block) have induced 
> > > > a regression in the unbinding of upper layer drivers.  If you're 
> > > > going to release the bdi in del_gendisk, you have to have some 
> > > > mechanism for re-acquiring it on re-probe (likely with the same 
> > > > name) otherwise rebind is broken for every block driver.
> > > 
> > > So my patch does not release bdi in del_gendisk(). Bdi has two
> > > initialization / destruction phases (similarly to request queue). You
> > > allocate and initialize bdi through bdi_init(), then you call
> > > bdi_register() to register it (which happens in device_add_disk()). 
> > > On shutdown you have to first call bdi_unregister() (used to be 
> > > called from blk_cleanup_queue(), my patch moved it to del_gendisk()). 
> > > After that the last reference to bdi may be dropped which does final 
> > > bdi destruction.
> > > 
> > > So do I understand correctly that SCSI may call device_add_disk(),
> > > del_gendisk() repeatedly for the same request queue?
> > 
> > Yes.  The upper drivers (sd, sr, st and sg) follow a device model. 
> >  They can thus be bound and unbound many times during the lifetime of a
> > SCSI device.
> > 
> > >  If yes, then indeed I have a bug to fix... But gendisk seems to get 
> > > allocated from scratch on each probe so we don't call 
> > > device_add_disk(), del_gendisk() more times on the same disk, right?
> > 
> > Right, gendisk, being a generic representation of a disk is a property
> > of the upper layer drivers.  We actually cheat and use it in all of
> > them (including the apparent character ones, they use alloc_disk,
> > put_disk but not add_disk).  So it has to be freed when the driver is
> > unbound and reallocated when it is bound.  It's the fundamental entity
> > which embeds the SCSI upper layer driver lifetime.
> > 
> > > > The fact that the second rebind tried with a different name 
> > > > indicates that sd_devt_release wasn't called, so some vestige of 
> > > > the devt remains on the subsequent rebind.
> > > 
> > > Yep, I guess that's caused by Dan's patch (commit 0dba1314d4f8 now) 
> > > which calls put_disk_devt() only in blk_cleanup_queue() which, if I
> > > understood you correctly, does not get called during unbind-bind 
> > > cycle? In fact Dan's patch would end up leaking devt's because of 
> > > repeated device_add_disk() calls for the same request queue...
> > 
> > That's a bit unfortunate.
> > 
> > > > Here's the problem: the queue belongs to SCSI (the lower layer), so 
> > > > it's not going to change because it doesn't see the release.  The
> > > > gendisk and its allied stuff belongs to sd so it gets freed and re
> > > > -created for the same queue.  Something in block is very confused
> > > > when this happens.
> > > 
> > > Yep, I think the binding of request queue to different gendisks is
> > > something I or Dan did not expect.
> > 
> > OK, so I think we now understand why this is happening ... the question
> > is what we do about it?  Is it fixable or do we need to go back to
> > basics?  The fundamental property which got broken by these patches is
> > the idea of the separation of the lifecycles of the queue and the
> > gendisk.  The only requirement is that the queue must exist before the
> > gendisk is created and must be destroyed after it.  This doesn't forbid
> > there being many gendisk lifetimes over one queue lifetime and this is
> > what we rely on for the SCSI driver model.  We rarely get called on it
> > because unbind/rebind of a ULD is rarely done in production (it's
> > mostly a devtest thing).
> 
> For now I think it's fixable. Let's see what I'll be able to come up with.
> Let me also note that before my patch moving bdi_unregister() to
> del_gendisk() (commit 165a5e22fafb), the SCSI driver model was buggy in a
> different way - because bdi_unregister() was called only from
> blk_cleanup_queue(), we were leaking bdi->owner reference to gendisk by
> repeated calls to bdi_register_owner() from device_add_disk() and
> thus gendisks were never freed after unbind/bind (I've actually verified
> this bug exists with debug messages). Also as a result request queue use
> count was steadily increasing with each new gendisk and request queue could
> never get freed after unbind / rebind.
> 
> My patch silently fixed this leakage but introduced a different set of
> problems which I'll try to fix.

Attached patch fixes the problem for me. If I also revert commit
0dba1314d4f81115dce711292ec7981d17231064 "scsi, block: fix duplicate bdi
name registration crashes" which is not needed with my
165a5e22fafb127ecb5914e12e8c32a1f0d3f820 "block: Move bdi_unregister() to
del_gendisk()", the device names also do not get incremented and I end up
with 'sda' device (otherwise I end up with sdc).

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

[-- Attachment #2: 0001-block-Allow-bdi-re-registration.patch --]
[-- Type: text/x-patch, Size: 1507 bytes --]

>From bfbccf28853d55d4418401d8eeb6b91c3050e316 Mon Sep 17 00:00:00 2001
From: Jan Kara <jack@suse.cz>
Date: Tue, 7 Mar 2017 18:01:51 +0100
Subject: [PATCH] block: Allow bdi re-registration

SCSI can call device_add_disk() several times for one request queue when
a device in unbound and bound, creating new gendisk each time. This will
lead to bdi being repeatedly registered and unregistered. Make sure
bdi_unregister() cleans up bdi in such a way that it is prepared for
the following bdi register call.

An easy way to provoke this behavior is to enable
CONFIG_DEBUG_TEST_DRIVER_REMOVE and use scsi_debug driver to create a
scsi disk which immediately hangs without this fix.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 mm/backing-dev.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index 6d861d090e9f..6ac932210f56 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -710,6 +710,11 @@ static void cgwb_bdi_destroy(struct backing_dev_info *bdi)
 	 */
 	atomic_dec(&bdi->usage_cnt);
 	wait_event(cgwb_release_wait, !atomic_read(&bdi->usage_cnt));
+	/*
+	 * Grab back our reference so that we hold it when @bdi gets
+	 * re-registered.
+	 */
+	atomic_inc(&bdi->usage_cnt);
 }
 
 /**
@@ -857,6 +862,8 @@ int bdi_register_owner(struct backing_dev_info *bdi, struct device *owner)
 			MINOR(owner->devt));
 	if (rc)
 		return rc;
+	/* Leaking owner reference... */
+	WARN_ON(bdi->owner);
 	bdi->owner = owner;
 	get_device(owner);
 	return 0;
-- 
2.10.2


^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-03-08  9:59 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20170305022111.yqtwyd6ognmco4gx@wfg-t540p.sh.intel.com>
     [not found] ` <20170306102733.GC14932@quack2.suse.cz>
2017-03-06 12:01   ` [bdi_unregister] 165a5e22fa INFO: task swapper:1 blocked for more than 120 seconds Jan Kara
2017-03-06 14:35     ` James Bottomley
2017-03-06 15:14       ` Jan Kara
2017-03-06 15:44         ` James Bottomley
2017-03-06 16:13           ` Jan Kara
2017-03-06 17:25             ` James Bottomley
2017-03-07 14:41               ` Jan Kara
2017-03-07 16:10                 ` James Bottomley
2017-03-07 16:26                   ` Jan Kara
2017-03-08  9:59                     ` Jan Kara

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox