* Re: req->nr_phys_segments > queue_max_segments (was Re: kernel BUG at drivers/block/virtio_blk.c:172!)
From: Jens Axboe @ 2015-10-01 10:16 UTC (permalink / raw)
To: Michael S. Tsirkin, Thomas D.; +Cc: Christoph Hellwig, virtualization
In-Reply-To: <20151001114817-mutt-send-email-mst@redhat.com>
On 10/01/2015 11:00 AM, Michael S. Tsirkin wrote:
> On Thu, Oct 01, 2015 at 03:10:14AM +0200, Thomas D. wrote:
>> Hi,
>>
>> I have a virtual machine which fails to boot linux-4.1.8 while mounting
>> file systems:
>>
>>> * Mounting local filesystem ...
>>> ------------[ cut here ]------------
>>> kernel BUG at drivers/block/virtio_blk.c:172!
>>> invalid opcode: 000 [#1] SMP
>>> Modules linked in: pcspkr psmouse dm_log_userspace virtio_net e1000 fuse nfs lockd grace sunrpc fscache dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log usbhid usb_storage sr_mod cdrom
>>> CPU: 7 PIDL 2254 Comm: dmcrypt_write Not tainted 4.1.8-gentoo #1
>>> Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
>>> task: ffff88061fb70000 ti: ffff88061ff30000 task.ti: ffff88061ff30000
>>> RIP: 0010:[<ffffffffb4557b30>] [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
>>> RSP: 0018:ffff88061ff33ba8 EFLAGS: 00010202
>>> RAX: 00000000000000b1 RBX: ffff88061fb2fc00 RCX: ffff88061ff33c30
>>> RDX: 0000000000000008 RSI: ffff88061ff33c50 RDI: ffff88061fb2fc00
>>> RBP: ffff88061ff33bf8 R08: ffff88061eef3540 R09: ffff88061ff33c30
>>> R10: 0000000000000000 R11: 00000000000000af R12: 0000000000000000
>>> R13: ffff88061eef3540 R14: ffff88061eef3540 R15: ffff880622c7ca80
>>> FS: 0000000000000000(0000) GS:ffff88063fdc0000(0000) knlGS:0000000000000000
>>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> CR2: 0000000001ffe468 CR3: 00000000bb343000 CR4: 00000000001406e0
>>> Stack:
>>> ffff880622d4c478 0000000000000000 ffff88061ff33bd8 ffff88061fb2f
>>> 0000000000000001 ffff88061fb2fc00 ffff88061ff33c30 0000000000000
>>> ffff88061eef3540 0000000000000000 ffff88061ff33c98 ffffffffb43eb
>>>
>>> Call Trace:
>>> [<ffffffffb43eb500>] __blk_mq_run_hw_queue+0x1d0/0x370
>>> [<ffffffffb43eb315>] blk_mq_run_hw_queue+0x95/0xb0
>>> [<ffffffffb43ec804>] blk_mq_flush_plug_list+0x129/0x140
>>> [<ffffffffb43e33d8>] blk_finish_plug+0x18/0x50
>>> [<ffffffffb45e3bea>] dmcrypt_write+0x1da/0x1f0
>>> [<ffffffffb4108c90>] ? wake_up_state+0x20/0x20
>>> [<ffffffffb45e3a10>] ? crypt_iv_lmk_dtr+0x60/0x60
>>> [<ffffffffb40fb789>] kthread_create_on_node+0x180/0x180
>>> [<ffffffffb4705e92>] ret_from_fork+0x42/0x70
>>> [<ffffffffb40fb6c0>] ? kthread_create_on_node+0x180/0x180
>>> Code: 00 0000 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 00 00 41 89 85 7c 01 00 00 e9 93 fe ff ff 66 0f 1f 44 00 00 <0f> 0b 66 0f 1f 44 00 00 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b
>>> RIP [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
>>> RSP: <ffff88061ff33ba8>
>>> ---[ end trace 8078357c459d5fc0 ]---
>
>
> So this BUG_ON is from 1cf7e9c68fe84248174e998922b39e508375e7c1.
> commit 1cf7e9c68fe84248174e998922b39e508375e7c1
> Author: Jens Axboe <axboe@kernel.dk>
> Date: Fri Nov 1 10:52:52 2013 -0600
>
> virtio_blk: blk-mq support
>
>
> BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems);
>
>
> On probe, we do
> /* We can handle whatever the host told us to handle. */
> blk_queue_max_segments(q, vblk->sg_elems-2);
>
>
> To debug this,
> maybe you can print out sg_elems at init time and when this fails,
> to make sure some kind of memory corruption
> does not change sg_elems after initialization?
>
>
> Jens, how may we get more segments than blk_queue_max_segments?
> Is driver expected to validate and drop such requests?
The answer is that this should not happen. If the driver informs of a
limit on the number of segments, that should never be exceeded. If it
does, then it's a bug in either the SG mapping, or in the building of
the request - either the request gets built too large for some reason,
or the mapping doesn't always coalesce segments even though it should.
The problem is that we get notified out-of-band, when we attempt to push
the request to the driver. At this point, much of the context could be
lost, like it is in your case.
Looking at the specific virtio_blk case, it does seem that it is
checking the segment count before mapping. Does the below fix the
problem, or does the BUG_ON() still trigger?
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index 6ca35495a5be..1501701b0202 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -169,8 +169,6 @@ static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
int err;
bool notify = false;
- BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems);
-
vbr->req = req;
if (req->cmd_flags & REQ_FLUSH) {
vbr->out_hdr.type = cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_FLUSH);
@@ -203,6 +201,7 @@ static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
num = blk_rq_map_sg(hctx->queue, vbr->req, vbr->sg);
if (num) {
+ BUG_ON(num + 2 > vblk->sg_elems);
if (rq_data_dir(vbr->req) == WRITE)
vbr->out_hdr.type |= cpu_to_virtio32(vblk->vdev, VIRTIO_BLK_T_OUT);
else
--
Jens Axboe
^ permalink raw reply related
* Re: req->nr_phys_segments > queue_max_segments (was Re: kernel BUG at drivers/block/virtio_blk.c:172!)
From: Hannes Reinecke @ 2015-10-01 9:20 UTC (permalink / raw)
To: Michael S. Tsirkin, Thomas D.
Cc: Jens Axboe, Christoph Hellwig, linux-scsi@vger.kernel.org,
virtualization
In-Reply-To: <20151001114817-mutt-send-email-mst@redhat.com>
On 10/01/2015 11:00 AM, Michael S. Tsirkin wrote:
> On Thu, Oct 01, 2015 at 03:10:14AM +0200, Thomas D. wrote:
>> Hi,
>>
>> I have a virtual machine which fails to boot linux-4.1.8 while mounting
>> file systems:
>>
>>> * Mounting local filesystem ...
>>> ------------[ cut here ]------------
>>> kernel BUG at drivers/block/virtio_blk.c:172!
>>> invalid opcode: 000 [#1] SMP
>>> Modules linked in: pcspkr psmouse dm_log_userspace virtio_net e1000 fuse nfs lockd grace sunrpc fscache dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log usbhid usb_storage sr_mod cdrom
>>> CPU: 7 PIDL 2254 Comm: dmcrypt_write Not tainted 4.1.8-gentoo #1
>>> Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
>>> task: ffff88061fb70000 ti: ffff88061ff30000 task.ti: ffff88061ff30000
>>> RIP: 0010:[<ffffffffb4557b30>] [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
>>> RSP: 0018:ffff88061ff33ba8 EFLAGS: 00010202
>>> RAX: 00000000000000b1 RBX: ffff88061fb2fc00 RCX: ffff88061ff33c30
>>> RDX: 0000000000000008 RSI: ffff88061ff33c50 RDI: ffff88061fb2fc00
>>> RBP: ffff88061ff33bf8 R08: ffff88061eef3540 R09: ffff88061ff33c30
>>> R10: 0000000000000000 R11: 00000000000000af R12: 0000000000000000
>>> R13: ffff88061eef3540 R14: ffff88061eef3540 R15: ffff880622c7ca80
>>> FS: 0000000000000000(0000) GS:ffff88063fdc0000(0000) knlGS:0000000000000000
>>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> CR2: 0000000001ffe468 CR3: 00000000bb343000 CR4: 00000000001406e0
>>> Stack:
>>> ffff880622d4c478 0000000000000000 ffff88061ff33bd8 ffff88061fb2f
>>> 0000000000000001 ffff88061fb2fc00 ffff88061ff33c30 0000000000000
>>> ffff88061eef3540 0000000000000000 ffff88061ff33c98 ffffffffb43eb
>>>
>>> Call Trace:
>>> [<ffffffffb43eb500>] __blk_mq_run_hw_queue+0x1d0/0x370
>>> [<ffffffffb43eb315>] blk_mq_run_hw_queue+0x95/0xb0
>>> [<ffffffffb43ec804>] blk_mq_flush_plug_list+0x129/0x140
>>> [<ffffffffb43e33d8>] blk_finish_plug+0x18/0x50
>>> [<ffffffffb45e3bea>] dmcrypt_write+0x1da/0x1f0
>>> [<ffffffffb4108c90>] ? wake_up_state+0x20/0x20
>>> [<ffffffffb45e3a10>] ? crypt_iv_lmk_dtr+0x60/0x60
>>> [<ffffffffb40fb789>] kthread_create_on_node+0x180/0x180
>>> [<ffffffffb4705e92>] ret_from_fork+0x42/0x70
>>> [<ffffffffb40fb6c0>] ? kthread_create_on_node+0x180/0x180
>>> Code: 00 0000 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 00 00 41 89 85 7c 01 00 00 e9 93 fe ff ff 66 0f 1f 44 00 00 <0f> 0b 66 0f 1f 44 00 00 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b
>>> RIP [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
>>> RSP: <ffff88061ff33ba8>
>>> ---[ end trace 8078357c459d5fc0 ]---
>
>
> So this BUG_ON is from 1cf7e9c68fe84248174e998922b39e508375e7c1.
> commit 1cf7e9c68fe84248174e998922b39e508375e7c1
> Author: Jens Axboe <axboe@kernel.dk>
> Date: Fri Nov 1 10:52:52 2013 -0600
>
> virtio_blk: blk-mq support
>
>
> BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems);
>
>
> On probe, we do
> /* We can handle whatever the host told us to handle. */
> blk_queue_max_segments(q, vblk->sg_elems-2);
>
>
> To debug this,
> maybe you can print out sg_elems at init time and when this fails,
> to make sure some kind of memory corruption
> does not change sg_elems after initialization?
>
>
> Jens, how may we get more segments than blk_queue_max_segments?
> Is driver expected to validate and drop such requests?
>
Whee! I'm not alone anymore!
I have seen similar issues even on non-mq systems; occasionally
I'm hitting this bug in drivers/scsi/scsi_lib.c:scsi_init_io()
count = blk_rq_map_sg(req->q, req, sdb->table.sgl);
BUG_ON(count > sdb->table.nents);
There are actually two problems here:
The one is that blk_rq_map_sg() requires a table (ie the last
argument), but doesn't have any indications on how large the
table is.
So one needs to check if the returned number of mapped sg
elements exceeds the number of elements in the table.
If so we already _have_ a memory overflow, and the only
thing we can to is sit in a corner and cry.
This really would need to be fixed up, eg by adding
another argument for the table size.
This other problem is that this _really_ shouldn't happen,
and points to some issue with the block layer in general.
Which I've been trying to find for several months now,
but no avail :-(
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
^ permalink raw reply
* req->nr_phys_segments > queue_max_segments (was Re: kernel BUG at drivers/block/virtio_blk.c:172!)
From: Michael S. Tsirkin @ 2015-10-01 9:00 UTC (permalink / raw)
To: Thomas D.; +Cc: Jens Axboe, Christoph Hellwig, virtualization
In-Reply-To: <560C87F6.4080708@whissi.de>
On Thu, Oct 01, 2015 at 03:10:14AM +0200, Thomas D. wrote:
> Hi,
>
> I have a virtual machine which fails to boot linux-4.1.8 while mounting
> file systems:
>
> > * Mounting local filesystem ...
> > ------------[ cut here ]------------
> > kernel BUG at drivers/block/virtio_blk.c:172!
> > invalid opcode: 000 [#1] SMP
> > Modules linked in: pcspkr psmouse dm_log_userspace virtio_net e1000 fuse nfs lockd grace sunrpc fscache dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log usbhid usb_storage sr_mod cdrom
> > CPU: 7 PIDL 2254 Comm: dmcrypt_write Not tainted 4.1.8-gentoo #1
> > Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
> > task: ffff88061fb70000 ti: ffff88061ff30000 task.ti: ffff88061ff30000
> > RIP: 0010:[<ffffffffb4557b30>] [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
> > RSP: 0018:ffff88061ff33ba8 EFLAGS: 00010202
> > RAX: 00000000000000b1 RBX: ffff88061fb2fc00 RCX: ffff88061ff33c30
> > RDX: 0000000000000008 RSI: ffff88061ff33c50 RDI: ffff88061fb2fc00
> > RBP: ffff88061ff33bf8 R08: ffff88061eef3540 R09: ffff88061ff33c30
> > R10: 0000000000000000 R11: 00000000000000af R12: 0000000000000000
> > R13: ffff88061eef3540 R14: ffff88061eef3540 R15: ffff880622c7ca80
> > FS: 0000000000000000(0000) GS:ffff88063fdc0000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 0000000001ffe468 CR3: 00000000bb343000 CR4: 00000000001406e0
> > Stack:
> > ffff880622d4c478 0000000000000000 ffff88061ff33bd8 ffff88061fb2f
> > 0000000000000001 ffff88061fb2fc00 ffff88061ff33c30 0000000000000
> > ffff88061eef3540 0000000000000000 ffff88061ff33c98 ffffffffb43eb
> >
> > Call Trace:
> > [<ffffffffb43eb500>] __blk_mq_run_hw_queue+0x1d0/0x370
> > [<ffffffffb43eb315>] blk_mq_run_hw_queue+0x95/0xb0
> > [<ffffffffb43ec804>] blk_mq_flush_plug_list+0x129/0x140
> > [<ffffffffb43e33d8>] blk_finish_plug+0x18/0x50
> > [<ffffffffb45e3bea>] dmcrypt_write+0x1da/0x1f0
> > [<ffffffffb4108c90>] ? wake_up_state+0x20/0x20
> > [<ffffffffb45e3a10>] ? crypt_iv_lmk_dtr+0x60/0x60
> > [<ffffffffb40fb789>] kthread_create_on_node+0x180/0x180
> > [<ffffffffb4705e92>] ret_from_fork+0x42/0x70
> > [<ffffffffb40fb6c0>] ? kthread_create_on_node+0x180/0x180
> > Code: 00 0000 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 00 00 41 89 85 7c 01 00 00 e9 93 fe ff ff 66 0f 1f 44 00 00 <0f> 0b 66 0f 1f 44 00 00 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b
> > RIP [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
> > RSP: <ffff88061ff33ba8>
> > ---[ end trace 8078357c459d5fc0 ]---
So this BUG_ON is from 1cf7e9c68fe84248174e998922b39e508375e7c1.
commit 1cf7e9c68fe84248174e998922b39e508375e7c1
Author: Jens Axboe <axboe@kernel.dk>
Date: Fri Nov 1 10:52:52 2013 -0600
virtio_blk: blk-mq support
BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems);
On probe, we do
/* We can handle whatever the host told us to handle. */
blk_queue_max_segments(q, vblk->sg_elems-2);
To debug this,
maybe you can print out sg_elems at init time and when this fails,
to make sure some kind of memory corruption
does not change sg_elems after initialization?
Jens, how may we get more segments than blk_queue_max_segments?
Is driver expected to validate and drop such requests?
>
> System details:
> ===============
> VM (KVM), I don't know any details about the host system.
>
> The disk (vda) is partitioned in vda1 (ext4, /boot) and vda2
> (LUKS-encrypted).
> On top of the LUKS volume I have LVM.
> On top of LVM I have several partitions, all XFS (so rootfs is on XFS!).
>
> 1/20 reboots seems to success.
>
> Today I tried to upgrade to linux-4.1.9. Mounting wasn't a problem
> anymore but after ~5min the system always dies with
>
> > NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s!
>
> (CPU number varies)
>
> dmesg:
>
> > [ 0.000000] Initializing cgroup subsys cpuset
> > [ 0.000000] Initializing cgroup subsys cpu
> > [ 0.000000] Initializing cgroup subsys cpuacct
> > [ 0.000000] Linux version 4.1.8-gentoo (root@sysresccd) (gcc version 4.9.3 (Gentoo 4.9.3 p1.2, pie-0.6.3) ) #1 SMP Thu Sep 24 03:44:37 CEST 2015
> > [ 0.000000] Command line: BOOT_IMAGE=/kernel-genkernel-x86_64-4.1.8-gentoo root=/dev/mapper/vps1storage-volRoot ro dolvm crypt_root=UUID=fdea18e4-ba0f-1234-1234-12345678 root=UUID=86dad6e6-e43e-1234-1234-987654321 rootfs=xfs scandelay=3 nomodeset
> > [ 0.000000] KERNEL supported cpus:
> > [ 0.000000] Intel GenuineIntel
> > [ 0.000000] e820: BIOS-provided physical RAM map:
> > [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
> > [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
> > [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
> > [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdefff] usable
> > [ 0.000000] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
> > [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
> > [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
> > [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000063fffffff] usable
> > [ 0.000000] NX (Execute Disable) protection: active
> > [ 0.000000] SMBIOS 2.8 present.
> > [ 0.000000] DMI: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
> > [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
> > [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
> > [ 0.000000] e820: last_pfn = 0x640000 max_arch_pfn = 0x400000000
> > [ 0.000000] MTRR default type: write-back
> > [ 0.000000] MTRR fixed ranges enabled:
> > [ 0.000000] 00000-9FFFF write-back
> > [ 0.000000] A0000-BFFFF uncachable
> > [ 0.000000] C0000-FFFFF write-protect
> > [ 0.000000] MTRR variable ranges enabled:
> > [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable
> > [ 0.000000] 1 disabled
> > [ 0.000000] 2 disabled
> > [ 0.000000] 3 disabled
> > [ 0.000000] 4 disabled
> > [ 0.000000] 5 disabled
> > [ 0.000000] 6 disabled
> > [ 0.000000] 7 disabled
> > [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WC UC- UC
> > [ 0.000000] e820: last_pfn = 0xbffdf max_arch_pfn = 0x400000000
> > [ 0.000000] found SMP MP-table at [mem 0x000f1fb0-0x000f1fbf] mapped at [ffff8800000f1fb0]
> > [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576
> > [ 0.000000] Using GB pages for direct mapping
> > [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
> > [ 0.000000] [mem 0x00000000-0x000fffff] page 4k
> > [ 0.000000] BRK [0x27016000, 0x27016fff] PGTABLE
> > [ 0.000000] BRK [0x27017000, 0x27017fff] PGTABLE
> > [ 0.000000] BRK [0x27018000, 0x27018fff] PGTABLE
> > [ 0.000000] init_memory_mapping: [mem 0x63fe00000-0x63fffffff]
> > [ 0.000000] [mem 0x63fe00000-0x63fffffff] page 1G
> > [ 0.000000] init_memory_mapping: [mem 0x620000000-0x63fdfffff]
> > [ 0.000000] [mem 0x620000000-0x63fdfffff] page 1G
> > [ 0.000000] init_memory_mapping: [mem 0x00100000-0xbffdefff]
> > [ 0.000000] [mem 0x00100000-0x001fffff] page 4k
> > [ 0.000000] [mem 0x00200000-0x3fffffff] page 2M
> > [ 0.000000] [mem 0x40000000-0x7fffffff] page 1G
> > [ 0.000000] [mem 0x80000000-0xbfdfffff] page 2M
> > [ 0.000000] [mem 0xbfe00000-0xbffdefff] page 4k
> > [ 0.000000] init_memory_mapping: [mem 0x100000000-0x61fffffff]
> > [ 0.000000] [mem 0x100000000-0x61fffffff] page 1G
> > [ 0.000000] RAMDISK: [mem 0x37a10000-0x37cfffff]
> > [ 0.000000] ACPI: Early table checksum verification disabled
> > [ 0.000000] ACPI: RSDP 0x00000000000F1F60 000014 (v00 BOCHS )
> > [ 0.000000] ACPI: RSDT 0x00000000BFFE163C 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
> > [ 0.000000] ACPI: FACP 0x00000000BFFE0677 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
> > [ 0.000000] ACPI: DSDT 0x00000000BFFDFB80 000AF7 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
> > [ 0.000000] ACPI: FACS 0x00000000BFFDFB40 000040
> > [ 0.000000] ACPI: SSDT 0x00000000BFFE06EB 000EA1 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
> > [ 0.000000] ACPI: APIC 0x00000000BFFE158C 0000B0 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
> > [ 0.000000] ACPI: Local APIC address 0xfee00000
> > [ 0.000000] No NUMA configuration found
> > [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000063fffffff]
> > [ 0.000000] NODE_DATA(0) allocated [mem 0x63ffd8000-0x63fffdfff]
> > [ 0.000000] [ffffea0000000000-ffffea0018ffffff] PMD -> [ffff880627600000-ffff88063f5fffff] on node 0
> > [ 0.000000] Zone ranges:
> > [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
> > [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
> > [ 0.000000] Normal [mem 0x0000000100000000-0x000000063fffffff]
> > [ 0.000000] Movable zone start for each node
> > [ 0.000000] Early memory node ranges
> > [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
> > [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffdefff]
> > [ 0.000000] node 0: [mem 0x0000000100000000-0x000000063fffffff]
> > [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000063fffffff]
> > [ 0.000000] On node 0 totalpages: 6291325
> > [ 0.000000] DMA zone: 64 pages used for memmap
> > [ 0.000000] DMA zone: 21 pages reserved
> > [ 0.000000] DMA zone: 3998 pages, LIFO batch:0
> > [ 0.000000] DMA32 zone: 12224 pages used for memmap
> > [ 0.000000] DMA32 zone: 782303 pages, LIFO batch:31
> > [ 0.000000] Normal zone: 86016 pages used for memmap
> > [ 0.000000] Normal zone: 5505024 pages, LIFO batch:31
> > [ 0.000000] ACPI: PM-Timer IO Port: 0x608
> > [ 0.000000] ACPI: Local APIC address 0xfee00000
> > [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
> > [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
> > [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
> > [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
> > [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
> > [ 0.000000] ACPI: IRQ0 used by override.
> > [ 0.000000] ACPI: IRQ5 used by override.
> > [ 0.000000] ACPI: IRQ9 used by override.
> > [ 0.000000] ACPI: IRQ10 used by override.
> > [ 0.000000] ACPI: IRQ11 used by override.
> > [ 0.000000] Using ACPI (MADT) for SMP configuration information
> > [ 0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
> > [ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
> > [ 0.000000] clocksource refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> > [ 0.000000] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:8 nr_node_ids:1
> > [ 0.000000] PERCPU: Embedded 33 pages/cpu @ffff88063fc00000 s95192 r8192 d31784 u262144
> > [ 0.000000] pcpu-alloc: s95192 r8192 d31784 u262144 alloc=1*2097152
> > [ 0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
> > [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 6193000
> > [ 0.000000] Policy zone: Normal
> > [ 0.000000] Kernel command line: BOOT_IMAGE=/kernel-genkernel-x86_64-4.1.8-gentoo root=/dev/mapper/vps1storage-volRoot ro dolvm crypt_root=UUID=fdea18e4-ba0f-4c82-8ebb-3dd160859ee8 root=UUID=86dad6e6-e43e-4b4e-a445-c2fe3d9aee13 rootfs=xfs scandelay=3 nomodeset
> > [ 0.000000] log_buf_len individual max cpu contribution: 32768 bytes
> > [ 0.000000] log_buf_len total cpu_extra contributions: 229376 bytes
> > [ 0.000000] log_buf_len min size: 262144 bytes
> > [ 0.000000] log_buf_len: 524288 bytes
> > [ 0.000000] early log buf free: 254352(97%)
> > [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> > [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form
> > [ 0.000000] Memory: 24684464K/25165300K available (7210K kernel code, 1199K rwdata, 2812K rodata, 1500K init, 1452K bss, 480836K reserved, 0K cma-reserved)
> > [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
> > [ 0.000000] Hierarchical RCU implementation.
> > [ 0.000000] RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=8.
> > [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
> > [ 0.000000] NR_IRQS:8448 nr_irqs:488 16
> > [ 0.000000] Console: colour VGA+ 80x25
> > [ 0.000000] console [tty0] enabled
> > [ 0.000000] tsc: Fast TSC calibration failed
> > [ 0.000000] tsc: Unable to calibrate against PIT
> > [ 0.000000] tsc: using PMTIMER reference calibration
> > [ 0.000000] tsc: Detected 2297.102 MHz processor
> > [ 0.060019] Calibrating delay loop (skipped), value calculated using timer frequency.. 4594.20 BogoMIPS (lpj=22971020)
> > [ 0.060712] pid_max: default: 32768 minimum: 301
> > [ 0.061048] ACPI: Core revision 20150410
> > [ 0.063591] ACPI: All ACPI Tables successfully acquired
> > [ 0.066330] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
> > [ 0.097999] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
> > [ 0.110609] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
> > [ 0.130058] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
> > [ 0.140947] Initializing cgroup subsys blkio
> > [ 0.141262] Initializing cgroup subsys memory
> > [ 0.141575] Initializing cgroup subsys devices
> > [ 0.141855] Initializing cgroup subsys freezer
> > [ 0.142136] Initializing cgroup subsys net_cls
> > [ 0.142434] Initializing cgroup subsys perf_event
> > [ 0.142727] Initializing cgroup subsys net_prio
> > [ 0.143007] Initializing cgroup subsys hugetlb
> > [ 0.144131] mce: CPU supports 10 MCE banks
> > [ 0.144436] Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
> > [ 0.144752] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
> > [ 0.145340] Freeing SMP alternatives memory: 24K (ffffffffa6ea4000 - ffffffffa6eaa000)
> > [ 0.145825] ftrace: allocating 24377 entries in 96 pages
> > [ 0.171174] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
> > [ 0.272232] TSC deadline timer enabled
> > [ 0.272240] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz (fam: 06, model: 3f, stepping: 02)
> > [ 0.275668] Performance Events: 16-deep LBR, Haswell events, Intel PMU driver.
> > [ 0.310508] ... version: 2
> > [ 0.310789] ... bit width: 48
> > [ 0.311062] ... generic registers: 4
> > [ 0.311334] ... value mask: 0000ffffffffffff
> > [ 0.311638] ... max period: 000000007fffffff
> > [ 0.311930] ... fixed-purpose events: 3
> > [ 0.312203] ... event mask: 000000070000000f
> > [ 0.312939] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
> > [ 0.313572] x86: Booting SMP configuration:
> > [ 0.313857] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
> > [ 1.530158] x86: Booted up 1 node, 8 CPUs
> > [ 1.530768] smpboot: Total of 8 processors activated (36791.87 BogoMIPS)
> > [ 1.532131] devtmpfs: initialized
> > [ 1.534359] clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> > [ 1.534359] NET: Registered protocol family 16
> > [ 1.560075] cpuidle: using governor ladder
> > [ 1.590031] cpuidle: using governor menu
> > [ 1.590464] ACPI: bus type PCI registered
> > [ 1.590878] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
> > [ 1.591402] PCI: Using configuration type 1 for base access
> > [ 1.593581] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
> > [ 1.594921] perf_event_intel: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off
> > [ 1.620474] ACPI: Added _OSI(Module Device)
> > [ 1.620995] ACPI: Added _OSI(Processor Device)
> > [ 1.621971] ACPI: Added _OSI(3.0 _SCP Extensions)
> > [ 1.622488] ACPI: Added _OSI(Processor Aggregator Device)
> > [ 1.626193] ACPI: Interpreter enabled
> > [ 1.626720] ACPI: (supports S0 S5)
> > [ 1.627200] ACPI: Using IOAPIC for interrupt routing
> > [ 1.627792] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
> > [ 1.635465] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
> > [ 1.636091] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
> > [ 1.636716] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
> > [ 1.637456] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
> > [ 1.638691] acpiphp: Slot [3] registered
> > [ 1.639231] acpiphp: Slot [4] registered
> > [ 1.639785] acpiphp: Slot [5] registered
> > [ 1.640041] acpiphp: Slot [6] registered
> > [ 1.640584] acpiphp: Slot [7] registered
> > [ 1.641115] acpiphp: Slot [8] registered
> > [ 1.641668] acpiphp: Slot [9] registered
> > [ 1.642212] acpiphp: Slot [10] registered
> > [ 1.643592] acpiphp: Slot [11] registered
> > [ 1.643899] acpiphp: Slot [12] registered
> > [ 1.644205] acpiphp: Slot [13] registered
> > [ 1.644510] acpiphp: Slot [14] registered
> > [ 1.644812] acpiphp: Slot [15] registered
> > [ 1.645135] acpiphp: Slot [16] registered
> > [ 1.645448] acpiphp: Slot [17] registered
> > [ 1.645758] acpiphp: Slot [18] registered
> > [ 1.646054] acpiphp: Slot [19] registered
> > [ 1.646365] acpiphp: Slot [20] registered
> > [ 1.646663] acpiphp: Slot [21] registered
> > [ 1.646964] acpiphp: Slot [22] registered
> > [ 1.647265] acpiphp: Slot [23] registered
> > [ 1.647573] acpiphp: Slot [24] registered
> > [ 1.647871] acpiphp: Slot [25] registered
> > [ 1.650061] acpiphp: Slot [26] registered
> > [ 1.650585] acpiphp: Slot [27] registered
> > [ 1.650885] acpiphp: Slot [28] registered
> > [ 1.651188] acpiphp: Slot [29] registered
> > [ 1.654491] acpiphp: Slot [30] registered
> > [ 1.654794] acpiphp: Slot [31] registered
> > [ 1.655088] PCI host bridge to bus 0000:00
> > [ 1.655375] pci_bus 0000:00: root bus resource [bus 00-ff]
> > [ 1.655681] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
> > [ 1.656004] pci_bus 0000:00: root bus resource [io 0x0d00-0xadff window]
> > [ 1.656331] pci_bus 0000:00: root bus resource [io 0xae0f-0xaeff window]
> > [ 1.656654] pci_bus 0000:00: root bus resource [io 0xaf20-0xafdf window]
> > [ 1.656976] pci_bus 0000:00: root bus resource [io 0xafe4-0xffff window]
> > [ 1.657302] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
> > [ 1.657764] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
> > [ 1.658260] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
> > [ 1.658612] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
> > [ 1.659003] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
> > [ 1.663759] pci 0000:00:01.1: reg 0x20: [io 0xc0a0-0xc0af]
> > [ 1.665700] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
> > [ 1.666060] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
> > [ 1.666379] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
> > [ 1.666709] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
> > [ 1.667227] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
> > [ 1.671870] pci 0000:00:01.2: reg 0x20: [io 0xc040-0xc05f]
> > [ 1.674388] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
> > [ 1.674684] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
> > [ 1.675393] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
> > [ 1.675919] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
> > [ 1.681068] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
> > [ 1.683771] pci 0000:00:02.0: reg 0x14: [mem 0xfebd0000-0xfebd0fff]
> > [ 1.694209] pci 0000:00:02.0: reg 0x30: [mem 0xfebc0000-0xfebcffff pref]
> > [ 1.694644] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
> > [ 1.696503] pci 0000:00:03.0: reg 0x10: [io 0xc060-0xc07f]
> > [ 1.698431] pci 0000:00:03.0: reg 0x14: [mem 0xfebd1000-0xfebd1fff]
> > [ 1.707348] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
> > [ 1.707672] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
> > [ 1.709600] pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc03f]
> > [ 1.710937] pci 0000:00:04.0: reg 0x14: [mem 0xfebd2000-0xfebd2fff]
> > [ 1.720532] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
> > [ 1.721499] pci 0000:00:05.0: reg 0x10: [io 0xc080-0xc09f]
> > [ 1.728430] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
> > [ 1.729565] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
> > [ 1.730332] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
> > [ 1.731157] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
> > [ 1.731921] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
> > [ 1.733477] ACPI: Enabled 16 GPEs in block 00 to 0F
> > [ 1.734272] SCSI subsystem initialized
> > [ 1.734701] libata version 3.00 loaded.
> > [ 1.734701] ACPI: bus type USB registered
> > [ 1.734701] usbcore: registered new interface driver usbfs
> > [ 1.734701] usbcore: registered new interface driver hub
> > [ 1.734701] usbcore: registered new device driver usb
> > [ 1.734701] PCI: Using ACPI for IRQ routing
> > [ 1.734701] PCI: pci_cache_line_size set to 64 bytes
> > [ 1.734701] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
> > [ 1.734701] e820: reserve RAM buffer [mem 0xbffdf000-0xbfffffff]
> > [ 1.734701] Switched to clocksource refined-jiffies
> > [ 1.780170] pnp: PnP ACPI init
> > [ 1.780817] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
> > [ 1.780951] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
> > [ 1.781017] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
> > [ 1.781091] pnp 00:03: [dma 2]
> > [ 1.781112] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
> > [ 1.781248] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
> > [ 1.781553] pnp: PnP ACPI: found 5 devices
> > [ 1.787555] clocksource acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
> > [ 1.790067] Switched to clocksource acpi_pm
> > [ 1.790589] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
> > [ 1.790591] pci_bus 0000:00: resource 5 [io 0x0d00-0xadff window]
> > [ 1.790593] pci_bus 0000:00: resource 6 [io 0xae0f-0xaeff window]
> > [ 1.790594] pci_bus 0000:00: resource 7 [io 0xaf20-0xafdf window]
> > [ 1.790596] pci_bus 0000:00: resource 8 [io 0xafe4-0xffff window]
> > [ 1.790598] pci_bus 0000:00: resource 9 [mem 0x000a0000-0x000bffff window]
> > [ 1.790600] pci_bus 0000:00: resource 10 [mem 0xc0000000-0xfebfffff window]
> > [ 1.790708] NET: Registered protocol family 2
> > [ 1.791746] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
> > [ 1.794097] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> > [ 1.795244] TCP: Hash tables configured (established 262144 bind 65536)
> > [ 1.795836] UDP hash table entries: 16384 (order: 7, 524288 bytes)
> > [ 1.796670] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
> > [ 1.797515] NET: Registered protocol family 1
> > [ 1.797916] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
> > [ 1.798317] pci 0000:00:01.0: PIIX3: Enabling Passive Release
> > [ 1.798725] pci 0000:00:01.0: Activating ISA DMA hang workarounds
> > [ 1.832711] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
> > [ 1.866825] pci 0000:00:02.0: Video device with shadowed ROM
> > [ 1.866862] PCI: CLS 0 bytes, default 64
> > [ 1.866987] Unpacking initramfs...
> > [ 2.333153] Freeing initrd memory: 3008K (ffff880037a10000 - ffff880037d00000)
> > [ 2.333728] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> > [ 2.334102] software IO TLB [mem 0xbbfdf000-0xbffdf000] (64MB) mapped at [ffff8800bbfdf000-ffff8800bffdefff]
> > [ 2.484518] microcode: CPU0 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.484952] microcode: CPU1 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.485386] microcode: CPU2 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.485996] microcode: CPU3 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.486562] microcode: CPU4 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.487083] microcode: CPU5 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.487659] microcode: CPU6 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.488220] microcode: CPU7 sig=0x306f2, pf=0x1, revision=0x1
> > [ 2.488694] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> > [ 2.502294] AVX2 version of gcm_enc/dec engaged.
> > [ 2.502789] AES CTR mode by8 optimization enabled
> > [ 2.508451] sha1_ssse3: Using AVX2 optimized SHA-1 implementation
> > [ 2.509347] sha256_ssse3: Using AVX2 optimized SHA-256 implementation
> > [ 2.510489] sha512_ssse3: Using AVX2 optimized SHA-512 implementation
> > [ 2.528966] futex hash table entries: 2048 (order: 5, 131072 bytes)
> > [ 2.530162] Initialise system trusted keyring
> > [ 2.531219] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> > [ 2.534975] zpool: loaded
> > [ 2.535407] zbud: loaded
> > [ 2.536346] VFS: Disk quotas dquot_6.6.0
> > [ 2.536897] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> > [ 2.537901] SGI XFS with ACLs, security attributes, no debug enabled
> > [ 2.552199] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
> > [ 2.553012] io scheduler noop registered
> > [ 2.553337] io scheduler deadline registered (default)
> > [ 2.553696] io scheduler cfq registered
> > [ 2.554114] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> > [ 2.554489] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> > [ 2.554875] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
> > [ 2.555304] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> > [ 2.730888] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
> > [ 2.732614] intel_idle: does not run on family 6 model 63
> > [ 2.732683] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
> > [ 2.733232] ACPI: Power Button [PWRF]
> > [ 2.737344] GHES: HEST is not enabled!
> > [ 2.771139] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
> > [ 2.771922] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
> > [ 2.808645] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
> > [ 2.857664] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [ 2.858195] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
> > [ 2.860777] [drm] Initialized drm 1.1.0 20060810
> > [ 2.904350] vda: vda1 vda2
> > [ 2.905228] ata_piix 0000:00:01.1: version 2.13
> > [ 2.909583] scsi host0: ata_piix
> > [ 2.910753] scsi host1: ata_piix
> > [ 2.911151] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
> > [ 2.911640] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
> > [ 2.912127] uhci_hcd: USB Universal Host Controller Interface driver
> > [ 2.951548] uhci_hcd 0000:00:01.2: UHCI Host Controller
> > [ 2.952000] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
> > [ 2.952776] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
> > [ 2.953361] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
> > [ 2.953817] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> > [ 2.954683] usb usb1: Product: UHCI Host Controller
> > [ 2.955100] usb usb1: Manufacturer: Linux 4.1.8-gentoo uhci_hcd
> > [ 2.955544] usb usb1: SerialNumber: 0000:00:01.2
> > [ 2.956124] hub 1-0:1.0: USB hub found
> > [ 2.956530] hub 1-0:1.0: 2 ports detected
> > [ 2.957113] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
> > [ 2.959024] serio: i8042 KBD port at 0x60,0x64 irq 1
> > [ 2.959415] serio: i8042 AUX port at 0x60,0x64 irq 12
> > [ 2.959991] mousedev: PS/2 mouse device common for all mice
> > [ 2.960736] rtc_cmos 00:00: RTC can wake from S4
> > [ 2.961887] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
> > [ 2.962357] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
> > [ 2.962989] device-mapper: uevent: version 1.0.3
> > [ 2.963915] device-mapper: ioctl: 4.31.0-ioctl (2015-3-12) initialised: dm-devel@redhat.com
> > [ 2.964689] hidraw: raw HID events driver (C) Jiri Kosina
> > [ 2.965923] NET: Registered protocol family 10
> > [ 2.967249] NET: Registered protocol family 17
> > [ 2.968481] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
> > [ 2.970994] Loading compiled-in X.509 certificates
> > [ 2.971462] registered taskstats version 1
> > [ 2.972893] rtc_cmos 00:00: setting system clock to 2015-10-01 00:53:23 UTC (1443660803)
> > [ 3.070910] ata2.01: NODEV after polling detection
> > [ 3.071169] ata2.00: ATAPI: QEMU DVD-ROM, 2.1.2, max UDMA/100
> > [ 3.072156] ata2.00: configured for MWDMA2
> > [ 3.073901] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.1. PQ: 0 ANSI: 5
> > [ 3.075516] scsi 1:0:0:0: Attached scsi generic sg0 type 5
> > [ 3.079023] Freeing unused kernel memory: 1500K (ffffffffa6d2d000 - ffffffffa6ea4000)
> > [ 3.079913] Write protecting the kernel read-only data: 12288k
> > [ 3.082417] Freeing unused kernel memory: 968K (ffff88002670e000 - ffff880026800000)
> > [ 3.084810] Freeing unused kernel memory: 1284K (ffff880026abf000 - ffff880026c00000)
> > [ 3.270109] usb 1-1: new full-speed USB device number 2 using uhci_hcd
> > [ 3.443465] usb 1-1: New USB device found, idVendor=0627, idProduct=0001
> > [ 3.443469] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=5
> > [ 3.443471] usb 1-1: Product: QEMU USB Tablet
> > [ 3.443473] usb 1-1: Manufacturer: QEMU
> > [ 3.443474] usb 1-1: SerialNumber: 42
> > [ 3.480221] tsc: Refined TSC clocksource calibration: 2297.342 MHz
> > [ 3.480229] clocksource tsc: mask: 0xffffffffffffffff max_cycles: 0x211d6678e63, max_idle_ns: 440795313325 ns
> > [ 3.701149] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
> > [ 3.701154] cdrom: Uniform CD-ROM driver Revision: 3.20
> > [ 3.701432] sr 1:0:0:0: Attached scsi CD-ROM sr0
> > [ 3.914781] usbcore: registered new interface driver usb-storage
> > [ 3.938615] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input3
> > [ 3.938854] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Pointer [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0
> > [ 3.938898] usbcore: registered new interface driver usbhid
> > [ 3.938900] usbhid: USB HID core driver
> > [ 4.093620] FS-Cache: Loaded
> > [ 4.097657] RPC: Registered named UNIX socket transport module.
> > [ 4.097660] RPC: Registered udp transport module.
> > [ 4.097662] RPC: Registered tcp transport module.
> > [ 4.097663] RPC: Registered tcp NFSv4.1 backchannel transport module.
> > [ 4.102111] FS-Cache: Netfs 'nfs' registered for caching
> > [ 4.111955] fuse init (API version 7.23)
> > [ 4.123036] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
> > [ 4.123039] e1000: Copyright (c) 1999-2006 Intel Corporation.
> > [ 4.237118] device-mapper: dm-log-userspace: version 1.3.0 loaded
> > [ 4.480564] Switched to clocksource tsc
> > [ 6.990017] timekeeping watchdog: Marking clocksource 'tsc' as unstable, because the skew is too large:
> > [ 6.990017] 'acpi_pm' wd_now: 54f66c wd_last: 3041f5 mask: ffffff
> > [ 6.990017] 'tsc' cs_now: 12fee71862 cs_last: 12ba72729d mask: ffffffffffffffff
> > [ 7.152645] Switched to clocksource acpi_pm
> > [ 7.307231] random: lvm urandom read with 16 bits of entropy available
> > [ 17.899633] XFS (dm-7): Mounting V4 Filesystem
> > [ 18.094485] XFS (dm-7): Starting recovery (logdev: internal)
> > [ 18.166829] XFS (dm-7): Ending recovery (logdev: internal)
> > [ 18.587634] random: nonblocking pool is initialized
> > [ 22.540164] udevd[2534]: starting version 225
> > [ 23.744732] input: PC Speaker as /devices/platform/pcspkr/input/input4
> > [ 23.774381] virtio_net virtio0 internet0: renamed from eth0
> > [ 23.812931] udevd[2604]: Process 'net.sh eth0 start' failed with exit code 1.
> > [ 23.821106] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input5
> > [ 25.879944] Adding 4194300k swap on /dev/mapper/vps1storage-volSwap. Priority:-1 extents:1 across:4194300k FS
> > [ 25.976101] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
> > [ 25.991743] XFS (dm-3): Mounting V4 Filesystem
> > [ 26.674155] XFS (dm-3): Ending clean mount
> > [ 26.709912] XFS (dm-4): Mounting V4 Filesystem
> > [ 26.906203] XFS (dm-4): Ending clean mount
> > [ 26.959643] XFS (dm-2): Mounting V4 Filesystem
> > [ 27.149336] XFS (dm-2): Starting recovery (logdev: internal)
> > [ 27.223253] XFS (dm-2): Ending recovery (logdev: internal)
> > [ 27.262127] XFS (dm-8): Mounting V4 Filesystem
> > [ 27.350733] XFS (dm-8): Ending clean mount
> > [ 27.379363] XFS (dm-6): Mounting V4 Filesystem
> > [ 27.567352] XFS (dm-6): Starting recovery (logdev: internal)
> > [ 28.010426] XFS (dm-6): Ending recovery (logdev: internal)
> > [ 28.056060] XFS (dm-5): Mounting V4 Filesystem
> > [ 28.188959] XFS (dm-5): Starting recovery (logdev: internal)
> > [ 28.284561] XFS (dm-5): Ending recovery (logdev: internal)
> > [ 29.940191] Netfilter messages via NETLINK v0.30.
> > [ 30.003286] ip_set: protocol 6
> > [ 32.450468] ip_tables: (C) 2000-2006 Netfilter Core Team
> > [ 32.559733] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
> > [ 33.093113] ip6_tables: (C) 2000-2006 Netfilter Core Team
> > [ 57.075187] xt_time: kernel timezone is +0200
> > [ 57.201413] ctnetlink v0.93: registering with nfnetlink.
> > [ 57.780880] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
> > [ 60.506072] u32 classifier
> > [ 60.506077] Performance counters on
> > [ 60.506078] Actions configured
> > [ 118.338005] INFO: NMI handler (perf_event_nmi_handler) took too long to run: 2.220 msecs
> > [ 118.338016] perf interrupt took too long (19624 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
> > [ 180.970220] perf interrupt took too long (19751 > 5000), lowering kernel.perf_event_max_sample_rate to 25000
>
>
>
> -Thomas
^ permalink raw reply
* kernel BUG at drivers/block/virtio_blk.c:172!
From: Thomas D. @ 2015-10-01 1:10 UTC (permalink / raw)
To: virtualization, mst
Hi,
I have a virtual machine which fails to boot linux-4.1.8 while mounting
file systems:
> * Mounting local filesystem ...
> ------------[ cut here ]------------
> kernel BUG at drivers/block/virtio_blk.c:172!
> invalid opcode: 000 [#1] SMP
> Modules linked in: pcspkr psmouse dm_log_userspace virtio_net e1000 fuse nfs lockd grace sunrpc fscache dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log usbhid usb_storage sr_mod cdrom
> CPU: 7 PIDL 2254 Comm: dmcrypt_write Not tainted 4.1.8-gentoo #1
> Hardware name: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
> task: ffff88061fb70000 ti: ffff88061ff30000 task.ti: ffff88061ff30000
> RIP: 0010:[<ffffffffb4557b30>] [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
> RSP: 0018:ffff88061ff33ba8 EFLAGS: 00010202
> RAX: 00000000000000b1 RBX: ffff88061fb2fc00 RCX: ffff88061ff33c30
> RDX: 0000000000000008 RSI: ffff88061ff33c50 RDI: ffff88061fb2fc00
> RBP: ffff88061ff33bf8 R08: ffff88061eef3540 R09: ffff88061ff33c30
> R10: 0000000000000000 R11: 00000000000000af R12: 0000000000000000
> R13: ffff88061eef3540 R14: ffff88061eef3540 R15: ffff880622c7ca80
> FS: 0000000000000000(0000) GS:ffff88063fdc0000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000001ffe468 CR3: 00000000bb343000 CR4: 00000000001406e0
> Stack:
> ffff880622d4c478 0000000000000000 ffff88061ff33bd8 ffff88061fb2f
> 0000000000000001 ffff88061fb2fc00 ffff88061ff33c30 0000000000000
> ffff88061eef3540 0000000000000000 ffff88061ff33c98 ffffffffb43eb
>
> Call Trace:
> [<ffffffffb43eb500>] __blk_mq_run_hw_queue+0x1d0/0x370
> [<ffffffffb43eb315>] blk_mq_run_hw_queue+0x95/0xb0
> [<ffffffffb43ec804>] blk_mq_flush_plug_list+0x129/0x140
> [<ffffffffb43e33d8>] blk_finish_plug+0x18/0x50
> [<ffffffffb45e3bea>] dmcrypt_write+0x1da/0x1f0
> [<ffffffffb4108c90>] ? wake_up_state+0x20/0x20
> [<ffffffffb45e3a10>] ? crypt_iv_lmk_dtr+0x60/0x60
> [<ffffffffb40fb789>] kthread_create_on_node+0x180/0x180
> [<ffffffffb4705e92>] ret_from_fork+0x42/0x70
> [<ffffffffb40fb6c0>] ? kthread_create_on_node+0x180/0x180
> Code: 00 0000 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 00 00 41 89 85 7c 01 00 00 e9 93 fe ff ff 66 0f 1f 44 00 00 <0f> 0b 66 0f 1f 44 00 00 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b
> RIP [<ffffffffb4557b30>] virtio_queue_rq+0x210/0x2b0
> RSP: <ffff88061ff33ba8>
> ---[ end trace 8078357c459d5fc0 ]---
System details:
===============
VM (KVM), I don't know any details about the host system.
The disk (vda) is partitioned in vda1 (ext4, /boot) and vda2
(LUKS-encrypted).
On top of the LUKS volume I have LVM.
On top of LVM I have several partitions, all XFS (so rootfs is on XFS!).
1/20 reboots seems to success.
Today I tried to upgrade to linux-4.1.9. Mounting wasn't a problem
anymore but after ~5min the system always dies with
> NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s!
(CPU number varies)
dmesg:
> [ 0.000000] Initializing cgroup subsys cpuset
> [ 0.000000] Initializing cgroup subsys cpu
> [ 0.000000] Initializing cgroup subsys cpuacct
> [ 0.000000] Linux version 4.1.8-gentoo (root@sysresccd) (gcc version 4.9.3 (Gentoo 4.9.3 p1.2, pie-0.6.3) ) #1 SMP Thu Sep 24 03:44:37 CEST 2015
> [ 0.000000] Command line: BOOT_IMAGE=/kernel-genkernel-x86_64-4.1.8-gentoo root=/dev/mapper/vps1storage-volRoot ro dolvm crypt_root=UUID=fdea18e4-ba0f-1234-1234-12345678 root=UUID=86dad6e6-e43e-1234-1234-987654321 rootfs=xfs scandelay=3 nomodeset
> [ 0.000000] KERNEL supported cpus:
> [ 0.000000] Intel GenuineIntel
> [ 0.000000] e820: BIOS-provided physical RAM map:
> [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
> [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdefff] usable
> [ 0.000000] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000063fffffff] usable
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] SMBIOS 2.8 present.
> [ 0.000000] DMI: Red Hat KVM, BIOS seabios-1.7.5-8.el7 04/01/2014
> [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
> [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
> [ 0.000000] e820: last_pfn = 0x640000 max_arch_pfn = 0x400000000
> [ 0.000000] MTRR default type: write-back
> [ 0.000000] MTRR fixed ranges enabled:
> [ 0.000000] 00000-9FFFF write-back
> [ 0.000000] A0000-BFFFF uncachable
> [ 0.000000] C0000-FFFFF write-protect
> [ 0.000000] MTRR variable ranges enabled:
> [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable
> [ 0.000000] 1 disabled
> [ 0.000000] 2 disabled
> [ 0.000000] 3 disabled
> [ 0.000000] 4 disabled
> [ 0.000000] 5 disabled
> [ 0.000000] 6 disabled
> [ 0.000000] 7 disabled
> [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WC UC- UC
> [ 0.000000] e820: last_pfn = 0xbffdf max_arch_pfn = 0x400000000
> [ 0.000000] found SMP MP-table at [mem 0x000f1fb0-0x000f1fbf] mapped at [ffff8800000f1fb0]
> [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576
> [ 0.000000] Using GB pages for direct mapping
> [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
> [ 0.000000] [mem 0x00000000-0x000fffff] page 4k
> [ 0.000000] BRK [0x27016000, 0x27016fff] PGTABLE
> [ 0.000000] BRK [0x27017000, 0x27017fff] PGTABLE
> [ 0.000000] BRK [0x27018000, 0x27018fff] PGTABLE
> [ 0.000000] init_memory_mapping: [mem 0x63fe00000-0x63fffffff]
> [ 0.000000] [mem 0x63fe00000-0x63fffffff] page 1G
> [ 0.000000] init_memory_mapping: [mem 0x620000000-0x63fdfffff]
> [ 0.000000] [mem 0x620000000-0x63fdfffff] page 1G
> [ 0.000000] init_memory_mapping: [mem 0x00100000-0xbffdefff]
> [ 0.000000] [mem 0x00100000-0x001fffff] page 4k
> [ 0.000000] [mem 0x00200000-0x3fffffff] page 2M
> [ 0.000000] [mem 0x40000000-0x7fffffff] page 1G
> [ 0.000000] [mem 0x80000000-0xbfdfffff] page 2M
> [ 0.000000] [mem 0xbfe00000-0xbffdefff] page 4k
> [ 0.000000] init_memory_mapping: [mem 0x100000000-0x61fffffff]
> [ 0.000000] [mem 0x100000000-0x61fffffff] page 1G
> [ 0.000000] RAMDISK: [mem 0x37a10000-0x37cfffff]
> [ 0.000000] ACPI: Early table checksum verification disabled
> [ 0.000000] ACPI: RSDP 0x00000000000F1F60 000014 (v00 BOCHS )
> [ 0.000000] ACPI: RSDT 0x00000000BFFE163C 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: FACP 0x00000000BFFE0677 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
> [ 0.000000] ACPI: DSDT 0x00000000BFFDFB80 000AF7 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: FACS 0x00000000BFFDFB40 000040
> [ 0.000000] ACPI: SSDT 0x00000000BFFE06EB 000EA1 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: APIC 0x00000000BFFE158C 0000B0 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
> [ 0.000000] ACPI: Local APIC address 0xfee00000
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000063fffffff]
> [ 0.000000] NODE_DATA(0) allocated [mem 0x63ffd8000-0x63fffdfff]
> [ 0.000000] [ffffea0000000000-ffffea0018ffffff] PMD -> [ffff880627600000-ffff88063f5fffff] on node 0
> [ 0.000000] Zone ranges:
> [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
> [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
> [ 0.000000] Normal [mem 0x0000000100000000-0x000000063fffffff]
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
> [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffdefff]
> [ 0.000000] node 0: [mem 0x0000000100000000-0x000000063fffffff]
> [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000063fffffff]
> [ 0.000000] On node 0 totalpages: 6291325
> [ 0.000000] DMA zone: 64 pages used for memmap
> [ 0.000000] DMA zone: 21 pages reserved
> [ 0.000000] DMA zone: 3998 pages, LIFO batch:0
> [ 0.000000] DMA32 zone: 12224 pages used for memmap
> [ 0.000000] DMA32 zone: 782303 pages, LIFO batch:31
> [ 0.000000] Normal zone: 86016 pages used for memmap
> [ 0.000000] Normal zone: 5505024 pages, LIFO batch:31
> [ 0.000000] ACPI: PM-Timer IO Port: 0x608
> [ 0.000000] ACPI: Local APIC address 0xfee00000
> [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
> [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
> [ 0.000000] ACPI: IRQ0 used by override.
> [ 0.000000] ACPI: IRQ5 used by override.
> [ 0.000000] ACPI: IRQ9 used by override.
> [ 0.000000] ACPI: IRQ10 used by override.
> [ 0.000000] ACPI: IRQ11 used by override.
> [ 0.000000] Using ACPI (MADT) for SMP configuration information
> [ 0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
> [ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
> [ 0.000000] clocksource refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [ 0.000000] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:8 nr_node_ids:1
> [ 0.000000] PERCPU: Embedded 33 pages/cpu @ffff88063fc00000 s95192 r8192 d31784 u262144
> [ 0.000000] pcpu-alloc: s95192 r8192 d31784 u262144 alloc=1*2097152
> [ 0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
> [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 6193000
> [ 0.000000] Policy zone: Normal
> [ 0.000000] Kernel command line: BOOT_IMAGE=/kernel-genkernel-x86_64-4.1.8-gentoo root=/dev/mapper/vps1storage-volRoot ro dolvm crypt_root=UUID=fdea18e4-ba0f-4c82-8ebb-3dd160859ee8 root=UUID=86dad6e6-e43e-4b4e-a445-c2fe3d9aee13 rootfs=xfs scandelay=3 nomodeset
> [ 0.000000] log_buf_len individual max cpu contribution: 32768 bytes
> [ 0.000000] log_buf_len total cpu_extra contributions: 229376 bytes
> [ 0.000000] log_buf_len min size: 262144 bytes
> [ 0.000000] log_buf_len: 524288 bytes
> [ 0.000000] early log buf free: 254352(97%)
> [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form
> [ 0.000000] Memory: 24684464K/25165300K available (7210K kernel code, 1199K rwdata, 2812K rodata, 1500K init, 1452K bss, 480836K reserved, 0K cma-reserved)
> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=8.
> [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
> [ 0.000000] NR_IRQS:8448 nr_irqs:488 16
> [ 0.000000] Console: colour VGA+ 80x25
> [ 0.000000] console [tty0] enabled
> [ 0.000000] tsc: Fast TSC calibration failed
> [ 0.000000] tsc: Unable to calibrate against PIT
> [ 0.000000] tsc: using PMTIMER reference calibration
> [ 0.000000] tsc: Detected 2297.102 MHz processor
> [ 0.060019] Calibrating delay loop (skipped), value calculated using timer frequency.. 4594.20 BogoMIPS (lpj=22971020)
> [ 0.060712] pid_max: default: 32768 minimum: 301
> [ 0.061048] ACPI: Core revision 20150410
> [ 0.063591] ACPI: All ACPI Tables successfully acquired
> [ 0.066330] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
> [ 0.097999] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
> [ 0.110609] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
> [ 0.130058] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
> [ 0.140947] Initializing cgroup subsys blkio
> [ 0.141262] Initializing cgroup subsys memory
> [ 0.141575] Initializing cgroup subsys devices
> [ 0.141855] Initializing cgroup subsys freezer
> [ 0.142136] Initializing cgroup subsys net_cls
> [ 0.142434] Initializing cgroup subsys perf_event
> [ 0.142727] Initializing cgroup subsys net_prio
> [ 0.143007] Initializing cgroup subsys hugetlb
> [ 0.144131] mce: CPU supports 10 MCE banks
> [ 0.144436] Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
> [ 0.144752] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
> [ 0.145340] Freeing SMP alternatives memory: 24K (ffffffffa6ea4000 - ffffffffa6eaa000)
> [ 0.145825] ftrace: allocating 24377 entries in 96 pages
> [ 0.171174] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
> [ 0.272232] TSC deadline timer enabled
> [ 0.272240] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz (fam: 06, model: 3f, stepping: 02)
> [ 0.275668] Performance Events: 16-deep LBR, Haswell events, Intel PMU driver.
> [ 0.310508] ... version: 2
> [ 0.310789] ... bit width: 48
> [ 0.311062] ... generic registers: 4
> [ 0.311334] ... value mask: 0000ffffffffffff
> [ 0.311638] ... max period: 000000007fffffff
> [ 0.311930] ... fixed-purpose events: 3
> [ 0.312203] ... event mask: 000000070000000f
> [ 0.312939] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
> [ 0.313572] x86: Booting SMP configuration:
> [ 0.313857] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
> [ 1.530158] x86: Booted up 1 node, 8 CPUs
> [ 1.530768] smpboot: Total of 8 processors activated (36791.87 BogoMIPS)
> [ 1.532131] devtmpfs: initialized
> [ 1.534359] clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [ 1.534359] NET: Registered protocol family 16
> [ 1.560075] cpuidle: using governor ladder
> [ 1.590031] cpuidle: using governor menu
> [ 1.590464] ACPI: bus type PCI registered
> [ 1.590878] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
> [ 1.591402] PCI: Using configuration type 1 for base access
> [ 1.593581] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
> [ 1.594921] perf_event_intel: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off
> [ 1.620474] ACPI: Added _OSI(Module Device)
> [ 1.620995] ACPI: Added _OSI(Processor Device)
> [ 1.621971] ACPI: Added _OSI(3.0 _SCP Extensions)
> [ 1.622488] ACPI: Added _OSI(Processor Aggregator Device)
> [ 1.626193] ACPI: Interpreter enabled
> [ 1.626720] ACPI: (supports S0 S5)
> [ 1.627200] ACPI: Using IOAPIC for interrupt routing
> [ 1.627792] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
> [ 1.635465] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
> [ 1.636091] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
> [ 1.636716] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
> [ 1.637456] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
> [ 1.638691] acpiphp: Slot [3] registered
> [ 1.639231] acpiphp: Slot [4] registered
> [ 1.639785] acpiphp: Slot [5] registered
> [ 1.640041] acpiphp: Slot [6] registered
> [ 1.640584] acpiphp: Slot [7] registered
> [ 1.641115] acpiphp: Slot [8] registered
> [ 1.641668] acpiphp: Slot [9] registered
> [ 1.642212] acpiphp: Slot [10] registered
> [ 1.643592] acpiphp: Slot [11] registered
> [ 1.643899] acpiphp: Slot [12] registered
> [ 1.644205] acpiphp: Slot [13] registered
> [ 1.644510] acpiphp: Slot [14] registered
> [ 1.644812] acpiphp: Slot [15] registered
> [ 1.645135] acpiphp: Slot [16] registered
> [ 1.645448] acpiphp: Slot [17] registered
> [ 1.645758] acpiphp: Slot [18] registered
> [ 1.646054] acpiphp: Slot [19] registered
> [ 1.646365] acpiphp: Slot [20] registered
> [ 1.646663] acpiphp: Slot [21] registered
> [ 1.646964] acpiphp: Slot [22] registered
> [ 1.647265] acpiphp: Slot [23] registered
> [ 1.647573] acpiphp: Slot [24] registered
> [ 1.647871] acpiphp: Slot [25] registered
> [ 1.650061] acpiphp: Slot [26] registered
> [ 1.650585] acpiphp: Slot [27] registered
> [ 1.650885] acpiphp: Slot [28] registered
> [ 1.651188] acpiphp: Slot [29] registered
> [ 1.654491] acpiphp: Slot [30] registered
> [ 1.654794] acpiphp: Slot [31] registered
> [ 1.655088] PCI host bridge to bus 0000:00
> [ 1.655375] pci_bus 0000:00: root bus resource [bus 00-ff]
> [ 1.655681] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
> [ 1.656004] pci_bus 0000:00: root bus resource [io 0x0d00-0xadff window]
> [ 1.656331] pci_bus 0000:00: root bus resource [io 0xae0f-0xaeff window]
> [ 1.656654] pci_bus 0000:00: root bus resource [io 0xaf20-0xafdf window]
> [ 1.656976] pci_bus 0000:00: root bus resource [io 0xafe4-0xffff window]
> [ 1.657302] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
> [ 1.657764] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
> [ 1.658260] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
> [ 1.658612] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
> [ 1.659003] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
> [ 1.663759] pci 0000:00:01.1: reg 0x20: [io 0xc0a0-0xc0af]
> [ 1.665700] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
> [ 1.666060] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
> [ 1.666379] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
> [ 1.666709] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
> [ 1.667227] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
> [ 1.671870] pci 0000:00:01.2: reg 0x20: [io 0xc040-0xc05f]
> [ 1.674388] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
> [ 1.674684] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
> [ 1.675393] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
> [ 1.675919] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
> [ 1.681068] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
> [ 1.683771] pci 0000:00:02.0: reg 0x14: [mem 0xfebd0000-0xfebd0fff]
> [ 1.694209] pci 0000:00:02.0: reg 0x30: [mem 0xfebc0000-0xfebcffff pref]
> [ 1.694644] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
> [ 1.696503] pci 0000:00:03.0: reg 0x10: [io 0xc060-0xc07f]
> [ 1.698431] pci 0000:00:03.0: reg 0x14: [mem 0xfebd1000-0xfebd1fff]
> [ 1.707348] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
> [ 1.707672] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
> [ 1.709600] pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc03f]
> [ 1.710937] pci 0000:00:04.0: reg 0x14: [mem 0xfebd2000-0xfebd2fff]
> [ 1.720532] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
> [ 1.721499] pci 0000:00:05.0: reg 0x10: [io 0xc080-0xc09f]
> [ 1.728430] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
> [ 1.729565] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
> [ 1.730332] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
> [ 1.731157] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
> [ 1.731921] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
> [ 1.733477] ACPI: Enabled 16 GPEs in block 00 to 0F
> [ 1.734272] SCSI subsystem initialized
> [ 1.734701] libata version 3.00 loaded.
> [ 1.734701] ACPI: bus type USB registered
> [ 1.734701] usbcore: registered new interface driver usbfs
> [ 1.734701] usbcore: registered new interface driver hub
> [ 1.734701] usbcore: registered new device driver usb
> [ 1.734701] PCI: Using ACPI for IRQ routing
> [ 1.734701] PCI: pci_cache_line_size set to 64 bytes
> [ 1.734701] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
> [ 1.734701] e820: reserve RAM buffer [mem 0xbffdf000-0xbfffffff]
> [ 1.734701] Switched to clocksource refined-jiffies
> [ 1.780170] pnp: PnP ACPI init
> [ 1.780817] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
> [ 1.780951] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
> [ 1.781017] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
> [ 1.781091] pnp 00:03: [dma 2]
> [ 1.781112] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
> [ 1.781248] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
> [ 1.781553] pnp: PnP ACPI: found 5 devices
> [ 1.787555] clocksource acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
> [ 1.790067] Switched to clocksource acpi_pm
> [ 1.790589] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
> [ 1.790591] pci_bus 0000:00: resource 5 [io 0x0d00-0xadff window]
> [ 1.790593] pci_bus 0000:00: resource 6 [io 0xae0f-0xaeff window]
> [ 1.790594] pci_bus 0000:00: resource 7 [io 0xaf20-0xafdf window]
> [ 1.790596] pci_bus 0000:00: resource 8 [io 0xafe4-0xffff window]
> [ 1.790598] pci_bus 0000:00: resource 9 [mem 0x000a0000-0x000bffff window]
> [ 1.790600] pci_bus 0000:00: resource 10 [mem 0xc0000000-0xfebfffff window]
> [ 1.790708] NET: Registered protocol family 2
> [ 1.791746] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
> [ 1.794097] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [ 1.795244] TCP: Hash tables configured (established 262144 bind 65536)
> [ 1.795836] UDP hash table entries: 16384 (order: 7, 524288 bytes)
> [ 1.796670] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
> [ 1.797515] NET: Registered protocol family 1
> [ 1.797916] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
> [ 1.798317] pci 0000:00:01.0: PIIX3: Enabling Passive Release
> [ 1.798725] pci 0000:00:01.0: Activating ISA DMA hang workarounds
> [ 1.832711] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
> [ 1.866825] pci 0000:00:02.0: Video device with shadowed ROM
> [ 1.866862] PCI: CLS 0 bytes, default 64
> [ 1.866987] Unpacking initramfs...
> [ 2.333153] Freeing initrd memory: 3008K (ffff880037a10000 - ffff880037d00000)
> [ 2.333728] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> [ 2.334102] software IO TLB [mem 0xbbfdf000-0xbffdf000] (64MB) mapped at [ffff8800bbfdf000-ffff8800bffdefff]
> [ 2.484518] microcode: CPU0 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.484952] microcode: CPU1 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.485386] microcode: CPU2 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.485996] microcode: CPU3 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.486562] microcode: CPU4 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.487083] microcode: CPU5 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.487659] microcode: CPU6 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.488220] microcode: CPU7 sig=0x306f2, pf=0x1, revision=0x1
> [ 2.488694] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [ 2.502294] AVX2 version of gcm_enc/dec engaged.
> [ 2.502789] AES CTR mode by8 optimization enabled
> [ 2.508451] sha1_ssse3: Using AVX2 optimized SHA-1 implementation
> [ 2.509347] sha256_ssse3: Using AVX2 optimized SHA-256 implementation
> [ 2.510489] sha512_ssse3: Using AVX2 optimized SHA-512 implementation
> [ 2.528966] futex hash table entries: 2048 (order: 5, 131072 bytes)
> [ 2.530162] Initialise system trusted keyring
> [ 2.531219] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 2.534975] zpool: loaded
> [ 2.535407] zbud: loaded
> [ 2.536346] VFS: Disk quotas dquot_6.6.0
> [ 2.536897] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [ 2.537901] SGI XFS with ACLs, security attributes, no debug enabled
> [ 2.552199] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
> [ 2.553012] io scheduler noop registered
> [ 2.553337] io scheduler deadline registered (default)
> [ 2.553696] io scheduler cfq registered
> [ 2.554114] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [ 2.554489] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> [ 2.554875] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
> [ 2.555304] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [ 2.730888] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
> [ 2.732614] intel_idle: does not run on family 6 model 63
> [ 2.732683] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
> [ 2.733232] ACPI: Power Button [PWRF]
> [ 2.737344] GHES: HEST is not enabled!
> [ 2.771139] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
> [ 2.771922] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
> [ 2.808645] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
> [ 2.857664] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> [ 2.858195] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
> [ 2.860777] [drm] Initialized drm 1.1.0 20060810
> [ 2.904350] vda: vda1 vda2
> [ 2.905228] ata_piix 0000:00:01.1: version 2.13
> [ 2.909583] scsi host0: ata_piix
> [ 2.910753] scsi host1: ata_piix
> [ 2.911151] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
> [ 2.911640] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
> [ 2.912127] uhci_hcd: USB Universal Host Controller Interface driver
> [ 2.951548] uhci_hcd 0000:00:01.2: UHCI Host Controller
> [ 2.952000] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
> [ 2.952776] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
> [ 2.953361] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
> [ 2.953817] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [ 2.954683] usb usb1: Product: UHCI Host Controller
> [ 2.955100] usb usb1: Manufacturer: Linux 4.1.8-gentoo uhci_hcd
> [ 2.955544] usb usb1: SerialNumber: 0000:00:01.2
> [ 2.956124] hub 1-0:1.0: USB hub found
> [ 2.956530] hub 1-0:1.0: 2 ports detected
> [ 2.957113] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
> [ 2.959024] serio: i8042 KBD port at 0x60,0x64 irq 1
> [ 2.959415] serio: i8042 AUX port at 0x60,0x64 irq 12
> [ 2.959991] mousedev: PS/2 mouse device common for all mice
> [ 2.960736] rtc_cmos 00:00: RTC can wake from S4
> [ 2.961887] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
> [ 2.962357] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
> [ 2.962989] device-mapper: uevent: version 1.0.3
> [ 2.963915] device-mapper: ioctl: 4.31.0-ioctl (2015-3-12) initialised: dm-devel@redhat.com
> [ 2.964689] hidraw: raw HID events driver (C) Jiri Kosina
> [ 2.965923] NET: Registered protocol family 10
> [ 2.967249] NET: Registered protocol family 17
> [ 2.968481] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
> [ 2.970994] Loading compiled-in X.509 certificates
> [ 2.971462] registered taskstats version 1
> [ 2.972893] rtc_cmos 00:00: setting system clock to 2015-10-01 00:53:23 UTC (1443660803)
> [ 3.070910] ata2.01: NODEV after polling detection
> [ 3.071169] ata2.00: ATAPI: QEMU DVD-ROM, 2.1.2, max UDMA/100
> [ 3.072156] ata2.00: configured for MWDMA2
> [ 3.073901] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.1. PQ: 0 ANSI: 5
> [ 3.075516] scsi 1:0:0:0: Attached scsi generic sg0 type 5
> [ 3.079023] Freeing unused kernel memory: 1500K (ffffffffa6d2d000 - ffffffffa6ea4000)
> [ 3.079913] Write protecting the kernel read-only data: 12288k
> [ 3.082417] Freeing unused kernel memory: 968K (ffff88002670e000 - ffff880026800000)
> [ 3.084810] Freeing unused kernel memory: 1284K (ffff880026abf000 - ffff880026c00000)
> [ 3.270109] usb 1-1: new full-speed USB device number 2 using uhci_hcd
> [ 3.443465] usb 1-1: New USB device found, idVendor=0627, idProduct=0001
> [ 3.443469] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=5
> [ 3.443471] usb 1-1: Product: QEMU USB Tablet
> [ 3.443473] usb 1-1: Manufacturer: QEMU
> [ 3.443474] usb 1-1: SerialNumber: 42
> [ 3.480221] tsc: Refined TSC clocksource calibration: 2297.342 MHz
> [ 3.480229] clocksource tsc: mask: 0xffffffffffffffff max_cycles: 0x211d6678e63, max_idle_ns: 440795313325 ns
> [ 3.701149] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
> [ 3.701154] cdrom: Uniform CD-ROM driver Revision: 3.20
> [ 3.701432] sr 1:0:0:0: Attached scsi CD-ROM sr0
> [ 3.914781] usbcore: registered new interface driver usb-storage
> [ 3.938615] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input3
> [ 3.938854] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Pointer [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0
> [ 3.938898] usbcore: registered new interface driver usbhid
> [ 3.938900] usbhid: USB HID core driver
> [ 4.093620] FS-Cache: Loaded
> [ 4.097657] RPC: Registered named UNIX socket transport module.
> [ 4.097660] RPC: Registered udp transport module.
> [ 4.097662] RPC: Registered tcp transport module.
> [ 4.097663] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [ 4.102111] FS-Cache: Netfs 'nfs' registered for caching
> [ 4.111955] fuse init (API version 7.23)
> [ 4.123036] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
> [ 4.123039] e1000: Copyright (c) 1999-2006 Intel Corporation.
> [ 4.237118] device-mapper: dm-log-userspace: version 1.3.0 loaded
> [ 4.480564] Switched to clocksource tsc
> [ 6.990017] timekeeping watchdog: Marking clocksource 'tsc' as unstable, because the skew is too large:
> [ 6.990017] 'acpi_pm' wd_now: 54f66c wd_last: 3041f5 mask: ffffff
> [ 6.990017] 'tsc' cs_now: 12fee71862 cs_last: 12ba72729d mask: ffffffffffffffff
> [ 7.152645] Switched to clocksource acpi_pm
> [ 7.307231] random: lvm urandom read with 16 bits of entropy available
> [ 17.899633] XFS (dm-7): Mounting V4 Filesystem
> [ 18.094485] XFS (dm-7): Starting recovery (logdev: internal)
> [ 18.166829] XFS (dm-7): Ending recovery (logdev: internal)
> [ 18.587634] random: nonblocking pool is initialized
> [ 22.540164] udevd[2534]: starting version 225
> [ 23.744732] input: PC Speaker as /devices/platform/pcspkr/input/input4
> [ 23.774381] virtio_net virtio0 internet0: renamed from eth0
> [ 23.812931] udevd[2604]: Process 'net.sh eth0 start' failed with exit code 1.
> [ 23.821106] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input5
> [ 25.879944] Adding 4194300k swap on /dev/mapper/vps1storage-volSwap. Priority:-1 extents:1 across:4194300k FS
> [ 25.976101] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
> [ 25.991743] XFS (dm-3): Mounting V4 Filesystem
> [ 26.674155] XFS (dm-3): Ending clean mount
> [ 26.709912] XFS (dm-4): Mounting V4 Filesystem
> [ 26.906203] XFS (dm-4): Ending clean mount
> [ 26.959643] XFS (dm-2): Mounting V4 Filesystem
> [ 27.149336] XFS (dm-2): Starting recovery (logdev: internal)
> [ 27.223253] XFS (dm-2): Ending recovery (logdev: internal)
> [ 27.262127] XFS (dm-8): Mounting V4 Filesystem
> [ 27.350733] XFS (dm-8): Ending clean mount
> [ 27.379363] XFS (dm-6): Mounting V4 Filesystem
> [ 27.567352] XFS (dm-6): Starting recovery (logdev: internal)
> [ 28.010426] XFS (dm-6): Ending recovery (logdev: internal)
> [ 28.056060] XFS (dm-5): Mounting V4 Filesystem
> [ 28.188959] XFS (dm-5): Starting recovery (logdev: internal)
> [ 28.284561] XFS (dm-5): Ending recovery (logdev: internal)
> [ 29.940191] Netfilter messages via NETLINK v0.30.
> [ 30.003286] ip_set: protocol 6
> [ 32.450468] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 32.559733] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
> [ 33.093113] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [ 57.075187] xt_time: kernel timezone is +0200
> [ 57.201413] ctnetlink v0.93: registering with nfnetlink.
> [ 57.780880] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
> [ 60.506072] u32 classifier
> [ 60.506077] Performance counters on
> [ 60.506078] Actions configured
> [ 118.338005] INFO: NMI handler (perf_event_nmi_handler) took too long to run: 2.220 msecs
> [ 118.338016] perf interrupt took too long (19624 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
> [ 180.970220] perf interrupt took too long (19751 > 5000), lowering kernel.perf_event_max_sample_rate to 25000
-Thomas
^ permalink raw reply
* Call for Papers - WorldCIST'15 - Proceedings by Springer
From: Maria Lemos @ 2015-09-29 19:02 UTC (permalink / raw)
To: virtualization
[-- Attachment #1: Type: text/plain, Size: 7211 bytes --]
--
-- We apologise for any cross-posting.
--
---- Please forward for your contacts. Thank you so much!
--
---------
WorldCIST'16 - 4th World Conference on Information Systems and Technologies
Recife, PE, Brazil
22th-24th of March 2016
http://www.aisti.eu/worldcist16/
-------------------------------------------
SCOPE
The WorldCist'16 - 4th World Conference on Information Systems and Technologies, to be held at Recife, PE, Brazil, 22 - 24 March 2016, is a global forum for researchers and practitioners to present and discuss the most recent innovations, trends, results, experiences and concerns in the several perspectives of Information Systems and Technologies.
We are pleased to invite you to submit your papers to WorldCist'16. All submissions will be reviewed on the basis of relevance, originality, importance and clarity.
THEMES
Submitted papers should be related with one or more of the main themes proposed for the Conference:
A) Information and Knowledge Management (IKM);
B) Organizational Models and Information Systems (OMIS);
C) Software and Systems Modeling (SSM);
D) Software Systems, Architectures, Applications and Tools (SSAAT);
E) Multimedia Systems and Applications (MSA);
F) Computer Networks, Mobility and Pervasive Systems (CNMPS);
G) Intelligent and Decision Support Systems (IDSS);
H) Big Data Analytics and Applications (BDAA);
I) Human-Computer Interaction (HCI);
J) Health Informatics (HIS);
K) Information Technologies in Education (ITE);
L) Information Technologies in Radiocommunications (ITR).
TYPES OF SUBMISSIONS AND DECISIONS
Four types of papers can be submitted:
- Full paper: Finished or consolidated R&D works, to be included in one of the Conference themes. These papers are assigned a 10-page limit.
- Short paper: Ongoing works with relevant preliminary results, open to discussion. These papers are assigned a 7-page limit.
-Poster paper: Initial work with relevant ideas, open to discussion. These papers are assigned to a 4-page limit.
- Company paper: Companies' papers that show practical experience, R & D, tools, etc., focused on some topics of the conference. These papers are assigned to a 4-page limit.
Submitted papers must comply with the format of Advances in Intelligent Systems and Computing Series (see Instructions for Authors at Springer Website or download a DOC example) be written in English, must not have been published before, not be under review for any other conference or publication and not include any information leading to the authors’ identification. Therefore, the authors’ names, affiliations and bibliographic references should not be included in the version for evaluation by the Program Committee. This information should only be included in the camera-ready version, saved in Word or Latex format and also in PDF format. These files must be accompanied by the Consent to Publication form filled out, in a ZIP file, and uploaded at the conference management system.
All papers will be subjected to a “double-blind review” by at least two members of the Program Committee.
Based on Program Committee evaluation, a paper can be rejected or accepted by the Conference Chairs. In the later case, it can be accepted as the type originally submitted or as another type. Thus, full papers can be accepted as short papers or poster papers only. Similarly, short papers can be accepted as poster papers only. In these cases, the authors will be allowed to maintain the original number of pages in the camera-ready version.
The authors of accepted poster papers must also build and print a poster to be exhibited during the Conference. This poster must follow an A1 or A2 vertical format. The Conference can includes Work Sessions where these posters are presented and orally discussed, with a 5 minute limit per poster.
The authors of accepted full papers will have 15 minutes to present their work in a Conference Work Session; approximately 5 minutes of discussion will follow each presentation. The authors of accepted short papers and company papers will have 11 minutes to present their work in a Conference Work Session; approximately 4 minutes of discussion will follow each presentation.
PUBLICATION AND INDEXING
To ensure that a full paper, short paper, poster paper or company paper is published in the Proceedings, at least one of the authors must be fully registered by the 27th of December 2015, and the paper must comply with the suggested layout and page-limit. Additionally, all recommended changes must be addressed by the authors before they submit the camera-ready version.
No more than one paper per registration will be published in the Conference Proceedings. An extra fee must be paid for publication of additional papers, with a maximum of one additional paper per registration.
Full and short papers will be published in Proceedings by Springer, in a book of Advances in Intelligent Systems and Computing series. Poster and company papers will be published by AISTI.
Published full and short papers will be submitted for indexation by ISI, EI-Compendex, SCOPUS and DBLP, among others, and will be available in the SpringerLink Digital Library.
The authors of the best selected papers will be invited to extend them for publication in international journals indexed by ISI/SCI, SCOPUS and DBLP, among others, such as:
- International Journal of Neural Systems (IF: 6.507)
- Integrated Computer-Aided Engineering (IF: 4.698)
- Computers in Human Behavior (IF: 2.694)
- Journal of Medical Systems (IF: 2.213)
- International Journal of Computer-Supported Collaborative Learning (IF: 1.841)
- Journal of Intelligent & Fuzzy Systems (IF: 1.812)
- Telemedicine and e-Health (IF: 1.668)
- International Journal of Information Management (IF: 1.550)
- Engineering Computations (IF: 1.495)
- Electronic Commerce Research and Applications (IF: 1.482)
- Telematics and Informatics (IF: 1.120)
- Journal of Evaluation in Clinical Practice (IF: 1.084)
- Ethics and Information Technology (IF: 1.021)
- Int. Journal of Computers Communications & Control (IF: 0.746)
- IET Software (IF: 0.595)
- Knowledge Management Research & Practice (IF: 0.554)
- AI Communications (IF: 0.547)
- Computing and Informatics (IF: 0.504)
- Universal Access in the Information Society (IF: 0.475)
- Journal of Global Information Management (IF: 0.424)
- Journal of Internet Services and Applications (SJR: 0.88)
- Journal of Hospitality and Tourism Technology (SJR: 0.41)
- VINE - The Journal of Information and Knowledge Management Systems (SJR: 0.24)
- International Journal of Online Engineering (SJR: 0.21)
- Int. Journal of Emerging Technologies in Learning (SJR: 0.12)
- Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization
IMPORTANT DATES
Paper Submission: October 31, 2015
Notification of Acceptance: December 13, 2015
Payment of Registration, to ensure the inclusion of an accepted paper in the conference proceedings: December 27, 2015.
Camera-ready Submission: December 31, 2015
-
WorldCIST'16
http://www.aisti.eu/worldcist16/
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [RFC PATCH 0/2] virtio nvme
From: Hannes Reinecke @ 2015-09-28 5:58 UTC (permalink / raw)
To: Nicholas A. Bellinger, Ming Lin
Cc: Minturn, Dave B, Michael S. Tsirkin, linux-nvme,
Linux Virtualization, target-devel, Christoph Hellwig
In-Reply-To: <1443330116.15432.57.camel@haakon3.risingtidesystems.com>
On 09/27/2015 07:01 AM, Nicholas A. Bellinger wrote:
> On Wed, 2015-09-23 at 15:58 -0700, Ming Lin wrote:
>> On Fri, 2015-09-18 at 14:09 -0700, Nicholas A. Bellinger wrote:
>>> On Fri, 2015-09-18 at 11:12 -0700, Ming Lin wrote:
>>>> On Thu, 2015-09-17 at 17:55 -0700, Nicholas A. Bellinger wrote:
>
> <SNIP>
>
<Even more SNIP>
>
> Btw, after chatting with Dr. Hannes this week at SDC here are his
> original rts-megasas -v6 patches from Feb 2013.
>
> Note they are standalone patches that require a sufficiently old enough
> LIO + QEMU to actually build + function.
>
> https://github.com/Datera/rts-megasas/blob/master/rts_megasas-qemu-v6.patch
> https://github.com/Datera/rts-megasas/blob/master/rts_megasas-fabric-v6.patch
>
> For groking purposes, they demonstrate the principle design for a host
> kernel level driver, along with the megasas firmware interface (MFI)
> specific emulation magic that makes up the bulk of the code.
>
And indeed, Nic persuaded me to have them updated to qemu latest.
Which I'll be doing shortly. Stay tuned.
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [RFC PATCH 0/2] virtio nvme
From: Ming Lin @ 2015-09-27 6:49 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Minturn, Dave B, Michael S. Tsirkin, linux-nvme,
Linux Virtualization, target-devel, Christoph Hellwig
In-Reply-To: <1443330116.15432.57.camel@haakon3.risingtidesystems.com>
On Sat, Sep 26, 2015 at 10:01 PM, Nicholas A. Bellinger
<nab@linux-iscsi.org> wrote:
>
> Btw, after chatting with Dr. Hannes this week at SDC here are his
> original rts-megasas -v6 patches from Feb 2013.
>
> Note they are standalone patches that require a sufficiently old enough
> LIO + QEMU to actually build + function.
>
> https://github.com/Datera/rts-megasas/blob/master/rts_megasas-qemu-v6.patch
> https://github.com/Datera/rts-megasas/blob/master/rts_megasas-fabric-v6.patch
>
> For groking purposes, they demonstrate the principle design for a host
> kernel level driver, along with the megasas firmware interface (MFI)
> specific emulation magic that makes up the bulk of the code.
>
> Take a look.
Big thanks. Reading the patches now.
>
> --nab
>
^ permalink raw reply
* Re: [RFC PATCH 0/2] virtio nvme
From: Nicholas A. Bellinger @ 2015-09-27 5:01 UTC (permalink / raw)
To: Ming Lin
Cc: Minturn, Dave B, Michael S. Tsirkin, linux-nvme,
Linux Virtualization, target-devel, Christoph Hellwig
In-Reply-To: <1443049097.28503.13.camel@ssi>
On Wed, 2015-09-23 at 15:58 -0700, Ming Lin wrote:
> On Fri, 2015-09-18 at 14:09 -0700, Nicholas A. Bellinger wrote:
> > On Fri, 2015-09-18 at 11:12 -0700, Ming Lin wrote:
> > > On Thu, 2015-09-17 at 17:55 -0700, Nicholas A. Bellinger wrote:
<SNIP>
> > IBLOCK + FILEIO + RD_MCP don't speak SCSI, they simply process I/Os with
> > LBA + length based on SGL memory or pass along a FLUSH with LBA +
> > length.
> >
> > So once the 'tcm_eventfd_nvme' driver on KVM host receives a nvme host
> > hardware frame via eventfd, it would decode the frame and send along the
> > Read/Write/Flush when exposing existing (non nvme native) backend
> > drivers.
>
> Learned vhost architecture:
> http://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html
>
> The nice thing is it is not tied to KVM in any way.
>
Yes.
There are assumptions vhost currently makes about the guest using virtio
queues however, and at least for an initial vhost_nvme prototype it's
probably easier to avoid hacking up drivers/vhost/* (for now)..
(Adding MST CC')
> For SCSI, there are "virtio-scsi" in guest kernel and "vhost-scsi" in
> host kernel.
>
> For NVMe, there is no "virtio-nvme" in guest kernel(just unmodified NVMe
> driver), but I'll do similar thing in Qemu with vhost infrastructure.
> And there is "vhost_nvme" in host kernel.
>
> For the "virtqueue" implementation in qemu-nvme, I'll possibly just
> use/copy drivers/virtio/virtio_ring.c, same as what
> linux/tools/virtio/virtio_test.c does.
>
> A bit more detail graph as below. What do you think?
>
> .-----------------------------------------. .------------------------.
> | Guest(Linux, Windows, FreeBSD, Solaris) | NVMe | qemu |
> | unmodified NVMe driver | command | NVMe device emulation |
> | | -------> | vhost + virtqueue |
> '-----------------------------------------' '------------------------'
> | | ^
> passthrough | kick/notify
> NVMe command | via eventfd
> userspace via virtqueue | | |
> v v |
> ----------------------------------------------------------------------------------
This should read something like:
Passthrough of nvme hardware frames via QEMU PCI-e struct vhost_mem into
a custom vhost_nvme kernel driver ioctl using struct file + struct
eventfd_ctx primitives.
Eg: QEMU user-space is not performing the nvme command decode before
passing emulated nvme hardware frame up to host kernel driver.
> .-----------------------------------------------------------------------.
> kernel | LIO frontend driver |
> | - vhost_nvme |
> '-----------------------------------------------------------------------'
> | translate ^
> | (NVMe command) |
> | to |
> v (LBA, length) |
vhost_nvme is performing host kernel level decode of user-space provided
nvme hardware frames into nvme command + LBA +length + SGL buffer for
target backend driver submission
> .----------------------------------------------------------------------.
> | LIO backend driver |
> | - fileio (/mnt/xxx.file) |
> | - iblock (/dev/sda1, /dev/nvme0n1, ...) |
> '----------------------------------------------------------------------'
> | ^
> | submit_bio() |
> v |
> .----------------------------------------------------------------------.
> | block layer |
> | |
> '----------------------------------------------------------------------'
For this part, HCH mentioned he is currently working on some code to
pass native NVMe commands + SGL memory via blk-mq struct request into
struct nvme_dev and/or struct nvme_queue.
> | ^
> | |
> v |
> .----------------------------------------------------------------------.
> | block device driver |
> | |
> '----------------------------------------------------------------------'
> | | | |
> | | | |
> v v v v
> .------------. .-----------. .------------. .---------------.
> | SATA | | SCSI | | NVMe | | .... |
> '------------' '-----------' '------------' '---------------'
>
>
Looks fine.
Btw, after chatting with Dr. Hannes this week at SDC here are his
original rts-megasas -v6 patches from Feb 2013.
Note they are standalone patches that require a sufficiently old enough
LIO + QEMU to actually build + function.
https://github.com/Datera/rts-megasas/blob/master/rts_megasas-qemu-v6.patch
https://github.com/Datera/rts-megasas/blob/master/rts_megasas-fabric-v6.patch
For groking purposes, they demonstrate the principle design for a host
kernel level driver, along with the megasas firmware interface (MFI)
specific emulation magic that makes up the bulk of the code.
Take a look.
--nab
^ permalink raw reply
* Re: [PATCH v2] Fix AF_PACKET ABI breakage in 4.2
From: Greg Kurz @ 2015-09-25 12:33 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: kvm, netdev, linux-kernel, virtualization, linux-api,
David Woodhouse, David Miller
In-Reply-To: <20150924124443-mutt-send-email-mst@redhat.com>
On Thu, 24 Sep 2015 12:50:59 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Sep 24, 2015 at 09:25:45AM +0200, Greg Kurz wrote:
> > On Wed, 23 Sep 2015 19:45:08 +0100
> > David Woodhouse <dwmw2@infradead.org> wrote:
> >
> > > Commit 7d82410950aa ("virtio: add explicit big-endian support to memory
> > > accessors") accidentally changed the virtio_net header used by
> > > AF_PACKET with PACKET_VNET_HDR from host-endian to big-endian.
> > >
> >
> > Hi David,
> >
> > Oops my bad... I obviously overlooked this one when adding cross-endian
> > support.
> >
> > > Since virtio_legacy_is_little_endian() is a very long identifier,
> > > define a VIO_LE macro and use that throughout the code instead of the
> >
> > VIO usually refers to virtual IO adapters for the PowerPC pSeries platform.
> >
> > > hard-coded 'false' for little-endian.
> > >
> >
> > What about introducing dedicated accessors as it is done in many other
> > locations where we do virtio byteswap ? Something like:
> >
> > static inline bool packet_is_little_endian(void)
> > {
> > return virtio_legacy_is_little_endian();
> > }
> >
> > static inline u16 packet16_to_cpu(__virtio16 val)
> > {
> > return __virtio16_to_cpu(packet_is_little_endian(), val);
> > }
> >
> > static inline __virtio16 cpu_to_packet16(u16 val)
> > {
> > return __cpu_to_virtio16(packet_is_little_endian(), val);
> > }
> >
> > It results in prettier code IMHO. Have a look at drivers/net/tun.c or
> > drivers/vhost/vhost.c.
> >
> > > This restores the ABI to match 4.1 and earlier kernels, and makes my
> > > test program work again.
> > >
> >
> > BTW, there is still work to do if we want to support cross-endian legacy or
> > virtio 1 on a big endian arch...
> >
> > Cheers.
> >
> > --
> > Greg
>
> It seems the API that we have is a confusing one.
>
> virtio endian-ness is either native or little, depending on a flag, so
> __virtio16_to_cpu seems to mean "either native to cpu or little to cpu
> depending on flag".
>
> It used to be like that, but not anymore.
>
> This leads to all kind of bugs.
>
> For example, I have only now realized vhost_is_little_endian isn't a
> constant on LE hosts if cross-endian support is not compiled.
>
> I think we need to fix it, but also think about a better API.
>
Your original API is good and works well for all the callers that
don't care for cross-endian support.
I guess we'd rather move the cross-endian burden to the few callers who
need it, i.e. tun, macvtap and vhost when cross-endian is compiled.
More or less like this:
/* Original API : either little-endian or native */
static inline u16 __virtio16_to_cpu(bool little_endian, __virtio16 val)
{
if (little_endian)
return le16_to_cpu((__force __le16)val);
else
return (__force u16)val;
}
#ifdef CONFIG_VHOST_CROSS_ENDIAN_LEGACY
static inline u16 vhost16_to_cpu(struct vhost_virtqueue *vq, __virtio16 val)
{
/* little-endian because of virtio 1 */
if (vhost_has_feature(vq, VIRTIO_F_VERSION_1))
return __virtio16_to_cpu(true, val);
#ifdef __LITTLE_ENDIAN
/* native little-endian */
return (__force u16)val;
#else
/* native big-endian */
return be16_to_cpu((__force __be16)val);
#endif
}
#else
static inline u16 vhost16_to_cpu(struct vhost_virtqueue *vq, __virtio16 val)
{
#ifdef __LITTLE_ENDIAN
/* fast path for little-endian host */
return __virtio16_to_cpu(true, val);
#else
/* slow path for big-endian host */
return __virtio16_to_cpu(vhost_has_feature(vq, VIRTIO_F_VERSION_1), val);
#endif
}
#endif
Does it make sense ?
>
> > > Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
> > > ---
> > > On Wed, 2015-09-23 at 11:09 -0700, David Miller wrote:
> > > > > +#define VIO_LE virtio_legacy_is_little_endian()
> > > >
> > > > When you define a shorthand macro, the defines to a function call,
> > > > make the macro have parenthesis too.
> > >
> > > In which case I suppose it also wants to be lower-case. Although
> > > "function call" is a bit strong since it's effectively just a constant.
> > > I'm still wondering if it'd be nicer just to use (__force u16) instead.
> > >
> > > diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
> > > index 7b8e39a..aa4b15c 100644
> > > --- a/net/packet/af_packet.c
> > > +++ b/net/packet/af_packet.c
> > > @@ -230,6 +230,8 @@ struct packet_skb_cb {
> > > } sa;
> > > };
> > >
> > > +#define vio_le() virtio_legacy_is_little_endian()
> > > +
> > > #define PACKET_SKB_CB(__skb) ((struct packet_skb_cb *)((__skb)->cb))
> > >
> > > #define GET_PBDQC_FROM_RB(x) ((struct tpacket_kbdq_core *)(&(x)->prb_bdqc))
> > > @@ -2680,15 +2682,15 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > > goto out_unlock;
> > >
> > > if ((vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
> > > - (__virtio16_to_cpu(false, vnet_hdr.csum_start) +
> > > - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2 >
> > > - __virtio16_to_cpu(false, vnet_hdr.hdr_len)))
> > > - vnet_hdr.hdr_len = __cpu_to_virtio16(false,
> > > - __virtio16_to_cpu(false, vnet_hdr.csum_start) +
> > > - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2);
> > > + (__virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2 >
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len))) actually
> > > + vnet_hdr.hdr_len = __cpu_to_virtio16(vio_le(),
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2);
> > >
> > > err = -EINVAL;
> > > - if (__virtio16_to_cpu(false, vnet_hdr.hdr_len) > len)
> > > + if (__virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len) > len)
> > > goto out_unlock;
> > >
> > > if (vnet_hdr.gso_type != VIRTIO_NET_HDR_GSO_NONE) {
> > > @@ -2731,7 +2733,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > > hlen = LL_RESERVED_SPACE(dev);
> > > tlen = dev->needed_tailroom;
> > > skb = packet_alloc_skb(sk, hlen + tlen, hlen, len,
> > > - __virtio16_to_cpu(false, vnet_hdr.hdr_len),
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len),
> > > msg->msg_flags & MSG_DONTWAIT, &err);
> > > if (skb == NULL)
> > > goto out_unlock;
> > > @@ -2778,8 +2780,8 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > >
> > > if (po->has_vnet_hdr) {
> > > if (vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
> > > - u16 s = __virtio16_to_cpu(false, vnet_hdr.csum_start);
> > > - u16 o = __virtio16_to_cpu(false, vnet_hdr.csum_offset);
> > > + u16 s = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start);
> > > + u16 o = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset);
> > > if (!skb_partial_csum_set(skb, s, o)) {
> > > err = -EINVAL;
> > > goto out_free;
> > > @@ -2787,7 +2789,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > > }
> > >
> > > skb_shinfo(skb)->gso_size =
> > > - __virtio16_to_cpu(false, vnet_hdr.gso_size);
> > > + __virtio16_to_cpu(vio_le(), vnet_hdr.gso_size);
> > > skb_shinfo(skb)->gso_type = gso_type;
> > >
> > > /* Header must be checked, and gso_segs computed. */
> > > @@ -3161,9 +3163,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
> > >
> > > /* This is a hint as to how much should be linear. */
> > > vnet_hdr.hdr_len =
> > > - __cpu_to_virtio16(false, skb_headlen(skb));
> > > + __cpu_to_virtio16(vio_le(), skb_headlen(skb));
> > > vnet_hdr.gso_size =
> > > - __cpu_to_virtio16(false, sinfo->gso_size);
> > > + __cpu_to_virtio16(vio_le(), sinfo->gso_size);
> > > if (sinfo->gso_type & SKB_GSO_TCPV4)
> > > vnet_hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
> > > else if (sinfo->gso_type & SKB_GSO_TCPV6)
> > > @@ -3181,9 +3183,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
> > >
> > > if (skb->ip_summed == CHECKSUM_PARTIAL) {
> > > vnet_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
> > > - vnet_hdr.csum_start = __cpu_to_virtio16(false,
> > > + vnet_hdr.csum_start = __cpu_to_virtio16(vio_le(),
> > > skb_checksum_start_offset(skb));
> > > - vnet_hdr.csum_offset = __cpu_to_virtio16(false,
> > > + vnet_hdr.csum_offset = __cpu_to_virtio16(vio_le(),
> > > skb->csum_offset);
> > > } else if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
> > > vnet_hdr.flags = VIRTIO_NET_HDR_F_DATA_VALID;
> > >
>
--
Gregory Kurz kurzgreg@fr.ibm.com
gkurz@linux.vnet.ibm.com
Software Engineer @ IBM/LTC http://www.ibm.com
Tel 33-5-6218-1607
"Anarchy is about taking complete responsibility for yourself."
Alan Moore.
^ permalink raw reply
* Re: [PATCH v2] Fix AF_PACKET ABI breakage in 4.2
From: David Woodhouse @ 2015-09-24 10:08 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: kvm, netdev, linux-kernel, virtualization, linux-api,
David Miller
In-Reply-To: <20150924095519.GA12444@redhat.com>
[-- Attachment #1.1: Type: text/plain, Size: 642 bytes --]
On Thu, 2015-09-24 at 12:55 +0300, Michael S. Tsirkin wrote:
>
> I'm fine with this patch
>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
Thanks. In fact Dave has already merged it.
> but if you want to re-work it along the lines suggested
> by Greg, that's also fine with me.
If I'm going to define my own accessors, I'd probably just make them
use (__force __virtio16). But TBH none of the options seemed
particularly pretty to me. I can live with what we have.
--
David Woodhouse Open Source Technology Centre
David.Woodhouse@intel.com Intel Corporation
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5691 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: RFC: virtio-peer shared memory based peer communication device
From: Michael S. Tsirkin @ 2015-09-24 10:04 UTC (permalink / raw)
To: Jan Kiszka
Cc: virtio-dev@lists.oasis-open.org, Claudio Fontana,
qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org,
Zhang, Yang Z, opnfv-tech-discuss@lists.opnfv.org
In-Reply-To: <55FFF8CA.3020008@siemens.com>
On Mon, Sep 21, 2015 at 02:32:10PM +0200, Jan Kiszka wrote:
> On 2015-09-21 14:13, Michael S. Tsirkin wrote:
> > On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
> >> Hello,
> >>
> >> this is a first RFC for virtio-peer 0.1, which is still very much a work in progress:
> >>
> >> https://github.com/hw-claudio/virtio-peer/wiki
> >>
> >> It is also available as PDF there, but the text is reproduced here for commenting:
> >>
> >> Peer shared memory communication device (virtio-peer)
> >>
> >> General Overview
> >>
> >> (I recommend looking at the PDF for some clarifying pictures)
> >>
> >> The Virtio Peer shared memory communication device (virtio-peer) is a
> >> virtual device which allows high performance low latency guest to
> >> guest communication. It uses a new queue extension feature tentatively
> >> called VIRTIO_F_WINDOW which indicates that descriptor tables,
> >> available and used rings and Queue Data reside in physical memory
> >> ranges called Windows, each identified with an unique identifier
> >> called WindowID.
> >
> > So if I had to summarize the difference from regular virtio,
> > I'd say the main one is that this uses window id + offset
> > instead of the physical address.
> >
> >
> > My question is - why do it?
> >
> > All windows are in memory space, are they not?
> >
> > How about guest using full physical addresses,
> > and hypervisor sending the window physical address
> > to VM2?
> >
> > VM2 can uses that to find both window id and offset.
> >
> >
> > This way at least VM1 can use regular virtio without changes.
>
> What would be the value of having different drivers in VM1 and VM2,
> specifically if both run Linux?
>
> Jan
It's common to have a VM act as a switch between others.
In this setup, there's value in being able to support existing guests as
endpoints, with new drivers only required for the switch.
> --
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux
^ permalink raw reply
* Re: [PATCH v2] Fix AF_PACKET ABI breakage in 4.2
From: Michael S. Tsirkin @ 2015-09-24 9:55 UTC (permalink / raw)
To: David Woodhouse
Cc: kvm, netdev, linux-kernel, virtualization, linux-api,
David Miller
In-Reply-To: <1443033908.74600.21.camel@infradead.org>
On Wed, Sep 23, 2015 at 07:45:08PM +0100, David Woodhouse wrote:
> Commit 7d82410950aa ("virtio: add explicit big-endian support to memory
> accessors") accidentally changed the virtio_net header used by
> AF_PACKET with PACKET_VNET_HDR from host-endian to big-endian.
>
> Since virtio_legacy_is_little_endian() is a very long identifier,
> define a VIO_LE macro and use that throughout the code instead of the
> hard-coded 'false' for little-endian.
>
> This restores the ABI to match 4.1 and earlier kernels, and makes my
> test program work again.
>
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
I'm fine with this patch
Acked-by: Michael S. Tsirkin <mst@redhat.com>
but if you want to re-work it along the lines suggested
by Greg, that's also fine with me.
> ---
> On Wed, 2015-09-23 at 11:09 -0700, David Miller wrote:
> > > +#define VIO_LE virtio_legacy_is_little_endian()
> >
> > When you define a shorthand macro, the defines to a function call,
> > make the macro have parenthesis too.
>
> In which case I suppose it also wants to be lower-case. Although
> "function call" is a bit strong since it's effectively just a constant.
> I'm still wondering if it'd be nicer just to use (__force u16) instead.
>
> diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
> index 7b8e39a..aa4b15c 100644
> --- a/net/packet/af_packet.c
> +++ b/net/packet/af_packet.c
> @@ -230,6 +230,8 @@ struct packet_skb_cb {
> } sa;
> };
>
> +#define vio_le() virtio_legacy_is_little_endian()
> +
> #define PACKET_SKB_CB(__skb) ((struct packet_skb_cb *)((__skb)->cb))
>
> #define GET_PBDQC_FROM_RB(x) ((struct tpacket_kbdq_core *)(&(x)->prb_bdqc))
> @@ -2680,15 +2682,15 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> goto out_unlock;
>
> if ((vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
> - (__virtio16_to_cpu(false, vnet_hdr.csum_start) +
> - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2 >
> - __virtio16_to_cpu(false, vnet_hdr.hdr_len)))
> - vnet_hdr.hdr_len = __cpu_to_virtio16(false,
> - __virtio16_to_cpu(false, vnet_hdr.csum_start) +
> - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2);
> + (__virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2 >
> + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len)))
> + vnet_hdr.hdr_len = __cpu_to_virtio16(vio_le(),
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2);
>
> err = -EINVAL;
> - if (__virtio16_to_cpu(false, vnet_hdr.hdr_len) > len)
> + if (__virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len) > len)
> goto out_unlock;
>
> if (vnet_hdr.gso_type != VIRTIO_NET_HDR_GSO_NONE) {
> @@ -2731,7 +2733,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> hlen = LL_RESERVED_SPACE(dev);
> tlen = dev->needed_tailroom;
> skb = packet_alloc_skb(sk, hlen + tlen, hlen, len,
> - __virtio16_to_cpu(false, vnet_hdr.hdr_len),
> + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len),
> msg->msg_flags & MSG_DONTWAIT, &err);
> if (skb == NULL)
> goto out_unlock;
> @@ -2778,8 +2780,8 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
>
> if (po->has_vnet_hdr) {
> if (vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
> - u16 s = __virtio16_to_cpu(false, vnet_hdr.csum_start);
> - u16 o = __virtio16_to_cpu(false, vnet_hdr.csum_offset);
> + u16 s = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start);
> + u16 o = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset);
> if (!skb_partial_csum_set(skb, s, o)) {
> err = -EINVAL;
> goto out_free;
> @@ -2787,7 +2789,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> }
>
> skb_shinfo(skb)->gso_size =
> - __virtio16_to_cpu(false, vnet_hdr.gso_size);
> + __virtio16_to_cpu(vio_le(), vnet_hdr.gso_size);
> skb_shinfo(skb)->gso_type = gso_type;
>
> /* Header must be checked, and gso_segs computed. */
> @@ -3161,9 +3163,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
>
> /* This is a hint as to how much should be linear. */
> vnet_hdr.hdr_len =
> - __cpu_to_virtio16(false, skb_headlen(skb));
> + __cpu_to_virtio16(vio_le(), skb_headlen(skb));
> vnet_hdr.gso_size =
> - __cpu_to_virtio16(false, sinfo->gso_size);
> + __cpu_to_virtio16(vio_le(), sinfo->gso_size);
> if (sinfo->gso_type & SKB_GSO_TCPV4)
> vnet_hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
> else if (sinfo->gso_type & SKB_GSO_TCPV6)
> @@ -3181,9 +3183,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
>
> if (skb->ip_summed == CHECKSUM_PARTIAL) {
> vnet_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
> - vnet_hdr.csum_start = __cpu_to_virtio16(false,
> + vnet_hdr.csum_start = __cpu_to_virtio16(vio_le(),
> skb_checksum_start_offset(skb));
> - vnet_hdr.csum_offset = __cpu_to_virtio16(false,
> + vnet_hdr.csum_offset = __cpu_to_virtio16(vio_le(),
> skb->csum_offset);
> } else if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
> vnet_hdr.flags = VIRTIO_NET_HDR_F_DATA_VALID;
>
> --
> David Woodhouse Open Source Technology Centre
> David.Woodhouse@intel.com Intel Corporation
>
^ permalink raw reply
* Re: [PATCH v2] Fix AF_PACKET ABI breakage in 4.2
From: Michael S. Tsirkin @ 2015-09-24 9:50 UTC (permalink / raw)
To: Greg Kurz
Cc: kvm, netdev, linux-kernel, virtualization, linux-api,
David Woodhouse, David Miller
In-Reply-To: <20150924092545.28d3ae52@bahia.local>
On Thu, Sep 24, 2015 at 09:25:45AM +0200, Greg Kurz wrote:
> On Wed, 23 Sep 2015 19:45:08 +0100
> David Woodhouse <dwmw2@infradead.org> wrote:
>
> > Commit 7d82410950aa ("virtio: add explicit big-endian support to memory
> > accessors") accidentally changed the virtio_net header used by
> > AF_PACKET with PACKET_VNET_HDR from host-endian to big-endian.
> >
>
> Hi David,
>
> Oops my bad... I obviously overlooked this one when adding cross-endian
> support.
>
> > Since virtio_legacy_is_little_endian() is a very long identifier,
> > define a VIO_LE macro and use that throughout the code instead of the
>
> VIO usually refers to virtual IO adapters for the PowerPC pSeries platform.
>
> > hard-coded 'false' for little-endian.
> >
>
> What about introducing dedicated accessors as it is done in many other
> locations where we do virtio byteswap ? Something like:
>
> static inline bool packet_is_little_endian(void)
> {
> return virtio_legacy_is_little_endian();
> }
>
> static inline u16 packet16_to_cpu(__virtio16 val)
> {
> return __virtio16_to_cpu(packet_is_little_endian(), val);
> }
>
> static inline __virtio16 cpu_to_packet16(u16 val)
> {
> return __cpu_to_virtio16(packet_is_little_endian(), val);
> }
>
> It results in prettier code IMHO. Have a look at drivers/net/tun.c or
> drivers/vhost/vhost.c.
>
> > This restores the ABI to match 4.1 and earlier kernels, and makes my
> > test program work again.
> >
>
> BTW, there is still work to do if we want to support cross-endian legacy or
> virtio 1 on a big endian arch...
>
> Cheers.
>
> --
> Greg
It seems the API that we have is a confusing one.
virtio endian-ness is either native or little, depending on a flag, so
__virtio16_to_cpu seems to mean "either native to cpu or little to cpu
depending on flag".
It used to be like that, but not anymore.
This leads to all kind of bugs.
For example, I have only now realized vhost_is_little_endian isn't a
constant on LE hosts if cross-endian support is not compiled.
I think we need to fix it, but also think about a better API.
> > Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
> > ---
> > On Wed, 2015-09-23 at 11:09 -0700, David Miller wrote:
> > > > +#define VIO_LE virtio_legacy_is_little_endian()
> > >
> > > When you define a shorthand macro, the defines to a function call,
> > > make the macro have parenthesis too.
> >
> > In which case I suppose it also wants to be lower-case. Although
> > "function call" is a bit strong since it's effectively just a constant.
> > I'm still wondering if it'd be nicer just to use (__force u16) instead.
> >
> > diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
> > index 7b8e39a..aa4b15c 100644
> > --- a/net/packet/af_packet.c
> > +++ b/net/packet/af_packet.c
> > @@ -230,6 +230,8 @@ struct packet_skb_cb {
> > } sa;
> > };
> >
> > +#define vio_le() virtio_legacy_is_little_endian()
> > +
> > #define PACKET_SKB_CB(__skb) ((struct packet_skb_cb *)((__skb)->cb))
> >
> > #define GET_PBDQC_FROM_RB(x) ((struct tpacket_kbdq_core *)(&(x)->prb_bdqc))
> > @@ -2680,15 +2682,15 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > goto out_unlock;
> >
> > if ((vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
> > - (__virtio16_to_cpu(false, vnet_hdr.csum_start) +
> > - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2 >
> > - __virtio16_to_cpu(false, vnet_hdr.hdr_len)))
> > - vnet_hdr.hdr_len = __cpu_to_virtio16(false,
> > - __virtio16_to_cpu(false, vnet_hdr.csum_start) +
> > - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2);
> > + (__virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2 >
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len)))
> > + vnet_hdr.hdr_len = __cpu_to_virtio16(vio_le(),
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2);
> >
> > err = -EINVAL;
> > - if (__virtio16_to_cpu(false, vnet_hdr.hdr_len) > len)
> > + if (__virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len) > len)
> > goto out_unlock;
> >
> > if (vnet_hdr.gso_type != VIRTIO_NET_HDR_GSO_NONE) {
> > @@ -2731,7 +2733,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > hlen = LL_RESERVED_SPACE(dev);
> > tlen = dev->needed_tailroom;
> > skb = packet_alloc_skb(sk, hlen + tlen, hlen, len,
> > - __virtio16_to_cpu(false, vnet_hdr.hdr_len),
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len),
> > msg->msg_flags & MSG_DONTWAIT, &err);
> > if (skb == NULL)
> > goto out_unlock;
> > @@ -2778,8 +2780,8 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> >
> > if (po->has_vnet_hdr) {
> > if (vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
> > - u16 s = __virtio16_to_cpu(false, vnet_hdr.csum_start);
> > - u16 o = __virtio16_to_cpu(false, vnet_hdr.csum_offset);
> > + u16 s = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start);
> > + u16 o = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset);
> > if (!skb_partial_csum_set(skb, s, o)) {
> > err = -EINVAL;
> > goto out_free;
> > @@ -2787,7 +2789,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> > }
> >
> > skb_shinfo(skb)->gso_size =
> > - __virtio16_to_cpu(false, vnet_hdr.gso_size);
> > + __virtio16_to_cpu(vio_le(), vnet_hdr.gso_size);
> > skb_shinfo(skb)->gso_type = gso_type;
> >
> > /* Header must be checked, and gso_segs computed. */
> > @@ -3161,9 +3163,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
> >
> > /* This is a hint as to how much should be linear. */
> > vnet_hdr.hdr_len =
> > - __cpu_to_virtio16(false, skb_headlen(skb));
> > + __cpu_to_virtio16(vio_le(), skb_headlen(skb));
> > vnet_hdr.gso_size =
> > - __cpu_to_virtio16(false, sinfo->gso_size);
> > + __cpu_to_virtio16(vio_le(), sinfo->gso_size);
> > if (sinfo->gso_type & SKB_GSO_TCPV4)
> > vnet_hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
> > else if (sinfo->gso_type & SKB_GSO_TCPV6)
> > @@ -3181,9 +3183,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
> >
> > if (skb->ip_summed == CHECKSUM_PARTIAL) {
> > vnet_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
> > - vnet_hdr.csum_start = __cpu_to_virtio16(false,
> > + vnet_hdr.csum_start = __cpu_to_virtio16(vio_le(),
> > skb_checksum_start_offset(skb));
> > - vnet_hdr.csum_offset = __cpu_to_virtio16(false,
> > + vnet_hdr.csum_offset = __cpu_to_virtio16(vio_le(),
> > skb->csum_offset);
> > } else if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
> > vnet_hdr.flags = VIRTIO_NET_HDR_F_DATA_VALID;
> >
^ permalink raw reply
* Re: [PATCH v2] Fix AF_PACKET ABI breakage in 4.2
From: Greg Kurz @ 2015-09-24 7:25 UTC (permalink / raw)
To: David Woodhouse
Cc: kvm, mst, netdev, linux-kernel, virtualization, linux-api,
David Miller
In-Reply-To: <1443033908.74600.21.camel@infradead.org>
On Wed, 23 Sep 2015 19:45:08 +0100
David Woodhouse <dwmw2@infradead.org> wrote:
> Commit 7d82410950aa ("virtio: add explicit big-endian support to memory
> accessors") accidentally changed the virtio_net header used by
> AF_PACKET with PACKET_VNET_HDR from host-endian to big-endian.
>
Hi David,
Oops my bad... I obviously overlooked this one when adding cross-endian
support.
> Since virtio_legacy_is_little_endian() is a very long identifier,
> define a VIO_LE macro and use that throughout the code instead of the
VIO usually refers to virtual IO adapters for the PowerPC pSeries platform.
> hard-coded 'false' for little-endian.
>
What about introducing dedicated accessors as it is done in many other
locations where we do virtio byteswap ? Something like:
static inline bool packet_is_little_endian(void)
{
return virtio_legacy_is_little_endian();
}
static inline u16 packet16_to_cpu(__virtio16 val)
{
return __virtio16_to_cpu(packet_is_little_endian(), val);
}
static inline __virtio16 cpu_to_packet16(u16 val)
{
return __cpu_to_virtio16(packet_is_little_endian(), val);
}
It results in prettier code IMHO. Have a look at drivers/net/tun.c or
drivers/vhost/vhost.c.
> This restores the ABI to match 4.1 and earlier kernels, and makes my
> test program work again.
>
BTW, there is still work to do if we want to support cross-endian legacy or
virtio 1 on a big endian arch...
Cheers.
--
Greg
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
> ---
> On Wed, 2015-09-23 at 11:09 -0700, David Miller wrote:
> > > +#define VIO_LE virtio_legacy_is_little_endian()
> >
> > When you define a shorthand macro, the defines to a function call,
> > make the macro have parenthesis too.
>
> In which case I suppose it also wants to be lower-case. Although
> "function call" is a bit strong since it's effectively just a constant.
> I'm still wondering if it'd be nicer just to use (__force u16) instead.
>
> diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
> index 7b8e39a..aa4b15c 100644
> --- a/net/packet/af_packet.c
> +++ b/net/packet/af_packet.c
> @@ -230,6 +230,8 @@ struct packet_skb_cb {
> } sa;
> };
>
> +#define vio_le() virtio_legacy_is_little_endian()
> +
> #define PACKET_SKB_CB(__skb) ((struct packet_skb_cb *)((__skb)->cb))
>
> #define GET_PBDQC_FROM_RB(x) ((struct tpacket_kbdq_core *)(&(x)->prb_bdqc))
> @@ -2680,15 +2682,15 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> goto out_unlock;
>
> if ((vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
> - (__virtio16_to_cpu(false, vnet_hdr.csum_start) +
> - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2 >
> - __virtio16_to_cpu(false, vnet_hdr.hdr_len)))
> - vnet_hdr.hdr_len = __cpu_to_virtio16(false,
> - __virtio16_to_cpu(false, vnet_hdr.csum_start) +
> - __virtio16_to_cpu(false, vnet_hdr.csum_offset) + 2);
> + (__virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2 >
> + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len)))
> + vnet_hdr.hdr_len = __cpu_to_virtio16(vio_le(),
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start) +
> + __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset) + 2);
>
> err = -EINVAL;
> - if (__virtio16_to_cpu(false, vnet_hdr.hdr_len) > len)
> + if (__virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len) > len)
> goto out_unlock;
>
> if (vnet_hdr.gso_type != VIRTIO_NET_HDR_GSO_NONE) {
> @@ -2731,7 +2733,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> hlen = LL_RESERVED_SPACE(dev);
> tlen = dev->needed_tailroom;
> skb = packet_alloc_skb(sk, hlen + tlen, hlen, len,
> - __virtio16_to_cpu(false, vnet_hdr.hdr_len),
> + __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len),
> msg->msg_flags & MSG_DONTWAIT, &err);
> if (skb == NULL)
> goto out_unlock;
> @@ -2778,8 +2780,8 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
>
> if (po->has_vnet_hdr) {
> if (vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
> - u16 s = __virtio16_to_cpu(false, vnet_hdr.csum_start);
> - u16 o = __virtio16_to_cpu(false, vnet_hdr.csum_offset);
> + u16 s = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_start);
> + u16 o = __virtio16_to_cpu(vio_le(), vnet_hdr.csum_offset);
> if (!skb_partial_csum_set(skb, s, o)) {
> err = -EINVAL;
> goto out_free;
> @@ -2787,7 +2789,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
> }
>
> skb_shinfo(skb)->gso_size =
> - __virtio16_to_cpu(false, vnet_hdr.gso_size);
> + __virtio16_to_cpu(vio_le(), vnet_hdr.gso_size);
> skb_shinfo(skb)->gso_type = gso_type;
>
> /* Header must be checked, and gso_segs computed. */
> @@ -3161,9 +3163,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
>
> /* This is a hint as to how much should be linear. */
> vnet_hdr.hdr_len =
> - __cpu_to_virtio16(false, skb_headlen(skb));
> + __cpu_to_virtio16(vio_le(), skb_headlen(skb));
> vnet_hdr.gso_size =
> - __cpu_to_virtio16(false, sinfo->gso_size);
> + __cpu_to_virtio16(vio_le(), sinfo->gso_size);
> if (sinfo->gso_type & SKB_GSO_TCPV4)
> vnet_hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
> else if (sinfo->gso_type & SKB_GSO_TCPV6)
> @@ -3181,9 +3183,9 @@ static int packet_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
>
> if (skb->ip_summed == CHECKSUM_PARTIAL) {
> vnet_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
> - vnet_hdr.csum_start = __cpu_to_virtio16(false,
> + vnet_hdr.csum_start = __cpu_to_virtio16(vio_le(),
> skb_checksum_start_offset(skb));
> - vnet_hdr.csum_offset = __cpu_to_virtio16(false,
> + vnet_hdr.csum_offset = __cpu_to_virtio16(vio_le(),
> skb->csum_offset);
> } else if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
> vnet_hdr.flags = VIRTIO_NET_HDR_F_DATA_VALID;
>
^ permalink raw reply
* Re: [RFC PATCH 0/2] virtio nvme
From: Ming Lin @ 2015-09-23 22:58 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Minturn, Dave B, linux-nvme, Linux Virtualization, target-devel,
Christoph Hellwig
In-Reply-To: <1442610544.10492.33.camel@haakon3.risingtidesystems.com>
On Fri, 2015-09-18 at 14:09 -0700, Nicholas A. Bellinger wrote:
> On Fri, 2015-09-18 at 11:12 -0700, Ming Lin wrote:
> > On Thu, 2015-09-17 at 17:55 -0700, Nicholas A. Bellinger wrote:
> > > On Thu, 2015-09-17 at 16:31 -0700, Ming Lin wrote:
> > > > On Wed, 2015-09-16 at 23:10 -0700, Nicholas A. Bellinger wrote:
> > > > > Hi Ming & Co,
>
> <SNIP>
>
> > > > > > I think the future "LIO NVMe target" only speaks NVMe protocol.
> > > > > >
> > > > > > Nick(CCed), could you correct me if I'm wrong?
> > > > > >
> > > > > > For SCSI stack, we have:
> > > > > > virtio-scsi(guest)
> > > > > > tcm_vhost(or vhost_scsi, host)
> > > > > > LIO-scsi-target
> > > > > >
> > > > > > For NVMe stack, we'll have similar components:
> > > > > > virtio-nvme(guest)
> > > > > > vhost_nvme(host)
> > > > > > LIO-NVMe-target
> > > > > >
> > > > >
> > > > > I think it's more interesting to consider a 'vhost style' driver that
> > > > > can be used with unmodified nvme host OS drivers.
> > > > >
> > > > > Dr. Hannes (CC'ed) had done something like this for megasas a few years
> > > > > back using specialized QEMU emulation + eventfd based LIO fabric driver,
> > > > > and got it working with Linux + MSFT guests.
> > > > >
> > > > > Doing something similar for nvme would (potentially) be on par with
> > > > > current virtio-scsi+vhost-scsi small-block performance for scsi-mq
> > > > > guests, without the extra burden of a new command set specific virtio
> > > > > driver.
> > > >
> > > > Trying to understand it.
> > > > Is it like below?
> > > >
> > > > .------------------------. MMIO .---------------------------------------.
> > > > | Guest |--------> | Qemu |
> > > > | Unmodified NVMe driver |<-------- | NVMe device simulation(eventfd based) |
> > > > '------------------------' '---------------------------------------'
> > > > | ^
> > > > write NVMe | | notify command
> > > > command | | completion
> > > > to eventfd | | to eventfd
> > > > v |
> > > > .--------------------------------------.
> > > > | Host: |
> > > > | eventfd based LIO NVMe fabric driver |
> > > > '--------------------------------------'
> > > > |
> > > > | nvme_queue_rq()
> > > > v
> > > > .--------------------------------------.
> > > > | NVMe driver |
> > > > '--------------------------------------'
> > > > |
> > > > |
> > > > v
> > > > .-------------------------------------.
> > > > | NVMe device |
> > > > '-------------------------------------'
> > > >
> > >
> > > Correct. The LIO driver on KVM host would be handling some amount of
> > > NVMe host interface emulation in kernel code, and would be able to
> > > decode nvme Read/Write/Flush operations and translate -> submit to
> > > existing backend drivers.
> >
> > Let me call the "eventfd based LIO NVMe fabric driver" as
> > "tcm_eventfd_nvme"
> >
> > Currently, LIO frontend driver(iscsi, fc, vhost-scsi etc) talk to LIO
> > backend driver(fileio, iblock etc) with SCSI commands.
> >
> > Did you mean the "tcm_eventfd_nvme" driver need to translate NVMe
> > commands to SCSI commands and then submit to backend driver?
> >
>
> IBLOCK + FILEIO + RD_MCP don't speak SCSI, they simply process I/Os with
> LBA + length based on SGL memory or pass along a FLUSH with LBA +
> length.
>
> So once the 'tcm_eventfd_nvme' driver on KVM host receives a nvme host
> hardware frame via eventfd, it would decode the frame and send along the
> Read/Write/Flush when exposing existing (non nvme native) backend
> drivers.
Learned vhost architecture:
http://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html
The nice thing is it is not tied to KVM in any way.
For SCSI, there are "virtio-scsi" in guest kernel and "vhost-scsi" in
host kernel.
For NVMe, there is no "virtio-nvme" in guest kernel(just unmodified NVMe
driver), but I'll do similar thing in Qemu with vhost infrastructure.
And there is "vhost_nvme" in host kernel.
For the "virtqueue" implementation in qemu-nvme, I'll possibly just
use/copy drivers/virtio/virtio_ring.c, same as what
linux/tools/virtio/virtio_test.c does.
A bit more detail graph as below. What do you think?
.-----------------------------------------. .------------------------.
| Guest(Linux, Windows, FreeBSD, Solaris) | NVMe | qemu |
| unmodified NVMe driver | command | NVMe device emulation |
| | -------> | vhost + virtqueue |
'-----------------------------------------' '------------------------'
| | ^
passthrough | kick/notify
NVMe command | via eventfd
userspace via virtqueue | | |
v v |
----------------------------------------------------------------------------------
.-----------------------------------------------------------------------.
kernel | LIO frontend driver |
| - vhost_nvme |
'-----------------------------------------------------------------------'
| translate ^
| (NVMe command) |
| to |
v (LBA, length) |
.----------------------------------------------------------------------.
| LIO backend driver |
| - fileio (/mnt/xxx.file) |
| - iblock (/dev/sda1, /dev/nvme0n1, ...) |
'----------------------------------------------------------------------'
| ^
| submit_bio() |
v |
.----------------------------------------------------------------------.
| block layer |
| |
'----------------------------------------------------------------------'
| ^
| |
v |
.----------------------------------------------------------------------.
| block device driver |
| |
'----------------------------------------------------------------------'
| | | |
| | | |
v v v v
.------------. .-----------. .------------. .---------------.
| SATA | | SCSI | | NVMe | | .... |
'------------' '-----------' '------------' '---------------'
^ permalink raw reply
* RE: [opnfv-tech-discuss] RFC: virtio-peer shared memory based peer communication device
From: Ergin, Mesut A @ 2015-09-23 18:53 UTC (permalink / raw)
To: Claudio Fontana, Zhang, Yang Z, Michael S. Tsirkin,
qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org,
virtio-dev@lists.oasis-open.org,
opnfv-tech-discuss@lists.opnfv.org
Cc: Jan Kiszka
On 18/09/2015 18:29, Claudio Fontana wrote:
> The Virtio Peer shared memory communication device (virtio-peer) is a
> virtual device which allows high performance low latency guest to guest
> communication. It uses a new queue extension feature tentatively called
> VIRTIO_F_WINDOW which indicates that descriptor tables, available and
> used rings and Queue Data reside in physical memory ranges called
> Windows, each identified with an unique identifier called WindowID.
>
Should we consider virtio-peer a pair-wise created device for two guests,
where sending side driver is operating receive side virtqueue to drop
packets directly? Do you have some architectural description of how
guests are expected to use virtio-peer? Thanks.
Mesut
^ permalink raw reply
* Re: [PATCH] Fix AF_PACKET ABI breakage in 4.2
From: David Miller @ 2015-09-23 18:09 UTC (permalink / raw)
To: dwmw2; +Cc: kvm, mst, netdev, linux-kernel, virtualization, linux-api
In-Reply-To: <1443019470.74600.6.camel@infradead.org>
From: David Woodhouse <dwmw2@infradead.org>
Date: Wed, 23 Sep 2015 15:44:30 +0100
> +#define VIO_LE virtio_legacy_is_little_endian()
When you define a shorthand macro, the defines to a function call,
make the macro have parenthesis too.
^ permalink raw reply
* Re: [PATCH v2 5/6] virtio-gpu: add basic prime support
From: Daniel Vetter @ 2015-09-22 16:15 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Dave Airlie, open list:VIRTIO GPU DRIVER, dri-devel,
Daniel Vetter, open list
In-Reply-To: <1442935469.13084.36.camel@redhat.com>
On Tue, Sep 22, 2015 at 05:24:29PM +0200, Gerd Hoffmann wrote:
> > > +int virtgpu_gem_prime_mmap(struct drm_gem_object *obj,
> > > + struct vm_area_struct *area)
> > > +{
> > > + WARN_ONCE(1, "not implemented");
> > > + return ENOSYS;
> >
> > This can get called by userspace, so please don't WARN here. Also missing
> > negate sign:
> >
> > return -ENOSYS;
>
> Hmm now checkpatch throws a warning at me:
>
> <quote>
> WARNING: ENOSYS means 'invalid syscall nr' and nothing else
> #12: FILE: drivers/gpu/drm/virtio/virtgpu_prime.c:70:
> + return -ENOSYS;
> </quote>
>
> I guess I should use something else then (here and elsewhere in the
> file)? Maybe -EINVAL? Other suggestions?
-ENODEV is what we occasionally pick. drm is fairly creative at errno
abuse, e.g. we already use -ENOENT to signal any kind of lookup failure in
ioctls (even if the fd itself is obviously there so not possible that the
fd isn't there).
Cheers, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply
* Re: [PATCH v7] pci: quirk to skip msi disable on shutdown
From: Bjorn Helgaas via Virtualization @ 2015-09-22 15:46 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Fam Zheng, linux-pci, linux-kernel, Ulrich Obergfell, Yinghai Lu,
Eric W. Biederman, virtualization
In-Reply-To: <20150922164107-mutt-send-email-mst@redhat.com>
On Tue, Sep 22, 2015 at 05:07:19PM +0300, Michael S. Tsirkin wrote:
> On Tue, Sep 22, 2015 at 07:36:40AM -0500, Bjorn Helgaas wrote:
> > On Tue, Sep 22, 2015 at 02:29:03PM +0300, Michael S. Tsirkin wrote:
> > > On Mon, Sep 21, 2015 at 05:10:43PM -0500, Bjorn Helgaas wrote:
> > > > Can you describe exactly what the device bug is? Apparently you're
> > > > saying that if we shut down MSI, it triggers the bug? And I guess
> > > > you're talking about a virtio device as implemented in qemu or other
> > > > hypervisors?
> > >
> > > Yes. Basically depending on an internal device state, disabling MSI
> > > sometimes wedges it. The most easy to debug effect is if it starts
> > > sending INTx interrupts, for which there's no handler currently.
> > > Full system reset always gets us out of the bad state.
> >
> > If disabling MSI causes the device to use INTx interrupts, that sounds
> > perfectly normal to me.
> >
> > If disabling MSI causes the device to hang, *that* sounds like a bug.
> > Since this is virtio, we should be able to figure out exactly where
> > that happens. Do you have a pointer to a virtio bug report, or even a
> > QEMU commit that fixes this virtio bug?
> >
> > I understand that even if there is a virtio fix in QEMU, we want a
> > solution that works even with an old QEMU that doesn't contain the
> > fix. But a pointer to a QEMU fix would really help understand and
> > document the Linux fix.
>
> I'm not sure we ever understood it completely.
>
> I think some of it has to do with the way the whole virtio 0
> device register layout changes when you enable/disable MSI. So should
> be ok when using the modern virtio 1 model since we fixed this thing.
If you know that:
- pci_device_shutdown() disables MSI,
- disabling MSI changes the virtio register layout, and
- the driver may touch the device after pci_device_shutdown(),
it seems like you might want a virtio shutdown method so you can
find out when the register layout changes. Changing the register
layout sounds completely broken, and dealing with it sounds racy, but
it sounds like a mess that should be handled by the driver.
> I was hoping that since disabling MSI in pci core is only useful as a
> work-around (for devices with a broken bus master enable - even though I
> don't think we know what these are exactly), a flag for not disabling it
> won't be held to such a high standard.
Well, I suppose you see the problem with adding workarounds on top of
workarounds, especially when we don't have a clear understanding of
why we need them. The "disable MSI on shutdown" code was added here:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d52877c7b1af
and there is no information in that patch about it being a workaround
for devices with broken bus master enable.
The cumulative effect of stuff like this is that it becomes impossible
to do any meaningful restructuring in the core without breaking some
corner case.
Bjorn
^ permalink raw reply
* Re: [PATCH v2 5/6] virtio-gpu: add basic prime support
From: Gerd Hoffmann @ 2015-09-22 15:24 UTC (permalink / raw)
To: Daniel Vetter
Cc: Dave Airlie, open list, open list:VIRTIO GPU DRIVER,
open list:VIRTIO GPU DRIVER
In-Reply-To: <20150922094027.GF3383@phenom.ffwll.local>
> > +int virtgpu_gem_prime_mmap(struct drm_gem_object *obj,
> > + struct vm_area_struct *area)
> > +{
> > + WARN_ONCE(1, "not implemented");
> > + return ENOSYS;
>
> This can get called by userspace, so please don't WARN here. Also missing
> negate sign:
>
> return -ENOSYS;
Hmm now checkpatch throws a warning at me:
<quote>
WARNING: ENOSYS means 'invalid syscall nr' and nothing else
#12: FILE: drivers/gpu/drm/virtio/virtgpu_prime.c:70:
+ return -ENOSYS;
</quote>
I guess I should use something else then (here and elsewhere in the
file)? Maybe -EINVAL? Other suggestions?
thanks,
Gerd
^ permalink raw reply
* Re: [PATCH v2 4/6] virtio-gpu: add 3d/virgl support
From: Gerd Hoffmann @ 2015-09-22 15:18 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: David Airlie, open list:ABI/API, open list,
open list:VIRTIO GPU DRIVER, open list:VIRTIO GPU DRIVER,
Dave Airlie
In-Reply-To: <20150921130110-mutt-send-email-mst@redhat.com>
Hi,
> > static unsigned int features[] = {
> > +#ifdef __LITTLE_ENDIAN
> > + VIRTIO_GPU_FEATURE_VIRGL,
> > +#endif
> > };
>
> Why is virgl LE specific? Just curious.
gallium command stream is native endian, and we only support little
endian guests on little endian hosts for that reason.
Supporting bigendian guests on bigendian hosts should be possible too
(with a VIRGl_BE feature flag), but with even the powerpc world moving
to little endian I see little reason to actually bother.
To be clear: this affects 3d acceleration only, 2d mode works fine on
every host/guest endian combination.
> > +#ifdef __LITTLE_ENDIAN
> > + if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_FEATURE_VIRGL))
> > + vgdev->has_virgl_3d = true;
> > +#endif
>
> You might be able to get by without this ifdef as
> VIRTIO_GPU_FEATURE_VIRGL won't be set without it.
Yes, right, this #ifdef can go away.
> > +int virtio_gpu_cmd_get_capset_info(struct virtio_gpu_device *vgdev, int idx)
> > +{
> > + struct virtio_gpu_get_capset_info *cmd_p;
> > + struct virtio_gpu_vbuffer *vbuf;
> > + void *resp_buf;
> > +
> > + resp_buf = kzalloc(sizeof(struct virtio_gpu_resp_capset_info),
> > + GFP_KERNEL);
> > + if (!resp_buf)
> > + return -ENOMEM;
> > +
> > + cmd_p = virtio_gpu_alloc_cmd_resp
> > + (vgdev, &virtio_gpu_cmd_get_capset_info_cb, &vbuf,
> > + sizeof(*cmd_p), sizeof(struct virtio_gpu_resp_capset_info),
> > + resp_buf);
> > + memset(cmd_p, 0, sizeof(*cmd_p));
> > +
> > + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_GET_CAPSET_INFO);
> > + cmd_p->capset_index = cpu_to_le32(idx);
> > + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> > + return 0;
> > +}
> > +
>
> I note that all callers ignore the recurn code from
> virtio_gpu_queue_ctrl_buffer.
>
> Is there a reason it can't fail?
It can only fail when vgdev->vqs_ready is false. Which can only happen
when we are about to unload the driver. In that case we probably don't
worry much about the failure ...
> Shouldn't it be made void then?
I guess we should ...
> > + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_CTX_CREATE);
> > + cmd_p->hdr.ctx_id = cpu_to_le32(id);
> > + cmd_p->nlen = cpu_to_le32(nlen);
> > + strncpy(cmd_p->debug_name, name, 63);
> > + cmd_p->debug_name[63] = 0;
>
> sizeof (cmd_p->debug_name) - 1 will be a but prettier.
Agree.
> > diff --git a/include/uapi/linux/virtio_gpu.h b/include/uapi/linux/virtio_gpu.h
> > index 478be52..7f4f9ce 100644
> > --- a/include/uapi/linux/virtio_gpu.h
> > +++ b/include/uapi/linux/virtio_gpu.h
> > @@ -40,6 +40,8 @@
> >
> > #include <linux/types.h>
> >
> > +#define VIRTIO_GPU_FEATURE_VIRGL 0
> > +
>
> I'd prefer it named VIRTIO_GPU_F_VIRGL for consistency with other
> devices.
Makes sense, I'll change it.
thanks,
Gerd
^ permalink raw reply
* Re: [PATCH v7] pci: quirk to skip msi disable on shutdown
From: Michael S. Tsirkin @ 2015-09-22 14:07 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Fam Zheng, linux-pci, linux-kernel, Ulrich Obergfell, Yinghai Lu,
Eric W. Biederman, virtualization
In-Reply-To: <20150922123640.GT25767@google.com>
On Tue, Sep 22, 2015 at 07:36:40AM -0500, Bjorn Helgaas wrote:
> On Tue, Sep 22, 2015 at 02:29:03PM +0300, Michael S. Tsirkin wrote:
> > On Mon, Sep 21, 2015 at 05:10:43PM -0500, Bjorn Helgaas wrote:
> > > On Mon, Sep 21, 2015 at 10:42:13PM +0300, Michael S. Tsirkin wrote:
> > > > On Mon, Sep 21, 2015 at 01:21:47PM -0500, Bjorn Helgaas wrote:
> > > > > On Sun, Sep 06, 2015 at 06:32:35PM +0300, Michael S. Tsirkin wrote:
> > > > > > On some hypervisors, virtio devices tend to generate spurious interrupts
> > > > > > when switching between MSI and non-MSI mode. Normally, either MSI or
> > > > > > non-MSI is used and all is well, but during shutdown, linux disables MSI
> > > > > > which then causes an "irq %d: nobody cared" message, with irq being
> > > > > > subsequently disabled.
> > > > >
> > > > > My understanding is:
> > > > >
> > > > > Linux disables MSI/MSI-X during device shutdown. If the device
> > > > > signals an interrupt after that, it may use INTx.
> > > > >
> > > > > This INTx interrupt is not necessarily spurious. Using INTx to signal an
> > > > > interrupt that occurs when MSI is disabled seems like reasonable behavior
> > > > > for any PCI device.
> > > > > And it doesn't seem related to switching between MSI and non-MSI mode.
> > > > > Yes, the INTx happens *after* disabling MSI, but it is not at all
> > > > > *because* we disabled MSI. So I wouldn't say "they generate spurious
> > > > > interrupts when switching between MSI and non-MSI."
> > > > >
> > > > > Why doesn't virtio-pci just register an INTx handler in addition to an MSI
> > > > > handler?
> > > >
> > > > The handler causes an expensive exit to the hypervisor,
> > > > and the INTx lines are shared with other devices.
> > >
> > > Do we care? Is this a performance path? I thought we were in a kexec
> > > shutdown path.
> >
> > Yes but the handler would always have to be registered, right?
>
> The pci_device_shutdown() path you're modifying calls drv->shutdown()
> immediately before disabling MSI, so I suppose you could register a
> handler in a virtio shutdown method.
I guess we could. Not sure what we'd do e.g. if that fails.
> > > > Seems silly to slow them down just so we can do something
> > > > that triggers the device bug. The bus master is disabled by that time,
> > > > if linux can just desist from touching MSI enable device won't
> > > > send either INTx (because MSI is on) or MSI
> > > > (because bus master is on) and all will be well.
> > >
> > > It would also be silly to put special-purpose code in the PCI core
> > > if there's a reasonable way to handle this in a driver.
> > >
> > > Can you describe exactly what the device bug is? Apparently you're
> > > saying that if we shut down MSI, it triggers the bug? And I guess
> > > you're talking about a virtio device as implemented in qemu or other
> > > hypervisors?
> >
> > Yes. Basically depending on an internal device state, disabling MSI
> > sometimes wedges it. The most easy to debug effect is if it starts
> > sending INTx interrupts, for which there's no handler currently.
> > Full system reset always gets us out of the bad state.
>
> If disabling MSI causes the device to use INTx interrupts, that sounds
> perfectly normal to me.
>
> If disabling MSI causes the device to hang, *that* sounds like a bug.
> Since this is virtio, we should be able to figure out exactly where
> that happens. Do you have a pointer to a virtio bug report, or even a
> QEMU commit that fixes this virtio bug?
>
> I understand that even if there is a virtio fix in QEMU, we want a
> solution that works even with an old QEMU that doesn't contain the
> fix. But a pointer to a QEMU fix would really help understand and
> document the Linux fix.
>
> Bjorn
I'm not sure we ever understood it completely.
I think some of it has to do with the way the whole virtio 0
device register layout changes when you enable/disable MSI. So should
be ok when using the modern virtio 1 model since we fixed this thing.
I was hoping that since disabling MSI in pci core is only useful as a
work-around (for devices with a broken bus master enable - even though I
don't think we know what these are exactly), a flag for not disabling it
won't be held to such a high standard.
--
MST
^ permalink raw reply
* Re: [PATCH v7] pci: quirk to skip msi disable on shutdown
From: Bjorn Helgaas via Virtualization @ 2015-09-22 12:36 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Fam Zheng, linux-pci, linux-kernel, Ulrich Obergfell, Yinghai Lu,
Eric W. Biederman, virtualization
In-Reply-To: <20150922131303-mutt-send-email-mst@redhat.com>
On Tue, Sep 22, 2015 at 02:29:03PM +0300, Michael S. Tsirkin wrote:
> On Mon, Sep 21, 2015 at 05:10:43PM -0500, Bjorn Helgaas wrote:
> > On Mon, Sep 21, 2015 at 10:42:13PM +0300, Michael S. Tsirkin wrote:
> > > On Mon, Sep 21, 2015 at 01:21:47PM -0500, Bjorn Helgaas wrote:
> > > > On Sun, Sep 06, 2015 at 06:32:35PM +0300, Michael S. Tsirkin wrote:
> > > > > On some hypervisors, virtio devices tend to generate spurious interrupts
> > > > > when switching between MSI and non-MSI mode. Normally, either MSI or
> > > > > non-MSI is used and all is well, but during shutdown, linux disables MSI
> > > > > which then causes an "irq %d: nobody cared" message, with irq being
> > > > > subsequently disabled.
> > > >
> > > > My understanding is:
> > > >
> > > > Linux disables MSI/MSI-X during device shutdown. If the device
> > > > signals an interrupt after that, it may use INTx.
> > > >
> > > > This INTx interrupt is not necessarily spurious. Using INTx to signal an
> > > > interrupt that occurs when MSI is disabled seems like reasonable behavior
> > > > for any PCI device.
> > > > And it doesn't seem related to switching between MSI and non-MSI mode.
> > > > Yes, the INTx happens *after* disabling MSI, but it is not at all
> > > > *because* we disabled MSI. So I wouldn't say "they generate spurious
> > > > interrupts when switching between MSI and non-MSI."
> > > >
> > > > Why doesn't virtio-pci just register an INTx handler in addition to an MSI
> > > > handler?
> > >
> > > The handler causes an expensive exit to the hypervisor,
> > > and the INTx lines are shared with other devices.
> >
> > Do we care? Is this a performance path? I thought we were in a kexec
> > shutdown path.
>
> Yes but the handler would always have to be registered, right?
The pci_device_shutdown() path you're modifying calls drv->shutdown()
immediately before disabling MSI, so I suppose you could register a
handler in a virtio shutdown method.
> > > Seems silly to slow them down just so we can do something
> > > that triggers the device bug. The bus master is disabled by that time,
> > > if linux can just desist from touching MSI enable device won't
> > > send either INTx (because MSI is on) or MSI
> > > (because bus master is on) and all will be well.
> >
> > It would also be silly to put special-purpose code in the PCI core
> > if there's a reasonable way to handle this in a driver.
> >
> > Can you describe exactly what the device bug is? Apparently you're
> > saying that if we shut down MSI, it triggers the bug? And I guess
> > you're talking about a virtio device as implemented in qemu or other
> > hypervisors?
>
> Yes. Basically depending on an internal device state, disabling MSI
> sometimes wedges it. The most easy to debug effect is if it starts
> sending INTx interrupts, for which there's no handler currently.
> Full system reset always gets us out of the bad state.
If disabling MSI causes the device to use INTx interrupts, that sounds
perfectly normal to me.
If disabling MSI causes the device to hang, *that* sounds like a bug.
Since this is virtio, we should be able to figure out exactly where
that happens. Do you have a pointer to a virtio bug report, or even a
QEMU commit that fixes this virtio bug?
I understand that even if there is a virtio fix in QEMU, we want a
solution that works even with an old QEMU that doesn't contain the
fix. But a pointer to a QEMU fix would really help understand and
document the Linux fix.
Bjorn
^ permalink raw reply
* Re: [PATCH v7] pci: quirk to skip msi disable on shutdown
From: Michael S. Tsirkin @ 2015-09-22 11:29 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Fam Zheng, linux-pci, linux-kernel, Ulrich Obergfell, Yinghai Lu,
Eric W. Biederman, virtualization
In-Reply-To: <20150921221043.GR25767@google.com>
On Mon, Sep 21, 2015 at 05:10:43PM -0500, Bjorn Helgaas wrote:
> On Mon, Sep 21, 2015 at 10:42:13PM +0300, Michael S. Tsirkin wrote:
> > On Mon, Sep 21, 2015 at 01:21:47PM -0500, Bjorn Helgaas wrote:
> > > On Sun, Sep 06, 2015 at 06:32:35PM +0300, Michael S. Tsirkin wrote:
> > > > On some hypervisors, virtio devices tend to generate spurious interrupts
> > > > when switching between MSI and non-MSI mode. Normally, either MSI or
> > > > non-MSI is used and all is well, but during shutdown, linux disables MSI
> > > > which then causes an "irq %d: nobody cared" message, with irq being
> > > > subsequently disabled.
> > >
> > > My understanding is:
> > >
> > > Linux disables MSI/MSI-X during device shutdown. If the device
> > > signals an interrupt after that, it may use INTx.
> > >
> > > This INTx interrupt is not necessarily spurious. Using INTx to signal an
> > > interrupt that occurs when MSI is disabled seems like reasonable behavior
> > > for any PCI device.
> > > And it doesn't seem related to switching between MSI and non-MSI mode.
> > > Yes, the INTx happens *after* disabling MSI, but it is not at all
> > > *because* we disabled MSI. So I wouldn't say "they generate spurious
> > > interrupts when switching between MSI and non-MSI."
> > >
> > > Why doesn't virtio-pci just register an INTx handler in addition to an MSI
> > > handler?
> >
> > The handler causes an expensive exit to the hypervisor,
> > and the INTx lines are shared with other devices.
>
> Do we care? Is this a performance path? I thought we were in a kexec
> shutdown path.
Yes but the handler would always have to be registered, right?
> > Seems silly to slow them down just so we can do something
> > that triggers the device bug. The bus master is disabled by that time,
> > if linux can just desist from touching MSI enable device won't
> > send either INTx (because MSI is on) or MSI
> > (because bus master is on) and all will be well.
>
> It would also be silly to put special-purpose code in the PCI core
> if there's a reasonable way to handle this in a driver.
>
> Can you describe exactly what the device bug is? Apparently you're
> saying that if we shut down MSI, it triggers the bug? And I guess
> you're talking about a virtio device as implemented in qemu or other
> hypervisors?
Yes. Basically depending on an internal device state, disabling MSI
sometimes wedges it. The most easy to debug effect is if it starts
sending INTx interrupts, for which there's no handler currently.
Full system reset always gets us out of the bad state.
> If we leave MSI enabled (as your patch does), then the device has MSI
> enabled and Bus Master disabled. I can see these possibilities:
>
> 1) the device never recognizes an interrupt condition
> 2) the device sets the pending bit but doesn't issue the MSI write,
> so the OS doesn't see the interrupt unless it polls for it
> 3) the device signals MSI and we still have an MSI handler
> registered, so we silently handle it
> 4) the device signals INTx
> You seem to suggest that if we leave MSI enabled (as your patch does),
> we're in case 1. But I doubt that disabling MSI causes the device to
> interrupt.
>
> Case 2 seems more likely to me: the device recognized an interrupt
> condition, e.g., an event occurred, and the OS simply doesn't see the
> interrupt because the device can't issue the MSI message.
>
> Case 3 does seem like it would be a device bug, because the device
> shouldn't do an MSI write when Bus Master is disabled. I don't see
> this case mentioned explicitly in the PCI spec, but PCIe r3.0 spec sec
> 7.5.1.1 does make it clear that disabling Bus Master also disables MSI
> messages.
>
> I don't know whether case 4 would be legal or not. But apparently it
> doesn't happen with the virtio device anyway, so it's not really a
> concern here.
>
> Bjorn
It's case 2. Except it doesn't actually set the pending
bit, because PCI spec has language like
"While a vector is masked, the function is prohibited from sending the
associated message, and the function must set the associated Pending bit"
but doesn't actually require to set pending if bus master enable
prevents sending MSI.
--
MST
^ permalink raw reply
* Re: [PATCH v2 5/6] virtio-gpu: add basic prime support
From: Daniel Vetter @ 2015-09-22 9:40 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Dave Airlie, open list:VIRTIO GPU DRIVER, open list,
open list:VIRTIO GPU DRIVER, dri-devel
In-Reply-To: <1442828417-6165-6-git-send-email-kraxel@redhat.com>
On Mon, Sep 21, 2015 at 11:40:16AM +0200, Gerd Hoffmann wrote:
> From: Dave Airlie <airlied@redhat.com>
>
> This is enough to enable DRI3.
>
> Signed-off-by: Dave Airlie <airlied@redhat.com>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> drivers/gpu/drm/virtio/Makefile | 2 +-
> drivers/gpu/drm/virtio/virtgpu_drv.c | 13 +++++-
> drivers/gpu/drm/virtio/virtgpu_drv.h | 12 ++++++
> drivers/gpu/drm/virtio/virtgpu_prime.c | 72 ++++++++++++++++++++++++++++++++++
> 4 files changed, 97 insertions(+), 2 deletions(-)
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_prime.c
>
> diff --git a/drivers/gpu/drm/virtio/Makefile b/drivers/gpu/drm/virtio/Makefile
> index da7bf19..3fb8eac 100644
> --- a/drivers/gpu/drm/virtio/Makefile
> +++ b/drivers/gpu/drm/virtio/Makefile
> @@ -7,6 +7,6 @@ ccflags-y := -Iinclude/drm
> virtio-gpu-y := virtgpu_drv.o virtgpu_kms.o virtgpu_drm_bus.o virtgpu_gem.o \
> virtgpu_fb.o virtgpu_display.o virtgpu_vq.o virtgpu_ttm.o \
> virtgpu_fence.o virtgpu_object.o virtgpu_debugfs.o virtgpu_plane.o \
> - virtgpu_ioctl.o
> + virtgpu_ioctl.o virtgpu_prime.o
>
> obj-$(CONFIG_DRM_VIRTIO_GPU) += virtio-gpu.o
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
> index 957e455..1245d09 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
> @@ -113,7 +113,7 @@ static const struct file_operations virtio_gpu_driver_fops = {
>
>
> static struct drm_driver driver = {
> - .driver_features = DRIVER_MODESET | DRIVER_GEM,
> + .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME,
> .set_busid = drm_virtio_set_busid,
> .load = virtio_gpu_driver_load,
> .unload = virtio_gpu_driver_unload,
> @@ -128,6 +128,17 @@ static struct drm_driver driver = {
> .debugfs_init = virtio_gpu_debugfs_init,
> .debugfs_cleanup = virtio_gpu_debugfs_takedown,
> #endif
> + .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
> + .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
> + .gem_prime_export = drm_gem_prime_export,
> + .gem_prime_import = drm_gem_prime_import,
> + .gem_prime_pin = virtgpu_gem_prime_pin,
> + .gem_prime_unpin = virtgpu_gem_prime_unpin,
> + .gem_prime_get_sg_table = virtgpu_gem_prime_get_sg_table,
> + .gem_prime_import_sg_table = virtgpu_gem_prime_import_sg_table,
> + .gem_prime_vmap = virtgpu_gem_prime_vmap,
> + .gem_prime_vunmap = virtgpu_gem_prime_vunmap,
> + .gem_prime_mmap = virtgpu_gem_prime_mmap,
>
> .gem_free_object = virtio_gpu_gem_free_object,
> .gem_open_object = virtio_gpu_gem_object_open,
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index 2719108..79f0abe 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -359,6 +359,18 @@ int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev,
> void virtio_gpu_object_free_sg_table(struct virtio_gpu_object *bo);
> int virtio_gpu_object_wait(struct virtio_gpu_object *bo, bool no_wait);
>
> +/* virtgpu_prime.c */
> +int virtgpu_gem_prime_pin(struct drm_gem_object *obj);
> +void virtgpu_gem_prime_unpin(struct drm_gem_object *obj);
> +struct sg_table *virtgpu_gem_prime_get_sg_table(struct drm_gem_object *obj);
> +struct drm_gem_object *virtgpu_gem_prime_import_sg_table(
> + struct drm_device *dev, struct dma_buf_attachment *attach,
> + struct sg_table *sgt);
> +void *virtgpu_gem_prime_vmap(struct drm_gem_object *obj);
> +void virtgpu_gem_prime_vunmap(struct drm_gem_object *obj, void *vaddr);
> +int virtgpu_gem_prime_mmap(struct drm_gem_object *obj,
> + struct vm_area_struct *vma);
> +
> static inline struct virtio_gpu_object*
> virtio_gpu_object_ref(struct virtio_gpu_object *bo)
> {
> diff --git a/drivers/gpu/drm/virtio/virtgpu_prime.c b/drivers/gpu/drm/virtio/virtgpu_prime.c
> new file mode 100644
> index 0000000..724c231
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_prime.c
> @@ -0,0 +1,72 @@
> +/*
> + * Copyright 2014 Canonical
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + *
> + * Authors: Andreas Pokorny
> + */
> +
> +#include "virtgpu_drv.h"
> +
> +/* Empty Implementations as there should not be any other driver for a virtual
> + * device that might share buffers with virtgpu */
> +
> +int virtgpu_gem_prime_pin(struct drm_gem_object *obj)
> +{
> + WARN_ONCE(1, "not implemented");
> + return -ENOSYS;
> +}
> +
> +void virtgpu_gem_prime_unpin(struct drm_gem_object *obj)
> +{
> + WARN_ONCE(1, "not implemented");
> +}
> +
> +
> +struct sg_table *virtgpu_gem_prime_get_sg_table(struct drm_gem_object *obj)
> +{
> + WARN_ONCE(1, "not implemented");
> + return ERR_PTR(-ENOSYS);
> +}
> +
> +struct drm_gem_object *virtgpu_gem_prime_import_sg_table(
> + struct drm_device *dev, struct dma_buf_attachment *attach,
> + struct sg_table *table)
> +{
> + WARN_ONCE(1, "not implemented");
> + return ERR_PTR(-ENOSYS);
> +}
> +
> +void *virtgpu_gem_prime_vmap(struct drm_gem_object *obj)
> +{
> + WARN_ONCE(1, "not implemented");
> + return ERR_PTR(-ENOSYS);
> +}
> +
> +void virtgpu_gem_prime_vunmap(struct drm_gem_object *obj, void *vaddr)
> +{
> + WARN_ONCE(1, "not implemented");
> +}
> +
> +int virtgpu_gem_prime_mmap(struct drm_gem_object *obj,
> + struct vm_area_struct *area)
> +{
> + WARN_ONCE(1, "not implemented");
> + return ENOSYS;
This can get called by userspace, so please don't WARN here. Also missing
negate sign:
return -ENOSYS;
Cheers, Daniel
> +}
> --
> 1.8.3.1
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox