[Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]

Hi,

I have two questions,

First, I'm developing for Libiscsi and trying to work with virtio-scsi
dataplane or even virtio-blk dataplane and it doesn't works well.

I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
package.

my iscsi xml part is :

virtio-blk -

    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
        <host name='11.212.32.52' port='3260'/>
      </source>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
function='0x0'/>
    </disk>

virtio-scsi -

    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
        <host name='11.212.32.52' port='3260'/>
      </source>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
    </controller>

I tried to add this code outdise of <devices> in my xml :

  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.scsi=off'/>
  </qemu:commandline>
  <!-- config-wce=off is not needed in RHEL 6.4 -->
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.config-wce=off'/>
  </qemu:commandline>
  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.virtio-disk0.x-data-plane=on'/>
  </qemu:commandline>

and nothing happened, can you think of something i'm missing?

second thing, I'm trying to look for the code where QEMU allocate all
guest memory (2 GB) in my case.

Thanks,

Roy

[-- Attachment #2: Type: text/html, Size: 3429 bytes --]

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Fam Zheng]

On Wed, 01/20 22:12, Roy Shterman wrote:
> and nothing happened, can you think of something i'm missing?

The "x-data-plane=on" option used to be the temporary flag and has been removed
since last release of QEMU.  In the command line, the syntax to use dataplane
for virtio-blk/scsi is:

  -object iothread,id=iothread0 \
  -device virtio-{blk,scsi}-pci,iothread=iothread0,$your_usual_opts

In libvirt xml, you need to define "iothread" objects, and assign to devices:

    <domain>
        ...
      <iothreads>1</iothreads>
      ...
      <devices>
        ...
        <disk type='file' device='disk'>
          <driver name='qemu' type='qcow2' iothread='1'/>
          ...
        </disk>
      </devices>
    </domain>

I haven't tried virtio-scsi dataplane with libvirt on my own, and I don't know
if it is supported there (note: even though in QEMU it is configurable,
dataplane is not completely thread safe, so be careful).

> second thing, I'm trying to look for the code where QEMU allocate all
> guest memory (2 GB) in my case.

The guest ram is allocated by the machine in the most simple case:

(gdb) bt
#0  memory_region_init_ram (mr=0x55555658d9c0, owner=0x0, name=0x555555b05a6f "pc.ram", size=4294967296, errp=0x5555564aa770 <error_fatal>) at /home/fam/work/qemu/memory.c:1226
#1  0x0000555555720f2c in allocate_system_memory_nonnuma (mr=0x55555658d9c0, owner=0x0, name=0x555555b05a6f "pc.ram", ram_size=4294967296) at /home/fam/work/qemu/numa.c:434
#2  0x0000555555720f9b in memory_region_allocate_system_memory (mr=0x55555658d9c0, owner=0x0, name=0x555555b05a6f "pc.ram", ram_size=4294967296)
    at /home/fam/work/qemu/numa.c:447
#3  0x000055555579092d in pc_memory_init (pcms=0x55555652e280, system_memory=0x555556531280, rom_memory=0x55555658cfd0, ram_memory=0x7fffffffdce8, guest_info=0x55555658d570)
    at /home/fam/work/qemu/hw/i386/pc.c:1307
#4  0x0000555555792f90 in pc_init1 (machine=0x55555652e280, host_type=0x555555b064f0 "i440FX-pcihost", pci_type=0x555555b064e9 "i440FX")
    at /home/fam/work/qemu/hw/i386/pc_piix.c:164
#5  0x00005555557939c2 in pc_init_v2_6 (machine=0x55555652e280) at /home/fam/work/qemu/hw/i386/pc_piix.c:429
#6  0x000055555583cf24 in main (argc=3, argv=0x7fffffffe1c8, envp=0x7fffffffe1e8) at /home/fam/work/qemu/vl.c:4511

Fam

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Paolo Bonzini]

On 20/01/2016 21:12, Roy Shterman wrote:
> Hi,
> 
> I have two questions,
> 
> First, I'm developing for Libiscsi and trying to work with virtio-scsi
> dataplane or even virtio-blk dataplane and it doesn't works well.
> 
> I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
> package.
> 
> my iscsi xml part is : 
> 
> virtio-blk - 
> 
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='vdb' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
>     </disk>
> 
> virtio-scsi - 
> 
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sda' bus='scsi'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='scsi' index='0' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>     </controller>

There is now support for dataplane in libvirt.  See
https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and
then you can add an iothread='NN' (NN is a number) to the <driver
name='qemu' type='raw'/> element.

> second thing, I'm trying to look for the code where QEMU allocate all
> guest memory (2 GB) in my case.

Start at memory_allocate_system_memory; ultimately you'll reach
qemu_anon_ram_alloc which is basically an mmap.

paolo

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

[-- Attachment #1: Type: text/plain, Size: 5623 bytes --]

Hi,

First of all thank very much for your help,

Second, unfortunately data-plane didn't worked well, I tried to add threads
from the instructions you gave me.

Here is my full xml file, maybe you can help me to understand why it didn't
worked :

<domain type='kvm'>
  <name>gen-r-vrt-105-007-RH7.0x64</name>
  <uuid>8f79e97e-d452-4577-82bd-2ed903773026</uuid>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memtune>
    <hard_limit unit='KiB'>8388608</hard_limit>
  </memtune>
  <memoryBacking>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>

<emulator>/.autodirect/mtrswgwork/roysh/git/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/images/gen-r-vrt-105-007/gen-r-vrt-105-007.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
    </disk>
    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1/iser'>
        <host name='11.212.32.52' port='3260'/>
      </source>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
function='0x0'/>
    </disk>
    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1'>
        <host name='11.212.32.52' port='3260'/>
      </source>
      <target dev='sda' bus='scsi'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='network' device='lun'>
      <driver name='qemu' type='raw'/>
      <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-2/1'>
        <host name='11.212.32.52' port='3260'/>
      </source>
      <target dev='sdb' bus='scsi'/>
      <address type='drive' controller='1' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
    </controller>
    <controller type='scsi' index='1' model='virtio-scsi'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
function='0x0'/>
    </controller>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='00:50:56:28:69:07'/>
      <source bridge='br0:'/>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
    </memballoon>
  </devices>
</domain>

BTW, in RH 7.2 data-plane is default if one is choosing to work with virtio?

Thank you very much,
Roy

On Thu, Jan 21, 2016 at 11:01 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:

>
>
> On 20/01/2016 21:12, Roy Shterman wrote:
> > Hi,
> >
> > I have two questions,
> >
> > First, I'm developing for Libiscsi and trying to work with virtio-scsi
> > dataplane or even virtio-blk dataplane and it doesn't works well.
> >
> > I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
> > package.
> >
> > my iscsi xml part is :
> >
> > virtio-blk -
> >
> >     <disk type='network' device='lun'>
> >       <driver name='qemu' type='raw'/>
> >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
> >         <host name='11.212.32.52' port='3260'/>
> >       </source>
> >       <target dev='vdb' bus='virtio'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> > function='0x0'/>
> >     </disk>
> >
> > virtio-scsi -
> >
> >     <disk type='network' device='lun'>
> >       <driver name='qemu' type='raw'/>
> >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
> >         <host name='11.212.32.52' port='3260'/>
> >       </source>
> >       <target dev='sda' bus='scsi'/>
> >       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
> >     </disk>
> >     <controller type='scsi' index='0' model='virtio-scsi'>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> > function='0x0'/>
> >     </controller>
>
> There is now support for dataplane in libvirt.  See
> https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and
> then you can add an iothread='NN' (NN is a number) to the <driver
> name='qemu' type='raw'/> element.
>
> > second thing, I'm trying to look for the code where QEMU allocate all
> > guest memory (2 GB) in my case.
>
> Start at memory_allocate_system_memory; ultimately you'll reach
> qemu_anon_ram_alloc which is basically an mmap.
>
> paolo
>

[-- Attachment #2: Type: text/html, Size: 9900 bytes --]

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Fam Zheng]

On Wed, 01/27 21:03, Roy Shterman wrote:
> Hi,
> 
> First of all thank very much for your help,
> 
> Second, unfortunately data-plane didn't worked well, I tried to add threads
> from the instructions you gave me.
> 
> Here is my full xml file, maybe you can help me to understand why it didn't
> worked :

Did you specify iothread in the xml?

> 
> <domain type='kvm'>
>   <name>gen-r-vrt-105-007-RH7.0x64</name>
>   <uuid>8f79e97e-d452-4577-82bd-2ed903773026</uuid>
>   <memory unit='KiB'>2097152</memory>
>   <currentMemory unit='KiB'>2097152</currentMemory>
>   <memtune>
>     <hard_limit unit='KiB'>8388608</hard_limit>
>   </memtune>
>   <memoryBacking>
>     <locked/>
>   </memoryBacking>
>   <vcpu placement='static'>2</vcpu>
>   <os>
>     <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
>     <boot dev='hd'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
>   <clock offset='utc'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>restart</on_crash>

Add

    <iothreads>1</iothreads>

>   <devices>
> 
> <emulator>/.autodirect/mtrswgwork/roysh/git/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='qcow2' cache='none'/>

and change these lines to:

        <driver name='qemu' type='qcow2' cache='none' iothread='1'/>

Fam

>       <source file='/images/gen-r-vrt-105-007/gen-r-vrt-105-007.img'/>
>       <target dev='vda' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1/iser'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='vdb' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sda' bus='scsi'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-2/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sdb' bus='scsi'/>
>       <address type='drive' controller='1' bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='scsi' index='0' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>     </controller>
>     <controller type='scsi' index='1' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
>     </controller>
>     <controller type='usb' index='0'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x2'/>
>     </controller>
>     <controller type='pci' index='0' model='pci-root'/>
>     <interface type='bridge'>
>       <mac address='00:50:56:28:69:07'/>
>       <source bridge='br0:'/>
>       <model type='e1000'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>     <serial type='pty'>
>       <target port='0'/>
>     </serial>
>     <console type='pty'>
>       <target type='serial' port='0'/>
>     </console>
>     <input type='tablet' bus='usb'/>
>     <input type='mouse' bus='ps2'/>
>     <graphics type='vnc' port='-1' autoport='yes'/>
>     <video>
>       <model type='cirrus' vram='9216' heads='1'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
>     </memballoon>
>   </devices>
> </domain>
> 
> BTW, in RH 7.2 data-plane is default if one is choosing to work with virtio?
> 
> Thank you very much,
> Roy
> 
> On Thu, Jan 21, 2016 at 11:01 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> >
> >
> > On 20/01/2016 21:12, Roy Shterman wrote:
> > > Hi,
> > >
> > > I have two questions,
> > >
> > > First, I'm developing for Libiscsi and trying to work with virtio-scsi
> > > dataplane or even virtio-blk dataplane and it doesn't works well.
> > >
> > > I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
> > > package.
> > >
> > > my iscsi xml part is :
> > >
> > > virtio-blk -
> > >
> > >     <disk type='network' device='lun'>
> > >       <driver name='qemu' type='raw'/>
> > >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
> > >         <host name='11.212.32.52' port='3260'/>
> > >       </source>
> > >       <target dev='vdb' bus='virtio'/>
> > >       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> > > function='0x0'/>
> > >     </disk>
> > >
> > > virtio-scsi -
> > >
> > >     <disk type='network' device='lun'>
> > >       <driver name='qemu' type='raw'/>
> > >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
> > >         <host name='11.212.32.52' port='3260'/>
> > >       </source>
> > >       <target dev='sda' bus='scsi'/>
> > >       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
> > >     </disk>
> > >     <controller type='scsi' index='0' model='virtio-scsi'>
> > >       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> > > function='0x0'/>
> > >     </controller>
> >
> > There is now support for dataplane in libvirt.  See
> > https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and
> > then you can add an iothread='NN' (NN is a number) to the <driver
> > name='qemu' type='raw'/> element.
> >
> > > second thing, I'm trying to look for the code where QEMU allocate all
> > > guest memory (2 GB) in my case.
> >
> > Start at memory_allocate_system_memory; ultimately you'll reach
> > qemu_anon_ram_alloc which is basically an mmap.
> >
> > paolo
> >

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

[-- Attachment #1: Type: text/plain, Size: 10230 bytes --]

Tried it again,

Important to understand that after modifying and saving configuration of
xml with virsh edit $name_of_guest

when i reenter the xml i can't see the iothread configuration in there.
don't understand why.

Also, after add what you suggested I checked with "info qtree" command on
the guest looking for sign of data-plane is working,

this is the output for info qtree:

      dev: virtio-blk-pci, id "virtio-disk1"
        class = 0 (0x0)
        ioeventfd = true
        vectors = 2 (0x2)
        virtio-pci-bus-master-bug-migration = false
        disable-legacy = false
        disable-modern = true
        migrate-extra = false
        modern-pio-notify = false
        x-disable-pcie = false
        addr = 07.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class SCSI controller, addr 00:07.0, pci id 1af4:1001 (sub
1af4:0002)
        bar 0: i/o at 0xc0c0 [0xc0ff]
        bar 1: mem at 0xfebf3000 [0xfebf3fff]
        bus: virtio-bus
          type virtio-pci-bus
          dev: virtio-blk-device, id ""
            drive = "drive-virtio-disk1"
            logical_block_size = 512 (0x200)
            physical_block_size = 512 (0x200)
            min_io_size = 0 (0x0)
            opt_io_size = 0 (0x0)
            discard_granularity = 4294967295 (0xffffffff)
            cyls = 16383 (0x3fff)
            heads = 16 (0x10)
            secs = 63 (0x3f)
            serial = ""
            config-wce = true
            scsi = true
            request-merging = true
            indirect_desc = true
            event_idx = true
            notify_on_empty = true
            any_layout = false
      dev: virtio-blk-pci, id "virtio-disk0"
        class = 0 (0x0)
        ioeventfd = true
        vectors = 2 (0x2)
        virtio-pci-bus-master-bug-migration = false
        disable-legacy = false
        disable-modern = true
        migrate-extra = false
        modern-pio-notify = false
        x-disable-pcie = false
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class SCSI controller, addr 00:04.0, pci id 1af4:1001 (sub
1af4:0002)
        bar 0: i/o at 0xc040 [0xc07f]
        bar 1: mem at 0xfebf1000 [0xfebf1fff]
        bus: virtio-bus
          type virtio-pci-bus
          dev: virtio-blk-device, id ""
            drive = "drive-virtio-disk0"
            logical_block_size = 512 (0x200)
            physical_block_size = 512 (0x200)
            min_io_size = 0 (0x0)
            opt_io_size = 0 (0x0)
            discard_granularity = 4294967295 (0xffffffff)
            cyls = 16383 (0x3fff)
            heads = 16 (0x10)
            secs = 63 (0x3f)
            serial = ""
            config-wce = true
            scsi = false
            request-merging = true
            indirect_desc = true
            event_idx = true
            notify_on_empty = true
            any_layout = false
      dev: virtio-scsi-pci, id "scsi1"
        ioeventfd = true
        vectors = 4 (0x4)
        virtio-pci-bus-master-bug-migration = false
        disable-legacy = false
        disable-modern = true
        migrate-extra = false
        modern-pio-notify = false
        x-disable-pcie = false
        addr = 08.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class SCSI controller, addr 00:08.0, pci id 1af4:1004 (sub
1af4:0008)
        bar 0: i/o at 0xc100 [0xc13f]
        bar 1: mem at 0xfebf4000 [0xfebf4fff]
        bus: virtio-bus
          type virtio-pci-bus
          dev: virtio-scsi-device, id ""
            num_queues = 1 (0x1)
            max_sectors = 65535 (0xffff)
            cmd_per_lun = 128 (0x80)
            hotplug = true
            param_change = true
            indirect_desc = true
            event_idx = true
            notify_on_empty = true
            any_layout = true
            bus: scsi1.0
              type SCSI
              dev: scsi-block, id "scsi1-0-0-0"
                drive = "drive-scsi1-0-0-0"
                channel = 0 (0x0)
                scsi-id = 0 (0x0)
                lun = 0 (0x0)

Thnaks,
Roy



On Wed, Jan 27, 2016 at 9:03 PM, Roy Shterman <roy.shterman@gmail.com>
wrote:

> Hi,
>
> First of all thank very much for your help,
>
> Second, unfortunately data-plane didn't worked well, I tried to add
> threads from the instructions you gave me.
>
> Here is my full xml file, maybe you can help me to understand why it
> didn't worked :
>
> <domain type='kvm'>
>   <name>gen-r-vrt-105-007-RH7.0x64</name>
>   <uuid>8f79e97e-d452-4577-82bd-2ed903773026</uuid>
>   <memory unit='KiB'>2097152</memory>
>   <currentMemory unit='KiB'>2097152</currentMemory>
>   <memtune>
>     <hard_limit unit='KiB'>8388608</hard_limit>
>   </memtune>
>   <memoryBacking>
>     <locked/>
>   </memoryBacking>
>   <vcpu placement='static'>2</vcpu>
>   <os>
>     <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
>     <boot dev='hd'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
>   <clock offset='utc'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>restart</on_crash>
>   <devices>
>
> <emulator>/.autodirect/mtrswgwork/roysh/git/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='qcow2' cache='none'/>
>       <source file='/images/gen-r-vrt-105-007/gen-r-vrt-105-007.img'/>
>       <target dev='vda' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1/iser'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='vdb' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sda' bus='scsi'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='network' device='lun'>
>       <driver name='qemu' type='raw'/>
>       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-2/1'>
>         <host name='11.212.32.52' port='3260'/>
>       </source>
>       <target dev='sdb' bus='scsi'/>
>       <address type='drive' controller='1' bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='scsi' index='0' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0'/>
>     </controller>
>     <controller type='scsi' index='1' model='virtio-scsi'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
>     </controller>
>     <controller type='usb' index='0'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x2'/>
>     </controller>
>     <controller type='pci' index='0' model='pci-root'/>
>     <interface type='bridge'>
>       <mac address='00:50:56:28:69:07'/>
>       <source bridge='br0:'/>
>       <model type='e1000'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>     <serial type='pty'>
>       <target port='0'/>
>     </serial>
>     <console type='pty'>
>       <target type='serial' port='0'/>
>     </console>
>     <input type='tablet' bus='usb'/>
>     <input type='mouse' bus='ps2'/>
>     <graphics type='vnc' port='-1' autoport='yes'/>
>     <video>
>       <model type='cirrus' vram='9216' heads='1'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
>     </memballoon>
>   </devices>
> </domain>
>
> BTW, in RH 7.2 data-plane is default if one is choosing to work with
> virtio?
>
> Thank you very much,
> Roy
>
> On Thu, Jan 21, 2016 at 11:01 AM, Paolo Bonzini <pbonzini@redhat.com>
> wrote:
>
>>
>>
>> On 20/01/2016 21:12, Roy Shterman wrote:
>> > Hi,
>> >
>> > I have two questions,
>> >
>> > First, I'm developing for Libiscsi and trying to work with virtio-scsi
>> > dataplane or even virtio-blk dataplane and it doesn't works well.
>> >
>> > I'm working with latest qemu and latest Libiscsi in RedHat 7 libvirt
>> > package.
>> >
>> > my iscsi xml part is :
>> >
>> > virtio-blk -
>> >
>> >     <disk type='network' device='lun'>
>> >       <driver name='qemu' type='raw'/>
>> >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-3/1'>
>> >         <host name='11.212.32.52' port='3260'/>
>> >       </source>
>> >       <target dev='vdb' bus='virtio'/>
>> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>> > function='0x0'/>
>> >     </disk>
>> >
>> > virtio-scsi -
>> >
>> >     <disk type='network' device='lun'>
>> >       <driver name='qemu' type='raw'/>
>> >       <source protocol='iscsi' name='iqn.2001-04.com.r-dcs03-tgt-1/1>
>> >         <host name='11.212.32.52' port='3260'/>
>> >       </source>
>> >       <target dev='sda' bus='scsi'/>
>> >       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>> >     </disk>
>> >     <controller type='scsi' index='0' model='virtio-scsi'>
>> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> > function='0x0'/>
>> >     </controller>
>>
>> There is now support for dataplane in libvirt.  See
>> https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and
>> then you can add an iothread='NN' (NN is a number) to the <driver
>> name='qemu' type='raw'/> element.
>>
>> > second thing, I'm trying to look for the code where QEMU allocate all
>> > guest memory (2 GB) in my case.
>>
>> Start at memory_allocate_system_memory; ultimately you'll reach
>> qemu_anon_ram_alloc which is basically an mmap.
>>
>> paolo
>>
>
>

[-- Attachment #2: Type: text/html, Size: 16700 bytes --]

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Fam Zheng]

On Thu, 01/28 09:28, Roy Shterman wrote:
> Tried it again,
> 
> Important to understand that after modifying and saving configuration of
> xml with virsh edit $name_of_guest
> 
> when i reenter the xml i can't see the iothread configuration in there.
> don't understand why.
> 
> Also, after add what you suggested I checked with "info qtree" command on
> the guest looking for sign of data-plane is working,

Currently "info qtree" won't reflect that information. You'll need to use ps to
see if the command line includes "iothread" parts as I suggested in the first
reply.

Fam

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Paolo Bonzini]

On 28/01/2016 08:28, Roy Shterman wrote:
> 
> Important to understand that after modifying and saving configuration of
> xml with virsh edit $name_of_guest
> 
> when i reenter the xml i can't see the iothread configuration in there.
> don't understand why.
> 

I suspect that your libvirt is too old.

Paolo

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Daniel P. Berrange]

On Thu, Jan 28, 2016 at 09:46:26AM +0100, Paolo Bonzini wrote:
> 
> 
> On 28/01/2016 08:28, Roy Shterman wrote:
> > 
> > Important to understand that after modifying and saving configuration of
> > xml with virsh edit $name_of_guest
> > 
> > when i reenter the xml i can't see the iothread configuration in there.
> > don't understand why.
> > 
> 
> I suspect that your libvirt is too old.

iothread support requires libvirt 1.2.8 or newer

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

You were right, I worked with old libvirt, Thanks!

I have another question about QEMU memory allocation,

There is a possibility to put in libvirt :

<memoryBacking>


On Thu, Jan 28, 2016 at 10:53 AM, Daniel P. Berrange <berrange@redhat.com>
wrote:

> On Thu, Jan 28, 2016 at 09:46:26AM +0100, Paolo Bonzini wrote:
> >
> >
> > On 28/01/2016 08:28, Roy Shterman wrote:
> > >
> > > Important to understand that after modifying and saving configuration
> of
> > > xml with virsh edit $name_of_guest
> > >
> > > when i reenter the xml i can't see the iothread configuration in there.
> > > don't understand why.
> > >
> >
> > I suspect that your libvirt is too old.
>
> iothread support requires libvirt 1.2.8 or newer
>
> Regards,
> Daniel
> --
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org              -o-             http://virt-manager.org
> :|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc
> :|
>

[-- Attachment #2: Type: text/html, Size: 2290 bytes --]

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

Sorry about the spam...

You were right, I worked with old libvirt, Thanks!

I have another question about QEMU memory allocation,

There is a possibility to put in libvirt :

<memoryBacking>
  </locked>
</memoryBacking>

This is suppose to lock all the memory that has been allocated for the
guest into host's ram,

Is there any way to just upraise the maximum available pinnable-memory of
guest? without locking all 2G (in my guest)

because default maximum pinnable memory in QEMU is 60k.

Thank you all for your help,
Roy

On Sat, Jan 30, 2016 at 10:29 AM, Roy Shterman <roy.shterman@gmail.com>
wrote:

> You were right, I worked with old libvirt, Thanks!
>
> I have another question about QEMU memory allocation,
>
> There is a possibility to put in libvirt :
>
> <memoryBacking>
>
>
> On Thu, Jan 28, 2016 at 10:53 AM, Daniel P. Berrange <berrange@redhat.com>
> wrote:
>
>> On Thu, Jan 28, 2016 at 09:46:26AM +0100, Paolo Bonzini wrote:
>> >
>> >
>> > On 28/01/2016 08:28, Roy Shterman wrote:
>> > >
>> > > Important to understand that after modifying and saving configuration
>> of
>> > > xml with virsh edit $name_of_guest
>> > >
>> > > when i reenter the xml i can't see the iothread configuration in
>> there.
>> > > don't understand why.
>> > >
>> >
>> > I suspect that your libvirt is too old.
>>
>> iothread support requires libvirt 1.2.8 or newer
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com      -o-
>> http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org              -o-
>> http://virt-manager.org :|
>> |: http://autobuild.org       -o-
>> http://search.cpan.org/~danberr/ :|
>> |: http://entangle-photo.org       -o-
>> http://live.gnome.org/gtk-vnc :|
>>
>
>

[-- Attachment #2: Type: text/html, Size: 3928 bytes --]