* NVMe and SR-IOV, how to use together? @ 2017-09-15 16:38 Bogdan Rudas 2017-09-18 16:11 ` Christoph Hellwig 0 siblings, 1 reply; 7+ messages in thread From: Bogdan Rudas @ 2017-09-15 16:38 UTC (permalink / raw) Hi! Today I've received from servers vendor modern Samsung SSD 1725a MZPLL1T6HEHP for evaluation. I would like to use it as storage for virtual machines. Before I will start doing anything with a device I would like to re-check if I understand technology basics: To divide the device among virtual machines I have to delete default namespace and create a dedicated namespace for each VM. When I will have at least 2 namespace, every of them will be exposed in host operating system as virtual function of PCI device (i.e. I will see lot of Virtual function for Non-Volatile memory controller) Then I can do PCI passthru like I do it for any other SR-IOV PCI device. Finally I have to remove all namespaces and re-create the single one which will be exposed as single PCI device with no virtual functions. Could someone review statements above and point me to useful additional information? There are no user friendly information about SR-IOV on NVMe devices in Linux, so I have to disturb the development community. Thank you! -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email. ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-15 16:38 NVMe and SR-IOV, how to use together? Bogdan Rudas @ 2017-09-18 16:11 ` Christoph Hellwig 2017-09-18 18:20 ` Bogdan Rudas 0 siblings, 1 reply; 7+ messages in thread From: Christoph Hellwig @ 2017-09-18 16:11 UTC (permalink / raw) SR-IOV itself only allows you to create new virtual functions, you can look at any number of Linux howtos on creating them using the sysfs interface. As far as NVMe is concerned there are multiple levels of interaction that could be possible, all optional. First NVMe since version 1.1 supports subsystems with multiple controlles, and you can use namespace management (if support by the device) to assign namespaces to one or more controllers. On Linux you can use the nvme-cli tool for that. Since NVMe 1.3 there are virtualization extensions that also allow to assign resources like queue pairs to specific VMs. There is no support in Linux or nvme-cli for that at the moment. ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-18 16:11 ` Christoph Hellwig @ 2017-09-18 18:20 ` Bogdan Rudas 2017-09-18 18:41 ` Keith Busch 0 siblings, 1 reply; 7+ messages in thread From: Bogdan Rudas @ 2017-09-18 18:20 UTC (permalink / raw) I run kernel 4.4.27 which don't have a support for SR-IOV in NVMe driver, that's why I do not see a VFs in lspci output. Thank you. On Mon, Sep 18, 2017@7:11 PM, Christoph Hellwig <hch@infradead.org> wrote: > SR-IOV itself only allows you to create new virtual functions, > you can look at any number of Linux howtos on creating them using > the sysfs interface. > > As far as NVMe is concerned there are multiple levels of interaction > that could be possible, all optional. > > First NVMe since version 1.1 supports subsystems with multiple > controlles, and you can use namespace management (if support by the > device) to assign namespaces to one or more controllers. On Linux > you can use the nvme-cli tool for that. > > Since NVMe 1.3 there are virtualization extensions that also allow to > assign resources like queue pairs to specific VMs. There is no support > in Linux or nvme-cli for that at the moment. -- Bogdan Rudas Director of IT Europe Exadel Inc. http://www.exadel.com/ E-mail: brudas at exadel.com Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email. ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-18 18:20 ` Bogdan Rudas @ 2017-09-18 18:41 ` Keith Busch 2017-09-19 11:30 ` Bogdan Rudas 0 siblings, 1 reply; 7+ messages in thread From: Keith Busch @ 2017-09-18 18:41 UTC (permalink / raw) On Mon, Sep 18, 2017@09:20:16PM +0300, Bogdan Rudas wrote: > I run kernel 4.4.27 which don't have a support for SR-IOV in NVMe > driver, that's why I do not see a VFs in lspci output. The driver has nothing to do with what lspci will output. If you don't see the SR-IOV capability structure there, that means your drive doesn't support it. ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-18 18:41 ` Keith Busch @ 2017-09-19 11:30 ` Bogdan Rudas 2017-09-19 12:49 ` Gruher, Joseph R 0 siblings, 1 reply; 7+ messages in thread From: Bogdan Rudas @ 2017-09-19 11:30 UTC (permalink / raw) Hi! I've checked the device on 4.14.0 rc1 kernel. And see no SR-IOV files for SSD PCI device in /sys/devices/..../sriov_*, SR-IOV works for Intel I350 LOM at the same host. However manufactures brochure claims the SR-IOV support for this series. Should I report this as a bug in Linux nvme driver? Is so, what the information should I gather? Here is a brochure with SR-IOV mention: http://www.samsung.com/semiconductor/global/file/insight/2016/08/Samsung_PM1725a-1.pdf Here is SSD information: root at ubuntu-test:~# nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 S3HBNA0J600073 SAMSUNG MZPLL1T6HEHP-00003 1 1.60 TB / 1.60 TB 512 B + 0 B GPNA4B3Q On Mon, Sep 18, 2017@9:41 PM, Keith Busch <keith.busch@intel.com> wrote: > On Mon, Sep 18, 2017@09:20:16PM +0300, Bogdan Rudas wrote: >> I run kernel 4.4.27 which don't have a support for SR-IOV in NVMe >> driver, that's why I do not see a VFs in lspci output. > > The driver has nothing to do with what lspci will output. If you don't > see the SR-IOV capability structure there, that means your drive doesn't > support it. -- Bogdan Rudas Director of IT Europe Exadel Inc. http://www.exadel.com/ E-mail: brudas at exadel.com Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email. ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-19 11:30 ` Bogdan Rudas @ 2017-09-19 12:49 ` Gruher, Joseph R 2017-09-19 13:59 ` Bogdan Rudas 0 siblings, 1 reply; 7+ messages in thread From: Gruher, Joseph R @ 2017-09-19 12:49 UTC (permalink / raw) It would be interesting to confirm the device really supports SR-IOV. At launch the device claimed it on paper but we were told firmware support had not been implemented. Do you have any support from the manufacturer to confirm the device HW/FW really supports SR-IOV today? > On Sep 19, 2017,@1:31 PM, Bogdan Rudas <brudas@exadel.com> wrote: > > Hi! > > I've checked the device on 4.14.0 rc1 kernel. And see no SR-IOV files > for SSD PCI device in /sys/devices/..../sriov_*, SR-IOV works for > Intel I350 LOM at the same host. > However manufactures brochure claims the SR-IOV support for this > series. Should I report this as a bug in Linux nvme driver? Is so, > what the information should I gather? > > Here is a brochure with SR-IOV mention: > http://www.samsung.com/semiconductor/global/file/insight/2016/08/Samsung_PM1725a-1.pdf > > Here is SSD information: > root at ubuntu-test:~# nvme list > Node SN Model > Namespace Usage Format FW Rev > ---------------- -------------------- > ---------------------------------------- --------- > -------------------------- ---------------- -------- > /dev/nvme0n1 S3HBNA0J600073 SAMSUNG MZPLL1T6HEHP-00003 > 1 1.60 TB / 1.60 TB 512 B + 0 B GPNA4B3Q > >> On Mon, Sep 18, 2017@9:41 PM, Keith Busch <keith.busch@intel.com> wrote: >>> On Mon, Sep 18, 2017@09:20:16PM +0300, Bogdan Rudas wrote: >>> I run kernel 4.4.27 which don't have a support for SR-IOV in NVMe >>> driver, that's why I do not see a VFs in lspci output. >> >> The driver has nothing to do with what lspci will output. If you don't >> see the SR-IOV capability structure there, that means your drive doesn't >> support it. > > > > -- > Bogdan Rudas > Director of IT Europe > Exadel Inc. > http://www.exadel.com/ > E-mail: brudas at exadel.com > Skype ID: bogdan.rudas > > -- > > > CONFIDENTIALITY NOTICE: This email and files attached to it are > confidential. If you are not the intended recipient you are hereby notified > that using, copying, distributing or taking any action in reliance on the > contents of this information is strictly prohibited. If you have received > this email in error please notify the sender and delete this email. > > _______________________________________________ > Linux-nvme mailing list > Linux-nvme at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-nvme ^ permalink raw reply [flat|nested] 7+ messages in thread
* NVMe and SR-IOV, how to use together? 2017-09-19 12:49 ` Gruher, Joseph R @ 2017-09-19 13:59 ` Bogdan Rudas 0 siblings, 0 replies; 7+ messages in thread From: Bogdan Rudas @ 2017-09-19 13:59 UTC (permalink / raw) Sadly no, I've got the drive for compatibility test from local hardware supplier. Samsung support said "We are very sorry. We could support only our direct customer." so I'm going to return the drive to supplier. On Tue, Sep 19, 2017 at 3:49 PM, Gruher, Joseph R <joseph.r.gruher@intel.com> wrote: > It would be interesting to confirm the device really supports SR-IOV. At launch the device claimed it on paper but we were told firmware support had not been implemented. Do you have any support from the manufacturer to confirm the device HW/FW really supports SR-IOV today? > > >> On Sep 19, 2017,@1:31 PM, Bogdan Rudas <brudas@exadel.com> wrote: >> >> Hi! >> >> I've checked the device on 4.14.0 rc1 kernel. And see no SR-IOV files >> for SSD PCI device in /sys/devices/..../sriov_*, SR-IOV works for >> Intel I350 LOM at the same host. >> However manufactures brochure claims the SR-IOV support for this >> series. Should I report this as a bug in Linux nvme driver? Is so, >> what the information should I gather? >> >> Here is a brochure with SR-IOV mention: >> http://www.samsung.com/semiconductor/global/file/insight/2016/08/Samsung_PM1725a-1.pdf >> >> Here is SSD information: >> root at ubuntu-test:~# nvme list >> Node SN Model >> Namespace Usage Format FW Rev >> ---------------- -------------------- >> ---------------------------------------- --------- >> -------------------------- ---------------- -------- >> /dev/nvme0n1 S3HBNA0J600073 SAMSUNG MZPLL1T6HEHP-00003 >> 1 1.60 TB / 1.60 TB 512 B + 0 B GPNA4B3Q >> >>> On Mon, Sep 18, 2017@9:41 PM, Keith Busch <keith.busch@intel.com> wrote: >>>> On Mon, Sep 18, 2017@09:20:16PM +0300, Bogdan Rudas wrote: >>>> I run kernel 4.4.27 which don't have a support for SR-IOV in NVMe >>>> driver, that's why I do not see a VFs in lspci output. >>> >>> The driver has nothing to do with what lspci will output. If you don't >>> see the SR-IOV capability structure there, that means your drive doesn't >>> support it. >> >> >> >> -- >> Bogdan Rudas >> Director of IT Europe >> Exadel Inc. >> http://www.exadel.com/ >> E-mail: brudas at exadel.com >> Skype ID: bogdan.rudas >> >> -- >> >> >> CONFIDENTIALITY NOTICE: This email and files attached to it are >> confidential. If you are not the intended recipient you are hereby notified >> that using, copying, distributing or taking any action in reliance on the >> contents of this information is strictly prohibited. If you have received >> this email in error please notify the sender and delete this email. >> >> _______________________________________________ >> Linux-nvme mailing list >> Linux-nvme at lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-nvme -- Bogdan Rudas Director of IT Europe Exadel Inc. http://www.exadel.com/ E-mail: brudas at exadel.com Skype ID: bogdan.rudas -- CONFIDENTIALITY NOTICE: This email and files attached to it are confidential. If you are not the intended recipient you are hereby notified that using, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error please notify the sender and delete this email. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-09-19 13:59 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-09-15 16:38 NVMe and SR-IOV, how to use together? Bogdan Rudas 2017-09-18 16:11 ` Christoph Hellwig 2017-09-18 18:20 ` Bogdan Rudas 2017-09-18 18:41 ` Keith Busch 2017-09-19 11:30 ` Bogdan Rudas 2017-09-19 12:49 ` Gruher, Joseph R 2017-09-19 13:59 ` Bogdan Rudas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox