* Re: [PATCH V5 0/4] Improve virtio-blk performance
From: Jens Axboe @ 2012-08-02 21:40 UTC (permalink / raw)
To: Asias He
Cc: kvm, Michael S. Tsirkin, linux-kernel, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-1-git-send-email-asias@redhat.com>
On 08/02/2012 08:25 AM, Asias He wrote:
> Hi folks,
>
> This version added REQ_FLUSH and REQ_FUA support as suggested by Christoph and
> rebased against latest linus's tree.
>
> Jens, could you please consider picking up the dependencies 1/4 and
> 2/4 in your tree. Thanks!
Pickedup, thanks for getting that done!
--
Jens Axboe
^ permalink raw reply
* Re: [GIT PULL] tcm_vhost: Initial merge of vhost level target fabric driver
From: Nicholas A. Bellinger @ 2012-08-02 20:23 UTC (permalink / raw)
To: Linus Torvalds
Cc: Jens Axboe, kvm-devel, linux-scsi, Michael S. Tsirkin, qemu-devel,
Zhi Yong Wu, target-devel, Anthony Liguori, Paolo Bonzini,
lf-virt, Christoph Hellwig, LKML
In-Reply-To: <1343697577.22538.661.camel@haakon2.linux-iscsi.org>
Hi Linus,
Ping on the initial tcm_vhost merge for-3.6..? I know it's been a
busier than usual merge window, but hopefully this one is still in your
PULL queue..
Otherwise if there is something else that you'd like to see different
from this PULL request, please let us know.
Thank you!
--nab
On Mon, 2012-07-30 at 18:19 -0700, Nicholas A. Bellinger wrote:
> Hi Linus,
>
> Here is the PULL request for the initial merge of tcm_vhost based on
> RFC-v5 code with MST's ACK appended to the initial merge commit.
> As promised, the commit is available from two different branches for you
> to consider merging as for-3.6 code.
>
> The 'for-next-merge' branch based on mainline commit 7409a6657ae using
> 3.5-rc2 code contains two duplicates of pre-merge vhost patch
> dependencies that have already been merged into mainline via net-next.
> This commit is also in the 07302012 -next patchset, and available here:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending.git for-next-merge
>
> Or the 'for-linus' branch containing an -rc0 head @ commit bdc0077af57:
>
> Merge tag 'scsi-misc' of git://git.kernel.org/../jejb/scsi)
>
> rebased up to the last commit in scsi-misc required for virtio-scsi
> client LLD scanning logic to function properly with tcm_vhost fabric
> ports, is available here:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending.git for-linus
>
> Both branches have gotten recent testing and have been running
> over-night small block random I/O tests connected to raw block flash
> backends. The same diffstat below will result from pulling either
> branch.
>
> Also, the incremental patch to address MST's last round of post-merge
> comments has been sent to the lists for feedback this afternoon. This
> will be included into the usual post -rc1 PULL via 3.6-rc-fixes, along
> with any other bits that end up changing post-merge.
>
> Please let us know if you have any concerns.
>
> Thank you!
>
> --nab
>
> Nicholas Bellinger (1):
> tcm_vhost: Initial merge for vhost level target fabric driver
>
> drivers/vhost/Kconfig | 3 +
> drivers/vhost/Kconfig.tcm | 6 +
> drivers/vhost/Makefile | 2 +
> drivers/vhost/tcm_vhost.c | 1628 +++++++++++++++++++++++++++++++++++++++++++++
> drivers/vhost/tcm_vhost.h | 101 +++
> 5 files changed, 1740 insertions(+), 0 deletions(-)
> create mode 100644 drivers/vhost/Kconfig.tcm
> create mode 100644 drivers/vhost/tcm_vhost.c
> create mode 100644 drivers/vhost/tcm_vhost.h
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply
* Re: [vmw_vmci 11/11] Apply the header code to make VMCI build
From: Sam Ravnborg @ 2012-08-02 20:22 UTC (permalink / raw)
To: Jan Engelhardt
Cc: pv-drivers, gregkh, linux-kernel, virtualization, vm-crosstalk,
Andrew Stiegmann (stieg), cschamp
In-Reply-To: <alpine.LNX.2.01.1208022145380.30631@frira.zrqbmnf.qr>
On Thu, Aug 02, 2012 at 09:50:02PM +0200, Jan Engelhardt wrote:
>
> On Friday 2012-07-27 12:34, Sam Ravnborg wrote:
> >> +#ifndef _VMCI_COMMONINT_H_
> >> +#define _VMCI_COMMONINT_H_
> >> +
> >> +#include <linux/printk.h>
> >> +#include <linux/vmw_vmci_defs.h>
> >
> >Use inverse chrismas tree here.
> >Longer include lines first, and soret alphabetically when
> >lines are of the same length.
>
> So that's where unreadable include lists come from.
> Depth-first lexicographically-sorted is a lot less hassle,
> especially when it comes to merging patches that each
> add one different include.
This is applied in many parts of the kernels and has some benefits:
- easy to spot duplicates
- clash is less likely when two commit adds includes
- easy to do so it looks the same across different files
Obviously <linux/*> comes before include <asm/*> as this is
separate blocks of includes.
net/ and arch/x86/ is two places where this is getting the norm,
and these are trendsetters for the rest of the kernel.
>
> >> +/*
> >> + * Utilility function that checks whether two entities are allowed
> >> + * to interact. If one of them is restricted, the other one must
> >> + * be trusted.
> >> + */
> >> +static inline bool vmci_deny_interaction(uint32_t partOne,
> >> + uint32_t partTwo)
> >
> >The kernel types are u32 not uint32_t - these types belongs in user-space.
>
> Not really. uint32_t is the C99 type for a 32-bit quantity, and I see
> absolutely zero reason not to use standardized things.
Found the following somewhere on the net:
On Mon, 29 Nov 2004, Paul Mackerras wrote:
>
> uint32_t is defined to be exactly 32 bits wide, so where's the problem
> in using it instead of __u32 in the headers that describe the
> user/kernel interface? (Ditto for uint{8,16,64}_t, of course.
Ok, this discussion has gone on for too long anyway, but let's make it
easier for everybody. The kernel uses u8/u16/u32 because:
- the kernel should not depend on, or pollute user-space naming.
YOU MUST NOT USE "uint32_t" when that may not be defined, and
user-space rules for when it is defined are arcane and totally
arbitrary.
...
See http://yarchive.net/comp/linux/kernel_headers.html for additional
rationale. (Second mail listed).
Sam
^ permalink raw reply
* Re: [vmw_vmci 11/11] Apply the header code to make VMCI build
From: Jan Engelhardt @ 2012-08-02 19:50 UTC (permalink / raw)
To: Sam Ravnborg
Cc: pv-drivers, gregkh, linux-kernel, virtualization, vm-crosstalk,
Andrew Stiegmann (stieg), cschamp
In-Reply-To: <20120727103455.GA4639@merkur.ravnborg.org>
On Friday 2012-07-27 12:34, Sam Ravnborg wrote:
>> +#ifndef _VMCI_COMMONINT_H_
>> +#define _VMCI_COMMONINT_H_
>> +
>> +#include <linux/printk.h>
>> +#include <linux/vmw_vmci_defs.h>
>
>Use inverse chrismas tree here.
>Longer include lines first, and soret alphabetically when
>lines are of the same length.
So that's where unreadable include lists come from.
Depth-first lexicographically-sorted is a lot less hassle,
especially when it comes to merging patches that each
add one different include.
>> +/*
>> + * Utilility function that checks whether two entities are allowed
>> + * to interact. If one of them is restricted, the other one must
>> + * be trusted.
>> + */
>> +static inline bool vmci_deny_interaction(uint32_t partOne,
>> + uint32_t partTwo)
>
>The kernel types are u32 not uint32_t - these types belongs in user-space.
Not really. uint32_t is the C99 type for a 32-bit quantity, and I see
absolutely zero reason not to use standardized things. The only
exception are header files visible to user space where __u32 should
be used for (obscure) reasons of avoiding naming clashes.
(Obscure because uint32_t is always supposed to be 32 bits.)
^ permalink raw reply
* CFP: 5th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2012 -- at IEEE/ACM Supercomputing 2012
From: zhaozhang @ 2012-08-02 15:56 UTC (permalink / raw)
To: virtualization
Call for Papers
--------------------------------------------------------------
-------------------------
The 5th Workshop on Many-Task Computing on Grids and
Supercomputers (MTAGS) 2012
http://datasys.cs.iit.edu/events/MTAGS12/
--------------------------------------------------------------
-------------------------
November 12th, 2012
Salt Lake City, Utah, USA
Co-located with with IEEE/ACM International Conference for
High Performance Computing, Networking, Storage and Analysis
(SC12)
==============================================================
=========================
The 5th workshop on Many-Task Computing on Grids and
Supercomputers (MTAGS) will
provide the scientific community a dedicated forum for
presenting new research,
development, and deployment efforts of large-scale many-task
computing (MTC)
applications on large scale clusters, Grids, Supercomputers,
and Cloud Computing
infrastructure. MTC, the theme of the workshop encompasses
loosely coupled applications,
which are generally composed of many tasks (both independent
and dependent tasks) to
achieve some larger application goal. This workshop will
cover challenges that can
hamper efficiency and utilization in running applications on
large-scale systems, such
as local resource manager scalability and granularity,
efficient utilization of raw
hardware, parallel file system contention and scalability,
data management, I/O
management, reliability at scale, and application scalability.
We welcome paper
submissions on all theoretical, simulations, and systems
topics related to MTC, but we
give special consideration to papers addressing petascale to
exascale challenges.
Papers will be peer-reviewed, and accepted papers will be
published in the workshop
proceedings as part of the IEEE digital library (pending
approval). The workshop will
be co-located with the IEEE/ACM Supercomputing 2012 Conference
in Salt Lake City Utah
on November 12th, 2012.
For more information, please see
http://datasys.cs.iit.edu/events/MTAGS12/. For more
information on past workshops, please see MTAGS11, MTAGS10,
MTAGS09, and MTAGS08. We also
ran a Special Issue on Many-Task Computing in the IEEE
Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011;
the proceedings can be found
online at
http://www.computer.org/portal/web/csdl/abs/trans/td/2011/06/t
td201106toc.htm.
We, the workshop organizers, also published a highly relevant
paper that defines
Many-Task Computing which was published in MTAGS08, titled
"Many-Task Computing for
Grids and Supercomputers"
(http://www.cs.iit.edu/~iraicu/research/publications/2008_MTAG
S08_MTC.pdf); we encourage
potential authors to read this paper.
Topics
--------------------------------------------------------------
-------------------------
We invite the submission of original work that is related to
the topics below. The
papers can be either short (4 pages) position papers, or long
(8 pages) research
papers. Topics of interest include (in the context of Many-
Task Computing):
* Compute Resource Management
* Scheduling
* Job execution frameworks
* Local resource manager extensions
* Performance evaluation of resource managers in use on
large scale systems
* Dynamic resource provisioning
* Techniques to manage many-core resources and/or GPUs
* Challenges and opportunities in running many-task
workloads on HPC systems
* Challenges and opportunities in running many-task
workloads on Cloud Computing
infrastructure
* Storage architectures and implementations
* Distributed file systems
* Parallel file systems
* Distributed meta-data management
* Content distribution systems for large data
* Data caching frameworks and techniques
* Data management within and across data centers
* Data-aware scheduling
* Data-intensive computing applications
* Eventual-consistency storage usage and management
* Programming models and tools
* Map-reduce and its generalizations
* Many-task computing middleware and applications
* Parallel programming frameworks
* Ensemble MPI techniques and frameworks
* Service-oriented science applications
* Large-Scale Workflow Systems
* Workflow system performance and scalability analysis
* Scalability of workflow systems
* Workflow infrastructure and e-Science middleware
* Programming Paradigms and Models
* Large-Scale Many-Task Applications
* High-throughput computing (HTC) applications
* Data-intensive applications
* Quasi-supercomputing applications, deployments, and
experiences
* Performance Evaluation
* Performance evaluation
* Real systems
* Simulations
* Reliability of large systems
* How MTC Addresses Challenges of Petascale and Exascale
Computing
* Concurrency & Programmability
* I/O & Memory
* Energy
* Resilience
* Heterogeneity
Paper Submission and Publication
--------------------------------------------------------------
-------------------------
Authors are invited to submit papers with unpublished,
original work of not more than 8
pages of double column text using single spaced 10 point size
on 8.5 x 11 inch pages,
as per IEEE 8.5 x 11 manuscript guidelines; document templates
can be found at
http://www.ieee.org/conferences_events/conferences/publishing/
templates.html. We are
also seeking position papers of no more than 4 pages in
length. The final 4/8 page
papers in PDF format must be submitted online at
https://cmt.research.microsoft.com/MTAGS2012/ before the
deadline. Papers will be
peer-reviewed, and accepted papers will be published in the
workshop proceedings as part
of the IEEE digital library (pending approval). Notifications
of the paper decisions
will be sent out by October 12th, 2012. Selected excellent
work may be eligible for
additional post-conference publication as journal articles or
book chapters, such as the
previous Special Issue on Many-Task Computing in the IEEE
Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011.
Submission implies the
willingness of at least one of the authors to register and
present the paper. For more
information, please http://datasys.cs.iit.edu/events/MTAGS12/,
or send email to
mtags12-chairs@datasys.cs.iit.edu.
Important Dates
--------------------------------------------------------------
-------------------------
* Abstract submission: September 10th, 2012 (11:59PM PST)
* Paper submission: September 17th, 2012 (11:59PM PST)
* Acceptance notification: October 12th, 2012
* Final papers due: November 7th, 2012
Committee Members
--------------------------------------------------------------
-------------------------
Workshop Chairs (mtags12-chairs@datasys.cs.iit.edu)
* Ioan Raicu, Illinois Institute of Technology & Argonne
National Laboratory
* Ian Foster, University of Chicago & Argonne National
Laboratory
* Yong Zhao, University of Electronic Science and Technology
of China
Steering Committee
* David Abramson, Monash University, Australia
* Jack Dongara, University of Tennessee, USA
* Geoffrey Fox, Indiana University, USA
* Manish Parashar, Rutgers University, USA
* Marc Snir, University of Illinois at Urbana Champaign, USA
* Xian-He Sun, Illinois Institute of Technology, USA
* Weimin Zheng, Tsinghua University, China
Publicity Chair (mtags12-publicity@datasys.cs.iit.edu)
* Zhao Zhang, University of Chicago, USA
Program Committee Chair (mtags12-pc-chair@datasys.cs.iit.edu)
* Justin Wozniak, Argonne National Laboratory, USA
Technical Committee
* Roger Barga, Microsoft Research, USA
* Kyle Chard, University of Chicago, USA
* Dennis Gannon, Microsoft Research, USA
* Florin Isaila, Universidad Carlos III de Madrid, Spain
* Hui Jin, Oracle Corporation, USA
* Daniel S. Katz, University of Chicago, USA
* Zhiling Lan, Illinois Institute of Technology, USA
* Mike Lang, Los Alamos National Laboratory, USA
* Gregor von Laszewski, Indiana University, USA
* Reagan Moore, University of North Carolina, Chappel Hill,
USA
* David O'Hallaron, Carnegie Mellon University, Intel Labs,
USA
* Marlon Pierce, Indiana University, USA
* Judy Qiu, Indiana University, USA
* Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory,
USA
* Kui Ren, SUNY Buffalo, USA
* Matei Ripeanu, University of British Columbia, Canada
* Wei Tang, Argonne National Laboratory, USA
* Valerie Taylor, Texas A&M, USA
* Ken Yocum, University of California, San Diego, USA
* Zhifeng Yun, Louisiana State University, USA
* Zhao Zhang, University of Chicago, USA
^ permalink raw reply
* DataCloud 2012 - The Third International Workshop on Data Intensive Computing in the Clouds -- Co-located with SC 12
From: Kyle Chard @ 2012-08-02 15:10 UTC (permalink / raw)
To: virtualization
[-- Attachment #1.1: Type: text/plain, Size: 4919 bytes --]
Call for Papers: The Third International Workshop on Data Intensive
Computing in the Clouds (DataCloud 2012)
November 11, Chicago IL, USA
(http://www.cse.buffalo.edu/faculty/tkosar/datacloud2012)
Co-Located with Super Computing 2012, November 10-16, Salt Lake City, UT,
USA (http://sc12.supercomputing.org/)
----------------------------------------------------------------------------
-----------------------------------------
IMPORTANT DATES
Abstract Submission: August 27, 2012
Paper Submission: September 10, 2012
Notification of Acceptance: October 8, 2012
Final Paper Due: October 29, 2012
Workshop: November 11, 2012
----------------------------------------------------------------------------
-----------------------------------------
OVERVIEW
Applications and experiments in all areas of science are becoming
increasingly complex and more demanding in terms of their computational and
data requirements. Some applications generate data volumes reaching hundreds
of terabytes and even petabytes. As scientific applications become more data
intensive, the management of data resources and dataflow between the storage
and compute resources is becoming the main bottleneck. Analyzing,
visualizing, and disseminating these large data sets has become a major
challenge and data intensive computing is now considered as the ''fourth
paradigm'' in scientific discovery after theoretical, experimental, and
computational science.
DataCloud 2012 will provide the scientific community a dedicated forum for
discussing new research, development, and deployment efforts in running
data-intensive computing workloads on Cloud Computing infrastructures. The
DataCloud 2012 workshop will focus on the use of cloud-based technologies to
meet the new data intensive scientific challenges that are not well served
by the current supercomputers, grids or compute-intensive clouds. We believe
the workshop will be an excellent place to help the community define the
current state, determine future goals, and present architectures and
services for future clouds supporting data intensive computing.
----------------------------------------------------------------------------
-----------------------------------------
WORKSHOP SCOPE
Topics of interest include, but are not limited to:
* Data-intensive cloud computing applications, characteristics,
challenges
* Case studies of data intensive computing in the clouds
* Performance evaluation of data clouds, data grids, and data centers
* Energy-efficient data cloud design and management
* Data placement, scheduling, and interoperability in the clouds
* Accountability, QoS, and SLAs
* Data privacy and protection in a public cloud environment
* Distributed file systems for clouds
* Data streaming and parallelization
* New programming models for data-intensive cloud computing
* Scalability issues in clouds
* Social computing and massively social gaming
* 3D Internet and implications
* Future research challenges in data-intensive cloud computing
----------------------------------------------------------------------------
-----------------------------------------
ORGANIZERS
* Tevfik Kosar, University at Buffalo
* Ioan Raicu, Illinois Institute of Technology & Argonne National
Laboratory, USA
* Roger Barga, Microsoft Research
STEERING COMMITTEE
* Ian Foster, University of Chicago & Argonne National Laboratory, USA
* Geoffrey Fox, Indiana University
* James Hamilton, Amazon Web Services
* Manish Parashar, Rutgers University & National Science Foundation
* Dan Reed, Microsoft Research
* Rich Wolski, University of California, Santa Barbara
PROGRAMME COMMITTEE
. Samer Al-Kiswany, University of British Columbia, Canada
. Abhishek Chandra, University of Minnesota
. Rong N. ChangI, BM Research
. Kyle Chard, University of Chicago & Argonne National Laboratory
. Terence Critchlow, Pacific Northwest National Laboratory
. Murat Demirbas, University at Buffalo
. Jaliya Ekanayake, Microsoft Research
. Geoffrey Fox, Indiana University
. Dennis Gannon, Microsoft Research
. Rob Gillen, Oak Ridge National Laboratory
. Maria Indrawan, Monash University, Australia
. Hui Jin, Oracle
. Dan Katz, National Science Foundation
. Steven Ko, University at Buffalo
. Reagan Moore, University of North Carolina
. Judy Qui, Indiana University
. Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory
. Kui Ren, University at Buffalo
. Yogesh Simmhan, University of Southern California, USA
. Borjo Sotomayor, University of Chicago
. Wei Tang, Argonne National Laboratory
. Bernard Traversat, Oracle
. Zhifeng Yun, Louisiana State University
. Ziming Zheng, Illinois Institute of Technology
[-- Attachment #1.2: Type: text/html, Size: 28586 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH V5 4/4] virtio-blk: Add REQ_FLUSH and REQ_FUA support to bio path
From: Asias He @ 2012-08-02 6:43 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, kvm, Michael S. Tsirkin, linux-kernel, virtualization,
Tejun Heo, Shaohua Li
In-Reply-To: <20120802062742.GA23573@lst.de>
On 08/02/2012 02:27 PM, Christoph Hellwig wrote:
> On Thu, Aug 02, 2012 at 02:25:56PM +0800, Asias He wrote:
>> We need to support both REQ_FLUSH and REQ_FUA for bio based path since
>> it does not get the sequencing of REQ_FUA into REQ_FLUSH that request
>> based drivers can request.
>>
>> REQ_FLUSH is emulated by:
>> 1. Send VIRTIO_BLK_T_FLUSH to device
>> 2. Wait until the flush is finished
>
> There is no need to wait for the flush to finish if the REQ_FLUSH
> request has no data payload.
>
> Even if it has a payload waiting is highly suboptimal and it should
> use a non-blocking sequencing like it is done in the request layer.
So, for REQ_FLUSH, what we need is that send out the VIRTIO_BLK_T_FLUSH
and not to wait.
>>
>> REQ_FUA is emulated by:
>> 1. Send the actual write
>> 2. Wait until the actual write is finished
>> 3. Send VIRTIO_BLK_T_FLUSH to device
>> 4. Wait until the flush is finished
>> 5. Signal the end of the write to upper layer
>
> The same comment about not blocking applies here as well.
We still need to wait until the actual write is finished here?
Like,
REQ_FUA is emulated by:
1. Send the actual write
2. Wait until the actual write is finished
3. Send VIRTIO_BLK_T_FLUSH to device
4. Signal the end of the write to upper layer
--
Asias
^ permalink raw reply
* Re: [PATCH V3 3/3] virtio-blk: Add bio-based IO path for virtio-blk
From: Asias He @ 2012-08-02 6:28 UTC (permalink / raw)
To: Christoph Hellwig
Cc: kvm, Michael S. Tsirkin, linux-kernel, virtualization,
Paolo Bonzini
In-Reply-To: <20120730134424.GC6041@lst.de>
On 07/30/2012 09:44 PM, Christoph Hellwig wrote:
> On Mon, Jul 30, 2012 at 09:31:06AM +0200, Paolo Bonzini wrote:
>> You only need to add REQ_FLUSH support. The virtio-blk protocol does
>> not support REQ_FUA, because there's no easy way to do it in userspace.
>
> A bio-based driver needs to handle both REQ_FLUSH and REQ_FUA as it does
> not get the sequencing of REQ_FUA into REQ_FLUSH that request based drivers
> can request. To what the REQ_FUA request gets translated is a different story.
I just sent out V5 to support both REQ_FLUSH AND REQ_FUA.
Thanks, Christoph!
--
Asias
^ permalink raw reply
* Re: [PATCH V5 4/4] virtio-blk: Add REQ_FLUSH and REQ_FUA support to bio path
From: Christoph Hellwig @ 2012-08-02 6:27 UTC (permalink / raw)
To: Asias He
Cc: Jens Axboe, kvm, Michael S. Tsirkin, linux-kernel, virtualization,
Tejun Heo, Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-5-git-send-email-asias@redhat.com>
On Thu, Aug 02, 2012 at 02:25:56PM +0800, Asias He wrote:
> We need to support both REQ_FLUSH and REQ_FUA for bio based path since
> it does not get the sequencing of REQ_FUA into REQ_FLUSH that request
> based drivers can request.
>
> REQ_FLUSH is emulated by:
> 1. Send VIRTIO_BLK_T_FLUSH to device
> 2. Wait until the flush is finished
There is no need to wait for the flush to finish if the REQ_FLUSH
request has no data payload.
Even if it has a payload waiting is highly suboptimal and it should
use a non-blocking sequencing like it is done in the request layer.
>
> REQ_FUA is emulated by:
> 1. Send the actual write
> 2. Wait until the actual write is finished
> 3. Send VIRTIO_BLK_T_FLUSH to device
> 4. Wait until the flush is finished
> 5. Signal the end of the write to upper layer
The same comment about not blocking applies here as well.
^ permalink raw reply
* [PATCH V5 4/4] virtio-blk: Add REQ_FLUSH and REQ_FUA support to bio path
From: Asias He @ 2012-08-02 6:25 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, kvm, Michael S. Tsirkin, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-1-git-send-email-asias@redhat.com>
We need to support both REQ_FLUSH and REQ_FUA for bio based path since
it does not get the sequencing of REQ_FUA into REQ_FLUSH that request
based drivers can request.
REQ_FLUSH is emulated by:
1. Send VIRTIO_BLK_T_FLUSH to device
2. Wait until the flush is finished
REQ_FUA is emulated by:
1. Send the actual write
2. Wait until the actual write is finished
3. Send VIRTIO_BLK_T_FLUSH to device
4. Wait until the flush is finished
5. Signal the end of the write to upper layer
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Shaohua Li <shli@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org
Signed-off-by: Asias He <asias@redhat.com>
---
drivers/block/virtio_blk.c | 104 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 91 insertions(+), 13 deletions(-)
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index 95cfeed..9ebaea7 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -54,6 +54,8 @@ struct virtio_blk
struct virtblk_req
{
+ struct completion *flush_done;
+ struct completion *bio_done;
struct request *req;
struct bio *bio;
struct virtio_blk_outhdr out_hdr;
@@ -95,14 +97,25 @@ static inline void virtblk_request_done(struct virtio_blk *vblk,
static inline void virtblk_bio_done(struct virtio_blk *vblk,
struct virtblk_req *vbr)
{
+ if (unlikely(vbr->bio_done)) {
+ complete(vbr->bio_done);
+ return;
+ }
bio_endio(vbr->bio, virtblk_result(vbr));
mempool_free(vbr, vblk->pool);
}
+static inline void virtblk_flush_done(struct virtio_blk *vblk,
+ struct virtblk_req *vbr)
+{
+ complete(vbr->flush_done);
+ mempool_free(vbr, vblk->pool);
+}
+
static void virtblk_done(struct virtqueue *vq)
{
+ unsigned long flush_done = 0, bio_done = 0, req_done = 0;
struct virtio_blk *vblk = vq->vdev->priv;
- unsigned long bio_done = 0, req_done = 0;
struct virtblk_req *vbr;
unsigned long flags;
unsigned int len;
@@ -112,9 +125,12 @@ static void virtblk_done(struct virtqueue *vq)
if (vbr->bio) {
virtblk_bio_done(vblk, vbr);
bio_done++;
- } else {
+ } else if (vbr->req) {
virtblk_request_done(vblk, vbr);
req_done++;
+ } else if (vbr->flush_done) {
+ virtblk_flush_done(vblk, vbr);
+ flush_done++;
}
}
/* In case queue is stopped waiting for more buffers. */
@@ -122,7 +138,7 @@ static void virtblk_done(struct virtqueue *vq)
blk_start_queue(vblk->disk->queue);
spin_unlock_irqrestore(vblk->disk->queue->queue_lock, flags);
- if (bio_done)
+ if (bio_done || flush_done)
wake_up(&vblk->queue_wait);
}
@@ -269,14 +285,65 @@ static void virtblk_add_buf_wait(struct virtio_blk *vblk,
finish_wait(&vblk->queue_wait, &wait);
}
+static inline void virtblk_add_req(struct virtio_blk *vblk,
+ struct virtblk_req *vbr,
+ unsigned int out, unsigned int in)
+{
+ spin_lock_irq(vblk->disk->queue->queue_lock);
+ if (unlikely(virtqueue_add_buf(vblk->vq, vbr->sg, out, in, vbr,
+ GFP_ATOMIC) < 0)) {
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+ virtblk_add_buf_wait(vblk, vbr, out, in);
+ return;
+ }
+ virtqueue_kick(vblk->vq);
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+}
+
+static int virtblk_flush(struct virtio_blk *vblk)
+{
+ DECLARE_COMPLETION_ONSTACK(done);
+ unsigned int out = 0, in = 0;
+ struct virtblk_req *vbr;
+
+ vbr = virtblk_alloc_req(vblk, GFP_NOIO);
+ if (!vbr)
+ return -ENOMEM;
+
+ vbr->flush_done = &done;
+ vbr->bio = NULL;
+ vbr->req = NULL;
+ vbr->out_hdr.type = VIRTIO_BLK_T_FLUSH;
+ vbr->out_hdr.sector = 0;
+ vbr->out_hdr.ioprio = 0;
+ sg_set_buf(&vbr->sg[out++], &vbr->out_hdr, sizeof(vbr->out_hdr));
+ sg_set_buf(&vbr->sg[out + in++], &vbr->status, sizeof(vbr->status));
+
+ virtblk_add_req(vblk, vbr, out, in);
+
+ wait_for_completion(&done);
+
+ return 0;
+}
+
static void virtblk_make_request(struct request_queue *q, struct bio *bio)
{
+ bool req_flush = false, req_fua = false;
struct virtio_blk *vblk = q->queuedata;
unsigned int num, out = 0, in = 0;
+ DECLARE_COMPLETION_ONSTACK(done);
struct virtblk_req *vbr;
BUG_ON(bio->bi_phys_segments + 2 > vblk->sg_elems);
- BUG_ON(bio->bi_rw & (REQ_FLUSH | REQ_FUA));
+
+ if (bio->bi_rw & REQ_FLUSH)
+ req_flush = true;
+ if (bio->bi_rw & REQ_FUA)
+ req_fua = true;
+
+ /* Execute a flush & wait until it finishes */
+ if (unlikely(req_flush))
+ virtblk_flush(vblk);
vbr = virtblk_alloc_req(vblk, GFP_NOIO);
if (!vbr) {
@@ -290,6 +357,11 @@ static void virtblk_make_request(struct request_queue *q, struct bio *bio)
vbr->out_hdr.sector = bio->bi_sector;
vbr->out_hdr.ioprio = bio_prio(bio);
+ if (unlikely(req_fua))
+ vbr->bio_done = &done;
+ else
+ vbr->bio_done = NULL;
+
sg_set_buf(&vbr->sg[out++], &vbr->out_hdr, sizeof(vbr->out_hdr));
num = blk_bio_map_sg(q, bio, vbr->sg + out);
@@ -307,15 +379,21 @@ static void virtblk_make_request(struct request_queue *q, struct bio *bio)
}
}
- spin_lock_irq(vblk->disk->queue->queue_lock);
- if (unlikely(virtqueue_add_buf(vblk->vq, vbr->sg, out, in, vbr,
- GFP_ATOMIC) < 0)) {
- spin_unlock_irq(vblk->disk->queue->queue_lock);
- virtblk_add_buf_wait(vblk, vbr, out, in);
- return;
+ virtblk_add_req(vblk, vbr, out, in);
+
+ if (unlikely(req_fua)) {
+ /*
+ * We emulate the REQ_FUA here:
+ *
+ * 1. Wait until the bio is finished
+ * 2. Execute a flush & wait until it finishes
+ * 3. Signal the end of the bio & free the vbr
+ */
+ wait_for_completion(vbr->bio_done);
+ virtblk_flush(vblk);
+ bio_endio(vbr->bio, virtblk_result(vbr));
+ mempool_free(vbr, vblk->pool);
}
- virtqueue_kick(vblk->vq);
- spin_unlock_irq(vblk->disk->queue->queue_lock);
}
/* return id (s/n) string for *disk to *id_str
@@ -529,7 +607,7 @@ static void virtblk_update_cache_mode(struct virtio_device *vdev)
u8 writeback = virtblk_get_cache_mode(vdev);
struct virtio_blk *vblk = vdev->priv;
- if (writeback && !use_bio)
+ if (writeback)
blk_queue_flush(vblk->disk->queue, REQ_FLUSH);
else
blk_queue_flush(vblk->disk->queue, 0);
--
1.7.11.2
^ permalink raw reply related
* [PATCH V5 3/4] virtio-blk: Add bio-based IO path for virtio-blk
From: Asias He @ 2012-08-02 6:25 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, kvm, Michael S. Tsirkin, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-1-git-send-email-asias@redhat.com>
This patch introduces bio-based IO path for virtio-blk.
Compared to request-based IO path, bio-based IO path uses driver
provided ->make_request_fn() method to bypasses the IO scheduler. It
handles the bio to device directly without allocating a request in block
layer. This reduces the IO path in guest kernel to achieve high IOPS
and lower latency. The downside is that guest can not use the IO
scheduler to merge and sort requests. However, this is not a big problem
if the backend disk in host side uses faster disk device.
When the bio-based IO path is not enabled, virtio-blk still uses the
original request-based IO path, no performance difference is observed.
Performance evaluation:
-----------------------------
1) Fio test is performed in a 8 vcpu guest with ramdisk based guest using
kvm tool.
Short version:
With bio-based IO path, sequential read/write, random read/write
IOPS boost : 28%, 24%, 21%, 16%
Latency improvement: 32%, 17%, 21%, 16%
Long version:
With bio-based IO path:
seq-read : io=2048.0MB, bw=116996KB/s, iops=233991 , runt= 17925msec
seq-write : io=2048.0MB, bw=100829KB/s, iops=201658 , runt= 20799msec
rand-read : io=3095.7MB, bw=112134KB/s, iops=224268 , runt= 28269msec
rand-write: io=3095.7MB, bw=96198KB/s, iops=192396 , runt= 32952msec
clat (usec): min=0 , max=2631.6K, avg=58716.99, stdev=191377.30
clat (usec): min=0 , max=1753.2K, avg=66423.25, stdev=81774.35
clat (usec): min=0 , max=2915.5K, avg=61685.70, stdev=120598.39
clat (usec): min=0 , max=1933.4K, avg=76935.12, stdev=96603.45
cpu : usr=74.08%, sys=703.84%, ctx=29661403, majf=21354, minf=22460954
cpu : usr=70.92%, sys=702.81%, ctx=77219828, majf=13980, minf=27713137
cpu : usr=72.23%, sys=695.37%, ctx=88081059, majf=18475, minf=28177648
cpu : usr=69.69%, sys=654.13%, ctx=145476035, majf=15867, minf=26176375
With request-based IO path:
seq-read : io=2048.0MB, bw=91074KB/s, iops=182147 , runt= 23027msec
seq-write : io=2048.0MB, bw=80725KB/s, iops=161449 , runt= 25979msec
rand-read : io=3095.7MB, bw=92106KB/s, iops=184211 , runt= 34416msec
rand-write: io=3095.7MB, bw=82815KB/s, iops=165630 , runt= 38277msec
clat (usec): min=0 , max=1932.4K, avg=77824.17, stdev=170339.49
clat (usec): min=0 , max=2510.2K, avg=78023.96, stdev=146949.15
clat (usec): min=0 , max=3037.2K, avg=74746.53, stdev=128498.27
clat (usec): min=0 , max=1363.4K, avg=89830.75, stdev=114279.68
cpu : usr=53.28%, sys=724.19%, ctx=37988895, majf=17531, minf=23577622
cpu : usr=49.03%, sys=633.20%, ctx=205935380, majf=18197, minf=27288959
cpu : usr=55.78%, sys=722.40%, ctx=101525058, majf=19273, minf=28067082
cpu : usr=56.55%, sys=690.83%, ctx=228205022, majf=18039, minf=26551985
2) Fio test is performed in a 8 vcpu guest with Fusion-IO based guest using
kvm tool.
Short version:
With bio-based IO path, sequential read/write, random read/write
IOPS boost : 11%, 11%, 13%, 10%
Latency improvement: 10%, 10%, 12%, 10%
Long Version:
With bio-based IO path:
read : io=2048.0MB, bw=58920KB/s, iops=117840 , runt= 35593msec
write: io=2048.0MB, bw=64308KB/s, iops=128616 , runt= 32611msec
read : io=3095.7MB, bw=59633KB/s, iops=119266 , runt= 53157msec
write: io=3095.7MB, bw=62993KB/s, iops=125985 , runt= 50322msec
clat (usec): min=0 , max=1284.3K, avg=128109.01, stdev=71513.29
clat (usec): min=94 , max=962339 , avg=116832.95, stdev=65836.80
clat (usec): min=0 , max=1846.6K, avg=128509.99, stdev=89575.07
clat (usec): min=0 , max=2256.4K, avg=121361.84, stdev=82747.25
cpu : usr=56.79%, sys=421.70%, ctx=147335118, majf=21080, minf=19852517
cpu : usr=61.81%, sys=455.53%, ctx=143269950, majf=16027, minf=24800604
cpu : usr=63.10%, sys=455.38%, ctx=178373538, majf=16958, minf=24822612
cpu : usr=62.04%, sys=453.58%, ctx=226902362, majf=16089, minf=23278105
With request-based IO path:
read : io=2048.0MB, bw=52896KB/s, iops=105791 , runt= 39647msec
write: io=2048.0MB, bw=57856KB/s, iops=115711 , runt= 36248msec
read : io=3095.7MB, bw=52387KB/s, iops=104773 , runt= 60510msec
write: io=3095.7MB, bw=57310KB/s, iops=114619 , runt= 55312msec
clat (usec): min=0 , max=1532.6K, avg=142085.62, stdev=109196.84
clat (usec): min=0 , max=1487.4K, avg=129110.71, stdev=114973.64
clat (usec): min=0 , max=1388.6K, avg=145049.22, stdev=107232.55
clat (usec): min=0 , max=1465.9K, avg=133585.67, stdev=110322.95
cpu : usr=44.08%, sys=590.71%, ctx=451812322, majf=14841, minf=17648641
cpu : usr=48.73%, sys=610.78%, ctx=418953997, majf=22164, minf=26850689
cpu : usr=45.58%, sys=581.16%, ctx=714079216, majf=21497, minf=22558223
cpu : usr=48.40%, sys=599.65%, ctx=656089423, majf=16393, minf=23824409
How to use:
-----------------------------
Add 'virtio_blk.use_bio=1' to kernel cmdline or 'modprobe virtio_blk
use_bio=1' to enable ->make_request_fn() based I/O path.
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Shaohua Li <shli@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Minchan Kim <minchan.kim@gmail.com>
Signed-off-by: Asias He <asias@redhat.com>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/block/virtio_blk.c | 203 ++++++++++++++++++++++++++++++++++++---------
1 file changed, 163 insertions(+), 40 deletions(-)
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index c0bbeb4..95cfeed 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -14,6 +14,9 @@
#define PART_BITS 4
+static bool use_bio;
+module_param(use_bio, bool, S_IRUGO);
+
static int major;
static DEFINE_IDA(vd_index_ida);
@@ -23,6 +26,7 @@ struct virtio_blk
{
struct virtio_device *vdev;
struct virtqueue *vq;
+ wait_queue_head_t queue_wait;
/* The disk structure for the kernel. */
struct gendisk *disk;
@@ -51,53 +55,87 @@ struct virtio_blk
struct virtblk_req
{
struct request *req;
+ struct bio *bio;
struct virtio_blk_outhdr out_hdr;
struct virtio_scsi_inhdr in_hdr;
u8 status;
+ struct scatterlist sg[];
};
-static void blk_done(struct virtqueue *vq)
+static inline int virtblk_result(struct virtblk_req *vbr)
+{
+ switch (vbr->status) {
+ case VIRTIO_BLK_S_OK:
+ return 0;
+ case VIRTIO_BLK_S_UNSUPP:
+ return -ENOTTY;
+ default:
+ return -EIO;
+ }
+}
+
+static inline void virtblk_request_done(struct virtio_blk *vblk,
+ struct virtblk_req *vbr)
+{
+ struct request *req = vbr->req;
+ int error = virtblk_result(vbr);
+
+ if (req->cmd_type == REQ_TYPE_BLOCK_PC) {
+ req->resid_len = vbr->in_hdr.residual;
+ req->sense_len = vbr->in_hdr.sense_len;
+ req->errors = vbr->in_hdr.errors;
+ } else if (req->cmd_type == REQ_TYPE_SPECIAL) {
+ req->errors = (error != 0);
+ }
+
+ __blk_end_request_all(req, error);
+ mempool_free(vbr, vblk->pool);
+}
+
+static inline void virtblk_bio_done(struct virtio_blk *vblk,
+ struct virtblk_req *vbr)
+{
+ bio_endio(vbr->bio, virtblk_result(vbr));
+ mempool_free(vbr, vblk->pool);
+}
+
+static void virtblk_done(struct virtqueue *vq)
{
struct virtio_blk *vblk = vq->vdev->priv;
+ unsigned long bio_done = 0, req_done = 0;
struct virtblk_req *vbr;
- unsigned int len;
unsigned long flags;
+ unsigned int len;
spin_lock_irqsave(vblk->disk->queue->queue_lock, flags);
while ((vbr = virtqueue_get_buf(vblk->vq, &len)) != NULL) {
- int error;
-
- switch (vbr->status) {
- case VIRTIO_BLK_S_OK:
- error = 0;
- break;
- case VIRTIO_BLK_S_UNSUPP:
- error = -ENOTTY;
- break;
- default:
- error = -EIO;
- break;
- }
-
- switch (vbr->req->cmd_type) {
- case REQ_TYPE_BLOCK_PC:
- vbr->req->resid_len = vbr->in_hdr.residual;
- vbr->req->sense_len = vbr->in_hdr.sense_len;
- vbr->req->errors = vbr->in_hdr.errors;
- break;
- case REQ_TYPE_SPECIAL:
- vbr->req->errors = (error != 0);
- break;
- default:
- break;
+ if (vbr->bio) {
+ virtblk_bio_done(vblk, vbr);
+ bio_done++;
+ } else {
+ virtblk_request_done(vblk, vbr);
+ req_done++;
}
-
- __blk_end_request_all(vbr->req, error);
- mempool_free(vbr, vblk->pool);
}
/* In case queue is stopped waiting for more buffers. */
- blk_start_queue(vblk->disk->queue);
+ if (req_done)
+ blk_start_queue(vblk->disk->queue);
spin_unlock_irqrestore(vblk->disk->queue->queue_lock, flags);
+
+ if (bio_done)
+ wake_up(&vblk->queue_wait);
+}
+
+static inline struct virtblk_req *virtblk_alloc_req(struct virtio_blk *vblk,
+ gfp_t gfp_mask)
+{
+ struct virtblk_req *vbr;
+
+ vbr = mempool_alloc(vblk->pool, gfp_mask);
+ if (vbr && use_bio)
+ sg_init_table(vbr->sg, vblk->sg_elems);
+
+ return vbr;
}
static bool do_req(struct request_queue *q, struct virtio_blk *vblk,
@@ -106,13 +144,13 @@ static bool do_req(struct request_queue *q, struct virtio_blk *vblk,
unsigned long num, out = 0, in = 0;
struct virtblk_req *vbr;
- vbr = mempool_alloc(vblk->pool, GFP_ATOMIC);
+ vbr = virtblk_alloc_req(vblk, GFP_ATOMIC);
if (!vbr)
/* When another request finishes we'll try again. */
return false;
vbr->req = req;
-
+ vbr->bio = NULL;
if (req->cmd_flags & REQ_FLUSH) {
vbr->out_hdr.type = VIRTIO_BLK_T_FLUSH;
vbr->out_hdr.sector = 0;
@@ -172,7 +210,8 @@ static bool do_req(struct request_queue *q, struct virtio_blk *vblk,
}
}
- if (virtqueue_add_buf(vblk->vq, vblk->sg, out, in, vbr, GFP_ATOMIC)<0) {
+ if (virtqueue_add_buf(vblk->vq, vblk->sg, out, in, vbr,
+ GFP_ATOMIC) < 0) {
mempool_free(vbr, vblk->pool);
return false;
}
@@ -180,7 +219,7 @@ static bool do_req(struct request_queue *q, struct virtio_blk *vblk,
return true;
}
-static void do_virtblk_request(struct request_queue *q)
+static void virtblk_request(struct request_queue *q)
{
struct virtio_blk *vblk = q->queuedata;
struct request *req;
@@ -203,6 +242,82 @@ static void do_virtblk_request(struct request_queue *q)
virtqueue_kick(vblk->vq);
}
+static void virtblk_add_buf_wait(struct virtio_blk *vblk,
+ struct virtblk_req *vbr,
+ unsigned long out,
+ unsigned long in)
+{
+ DEFINE_WAIT(wait);
+
+ for (;;) {
+ prepare_to_wait_exclusive(&vblk->queue_wait, &wait,
+ TASK_UNINTERRUPTIBLE);
+
+ spin_lock_irq(vblk->disk->queue->queue_lock);
+ if (virtqueue_add_buf(vblk->vq, vbr->sg, out, in, vbr,
+ GFP_ATOMIC) < 0) {
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+ io_schedule();
+ } else {
+ virtqueue_kick(vblk->vq);
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+ break;
+ }
+
+ }
+
+ finish_wait(&vblk->queue_wait, &wait);
+}
+
+static void virtblk_make_request(struct request_queue *q, struct bio *bio)
+{
+ struct virtio_blk *vblk = q->queuedata;
+ unsigned int num, out = 0, in = 0;
+ struct virtblk_req *vbr;
+
+ BUG_ON(bio->bi_phys_segments + 2 > vblk->sg_elems);
+ BUG_ON(bio->bi_rw & (REQ_FLUSH | REQ_FUA));
+
+ vbr = virtblk_alloc_req(vblk, GFP_NOIO);
+ if (!vbr) {
+ bio_endio(bio, -ENOMEM);
+ return;
+ }
+
+ vbr->bio = bio;
+ vbr->req = NULL;
+ vbr->out_hdr.type = 0;
+ vbr->out_hdr.sector = bio->bi_sector;
+ vbr->out_hdr.ioprio = bio_prio(bio);
+
+ sg_set_buf(&vbr->sg[out++], &vbr->out_hdr, sizeof(vbr->out_hdr));
+
+ num = blk_bio_map_sg(q, bio, vbr->sg + out);
+
+ sg_set_buf(&vbr->sg[num + out + in++], &vbr->status,
+ sizeof(vbr->status));
+
+ if (num) {
+ if (bio->bi_rw & REQ_WRITE) {
+ vbr->out_hdr.type |= VIRTIO_BLK_T_OUT;
+ out += num;
+ } else {
+ vbr->out_hdr.type |= VIRTIO_BLK_T_IN;
+ in += num;
+ }
+ }
+
+ spin_lock_irq(vblk->disk->queue->queue_lock);
+ if (unlikely(virtqueue_add_buf(vblk->vq, vbr->sg, out, in, vbr,
+ GFP_ATOMIC) < 0)) {
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+ virtblk_add_buf_wait(vblk, vbr, out, in);
+ return;
+ }
+ virtqueue_kick(vblk->vq);
+ spin_unlock_irq(vblk->disk->queue->queue_lock);
+}
+
/* return id (s/n) string for *disk to *id_str
*/
static int virtblk_get_id(struct gendisk *disk, char *id_str)
@@ -360,7 +475,7 @@ static int init_vq(struct virtio_blk *vblk)
int err = 0;
/* We expect one virtqueue, for output. */
- vblk->vq = virtio_find_single_vq(vblk->vdev, blk_done, "requests");
+ vblk->vq = virtio_find_single_vq(vblk->vdev, virtblk_done, "requests");
if (IS_ERR(vblk->vq))
err = PTR_ERR(vblk->vq);
@@ -414,7 +529,7 @@ static void virtblk_update_cache_mode(struct virtio_device *vdev)
u8 writeback = virtblk_get_cache_mode(vdev);
struct virtio_blk *vblk = vdev->priv;
- if (writeback)
+ if (writeback && !use_bio)
blk_queue_flush(vblk->disk->queue, REQ_FLUSH);
else
blk_queue_flush(vblk->disk->queue, 0);
@@ -477,6 +592,8 @@ static int __devinit virtblk_probe(struct virtio_device *vdev)
struct virtio_blk *vblk;
struct request_queue *q;
int err, index;
+ int pool_size;
+
u64 cap;
u32 v, blk_size, sg_elems, opt_io_size;
u16 min_io_size;
@@ -506,10 +623,12 @@ static int __devinit virtblk_probe(struct virtio_device *vdev)
goto out_free_index;
}
+ init_waitqueue_head(&vblk->queue_wait);
vblk->vdev = vdev;
vblk->sg_elems = sg_elems;
sg_init_table(vblk->sg, vblk->sg_elems);
mutex_init(&vblk->config_lock);
+
INIT_WORK(&vblk->config_work, virtblk_config_changed_work);
vblk->config_enable = true;
@@ -517,7 +636,10 @@ static int __devinit virtblk_probe(struct virtio_device *vdev)
if (err)
goto out_free_vblk;
- vblk->pool = mempool_create_kmalloc_pool(1,sizeof(struct virtblk_req));
+ pool_size = sizeof(struct virtblk_req);
+ if (use_bio)
+ pool_size += sizeof(struct scatterlist) * sg_elems;
+ vblk->pool = mempool_create_kmalloc_pool(1, pool_size);
if (!vblk->pool) {
err = -ENOMEM;
goto out_free_vq;
@@ -530,12 +652,14 @@ static int __devinit virtblk_probe(struct virtio_device *vdev)
goto out_mempool;
}
- q = vblk->disk->queue = blk_init_queue(do_virtblk_request, NULL);
+ q = vblk->disk->queue = blk_init_queue(virtblk_request, NULL);
if (!q) {
err = -ENOMEM;
goto out_put_disk;
}
+ if (use_bio)
+ blk_queue_make_request(q, virtblk_make_request);
q->queuedata = vblk;
virtblk_name_format("vd", index, vblk->disk->disk_name, DISK_NAME_LEN);
@@ -620,7 +744,6 @@ static int __devinit virtblk_probe(struct virtio_device *vdev)
if (!err && opt_io_size)
blk_queue_io_opt(q, blk_size * opt_io_size);
-
add_disk(vblk->disk);
err = device_create_file(disk_to_dev(vblk->disk), &dev_attr_serial);
if (err)
--
1.7.11.2
^ permalink raw reply related
* [PATCH V5 2/4] block: Add blk_bio_map_sg() helper
From: Asias He @ 2012-08-02 6:25 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, kvm, Michael S. Tsirkin, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-1-git-send-email-asias@redhat.com>
Add a helper to map a bio to a scatterlist, modelled after
blk_rq_map_sg.
This helper is useful for any driver that wants to create
a scatterlist from its ->make_request_fn method.
Changes in v2:
- Use __blk_segment_map_sg to avoid duplicated code
- Add cocbook style function comment
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Shaohua Li <shli@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Minchan Kim <minchan.kim@gmail.com>
Signed-off-by: Asias He <asias@redhat.com>
---
block/blk-merge.c | 37 +++++++++++++++++++++++++++++++++++++
include/linux/blkdev.h | 2 ++
2 files changed, 39 insertions(+)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 576b68e..e76279e 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -209,6 +209,43 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
}
EXPORT_SYMBOL(blk_rq_map_sg);
+/**
+ * blk_bio_map_sg - map a bio to a scatterlist
+ * @q: request_queue in question
+ * @bio: bio being mapped
+ * @sglist: scatterlist being mapped
+ *
+ * Note:
+ * Caller must make sure sg can hold bio->bi_phys_segments entries
+ *
+ * Will return the number of sg entries setup
+ */
+int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist)
+{
+ struct bio_vec *bvec, *bvprv;
+ struct scatterlist *sg;
+ int nsegs, cluster;
+ unsigned long i;
+
+ nsegs = 0;
+ cluster = blk_queue_cluster(q);
+
+ bvprv = NULL;
+ sg = NULL;
+ bio_for_each_segment(bvec, bio, i) {
+ __blk_segment_map_sg(q, bvec, sglist, &bvprv, &sg,
+ &nsegs, &cluster);
+ } /* segments in bio */
+
+ if (sg)
+ sg_mark_end(sg);
+
+ BUG_ON(bio->bi_phys_segments && nsegs > bio->bi_phys_segments);
+ return nsegs;
+}
+EXPORT_SYMBOL(blk_bio_map_sg);
+
static inline int ll_new_hw_segment(struct request_queue *q,
struct request *req,
struct bio *bio)
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4e72a9d..d261cbb 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -894,6 +894,8 @@ extern void blk_queue_flush_queueable(struct request_queue *q, bool queueable);
extern struct backing_dev_info *blk_get_backing_dev_info(struct block_device *bdev);
extern int blk_rq_map_sg(struct request_queue *, struct request *, struct scatterlist *);
+extern int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist);
extern void blk_dump_rq_flags(struct request *, char *);
extern long nr_blockdev_pages(void);
--
1.7.11.2
^ permalink raw reply related
* [PATCH V5 1/4] block: Introduce __blk_segment_map_sg() helper
From: Asias He @ 2012-08-02 6:25 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, kvm, Michael S. Tsirkin, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
In-Reply-To: <1343888757-25723-1-git-send-email-asias@redhat.com>
Split the mapping code in blk_rq_map_sg() to a helper
__blk_segment_map_sg(), so that other mapping function, e.g.
blk_bio_map_sg(), can share the code.
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Shaohua Li <shli@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org
Suggested-by: Jens Axboe <axboe@kernel.dk>
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Asias He <asias@redhat.com>
---
block/blk-merge.c | 80 +++++++++++++++++++++++++++++++------------------------
1 file changed, 45 insertions(+), 35 deletions(-)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 160035f..576b68e 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -110,6 +110,49 @@ static int blk_phys_contig_segment(struct request_queue *q, struct bio *bio,
return 0;
}
+static void
+__blk_segment_map_sg(struct request_queue *q, struct bio_vec *bvec,
+ struct scatterlist *sglist, struct bio_vec **bvprv,
+ struct scatterlist **sg, int *nsegs, int *cluster)
+{
+
+ int nbytes = bvec->bv_len;
+
+ if (*bvprv && *cluster) {
+ if ((*sg)->length + nbytes > queue_max_segment_size(q))
+ goto new_segment;
+
+ if (!BIOVEC_PHYS_MERGEABLE(*bvprv, bvec))
+ goto new_segment;
+ if (!BIOVEC_SEG_BOUNDARY(q, *bvprv, bvec))
+ goto new_segment;
+
+ (*sg)->length += nbytes;
+ } else {
+new_segment:
+ if (!*sg)
+ *sg = sglist;
+ else {
+ /*
+ * If the driver previously mapped a shorter
+ * list, we could see a termination bit
+ * prematurely unless it fully inits the sg
+ * table on each mapping. We KNOW that there
+ * must be more entries here or the driver
+ * would be buggy, so force clear the
+ * termination bit to avoid doing a full
+ * sg_init_table() in drivers for each command.
+ */
+ (*sg)->page_link &= ~0x02;
+ *sg = sg_next(*sg);
+ }
+
+ sg_set_page(*sg, bvec->bv_page, nbytes, bvec->bv_offset);
+ (*nsegs)++;
+ }
+ *bvprv = bvec;
+}
+
/*
* map a request to scatterlist, return number of sg entries setup. Caller
* must make sure sg can hold rq->nr_phys_segments entries
@@ -131,41 +174,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
bvprv = NULL;
sg = NULL;
rq_for_each_segment(bvec, rq, iter) {
- int nbytes = bvec->bv_len;
-
- if (bvprv && cluster) {
- if (sg->length + nbytes > queue_max_segment_size(q))
- goto new_segment;
-
- if (!BIOVEC_PHYS_MERGEABLE(bvprv, bvec))
- goto new_segment;
- if (!BIOVEC_SEG_BOUNDARY(q, bvprv, bvec))
- goto new_segment;
-
- sg->length += nbytes;
- } else {
-new_segment:
- if (!sg)
- sg = sglist;
- else {
- /*
- * If the driver previously mapped a shorter
- * list, we could see a termination bit
- * prematurely unless it fully inits the sg
- * table on each mapping. We KNOW that there
- * must be more entries here or the driver
- * would be buggy, so force clear the
- * termination bit to avoid doing a full
- * sg_init_table() in drivers for each command.
- */
- sg->page_link &= ~0x02;
- sg = sg_next(sg);
- }
-
- sg_set_page(sg, bvec->bv_page, nbytes, bvec->bv_offset);
- nsegs++;
- }
- bvprv = bvec;
+ __blk_segment_map_sg(q, bvec, sglist, &bvprv, &sg,
+ &nsegs, &cluster);
} /* segments in rq */
--
1.7.11.2
^ permalink raw reply related
* [PATCH V5 0/4] Improve virtio-blk performance
From: Asias He @ 2012-08-02 6:25 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, kvm, Michael S. Tsirkin, virtualization, Tejun Heo,
Shaohua Li, Christoph Hellwig
Hi folks,
This version added REQ_FLUSH and REQ_FUA support as suggested by Christoph and
rebased against latest linus's tree.
Jens, could you please consider picking up the dependencies 1/4 and 2/4 in your
tree. Thanks!
This patchset implements bio-based IO path for virito-blk to improve
performance.
Fio test shows bio-based IO path gives the following performance improvement:
1) Ramdisk device
With bio-based IO path, sequential read/write, random read/write
IOPS boost : 28%, 24%, 21%, 16%
Latency improvement: 32%, 17%, 21%, 16%
2) Fusion IO device
With bio-based IO path, sequential read/write, random read/write
IOPS boost : 11%, 11%, 13%, 10%
Latency improvement: 10%, 10%, 12%, 10%
Asias He (4):
block: Introduce __blk_segment_map_sg() helper
block: Add blk_bio_map_sg() helper
virtio-blk: Add bio-based IO path for virtio-blk
virtio-blk: Add REQ_FLUSH and REQ_FUA support to bio path
block/blk-merge.c | 117 +++++++++++++------
drivers/block/virtio_blk.c | 279 ++++++++++++++++++++++++++++++++++++++-------
include/linux/blkdev.h | 2 +
3 files changed, 324 insertions(+), 74 deletions(-)
--
1.7.11.2
^ permalink raw reply
* Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages
From: Rik van Riel @ 2012-08-01 20:53 UTC (permalink / raw)
To: Rafael Aquini
Cc: Konrad Rzeszutek Wilk, Michael S. Tsirkin, linux-kernel,
virtualization, linux-mm, Minchan Kim, Andi Kleen, Andrew Morton,
Rafael Aquini
In-Reply-To: <20120723181952.GA27373@t510.redhat.com>
On 07/23/2012 02:19 PM, Rafael Aquini wrote:
> In a glance, I believe this whole dance you're suggesting might just be too much
> of an overcomplication, and the best approach would be simply teaching the
> hotplug bits about the ballooned corner case just like it's being done to
> compaction/migration. However, I'll look at it carefully before making any other
> adjustments/propositions over here.
Compaction and hotplug do essentially the same thing
here: "collect all the movable pages from a page block,
and move them elsewhere".
Whether or not it is easier for them to share code, or
to duplicate a few lines of code, is something that can
be looked into later.
^ permalink raw reply
* Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages
From: Rik van Riel @ 2012-08-01 20:51 UTC (permalink / raw)
To: Minchan Kim
Cc: Rafael Aquini, Konrad Rzeszutek Wilk, Michael S. Tsirkin,
linux-kernel, virtualization, linux-mm, Andi Kleen, Andrew Morton,
Rafael Aquini
In-Reply-To: <20120723023332.GA6832@bbox>
On 07/22/2012 10:33 PM, Minchan Kim wrote:
> IMHO, better approach is that after we can get complete free pageblocks
> by compaction or reclaim, move balloon pages into that pageblocks and make
> that blocks to unmovable. It can prevent fragmentation and it makes
> current or future code don't need to consider balloon page.
I believe this is the wrong thing to do.
In a KVM guest, getting applications in transparent
huge pages can be a 10-25% performance benefit.
Therefore, we need to make all the 2MB pageblocks
we can available for use by userland.
Using 2MB blocks for the balloon (which is never
touched) is extremely wasteful and could result in
a large performance penalty, if we cannot defragment
the remaining memory enough to give 2MB pages to
applications.
The 2MB blocks are prime real estate. They should
remain available for applications.
^ permalink raw reply
* Re: [RFC-v5] tcm_vhost: Initial merge for vhost level target fabric driver
From: Paolo Bonzini @ 2012-08-01 6:42 UTC (permalink / raw)
To: Eric Northup
Cc: Jens Axboe, Anthony Liguori, linux-scsi, kvm-devel,
Michael S. Tsirkin, Greg Kroah-Hartman, qemu-devel, lf-virt,
Anthony Liguori, target-devel, Zhi Yong Wu, Christoph Hellwig,
Stefan Hajnoczi
In-Reply-To: <CAG7+5M35QAK=ggAJRZtga0PWw5VZFi-mraKEAkWiq16U3ysxjg@mail.gmail.com>
Il 31/07/2012 22:52, Eric Northup ha scritto:
> It seems to me like this is not the way that virtio devices are supposed
> to behave - if a guest splits a virtio_scsi_cmd_req or _resp across a
> page boundary, then this code won't work.
Buffers can cover several pages. Of course, data buffers have to be at
least sector aligned, but this restriction does not apply to
request/response descriptors.
> Quoting the 'Message Framing' part of the virtio spec:
>
> "In particular, no implementation should use the descriptor boundaries
> to determine the size of any header in a request. "
True, but this has never matched reality.
Paolo
^ permalink raw reply
* Call for Participation: IEEE eScience 2012 in Chicago, IL, October 8-12, 2012
From: Ioan Raicu @ 2012-07-31 21:47 UTC (permalink / raw)
To: virtualization
[-- Attachment #1.1: Type: text/plain, Size: 4361 bytes --]
*Call for Participation*
*IEEE eScience 2012*
*http://www.ci.uchicago.edu/escience2012/index.php*
*October 8^th -12^th , 2012 -- Chicago, IL, USA*
The 8th IEEE International Conference on eScience (eScience 2012) will
be held at the Hyatt Regency Chicago, Chicago, Illinois, *8-12 October
2012*, with workshops (including the Microsoft eScience Workshop) on 8-9
October and the main conference events on 10-12 October.
Scientific research is increasingly carried out by communities of
researchers that span disciplines, laboratories, organizations and
national boundaries. These activities involve geographically distributed
and heterogeneous resources such as computational systems, scientific
instruments, databases, sensors, software components, networks, and
people. Such large-scale and enhanced scientific endeavors are carried
out via collaborations on a global scale in which information and
computing technology plays a vital role and are thus popularly termed as
e-Science.
Keynote Speakers
Description: Professor Gerhard Klimeck
Professor Gerhard Klimeck
Director of the Network for Computational Nanotechnology and Professor
of Electrical and Computer Engineering
/Purdue University///
Description: Professor Leonard Smith
Professor Leonard Smith
Director of the Centre for the Analysis of Time Series (CATS)
/London School of Economics and Political Science/
Description: Dr. Gregory Wilson
Dr. Gregory Wilson
Software Carpentry
/Mozilla Foundation/
Description: Professor Carole Goble
Professor Carole Goble
/University of Manchester, UK/
Workshops
eScience 2012 will include workshops on 8-9 October. In addition to the
Microsoft eScience Workshop, other workshops were solicited in the Call
for Workshops
<http://www.ci.uchicago.edu/escience2012/call_for_workshops.php>. Six
workshops were accepted:
* 8 October: Extending High-Performance Computing Beyond its
Traditional User Communities
<http://www.psc.edu/index.php/escience-2012-workshop>, Papers due 6
August, Contact Person/email: Sergiu Sanielevici <sergiu@psc.edu>
* 8 October: 2nd International Workshop on Analyzing and Improving
Collaborative eScience with Social Networks (eSoN 12)
<http://www.ci.uchicago.edu/eson2012/>, Papers due 17 August,
Contact Person/email: Kyle Chard <kyle@ci.uchicago.edu>
* 8 October: Advances in eHealth
<http://www.scalalife.eu/content/advances-ehealth-2012-workshop>,
Abstracts due 4 July, Papers due 11 July, Contact Person/email:
Rossen Apostolov <rossen@kth.se>
* 9 October: Maintainable Software Practices in e-Science
<http://software.ac.uk/maintainable-software-practice-workshop>,
Papers due 20 July, Contact Person/email: Neil Chue Hong and
Jennifer Schopf <softwarepractice2012@easychair.org>
* 9 October: eScience Meets the Instrument,
<https://confluence-vre.its.monash.edu.au/display/escience2012/eScience+Meets+the+Instrument>
Contact Person/email: Richard Farnsworth <rif@aps.anl.gov>
* 9 October am: Collaborative research using eScience infrastructure
and high speed networks
<http://www.surfnet.nl/en/Hybride_netwerk/SURFlichtpaden/Pages/CollaborativeresearchusingeScienceinfrastructureandhighspeednetworks.aspx>,
Contact Person/email: Peter Hinrich <peter.hinrich@surfnet.nl>
In addition, there will also be one tutorial:
* 9 October pm: Big Data Processing: Lessons from Industry and
Applications in Science
<http://www.ci.uchicago.edu/escience2012/tutorial.php>, Contact
Person/email: Roger Barga <barga@microsoft.com>
--
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel: 1-847-722-0876
Office: 1-312-567-5704
Email:iraicu@cs.iit.edu
Web:http://www.cs.iit.edu/~iraicu/
Web:http://datasys.cs.iit.edu/
=================================================================
=================================================================
[-- Attachment #1.2.1: Type: text/html, Size: 45651 bytes --]
[-- Attachment #1.2.2: Type: image/png, Size: 95497 bytes --]
[-- Attachment #1.2.3: Type: image/png, Size: 92700 bytes --]
[-- Attachment #1.2.4: Type: image/png, Size: 92056 bytes --]
[-- Attachment #1.2.5: Type: image/png, Size: 82775 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [RFC-v5] tcm_vhost: Initial merge for vhost level target fabric driver
From: Eric Northup @ 2012-07-31 20:52 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Jens Axboe, Anthony Liguori, Stefan Hajnoczi, kvm-devel,
Michael S. Tsirkin, Greg Kroah-Hartman, qemu-devel, lf-virt,
Anthony Liguori, target-devel, linux-scsi, Paolo Bonzini,
Zhi Yong Wu, Christoph Hellwig
In-Reply-To: <1343346200-19850-1-git-send-email-nab@linux-iscsi.org>
[-- Attachment #1.1: Type: text/plain, Size: 8231 bytes --]
On Thu, Jul 26, 2012 at 4:43 PM, Nicholas A. Bellinger
<nab@linux-iscsi.org>wrote:
> [...]
> +static void vhost_scsi_handle_vq(struct vhost_scsi *vs)
> +{
> + struct vhost_virtqueue *vq = &vs->vqs[2];
> + struct virtio_scsi_cmd_req v_req;
> + struct tcm_vhost_tpg *tv_tpg;
> + struct tcm_vhost_cmd *tv_cmd;
> + u32 exp_data_len, data_first, data_num, data_direction;
> + unsigned out, in, i;
> + int head, ret;
> +
> + /* Must use ioctl VHOST_SCSI_SET_ENDPOINT */
> + tv_tpg = vs->vs_tpg;
> + if (unlikely(!tv_tpg)) {
> + pr_err("%s endpoint not set\n", __func__);
> + return;
> + }
> +
> + mutex_lock(&vq->mutex);
> + vhost_disable_notify(&vs->dev, vq);
> +
> + for (;;) {
> + head = vhost_get_vq_desc(&vs->dev, vq, vq->iov,
> + ARRAY_SIZE(vq->iov), &out, &in,
> + NULL, NULL);
> + pr_debug("vhost_get_vq_desc: head: %d, out: %u in: %u\n",
> + head, out, in);
> + /* On error, stop handling until the next kick. */
> + if (unlikely(head < 0))
> + break;
> + /* Nothing new? Wait for eventfd to tell us they
> refilled. */
> + if (head == vq->num) {
> + if (unlikely(vhost_enable_notify(&vs->dev, vq))) {
> + vhost_disable_notify(&vs->dev, vq);
> + continue;
> + }
> + break;
> + }
> +
> +/* FIXME: BIDI operation */
> + if (out == 1 && in == 1) {
>
It seems to me like this is not the way that virtio devices are supposed to
behave - if a guest splits a virtio_scsi_cmd_req or _resp across a page
boundary, then this code won't work.
Quoting the 'Message Framing' part of the virtio spec:
"In particular, no implementation should use the descriptor boundaries to
determine the size of any header in a request. "
+ data_direction = DMA_NONE;
> + data_first = 0;
> + data_num = 0;
> + } else if (out == 1 && in > 1) {
> + data_direction = DMA_FROM_DEVICE;
> + data_first = out + 1;
> + data_num = in - 1;
> + } else if (out > 1 && in == 1) {
> + data_direction = DMA_TO_DEVICE;
> + data_first = 1;
> + data_num = out - 1;
> + } else {
> + vq_err(vq, "Invalid buffer layout out: %u in:
> %u\n",
> + out, in);
> + break;
> + }
> +
> + /*
> + * Check for a sane resp buffer so we can report errors to
> + * the guest.
> + */
> + if (unlikely(vq->iov[out].iov_len !=
> + sizeof(struct
> virtio_scsi_cmd_resp))) {
> + vq_err(vq, "Expecting virtio_scsi_cmd_resp, got
> %zu"
> + " bytes\n", vq->iov[out].iov_len);
> + break;
> + }
> +
> + if (unlikely(vq->iov[0].iov_len != sizeof(v_req))) {
> + vq_err(vq, "Expecting virtio_scsi_cmd_req, got %zu"
> + " bytes\n", vq->iov[0].iov_len);
> + break;
> + }
> + pr_debug("Calling __copy_from_user: vq->iov[0].iov_base:
> %p,"
> + " len: %zu\n", vq->iov[0].iov_base, sizeof(v_req));
> + ret = __copy_from_user(&v_req, vq->iov[0].iov_base,
> + sizeof(v_req));
> + if (unlikely(ret)) {
> + vq_err(vq, "Faulted on virtio_scsi_cmd_req\n");
> + break;
> + }
> +
> + exp_data_len = 0;
> + for (i = 0; i < data_num; i++)
> + exp_data_len += vq->iov[data_first + i].iov_len;
> +
> + tv_cmd = vhost_scsi_allocate_cmd(tv_tpg, &v_req,
> + exp_data_len, data_direction);
> + if (IS_ERR(tv_cmd)) {
> + vq_err(vq, "vhost_scsi_allocate_cmd failed %ld\n",
> + PTR_ERR(tv_cmd));
> + break;
> + }
> + pr_debug("Allocated tv_cmd: %p exp_data_len: %d,
> data_direction"
> + ": %d\n", tv_cmd, exp_data_len, data_direction);
> +
> + tv_cmd->tvc_vhost = vs;
> +
> + if (unlikely(vq->iov[out].iov_len !=
> + sizeof(struct virtio_scsi_cmd_resp))) {
> + vq_err(vq, "Expecting virtio_scsi_cmd_resp, got
> %zu"
> + " bytes, out: %d, in: %d\n",
> + vq->iov[out].iov_len, out, in);
> + break;
> + }
> +
> + tv_cmd->tvc_resp = vq->iov[out].iov_base;
> +
> + /*
> + * Copy in the recieved CDB descriptor into tv_cmd->tvc_cdb
> + * that will be used by tcm_vhost_new_cmd_map() and down
> into
> + * target_setup_cmd_from_cdb()
> + */
> + memcpy(tv_cmd->tvc_cdb, v_req.cdb, TCM_VHOST_MAX_CDB_SIZE);
> + /*
> + * Check that the recieved CDB size does not exceeded our
> + * hardcoded max for tcm_vhost
> + */
> + /* TODO what if cdb was too small for varlen cdb header? */
> + if (unlikely(scsi_command_size(tv_cmd->tvc_cdb) >
> + TCM_VHOST_MAX_CDB_SIZE)) {
> + vq_err(vq, "Received SCSI CDB with command_size:
> %d that"
> + " exceeds SCSI_MAX_VARLEN_CDB_SIZE: %d\n",
> + scsi_command_size(tv_cmd->tvc_cdb),
> + TCM_VHOST_MAX_CDB_SIZE);
> + break; /* TODO */
> + }
> + tv_cmd->tvc_lun = ((v_req.lun[2] << 8) | v_req.lun[3]) &
> 0x3FFF;
> +
> + pr_debug("vhost_scsi got command opcode: %#02x, lun: %d\n",
> + tv_cmd->tvc_cdb[0], tv_cmd->tvc_lun);
> +
> + if (data_direction != DMA_NONE) {
> + ret = vhost_scsi_map_iov_to_sgl(tv_cmd,
> + &vq->iov[data_first], data_num,
> + data_direction == DMA_TO_DEVICE);
> + if (unlikely(ret)) {
> + vq_err(vq, "Failed to map iov to sgl\n");
> + break; /* TODO */
> + }
> + }
> +
> + /*
> + * Save the descriptor from vhost_get_vq_desc() to be used
> to
> + * complete the virtio-scsi request in TCM callback
> context via
> + * tcm_vhost_queue_data_in() and tcm_vhost_queue_status()
> + */
> + tv_cmd->tvc_vq_desc = head;
> + /*
> + * Dispatch tv_cmd descriptor for cmwq execution in process
> + * context provided by tcm_vhost_workqueue. This also
> ensures
> + * tv_cmd is executed on the same kworker CPU as this vhost
> + * thread to gain positive L2 cache locality effects..
> + */
> + INIT_WORK(&tv_cmd->work, tcm_vhost_submission_work);
> + queue_work(tcm_vhost_workqueue, &tv_cmd->work);
> + }
> +
> + mutex_unlock(&vq->mutex);
> +}
> [...]
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Typing one-handed, please don't mistake brevity for rudeness.
[-- Attachment #1.2: Type: text/html, Size: 9808 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [vmw_vmci 00/11] VMCI for Linux
From: Josh Boyer @ 2012-07-31 12:48 UTC (permalink / raw)
To: Greg KH
Cc: pv-drivers, linux-kernel, virtualization, vm-crosstalk,
Andrew Stiegmann (stieg), cschamp
In-Reply-To: <20120727014650.GB17168@kroah.com>
On Thu, Jul 26, 2012 at 9:46 PM, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Thu, Jul 26, 2012 at 09:06:25PM -0400, Josh Boyer wrote:
>> On Thu, Jul 26, 2012 at 7:39 PM, Andrew Stiegmann (stieg)
>> <astiegmann@vmware.com> wrote:
>> > drivers/misc/Kconfig | 1 +
>> > drivers/misc/Makefile | 1 +
>> > drivers/misc/vmw_vmci/Kconfig | 16 +
>>
>> Is there a reason this isn't going into staging first? The Hyper-V
>> drivers went through staging and that actually seemed to work fairly
>> well.
>
> Is there some reason you feel this should be in the staging tree now?
> Why?
Apologies for the delayed reply. Was on vacation.
Mostly because this is only one of several drivers. One that the
other drivers depend on, and I don't see those posted at all. I'm
guessing we'll want changes that will cause those unposted drivers to
break. It just seems to make sense to work on the API in staging
rather than slam it into drivers/misc/.
josh
^ permalink raw reply
* Re: [PATCH V3 3/3] virtio-blk: Add bio-based IO path for virtio-blk
From: Asias He @ 2012-07-31 8:43 UTC (permalink / raw)
To: Christoph Hellwig
Cc: kvm, Michael S. Tsirkin, linux-kernel, virtualization,
Paolo Bonzini
In-Reply-To: <20120730134322.GB6041@lst.de>
On 07/30/2012 09:43 PM, Christoph Hellwig wrote:
> On Mon, Jul 30, 2012 at 12:43:12PM +0800, Asias He wrote:
>> I think we can add REQ_FLUSH & REQ_FUA support to bio path and that
>> deserves another patch.
>
> Adding it is a requirement for merging the code.
>
OK. Will add that.
--
Asias
^ permalink raw reply
* [GIT PULL] tcm_vhost: Initial merge of vhost level target fabric driver
From: Nicholas A. Bellinger @ 2012-07-31 1:19 UTC (permalink / raw)
To: Linus Torvalds
Cc: Jens Axboe, kvm-devel, linux-scsi, Michael S. Tsirkin, qemu-devel,
Zhi Yong Wu, target-devel, Anthony Liguori, Paolo Bonzini,
lf-virt, Christoph Hellwig, LKML
Hi Linus,
Here is the PULL request for the initial merge of tcm_vhost based on
RFC-v5 code with MST's ACK appended to the initial merge commit.
As promised, the commit is available from two different branches for you
to consider merging as for-3.6 code.
The 'for-next-merge' branch based on mainline commit 7409a6657ae using
3.5-rc2 code contains two duplicates of pre-merge vhost patch
dependencies that have already been merged into mainline via net-next.
This commit is also in the 07302012 -next patchset, and available here:
git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending.git for-next-merge
Or the 'for-linus' branch containing an -rc0 head @ commit bdc0077af57:
Merge tag 'scsi-misc' of git://git.kernel.org/../jejb/scsi)
rebased up to the last commit in scsi-misc required for virtio-scsi
client LLD scanning logic to function properly with tcm_vhost fabric
ports, is available here:
git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending.git for-linus
Both branches have gotten recent testing and have been running
over-night small block random I/O tests connected to raw block flash
backends. The same diffstat below will result from pulling either
branch.
Also, the incremental patch to address MST's last round of post-merge
comments has been sent to the lists for feedback this afternoon. This
will be included into the usual post -rc1 PULL via 3.6-rc-fixes, along
with any other bits that end up changing post-merge.
Please let us know if you have any concerns.
Thank you!
--nab
Nicholas Bellinger (1):
tcm_vhost: Initial merge for vhost level target fabric driver
drivers/vhost/Kconfig | 3 +
drivers/vhost/Kconfig.tcm | 6 +
drivers/vhost/Makefile | 2 +
drivers/vhost/tcm_vhost.c | 1628 +++++++++++++++++++++++++++++++++++++++++++++
drivers/vhost/tcm_vhost.h | 101 +++
5 files changed, 1740 insertions(+), 0 deletions(-)
create mode 100644 drivers/vhost/Kconfig.tcm
create mode 100644 drivers/vhost/tcm_vhost.c
create mode 100644 drivers/vhost/tcm_vhost.h
^ permalink raw reply
* Re: Re: Re: [RFC PATCH 0/6] virtio-trace: Support virtio-trace
From: Yoshihiro YUNOMAE @ 2012-07-31 0:52 UTC (permalink / raw)
To: Amit Shah
Cc: Arnd Bergmann, Greg Kroah-Hartman, Frederic Weisbecker,
qemu-devel, Borislav Petkov, linux-kernel, Herbert Xu,
Franch Ch. Eigler, Ingo Molnar, Mathieu Desnoyers, Steven Rostedt,
Anthony Liguori, yrl.pp-manager.tt, Masami Hiramatsu,
virtualization
In-Reply-To: <20120727094320.GK18651@amit.redhat.com>
Hi Amit,
Sorry for the late reply.
(2012/07/27 18:43), Amit Shah wrote:
> On (Fri) 27 Jul 2012 [17:55:11], Yoshihiro YUNOMAE wrote:
>> Hi Amit,
>>
>> Thank you for commenting on our work.
>>
>> (2012/07/26 20:35), Amit Shah wrote:
>>> On (Tue) 24 Jul 2012 [11:36:57], Yoshihiro YUNOMAE wrote:
>>
[...]
>>>>
>>>> ***Just enhancement ideas***
>>>> - Support for trace-cmd
>>>> - Support for 9pfs protocol
>>>> - Support for non-blocking mode in QEMU
>>>
>>> There were patches long back (by me) to make chardevs non-blocking but
>>> they didn't make it upstream. Fedora carries them, if you want to try
>>> out. Though we want to converge on a reasonable solution that's
>>> acceptable upstream as well. Just that no one's working on it
>>> currently. Any help here will be appreciated.
>>
>> Thanks! In this case, since a guest will stop to run when host reads
>> trace data of the guest, char device is needed to add a non-blocking
>> mode. I'll read your patch series. Is the latest version 8?
>> http://lists.gnu.org/archive/html/qemu-devel/2010-12/msg00035.html
>
> I suppose the latest version on-list is what you quote above. The
> objections to the patch series are mentioned in Anthony's mails.
I'll check the mails.
> Hans maintains a rebased version of the patches in his tree at
>
> http://cgit.freedesktop.org/~jwrdegoede/qemu/
>
> those patches are included in Fedora's qemu-kvm, so you can try that
> out if it improves performance for you.
Thanks. I'll check those patches.
>>>> - Make "vhost-serial"
>>>
>>> I need to understand a) why it's perf-critical, and b) why should the
>>> host be involved at all, to comment on these.
>>
>> a) To make collecting overhead decrease for application on a guest.
>> (see above)
>> b) Trace data of host kernel is not involved even if we introduce this
>> patch set.
>
> I see, so you suggested vhost-serial only because you saw the guest
> stopping problem due to the absence of non-blocking code? If so, it
> now makes sense. I don't think we need vhost-serial in any way yet.
I understood. We suggested vhost-serial as one of the ideas for
improving performances. Other features(trace-cmd, 9pfs, and
non-blocking chardev) should be supported first, I think.
> BTW where do you parse the trace data obtained from guests? On a
> remote host?
It is the best that we can parse the data on a remote host in this
tracing system. Existing trace-cmd can already parse it on a remote
site. If we add the feature collecting event-format data(guest's
debugfs has that) from guests, we can parse tracing data on a remote
host as well as on a host running guests.
Thank you,
--
Yoshihiro YUNOMAE
Software Platform Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: yoshihiro.yunomae.ez@hitachi.com
^ permalink raw reply
* [PATCH] tcm_vhost: Post-merge review changes requested by MST
From: Nicholas A. Bellinger @ 2012-07-30 22:13 UTC (permalink / raw)
To: target-devel
Cc: Jens Axboe, Anthony Liguori, Stefan Hajnoczi, kvm-devel,
Michael S. Tsirkin, Greg Kroah-Hartman, qemu-devel, Zhi Yong Wu,
Anthony Liguori, linux-scsi, Paolo Bonzini, lf-virt,
Christoph Hellwig
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch contains the post RFC-v5 (post-merge) changes, this includes:
- Add locking comment
- Move vhost_scsi_complete_cmd ahead of TFO callbacks in order to
drop forward declarations
- Drop extra '!= NULL' usage in vhost_scsi_complete_cmd_work()
- Change vhost_scsi_*_handle_kick() to use pr_debug
- Fix possible race in vhost_scsi_set_endpoint() for vs->vs_tpg checking
+ assignment.
- Convert tv_tpg->tpg_vhost_count + ->tv_tpg_port_count from atomic_t ->
int, and make sure reference is protected by ->tv_tpg_mutex.
- Drop unnecessary vhost_scsi->vhost_ref_cnt
- Add 'err:' label for exception path in vhost_scsi_clear_endpoint()
- Add enum for VQ numbers, add usage in vhost_scsi_open()
- Add vhost_scsi_flush() + vhost_scsi_flush_vq() following
drivers/vhost/net.c
- Add smp_wmb() + vhost_scsi_flush() call during vhost_scsi_set_features()
- Drop unnecessary copy_from_user() usage with GET_ABI_VERSION ioctl
- Add missing vhost_scsi_compat_ioctl() caller for vhost_scsi_fops
- Fix function parameter definition first line to follow existing
vhost code style
- Change 'vHost' usage -> 'vhost' in handful of locations
- Change -EPERM -> -EBUSY usage for two failures in tcm_vhost_drop_nexus()
- Add comment for tcm_vhost_workqueue in tcm_vhost_init()
- Make GET_ABI_VERSION return 'int' + add comment in tcm_vhost.h
Reported-by: Michael S. Tsirkin <mst@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Cc: Anthony Liguori <aliguori@us.ibm.com>
Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/vhost/tcm_vhost.c | 195 ++++++++++++++++++++++++--------------------
drivers/vhost/tcm_vhost.h | 9 +-
2 files changed, 111 insertions(+), 93 deletions(-)
diff --git a/drivers/vhost/tcm_vhost.c b/drivers/vhost/tcm_vhost.c
index 481af88..74b2eda 100644
--- a/drivers/vhost/tcm_vhost.c
+++ b/drivers/vhost/tcm_vhost.c
@@ -53,9 +53,14 @@
#include "vhost.h"
#include "tcm_vhost.h"
+enum {
+ VHOST_SCSI_VQ_CTL = 0,
+ VHOST_SCSI_VQ_EVT = 1,
+ VHOST_SCSI_VQ_IO = 2,
+};
+
struct vhost_scsi {
- atomic_t vhost_ref_cnt;
- struct tcm_vhost_tpg *vs_tpg;
+ struct tcm_vhost_tpg *vs_tpg; /* Protected by vhost_scsi->dev.mutex */
struct vhost_dev dev;
struct vhost_virtqueue vqs[3];
@@ -131,8 +136,7 @@ static u32 tcm_vhost_get_default_depth(struct se_portal_group *se_tpg)
return 1;
}
-static u32 tcm_vhost_get_pr_transport_id(
- struct se_portal_group *se_tpg,
+static u32 tcm_vhost_get_pr_transport_id(struct se_portal_group *se_tpg,
struct se_node_acl *se_nacl,
struct t10_pr_registration *pr_reg,
int *format_code,
@@ -162,8 +166,7 @@ static u32 tcm_vhost_get_pr_transport_id(
format_code, buf);
}
-static u32 tcm_vhost_get_pr_transport_id_len(
- struct se_portal_group *se_tpg,
+static u32 tcm_vhost_get_pr_transport_id_len(struct se_portal_group *se_tpg,
struct se_node_acl *se_nacl,
struct t10_pr_registration *pr_reg,
int *format_code)
@@ -192,8 +195,7 @@ static u32 tcm_vhost_get_pr_transport_id_len(
format_code);
}
-static char *tcm_vhost_parse_pr_out_transport_id(
- struct se_portal_group *se_tpg,
+static char *tcm_vhost_parse_pr_out_transport_id(struct se_portal_group *se_tpg,
const char *buf,
u32 *out_tid_len,
char **port_nexus_ptr)
@@ -236,8 +238,7 @@ static struct se_node_acl *tcm_vhost_alloc_fabric_acl(
return &nacl->se_node_acl;
}
-static void tcm_vhost_release_fabric_acl(
- struct se_portal_group *se_tpg,
+static void tcm_vhost_release_fabric_acl(struct se_portal_group *se_tpg,
struct se_node_acl *se_nacl)
{
struct tcm_vhost_nacl *nacl = container_of(se_nacl,
@@ -297,7 +298,16 @@ static int tcm_vhost_get_cmd_state(struct se_cmd *se_cmd)
return 0;
}
-static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *);
+static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *tv_cmd)
+{
+ struct vhost_scsi *vs = tv_cmd->tvc_vhost;
+
+ spin_lock_bh(&vs->vs_completion_lock);
+ list_add_tail(&tv_cmd->tvc_completion_list, &vs->vs_completion_list);
+ spin_unlock_bh(&vs->vs_completion_lock);
+
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+}
static int tcm_vhost_queue_data_in(struct se_cmd *se_cmd)
{
@@ -381,7 +391,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
vs_completion_work);
struct tcm_vhost_cmd *tv_cmd;
- while ((tv_cmd = vhost_scsi_get_cmd_from_completion(vs)) != NULL) {
+ while ((tv_cmd = vhost_scsi_get_cmd_from_completion(vs))) {
struct virtio_scsi_cmd_resp v_rsp;
struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
int ret;
@@ -408,19 +418,6 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
vhost_signal(&vs->dev, &vs->vqs[2]);
}
-static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *tv_cmd)
-{
- struct vhost_scsi *vs = tv_cmd->tvc_vhost;
-
- pr_debug("%s tv_cmd %p\n", __func__, tv_cmd);
-
- spin_lock_bh(&vs->vs_completion_lock);
- list_add_tail(&tv_cmd->tvc_completion_list, &vs->vs_completion_list);
- spin_unlock_bh(&vs->vs_completion_lock);
-
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
-}
-
static struct tcm_vhost_cmd *vhost_scsi_allocate_cmd(
struct tcm_vhost_tpg *tv_tpg,
struct virtio_scsi_cmd_req *v_req,
@@ -787,12 +784,12 @@ static void vhost_scsi_handle_vq(struct vhost_scsi *vs)
static void vhost_scsi_ctl_handle_kick(struct vhost_work *work)
{
- pr_err("%s: The handling func for control queue.\n", __func__);
+ pr_debug("%s: The handling func for control queue.\n", __func__);
}
static void vhost_scsi_evt_handle_kick(struct vhost_work *work)
{
- pr_err("%s: The handling func for event queue.\n", __func__);
+ pr_debug("%s: The handling func for event queue.\n", __func__);
}
static void vhost_scsi_handle_kick(struct vhost_work *work)
@@ -825,11 +822,6 @@ static int vhost_scsi_set_endpoint(
return -EFAULT;
}
}
-
- if (vs->vs_tpg) {
- mutex_unlock(&vs->dev.mutex);
- return -EEXIST;
- }
mutex_unlock(&vs->dev.mutex);
mutex_lock(&tcm_vhost_mutex);
@@ -839,7 +831,7 @@ static int vhost_scsi_set_endpoint(
mutex_unlock(&tv_tpg->tv_tpg_mutex);
continue;
}
- if (atomic_read(&tv_tpg->tv_tpg_vhost_count)) {
+ if (tv_tpg->tv_tpg_vhost_count != 0) {
mutex_unlock(&tv_tpg->tv_tpg_mutex);
continue;
}
@@ -847,14 +839,20 @@ static int vhost_scsi_set_endpoint(
if (!strcmp(tv_tport->tport_name, t->vhost_wwpn) &&
(tv_tpg->tport_tpgt == t->vhost_tpgt)) {
- atomic_inc(&tv_tpg->tv_tpg_vhost_count);
- smp_mb__after_atomic_inc();
+ tv_tpg->tv_tpg_vhost_count++;
mutex_unlock(&tv_tpg->tv_tpg_mutex);
mutex_unlock(&tcm_vhost_mutex);
mutex_lock(&vs->dev.mutex);
+ if (vs->vs_tpg) {
+ mutex_unlock(&vs->dev.mutex);
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ tv_tpg->tv_tpg_vhost_count--;
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ return -EEXIST;
+ }
+
vs->vs_tpg = tv_tpg;
- atomic_inc(&vs->vhost_ref_cnt);
smp_mb__after_atomic_inc();
mutex_unlock(&vs->dev.mutex);
return 0;
@@ -871,38 +869,42 @@ static int vhost_scsi_clear_endpoint(
{
struct tcm_vhost_tport *tv_tport;
struct tcm_vhost_tpg *tv_tpg;
- int index;
+ int index, ret;
mutex_lock(&vs->dev.mutex);
/* Verify that ring has been setup correctly. */
for (index = 0; index < vs->dev.nvqs; ++index) {
if (!vhost_vq_access_ok(&vs->vqs[index])) {
- mutex_unlock(&vs->dev.mutex);
- return -EFAULT;
+ ret = -EFAULT;
+ goto err;
}
}
if (!vs->vs_tpg) {
- mutex_unlock(&vs->dev.mutex);
- return -ENODEV;
+ ret = -ENODEV;
+ goto err;
}
tv_tpg = vs->vs_tpg;
tv_tport = tv_tpg->tport;
if (strcmp(tv_tport->tport_name, t->vhost_wwpn) ||
(tv_tpg->tport_tpgt != t->vhost_tpgt)) {
- mutex_unlock(&vs->dev.mutex);
pr_warn("tv_tport->tport_name: %s, tv_tpg->tport_tpgt: %hu"
" does not match t->vhost_wwpn: %s, t->vhost_tpgt: %hu\n",
tv_tport->tport_name, tv_tpg->tport_tpgt,
t->vhost_wwpn, t->vhost_tpgt);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err;
}
- atomic_dec(&tv_tpg->tv_tpg_vhost_count);
+ tv_tpg->tv_tpg_vhost_count--;
vs->vs_tpg = NULL;
mutex_unlock(&vs->dev.mutex);
return 0;
+
+err:
+ mutex_unlock(&vs->dev.mutex);
+ return ret;
}
static int vhost_scsi_open(struct inode *inode, struct file *f)
@@ -918,9 +920,9 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
INIT_LIST_HEAD(&s->vs_completion_list);
spin_lock_init(&s->vs_completion_lock);
- s->vqs[0].handle_kick = vhost_scsi_ctl_handle_kick;
- s->vqs[1].handle_kick = vhost_scsi_evt_handle_kick;
- s->vqs[2].handle_kick = vhost_scsi_handle_kick;
+ s->vqs[VHOST_SCSI_VQ_CTL].handle_kick = vhost_scsi_ctl_handle_kick;
+ s->vqs[VHOST_SCSI_VQ_EVT].handle_kick = vhost_scsi_evt_handle_kick;
+ s->vqs[VHOST_SCSI_VQ_IO].handle_kick = vhost_scsi_handle_kick;
r = vhost_dev_init(&s->dev, s->vqs, 3);
if (r < 0) {
kfree(s);
@@ -949,6 +951,18 @@ static int vhost_scsi_release(struct inode *inode, struct file *f)
return 0;
}
+static void vhost_scsi_flush_vq(struct vhost_scsi *vs, int index)
+{
+ vhost_poll_flush(&vs->dev.vqs[index].poll);
+}
+
+static void vhost_scsi_flush(struct vhost_scsi *vs)
+{
+ vhost_scsi_flush_vq(vs, VHOST_SCSI_VQ_CTL);
+ vhost_scsi_flush_vq(vs, VHOST_SCSI_VQ_EVT);
+ vhost_scsi_flush_vq(vs, VHOST_SCSI_VQ_IO);
+}
+
static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
{
if (features & ~VHOST_FEATURES)
@@ -961,7 +975,8 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
return -EFAULT;
}
vs->dev.acked_features = features;
- /* TODO possibly smp_wmb() and flush vqs */
+ smp_wmb();
+ vhost_scsi_flush(vs);
mutex_unlock(&vs->dev.mutex);
return 0;
}
@@ -974,7 +989,7 @@ static long vhost_scsi_ioctl(struct file *f, unsigned int ioctl,
void __user *argp = (void __user *)arg;
u64 __user *featurep = argp;
u64 features;
- int r;
+ int r, abi_version = VHOST_SCSI_ABI_VERSION;
switch (ioctl) {
case VHOST_SCSI_SET_ENDPOINT:
@@ -988,12 +1003,7 @@ static long vhost_scsi_ioctl(struct file *f, unsigned int ioctl,
return vhost_scsi_clear_endpoint(vs, &backend);
case VHOST_SCSI_GET_ABI_VERSION:
- if (copy_from_user(&backend, argp, sizeof backend))
- return -EFAULT;
-
- backend.abi_version = VHOST_SCSI_ABI_VERSION;
-
- if (copy_to_user(argp, &backend, sizeof backend))
+ if (copy_to_user(argp, &abi_version, sizeof abi_version))
return -EFAULT;
return 0;
case VHOST_GET_FEATURES:
@@ -1013,11 +1023,21 @@ static long vhost_scsi_ioctl(struct file *f, unsigned int ioctl,
}
}
+#ifdef CONFIG_COMPAT
+static long vhost_scsi_compat_ioctl(struct file *f, unsigned int ioctl,
+ unsigned long arg)
+{
+ return vhost_scsi_ioctl(f, ioctl, (unsigned long)compat_ptr(arg));
+}
+#endif
+
static const struct file_operations vhost_scsi_fops = {
.owner = THIS_MODULE,
.release = vhost_scsi_release,
.unlocked_ioctl = vhost_scsi_ioctl,
- /* TODO compat ioctl? */
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = vhost_scsi_compat_ioctl,
+#endif
.open = vhost_scsi_open,
.llseek = noop_llseek,
};
@@ -1054,28 +1074,28 @@ static char *tcm_vhost_dump_proto_id(struct tcm_vhost_tport *tport)
return "Unknown";
}
-static int tcm_vhost_port_link(
- struct se_portal_group *se_tpg,
+static int tcm_vhost_port_link(struct se_portal_group *se_tpg,
struct se_lun *lun)
{
struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
struct tcm_vhost_tpg, se_tpg);
- atomic_inc(&tv_tpg->tv_tpg_port_count);
- smp_mb__after_atomic_inc();
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ tv_tpg->tv_tpg_port_count++;
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
return 0;
}
-static void tcm_vhost_port_unlink(
- struct se_portal_group *se_tpg,
+static void tcm_vhost_port_unlink(struct se_portal_group *se_tpg,
struct se_lun *se_lun)
{
struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
struct tcm_vhost_tpg, se_tpg);
- atomic_dec(&tv_tpg->tv_tpg_port_count);
- smp_mb__after_atomic_dec();
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ tv_tpg->tv_tpg_port_count--;
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
}
static struct se_node_acl *tcm_vhost_make_nodeacl(
@@ -1122,8 +1142,7 @@ static void tcm_vhost_drop_nodeacl(struct se_node_acl *se_acl)
kfree(nacl);
}
-static int tcm_vhost_make_nexus(
- struct tcm_vhost_tpg *tv_tpg,
+static int tcm_vhost_make_nexus(struct tcm_vhost_tpg *tv_tpg,
const char *name)
{
struct se_portal_group *se_tpg;
@@ -1168,7 +1187,7 @@ static int tcm_vhost_make_nexus(
return -ENOMEM;
}
/*
- * Now register the TCM vHost virtual I_T Nexus as active with the
+ * Now register the TCM vhost virtual I_T Nexus as active with the
* call to __transport_register_session()
*/
__transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
@@ -1179,8 +1198,7 @@ static int tcm_vhost_make_nexus(
return 0;
}
-static int tcm_vhost_drop_nexus(
- struct tcm_vhost_tpg *tpg)
+static int tcm_vhost_drop_nexus(struct tcm_vhost_tpg *tpg)
{
struct se_session *se_sess;
struct tcm_vhost_nexus *tv_nexus;
@@ -1198,27 +1216,27 @@ static int tcm_vhost_drop_nexus(
return -ENODEV;
}
- if (atomic_read(&tpg->tv_tpg_port_count)) {
+ if (tpg->tv_tpg_port_count != 0) {
mutex_unlock(&tpg->tv_tpg_mutex);
- pr_err("Unable to remove TCM_vHost I_T Nexus with"
+ pr_err("Unable to remove TCM_vhost I_T Nexus with"
" active TPG port count: %d\n",
- atomic_read(&tpg->tv_tpg_port_count));
- return -EPERM;
+ tpg->tv_tpg_port_count);
+ return -EBUSY;
}
- if (atomic_read(&tpg->tv_tpg_vhost_count)) {
+ if (tpg->tv_tpg_vhost_count != 0) {
mutex_unlock(&tpg->tv_tpg_mutex);
- pr_err("Unable to remove TCM_vHost I_T Nexus with"
+ pr_err("Unable to remove TCM_vhost I_T Nexus with"
" active TPG vhost count: %d\n",
- atomic_read(&tpg->tv_tpg_vhost_count));
- return -EPERM;
+ tpg->tv_tpg_vhost_count);
+ return -EBUSY;
}
- pr_debug("TCM_vHost_ConfigFS: Removing I_T Nexus to emulated"
+ pr_debug("TCM_vhost_ConfigFS: Removing I_T Nexus to emulated"
" %s Initiator Port: %s\n", tcm_vhost_dump_proto_id(tpg->tport),
tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
/*
- * Release the SCSI I_T Nexus to the emulated vHost Target Port
+ * Release the SCSI I_T Nexus to the emulated vhost Target Port
*/
transport_deregister_session(tv_nexus->tvn_se_sess);
tpg->tpg_nexus = NULL;
@@ -1228,8 +1246,7 @@ static int tcm_vhost_drop_nexus(
return 0;
}
-static ssize_t tcm_vhost_tpg_show_nexus(
- struct se_portal_group *se_tpg,
+static ssize_t tcm_vhost_tpg_show_nexus(struct se_portal_group *se_tpg,
char *page)
{
struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
@@ -1250,8 +1267,7 @@ static ssize_t tcm_vhost_tpg_show_nexus(
return ret;
}
-static ssize_t tcm_vhost_tpg_store_nexus(
- struct se_portal_group *se_tpg,
+static ssize_t tcm_vhost_tpg_store_nexus(struct se_portal_group *se_tpg,
const char *page,
size_t count)
{
@@ -1336,8 +1352,7 @@ static struct configfs_attribute *tcm_vhost_tpg_attrs[] = {
NULL,
};
-static struct se_portal_group *tcm_vhost_make_tpg(
- struct se_wwn *wwn,
+static struct se_portal_group *tcm_vhost_make_tpg(struct se_wwn *wwn,
struct config_group *group,
const char *name)
{
@@ -1385,7 +1400,7 @@ static void tcm_vhost_drop_tpg(struct se_portal_group *se_tpg)
list_del(&tpg->tv_tpg_list);
mutex_unlock(&tcm_vhost_mutex);
/*
- * Release the virtual I_T Nexus for this vHost TPG
+ * Release the virtual I_T Nexus for this vhost TPG
*/
tcm_vhost_drop_nexus(tpg);
/*
@@ -1395,8 +1410,7 @@ static void tcm_vhost_drop_tpg(struct se_portal_group *se_tpg)
kfree(tpg);
}
-static struct se_wwn *tcm_vhost_make_tport(
- struct target_fabric_configfs *tf,
+static struct se_wwn *tcm_vhost_make_tport(struct target_fabric_configfs *tf,
struct config_group *group,
const char *name)
{
@@ -1592,7 +1606,10 @@ static void tcm_vhost_deregister_configfs(void)
static int __init tcm_vhost_init(void)
{
int ret = -ENOMEM;
-
+ /*
+ * Use our own dedicated workqueue for submitting I/O into
+ * target core to avoid contention within system_wq.
+ */
tcm_vhost_workqueue = alloc_workqueue("tcm_vhost", 0, 0);
if (!tcm_vhost_workqueue)
goto out;
diff --git a/drivers/vhost/tcm_vhost.h b/drivers/vhost/tcm_vhost.h
index c983ed2..eff42df 100644
--- a/drivers/vhost/tcm_vhost.h
+++ b/drivers/vhost/tcm_vhost.h
@@ -47,9 +47,9 @@ struct tcm_vhost_tpg {
/* Vhost port target portal group tag for TCM */
u16 tport_tpgt;
/* Used to track number of TPG Port/Lun Links wrt to explict I_T Nexus shutdown */
- atomic_t tv_tpg_port_count;
- /* Used for vhost_scsi device reference to tpg_nexus */
- atomic_t tv_tpg_vhost_count;
+ int tv_tpg_port_count;
+ /* Used for vhost_scsi device reference to tpg_nexus, protected by tv_tpg_mutex */
+ int tv_tpg_vhost_count;
/* list for tcm_vhost_list */
struct list_head tv_tpg_list;
/* Used to protect access for tpg_nexus */
@@ -98,4 +98,5 @@ struct vhost_scsi_target {
/* VHOST_SCSI specific defines */
#define VHOST_SCSI_SET_ENDPOINT _IOW(VHOST_VIRTIO, 0x40, struct vhost_scsi_target)
#define VHOST_SCSI_CLEAR_ENDPOINT _IOW(VHOST_VIRTIO, 0x41, struct vhost_scsi_target)
-#define VHOST_SCSI_GET_ABI_VERSION _IOW(VHOST_VIRTIO, 0x42, struct vhost_scsi_target)
+/* Changing this breaks userspace. */
+#define VHOST_SCSI_GET_ABI_VERSION _IOW(VHOST_VIRTIO, 0x42, int)
--
1.7.2.5
^ permalink raw reply related
* Re: [RFC PATCH 4/6] ftrace: Allow stealing pages from pipe buffer
From: Steven Rostedt @ 2012-07-30 22:12 UTC (permalink / raw)
To: Yoshihiro YUNOMAE
Cc: Herbert Xu, Arnd Bergmann, qemu-devel, Frederic Weisbecker,
yrl.pp-manager.tt, linux-kernel, Borislav Petkov, virtualization,
Masami Hiramatsu, Franch Ch. Eigler, Ingo Molnar,
Mathieu Desnoyers, Anthony Liguori, Greg Kroah-Hartman, Amit Shah
In-Reply-To: <20120724023738.6600.59837.stgit@ltc189.sdl.hitachi.co.jp>
On Tue, 2012-07-24 at 11:37 +0900, Yoshihiro YUNOMAE wrote:
> From: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>
> Use generic steal operation on pipe buffer to allow stealing
> ring buffer's read page from pipe buffer.
>
> Note that this could reduce the performance of splice on the
> splice_write side operation without affinity setting.
> Since the ring buffer's read pages are allocated on the
> tracing-node, but the splice user does not always execute
> splice write side operation on the same node. In this case,
> the page will be accessed from the another node.
> Thus, it is strongly recommended to assign the splicing
> thread to corresponding node.
>
> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
-- Steve
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> ---
>
> kernel/trace/trace.c | 8 +-------
> 1 files changed, 1 insertions(+), 7 deletions(-)
>
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index a120f98..ae01930 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -4194,12 +4194,6 @@ static void buffer_pipe_buf_release(struct pipe_inode_info *pipe,
> buf->private = 0;
> }
>
> -static int buffer_pipe_buf_steal(struct pipe_inode_info *pipe,
> - struct pipe_buffer *buf)
> -{
> - return 1;
> -}
> -
> static void buffer_pipe_buf_get(struct pipe_inode_info *pipe,
> struct pipe_buffer *buf)
> {
> @@ -4215,7 +4209,7 @@ static const struct pipe_buf_operations buffer_pipe_buf_ops = {
> .unmap = generic_pipe_buf_unmap,
> .confirm = generic_pipe_buf_confirm,
> .release = buffer_pipe_buf_release,
> - .steal = buffer_pipe_buf_steal,
> + .steal = generic_pipe_buf_steal,
> .get = buffer_pipe_buf_get,
> };
>
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox