* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 19:35 UTC (permalink / raw)
To: Ingo Molnar
Cc: Avi Kivity, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <20090817151449.GA19384@elte.hu>
[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]
Ingo Molnar wrote:
> * Gregory Haskins <gregory.haskins@gmail.com> wrote:
>
>> Avi Kivity wrote:
>>> On 08/17/2009 05:16 PM, Gregory Haskins wrote:
>>>>> My opinion is that this is a duplication of effort and we'd be better
>>>>> off if everyone contributed to enhancing virtio, which already has
>>>>> widely deployed guest drivers and non-Linux guest support.
>>>>>
>>>>> It may have merit if it is proven that it is technically superior to
>>>>> virtio (and I don't mean some benchmark in some point in time; I mean
>>>>> design wise). So far I haven't seen any indications that it is.
>>>>>
>>>>>
>>>> The design is very different, so hopefully I can start to convince you
>>>> why it might be interesting.
>>>>
>>> We've been through this before I believe. If you can point out
>>> specific differences that make venet outperform virtio-net I'll
>>> be glad to hear (and steal) them though.
>> You sure know how to convince someone to collaborate with you, eh?
>>
>> Unforunately, i've answered that question numerous times, but it
>> apparently falls on deaf ears.
>
> I'm trying to find the relevant discussion. The link you gave in the
> previous mail:
>
> http://lkml.org/lkml/2009/4/21/408
>
> does not offer any design analysis of vbus versus virtio, and why
> the only fix to virtio is vbus. It offers a comparison and a blanket
> statement that vbus is superior but no arguments.
>
> (If you've already explained in a past thread then please give me an
> URL to that reply if possible, or forward me that prior reply.
> Thanks!)
Sorry, it was a series of long threads from quite a while back. I will
see if I can find some references, but it might be easier to just start
fresh (see the last reply I sent).
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 19:33 UTC (permalink / raw)
To: Ingo Molnar
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <20090817150844.GA3307@elte.hu>
[-- Attachment #1: Type: text/plain, Size: 11547 bytes --]
Ingo Molnar wrote:
> * Gregory Haskins <gregory.haskins@gmail.com> wrote:
>
>> Hi Ingo,
>>
>> 1) First off, let me state that I have made every effort to
>> propose this as a solution to integrate with KVM, the most recent
>> of which is April:
>>
>> http://lkml.org/lkml/2009/4/21/408
>>
>> If you read through the various vbus related threads on LKML/KVM
>> posted this year, I think you will see that I made numerous polite
>> offerings to work with people on finding a common solution here,
>> including Michael.
>>
>> In the end, Michael decided that go a different route using some
>> of the ideas proposed in vbus + venet-tap to create vhost-net.
>> This is fine, and I respect his decision. But do not try to pin
>> "fracturing" on me, because I tried everything to avoid it. :)
>
> That's good.
>
> So if virtio is fixed to be as fast as vbus, and if there's no other
> techical advantages of vbus over virtio you'll be glad to drop vbus
> and stand behind virtio?
To reiterate: vbus and virtio are not mutually exclusive. The virtio
device model rides happily on top of the vbus bus model.
This is primarily a question of the virtio-pci adapter, vs virtio-vbus.
For more details, see this post: http://lkml.org/lkml/2009/8/6/244
There is a secondary question of venet (a vbus native device) verses
virtio-net (a virtio native device that works with PCI or VBUS). If
this contention is really around venet vs virtio-net, I may possibly
conceed and retract its submission to mainline. I've been pushing it to
date because people are using it and I don't see any reason that the
driver couldn't be upstream.
>
> Also, are you willing to help virtio to become faster?
Yes, that is not a problem. Note that virtio in general, and
virtio-net/venet in particular are not the primary goal here, however.
Improved 802.x and block IO are just positive side-effects of the
effort. I started with 802.x networking just to demonstrate the IO
layer capabilities, and to test it. It ended up being so good on
contrast to existing facilities, that developers in the vbus community
started using it for production development.
Ultimately, I created vbus to address areas of performance that have not
yet been addressed in things like KVM. Areas such as real-time guests,
or RDMA (host bypass) interfaces. I also designed it in such a way that
we could, in theory, write one set of (linux-based) backends, and have
them work across a variety of environments (such as containers/VMs like
KVM, lguest, openvz, but also physical systems like blade enclosures and
clusters, or even applications running on the host).
> Or do you
> have arguments why that is impossible to do so and why the only
> possible solution is vbus? Avi says no such arguments were offered
> so far.
Not for lack of trying. I think my points have just been missed
everytime I try to describe them. ;) Basically I write a message very
similar to this one, and the next conversation starts back from square
one. But I digress, let me try again..
Noting that this discussion is really about the layer *below* virtio,
not virtio itself (e.g. PCI vs vbus). Lets start with a little background:
-- Background --
So on one level, we have the resource-container technology called
"vbus". It lets you create a container on the host, fill it with
virtual devices, and assign that container to some context (such as a
KVM guest). These "devices" are LKMs, and each device has a very simple
verb namespace consisting of a synchronous "call()" method, and a
"shm()" method for establishing async channels.
The async channels are just shared-memory with a signal path (e.g.
interrupts and hypercalls), which the device+driver can use to overlay
things like rings (virtqueues, IOQs), or other shared-memory based
constructs of their choosing (such as a shared table). The signal path
is designed to minimize enter/exits and reduce spurious signals in a
unified way (see shm-signal patch).
call() can be used both for config-space like details, as well as
fast-path messaging that require synchronous behavior (such as guest
scheduler updates).
All of this is managed via sysfs/configfs.
On the guest, we have a "vbus-proxy" which is how the guest gets access
to devices assigned to its container. (as an aside, "virtio" devices
can be populated in the container, and then surfaced up to the
virtio-bus via that virtio-vbus patch I mentioned).
There is a thing called a "vbus-connector" which is the guest specific
part. Its job is to connect the vbus-proxy in the guest, to the vbus
container on the host. How it does its job is specific to the connector
implementation, but its role is to transport messages between the guest
and the host (such as for call() and shm() invocations) and to handle
things like discovery and hotswap.
-- Issues --
Out of all this, I think the biggest contention point is the design of
the vbus-connector that I use in AlacrityVM (Avi, correct me if I am
wrong and you object to other aspects as well). I suspect that if I had
designed the vbus-connector to surface vbus devices as PCI devices via
QEMU, the patches would potentially have been pulled in a while ago.
There are, of course, reasons why vbus does *not* render as PCI, so this
is the meat of of your question, I believe.
At a high level, PCI was designed for software-to-hardware interaction,
so it makes assumptions about that relationship that do not necessarily
apply to virtualization.
For instance:
A) hardware can only generate byte/word sized requests at a time because
that is all the pcb-etch and silicon support. So hardware is usually
expressed in terms of some number of "registers".
B) each access to one of these registers is relatively cheap
C) the target end-point has no visibility into the CPU machine state
other than the parameters passed in the bus-cycle (usually an address
and data tuple).
D) device-ids are in a fixed width register and centrally assigned from
an authority (e.g. PCI-SIG).
E) Interrupt/MSI routing is per-device oriented
F) Interrupts/MSI are assumed cheap to inject
G) Interrupts/MSI are non-priortizable.
H) Interrupts/MSI are statically established
These assumptions and constraints may be completely different or simply
invalid in a virtualized guest. For instance, the hypervisor is just
software, and therefore it's not restricted to "etch" constraints. IO
requests can be arbitrarily large, just as if you are invoking a library
function-call or OS system-call. Likewise, each one of those requests is
a branch and a context switch, so it has often has greater performance
implications than a simple register bus-cycle in hardware. If you use
an MMIO variant, it has to run through the page-fault code to be decoded.
The result is typically decreased performance if you try to do the same
thing real hardware does. This is why you usually see hypervisor
specific drivers (e.g. virtio-net, vmnet, etc) a common feature.
_Some_ performance oriented items can technically be accomplished in
PCI, albeit in a much more awkward way. For instance, you can set up a
really fast, low-latency "call()" mechanism using a PIO port on a
PCI-model and ioeventfd. As a matter of fact, this is exactly what the
vbus pci-bridge does:
http://git.kernel.org/?p=linux/kernel/git/ghaskins/alacrityvm/linux-2.6.git;a=blob;f=drivers/vbus/pci-bridge.c;h=f0ed51af55b5737b3ae4239ed2adfe12c7859941;hb=ee557a5976921650b792b19e6a93cd03fcad304a#l102
(Also note that the enabling technology, ioeventfd, is something that
came out of my efforts on vbus).
The problem here is that this is incredibly awkward to setup. You have
all that per-cpu goo and the registration of the memory on the guest.
And on the host side, you have all the vmapping of the registered
memory, and the file-descriptor to manage. In short, its really painful.
I would much prefer to do this *once*, and then let all my devices
simple re-use that infrastructure. This is, in fact, what I do. Here
is the device model that a guest sees:
struct vbus_device_proxy_ops {
int (*open)(struct vbus_device_proxy *dev, int version, int
flags);
int (*close)(struct vbus_device_proxy *dev, int flags);
int (*shm)(struct vbus_device_proxy *dev, int id, int prio,
void *ptr, size_t len,
struct shm_signal_desc *sigdesc, struct shm_signal
**signal,
int flags);
int (*call)(struct vbus_device_proxy *dev, u32 func,
void *data, size_t len, int flags);
void (*release)(struct vbus_device_proxy *dev);
};
Now the client just calls dev->call() and its lighting quick, and they
don't have to worry about all the details of making it quick, nor expend
addition per-cpu heap and address space to get it.
Moving on: _Other_ items cannot be replicated (at least, not without
hacking it into something that is no longer PCI.
Things like the pci-id namespace are just silly for software. I would
rather have a namespace that does not require central management so
people are free to create vbus-backends at will. This is akin to
registering a device MAJOR/MINOR, verses using the various dynamic
assignment mechanisms. vbus uses a string identifier in place of a
pci-id. This is superior IMHO, and not compatible with PCI.
As another example, the connector design coalesces *all* shm-signals
into a single interrupt (by prio) that uses the same context-switch
mitigation techniques that help boost things like networking. This
effectively means we can detect and optimize out ack/eoi cycles from the
APIC as the IO load increases (which is when you need it most). PCI has
no such concept.
In addition, the signals and interrupts are priority aware, which is
useful for things like 802.1p networking where you may establish 8-tx
and 8-rx queues for your virtio-net device. x86 APIC really has no
usable equivalent, so PCI is stuck here.
Also, the signals can be allocated on-demand for implementing things
like IPC channels in response to guest requests since there is no
assumption about device-to-interrupt mappings. This is more flexible.
And through all of this, this design would work in any guest even if it
doesn't have PCI (e.g. lguest, UML, physical systems, etc).
-- Bottom Line --
The idea here is to generalize all the interesting parts that are common
(fast sync+async io, context-switch mitigation, back-end models, memory
abstractions, signal-path routing, etc) that a variety of linux based
technologies can use (kvm, lguest, openvz, uml, physical systems) and
only require the thin "connector" code to port the system around. The
idea is to try to get this aspect of PV right once, and at some point in
the future, perhaps vbus will be as ubiquitous as PCI. Well, perhaps
not *that* ubiquitous, but you get the idea ;)
Then device models like virtio can ride happily on top and we end up
with a really robust and high-performance Linux-based stack. I don't
buy the argument that we already have PCI so lets use it. I don't think
its the best design and I am not afraid to make an investment in a
change here because I think it will pay off in the long run.
I hope this helps to clarify my motivation.
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* mlx4 2.6.31-rc5: SW2HW_EQ failed.
From: Christoph Lameter @ 2009-08-17 19:26 UTC (permalink / raw)
To: Roland Dreier; +Cc: netdev, Yevgeny Petrilin
mlx4 fails to initialize here:
[ 9.973940] mlx4_core 0000:04:00.0: irq 93 for MSI/MSI-X
[ 9.983108] sr 1:0:0:0: Attached scsi CD-ROM sr0
[ 9.988209] ses 0:0:32:0: Attached scsi generic sg0 type 13
[ 9.999376] sd 0:2:0:0: Attached scsi generic sg1 type 0
[ 10.010024] sr 1:0:0:0: Attached scsi generic sg2 type 5
[ 10.256371] mlx4_core 0000:04:00.0: SW2HW_EQ failed (-5)
[ 10.270103] mlx4_core 0000:04:00.0: Failed to initialize event queue
table, aborting.
[ 10.288768] mlx4_core 0000:04:00.0: PCI INT A disabled
[ 10.299057] mlx4_core: probe of 0000:04:00.0 failed with error -5
^ permalink raw reply
* Re: Very strange issues with ethernet wake on lan
From: Rafael J. Wysocki @ 2009-08-17 18:30 UTC (permalink / raw)
To: Maxim Levitsky
Cc: netdev@vger.kernel.org, linux-pm@lists.linux-foundation.org
In-Reply-To: <1250500588.4301.5.camel@localhost.localdomain>
On Monday 17 August 2009, Maxim Levitsky wrote:
> On Sun, 2009-08-16 at 06:42 +0300, Maxim Levitsky wrote:
> > Hi,
> >
> > I have recently put back the davicom dm9009 ethernet card into my
> > computer.
> >
> > Some long time ago, I have written its suspend/resume routines.
> > Now I see that few things have changed, like I need to enable wake in
> > sysfs or better patch the code to do so, some nice helpers like
> > pci_prepare_to_sleep have arrived, etc.
> >
> >
> > I narrowed the strange issue down to following situation:
> >
> > I reload dmfe.ko (and networkmanager is disabled)
> > I don't ifup the device, thus pretty much no hardware initialization
> > takes place (but this appears not to matter anyway)
> >
> > I then suspend the system, and WOL doesn't work (I have patched the
> > driver to enable WOL automaticly)
> >
> > I then, suspend again. WOL works, and continues to work as long as I
> > don't reload the driver. If I do, same situation repeats.
> >
> > Also, after a boot, WOL works, so a reload cycle triggers that issue.
> >
> > And most importantly, if I don't do a
> >
> > pci_set_power_state(pci_dev, pci_choose_state (pci_dev, state));
> >
> > in .suspend, then WOL always works.
> >
> > and I have even tried to set state manually to PCI_D3hot or PCI_D3cold,
> >
> > I also tried to use pci_save_state
> >
> >
> > I also have 2 copies of this card, and both have this issue.
> > I also tried 2 pci slots.
> >
> > Kernel is vanilla 2.6.31-rc5
I need a dmesg output containing at least one suspend-resume cycle
(preferably two cycles, the first one where WoL worked and another one
where it didn't).
Best,
Rafael
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Michael S. Tsirkin @ 2009-08-17 17:41 UTC (permalink / raw)
To: Gregory Haskins
Cc: Anthony Liguori, Ingo Molnar, Gregory Haskins, kvm, Avi Kivity,
alacrityvm-devel, linux-kernel, netdev
In-Reply-To: <4A8965E0.8050608@gmail.com>
On Mon, Aug 17, 2009 at 10:14:56AM -0400, Gregory Haskins wrote:
> Case in point: Take an upstream kernel and you can modprobe the
> vbus-pcibridge in and virtio devices will work over that transport
> unmodified.
>
> See http://lkml.org/lkml/2009/8/6/244 for details.
The modprobe you are talking about would need
to be done in guest kernel, correct?
> OTOH, Michael's patch is purely targeted at improving virtio-net on kvm,
> and its likewise constrained by various limitations of that decision
> (such as its reliance of the PCI model, and the kvm memory scheme).
vhost is actually not related to PCI in any way. It simply leaves all
setup for userspace to do. And the memory scheme was intentionally
separated from kvm so that it can easily support e.g. lguest.
--
MST
^ permalink raw reply
* Re: Receive side performance issue with multi-10-GigE and NUMA
From: Jesse Barnes @ 2009-08-17 16:53 UTC (permalink / raw)
To: Bill Fink
Cc: Brandeburg, Jesse, Neil Horman, Andrew Gallatin, Brice Goglin,
Linux Network Developers, Yinghai Lu
In-Reply-To: <20090814163155.968872fe.billfink@mindspring.com>
On Fri, 14 Aug 2009 16:31:55 -0400
Bill Fink <billfink@mindspring.com> wrote:
> On Wed, 12 Aug 2009, Bill Fink wrote:
>
> > On Tue, 11 Aug 2009, Brandeburg, Jesse wrote:
> >
> > > Bill Fink wrote:
> > > > On Sat, 8 Aug 2009, Neil Horman wrote:
> > > >
> > > >> On Sat, Aug 08, 2009 at 02:21:36PM -0400, Andrew Gallatin
> > > >> wrote:
> > > >>> Neil Horman wrote:
> > > >>>> On Sat, Aug 08, 2009 at 07:08:20AM -0400, Andrew Gallatin
> > > >>>> wrote:
> > > >>>>> Bill Fink wrote:
> > > >>>>>> On Fri, 07 Aug 2009, Andrew Gallatin wrote:
> > > >>>>>>
> > > >>>>>>> Bill Fink wrote:
> > > >>>>>>>
> > > >>>>>>>> All sysfs local_cpus values are the same
> > > >>>>>>>> (00000000,000000ff), so yes they are also wrong.
> > >
> > > bill, I recently helped Jesse Barnes push a patch that addresses
> > > this kind of issue on CoreI7, the root cause was the numa_node
> > > variable was initialized based on slot on AMD systems, but needed
> > > to be set to -1 by default on systems with a uniform IOH to slot
> > > architecture.
> > >
> > > here is the commit ID:
> > > http://git.kernel.org/?p=linux/kernel/git/sfr/linux-next.git;a=commit;h=3c38
> > > d674be519109696746192943a6d524019f7f
> > >
> > > I'm not sure it is in linus' tree yet, this link is to net-next
> > >
> > > Maybe see if it helps?
> >
> > It's worth a shot.
> >
> > Hopefully I can get a chance to build a new kernel tomorrow to check
> > out some of the suggestions, like this one, the setting of
> > ACPI_DEBUG, and the new ftrace module for checking NUMA affinity of
> > skbs.
>
> I applied this patch to my 2.6.29.6 kernel (from Fedora 11).
>
> Now when I do:
>
> find /sys -name numa_node -exec grep . {} /dev/null \;
>
> the numa_node for _all_ PCI devices is -1.
Yeah, that sounds right (indicates they're not really tied to a
specific node).
> When I do:
>
> find /sys -name local_cpus -exec grep . {} /dev/null \;
>
> I find that local_cpus is always 00000000,00000000.
>
> Is that OK or should it be 00000000,000000ff (for my dual quad-core
> Xeon 5580 system with no hyperthreading)?
Hm, yeah it probably should have the full CPU mask...
> Also, is it just not possible on this type of Intel Xeon system to
> properly associate the PCI devices with the nearest NUMA node?
All the PCI devices hang off the root complex, which is the same
distance to each node of memory (at least that's my understanding for
current platforms).
> In any event, the patch didn't help (or hurt). The transmit
> performance remained at ~100 Gbps while the receive performance
> remained at 55 Gbps.
Maybe the other Jesse has some ideas here.
--
Jesse Barnes, Intel Open Source Technology Center
^ permalink raw reply
* NETROM: Fix use of static buffer
From: Ralf Baechle @ 2009-08-17 16:16 UTC (permalink / raw)
To: netdev, David S. Miller; +Cc: linux-hams
The static variable used by nr_call_to_digi might result in corruption if
multiple threads are trying to usee a node or neighbour via ioctl. Fixed
by having the caller pass a structure in. This is safe because nr_add_node
rsp. nr_add_neigh will allocate a permanent structure, if needed.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
net/netrom/nr_route.c | 21 ++++++++++++---------
1 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/net/netrom/nr_route.c b/net/netrom/nr_route.c
index e943c16..4eb1ac9 100644
--- a/net/netrom/nr_route.c
+++ b/net/netrom/nr_route.c
@@ -630,23 +630,23 @@ out:
return dev;
}
-static ax25_digi *nr_call_to_digi(int ndigis, ax25_address *digipeaters)
+static ax25_digi *nr_call_to_digi(ax25_digi *digi, int ndigis,
+ ax25_address *digipeaters)
{
- static ax25_digi ax25_digi;
int i;
if (ndigis == 0)
return NULL;
for (i = 0; i < ndigis; i++) {
- ax25_digi.calls[i] = digipeaters[i];
- ax25_digi.repeated[i] = 0;
+ digi->calls[i] = digipeaters[i];
+ digi->repeated[i] = 0;
}
- ax25_digi.ndigi = ndigis;
- ax25_digi.lastrepeat = -1;
+ digi->ndigi = ndigis;
+ digi->lastrepeat = -1;
- return &ax25_digi;
+ return digi;
}
/*
@@ -656,6 +656,7 @@ int nr_rt_ioctl(unsigned int cmd, void __user *arg)
{
struct nr_route_struct nr_route;
struct net_device *dev;
+ ax25_digi digi;
int ret;
switch (cmd) {
@@ -673,13 +674,15 @@ int nr_rt_ioctl(unsigned int cmd, void __user *arg)
ret = nr_add_node(&nr_route.callsign,
nr_route.mnemonic,
&nr_route.neighbour,
- nr_call_to_digi(nr_route.ndigis, nr_route.digipeaters),
+ nr_call_to_digi(&digi, nr_route.ndigis,
+ nr_route.digipeaters),
dev, nr_route.quality,
nr_route.obs_count);
break;
case NETROM_NEIGH:
ret = nr_add_neigh(&nr_route.callsign,
- nr_call_to_digi(nr_route.ndigis, nr_route.digipeaters),
+ nr_call_to_digi(&digi, nr_route.ndigis,
+ nr_route.digipeaters),
dev, nr_route.quality);
break;
default:
^ permalink raw reply related
* Re: [PATCH] lib/vsprintf.c: Add "%pI6c" - print pointer as compressed ipv6 address
From: Jens Rosenboom @ 2009-08-17 15:18 UTC (permalink / raw)
To: Joe Perches; +Cc: David Miller, chuck.lever, brian.haley, netdev
In-Reply-To: <1250349894.4620.5.camel@Joe-Laptop.home>
On Sat, 2009-08-15 at 08:24 -0700, Joe Perches wrote:
> On Fri, 2009-08-14 at 13:12 -0700, David Miller wrote:
> > I'd say that kernel log messages are OK to tinker with, whereas procfs
> > and sysfs file contents are not.
>
> Here's a patch to start that tinkering with log messages
Two small optimizations:
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 9b79536..a80ef3d 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -677,12 +677,11 @@ static char *ip6_compressed_string(char *p, const
struct in6_addr *addr)
int j;
int range;
unsigned char zerolength[8];
- int longest = 0;
+ int longest = 1;
int colonpos = -1;
u16 word;
u8 hi;
u8 lo;
- bool printhi;
bool needcolon = false;
bool useIPv4 = ipv6_addr_v4mapped(addr) || ipv6_addr_is_isatap(addr);
@@ -707,8 +706,6 @@ static char *ip6_compressed_string(char *p, const
struct in6_addr *addr)
colonpos = i;
}
}
- if (colonpos != -1 && zerolength[colonpos] < 2)
- colonpos = -1;
for (i = 0; i < range; i++) {
if (i == colonpos) {
@@ -729,15 +726,13 @@ static char *ip6_compressed_string(char *p, const
struct in6_addr *addr)
word = ntohs(addr->s6_addr16[i]);
hi = word >> 8;
lo = word & 0xff;
- printhi = false;
if (hi) {
if (hi > 0x0f)
p = pack_hex_byte(p, hi);
else
*p++ = hex_asc_lo(hi);
- printhi = true;
}
- if (printhi || lo > 0x0f)
+ if (hi || lo > 0x0f)
p = pack_hex_byte(p, lo);
else
*p++ = hex_asc_lo(lo);
Also I'm wondering whether it makes sense to pull the format code
checking into all the sub-routines. It might be easier to maintain if it
is all kept together in pointer().
^ permalink raw reply related
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-17 15:18 UTC (permalink / raw)
To: Gregory Haskins
Cc: Ingo Molnar, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A8972C3.30202@gmail.com>
On 08/17/2009 06:09 PM, Gregory Haskins wrote:
>
>> We've been through this before I believe. If you can point out specific
>> differences that make venet outperform virtio-net I'll be glad to hear
>> (and steal) them though.
>>
>>
> You sure know how to convince someone to collaborate with you, eh?
>
>
If I've offended you, I apologize.
> Unforunately, i've answered that question numerous times, but it
> apparently falls on deaf ears.
>
Well, I'm sorry, I truly don't think I've had that question answered
with specificity. I'm really interested in it (out of a selfish desire
to improve virtio), but the only comment I recall from you was to the
effect that the virtio rings were better than ioq in terms of cache
placement.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Ingo Molnar @ 2009-08-17 15:14 UTC (permalink / raw)
To: Gregory Haskins
Cc: Avi Kivity, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A8972C3.30202@gmail.com>
* Gregory Haskins <gregory.haskins@gmail.com> wrote:
> Avi Kivity wrote:
> > On 08/17/2009 05:16 PM, Gregory Haskins wrote:
> >>> My opinion is that this is a duplication of effort and we'd be better
> >>> off if everyone contributed to enhancing virtio, which already has
> >>> widely deployed guest drivers and non-Linux guest support.
> >>>
> >>> It may have merit if it is proven that it is technically superior to
> >>> virtio (and I don't mean some benchmark in some point in time; I mean
> >>> design wise). So far I haven't seen any indications that it is.
> >>>
> >>>
> >>
> >> The design is very different, so hopefully I can start to convince you
> >> why it might be interesting.
> >>
> >
> > We've been through this before I believe. If you can point out
> > specific differences that make venet outperform virtio-net I'll
> > be glad to hear (and steal) them though.
>
> You sure know how to convince someone to collaborate with you, eh?
>
> Unforunately, i've answered that question numerous times, but it
> apparently falls on deaf ears.
I'm trying to find the relevant discussion. The link you gave in the
previous mail:
http://lkml.org/lkml/2009/4/21/408
does not offer any design analysis of vbus versus virtio, and why
the only fix to virtio is vbus. It offers a comparison and a blanket
statement that vbus is superior but no arguments.
(If you've already explained in a past thread then please give me an
URL to that reply if possible, or forward me that prior reply.
Thanks!)
Ingo
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-17 15:13 UTC (permalink / raw)
To: Gregory Haskins
Cc: Ingo Molnar, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <4A8971A8.2040102@gmail.com>
On 08/17/2009 06:05 PM, Gregory Haskins wrote:
> Hi Ingo,
>
> 1) First off, let me state that I have made every effort to propose this
> as a solution to integrate with KVM, the most recent of which is April:
>
> http://lkml.org/lkml/2009/4/21/408
>
> If you read through the various vbus related threads on LKML/KVM posted
> this year, I think you will see that I made numerous polite offerings to
> work with people on finding a common solution here, including Michael.
>
> In the end, Michael decided that go a different route using some of the
> ideas proposed in vbus + venet-tap to create vhost-net. This is fine,
> and I respect his decision. But do not try to pin "fracturing" on me,
> because I tried everything to avoid it. :)
>
Given your post, there are only three possible ways to continue kvm
guest driver development:
- develop virtio/vhost, drop vbus/venet
- develop vbus/venet, drop virtio
- develop both
Developing both fractures the community. Dropping virtio invalidates
the installed base and Windows effort. There were no strong technical
reasons shown in favor of the remaining option.
> Since I still disagree with the fundamental approach of how KVM IO
> works,
What's that?
> Prior to my effort, KVM was humming along at the status quo and I came
> along with a closer eye and almost doubled the throughput and cut
> latency by 78%. Given an apparent disagreement with aspects of my
> approach, Michael went off and created a counter example that was
> motivated by my performance findings.
>
Oh, virtio-net performance was a thorn in our side for a long time. I
agree that venet was an additional spur.
> Therefore, even if Avi ultimately accepts Michaels vhost approach
> instead of mine, Linux as a hypervisor platform has been significantly
> _improved_ by a little friendly competition, not somehow damaged by it.
>
Certainly, and irqfd/ioeventfd are a net win in any case.
> 4) Lastly, these patches are almost entirely just stand alone Linux
> drivers that do not affect KVM if KVM doesn't wish to acknowledge them.
> Its just like any of the other numerous drivers that are accepted
> upstream into Linux every day. The only maintained subsystem that is
> technically touched by this series is netdev, and David Miller already
> approved of the relevant patch's inclusion:
>
> http://lkml.org/lkml/2009/8/3/505
>
> So with all due respect, where is the problem? The patches are all
> professionally developed according to the Linux coding standards, pass
> checkpatch, are GPL'ed, and work with a freely available platform which
> you can download today
> (http://git.kernel.org/?p=linux/kernel/git/ghaskins/alacrityvm/linux-2.6.git;a=summary)
>
As I mentioned before, I have no technical objections to the patches, I
just wish the effort could be concentrated in one direction.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 15:09 UTC (permalink / raw)
To: Avi Kivity
Cc: Ingo Molnar, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A897057.2050504@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]
Avi Kivity wrote:
> On 08/17/2009 05:16 PM, Gregory Haskins wrote:
>>> My opinion is that this is a duplication of effort and we'd be better
>>> off if everyone contributed to enhancing virtio, which already has
>>> widely deployed guest drivers and non-Linux guest support.
>>>
>>> It may have merit if it is proven that it is technically superior to
>>> virtio (and I don't mean some benchmark in some point in time; I mean
>>> design wise). So far I haven't seen any indications that it is.
>>>
>>>
>>
>> The design is very different, so hopefully I can start to convince you
>> why it might be interesting.
>>
>
> We've been through this before I believe. If you can point out specific
> differences that make venet outperform virtio-net I'll be glad to hear
> (and steal) them though.
>
You sure know how to convince someone to collaborate with you, eh?
Unforunately, i've answered that question numerous times, but it
apparently falls on deaf ears.
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Ingo Molnar @ 2009-08-17 15:08 UTC (permalink / raw)
To: Gregory Haskins
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <4A8971A8.2040102@gmail.com>
* Gregory Haskins <gregory.haskins@gmail.com> wrote:
> Hi Ingo,
>
> 1) First off, let me state that I have made every effort to
> propose this as a solution to integrate with KVM, the most recent
> of which is April:
>
> http://lkml.org/lkml/2009/4/21/408
>
> If you read through the various vbus related threads on LKML/KVM
> posted this year, I think you will see that I made numerous polite
> offerings to work with people on finding a common solution here,
> including Michael.
>
> In the end, Michael decided that go a different route using some
> of the ideas proposed in vbus + venet-tap to create vhost-net.
> This is fine, and I respect his decision. But do not try to pin
> "fracturing" on me, because I tried everything to avoid it. :)
That's good.
So if virtio is fixed to be as fast as vbus, and if there's no other
techical advantages of vbus over virtio you'll be glad to drop vbus
and stand behind virtio?
Also, are you willing to help virtio to become faster? Or do you
have arguments why that is impossible to do so and why the only
possible solution is vbus? Avi says no such arguments were offered
so far.
Ingo
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Ingo Molnar @ 2009-08-17 15:05 UTC (permalink / raw)
To: Avi Kivity
Cc: Gregory Haskins, Anthony Liguori, Gregory Haskins, kvm,
alacrityvm-devel, linux-kernel, netdev, Michael S. Tsirkin
In-Reply-To: <4A896FFE.5050207@redhat.com>
* Avi Kivity <avi@redhat.com> wrote:
> I don't have any technical objections to vbus/venet (I had in the
> past re interrupts but I believe you've addressed them), and it
> appears to perform very well. However I still think we should
> address virtio's shortcomings (as Michael is doing) rather than
> create a competitor. We have enough external competition, we
> don't need in-tree competitors.
I do have strong technical objections: distributions really want to
standardize on as few Linux internal virtualization APIs as
possible, so splintering it just because /bin/cp is easy to do is
bad.
If virtio pulls even with vbus's performance and vbus has no
advantages over virtio i do NAK vbus on that basis. Lets stop the
sillyness before it starts hurting users. Coming up with something
better is good, but doing an incompatible, duplicative framework
just for NIH reasons is stupid and should be resisted.
People dont get to add a new sys_read_v2() without strong technical
arguments either - the same holds for our Linux internal driver
abstractions, APIs and ABIs.
ingo
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 15:05 UTC (permalink / raw)
To: Ingo Molnar
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <20090817142506.GB3602@elte.hu>
[-- Attachment #1: Type: text/plain, Size: 5022 bytes --]
Ingo Molnar wrote:
> * Gregory Haskins <gregory.haskins@gmail.com> wrote:
>
>> Ingo Molnar wrote:
>>> * Gregory Haskins <ghaskins@novell.com> wrote:
>>>
>>>> This will generally be used for hypervisors to publish any host-side
>>>> virtual devices up to a guest. The guest will have the opportunity
>>>> to consume any devices present on the vbus-proxy as if they were
>>>> platform devices, similar to existing buses like PCI.
>>>>
>>>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>>>> ---
>>>>
>>>> MAINTAINERS | 6 ++
>>>> arch/x86/Kconfig | 2 +
>>>> drivers/Makefile | 1
>>>> drivers/vbus/Kconfig | 14 ++++
>>>> drivers/vbus/Makefile | 3 +
>>>> drivers/vbus/bus-proxy.c | 152 +++++++++++++++++++++++++++++++++++++++++++
>>>> include/linux/vbus_driver.h | 73 +++++++++++++++++++++
>>>> 7 files changed, 251 insertions(+), 0 deletions(-)
>>>> create mode 100644 drivers/vbus/Kconfig
>>>> create mode 100644 drivers/vbus/Makefile
>>>> create mode 100644 drivers/vbus/bus-proxy.c
>>>> create mode 100644 include/linux/vbus_driver.h
>>> Is there a consensus on this with the KVM folks? (i've added the KVM
>>> list to the Cc:)
>>>
>>>
>> Hi Ingo,
>>
>> Avi can correct me if I am wrong, but the agreement that he and I
>> came to a few months ago was something to the effect of:
>>
>> kvm will be neutral towards various external IO subsystems, and
>> instead provide various hooks (see irqfd, ioeventfd) to permit
>> these IO subsystems to interface with kvm.
>>
>> AlacrityVM is one of the first projects to take advantage of that
>> interface. AlacrityVM is kvm-core + vbus-core +
>> vbus-kvm-connector + vbus-enhanced qemu + guest drivers. This
>> thread is part of the guest-drivers portion. Note that it is
>> specific to alacrityvm, not kvm, which is why the kvm list was not
>> included in the conversation (also an agreement with Avi:
>> http://lkml.org/lkml/2009/8/6/231).
>
> Well my own opinion is that the fracturing of the Linux internal
> driver space into diverging pieces of duplicate functionality
> (absent compelling technical reasons) is harmful.
[Adding Michael Tsirkin]
Hi Ingo,
1) First off, let me state that I have made every effort to propose this
as a solution to integrate with KVM, the most recent of which is April:
http://lkml.org/lkml/2009/4/21/408
If you read through the various vbus related threads on LKML/KVM posted
this year, I think you will see that I made numerous polite offerings to
work with people on finding a common solution here, including Michael.
In the end, Michael decided that go a different route using some of the
ideas proposed in vbus + venet-tap to create vhost-net. This is fine,
and I respect his decision. But do not try to pin "fracturing" on me,
because I tried everything to avoid it. :)
Since I still disagree with the fundamental approach of how KVM IO
works, I am continuing my effort in the downstream project "AlacrityVM"
which will hopefully serve to build a better understanding of what it is
I am doing with the vbus technology, and a point to maintain the subsystem.
2) There *are* technical reasons for this change (and IMHO, they are
compelling), many of which have already been previously discussed
(including my last reply to Anthony) so I wont rehash them here.
3) Even if there really is some duplication here, I disagree with you
that it is somehow harmful to the Linux community per se. Case in
point, look at the graphs posted on the AlacrityVM wiki:
http://developer.novell.com/wiki/index.php/AlacrityVM
Prior to my effort, KVM was humming along at the status quo and I came
along with a closer eye and almost doubled the throughput and cut
latency by 78%. Given an apparent disagreement with aspects of my
approach, Michael went off and created a counter example that was
motivated by my performance findings.
Therefore, even if Avi ultimately accepts Michaels vhost approach
instead of mine, Linux as a hypervisor platform has been significantly
_improved_ by a little friendly competition, not somehow damaged by it.
4) Lastly, these patches are almost entirely just stand alone Linux
drivers that do not affect KVM if KVM doesn't wish to acknowledge them.
Its just like any of the other numerous drivers that are accepted
upstream into Linux every day. The only maintained subsystem that is
technically touched by this series is netdev, and David Miller already
approved of the relevant patch's inclusion:
http://lkml.org/lkml/2009/8/3/505
So with all due respect, where is the problem? The patches are all
professionally developed according to the Linux coding standards, pass
checkpatch, are GPL'ed, and work with a freely available platform which
you can download today
(http://git.kernel.org/?p=linux/kernel/git/ghaskins/alacrityvm/linux-2.6.git;a=summary)
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-17 14:59 UTC (permalink / raw)
To: Gregory Haskins
Cc: Ingo Molnar, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A896648.9040707@gmail.com>
On 08/17/2009 05:16 PM, Gregory Haskins wrote:
>> My opinion is that this is a duplication of effort and we'd be better
>> off if everyone contributed to enhancing virtio, which already has
>> widely deployed guest drivers and non-Linux guest support.
>>
>> It may have merit if it is proven that it is technically superior to
>> virtio (and I don't mean some benchmark in some point in time; I mean
>> design wise). So far I haven't seen any indications that it is.
>>
>>
>
> The design is very different, so hopefully I can start to convince you
> why it might be interesting.
>
We've been through this before I believe. If you can point out specific
differences that make venet outperform virtio-net I'll be glad to hear
(and steal) them though.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-17 14:58 UTC (permalink / raw)
To: Gregory Haskins
Cc: Anthony Liguori, Ingo Molnar, Gregory Haskins, kvm,
alacrityvm-devel, linux-kernel, netdev, Michael S. Tsirkin
In-Reply-To: <4A8965E0.8050608@gmail.com>
On 08/17/2009 05:14 PM, Gregory Haskins wrote:
> Note: No one has ever proposed to change the virtio-ABI. In fact, this
> thread in question doesn't even touch virtio, and even the patches that
> I have previously posted to add virtio-capability do it in a backwards
> compatible way
>
Your patches include venet, which is a direct competitor to virtio-net,
so it splits the development effort.
> Case in point: Take an upstream kernel and you can modprobe the
> vbus-pcibridge in and virtio devices will work over that transport
> unmodified.
>
Older kernels don't support it, and Windows doesn't support it.
>> vbus
>> does things in different ways (paravirtual bus vs. pci for discovery)
>> but I think we're happy with how virtio does things today.
>>
>>
> Thats fine. KVM can stick with virtio-pci if it wants. AlacrityVM will
> support virtio-pci and vbus (with possible convergence with
> virtio-vbus). If at some point KVM thinks vbus is interesting, I will
> gladly work with getting it integrated into upstream KVM as well. Until
> then, they can happily coexist without issue between the two projects.
>
If vbus is to go upstream, it must go via the same path other drivers
go. Virtio wasn't merged via the kvm tree and virtio-host won't be either.
I don't have any technical objections to vbus/venet (I had in the past
re interrupts but I believe you've addressed them), and it appears to
perform very well. However I still think we should address virtio's
shortcomings (as Michael is doing) rather than create a competitor. We
have enough external competition, we don't need in-tree competitors.
>> I think the reason vbus gets better performance for networking today is
>> that vbus' backends are in the kernel while virtio's backends are
>> currently in userspace.
>>
> Well, with all due respect, you also said initially when I announced
> vbus that in-kernel doesn't matter, and tried to make virtio-net run as
> fast as venet from userspace ;) Given that we never saw those userspace
> patches from you that in fact equaled my performance, I assume you were
> wrong about that statement.
I too thought that if we'd improved the userspace interfaces we'd get
fast networking without pushing virtio details into the kernels,
benefiting not just kvm but the Linux community at large. This might
still be correct but in fact no one turned up with the patches. Maybe
they're impossible to write, hard to write, or uninteresting to write
for those who are capable of writing them. As it is, we've given up and
Michael wrote vhost.
> Perhaps you were wrong about other things too?
>
I'm pretty sure Anthony doesn't posses a Diploma of Perpetual Omniscience.
>> Since Michael has a functioning in-kernel
>> backend for virtio-net now, I suspect we're weeks (maybe days) away from
>> performance results. My expectation is that vhost + virtio-net will be
>> as good as venet + vbus.
>>
> This is not entirely impossible, at least for certain simple benchmarks
> like singleton throughput and latency.
What about more complex benchmarks? Do you thing vbus+venet has an
advantage there?
> But if you think that this
> somehow invalidates vbus as a concept, you have missed the point entirely.
>
> vbus is about creating a flexible (e.g. cross hypervisor, and even
> physical system or userspace application) in-kernel IO containers with
> linux. The "guest" interface represents what I believe to be the ideal
> interface for ease of use, yet maximum performance for
> software-to-software interaction.
Maybe. But layering venet or vblock on top of it makes it specific to
hypervisors. The venet/vblock ABIs are not very interesting for
user-to-user (and anyway, they could use virtio just as well).
> venet was originally crafted just to validate the approach and test the
> vbus interface. It ended up being so much faster that virtio-net, that
> people in the vbus community started coding against its ABI.
It ended up being much faster than qemu's host implementation, not the
virtio ABI. When asked you've indicated that you don't see any
deficiencies in the virtio protocol.
> OTOH, Michael's patch is purely targeted at improving virtio-net on kvm,
> and its likewise constrained by various limitations of that decision
> (such as its reliance of the PCI model, and the kvm memory scheme). The
> tradeoff is that his approach will work in all existing virtio-net kvm
> guests, and is probably significantly less code since he can re-use the
> qemu PCI bus model.
>
virtio does not depend on PCI and virtio-host does not either.
> Conversely, I am not afraid of requiring a new driver to optimize the
> general PV interface. In the long term, this will reduce the amount of
> reimplementing the same code over and over, reduce system overhead, and
> it adds new features not previously available (for instance, coalescing
> and prioritizing interrupts).
>
If it were proven to me a new driver is needed I'd switch too. So far
no proof has materialized.
>> If that's the case, then I don't see any
>> reason to adopt vbus unless Greg things there are other compelling
>> features over virtio.
>>
> Aside from the fact that this is another confusion of the vbus/virtio
> relationship...yes, of course there are compelling features (IMHO) or I
> wouldn't be expending effort ;) They are at least compelling enough to
> put in AlacrityVM. If upstream KVM doesn't want them, that's KVMs
> decision and I am fine with that. Simply never apply my qemu patches to
> qemu-kvm.git, and KVM will be blissfully unaware if vbus is present. I
> do hope that I can convince the KVM community otherwise, however. :)
>
If the vbus patches make it into the kernel I see no reason not to
support them in qemu. qemu supports dozens if not hundreds of devices,
one more wouldn't matter.
But there's a lot of work before that can happen; for example you must
support save/restore/migrate for vbus to be mergable.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: question about drivers/atm/firestream.c
From: Chas Williams (CONTRACTOR) @ 2009-08-17 14:25 UTC (permalink / raw)
To: Julia Lawall; +Cc: linux-atm-general, netdev
In-Reply-To: <Pine.LNX.4.64.0908151906020.20096@ask.diku.dk>
In message <Pine.LNX.4.64.0908151906020.20096@ask.diku.dk>,Julia Lawall writes:
>In the file drivers/atm/firestream.c in the function fs_open, there is the
>following code:
>
> if (vci != ATM_VPI_UNSPEC && vpi != ATM_VCI_UNSPEC)
> set_bit(ATM_VF_ADDR, &atm_vcc->flags);
>
>Should ATM_VPI_UNSPEC and ATM_VCI_UNSPEC be exchanged? They have the same
>value, but the code looks a bit odd this way nonetheless.
yes this is backwards. this whole UNSPEC business should really go away.
drivers should never need to worry about getting an unspecified vpi or
vci. this should be chosen by the next layer up in the stack.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Ingo Molnar @ 2009-08-17 14:25 UTC (permalink / raw)
To: Gregory Haskins
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A8954F0.3040402@gmail.com>
* Gregory Haskins <gregory.haskins@gmail.com> wrote:
> Ingo Molnar wrote:
> > * Gregory Haskins <ghaskins@novell.com> wrote:
> >
> >> This will generally be used for hypervisors to publish any host-side
> >> virtual devices up to a guest. The guest will have the opportunity
> >> to consume any devices present on the vbus-proxy as if they were
> >> platform devices, similar to existing buses like PCI.
> >>
> >> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
> >> ---
> >>
> >> MAINTAINERS | 6 ++
> >> arch/x86/Kconfig | 2 +
> >> drivers/Makefile | 1
> >> drivers/vbus/Kconfig | 14 ++++
> >> drivers/vbus/Makefile | 3 +
> >> drivers/vbus/bus-proxy.c | 152 +++++++++++++++++++++++++++++++++++++++++++
> >> include/linux/vbus_driver.h | 73 +++++++++++++++++++++
> >> 7 files changed, 251 insertions(+), 0 deletions(-)
> >> create mode 100644 drivers/vbus/Kconfig
> >> create mode 100644 drivers/vbus/Makefile
> >> create mode 100644 drivers/vbus/bus-proxy.c
> >> create mode 100644 include/linux/vbus_driver.h
> >
> > Is there a consensus on this with the KVM folks? (i've added the KVM
> > list to the Cc:)
> >
> >
>
> Hi Ingo,
>
> Avi can correct me if I am wrong, but the agreement that he and I
> came to a few months ago was something to the effect of:
>
> kvm will be neutral towards various external IO subsystems, and
> instead provide various hooks (see irqfd, ioeventfd) to permit
> these IO subsystems to interface with kvm.
>
> AlacrityVM is one of the first projects to take advantage of that
> interface. AlacrityVM is kvm-core + vbus-core +
> vbus-kvm-connector + vbus-enhanced qemu + guest drivers. This
> thread is part of the guest-drivers portion. Note that it is
> specific to alacrityvm, not kvm, which is why the kvm list was not
> included in the conversation (also an agreement with Avi:
> http://lkml.org/lkml/2009/8/6/231).
Well my own opinion is that the fracturing of the Linux internal
driver space into diverging pieces of duplicate functionality
(absent compelling technical reasons) is harmful.
Ingo
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Ingo Molnar @ 2009-08-17 14:23 UTC (permalink / raw)
To: Anthony Liguori
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <4A896112.9030407@codemonkey.ws>
* Anthony Liguori <anthony@codemonkey.ws> wrote:
> Ingo Molnar wrote:
>>> I think the reason vbus gets better performance for networking today
>>> is that vbus' backends are in the kernel while virtio's backends are
>>> currently in userspace. Since Michael has a functioning in-kernel
>>> backend for virtio-net now, I suspect we're weeks (maybe days) away
>>> from performance results. My expectation is that vhost + virtio-net
>>> will be as good as venet + vbus. If that's the case, then I don't
>>> see any reason to adopt vbus unless Greg things there are other
>>> compelling features over virtio.
>>>
>>
>> Keeping virtio's backend in user-space was rather stupid IMHO.
>
> I don't think it's quite so clear.
in such a narrow quote it's not so clear indeed - that's why i
qualified it with:
>> Having the _option_ to piggyback to user-space (for flexibility,
>> extensibility, etc.) is OK, but not having kernel acceleration is
>> bad.
Ingo
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 14:16 UTC (permalink / raw)
To: Avi Kivity
Cc: Ingo Molnar, Gregory Haskins, kvm, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <4A87C3B9.2040206@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]
Avi Kivity wrote:
> On 08/15/2009 01:32 PM, Ingo Molnar wrote:
>>> This will generally be used for hypervisors to publish any host-side
>>> virtual devices up to a guest. The guest will have the opportunity
>>> to consume any devices present on the vbus-proxy as if they were
>>> platform devices, similar to existing buses like PCI.
>>>
>>>
>> Is there a consensus on this with the KVM folks? (i've added the KVM
>> list to the Cc:)
>>
>
> My opinion is that this is a duplication of effort and we'd be better
> off if everyone contributed to enhancing virtio, which already has
> widely deployed guest drivers and non-Linux guest support.
>
> It may have merit if it is proven that it is technically superior to
> virtio (and I don't mean some benchmark in some point in time; I mean
> design wise). So far I haven't seen any indications that it is.
>
The design is very different, so hopefully I can start to convince you
why it might be interesting.
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 14:14 UTC (permalink / raw)
To: Anthony Liguori
Cc: Ingo Molnar, Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel,
linux-kernel, netdev, Michael S. Tsirkin
In-Reply-To: <4A870964.9090408@codemonkey.ws>
[-- Attachment #1: Type: text/plain, Size: 6816 bytes --]
Anthony Liguori wrote:
> Ingo Molnar wrote:
>> * Gregory Haskins <ghaskins@novell.com> wrote:
>>
>>
>>> This will generally be used for hypervisors to publish any host-side
>>> virtual devices up to a guest. The guest will have the opportunity
>>> to consume any devices present on the vbus-proxy as if they were
>>> platform devices, similar to existing buses like PCI.
>>>
>>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>>> ---
>>>
>>> MAINTAINERS | 6 ++
>>> arch/x86/Kconfig | 2 +
>>> drivers/Makefile | 1 drivers/vbus/Kconfig |
>>> 14 ++++
>>> drivers/vbus/Makefile | 3 +
>>> drivers/vbus/bus-proxy.c | 152
>>> +++++++++++++++++++++++++++++++++++++++++++
>>> include/linux/vbus_driver.h | 73 +++++++++++++++++++++
>>> 7 files changed, 251 insertions(+), 0 deletions(-)
>>> create mode 100644 drivers/vbus/Kconfig
>>> create mode 100644 drivers/vbus/Makefile
>>> create mode 100644 drivers/vbus/bus-proxy.c
>>> create mode 100644 include/linux/vbus_driver.h
>>>
>>
>> Is there a consensus on this with the KVM folks? (i've added the KVM
>> list to the Cc:)
>>
>
> I'll let Avi comment about it from a KVM perspective but from a QEMU
> perspective, I don't think we want to support two paravirtual IO
> frameworks. I'd like to see them converge. Since there's an install
> base of guests today with virtio drivers, there really ought to be a
> compelling reason to change the virtio ABI in a non-backwards compatible
> way.
Note: No one has ever proposed to change the virtio-ABI. In fact, this
thread in question doesn't even touch virtio, and even the patches that
I have previously posted to add virtio-capability do it in a backwards
compatible way
Case in point: Take an upstream kernel and you can modprobe the
vbus-pcibridge in and virtio devices will work over that transport
unmodified.
See http://lkml.org/lkml/2009/8/6/244 for details.
Note that I have tentatively dropped the virtio-vbus patch from the
queue due to lack of interest, but I can resurrect it if need be.
> This means convergence really ought to be adding features to virtio.
virtio is a device model. vbus is a bus model and a host backend
facility. Adding features to virtio would be orthogonal to some kind of
convergence goal. virtio can run unmodified or add new features within
its own namespace independent of vbus, as it pleases. vbus will simply
transport those changes.
>
> On paper, I don't think vbus really has any features over virtio.
Again, do not confuse vbus with virtio. They are different layers of
the stack.
> vbus
> does things in different ways (paravirtual bus vs. pci for discovery)
> but I think we're happy with how virtio does things today.
>
Thats fine. KVM can stick with virtio-pci if it wants. AlacrityVM will
support virtio-pci and vbus (with possible convergence with
virtio-vbus). If at some point KVM thinks vbus is interesting, I will
gladly work with getting it integrated into upstream KVM as well. Until
then, they can happily coexist without issue between the two projects.
> I think the reason vbus gets better performance for networking today is
> that vbus' backends are in the kernel while virtio's backends are
> currently in userspace.
Well, with all due respect, you also said initially when I announced
vbus that in-kernel doesn't matter, and tried to make virtio-net run as
fast as venet from userspace ;) Given that we never saw those userspace
patches from you that in fact equaled my performance, I assume you were
wrong about that statement. Perhaps you were wrong about other things too?
> Since Michael has a functioning in-kernel
> backend for virtio-net now, I suspect we're weeks (maybe days) away from
> performance results. My expectation is that vhost + virtio-net will be
> as good as venet + vbus.
This is not entirely impossible, at least for certain simple benchmarks
like singleton throughput and latency. But if you think that this
somehow invalidates vbus as a concept, you have missed the point entirely.
vbus is about creating a flexible (e.g. cross hypervisor, and even
physical system or userspace application) in-kernel IO containers with
linux. The "guest" interface represents what I believe to be the ideal
interface for ease of use, yet maximum performance for
software-to-software interaction. This means very low latency and
high-throughput for both synchronous and asynchronous IO, minimizing
enters/exits, reducing enter/exit cost, prioritization, parallel
computation, etc. The things that we (the alacrityvm community) have
coming down the pipeline for high-performance virtualization require
that these issues be addressed.
venet was originally crafted just to validate the approach and test the
vbus interface. It ended up being so much faster that virtio-net, that
people in the vbus community started coding against its ABI. Therefore,
I decided to support it formally and indefinately. If I can get
consensus on virtio-vbus going forward, it will probably be the last
vbus-specific driver for which there is overlap with virtio (e.g.
virtio-block, virtio-console, etc). Instead, you will only see native
vbus devices for non-native virtio type things, like real-time and
advanced fabric support.
OTOH, Michael's patch is purely targeted at improving virtio-net on kvm,
and its likewise constrained by various limitations of that decision
(such as its reliance of the PCI model, and the kvm memory scheme). The
tradeoff is that his approach will work in all existing virtio-net kvm
guests, and is probably significantly less code since he can re-use the
qemu PCI bus model.
Conversely, I am not afraid of requiring a new driver to optimize the
general PV interface. In the long term, this will reduce the amount of
reimplementing the same code over and over, reduce system overhead, and
it adds new features not previously available (for instance, coalescing
and prioritizing interrupts).
> If that's the case, then I don't see any
> reason to adopt vbus unless Greg things there are other compelling
> features over virtio.
Aside from the fact that this is another confusion of the vbus/virtio
relationship...yes, of course there are compelling features (IMHO) or I
wouldn't be expending effort ;) They are at least compelling enough to
put in AlacrityVM. If upstream KVM doesn't want them, that's KVMs
decision and I am fine with that. Simply never apply my qemu patches to
qemu-kvm.git, and KVM will be blissfully unaware if vbus is present. I
do hope that I can convince the KVM community otherwise, however. :)
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: System freeze on reboot - general protection fault
From: Patrick McHardy @ 2009-08-17 14:03 UTC (permalink / raw)
To: Eric Dumazet
Cc: Zdenek Kabelac, Christoph Lameter, Robin Holt,
Linux Kernel Mailing List, Pekka Enberg, Jesper Dangaard Brouer,
Linux Netdev List, Netfilter Developers
In-Reply-To: <4A87CE60.4020506@gmail.com>
Eric Dumazet wrote:
> Zdenek Kabelac a écrit :
>> [<ffffffffa02c502f>] nf_conntrack_ftp_fini+0x2f/0x70 [nf_conntrack_ftp]
>> [<ffffffff8027bcc5>] sys_delete_module+0x1a5/0x270
>> [<ffffffff8020d329>] ? retint_swapgs+0xe/0x13
>> [<ffffffff80271bf2>] ? trace_hardirqs_on_caller+0x162/0x1b0
>> [<ffffffff80292121>] ? audit_syscall_entry+0x191/0x1c0
>> [<ffffffff80526dae>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>> [<ffffffff8020c84b>] system_call_fastpath+0x16/0x1b
>> Code: c6 00 00 0f 82 66 ff ff ff 49 8b 9e d8 05 00 00 48 85 db 75 16
>> e9 8e 00 00 00 0f 1f 44 00 00 48 85 c0 0f 84 80 00 00 00 48 89 c3 <0f>
>> b6 4b 37 48 8b 03 48 8d 14 cd 00 00 00 00 0f 18 08 48 29 ca
>> RIP [<ffffffffa02b2c2c>] nf_conntrack_helper_unregister+0x16c/0x320
>> [nf_conntrack]
>> RSP <ffff88013982fe68>
>> CR2: 0000000000000038
>> ---[ end trace bc3a0ede3d0084db ]---
>>
> I am currently traveling and wont be able to help you before next week.
>
> I added netdev, Patrick, and netfilter-devel in CC so that more eyes can take a look.
Thanks for the report, I'll have a look at this. Zdenek, please
send me the nf_conntrack.ko file used in the above oops. Thanks.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Anthony Liguori @ 2009-08-17 13:54 UTC (permalink / raw)
To: Ingo Molnar
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <20090816071607.GB29537@elte.hu>
Ingo Molnar wrote:
>> I think the reason vbus gets better performance for networking
>> today is that vbus' backends are in the kernel while virtio's
>> backends are currently in userspace. Since Michael has a
>> functioning in-kernel backend for virtio-net now, I suspect we're
>> weeks (maybe days) away from performance results. My expectation
>> is that vhost + virtio-net will be as good as venet + vbus. If
>> that's the case, then I don't see any reason to adopt vbus unless
>> Greg things there are other compelling features over virtio.
>>
>
> Keeping virtio's backend in user-space was rather stupid IMHO.
>
I don't think it's quite so clear.
There's nothing about vhost_net that would prevent a userspace
application from using it as a higher performance replacement for tun/tap.
The fact that we can avoid userspace for most of the fast paths is nice
but that's really an issue of vhost_net vs. tun/tap.
From the kernel's perspective, a KVM guest is just a userspace
process. Having new userspace interfaces that are only useful to KVM
guests would be a bad thing.
Regards,
Anthony Liguori
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-17 13:02 UTC (permalink / raw)
To: Ingo Molnar
Cc: Gregory Haskins, kvm, Avi Kivity, alacrityvm-devel, linux-kernel,
netdev
In-Reply-To: <20090815103243.GA26749@elte.hu>
[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]
Ingo Molnar wrote:
> * Gregory Haskins <ghaskins@novell.com> wrote:
>
>> This will generally be used for hypervisors to publish any host-side
>> virtual devices up to a guest. The guest will have the opportunity
>> to consume any devices present on the vbus-proxy as if they were
>> platform devices, similar to existing buses like PCI.
>>
>> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
>> ---
>>
>> MAINTAINERS | 6 ++
>> arch/x86/Kconfig | 2 +
>> drivers/Makefile | 1
>> drivers/vbus/Kconfig | 14 ++++
>> drivers/vbus/Makefile | 3 +
>> drivers/vbus/bus-proxy.c | 152 +++++++++++++++++++++++++++++++++++++++++++
>> include/linux/vbus_driver.h | 73 +++++++++++++++++++++
>> 7 files changed, 251 insertions(+), 0 deletions(-)
>> create mode 100644 drivers/vbus/Kconfig
>> create mode 100644 drivers/vbus/Makefile
>> create mode 100644 drivers/vbus/bus-proxy.c
>> create mode 100644 include/linux/vbus_driver.h
>
> Is there a consensus on this with the KVM folks? (i've added the KVM
> list to the Cc:)
>
>
Hi Ingo,
Avi can correct me if I am wrong, but the agreement that he and I came
to a few months ago was something to the effect of:
kvm will be neutral towards various external IO subsystems, and instead
provide various hooks (see irqfd, ioeventfd) to permit these IO
subsystems to interface with kvm.
AlacrityVM is one of the first projects to take advantage of that
interface. AlacrityVM is kvm-core + vbus-core + vbus-kvm-connector +
vbus-enhanced qemu + guest drivers. This thread is part of the
guest-drivers portion. Note that it is specific to alacrityvm, not kvm,
which is why the kvm list was not included in the conversation (also an
agreement with Avi: http://lkml.org/lkml/2009/8/6/231).
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox