* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 6:40 UTC (permalink / raw)
To: Gregory Haskins, Avi Kivity
Cc: Anthony Liguori, Ingo Molnar, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8B9241.20300@redhat.com>
>>> On 8/19/2009 at 1:48 AM, in message <4A8B9241.20300@redhat.com>, Avi Kivity
<avi@redhat.com> wrote:
> On 08/19/2009 08:36 AM, Gregory Haskins wrote:
>>> If virtio net in guest could be improved instead, everyone would
>>> benefit.
>>>
>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>> connector, you are happy?
>>
>
> This doesn't improve virtio-net in any way.
Any why not? (Did you notice I said "PCI compliant", i.e. over virtio-pci)
>
>>> I am doing this, and I wish more people would join. Instead,
>>> you change ABI in a incompatible way.
>>>
>> Only by choice of my particular connector. The ABI is a function of the
>> connector design. So one such model is to terminate the connector in
>> qemu, and surface the resulting objects as PCI devices. I choose not to
>> use this particular design for my connector that I am pushing upstream
>> because I am of the opinion that I can do better by terminating it in
>> the guest directly as a PV optimized bus. However, both connectors can
>> theoretically coexist peacefully.
>>
>
> virtio already supports this model; see lguest and s390. Transporting
> virtio over vbus and vbus over something else doesn't gain anything over
> directly transporting virtio over that something else.
This is not what I am advocating.
Kind Regards,
-Greg
^ permalink raw reply
* Re: [PATCH] net: Rename MAC platform driver for w90p910 platform
From: David Miller @ 2009-08-19 6:35 UTC (permalink / raw)
To: mcuos.com; +Cc: linux-arm-kernel, netdev
In-Reply-To: <4A88D5E8.9070708@gmail.com>
From: Wan ZongShun <mcuos.com@gmail.com>
Date: Mon, 17 Aug 2009 12:00:40 +0800
> Due to I modified the corresponding platform device name,
> so I make the patch to rename MAC platform driver
> for w90p910 platform.
>
> Signed-off-by: Wan ZongShun <mcuos.com@gmail.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH] revert TCP retransmission backoff on ICMP destination unreachable
From: David Miller @ 2009-08-19 6:29 UTC (permalink / raw)
To: damian; +Cc: ilpo.jarvinen, netdev
In-Reply-To: <4A8AE771.1070308@tvk.rwth-aachen.de>
From: Damian Lukowski <damian@tvk.rwth-aachen.de>
Date: Tue, 18 Aug 2009 19:40:01 +0200
>> On Fri, 14 Aug 2009, Damian Lukowski wrote:
>>
>>>> Longer than 80 columns, and use an inline function instead
>>>> of a macro in order to get proper type checking.
>>>> [...]
>>>> Do not break up the function local variables with spurious new lines
>>>> like this, please.
>>>> [...]
>>>> The indentation and tabbing is messed up in all of the code you are
>>>> adding, please fix it up to be consistent with the surrounding code
>>>> and the rest of the TCP stack.
>>>>
>>>> Do not use C++ style // comments.
>>>
>>> Better?
>>
>> Please, include the changelog message on resubmits too next time.
>
> I'm sorry, which message do you mean? I used plain diff without GIT or
> anything.
He means the commit message.
When you fix up patches, don't just reply and say "here's the fixed
up patch". That doesn't work.
We need the full context, the full commit message, and your signoffs,
when you submit any patch you intend us to consider applying.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 6:28 UTC (permalink / raw)
To: Avi Kivity
Cc: Ingo Molnar, kvm, alacrityvm-devel, linux-kernel, netdev,
Michael S. Tsirkin
In-Reply-To: <4A8AD678.7050609@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 20993 bytes --]
Avi Kivity wrote:
> On 08/18/2009 05:46 PM, Gregory Haskins wrote:
>>
>>> Can you explain how vbus achieves RDMA?
>>>
>>> I also don't see the connection to real time guests.
>>>
>> Both of these are still in development. Trying to stay true to the
>> "release early and often" mantra, the core vbus technology is being
>> pushed now so it can be reviewed. Stay tuned for these other
>> developments.
>>
>
> Hopefully you can outline how it works. AFAICT, RDMA and kernel bypass
> will need device assignment. If you're bypassing the call into the host
> kernel, it doesn't really matter how that call is made, does it?
This is for things like the setup of queue-pairs, and the transport of
door-bells, and ib-verbs. I am not on the team doing that work, so I am
not an expert in this area. What I do know is having a flexible and
low-latency signal-path was deemed a key requirement.
For real-time, a big part of it is relaying the guest scheduler state to
the host, but in a smart way. For instance, the cpu priority for each
vcpu is in a shared-table. When the priority is raised, we can simply
update the table without taking a VMEXIT. When it is lowered, we need
to inform the host of the change in case the underlying task needs to
reschedule.
This is where the really fast call() type mechanism is important.
Its also about having the priority flow-end to end, and having the vcpu
interrupt state affect the task-priority, etc (e.g. pending interrupts
affect the vcpu task prio).
etc, etc.
I can go on and on (as you know ;), but will wait till this work is more
concrete and proven.
>
>>>> I also designed it in such a way that
>>>> we could, in theory, write one set of (linux-based) backends, and have
>>>> them work across a variety of environments (such as containers/VMs like
>>>> KVM, lguest, openvz, but also physical systems like blade enclosures
>>>> and
>>>> clusters, or even applications running on the host).
>>>>
>>>>
>>> Sorry, I'm still confused. Why would openvz need vbus?
>>>
>> Its just an example. The point is that I abstracted what I think are
>> the key points of fast-io, memory routing, signal routing, etc, so that
>> it will work in a variety of (ideally, _any_) environments.
>>
>> There may not be _performance_ motivations for certain classes of VMs
>> because they already have decent support, but they may want a connector
>> anyway to gain some of the new features available in vbus.
>>
>> And looking forward, the idea is that we have commoditized the backend
>> so we don't need to redo this each time a new container comes along.
>>
>
> I'll wait until a concrete example shows up as I still don't understand.
Ok.
>
>>> One point of contention is that this is all managementy stuff and should
>>> be kept out of the host kernel. Exposing shared memory, interrupts, and
>>> guest hypercalls can all be easily done from userspace (as virtio
>>> demonstrates). True, some devices need kernel acceleration, but that's
>>> no reason to put everything into the host kernel.
>>>
>> See my last reply to Anthony. My two points here are that:
>>
>> a) having it in-kernel makes it a complete subsystem, which perhaps has
>> diminished value in kvm, but adds value in most other places that we are
>> looking to use vbus.
>>
>
> It's not a complete system unless you want users to administer VMs using
> echo and cat and configfs. Some userspace support will always be
> necessary.
Well, more specifically, it doesn't require a userspace app to hang
around. For instance, you can set up your devices with udev scripts, or
whatever.
But that is kind of a silly argument, since the kernel always needs
userspace around to give it something interesting, right? ;)
Basically, what it comes down to is both vbus and vhost need
configuration/management. Vbus does it with sysfs/configfs, and vhost
does it with ioctls. I ultimately decided to go with sysfs/configfs
because, at least that the time I looked, it seemed like the "blessed"
way to do user->kernel interfaces.
>
>> b) the in-kernel code is being overstated as "complex". We are not
>> talking about your typical virt thing, like an emulated ICH/PCI chipset.
>> Its really a simple list of devices with a handful of attributes. They
>> are managed using established linux interfaces, like sysfs/configfs.
>>
>
> They need to be connected to the real world somehow. What about
> security? can any user create a container and devices and link them to
> real interfaces? If not, do you need to run the VM as root?
Today it has to be root as a result of weak mode support in configfs, so
you have me there. I am looking for help patching this limitation, though.
Also, venet-tap uses a bridge, which of course is not as slick as a
raw-socket w.r.t. perms.
>
> virtio and vhost-net solve these issues. Does vbus?
>
> The code may be simple to you. But the question is whether it's
> necessary, not whether it's simple or complex.
>
>>> Exposing devices as PCI is an important issue for me, as I have to
>>> consider non-Linux guests.
>>>
>> Thats your prerogative, but obviously not everyone agrees with you.
>>
>
> I hope everyone agrees that it's an important issue for me and that I
> have to consider non-Linux guests. I also hope that you're considering
> non-Linux guests since they have considerable market share.
I didn't mean non-Linux guests are not important. I was disagreeing
with your assertion that it only works if its PCI. There are numerous
examples of IHV/ISV "bridge" implementations deployed in Windows, no?
If vbus is exposed as a PCI-BRIDGE, how is this different?
>
>> Getting non-Linux guests to work is my problem if you chose to not be
>> part of the vbus community.
>>
>
> I won't be writing those drivers in any case.
Ok.
>
>>> Another issue is the host kernel management code which I believe is
>>> superfluous.
>>>
>> In your opinion, right?
>>
>
> Yes, this is why I wrote "I believe".
Fair enough.
>
>
>>> Given that, why spread to a new model?
>>>
>> Note: I haven't asked you to (at least, not since April with the vbus-v3
>> release). Spreading to a new model is currently the role of the
>> AlacrityVM project, since we disagree on the utility of a new model.
>>
>
> Given I'm not the gateway to inclusion of vbus/venet, you don't need to
> ask me anything. I'm still free to give my opinion.
Agreed, and I didn't mean to suggest otherwise. It not clear if you are
wearing the "kvm maintainer" hat, or the "lkml community member" hat at
times, so its important to make that distinction. Otherwise, its not
clear if this is edict as my superior, or input as my peer. ;)
>
>>>> A) hardware can only generate byte/word sized requests at a time
>>>> because
>>>> that is all the pcb-etch and silicon support. So hardware is usually
>>>> expressed in terms of some number of "registers".
>>>>
>>>>
>>> No, hardware happily DMAs to and fro main memory.
>>>
>> Yes, now walk me through how you set up DMA to do something like a call
>> when you do not know addresses apriori. Hint: count the number of
>> MMIO/PIOs you need. If the number is> 1, you've lost.
>>
>
> With virtio, the number is 1 (or less if you amortize). Set up the ring
> entries and kick.
Again, I am just talking about basic PCI here, not the things we build
on top.
The point is: the things we build on top have costs associated with
them, and I aim to minimize it. For instance, to do a "call()" kind of
interface, you generally need to pre-setup some per-cpu mappings so that
you can just do a single iowrite32() to kick the call off. Those
per-cpu mappings have a cost if you want them to be high-performance, so
my argument is that you ideally want to limit the number of times you
have to do this. My current design reduces this to "once".
>
>>> Some hardware of
>>> course uses mmio registers extensively, but not virtio hardware. With
>>> the recent MSI support no registers are touched in the fast path.
>>>
>> Note we are not talking about virtio here. Just raw PCI and why I
>> advocate vbus over it.
>>
>
> There's no such thing as raw PCI. Every PCI device has a protocol. The
> protocol virtio chose is optimized for virtualization.
And its a question of how that protocol scales, more than how the
protocol works.
Obviously the general idea of the protocol works, as vbus itself is
implemented as a PCI-BRIDGE and is therefore limited to the underlying
characteristics that I can get out of PCI (like PIO latency).
>
>
>>>> D) device-ids are in a fixed width register and centrally assigned from
>>>> an authority (e.g. PCI-SIG).
>>>>
>>>>
>>> That's not an issue either. Qumranet/Red Hat has donated a range of
>>> device IDs for use in virtio.
>>>
>> Yes, and to get one you have to do what? Register it with kvm.git,
>> right? Kind of like registering a MAJOR/MINOR, would you agree? Maybe
>> you do not mind (especially given your relationship to kvm.git), but
>> there are disadvantages to that model for most of the rest of us.
>>
>
> Send an email, it's not that difficult. There's also an experimental
> range.
Ugly....
>
>>> Device IDs are how devices are associated
>>> with drivers, so you'll need something similar for vbus.
>>>
>> Nope, just like you don't need to do anything ahead of time for using a
>> dynamic misc-device name. You just have both the driver and device know
>> what they are looking for (its part of the ABI).
>>
>
> If you get a device ID clash, you fail. If you get a device name clash,
> you fail in the same way.
No argument here.
>
>>>> E) Interrupt/MSI routing is per-device oriented
>>>>
>>>>
>>> Please elaborate. What is the issue? How does vbus solve it?
>>>
>> There are no "interrupts" in vbus..only shm-signals. You can establish
>> an arbitrary amount of shm regions, each with an optional shm-signal
>> associated with it. To do this, the driver calls dev->shm(), and you
>> get back a shm_signal object.
>>
>> Underneath the hood, the vbus-connector (e.g. vbus-pcibridge) decides
>> how it maps real interrupts to shm-signals (on a system level, not per
>> device). This can be 1:1, or any other scheme. vbus-pcibridge uses one
>> system-wide interrupt per priority level (today this is 8 levels), each
>> with an IOQ based event channel. "signals" come as an event on that
>> channel.
>>
>> So the "issue" is that you have no real choice with PCI. You just get
>> device oriented interrupts. With vbus, its abstracted. So you can
>> still get per-device standard MSI, or you can do fancier things like do
>> coalescing and prioritization.
>>
>
> As I've mentioned before, prioritization is available on x86
But as Ive mentioned, it doesn't work very well.
>, and coalescing scales badly.
Depends on what is scaling. Scaling vcpus? Yes, you are right.
Scaling the number of devices? No, this is where it improves.
>
>>>> F) Interrupts/MSI are assumed cheap to inject
>>>>
>>>>
>>> Interrupts are not assumed cheap; that's why interrupt mitigation is
>>> used (on real and virtual hardware).
>>>
>> Its all relative. IDT dispatch and EOI overhead are "baseline" on real
>> hardware, whereas they are significantly more expensive to do the
>> vmenters and vmexits on virt (and you have new exit causes, like
>> irq-windows, etc, that do not exist in real HW).
>>
>
> irq window exits ought to be pretty rare, so we're only left with
> injection vmexits. At around 1us/vmexit, even 100,000 interrupts/vcpu
> (which is excessive) will only cost you 10% cpu time.
1us is too much for what I am building, IMHO. Besides, there are a slew
of older machines (like Woodcrests) that are more like 2+us per exit, so
1us is a best-case scenario.
>
>>>> G) Interrupts/MSI are non-priortizable.
>>>>
>>>>
>>> They are prioritizable; Linux ignores this though (Windows doesn't).
>>> Please elaborate on what the problem is and how vbus solves it.
>>>
>> It doesn't work right. The x86 sense of interrupt priority is, sorry to
>> say it, half-assed at best. I've worked with embedded systems that have
>> real interrupt priority support in the hardware, end to end, including
>> the PIC. The LAPIC on the other hand is really weak in this dept, and
>> as you said, Linux doesn't even attempt to use whats there.
>>
>
> Maybe prioritization is not that important then. If it is, it needs to
> be fixed at the lapic level, otherwise you have no real prioritization
> wrt non-vbus interrupts.
While this is true, I am generally not worried about it. For the
environments that care, I plan on having it be predominantly vbus
devices and using an -rt kernel (with irq-threads).
>
>>>> H) Interrupts/MSI are statically established
>>>>
>>>>
>>> Can you give an example of why this is a problem?
>>>
>> Some of the things we are building use the model of having a device that
>> hands out shm-signal in response to guest events (say, the creation of
>> an IPC channel). This would generally be handled by a specific device
>> model instance, and it would need to do this without pre-declaring the
>> MSI vectors (to use PCI as an example).
>>
>
> You're free to demultiplex an MSI to however many consumers you want,
> there's no need for a new bus for that.
Hmmm...can you elaborate?
>
>>> What performance oriented items have been left unaddressed?
>>>
>> Well, the interrupt model to name one.
>>
>
> Like I mentioned, you can merge MSI interrupts, but that's not
> necessarily a good idea.
>
>>> How do you handle conflicts? Again you need a central authority to hand
>>> out names or prefixes.
>>>
>> Not really, no. If you really wanted to be formal about it, you could
>> adopt any series of UUID schemes. For instance, perhaps venet should be
>> "com.novell::virtual-ethernet". Heck, I could use uuidgen.
>>
>
> Do you use DNS. We use PCI-SIG. If Novell is a PCI-SIG member you can
> get a vendor ID and control your own virtio space.
Yeah, we have our own id. I am more concerned about making this design
make sense outside of PCI oriented environments.
>
>>>> As another example, the connector design coalesces *all* shm-signals
>>>> into a single interrupt (by prio) that uses the same context-switch
>>>> mitigation techniques that help boost things like networking. This
>>>> effectively means we can detect and optimize out ack/eoi cycles from
>>>> the
>>>> APIC as the IO load increases (which is when you need it most). PCI
>>>> has
>>>> no such concept.
>>>>
>>>>
>>> That's a bug, not a feature. It means poor scaling as the number of
>>> vcpus increases and as the number of devices increases.
vcpu increases, I agree (and am ok with, as I expect low vcpu count
machines to be typical). nr of devices, I disagree. can you elaborate?
>>>
>> So the "avi-vbus-connector" can use 1:1, if you prefer. Large vcpu
>> counts (which are not typical) and irq-affinity is not a target
>> application for my design, so I prefer the coalescing model in the
>> vbus-pcibridge included in this series. YMMV
>>
>
> So far you've left out live migration
guilty as charged.
> Windows,
Work in progress.
> large guests
Can you elaborate? I am not familiar with the term.
> and multiqueue out of your design.
AFAICT, multiqueue should work quite nicely with vbus. Can you
elaborate on where you see the problem?
> If you wish to position vbus/venet for
> large scale use you'll need to address all of them.
>
>>> Note nothing prevents steering multiple MSIs into a single vector. It's
>>> a bad idea though.
>>>
>> Yes, it is a bad idea...and not the same thing either. This would
>> effectively create a shared-line scenario in the irq code, which is not
>> what happens in vbus.
>>
>
> Ok.
>
>>>> In addition, the signals and interrupts are priority aware, which is
>>>> useful for things like 802.1p networking where you may establish 8-tx
>>>> and 8-rx queues for your virtio-net device. x86 APIC really has no
>>>> usable equivalent, so PCI is stuck here.
>>>>
>>>>
>>> x86 APIC is priority aware.
>>>
>> Have you ever tried to use it?
>>
>
> I haven't, but Windows does.
Yeah, it doesn't really work well. Its an extremely rigid model that
(IIRC) only lets you prioritize in 16 groups spaced by IDT (0-15 are one
level, 16-31 are another, etc). Most of the embedded PICs I have worked
with supported direct remapping, etc. But in any case, Linux doesn't
support it so we are hosed no matter how good it is.
>
>>>> Also, the signals can be allocated on-demand for implementing things
>>>> like IPC channels in response to guest requests since there is no
>>>> assumption about device-to-interrupt mappings. This is more flexible.
>>>>
>>>>
>>> Yes. However given that vectors are a scarce resource you're severely
>>> limited in that.
>>>
>> The connector I am pushing out does not have this limitation.
>>
>
> Okay.
>
>>
>>> And if you're multiplexing everything on one vector,
>>> then you can just as well demultiplex your channels in the virtio driver
>>> code.
>>>
>> Only per-device, not system wide.
>>
>
> Right. I still think multiplexing interrupts is a bad idea in a large
> system. In a small system... why would you do it at all?
device scaling, like for running a device-domain / bridge in a guest.
>
>>>> And through all of this, this design would work in any guest even if it
>>>> doesn't have PCI (e.g. lguest, UML, physical systems, etc).
>>>>
>>>>
>>> That is true for virtio which works on pci-less lguest and s390.
>>>
>> Yes, and lguest and s390 had to build their own bus-model to do it,
>> right?
>>
>
> They had to build connectors just like you propose to do.
More importantly, they had to build back-end busses too, no?
>
>> Thank you for bringing this up, because it is one of the main points
>> here. What I am trying to do is generalize the bus to prevent the
>> proliferation of more of these isolated models in the future. Build
>> one, fast, in-kernel model so that we wouldn't need virtio-X, and
>> virtio-Y in the future. They can just reuse the (performance optimized)
>> bus and models, and only need to build the connector to bridge them.
>>
>
> But you still need vbus-connector-lguest and vbus-connector-s390 because
> they all talk to the host differently. So what's changed? the names?
The fact that they don't need to redo most of the in-kernel backend
stuff. Just the connector.
>
>>> That is exactly the design goal of virtio (except it limits itself to
>>> virtualization).
>>>
>> No, virtio is only part of the picture. It not including the backend
>> models, or how to do memory/signal-path abstraction for in-kernel, for
>> instance. But otherwise, virtio as a device model is compatible with
>> vbus as a bus model. They compliment one another.
>>
>
> Well, venet doesn't complement virtio-net, and virtio-pci doesn't
> complement vbus-connector.
Agreed, but virtio complements vbus by virtue of virtio-vbus.
>
>>>> Then device models like virtio can ride happily on top and we end up
>>>> with a really robust and high-performance Linux-based stack. I don't
>>>> buy the argument that we already have PCI so lets use it. I don't
>>>> think
>>>> its the best design and I am not afraid to make an investment in a
>>>> change here because I think it will pay off in the long run.
>>>>
>>>>
>>> Sorry, I don't think you've shown any quantifiable advantages.
>>>
>> We can agree to disagree then, eh? There are certainly quantifiable
>> differences. Waving your hand at the differences to say they are not
>> advantages is merely an opinion, one that is not shared universally.
>>
>
> I've addressed them one by one. We can agree to disagree on interrupt
> multiplexing, and the importance of compatibility, Windows, large
> guests, multiqueue, and DNS vs. PCI-SIG.
>
>> The bottom line is all of these design distinctions are encapsulated
>> within the vbus subsystem and do not affect the kvm code-base. So
>> agreement with kvm upstream is not a requirement, but would be
>> advantageous for collaboration.
>>
>
> Certainly.
>
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH] net: add Xilinx emac lite device driver
From: David Miller @ 2009-08-19 6:28 UTC (permalink / raw)
To: john.linn
Cc: netdev, linuxppc-dev, jgarzik, grant.likely, jwboyer,
john.williams, michal.simek, sadanan
In-Reply-To: <20090818153047.48E42ED8051@mail66-dub.bigfish.com>
From: John Linn <john.linn@xilinx.com>
Date: Tue, 18 Aug 2009 09:30:41 -0600
> @@ -1926,6 +1926,11 @@ config ATL2
> To compile this driver as a module, choose M here. The module
> will be called atl2.
>
> +config XILINX_EMACLITE
> + tristate "Xilinx 10/100 Ethernet Lite support"
> + help
> + This driver supports the 10/100 Ethernet Lite from Xilinx.
> +
> source "drivers/net/fs_enet/Kconfig"
>
> endif # NET_ETHERNET
You can't unconditionally enable this driver everywhere.
It depends upon things like openfirmware support, etc.
so the driver will fail to link if the architecture doesn't
have those support routines.
Please fix this up, thanks.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 5:48 UTC (permalink / raw)
To: Gregory Haskins
Cc: Michael S. Tsirkin, Anthony Liguori, Ingo Molnar, Gregory Haskins,
kvm, alacrityvm-devel, linux-kernel, netdev
In-Reply-To: <4A8B8F4E.80207@gmail.com>
On 08/19/2009 08:36 AM, Gregory Haskins wrote:
>> If virtio net in guest could be improved instead, everyone would
>> benefit.
>>
> So if I whip up a virtio-net backend for vbus with a PCI compliant
> connector, you are happy?
>
This doesn't improve virtio-net in any way.
>> I am doing this, and I wish more people would join. Instead,
>> you change ABI in a incompatible way.
>>
> Only by choice of my particular connector. The ABI is a function of the
> connector design. So one such model is to terminate the connector in
> qemu, and surface the resulting objects as PCI devices. I choose not to
> use this particular design for my connector that I am pushing upstream
> because I am of the opinion that I can do better by terminating it in
> the guest directly as a PV optimized bus. However, both connectors can
> theoretically coexist peacefully.
>
virtio already supports this model; see lguest and s390. Transporting
virtio over vbus and vbus over something else doesn't gain anything over
directly transporting virtio over that something else.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: [Alacrityvm-devel] [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 5:40 UTC (permalink / raw)
To: Ira W. Snyder
Cc: Michael S. Tsirkin, Gregory Haskins, kvm, netdev, linux-kernel,
alacrityvm-devel, Anthony Liguori, Ingo Molnar, Gregory Haskins
In-Reply-To: <20090819003812.GA11168@ovro.caltech.edu>
On 08/19/2009 03:38 AM, Ira W. Snyder wrote:
> On Wed, Aug 19, 2009 at 12:26:23AM +0300, Avi Kivity wrote:
>
>> On 08/18/2009 11:59 PM, Ira W. Snyder wrote:
>>
>>> On a non shared-memory system (where the guest's RAM is not just a chunk
>>> of userspace RAM in the host system), virtio's management model seems to
>>> fall apart. Feature negotiation doesn't work as one would expect.
>>>
>>>
>> In your case, virtio-net on the main board accesses PCI config space
>> registers to perform the feature negotiation; software on your PCI cards
>> needs to trap these config space accesses and respond to them according
>> to virtio ABI.
>>
>>
> Is this "real PCI" (physical hardware) or "fake PCI" (software PCI
> emulation) that you are describing?
>
>
Real PCI.
> The host (x86, PCI master) must use "real PCI" to actually configure the
> boards, enable bus mastering, etc. Just like any other PCI device, such
> as a network card.
>
> On the guests (ppc, PCI agents) I cannot add/change PCI functions (the
> last .[0-9] in the PCI address) nor can I change PCI BAR's once the
> board has started. I'm pretty sure that would violate the PCI spec,
> since the PCI master would need to re-scan the bus, and re-assign
> addresses, which is a task for the BIOS.
>
Yes. Can the boards respond to PCI config space cycles coming from the
host, or is the config space implemented in silicon and immutable?
(reading on, I see the answer is no). virtio-pci uses the PCI config
space to configure the hardware.
>> (There's no real guest on your setup, right? just a kernel running on
>> and x86 system and other kernels running on the PCI cards?)
>>
>>
> Yes, the x86 (PCI master) runs Linux (booted via PXELinux). The ppc's
> (PCI agents) also run Linux (booted via U-Boot). They are independent
> Linux systems, with a physical PCI interconnect.
>
> The x86 has CONFIG_PCI=y, however the ppc's have CONFIG_PCI=n. Linux's
> PCI stack does bad things as a PCI agent. It always assumes it is a PCI
> master.
>
> It is possible for me to enable CONFIG_PCI=y on the ppc's by removing
> the PCI bus from their list of devices provided by OpenFirmware. They
> can not access PCI via normal methods. PCI drivers cannot work on the
> ppc's, because Linux assumes it is a PCI master.
>
> To the best of my knowledge, I cannot trap configuration space accesses
> on the PCI agents. I haven't needed that for anything I've done thus
> far.
>
>
Well, if you can't do that, you can't use virtio-pci on the host.
You'll need another virtio transport (equivalent to "fake pci" you
mentioned above).
>>> This does appear to be solved by vbus, though I haven't written a
>>> vbus-over-PCI implementation, so I cannot be completely sure.
>>>
>>>
>> Even if virtio-pci doesn't work out for some reason (though it should),
>> you can write your own virtio transport and implement its config space
>> however you like.
>>
>>
> This is what I did with virtio-over-PCI. The way virtio-net negotiates
> features makes this work non-intuitively.
>
I think you tried to take two virtio-nets and make them talk together?
That won't work. You need the code from qemu to talk to virtio-net
config space, and vhost-net to pump the rings.
>>> I'm not at all clear on how to get feature negotiation to work on a
>>> system like mine. From my study of lguest and kvm (see below) it looks
>>> like userspace will need to be involved, via a miscdevice.
>>>
>>>
>> I don't see why. Is the kernel on the PCI cards in full control of all
>> accesses?
>>
>>
> I'm not sure what you mean by this. Could you be more specific? This is
> a normal, unmodified vanilla Linux kernel running on the PCI agents.
>
I meant, does board software implement the config space accesses issued
from the host, and it seems the answer is no.
> In my virtio-over-PCI patch, I hooked two virtio-net's together. I wrote
> an algorithm to pair the tx/rx queues together. Since virtio-net
> pre-fills its rx queues with buffers, I was able to use the DMA engine
> to copy from the tx queue into the pre-allocated memory in the rx queue.
>
>
Please find a name other than virtio-over-PCI since it conflicts with
virtio-pci. You're tunnelling virtio config cycles (which are usually
done on pci config cycles) on a new protocol which is itself tunnelled
over PCI shared memory.
>>>
>>>
>> Yeah. You'll need to add byteswaps.
>>
>>
> I wonder if Rusty would accept a new feature:
> VIRTIO_F_NET_LITTLE_ENDIAN, which would allow the virtio-net driver to
> use LE for all of it's multi-byte fields.
>
> I don't think the transport should have to care about the endianness.
>
Given this is not mainstream use, it would have to have zero impact when
configured out.
> True. It's slowpath setup, so I don't care how fast it is. For reasons
> outside my control, the x86 (PCI master) is running a RHEL5 system. This
> means glibc-2.5, which doesn't have eventfd support, AFAIK. I could try
> and push for an upgrade. This obviously makes cat/echo really nice, it
> doesn't depend on glibc, only the kernel version.
>
> I don't give much weight to the above, because I can use the eventfd
> syscalls directly, without glibc support. It is just more painful.
>
The x86 side only needs to run virtio-net, which is present in RHEL
5.3. You'd only need to run virtio-tunnel or however it's called. All
the eventfd magic takes place on the PCI agents.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 5:36 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Avi Kivity, Ingo Molnar, kvm, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin
In-Reply-To: <200908182020.11126.arnd@arndb.de>
[-- Attachment #1: Type: text/plain, Size: 2718 bytes --]
Arnd Bergmann wrote:
> On Tuesday 18 August 2009, Gregory Haskins wrote:
>> Avi Kivity wrote:
>>> On 08/17/2009 10:33 PM, Gregory Haskins wrote:
>>>
>>> One point of contention is that this is all managementy stuff and should
>>> be kept out of the host kernel. Exposing shared memory, interrupts, and
>>> guest hypercalls can all be easily done from userspace (as virtio
>>> demonstrates). True, some devices need kernel acceleration, but that's
>>> no reason to put everything into the host kernel.
>> See my last reply to Anthony. My two points here are that:
>>
>> a) having it in-kernel makes it a complete subsystem, which perhaps has
>> diminished value in kvm, but adds value in most other places that we are
>> looking to use vbus.
>>
>> b) the in-kernel code is being overstated as "complex". We are not
>> talking about your typical virt thing, like an emulated ICH/PCI chipset.
>> Its really a simple list of devices with a handful of attributes. They
>> are managed using established linux interfaces, like sysfs/configfs.
>
> IMHO the complexity of the code is not so much of a problem. What I
> see as a problem is the complexity a kernel/user space interface that
> manages a the devices with global state.
>
> One of the greatest features of Michaels vhost driver is that all
> the state is associated with open file descriptors that either exist
> already or belong to the vhost_net misc device. When a process dies,
> all the file descriptors get closed and the whole state is cleaned
> up implicitly.
>
> AFAICT, you can't do that with the vbus host model.
It should work the same. When a driver opens a vbus device, it calls
"interface->connect()" and gets back a "connection" object. The
connection->release() method is invoked when the driver "goes away",
which would include the scenario you present. This gives the
device-model the opportunity to cleanup in the same way.
>
>>> What performance oriented items have been left unaddressed?
>> Well, the interrupt model to name one.
>
> The performance aspects of your interrupt model are independent
> of the vbus proxy, or at least they should be. Let's assume for
> now that your event notification mechanism gives significant
> performance improvements (which we can't measure independently
> right now). I don't see a reason why we could not get the
> same performance out of a paravirtual interrupt controller
> that uses the same method, and it would be straightforward
> to implement one and use that together with all the existing
> emulated PCI devices and virtio devices including vhost_net.
Agreed. I proposed this before and Avi rejected the idea.
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 5:36 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Avi Kivity, Anthony Liguori, Ingo Molnar, Gregory Haskins, kvm,
alacrityvm-devel, linux-kernel, netdev
In-Reply-To: <20090818165139.GC19846@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 3234 bytes --]
Michael S. Tsirkin wrote:
> On Tue, Aug 18, 2009 at 11:51:59AM -0400, Gregory Haskins wrote:
>>> It's not laughably trivial when you try to support the full feature set
>>> of kvm (for example, live migration will require dirty memory tracking,
>>> and exporting all state stored in the kernel to userspace).
>> Doesn't vhost suffer from the same issue? If not, could I also apply
>> the same technique to support live-migration in vbus?
>
> vhost does this by switching to userspace for the duration of live
> migration. venet could do this I guess, but you'd need to write a
> userspace implementation. vhost just reuses existing userspace virtio.
>
>> With all due respect, I didnt ask you do to anything, especially not
>> abandon something you are happy with.
>>
>> All I did was push guest drivers to LKML. The code in question is
>> independent of KVM, and its proven to improve the experience of using
>> Linux as a platform. There are people interested in using them (by
>> virtue of the number of people that have signed up for the AlacrityVM
>> list, and have mailed me privately about this work).
>>
>> So where is the problem here?
>
> If virtio net in guest could be improved instead, everyone would
> benefit.
So if I whip up a virtio-net backend for vbus with a PCI compliant
connector, you are happy?
> I am doing this, and I wish more people would join. Instead,
> you change ABI in a incompatible way.
Only by choice of my particular connector. The ABI is a function of the
connector design. So one such model is to terminate the connector in
qemu, and surface the resulting objects as PCI devices. I choose not to
use this particular design for my connector that I am pushing upstream
because I am of the opinion that I can do better by terminating it in
the guest directly as a PV optimized bus. However, both connectors can
theoretically coexist peacefully.
The advantage that this would give us is that one in-kernel virtio-net
model could be surfaced to all vbus users (pci, or otherwise), which
will hopefully be growing over time. This would have gained vbus a
virtio-net backend, and it would have saved you from re-inventing the
various abstractions and management interfaces that vbus has in place.
> So now, there's no single place to
> work on kvm networking performance. Now, it would all be understandable
> if the reason was e.g. better performance. But you say yourself it
> isn't.
Actually, I really didn't say that. As far as I know, your patch hasnt
been performance proven to my knowledge, but I just gave you the benefit
of the doubt. What I said was that for a limited type of benchmark, it
*may* get similar numbers if you implemented vhost optimally. For
others (for instance, when we can start to take advantage of priority,
or scaling the number of interfaces) it may not since my proposed
connector was designed to optimize this over raw PCI facilities.
But I digress. Please post results when you have numbers, as I had to
give up my 10GE rig in the lab. I suspect you will have performance
issues until you at least address GSO, but you may already be there by now.
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [Alacrityvm-devel] [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 5:26 UTC (permalink / raw)
To: Ira W. Snyder
Cc: Michael S. Tsirkin, Gregory Haskins, kvm, netdev, linux-kernel,
alacrityvm-devel, Anthony Liguori, Ingo Molnar, Gregory Haskins
In-Reply-To: <20090819004445.GB11168@ovro.caltech.edu>
On 08/19/2009 03:44 AM, Ira W. Snyder wrote:
>> You don't need in fact a third mode. You can mmap the x86 address space
>> into your ppc userspace and use the second mode. All you need then is
>> the dma engine glue and byte swapping.
>>
>>
> Hmm, I'll have to think about that.
>
> The ppc is a 32-bit processor, so it has 4GB of address space for
> everything, including PCI, SDRAM, flash memory, and all other
> peripherals.
>
> This is exactly like 32bit x86, where you cannot have a PCI card that
> exposes a 4GB PCI BAR. The system would have no address space left for
> its own SDRAM.
>
(you actually can, since x86 has a 36-40 bit physical address space even
with a 32-bit virtual address space, but that doesn't help you).
> On my x86 computers, I only have 1GB of physical RAM, and so the ppc's
> have plenty of room in their address spaces to map the entire x86 RAM
> into their own address space. That is exactly what I do now. Accesses to
> ppc physical address 0x80000000 "magically" hit x86 physical address
> 0x0.
>
So if you mmap() that, you could work with virtual addresses. It may be
more efficient to work with physical addresses directly though.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 5:22 UTC (permalink / raw)
To: Gregory Haskins
Cc: Ingo Molnar, Anthony Liguori, Gregory Haskins, kvm,
alacrityvm-devel, linux-kernel, netdev, Michael S. Tsirkin
In-Reply-To: <4A8B7F17.6050100@gmail.com>
On 08/19/2009 07:27 AM, Gregory Haskins wrote:
>
>> This thread started because i asked you about your technical
>> arguments why we'd want vbus instead of virtio.
>>
> (You mean vbus vs pci, right? virtio works fine, is untouched, and is
> out-of-scope here)
>
I guess he meant venet vs virtio-net. Without venet vbus is currently
userless.
> Right, and I do believe I answered your questions. Do you feel as
> though this was not a satisfactory response?
>
Others and I have shown you its wrong. There's no inherent performance
problem in pci. The vbus approach has inherent problems (the biggest of
which is compatibility, the second managability).
>> Your answer above
>> now basically boils down to: "because I want it so, why dont you
>> leave me alone".
>>
> Well, with all due respect, please do not put words in my mouth. This
> is not what I am saying at all.
>
> What I *am* saying is:
>
> fact: this thread is about linux guest drivers to support vbus
>
> fact: these drivers do not touch kvm code.
>
> fact: these drivers to not force kvm to alter its operation in any way.
>
> fact: these drivers do not alter ABIs that KVM currently supports.
>
> Therefore, all this talk about "abandoning", "supporting", and
> "changing" things in KVM is, premature, irrelevant, and/or, FUD. No one
> proposed such changes, so I am highlighting this fact to bring the
> thread back on topic. That KVM talk is merely a distraction at this
> point in time.
>
s/kvm/kvm stack/. virtio/pci is part of the kvm stack, even if it is
not part of kvm itself. If vbus/venet were to be merged, users and
developers would have to choose one or the other. That's the
fragmentation I'm worried about. And you can prefix that with "fact:"
as well.
>> We all love faster code and better management interfaces and tons
>> of your prior patches got accepted by Avi. This time you didnt even
>> _try_ to improve virtio.
>>
> Im sorry, but you are mistaken:
>
> http://lkml.indiana.edu/hypermail/linux/kernel/0904.2/02443.html
>
That does nothing to improve virtio. Existing guests (Linux and
Windows) which support virtio will cease to work if the host moves to
vbus-virtio. Existing hosts (running virtio-pci) won't be able to talk
to newer guests running virtio-vbus. The patch doesn't improve
performance without the entire vbus stack in the host kernel and a
vbus-virtio-net-host host kernel driver.
Perhaps if you posted everything needed to make vbus-virtio work and
perform we could compare that to vhost-net and you'll see another reason
why vhost-net is the better approach.
> You are also wrong to say that I didn't try to avoid creating a
> downstream effort first. I believe the public record of the mailing
> lists will back me up that I tried politely pushing this directly though
> kvm first. It was only after Avi recently informed me that they would
> be building their own version of an in-kernel backend in lieu of working
> with me to adapt vbus to their needs that I decided to put my own
> project together.
>
There's no way we can adapt vbus to our needs. Don't you think we'd
preferred it rather than writing our own? the current virtio-net issues
are hurting us.
Our needs are compatibility, performance, and managability. vbus fails
all three, your impressive venet numbers notwithstanding.
> What should I have done otherwise, in your opinion?
>
You could come up with uses where vbus truly is superior to
virtio/pci/whatever (not words about etch constraints). Showing some of
those non-virt uses, for example. The fact that your only user
duplicates existing functionality doesn't help.
>> And fragmentation matters quite a bit. To Linux users, developers,
>> administrators, packagers it's a big deal whether two overlapping
>> pieces of functionality for the same thing exist within the same
>> kernel.
>>
> So the only thing that could be construed as overlapping here is venet
> vs virtio-net. If I dropped the contentious venet and focused on making
> a virtio-net backend that we can all re-use, do you see that as a path
> of compromise here?
>
That's a step in the right direction.
>> I certainly dont want that. Instead we (at great expense and work)
>> try to reach the best technical solution.
>>
> This is all I want, as well.
>
Note whenever I mention migration, large guests, or Windows you say
these are not your design requirements. The best technical solution
will have to consider those.
>> If the community wants this then why cannot you convince one of the
>> most prominent representatives of that community, the KVM
>> developers?
>>
> Its a chicken and egg at times. Perhaps the KVM developers do not have
> the motivation or time to properly consider such a proposal _until_ the
> community presents its demand.
I've spent quite a lot of time arguing with you, no doubt influenced by
the fact that you can write a lot faster than I can read.
>> Furthermore, 99% of your work is KVM
>>
> Actually, no. Almost none of it is. I think there are about 2-3
> patches in the series that touch KVM, the rest are all original (and
> primarily stand-alone code). AlacrityVM is the application of kvm and
> vbus (and, of course, Linux) together as a complete unit, but I do not
> try to hide this relationship.
>
> By your argument, KVM is 99% QEMU+Linux. ;)
>
That's one of the kvm strong points...
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 4:27 UTC (permalink / raw)
To: Ingo Molnar
Cc: Avi Kivity, Anthony Liguori, Gregory Haskins, kvm,
alacrityvm-devel, linux-kernel, netdev, Michael S. Tsirkin
In-Reply-To: <20090818161434.GA15884@elte.hu>
[-- Attachment #1: Type: text/plain, Size: 9045 bytes --]
Ingo Molnar wrote:
> * Gregory Haskins <gregory.haskins@gmail.com> wrote:
>
>>> You haven't convinced me that your ideas are worth the effort
>>> of abandoning virtio/pci or maintaining both venet/vbus and
>>> virtio/pci.
>> With all due respect, I didnt ask you do to anything, especially
>> not abandon something you are happy with.
>>
>> All I did was push guest drivers to LKML. The code in question
>> is independent of KVM, and its proven to improve the experience
>> of using Linux as a platform. There are people interested in
>> using them (by virtue of the number of people that have signed up
>> for the AlacrityVM list, and have mailed me privately about this
>> work).
>
> This thread started because i asked you about your technical
> arguments why we'd want vbus instead of virtio.
(You mean vbus vs pci, right? virtio works fine, is untouched, and is
out-of-scope here)
Right, and I do believe I answered your questions. Do you feel as
though this was not a satisfactory response?
> Your answer above
> now basically boils down to: "because I want it so, why dont you
> leave me alone".
Well, with all due respect, please do not put words in my mouth. This
is not what I am saying at all.
What I *am* saying is:
fact: this thread is about linux guest drivers to support vbus
fact: these drivers do not touch kvm code.
fact: these drivers to not force kvm to alter its operation in any way.
fact: these drivers do not alter ABIs that KVM currently supports.
Therefore, all this talk about "abandoning", "supporting", and
"changing" things in KVM is, premature, irrelevant, and/or, FUD. No one
proposed such changes, so I am highlighting this fact to bring the
thread back on topic. That KVM talk is merely a distraction at this
point in time.
>
> What you are doing here is to in essence to fork KVM,
No, that is incorrect. What I am doing here is a downstream development
point for the integration of KVM and vbus. Its akin to kvm.git or
tip.git to develop a subsystem intended for eventual inclusion upstream.
If and when the code goes upstream in a manner acceptable to all
parties involved, and AlacrityVM exceeds its utility as a separate
project, I will _gladly_ dissolve it and migrate to use upstream KVM
instead.
As stated on the project wiki: "It is a goal of AlacrityVM to work
towards upstream acceptance of the project on a timeline that suits the
community. In the meantime, this wiki will serve as the central
coordination point for development and discussion of the technology"
(citation: http://developer.novell.com/wiki/index.php/AlacrityVM)
And I meant it when I said it.
Until then, the project is a much more efficient way for us (the vbus
developers) to work together than pointing people at my patch series
posted to kvm@vger. I tried that way first. It sucked, and didn't
work. Users were having trouble patching the various pieces, building,
etc. Now I can offer a complete solution from a central point, with all
the proper pieces in place to play around with it.
Ultimately, it is up to upstream to decide if this is to become merged
or remain out of tree forever as a "fork". Not me. I will continue to
make every effort to find common ground with my goals coincident with
the blessing of upstream, as I have been from the beginning. Now I have
a more official forum to do it in.
> regardless of
> the technical counter arguments given against such a fork and
> regardless of the ample opportunity given to you to demostrate the
> technical advantages of your code. (in which case KVM would happily
> migrate to your code)
In an ideal world, perhaps. Avi and I currently have a fundamental
disagreement about the best way to do PV. He sees the world through PCI
glasses, and I don't. Despite attempts on both sides to rectify this
disagreement, we currently do not see eye to eye on every front.
This doesn't mean he is right, and I am wrong per se. It just means we
disagree. Period. Avi is a sharp guy, and I respect him. But upstream
KVM doesn't have a corner on "correct" ;) The community as a whole will
ultimately decide if my ideas live or die, wouldn't you agree?
Avi can correct me if I am wrong, but what we _do_ agree on is that core
KVM doesn't need to be directly involved in this vbus (or vhost)
discussion, per se. It just wants to have the hooks to support various
PV solutions (such as irqfd/ioeventfd), and vbus is one such solution.
>
> We all love faster code and better management interfaces and tons
> of your prior patches got accepted by Avi. This time you didnt even
> _try_ to improve virtio.
Im sorry, but you are mistaken:
http://lkml.indiana.edu/hypermail/linux/kernel/0904.2/02443.html
> It's not like you posted a lot of virtio
> patches which were not applied. You didnt even try and you need to
> try _much_ harder than that before forking a project.
I really do not think you are in a position to say when someone can or
cannot fork a project, so please do not try to lecture on that. Perhaps
you could offer advice on when someone, in your opinion, *should* or
*should not* fork, because that would be interesting to hear.
You are also wrong to say that I didn't try to avoid creating a
downstream effort first. I believe the public record of the mailing
lists will back me up that I tried politely pushing this directly though
kvm first. It was only after Avi recently informed me that they would
be building their own version of an in-kernel backend in lieu of working
with me to adapt vbus to their needs that I decided to put my own
project together.
What should I have done otherwise, in your opinion?
>
> And fragmentation matters quite a bit. To Linux users, developers,
> administrators, packagers it's a big deal whether two overlapping
> pieces of functionality for the same thing exist within the same
> kernel.
So the only thing that could be construed as overlapping here is venet
vs virtio-net. If I dropped the contentious venet and focused on making
a virtio-net backend that we can all re-use, do you see that as a path
of compromise here?
> The kernel is not an anarchy where everyone can have their
> own sys_fork() version or their own sys_write() version. Would you
> want to have two dozen read() variants, sys_read_oracle() and a
> sys_read_db2()?
No, and I am not advocating that either.
>
> I certainly dont want that. Instead we (at great expense and work)
> try to reach the best technical solution.
This is all I want, as well.
> That means we throw away
> inferior code and adopt the better one. (with a reasonable
> migration period)
>
> You are ignoring that principle with hand-waving about 'the
> community wants this'.
I call it like I see it. I get private emails all the time encouraging
my efforts and asking about the project. I'm sorry if you see this as
hand-waving. Perhaps the people involved will become more vocal in the
community to back me up, perhaps not. Time will tell.
> I can assure you, users _DONT WANT_ split
> interfaces and incompatible drivers for the same thing. They want
> stuff that works well.
And I can respect that, and am trying to provide that.
>
> If the community wants this then why cannot you convince one of the
> most prominent representatives of that community, the KVM
> developers?
Its a chicken and egg at times. Perhaps the KVM developers do not have
the motivation or time to properly consider such a proposal _until_ the
community presents its demand. And sometimes you cannot build demand
unless you have an easy way to use the idea, such as a project to back
it. Since vbus+kvm has many moving parts (guest side, host-side,
userspace-side, etc), its difficult to use as a patch series pulled in
from a mailing list.
This is the role of the AlacrityVM project. Make it easy to use and
develop. If it draws a community, perhaps KVM will reconsider its
stance. If it does not draw a community, it will naturally die. End of
story.
But please do not confuse one particular groups opinion as the sole
validation of an idea, no matter how "prominent". There are numerous
reasons why one group may hold an opinion that have nothing to do with
the actual technical merits of the idea, or the community demand for it.
>
> Furthermore, 99% of your work is KVM
Actually, no. Almost none of it is. I think there are about 2-3
patches in the series that touch KVM, the rest are all original (and
primarily stand-alone code). AlacrityVM is the application of kvm and
vbus (and, of course, Linux) together as a complete unit, but I do not
try to hide this relationship.
By your argument, KVM is 99% QEMU+Linux. ;)
> why dont you respect that work by not forking it?
Lighten up on the fork FUD, please. It's counter productive.
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: Socket C/R additional features
From: David Miller @ 2009-08-19 3:36 UTC (permalink / raw)
To: danms; +Cc: orenl, containers, netdev
In-Reply-To: <1250625435-16299-1-git-send-email-danms@us.ibm.com>
From: Dan Smith <danms@us.ibm.com>
Date: Tue, 18 Aug 2009 12:57:12 -0700
> This is an updated version of my previous patch set, with the fourth patch
> removed. This is rebased on current v17-dev and includes Serge's
> recommended changes to may_setgid().
I have no fundamental problems with these changes, you can
attach my ACK to the networking bits if you want:
Acked-by: David S. Miller <davem@davemloft.net>
^ permalink raw reply
* Re: [PATCH 1/4] net: skb_orphan on dev_hard_start_xmit
From: David Miller @ 2009-08-19 3:34 UTC (permalink / raw)
To: herbert
Cc: rusty, patrick.ohly, netdev, virtualization, divy, rolandd, xemul,
dcbw, libertas-dev
In-Reply-To: <20090819031926.GA17367@gondor.apana.org.au>
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Wed, 19 Aug 2009 13:19:26 +1000
> I'm in the process of repeating the same experiment with cxgb3
> which hopefully should let me turn interrupts off on descriptors
> while still reporting completion status.
Ok, I look forward to seeing your work however it turns out.
Once I see what you've done, I'll give it a spin on niu.
^ permalink raw reply
* Re: [PATCH 0/7] sky2: tx performance improvement
From: David Miller @ 2009-08-19 3:28 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 18 Aug 2009 18:17:04 -0700
> Bunch of patches for net-next (2.6.32) which boost transmit performance.
This series looks good, applied to net-next-2.6
Thanks!
^ permalink raw reply
* Re: [PATCH] yellowfin: Fix buffer underrun after dev_alloc_skb() failure
From: David Miller @ 2009-08-19 3:21 UTC (permalink / raw)
To: roel.kluin; +Cc: netdev, akpm
In-Reply-To: <4A8A6B5A.7030104@gmail.com>
From: Roel Kluin <roel.kluin@gmail.com>
Date: Tue, 18 Aug 2009 10:50:34 +0200
> yellowfin_init_ring() needs to clean up if dev_alloc_skb() fails and
> should pass an error status up to the caller. This also prevents an
> buffer underrun if failure occurred in the first iteration.
> yellowfin_open() which calls yellowfin_init_ring() should free its
> requested irq upon failure.
>
> Signed-off-by: Roel Kluin <roel.kluin@gmail.com>
Looks good, applied, thanks.
^ permalink raw reply
* Re: [PATCH 1/4] net: skb_orphan on dev_hard_start_xmit
From: Herbert Xu @ 2009-08-19 3:19 UTC (permalink / raw)
To: David Miller
Cc: rusty, patrick.ohly, netdev, virtualization, divy, rolandd, xemul,
dcbw, libertas-dev
In-Reply-To: <20090817.184712.180559154.davem@davemloft.net>
On Mon, Aug 17, 2009 at 06:47:12PM -0700, David Miller wrote:
>
> I'm pretty sure that for normal TCP and UDP workloads, this is just
> going to set the interrupt bit on the first packet that gets into the
> queue, and then not in the rest.
>
> TCP just loops over packets in the send queue, and at initial state
> the qdisc will be empty.
The scheme I actually tried on the e1000e is not quite the same
as what I had here. I'm basically just adding descriptors per
usual.
However, I don't give them to the hardware immediately. Instead
they're held back so that on average we have about three descriptors
in the queue spaced equally that will cause interrupts. It seems
to work fairly well for netperf-generated TCP/UDP traffic in terms
of generating the right amount of interrupts (once every qlen/3
packets).
I haven't posted anything yet because I ran into a weird problem
with the e1000e. It was generating a lot more interrupts than
what I'm telling it to do. Even if I hard-code the interrupts
at 0, qlen/3, 2qlen/2 I still get way more than qlen/3 interrupts
for qlen packets.
This may be related to the fact that e1000e doesn't really have
a way of turning the interrupts off for a given descriptor. Well
actually it does but it also turns off status reporting which means
that we will never know whether that entry has finished processing.
So I've had to rely on using its TX interrupt delay mechanism as an
approximation of turning interrupt notification off. Evidently that
is not working in the way I intended it to.
I'm in the process of repeating the same experiment with cxgb3
which hopefully should let me turn interrupts off on descriptors
while still reporting completion status.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] mdio: mdio_if_info::mmds should not be __bitwise
From: David Miller @ 2009-08-19 3:13 UTC (permalink / raw)
To: bhutchings; +Cc: netdev, linux-net-drivers
In-Reply-To: <1250614778.2782.14.camel@achroite>
From: Ben Hutchings <bhutchings@solarflare.com>
Date: Tue, 18 Aug 2009 17:59:38 +0100
> I misunderstood the meaning of __bitwise. In practice it makes sparse
> warn about every use of mmds which is certainly not what we want.
>
> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
> ---
> This should be safe to include in 2.6.31 but it's hardly critical.
Applied to net-next-2.6, thanks.
^ permalink raw reply
* Re: [PATCH -next] net: fix ks8851 build errors
From: David Miller @ 2009-08-19 3:12 UTC (permalink / raw)
To: randy.dunlap; +Cc: sfr, netdev, akpm, linux-next, linux-kernel
In-Reply-To: <20090818152705.64557a0b.randy.dunlap@oracle.com>
From: Randy Dunlap <randy.dunlap@oracle.com>
Date: Tue, 18 Aug 2009 15:27:05 -0700
> --- linux-next-20090818.orig/drivers/net/Kconfig
> +++ linux-next-20090818/drivers/net/Kconfig
> @@ -1733,6 +1733,8 @@ config KS8851
> tristate "Micrel KS8851 SPI"
> depends on SPI
> select MII
> + select CRC32
> + select BITREVERSE
> help
> SPI driver for Micrel KS8851 SPI attached network chip.
>
CRC32 alone is sufficient, because it, in turn, selects BITREVERSE
When you resubmit with this simplification, I'll apply the
patch to net-2.6 since the problem exists in Linus's tree
too, not just -next.
^ permalink raw reply
* [PATCH 7/7] sky2: version 1.25
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
[-- Attachment #1: sky2-1.25.patch --]
[-- Type: text/plain, Size: 343 bytes --]
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/drivers/net/sky2.c 2009-08-18 17:53:33.772652966 -0700
+++ b/drivers/net/sky2.c 2009-08-18 17:53:42.177674500 -0700
@@ -50,7 +50,7 @@
#include "sky2.h"
#define DRV_NAME "sky2"
-#define DRV_VERSION "1.24"
+#define DRV_VERSION "1.25"
#define PFX DRV_NAME " "
/*
--
^ permalink raw reply
* [PATCH 0/7] sky2: tx performance improvement
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
Bunch of patches for net-next (2.6.32) which boost transmit performance.
--
^ permalink raw reply
* [PATCH 1/7] sky2: use upper/lower 32 bits
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
[-- Attachment #1: sky2-upper32.patch --]
[-- Type: text/plain, Size: 1948 bytes --]
Use the existing macros to show where DMA address is being broken
apart. This is cosmetic only.
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/drivers/net/sky2.c 2009-08-18 09:30:17.292586888 -0700
+++ b/drivers/net/sky2.c 2009-08-18 09:32:29.143195273 -0700
@@ -984,12 +984,12 @@ static void sky2_qset(struct sky2_hw *hw
* hardware and driver list elements
*/
static void sky2_prefetch_init(struct sky2_hw *hw, u32 qaddr,
- u64 addr, u32 last)
+ dma_addr_t addr, u32 last)
{
sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_CTRL), PREF_UNIT_RST_SET);
sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_CTRL), PREF_UNIT_RST_CLR);
- sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_ADDR_HI), addr >> 32);
- sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_ADDR_LO), (u32) addr);
+ sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_ADDR_HI), upper_32_bits(addr));
+ sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_ADDR_LO), lower_32_bits(addr));
sky2_write16(hw, Y2_QADDR(qaddr, PREF_UNIT_LAST_IDX), last);
sky2_write32(hw, Y2_QADDR(qaddr, PREF_UNIT_CTRL), PREF_UNIT_OP_ON);
@@ -1057,7 +1057,7 @@ static void sky2_rx_add(struct sky2_port
}
le = sky2_next_rx(sky2);
- le->addr = cpu_to_le32((u32) map);
+ le->addr = cpu_to_le32(lower_32_bits(map));
le->length = cpu_to_le16(len);
le->opcode = op | HW_OWNER;
}
@@ -1662,7 +1662,7 @@ static int sky2_xmit_frame(struct sk_buf
}
le = get_tx_le(sky2, &slot);
- le->addr = cpu_to_le32((u32) mapping);
+ le->addr = cpu_to_le32(lower_32_bits(mapping));
le->length = cpu_to_le16(len);
le->ctrl = ctrl;
le->opcode = mss ? (OP_LARGESEND | HW_OWNER) : (OP_PACKET | HW_OWNER);
@@ -1689,7 +1689,7 @@ static int sky2_xmit_frame(struct sk_buf
}
le = get_tx_le(sky2, &slot);
- le->addr = cpu_to_le32((u32) mapping);
+ le->addr = cpu_to_le32(lower_32_bits(mapping));
le->length = cpu_to_le16(frag->size);
le->ctrl = ctrl;
le->opcode = OP_BUFFER | HW_OWNER;
--
^ permalink raw reply
* [PATCH 6/7] sky2: no recycling
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
[-- Attachment #1: sky2-no-recycle.patch --]
[-- Type: text/plain, Size: 2058 bytes --]
Recycling turns out to be a bad idea! For most use cases, the
packet can not be reused: TCP packets are cloned. Even for the ideal
case of forwarding, it hurts performance because of CPU ping/pong.
On a multi-core system forwarding of 64 byte packets is worse
much worse: recycling = 24% forwarded vs no recycling = 42% forwarded
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/drivers/net/sky2.c 2009-08-18 17:50:31.591887510 -0700
+++ b/drivers/net/sky2.c 2009-08-18 17:53:04.506927814 -0700
@@ -1177,7 +1177,6 @@ static void sky2_rx_clean(struct sky2_po
re->skb = NULL;
}
}
- skb_queue_purge(&sky2->rx_recycle);
}
/* Basic MII support */
@@ -1269,10 +1268,8 @@ static struct sk_buff *sky2_rx_alloc(str
struct sk_buff *skb;
int i;
- skb = __skb_dequeue(&sky2->rx_recycle);
- if (!skb)
- skb = netdev_alloc_skb(sky2->netdev, sky2->rx_data_size
- + sky2_rx_pad(sky2->hw));
+ skb = netdev_alloc_skb(sky2->netdev,
+ sky2->rx_data_size + sky2_rx_pad(sky2->hw));
if (!skb)
goto nomem;
@@ -1364,8 +1361,6 @@ static int sky2_rx_start(struct sky2_por
sky2->rx_data_size = size;
- skb_queue_head_init(&sky2->rx_recycle);
-
/* Fill Rx ring */
for (i = 0; i < sky2->rx_pending; i++) {
re = sky2->rx_ring + i;
@@ -1776,12 +1771,7 @@ static void sky2_tx_complete(struct sky2
dev->stats.tx_packets++;
dev->stats.tx_bytes += skb->len;
- if (skb_queue_len(&sky2->rx_recycle) < sky2->rx_pending
- && skb_recycle_check(skb, sky2->rx_data_size
- + sky2_rx_pad(sky2->hw)))
- __skb_queue_head(&sky2->rx_recycle, skb);
- else
- dev_kfree_skb_any(skb);
+ dev_kfree_skb_any(skb);
sky2->tx_next = RING_NEXT(idx, sky2->tx_ring_size);
}
--- a/drivers/net/sky2.h 2009-08-18 17:50:34.706665948 -0700
+++ b/drivers/net/sky2.h 2009-08-18 17:51:02.596652122 -0700
@@ -2032,7 +2032,6 @@ struct sky2_port {
u16 rx_pending;
u16 rx_data_size;
u16 rx_nfrags;
- struct sk_buff_head rx_recycle;
#ifdef SKY2_VLAN_TAG_USED
u16 rx_tag;
--
^ permalink raw reply
* [PATCH 2/7] sky2: transmit ring 64 bit conservation
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
[-- Attachment #1: sky2-64.patch --]
[-- Type: text/plain, Size: 2091 bytes --]
This patch saves elements on transmit ring by only updating the upper
64 bit address when it changes. With many workloads skb's are located
in same region, so it saves space.
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/drivers/net/sky2.c 2009-08-18 09:32:29.143195273 -0700
+++ b/drivers/net/sky2.c 2009-08-18 09:33:15.490426779 -0700
@@ -1016,6 +1016,7 @@ static void tx_init(struct sky2_port *sk
le = get_tx_le(sky2, &sky2->tx_prod);
le->addr = 0;
le->opcode = OP_ADDR64 | HW_OWNER;
+ sky2->tx_last_upper = 0;
}
static inline struct tx_ring_info *tx_le_re(struct sky2_port *sky2,
@@ -1573,8 +1574,9 @@ static int sky2_xmit_frame(struct sk_buf
struct sky2_tx_le *le = NULL;
struct tx_ring_info *re;
unsigned i, len;
- u16 slot;
dma_addr_t mapping;
+ u32 upper;
+ u16 slot;
u16 mss;
u8 ctrl;
@@ -1593,9 +1595,11 @@ static int sky2_xmit_frame(struct sk_buf
dev->name, slot, skb->len);
/* Send high bits if needed */
- if (sizeof(dma_addr_t) > sizeof(u32)) {
+ upper = upper_32_bits(mapping);
+ if (upper != sky2->tx_last_upper) {
le = get_tx_le(sky2, &slot);
- le->addr = cpu_to_le32(upper_32_bits(mapping));
+ le->addr = cpu_to_le32(upper);
+ sky2->tx_last_upper = upper;
le->opcode = OP_ADDR64 | HW_OWNER;
}
@@ -1681,10 +1685,11 @@ static int sky2_xmit_frame(struct sk_buf
if (pci_dma_mapping_error(hw->pdev, mapping))
goto mapping_unwind;
- if (sizeof(dma_addr_t) > sizeof(u32)) {
+ upper = upper_32_bits(mapping);
+ if (upper != sky2->tx_last_upper) {
le = get_tx_le(sky2, &slot);
- le->addr = cpu_to_le32(upper_32_bits(mapping));
- le->ctrl = 0;
+ le->addr = cpu_to_le32(upper);
+ sky2->tx_last_upper = upper;
le->opcode = OP_ADDR64 | HW_OWNER;
}
--- a/drivers/net/sky2.h 2009-08-18 09:29:59.224176503 -0700
+++ b/drivers/net/sky2.h 2009-08-18 09:33:15.491426771 -0700
@@ -2017,6 +2017,7 @@ struct sky2_port {
u16 tx_pending;
u16 tx_last_mss;
+ u32 tx_last_upper;
u32 tx_tcpsum;
struct rx_ring_info *rx_ring ____cacheline_aligned_in_smp;
--
^ permalink raw reply
* [PATCH 4/7] sky2: dynamic size transmit ring
From: Stephen Hemminger @ 2009-08-19 1:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20090819011704.685802801@vyatta.com>
[-- Attachment #1: sky2-tx-size.patch --]
[-- Type: text/plain, Size: 6748 bytes --]
Allocate and size transmit ring based on parameters. Saves excess
space and allows configuring larger rings for testing.
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
--- a/drivers/net/sky2.c 2009-08-18 09:34:30.664368374 -0700
+++ b/drivers/net/sky2.c 2009-08-18 09:34:35.094550982 -0700
@@ -64,10 +64,12 @@
#define RX_MAX_PENDING (RX_LE_SIZE/6 - 2)
#define RX_DEF_PENDING RX_MAX_PENDING
-#define TX_RING_SIZE 512
-#define TX_DEF_PENDING 128
-#define MAX_SKB_TX_LE (4 + (sizeof(dma_addr_t)/sizeof(u32))*MAX_SKB_FRAGS)
+/* This is the worst case number of transmit list elements for a single skb:
+ VLAN + TSO + CKSUM + Data + skb_frags * DMA */
+#define MAX_SKB_TX_LE (4 + (sizeof(dma_addr_t)/sizeof(u32))*MAX_SKB_FRAGS)
#define TX_MIN_PENDING (MAX_SKB_TX_LE+1)
+#define TX_MAX_PENDING 4096
+#define TX_DEF_PENDING 127
#define STATUS_RING_SIZE 2048 /* 2 ports * (TX + 2*RX) */
#define STATUS_LE_BYTES (STATUS_RING_SIZE*sizeof(struct sky2_status_le))
@@ -1000,7 +1002,7 @@ static inline struct sky2_tx_le *get_tx_
{
struct sky2_tx_le *le = sky2->tx_le + *slot;
- *slot = RING_NEXT(*slot, TX_RING_SIZE);
+ *slot = RING_NEXT(*slot, sky2->tx_ring_size);
le->ctrl = 0;
return le;
}
@@ -1433,13 +1435,13 @@ static int sky2_up(struct net_device *de
/* must be power of 2 */
sky2->tx_le = pci_alloc_consistent(hw->pdev,
- TX_RING_SIZE *
+ sky2->tx_ring_size *
sizeof(struct sky2_tx_le),
&sky2->tx_le_map);
if (!sky2->tx_le)
goto err_out;
- sky2->tx_ring = kcalloc(TX_RING_SIZE, sizeof(struct tx_ring_info),
+ sky2->tx_ring = kcalloc(sky2->tx_ring_size, sizeof(struct tx_ring_info),
GFP_KERNEL);
if (!sky2->tx_ring)
goto err_out;
@@ -1491,7 +1493,7 @@ static int sky2_up(struct net_device *de
sky2_write16(hw, Q_ADDR(txqaddr[port], Q_AL), ECU_TXFF_LEV);
sky2_prefetch_init(hw, txqaddr[port], sky2->tx_le_map,
- TX_RING_SIZE - 1);
+ sky2->tx_ring_size - 1);
#ifdef SKY2_VLAN_TAG_USED
sky2_set_vlan_mode(hw, port, sky2->vlgrp != NULL);
@@ -1520,7 +1522,7 @@ err_out:
}
if (sky2->tx_le) {
pci_free_consistent(hw->pdev,
- TX_RING_SIZE * sizeof(struct sky2_tx_le),
+ sky2->tx_ring_size * sizeof(struct sky2_tx_le),
sky2->tx_le, sky2->tx_le_map);
sky2->tx_le = NULL;
}
@@ -1533,15 +1535,15 @@ err_out:
}
/* Modular subtraction in ring */
-static inline int tx_dist(unsigned tail, unsigned head)
+static inline int tx_inuse(const struct sky2_port *sky2)
{
- return (head - tail) & (TX_RING_SIZE - 1);
+ return (sky2->tx_prod - sky2->tx_cons) & (sky2->tx_ring_size - 1);
}
/* Number of list elements available for next tx */
static inline int tx_avail(const struct sky2_port *sky2)
{
- return sky2->tx_pending - tx_dist(sky2->tx_cons, sky2->tx_prod);
+ return sky2->tx_pending - tx_inuse(sky2);
}
/* Estimate of number of transmit list elements required */
@@ -1717,7 +1719,7 @@ static int sky2_xmit_frame(struct sk_buf
return NETDEV_TX_OK;
mapping_unwind:
- for (i = sky2->tx_prod; i != slot; i = RING_NEXT(i, TX_RING_SIZE)) {
+ for (i = sky2->tx_prod; i != slot; i = RING_NEXT(i, sky2->tx_ring_size)) {
le = sky2->tx_le + i;
re = sky2->tx_ring + i;
@@ -1760,10 +1762,10 @@ static void sky2_tx_complete(struct sky2
struct pci_dev *pdev = sky2->hw->pdev;
unsigned idx;
- BUG_ON(done >= TX_RING_SIZE);
+ BUG_ON(done >= sky2->tx_ring_size);
for (idx = sky2->tx_cons; idx != done;
- idx = RING_NEXT(idx, TX_RING_SIZE)) {
+ idx = RING_NEXT(idx, sky2->tx_ring_size)) {
struct sky2_tx_le *le = sky2->tx_le + idx;
struct tx_ring_info *re = sky2->tx_ring + idx;
@@ -1799,7 +1801,7 @@ static void sky2_tx_complete(struct sky2
else
dev_kfree_skb_any(skb);
- sky2->tx_next = RING_NEXT(idx, TX_RING_SIZE);
+ sky2->tx_next = RING_NEXT(idx, sky2->tx_ring_size);
}
}
@@ -1907,7 +1909,7 @@ static int sky2_down(struct net_device *
kfree(sky2->rx_ring);
pci_free_consistent(hw->pdev,
- TX_RING_SIZE * sizeof(struct sky2_tx_le),
+ sky2->tx_ring_size * sizeof(struct sky2_tx_le),
sky2->tx_le, sky2->tx_le_map);
kfree(sky2->tx_ring);
@@ -2517,7 +2519,6 @@ static int sky2_status_intr(struct sky2_
case OP_TXINDEXLE:
/* TX index reports status for both ports */
- BUILD_BUG_ON(TX_RING_SIZE > 0x1000);
sky2_tx_done(hw->dev[0], status & 0xfff);
if (hw->dev[1])
sky2_tx_done(hw->dev[1],
@@ -3669,7 +3670,7 @@ static int sky2_set_coalesce(struct net_
ecmd->rx_coalesce_usecs_irq > tmax)
return -EINVAL;
- if (ecmd->tx_max_coalesced_frames >= TX_RING_SIZE-1)
+ if (ecmd->tx_max_coalesced_frames >= sky2->tx_ring_size-1)
return -EINVAL;
if (ecmd->rx_max_coalesced_frames > RX_MAX_PENDING)
return -EINVAL;
@@ -3713,7 +3714,7 @@ static void sky2_get_ringparam(struct ne
ering->rx_max_pending = RX_MAX_PENDING;
ering->rx_mini_max_pending = 0;
ering->rx_jumbo_max_pending = 0;
- ering->tx_max_pending = TX_RING_SIZE - 1;
+ ering->tx_max_pending = TX_MAX_PENDING;
ering->rx_pending = sky2->rx_pending;
ering->rx_mini_pending = 0;
@@ -3728,14 +3729,15 @@ static int sky2_set_ringparam(struct net
if (ering->rx_pending > RX_MAX_PENDING ||
ering->rx_pending < 8 ||
- ering->tx_pending < MAX_SKB_TX_LE ||
- ering->tx_pending > TX_RING_SIZE - 1)
+ ering->tx_pending < TX_MIN_PENDING ||
+ ering->tx_pending > TX_MAX_PENDING)
return -EINVAL;
sky2_detach(dev);
sky2->rx_pending = ering->rx_pending;
sky2->tx_pending = ering->tx_pending;
+ sky2->tx_ring_size = roundup_pow_of_two(sky2->tx_pending+1);
return sky2_reattach(dev);
}
@@ -4105,8 +4107,8 @@ static int sky2_debug_show(struct seq_fi
/* Dump contents of tx ring */
sop = 1;
- for (idx = sky2->tx_next; idx != sky2->tx_prod && idx < TX_RING_SIZE;
- idx = RING_NEXT(idx, TX_RING_SIZE)) {
+ for (idx = sky2->tx_next; idx != sky2->tx_prod && idx < sky2->tx_ring_size;
+ idx = RING_NEXT(idx, sky2->tx_ring_size)) {
const struct sky2_tx_le *le = sky2->tx_le + idx;
u32 a = le32_to_cpu(le->addr);
@@ -4315,7 +4317,9 @@ static __devinit struct net_device *sky2
sky2->wol = wol;
spin_lock_init(&sky2->phy_lock);
+
sky2->tx_pending = TX_DEF_PENDING;
+ sky2->tx_ring_size = roundup_pow_of_two(TX_DEF_PENDING+1);
sky2->rx_pending = RX_DEF_PENDING;
hw->dev[port] = dev;
--- a/drivers/net/sky2.h 2009-08-18 09:33:15.491426771 -0700
+++ b/drivers/net/sky2.h 2009-08-18 09:34:35.095551088 -0700
@@ -2011,6 +2011,7 @@ struct sky2_port {
struct tx_ring_info *tx_ring;
struct sky2_tx_le *tx_le;
+ u16 tx_ring_size;
u16 tx_cons; /* next le to check */
u16 tx_prod; /* next le to use */
u16 tx_next; /* debug only */
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox