* Re: [PATCHv3 2/2] vhost_net: a kernel-level virtio server
From: Arnd Bergmann @ 2009-08-19 13:46 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtualization, netdev, kvm, linux-kernel, mingo, linux-mm, akpm,
hpa, gregory.haskins, Or Gerlitz
In-Reply-To: <20090819130417.GB3080@redhat.com>
On Wednesday 19 August 2009, Michael S. Tsirkin wrote:
> > Maybe we could instead extend the 'splice' system call to work on a
> > vhost_net file descriptor. If we do that, we can put the access back
> > into a user thread (or two) that stays in splice indefinetely
>
> An issue with exposing internal threading model to userspace
> in this way is that we lose control of e.g. CPU locality -
> and it is very hard for userspace to get it right.
Good point, I hadn't thought about that in this context.
For macvtap, my idea was to open the same tap char device multiple
times and use each fd exclusively on one *guest* CPU. I'm not sure
if virtio-net can already handle SMP guests efficiently. We might
actually need to extend it to have more pairs of virtqueues, one
for each guest CPU, which can then be bound to a host queue (or queue
pair) in the physical nic.
Leaving that aside for now, you could replace VHOST_NET_SET_SOCKET,
VHOST_SET_OWNER, VHOST_RESET_OWNER and your kernel thread with a new
VHOST_NET_SPLICE blocking ioctl that does all the transfers in the
context of the calling thread.
This would improve the driver on various fronts:
- no need for playing tricks with use_mm/unuse_mm
- possibly fewer global TLB flushes from switch_mm, which
may improve performance.
- ability to pass down error codes from socket or guest to
user space by returning from ioctl
- based on that, the ability to use any kind of file
descriptor that can do writev/readv or sendmsg/recvmsg
without the nastiness you mentioned.
The disadvantage of course is that you need to add a user
thread for each guest device to make up for the workqueue
that you save.
> > to
> > avoid some of the implications of kernel threads like the missing
> > ability to handle transfer errors in user space.
>
> Are you talking about TCP here?
> Transfer errors are typically asynchronous - possibly eventfd
> as I expose for vhost net is sufficient there.
I mean errors in general if we allow random file descriptors to be used.
E.g. tun_chr_aio_read could return EBADFD, EINVAL, EFAULT, ERESTARTSYS,
EIO, EAGAIN and possibly others. We can handle some in kernel, others
should never happen with vhost_net, but if something unexpected happens
it would be nice to just bail out to user space.
> > > I wonder - can we expose the underlying socket used by tap, or will that
> > > create complex lifetime issues?
> >
> > I think this could get more messy in the long run than calling vfs_readv
> > on a random fd. It would mean deep internal knowledge of the tap driver
> > in vhost_net, which I really would prefer to avoid.
>
> No, what I had in mind is adding a GET_SOCKET ioctl to tap.
> vhost would then just use the socket.
Right, that would work with tun/tap at least. It sounds a bit fishy
but I can't see a reason why it would be hard to do.
I'd have to think about how to get it working with macvtap, or if
there is much value left in macvtap after that anyway.
> > So how about making the qemu command line interface an extension to
> > what Or Gerlitz has done for the raw packet sockets?
>
> Not sure I see the connection, but I have not thought about qemu
> side of things too much yet - trying to get kernel bits in place
> first so that there's a stable ABI to work with.
Ok, fair enough. The kernel bits are obviously more time critical
right now, since they should get into 2.6.32.
Arnd <><
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 13:27 UTC (permalink / raw)
To: Avi Kivity
Cc: Ingo Molnar, Anthony Liguori, kvm, alacrityvm-devel, linux-kernel,
netdev, Michael S. Tsirkin, Ira W. Snyder
In-Reply-To: <4A8B8C11.9030800@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 11351 bytes --]
Avi Kivity wrote:
> On 08/19/2009 07:27 AM, Gregory Haskins wrote:
>>
>>> This thread started because i asked you about your technical
>>> arguments why we'd want vbus instead of virtio.
>>>
>> (You mean vbus vs pci, right? virtio works fine, is untouched, and is
>> out-of-scope here)
>>
>
> I guess he meant venet vs virtio-net. Without venet vbus is currently
> userless.
>
>> Right, and I do believe I answered your questions. Do you feel as
>> though this was not a satisfactory response?
>>
>
> Others and I have shown you its wrong.
No, you have shown me that you disagree. I'm sorry, but do not assume
they are the same.
Case in point: You also said that threading the ethernet model was wrong
when I proposed it, and later conceded when I showed you the numbers
that you were wrong. I don't say this to be a jerk. I am wrong myself
all the time too.
I only say it to highlight that perhaps we just don't (yet) see each
others POV. Therefore, do not be so quick to put a "wrong" label on
something, especially when the line of questioning/debate indicates to
me that there are still fundamental issues in understanding exactly how
things work.
> There's no inherent performance
> problem in pci. The vbus approach has inherent problems (the biggest of
> which is compatibility
Trying to be backwards compatible in all dimensions is not a design
goal, as already stated.
, the second managability).
>
Where are the management problems?
>>> Your answer above
>>> now basically boils down to: "because I want it so, why dont you
>>> leave me alone".
>>>
>> Well, with all due respect, please do not put words in my mouth. This
>> is not what I am saying at all.
>>
>> What I *am* saying is:
>>
>> fact: this thread is about linux guest drivers to support vbus
>>
>> fact: these drivers do not touch kvm code.
>>
>> fact: these drivers to not force kvm to alter its operation in any way.
>>
>> fact: these drivers do not alter ABIs that KVM currently supports.
>>
>> Therefore, all this talk about "abandoning", "supporting", and
>> "changing" things in KVM is, premature, irrelevant, and/or, FUD. No one
>> proposed such changes, so I am highlighting this fact to bring the
>> thread back on topic. That KVM talk is merely a distraction at this
>> point in time.
>>
>
> s/kvm/kvm stack/. virtio/pci is part of the kvm stack, even if it is
> not part of kvm itself. If vbus/venet were to be merged, users and
> developers would have to choose one or the other. That's the
> fragmentation I'm worried about. And you can prefix that with "fact:"
> as well.
Noted
>
>>> We all love faster code and better management interfaces and tons
>>> of your prior patches got accepted by Avi. This time you didnt even
>>> _try_ to improve virtio.
>>>
>> Im sorry, but you are mistaken:
>>
>> http://lkml.indiana.edu/hypermail/linux/kernel/0904.2/02443.html
>>
>
> That does nothing to improve virtio.
I'm sorry, but thats just plain false.
> Existing guests (Linux and
> Windows) which support virtio will cease to work if the host moves to
> vbus-virtio.
Sigh...please re-read "fact" section. And even if this work is accepted
upstream as it is, how you configure the host and guest is just that: a
configuration. If your guest and host both speak vbus, use it. If they
don't, don't use it. Simple as that. Saying anything else is just more
FUD, and I can say the same thing about a variety of other configuration
options currently available.
> Existing hosts (running virtio-pci) won't be able to talk
> to newer guests running virtio-vbus. The patch doesn't improve
> performance without the entire vbus stack in the host kernel and a
> vbus-virtio-net-host host kernel driver.
<rewind years=2>Existing hosts (running realtek emulation) won't be able
to talk to newer guests running virtio-net. Virtio-net doesn't do
anything to improve realtek emulation without the entire virtio stack in
the host.</rewind>
You gotta start somewhere. You're argument buys you nothing other than
backwards compat, which I've already stated is not a specific goal here.
I am not against "modprobe vbus-pcibridge", and I am sure there are
users out that that do not object to this either.
>
> Perhaps if you posted everything needed to make vbus-virtio work and
> perform we could compare that to vhost-net and you'll see another reason
> why vhost-net is the better approach.
Yet, you must recognize that an alternative outcome is that we can look
at issues outside of virtio-net on KVM and perhaps you will see vbus is
a better approach.
>
>> You are also wrong to say that I didn't try to avoid creating a
>> downstream effort first. I believe the public record of the mailing
>> lists will back me up that I tried politely pushing this directly though
>> kvm first. It was only after Avi recently informed me that they would
>> be building their own version of an in-kernel backend in lieu of working
>> with me to adapt vbus to their needs that I decided to put my own
>> project together.
>>
>
> There's no way we can adapt vbus to our needs.
Really? Did you ever bother to ask how? I'm pretty sure you can. And
if you couldn't, I would have considered changes to make it work.
> Don't you think we'd preferred it rather than writing our own?
Honestly, I am not so sure based on your responses.
> the current virtio-net issues
> are hurting us.
Indeed.
>
> Our needs are compatibility, performance, and managability. vbus fails
> all three, your impressive venet numbers notwithstanding.
>
>> What should I have done otherwise, in your opinion?
>>
>
> You could come up with uses where vbus truly is superior to
> virtio/pci/whatever
I've already listed numerous examples on why I advocate vbus over PCI,
and have already stated I am not competing against virtio.
> (not words about etch constraints).
I was asked about the design, and that was background on some of my
motivations. Don't try to spin that into something its not.
> Showing some of those non-virt uses, for example.
Actually, Ira's chassis discussed earlier is a classic example. Vbus
actually fits neatly into his model, I believe (and much better than the
vhost proposals, IMO).
Basically, IMO we want to invert Ira's bus (so that the PPC boards see
host-based devices, instead of the other way around). You write a
connector that transports the vbus verbs over the PCI link. You write a
udev rule that responds to the PPC board "arrival" event to create a new
vbus container, and assign the board to that context.
Then, whatever devices you instantiate in the vbus container will
surface on the PPC board's "vbus-proxy" bus. This can include "virtio"
type devices which are serviced by the virtio-vbus code to render these
devices to the virtio-bus. Finally, drivers like virtio-net and
virtio-console load and run normally.
The host-side administers the available inventory on a per-board basis
and its configuration using sysfs operations.
> The fact that your only user duplicates existing functionality doesn't help.
Certainly at some level, that is true and is unfortunate, I agree. In
retrospect, I wish I started with something non-overlapping with virtio
as the demo, just to avoid this aspect of controversy.
At another level, its the highest-performance 802.x interface for KVM at
the moment, since we still have not seen benchmarks for vhost. Given
that I have spent a lot of time lately optimizing KVM, I can tell you
its not trivial to get it to work better than the userspace virtio.
Michael is clearly a smart guy, so the odds are in his favor. But do
not count your chickens before they hatch, because its not guaranteed
success.
Long story short, my patches are not duplicative on all levels (i.e.
performance). Its just another ethernet driver, of which there are
probably hundreds of alternatives in the kernel already. You could also
argue that we already have multiple models in qemu (realtek, e1000,
virtio-net, etc) so this is not without precedent. So really all this
"fragmentation" talk is FUD. Lets stay on-point, please.
>
>
>>> And fragmentation matters quite a bit. To Linux users, developers,
>>> administrators, packagers it's a big deal whether two overlapping
>>> pieces of functionality for the same thing exist within the same
>>> kernel.
>>>
>> So the only thing that could be construed as overlapping here is venet
>> vs virtio-net. If I dropped the contentious venet and focused on making
>> a virtio-net backend that we can all re-use, do you see that as a path
>> of compromise here?
>>
>
> That's a step in the right direction.
Ok. I am concerned it would be a waste of my time given your current
statements regarding the backend aspects of my design.
Can we talk more about that at some point? I think you will see its not
some "evil, heavy duty" infrastructure that some comments seem to be
trying to paint it as. I think its similar in concept to what you need
to do for a vhost like design, but (with all due respect to Michael) a
little bit more thought into the necessary abstraction points to allow
broader application.
>
>>> I certainly dont want that. Instead we (at great expense and work)
>>> try to reach the best technical solution.
>>>
>> This is all I want, as well.
>>
>
> Note whenever I mention migration, large guests, or Windows you say
> these are not your design requirements.
Actually, I don't think I've ever said that, per se. I said that those
things are not a priority for me, personally. I never made a design
decision that I knew would preclude the support for such concepts. In
fact, afaict, the design would support them just fine, given resources
the develop them.
For the record: I never once said "vbus is done". There is plenty of
work left to do. This is natural (kvm I'm sure wasn't 100% when it went
in either, nor is it today)
> The best technical solution will have to consider those.
We are on the same page here.
>
>>> If the community wants this then why cannot you convince one of the
>>> most prominent representatives of that community, the KVM
>>> developers?
>>>
>> Its a chicken and egg at times. Perhaps the KVM developers do not have
>> the motivation or time to properly consider such a proposal _until_ the
>> community presents its demand.
>
> I've spent quite a lot of time arguing with you, no doubt influenced by
> the fact that you can write a lot faster than I can read.
:)
>
>>> Furthermore, 99% of your work is KVM
>>>
>> Actually, no. Almost none of it is. I think there are about 2-3
>> patches in the series that touch KVM, the rest are all original (and
>> primarily stand-alone code). AlacrityVM is the application of kvm and
>> vbus (and, of course, Linux) together as a complete unit, but I do not
>> try to hide this relationship.
>>
>> By your argument, KVM is 99% QEMU+Linux. ;)
>>
>
> That's one of the kvm strong points...
As AlacrityVMs, as well ;)
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCHv3 2/2] vhost_net: a kernel-level virtio server
From: Michael S. Tsirkin @ 2009-08-19 13:04 UTC (permalink / raw)
To: Arnd Bergmann
Cc: virtualization, netdev, kvm, linux-kernel, mingo, linux-mm, akpm,
hpa, gregory.haskins, Or Gerlitz
In-Reply-To: <200908191104.50672.arnd@arndb.de>
On Wed, Aug 19, 2009 at 11:04:50AM +0200, Arnd Bergmann wrote:
> On Sunday 16 August 2009, Michael S. Tsirkin wrote:
> > On Fri, Aug 14, 2009 at 01:40:36PM +0200, Arnd Bergmann wrote:
> > >
> > > * most of the transports are sockets, tap uses a character device.
> > > This could be dealt with by having both a struct socket * in
> > > struct vhost_net *and* a struct file *, or by always keeping the
> > > struct file and calling vfs_readv/vfs_writev for the data transport
> > > in both cases.
> >
> > I am concerned that character devices might have weird side effects with
> > read/write operations and that calling them from kernel thread the way I
> > do might have security implications. Can't point at anything specific
> > though at the moment.
>
> I understand your feelings about passing a chardev fd into your driver
> and I agree that we need to be very careful if we want to allow it.
>
> Maybe we could instead extend the 'splice' system call to work on a
> vhost_net file descriptor. If we do that, we can put the access back
> into a user thread (or two) that stays in splice indefinetely
An issue with exposing internal threading model to userspace
in this way is that we lose control of e.g. CPU locality -
and it is very hard for userspace to get it right.
> to
> avoid some of the implications of kernel threads like the missing
> ability to handle transfer errors in user space.
Are you talking about TCP here?
Transfer errors are typically asynchronous - possibly eventfd
as I expose for vhost net is sufficient there.
> > I wonder - can we expose the underlying socket used by tap, or will that
> > create complex lifetime issues?
>
> I think this could get more messy in the long run than calling vfs_readv
> on a random fd. It would mean deep internal knowledge of the tap driver
> in vhost_net, which I really would prefer to avoid.
No, what I had in mind is adding a GET_SOCKET ioctl to tap.
vhost would then just use the socket.
> > > * Each transport has a slightly different header, we have
> > > - raw ethernet frames (raw, udp multicast, tap)
> > > - 32-bit length + raw frames, possibly fragmented (tcp)
> > > - 80-bit header + raw frames, possibly fragmented (tap with vnet_hdr)
> > > To handle these three cases, we need either different ioctl numbers
> > > so that vhost_net can choose the right one, or a flags field in
> > > VHOST_NET_SET_SOCKET, like
> > >
> > > #define VHOST_NET_RAW 1
> > > #define VHOST_NET_LEN_HDR 2
> > > #define VHOST_NET_VNET_HDR 4
> > >
> > > struct vhost_net_socket {
> > > unsigned int flags;
> > > int fd;
> > > };
> > > #define VHOST_NET_SET_SOCKET _IOW(VHOST_VIRTIO, 0x30, struct vhost_net_socket)
> >
> > It seems we can query the socket to find out the type,
>
> yes, I understand that you can do that, but I still think that decision
> should be left to user space. Adding a length header for TCP streams but
> not for UDP is something that we would normally want to do, but IMHO
> vhost_net should not need to know about this.
>
> > or use the features ioctl.
>
> Right, I had forgotten about that one. It's probably equivalent
> to the flags I suggested, except that one allows you to set features
> after starting the communication, while the other one prevents
> you from doing that.
>
> > > Qemu could then automatically try to use vhost_net, if it's available
> > > in the kernel, or just fall back on software vlan otherwise.
> > > Does that make sense?
> >
> > I agree, long term it should be enabled automatically when possible.
>
> So how about making the qemu command line interface an extension to
> what Or Gerlitz has done for the raw packet sockets?
>
> Arnd <><
Not sure I see the connection, but I have not thought about qemu
side of things too much yet - trying to get kernel bits in place
first so that there's a stable ABI to work with.
--
MST
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] [V2] net: add Xilinx emac lite device driver
From: John Linn @ 2009-08-19 12:29 UTC (permalink / raw)
To: netdev, linuxppc-dev, jgarzik, davem
Cc: John Linn, Grant Likely, Josh Boyer, John Williams, Michal Simek,
Sadanand M
This patch adds support for the Xilinx Ethernet Lite device. The
soft logic core from Xilinx is typically used on Virtex and Spartan
designs attached to either a PowerPC or a Microblaze processor.
CC: Grant Likely <grant.likely@secretlab.ca>
CC: Josh Boyer <jwboyer@linux.vnet.ibm.com>
CC: John Williams <john.williams@petalogix.com>
CC: Michal Simek <michal.simek@petalogix.com>
Signed-off-by: Sadanand M <sadanan@xilinx.com>
Signed-off-by: John Linn <john.linn@xilinx.com>
---
V2 - cleanup based on review, added depends for ppc and mb in Kconfig
---
drivers/net/Kconfig | 6 +
drivers/net/Makefile | 1 +
drivers/net/xilinx_emaclite.c | 1040 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 1047 insertions(+), 0 deletions(-)
create mode 100755 drivers/net/xilinx_emaclite.c
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 5f6509a..ec77b69 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -1926,6 +1926,12 @@ config ATL2
To compile this driver as a module, choose M here. The module
will be called atl2.
+config XILINX_EMACLITE
+ tristate "Xilinx 10/100 Ethernet Lite support"
+ depends on PPC32 || MICROBLAZE
+ help
+ This driver supports the 10/100 Ethernet Lite from Xilinx.
+
source "drivers/net/fs_enet/Kconfig"
endif # NET_ETHERNET
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index ead8cab..99ae6d7 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -142,6 +142,7 @@ obj-$(CONFIG_TSI108_ETH) += tsi108_eth.o
obj-$(CONFIG_MV643XX_ETH) += mv643xx_eth.o
ll_temac-objs := ll_temac_main.o ll_temac_mdio.o
obj-$(CONFIG_XILINX_LL_TEMAC) += ll_temac.o
+obj-$(CONFIG_XILINX_EMACLITE) += xilinx_emaclite.o
obj-$(CONFIG_QLA3XXX) += qla3xxx.o
obj-$(CONFIG_QLGE) += qlge/
diff --git a/drivers/net/xilinx_emaclite.c b/drivers/net/xilinx_emaclite.c
new file mode 100755
index 0000000..3716e20
--- /dev/null
+++ b/drivers/net/xilinx_emaclite.c
@@ -0,0 +1,1040 @@
+/*
+ * Xilinx EmacLite Linux driver for the Xilinx Ethernet MAC Lite device.
+ *
+ * This is a new flat driver which is based on the original emac_lite
+ * driver from John Williams <john.williams@petalogix.com>.
+ *
+ * 2007-2009 (c) Xilinx, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/uaccess.h>
+#include <linux/init.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/skbuff.h>
+#include <linux/io.h>
+
+#include <linux/of_device.h>
+#include <linux/of_platform.h>
+
+#define DRIVER_NAME "xilinx_emaclite"
+
+/* Register offsets for the EmacLite Core */
+#define XEL_TXBUFF_OFFSET 0x0 /* Transmit Buffer */
+#define XEL_GIER_OFFSET 0x07F8 /* GIE Register */
+#define XEL_TSR_OFFSET 0x07FC /* Tx status */
+#define XEL_TPLR_OFFSET 0x07F4 /* Tx packet length */
+
+#define XEL_RXBUFF_OFFSET 0x1000 /* Receive Buffer */
+#define XEL_RPLR_OFFSET 0x100C /* Rx packet length */
+#define XEL_RSR_OFFSET 0x17FC /* Rx status */
+
+#define XEL_BUFFER_OFFSET 0x0800 /* Next Tx/Rx buffer's offset */
+
+/* Global Interrupt Enable Register (GIER) Bit Masks */
+#define XEL_GIER_GIE_MASK 0x80000000 /* Global Enable */
+
+/* Transmit Status Register (TSR) Bit Masks */
+#define XEL_TSR_XMIT_BUSY_MASK 0x00000001 /* Tx complete */
+#define XEL_TSR_PROGRAM_MASK 0x00000002 /* Program the MAC address */
+#define XEL_TSR_XMIT_IE_MASK 0x00000008 /* Tx interrupt enable bit */
+#define XEL_TSR_XMIT_ACTIVE_MASK 0x80000000 /* Buffer is active, SW bit
+ * only. This is not documented
+ * in the HW spec */
+
+/* Define for programming the MAC address into the EmacLite */
+#define XEL_TSR_PROG_MAC_ADDR (XEL_TSR_XMIT_BUSY_MASK | XEL_TSR_PROGRAM_MASK)
+
+/* Receive Status Register (RSR) */
+#define XEL_RSR_RECV_DONE_MASK 0x00000001 /* Rx complete */
+#define XEL_RSR_RECV_IE_MASK 0x00000008 /* Rx interrupt enable bit */
+
+/* Transmit Packet Length Register (TPLR) */
+#define XEL_TPLR_LENGTH_MASK 0x0000FFFF /* Tx packet length */
+
+/* Receive Packet Length Register (RPLR) */
+#define XEL_RPLR_LENGTH_MASK 0x0000FFFF /* Rx packet length */
+
+#define XEL_HEADER_OFFSET 12 /* Offset to length field */
+#define XEL_HEADER_SHIFT 16 /* Shift value for length */
+
+/* General Ethernet Definitions */
+#define XEL_ARP_PACKET_SIZE 28 /* Max ARP packet size */
+#define XEL_HEADER_IP_LENGTH_OFFSET 16 /* IP Length Offset */
+
+
+
+#define TX_TIMEOUT (60*HZ) /* Tx timeout is 60 seconds. */
+#define ALIGNMENT 4
+
+/* BUFFER_ALIGN(adr) calculates the number of bytes to the next alignment. */
+#define BUFFER_ALIGN(adr) ((ALIGNMENT - ((u32) adr)) % ALIGNMENT)
+
+/**
+ * struct net_local - Our private per device data
+ * @ndev: instance of the network device
+ * @tx_ping_pong: indicates whether Tx Pong buffer is configured in HW
+ * @rx_ping_pong: indicates whether Rx Pong buffer is configured in HW
+ * @next_tx_buf_to_use: next Tx buffer to write to
+ * @next_rx_buf_to_use: next Rx buffer to read from
+ * @base_addr: base address of the Emaclite device
+ * @reset_lock: lock used for synchronization
+ * @deferred_skb: holds an skb (for transmission at a later time) when the
+ * Tx buffer is not free
+ */
+struct net_local {
+
+ struct net_device *ndev;
+
+ bool tx_ping_pong;
+ bool rx_ping_pong;
+ u32 next_tx_buf_to_use;
+ u32 next_rx_buf_to_use;
+ void __iomem *base_addr;
+
+ spinlock_t reset_lock;
+ struct sk_buff *deferred_skb;
+};
+
+
+/*************************/
+/* EmacLite driver calls */
+/*************************/
+
+/**
+ * xemaclite_enable_interrupts - Enable the interrupts for the EmacLite device
+ * @drvdata: Pointer to the Emaclite device private data
+ *
+ * This function enables the Tx and Rx interrupts for the Emaclite device along
+ * with the Global Interrupt Enable.
+ */
+static void xemaclite_enable_interrupts(struct net_local *drvdata)
+{
+ u32 reg_data;
+
+ /* Enable the Tx interrupts for the first Buffer */
+ reg_data = in_be32(drvdata->base_addr + XEL_TSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_TSR_OFFSET,
+ reg_data | XEL_TSR_XMIT_IE_MASK);
+
+ /* Enable the Tx interrupts for the second Buffer if
+ * configured in HW */
+ if (drvdata->tx_ping_pong != 0) {
+ reg_data = in_be32(drvdata->base_addr +
+ XEL_BUFFER_OFFSET + XEL_TSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_TSR_OFFSET,
+ reg_data | XEL_TSR_XMIT_IE_MASK);
+ }
+
+ /* Enable the Rx interrupts for the first buffer */
+ reg_data = in_be32(drvdata->base_addr + XEL_RSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_RSR_OFFSET,
+ reg_data | XEL_RSR_RECV_IE_MASK);
+
+ /* Enable the Rx interrupts for the second Buffer if
+ * configured in HW */
+ if (drvdata->rx_ping_pong != 0) {
+ reg_data = in_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_RSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_RSR_OFFSET,
+ reg_data | XEL_RSR_RECV_IE_MASK);
+ }
+
+ /* Enable the Global Interrupt Enable */
+ out_be32(drvdata->base_addr + XEL_GIER_OFFSET, XEL_GIER_GIE_MASK);
+}
+
+/**
+ * xemaclite_disable_interrupts - Disable the interrupts for the EmacLite device
+ * @drvdata: Pointer to the Emaclite device private data
+ *
+ * This function disables the Tx and Rx interrupts for the Emaclite device,
+ * along with the Global Interrupt Enable.
+ */
+static void xemaclite_disable_interrupts(struct net_local *drvdata)
+{
+ u32 reg_data;
+
+ /* Disable the Global Interrupt Enable */
+ out_be32(drvdata->base_addr + XEL_GIER_OFFSET, XEL_GIER_GIE_MASK);
+
+ /* Disable the Tx interrupts for the first buffer */
+ reg_data = in_be32(drvdata->base_addr + XEL_TSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_TSR_OFFSET,
+ reg_data & (~XEL_TSR_XMIT_IE_MASK));
+
+ /* Disable the Tx interrupts for the second Buffer
+ * if configured in HW */
+ if (drvdata->tx_ping_pong != 0) {
+ reg_data = in_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_TSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_TSR_OFFSET,
+ reg_data & (~XEL_TSR_XMIT_IE_MASK));
+ }
+
+ /* Disable the Rx interrupts for the first buffer */
+ reg_data = in_be32(drvdata->base_addr + XEL_RSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_RSR_OFFSET,
+ reg_data & (~XEL_RSR_RECV_IE_MASK));
+
+ /* Disable the Rx interrupts for the second buffer
+ * if configured in HW */
+ if (drvdata->rx_ping_pong != 0) {
+
+ reg_data = in_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_RSR_OFFSET);
+ out_be32(drvdata->base_addr + XEL_BUFFER_OFFSET +
+ XEL_RSR_OFFSET,
+ reg_data & (~XEL_RSR_RECV_IE_MASK));
+ }
+}
+
+/**
+ * xemaclite_aligned_write - Write from 16-bit aligned to 32-bit aligned address
+ * @src_ptr: Void pointer to the 16-bit aligned source address
+ * @dest_ptr: Pointer to the 32-bit aligned destination address
+ * @length: Number bytes to write from source to destination
+ *
+ * This function writes data from a 16-bit aligned buffer to a 32-bit aligned
+ * address in the EmacLite device.
+ */
+static void xemaclite_aligned_write(void *src_ptr, u32 *dest_ptr,
+ unsigned length)
+{
+ u32 align_buffer;
+ u32 *to_u32_ptr;
+ u16 *from_u16_ptr, *to_u16_ptr;
+
+ to_u32_ptr = dest_ptr;
+ from_u16_ptr = (u16 *) src_ptr;
+ align_buffer = 0;
+
+ for (; length > 3; length -= 4) {
+ to_u16_ptr = (u16 *) ((void *) &align_buffer);
+ *to_u16_ptr++ = *from_u16_ptr++;
+ *to_u16_ptr++ = *from_u16_ptr++;
+
+ /* Output a word */
+ *to_u32_ptr++ = align_buffer;
+ }
+ if (length) {
+ u8 *from_u8_ptr, *to_u8_ptr;
+
+ /* Set up to output the remaining data */
+ align_buffer = 0;
+ to_u8_ptr = (u8 *) &align_buffer;
+ from_u8_ptr = (u8 *) from_u16_ptr;
+
+ /* Output the remaining data */
+ for (; length > 0; length--)
+ *to_u8_ptr++ = *from_u8_ptr++;
+
+ *to_u32_ptr = align_buffer;
+ }
+}
+
+/**
+ * xemaclite_aligned_read - Read from 32-bit aligned to 16-bit aligned buffer
+ * @src_ptr: Pointer to the 32-bit aligned source address
+ * @dest_ptr: Pointer to the 16-bit aligned destination address
+ * @length: Number bytes to read from source to destination
+ *
+ * This function reads data from a 32-bit aligned address in the EmacLite device
+ * to a 16-bit aligned buffer.
+ */
+static void xemaclite_aligned_read(u32 *src_ptr, u8 *dest_ptr,
+ unsigned length)
+{
+ u16 *to_u16_ptr, *from_u16_ptr;
+ u32 *from_u32_ptr;
+ u32 align_buffer;
+
+ from_u32_ptr = src_ptr;
+ to_u16_ptr = (u16 *) dest_ptr;
+
+ for (; length > 3; length -= 4) {
+ /* Copy each word into the temporary buffer */
+ align_buffer = *from_u32_ptr++;
+ from_u16_ptr = (u16 *)&align_buffer;
+
+ /* Read data from source */
+ *to_u16_ptr++ = *from_u16_ptr++;
+ *to_u16_ptr++ = *from_u16_ptr++;
+ }
+
+ if (length) {
+ u8 *to_u8_ptr, *from_u8_ptr;
+
+ /* Set up to read the remaining data */
+ to_u8_ptr = (u8 *) to_u16_ptr;
+ align_buffer = *from_u32_ptr++;
+ from_u8_ptr = (u8 *) &align_buffer;
+
+ /* Read the remaining data */
+ for (; length > 0; length--)
+ *to_u8_ptr = *from_u8_ptr;
+ }
+}
+
+/**
+ * xemaclite_send_data - Send an Ethernet frame
+ * @drvdata: Pointer to the Emaclite device private data
+ * @data: Pointer to the data to be sent
+ * @byte_count: Total frame size, including header
+ *
+ * This function checks if the Tx buffer of the Emaclite device is free to send
+ * data. If so, it fills the Tx buffer with data for transmission. Otherwise, it
+ * returns an error.
+ *
+ * Return: 0 upon success or -1 if the buffer(s) are full.
+ *
+ * Note: The maximum Tx packet size can not be more than Ethernet header
+ * (14 Bytes) + Maximum MTU (1500 bytes). This is excluding FCS.
+ */
+static int xemaclite_send_data(struct net_local *drvdata, u8 *data,
+ unsigned int byte_count)
+{
+ u32 reg_data;
+ void __iomem *addr;
+
+ /* Determine the expected Tx buffer address */
+ addr = drvdata->base_addr + drvdata->next_tx_buf_to_use;
+
+ /* If the length is too large, truncate it */
+ if (byte_count > ETH_FRAME_LEN)
+ byte_count = ETH_FRAME_LEN;
+
+ /* Check if the expected buffer is available */
+ reg_data = in_be32(addr + XEL_TSR_OFFSET);
+ if ((reg_data & (XEL_TSR_XMIT_BUSY_MASK |
+ XEL_TSR_XMIT_ACTIVE_MASK)) == 0) {
+
+ /* Switch to next buffer if configured */
+ if (drvdata->tx_ping_pong != 0)
+ drvdata->next_tx_buf_to_use ^= XEL_BUFFER_OFFSET;
+ } else if (drvdata->tx_ping_pong != 0) {
+ /* If the expected buffer is full, try the other buffer,
+ * if it is configured in HW */
+
+ addr = (void __iomem __force *)((u32 __force)addr ^
+ XEL_BUFFER_OFFSET);
+ reg_data = in_be32(addr + XEL_TSR_OFFSET);
+
+ if ((reg_data & (XEL_TSR_XMIT_BUSY_MASK |
+ XEL_TSR_XMIT_ACTIVE_MASK)) != 0)
+ return -1; /* Buffers were full, return failure */
+ } else
+ return -1; /* Buffer was full, return failure */
+
+ /* Write the frame to the buffer */
+ xemaclite_aligned_write(data, (u32 __force *) addr, byte_count);
+
+ out_be32(addr + XEL_TPLR_OFFSET, (byte_count & XEL_TPLR_LENGTH_MASK));
+
+ /* Update the Tx Status Register to indicate that there is a
+ * frame to send. Set the XEL_TSR_XMIT_ACTIVE_MASK flag which
+ * is used by the interrupt handler to check whether a frame
+ * has been transmitted */
+ reg_data = in_be32(addr + XEL_TSR_OFFSET);
+ reg_data |= (XEL_TSR_XMIT_BUSY_MASK | XEL_TSR_XMIT_ACTIVE_MASK);
+ out_be32(addr + XEL_TSR_OFFSET, reg_data);
+
+ return 0;
+}
+
+/**
+ * xemaclite_recv_data - Receive a frame
+ * @drvdata: Pointer to the Emaclite device private data
+ * @data: Address where the data is to be received
+ *
+ * This function is intended to be called from the interrupt context or
+ * with a wrapper which waits for the receive frame to be available.
+ *
+ * Return: Total number of bytes received
+ */
+static u16 xemaclite_recv_data(struct net_local *drvdata, u8 *data)
+{
+ void __iomem *addr;
+ u16 length, proto_type;
+ u32 reg_data;
+
+ /* Determine the expected buffer address */
+ addr = (drvdata->base_addr + drvdata->next_rx_buf_to_use);
+
+ /* Verify which buffer has valid data */
+ reg_data = in_be32(addr + XEL_RSR_OFFSET);
+
+ if ((reg_data & XEL_RSR_RECV_DONE_MASK) == XEL_RSR_RECV_DONE_MASK) {
+ if (drvdata->rx_ping_pong != 0)
+ drvdata->next_rx_buf_to_use ^= XEL_BUFFER_OFFSET;
+ } else {
+ /* The instance is out of sync, try other buffer if other
+ * buffer is configured, return 0 otherwise. If the instance is
+ * out of sync, do not update the 'next_rx_buf_to_use' since it
+ * will correct on subsequent calls */
+ if (drvdata->rx_ping_pong != 0)
+ addr = (void __iomem __force *)((u32 __force)addr ^
+ XEL_BUFFER_OFFSET);
+ else
+ return 0; /* No data was available */
+
+ /* Verify that buffer has valid data */
+ reg_data = in_be32(addr + XEL_RSR_OFFSET);
+ if ((reg_data & XEL_RSR_RECV_DONE_MASK) !=
+ XEL_RSR_RECV_DONE_MASK)
+ return 0; /* No data was available */
+ }
+
+ /* Get the protocol type of the ethernet frame that arrived */
+ proto_type = ((in_be32(addr + XEL_HEADER_OFFSET +
+ XEL_RXBUFF_OFFSET) >> XEL_HEADER_SHIFT) &
+ XEL_RPLR_LENGTH_MASK);
+
+ /* Check if received ethernet frame is a raw ethernet frame
+ * or an IP packet or an ARP packet */
+ if (proto_type > (ETH_FRAME_LEN + ETH_FCS_LEN)) {
+
+ if (proto_type == ETH_P_IP) {
+ length = ((in_be32(addr +
+ XEL_HEADER_IP_LENGTH_OFFSET +
+ XEL_RXBUFF_OFFSET) >>
+ XEL_HEADER_SHIFT) &
+ XEL_RPLR_LENGTH_MASK);
+ length += ETH_HLEN + ETH_FCS_LEN;
+
+ } else if (proto_type == ETH_P_ARP)
+ length = XEL_ARP_PACKET_SIZE + ETH_HLEN + ETH_FCS_LEN;
+ else
+ /* Field contains type other than IP or ARP, use max
+ * frame size and let user parse it */
+ length = ETH_FRAME_LEN + ETH_FCS_LEN;
+ } else
+ /* Use the length in the frame, plus the header and trailer */
+ length = proto_type + ETH_HLEN + ETH_FCS_LEN;
+
+ /* Read from the EmacLite device */
+ xemaclite_aligned_read((u32 __force *) (addr + XEL_RXBUFF_OFFSET),
+ data, length);
+
+ /* Acknowledge the frame */
+ reg_data = in_be32(addr + XEL_RSR_OFFSET);
+ reg_data &= ~XEL_RSR_RECV_DONE_MASK;
+ out_be32(addr + XEL_RSR_OFFSET, reg_data);
+
+ return length;
+}
+
+/**
+ * xemaclite_set_mac_address - Set the MAC address for this device
+ * @drvdata: Pointer to the Emaclite device private data
+ * @address_ptr:Pointer to the MAC address (MAC address is a 48-bit value)
+ *
+ * Tx must be idle and Rx should be idle for deterministic results.
+ * It is recommended that this function should be called after the
+ * initialization and before transmission of any packets from the device.
+ * The MAC address can be programmed using any of the two transmit
+ * buffers (if configured).
+ */
+static void xemaclite_set_mac_address(struct net_local *drvdata,
+ u8 *address_ptr)
+{
+ void __iomem *addr;
+ u32 reg_data;
+
+ /* Determine the expected Tx buffer address */
+ addr = drvdata->base_addr + drvdata->next_tx_buf_to_use;
+
+ xemaclite_aligned_write(address_ptr, (u32 __force *) addr, ETH_ALEN);
+
+ out_be32(addr + XEL_TPLR_OFFSET, ETH_ALEN);
+
+ /* Update the MAC address in the EmacLite */
+ reg_data = in_be32(addr + XEL_TSR_OFFSET);
+ out_be32(addr + XEL_TSR_OFFSET, reg_data | XEL_TSR_PROG_MAC_ADDR);
+
+ /* Wait for EmacLite to finish with the MAC address update */
+ while ((in_be32(addr + XEL_TSR_OFFSET) &
+ XEL_TSR_PROG_MAC_ADDR) != 0)
+ ;
+}
+
+/**
+ * xemaclite_tx_timeout - Callback for Tx Timeout
+ * @dev: Pointer to the network device
+ *
+ * This function is called when Tx time out occurs for Emaclite device.
+ */
+static void xemaclite_tx_timeout(struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ unsigned long flags;
+
+ dev_err(&lp->ndev->dev, "Exceeded transmit timeout of %lu ms\n",
+ TX_TIMEOUT * 1000UL / HZ);
+
+ dev->stats.tx_errors++;
+
+ /* Reset the device */
+ spin_lock_irqsave(&lp->reset_lock, flags);
+
+ /* Shouldn't really be necessary, but shouldn't hurt */
+ netif_stop_queue(dev);
+
+ xemaclite_disable_interrupts(lp);
+ xemaclite_enable_interrupts(lp);
+
+ if (lp->deferred_skb) {
+ dev_kfree_skb(lp->deferred_skb);
+ lp->deferred_skb = NULL;
+ dev->stats.tx_errors++;
+ }
+
+ /* To exclude tx timeout */
+ dev->trans_start = 0xffffffff - TX_TIMEOUT - TX_TIMEOUT;
+
+ /* We're all ready to go. Start the queue */
+ netif_wake_queue(dev);
+ spin_unlock_irqrestore(&lp->reset_lock, flags);
+}
+
+/**********************/
+/* Interrupt Handlers */
+/**********************/
+
+/**
+ * xemaclite_tx_handler - Interrupt handler for frames sent
+ * @dev: Pointer to the network device
+ *
+ * This function updates the number of packets transmitted and handles the
+ * deferred skb, if there is one.
+ */
+static void xemaclite_tx_handler(struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+
+ dev->stats.tx_packets++;
+ if (lp->deferred_skb) {
+ if (xemaclite_send_data(lp,
+ (u8 *) lp->deferred_skb->data,
+ lp->deferred_skb->len) != 0)
+ return;
+ else {
+ dev->stats.tx_bytes += lp->deferred_skb->len;
+ dev_kfree_skb_irq(lp->deferred_skb);
+ lp->deferred_skb = NULL;
+ dev->trans_start = jiffies;
+ netif_wake_queue(dev);
+ }
+ }
+}
+
+/**
+ * xemaclite_rx_handler- Interrupt handler for frames received
+ * @dev: Pointer to the network device
+ *
+ * This function allocates memory for a socket buffer, fills it with data
+ * received and hands it over to the TCP/IP stack.
+ */
+static void xemaclite_rx_handler(struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ struct sk_buff *skb;
+ unsigned int align;
+ u32 len;
+
+ len = ETH_FRAME_LEN + ETH_FCS_LEN;
+ skb = dev_alloc_skb(len + ALIGNMENT);
+ if (!skb) {
+ /* Couldn't get memory. */
+ dev->stats.rx_dropped++;
+ dev_err(&lp->ndev->dev, "Could not allocate receive buffer\n");
+ return;
+ }
+
+ /*
+ * A new skb should have the data halfword aligned, but this code is
+ * here just in case that isn't true. Calculate how many
+ * bytes we should reserve to get the data to start on a word
+ * boundary */
+ align = BUFFER_ALIGN(skb->data);
+ if (align)
+ skb_reserve(skb, align);
+
+ skb_reserve(skb, 2);
+
+ len = xemaclite_recv_data(lp, (u8 *) skb->data);
+
+ if (!len) {
+ dev->stats.rx_errors++;
+ dev_kfree_skb_irq(skb);
+ return;
+ }
+
+ skb_put(skb, len); /* Tell the skb how much data we got */
+ skb->dev = dev; /* Fill out required meta-data */
+
+ skb->protocol = eth_type_trans(skb, dev);
+ skb->ip_summed = CHECKSUM_NONE;
+
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += len;
+ dev->last_rx = jiffies;
+
+ netif_rx(skb); /* Send the packet upstream */
+}
+
+/**
+ * xemaclite_interrupt - Interrupt handler for this driver
+ * @irq: Irq of the Emaclite device
+ * @dev_id: Void pointer to the network device instance used as callback
+ * reference
+ *
+ * This function handles the Tx and Rx interrupts of the EmacLite device.
+ */
+static irqreturn_t xemaclite_interrupt(int irq, void *dev_id)
+{
+ bool tx_complete = 0;
+ struct net_device *dev = dev_id;
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ void __iomem *base_addr = lp->base_addr;
+ u32 tx_status;
+
+ /* Check if there is Rx Data available */
+ if ((in_be32(base_addr + XEL_RSR_OFFSET) & XEL_RSR_RECV_DONE_MASK) ||
+ (in_be32(base_addr + XEL_BUFFER_OFFSET + XEL_RSR_OFFSET)
+ & XEL_RSR_RECV_DONE_MASK))
+
+ xemaclite_rx_handler(dev);
+
+ /* Check if the Transmission for the first buffer is completed */
+ tx_status = in_be32(base_addr + XEL_TSR_OFFSET);
+ if (((tx_status & XEL_TSR_XMIT_BUSY_MASK) == 0) &&
+ (tx_status & XEL_TSR_XMIT_ACTIVE_MASK) != 0) {
+
+ tx_status &= ~XEL_TSR_XMIT_ACTIVE_MASK;
+ out_be32(base_addr + XEL_TSR_OFFSET, tx_status);
+
+ tx_complete = 1;
+ }
+
+ /* Check if the Transmission for the second buffer is completed */
+ tx_status = in_be32(base_addr + XEL_BUFFER_OFFSET + XEL_TSR_OFFSET);
+ if (((tx_status & XEL_TSR_XMIT_BUSY_MASK) == 0) &&
+ (tx_status & XEL_TSR_XMIT_ACTIVE_MASK) != 0) {
+
+ tx_status &= ~XEL_TSR_XMIT_ACTIVE_MASK;
+ out_be32(base_addr + XEL_BUFFER_OFFSET + XEL_TSR_OFFSET,
+ tx_status);
+
+ tx_complete = 1;
+ }
+
+ /* If there was a Tx interrupt, call the Tx Handler */
+ if (tx_complete != 0)
+ xemaclite_tx_handler(dev);
+
+ return IRQ_HANDLED;
+}
+
+/**
+ * xemaclite_open - Open the network device
+ * @dev: Pointer to the network device
+ *
+ * This function sets the MAC address, requests an IRQ and enables interrupts
+ * for the Emaclite device and starts the Tx queue.
+ */
+static int xemaclite_open(struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ int retval;
+
+ /* Just to be safe, stop the device first */
+ xemaclite_disable_interrupts(lp);
+
+ /* Set the MAC address each time opened */
+ xemaclite_set_mac_address(lp, dev->dev_addr);
+
+ /* Grab the IRQ */
+ retval = request_irq(dev->irq, &xemaclite_interrupt, 0, dev->name, dev);
+ if (retval) {
+ dev_err(&lp->ndev->dev, "Could not allocate interrupt %d\n",
+ dev->irq);
+ return retval;
+ }
+
+ /* Enable Interrupts */
+ xemaclite_enable_interrupts(lp);
+
+ /* We're ready to go */
+ netif_start_queue(dev);
+
+ return 0;
+}
+
+/**
+ * xemaclite_close - Close the network device
+ * @dev: Pointer to the network device
+ *
+ * This function stops the Tx queue, disables interrupts and frees the IRQ for
+ * the Emaclite device.
+ */
+static int xemaclite_close(struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+
+ netif_stop_queue(dev);
+ xemaclite_disable_interrupts(lp);
+ free_irq(dev->irq, dev);
+
+ return 0;
+}
+
+/**
+ * xemaclite_get_stats - Get the stats for the net_device
+ * @dev: Pointer to the network device
+ *
+ * This function returns the address of the 'net_device_stats' structure for the
+ * given network device. This structure holds usage statistics for the network
+ * device.
+ *
+ * Return: Pointer to the net_device_stats structure.
+ */
+static struct net_device_stats *xemaclite_get_stats(struct net_device *dev)
+{
+ return &dev->stats;
+}
+
+/**
+ * xemaclite_send - Transmit a frame
+ * @orig_skb: Pointer to the socket buffer to be transmitted
+ * @dev: Pointer to the network device
+ *
+ * This function checks if the Tx buffer of the Emaclite device is free to send
+ * data. If so, it fills the Tx buffer with data from socket buffer data,
+ * updates the stats and frees the socket buffer. The Tx completion is signaled
+ * by an interrupt. If the Tx buffer isn't free, then the socket buffer is
+ * deferred and the Tx queue is stopped so that the deferred socket buffer can
+ * be transmitted when the Emaclite device is free to transmit data.
+ *
+ * Return: 0, always.
+ */
+static int xemaclite_send(struct sk_buff *orig_skb, struct net_device *dev)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ struct sk_buff *new_skb;
+ unsigned int len;
+ unsigned long flags;
+
+ len = orig_skb->len;
+
+ new_skb = orig_skb;
+
+ spin_lock_irqsave(&lp->reset_lock, flags);
+ if (xemaclite_send_data(lp, (u8 *) new_skb->data, len) != 0) {
+ /* If the Emaclite Tx buffer is busy, stop the Tx queue and
+ * defer the skb for transmission at a later point when the
+ * current transmission is complete */
+ netif_stop_queue(dev);
+ lp->deferred_skb = new_skb;
+ spin_unlock_irqrestore(&lp->reset_lock, flags);
+ return 0;
+ }
+ spin_unlock_irqrestore(&lp->reset_lock, flags);
+
+ dev->stats.tx_bytes += len;
+ dev_kfree_skb(new_skb);
+ dev->trans_start = jiffies;
+
+ return 0;
+}
+
+/**
+ * xemaclite_ioctl - Perform IO Control operations on the network device
+ * @dev: Pointer to the network device
+ * @rq: Pointer to the interface request structure
+ * @cmd: IOCTL command
+ *
+ * The only IOCTL operation supported by this function is setting the MAC
+ * address. An error is reported if any other operations are requested.
+ *
+ * Return: 0 to indicate success, or a negative error for failure.
+ */
+static int xemaclite_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
+{
+ struct net_local *lp = (struct net_local *) netdev_priv(dev);
+ struct hw_addr_data *hw_addr = (struct hw_addr_data *) &rq->ifr_hwaddr;
+
+ switch (cmd) {
+ case SIOCETHTOOL:
+ return -EIO;
+
+ case SIOCSIFHWADDR:
+ dev_err(&lp->ndev->dev, "SIOCSIFHWADDR\n");
+
+ /* Copy MAC address in from user space */
+ copy_from_user((void __force *) dev->dev_addr,
+ (void __user __force *) hw_addr,
+ IFHWADDRLEN);
+ xemaclite_set_mac_address(lp, dev->dev_addr);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+/**
+ * xemaclite_remove_ndev - Free the network device
+ * @ndev: Pointer to the network device to be freed
+ *
+ * This function un maps the IO region of the Emaclite device and frees the net
+ * device.
+ */
+static void xemaclite_remove_ndev(struct net_device *ndev)
+{
+ if (ndev) {
+ struct net_local *lp = (struct net_local *) netdev_priv(ndev);
+
+ if (lp->base_addr)
+ iounmap((void __iomem __force *) (lp->base_addr));
+ free_netdev(ndev);
+ }
+}
+
+/**
+ * get_bool - Get a parameter from the OF device
+ * @ofdev: Pointer to OF device structure
+ * @s: Property to be retrieved
+ *
+ * This function looks for a property in the device node and returns the value
+ * of the property if its found or 0 if the property is not found.
+ *
+ * Return: Value of the parameter if the parameter is found, or 0 otherwise
+ */
+static bool get_bool(struct of_device *ofdev, const char *s)
+{
+ u32 *p = (u32 *)of_get_property(ofdev->node, s, NULL);
+
+ if (p) {
+ return (bool)*p;
+ } else {
+ dev_warn(&ofdev->dev, "Parameter %s not found,"
+ "defaulting to false\n", s);
+ return 0;
+ }
+}
+
+static struct net_device_ops xemaclite_netdev_ops;
+
+/**
+ * xemaclite_of_probe - Probe method for the Emaclite device.
+ * @ofdev: Pointer to OF device structure
+ * @match: Pointer to the structure used for matching a device
+ *
+ * This function probes for the Emaclite device in the device tree.
+ * It initializes the driver data structure and the hardware, sets the MAC
+ * address and registers the network device.
+ *
+ * Return: 0, if the driver is bound to the Emaclite device, or
+ * a negative error if there is failure.
+ */
+static int __devinit xemaclite_of_probe(struct of_device *ofdev,
+ const struct of_device_id *match)
+{
+ struct resource r_irq; /* Interrupt resources */
+ struct resource r_mem; /* IO mem resources */
+ struct net_device *ndev = NULL;
+ struct net_local *lp = NULL;
+ struct device *dev = &ofdev->dev;
+ const void *mac_address;
+
+ int rc = 0;
+
+ dev_info(dev, "Device Tree Probing\n");
+
+ /* Get iospace for the device */
+ rc = of_address_to_resource(ofdev->node, 0, &r_mem);
+ if (rc) {
+ dev_err(dev, "invalid address\n");
+ return rc;
+ }
+
+ /* Get IRQ for the device */
+ rc = of_irq_to_resource(ofdev->node, 0, &r_irq);
+ if (rc == NO_IRQ) {
+ dev_err(dev, "no IRQ found\n");
+ return rc;
+ }
+
+ /* Create an ethernet device instance */
+ ndev = alloc_etherdev(sizeof(struct net_local));
+ if (!ndev) {
+ dev_err(dev, "Could not allocate network device\n");
+ return -ENOMEM;
+ }
+
+ dev_set_drvdata(dev, ndev);
+
+ ndev->irq = r_irq.start;
+ ndev->mem_start = r_mem.start;
+ ndev->mem_end = r_mem.end;
+
+ lp = netdev_priv(ndev);
+ lp->ndev = ndev;
+
+ if (!request_mem_region(ndev->mem_start,
+ ndev->mem_end - ndev->mem_start + 1,
+ DRIVER_NAME)) {
+ dev_err(dev, "Couldn't lock memory region at %p\n",
+ (void *)ndev->mem_start);
+ rc = -EBUSY;
+ goto error2;
+ }
+
+ /* Get the virtual base address for the device */
+ lp->base_addr = ioremap(r_mem.start, r_mem.end - r_mem.start + 1);
+ if (NULL == lp->base_addr) {
+ dev_err(dev, "EmacLite: Could not allocate iomem\n");
+ rc = -EIO;
+ goto error1;
+ }
+
+ lp->next_tx_buf_to_use = 0x0;
+ lp->next_rx_buf_to_use = 0x0;
+ lp->tx_ping_pong = get_bool(ofdev, "xlnx,tx-ping-pong");
+ lp->rx_ping_pong = get_bool(ofdev, "xlnx,rx-ping-pong");
+ mac_address = of_get_mac_address(ofdev->node);
+
+ if (mac_address)
+ /* Set the MAC address. */
+ memcpy(ndev->dev_addr, mac_address, 6);
+ else
+ dev_warn(dev, "No MAC address found\n");
+
+ /* Clear the Tx CSR's in case this is a restart */
+ out_be32(lp->base_addr + XEL_TSR_OFFSET, 0);
+ out_be32(lp->base_addr + XEL_BUFFER_OFFSET + XEL_TSR_OFFSET, 0);
+
+ /* Set the MAC address in the EmacLite device */
+ xemaclite_set_mac_address(lp, ndev->dev_addr);
+
+ dev_info(dev,
+ "MAC address is now %2x:%2x:%2x:%2x:%2x:%2x\n",
+ ndev->dev_addr[0], ndev->dev_addr[1],
+ ndev->dev_addr[2], ndev->dev_addr[3],
+ ndev->dev_addr[4], ndev->dev_addr[5]);
+
+ ndev->netdev_ops = &xemaclite_netdev_ops;
+ ndev->flags &= ~IFF_MULTICAST;
+ ndev->watchdog_timeo = TX_TIMEOUT;
+
+ /* Finally, register the device */
+ rc = register_netdev(ndev);
+ if (rc) {
+ dev_err(dev,
+ "Cannot register network device, aborting\n");
+ goto error1;
+ }
+
+ dev_info(dev,
+ "Xilinx EmacLite at 0x%08X mapped to 0x%08X, irq=%d\n",
+ (unsigned int __force)ndev->mem_start,
+ (unsigned int __force)lp->base_addr, ndev->irq);
+ return 0;
+
+error1:
+ release_mem_region(ndev->mem_start, r_mem.end - r_mem.start + 1);
+
+error2:
+ xemaclite_remove_ndev(ndev);
+ return rc;
+}
+
+/**
+ * xemaclite_of_remove - Unbind the driver from the Emaclite device.
+ * @of_dev: Pointer to OF device structure
+ *
+ * This function is called if a device is physically removed from the system or
+ * if the driver module is being unloaded. It frees any resources allocated to
+ * the device.
+ *
+ * Return: 0, always.
+ */
+static int __devexit xemaclite_of_remove(struct of_device *of_dev)
+{
+ struct device *dev = &of_dev->dev;
+ struct net_device *ndev = dev_get_drvdata(dev);
+
+ unregister_netdev(ndev);
+
+ release_mem_region(ndev->mem_start, ndev->mem_end-ndev->mem_start + 1);
+
+ xemaclite_remove_ndev(ndev);
+
+ dev_set_drvdata(dev, NULL);
+
+ return 0;
+}
+
+static struct net_device_ops xemaclite_netdev_ops = {
+ .ndo_open = xemaclite_open,
+ .ndo_stop = xemaclite_close,
+ .ndo_start_xmit = xemaclite_send,
+ .ndo_do_ioctl = xemaclite_ioctl,
+ .ndo_tx_timeout = xemaclite_tx_timeout,
+ .ndo_get_stats = xemaclite_get_stats,
+};
+
+/* Match table for OF platform binding */
+static struct of_device_id xemaclite_of_match[] __devinitdata = {
+ { .compatible = "xlnx,opb-ethernetlite-1.01.a", },
+ { .compatible = "xlnx,opb-ethernetlite-1.01.b", },
+ { .compatible = "xlnx,xps-ethernetlite-1.00.a", },
+ { .compatible = "xlnx,xps-ethernetlite-2.00.a", },
+ { .compatible = "xlnx,xps-ethernetlite-2.01.a", },
+ { /* end of list */ },
+};
+MODULE_DEVICE_TABLE(of, xemaclite_of_match);
+
+static struct of_platform_driver xemaclite_of_driver = {
+ .name = DRIVER_NAME,
+ .match_table = xemaclite_of_match,
+ .probe = xemaclite_of_probe,
+ .remove = __devexit_p(xemaclite_of_remove),
+};
+
+/**
+ * xgpiopss_init - Initial driver registration call
+ *
+ * Return: 0 upon success, or a negative error upon failure.
+ */
+static int __init xemaclite_init(void)
+{
+ /* No kernel boot options used, we just need to register the driver */
+ return of_register_platform_driver(&xemaclite_of_driver);
+}
+
+/**
+ * xemaclite_cleanup - Driver un-registration call
+ */
+static void __exit xemaclite_cleanup(void)
+{
+ of_unregister_platform_driver(&xemaclite_of_driver);
+}
+
+module_init(xemaclite_init);
+module_exit(xemaclite_cleanup);
+
+MODULE_AUTHOR("Xilinx, Inc.");
+MODULE_DESCRIPTION("Xilinx Ethernet MAC Lite driver");
+MODULE_LICENSE("GPL");
--
1.6.2.1
This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.
^ permalink raw reply related
* bnx2: module problem with 2.6.30.y or later
From: Ryota Ozaki @ 2009-08-19 12:00 UTC (permalink / raw)
To: netdev; +Cc: Ryota OZAKI
Hi,
I'm facing a problem that my kernel freezes on kexec
with bnx2 configured as a module. A workaround embedding
it and its firmware into the kernel makes out as
it was reported somewhere. However, I want to compile
it as a module due to our system limitation.
Is anyone working on the problem?
The problem appears with 2.6.30.y and 2.6.31-rc? and
not with 2.6.29.y.
Thanks,
ozaki-r
^ permalink raw reply
* Re: new ipdelay= option for faster netboot
From: Jamie Lokier @ 2009-08-19 11:57 UTC (permalink / raw)
To: Tim Bird
Cc: David Miller, r.schwebel, vda.linux, linux-kernel, linux-embedded,
arjan, kernel, netdev
In-Reply-To: <4A8A06A0.2070402@am.sony.com>
Tim Bird wrote:
> David Miller wrote:
> > From: Tim Bird <tim.bird@am.sony.com>
> > Date: Mon, 17 Aug 2009 18:24:26 -0700
> >
> >> David Miller wrote:
> >>> I have card/switch combinations that take up to 10 seconds to
> >>> negotiate a proper link.
> >> What types of delays are these timeouts supposed to
> >> cover?
> >
> > The problem is that if you don't first give at least some time for the
> > link to come up, the remaining time it takes the link to come up will
> > end up chewing into the actual bootp/dhcp protocol timeouts. And
> > that's what we're trying to avoid.
>
> What link? I'm not that familiar with networking.
>
> Assuming I'm using ethernet, what link needs to come up?
When you plug an ethernet cable in, you may have noticed it takes a
short time before the signal light comes on. That's negotiation time.
Some are slower than others, but none of them do it instantly.
> Is this something to do with power propagation to the
> physical wire?
Not really.
> Is there some MAC layer negotiation between the card and the switch?
> Is it the time for the switch to do speed detection?
Yes and yes.
> And, can any of this be more accurately determined
> or guessed-at with knowledge of the onboard hardware?
> Or is it dependent on external conditions?
It can be accurately determined with most cards (all modern ones)
because you get a notification when it's done, or you can poll the card.
That's why on the desktop it's able to detect when you plug in an
ethernet cable and start DHCP as soon as link negotiation is complete.
So the right thing to do, as David Miller suggested too, isn't a fixed
timeout. It should wait for link state UP and then start DHCP
immediately.
-- Jamie
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 11:52 UTC (permalink / raw)
To: Avi Kivity
Cc: Gregory Haskins, Anthony Liguori, Ingo Molnar, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8BE6D2.3090107@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]
Avi Kivity wrote:
> On 08/19/2009 02:40 PM, Gregory Haskins wrote:
>>
>>>>>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>>>>>> connector, you are happy?
>>>>>>
>>>>>>
>>>>>>
>>>>> This doesn't improve virtio-net in any way.
>>>>>
>>>>>
>>>> Any why not? (Did you notice I said "PCI compliant", i.e. over
>>>> virtio-pci)
>>>>
>>>>
>>> Because virtio-net will have gained nothing that it didn't have before.
>>>
>> ??
>>
>> *) ABI is virtio-pci compatible, as you like
>>
>
> That's not a gain, that's staying in the same place.
>
>> *) fast-path is in-kernel, as we all like
>>
>
> That's not a gain as we have vhost-net (sure, in development, but your
> proposed backend isn't even there yet).
>
>> *) model is in vbus so it would work in all environments that vbus
>> supports.
>>
>
> The ABI can be virtio-pci compatible or it can be vbus-comaptible. How
> can it be both? The ABIs are different.
>
> Note that if you had submitted a virtio-net backend I'd have asked you
> to strip away all the management / bus layers and we'd have ended up
> with vhost-net.
Sigh...
>
>>>>> virtio already supports this model; see lguest and s390. Transporting
>>>>> virtio over vbus and vbus over something else doesn't gain anything
>>>>> over
>>>>> directly transporting virtio over that something else.
>>>>>
>>>>>
>>>> This is not what I am advocating.
>>>>
>>>>
>>>>
>>> What are you advocating? As far as I can tell your virtio-vbus
>>> connector plus the vbus-kvm connector is just that.
>>>
>> I wouldn't classify it anything like that, no. Its just virtio over
>> vbus.
>>
>
> We're in a loop. Doesn't virtio over vbus need a virtio-vbus
> connector? and doesn't vbus need a connector to talk to the hypervisor?
>
No, it doesnt work like that. There is only one connector.
Kind Regards,
-Greg
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 267 bytes --]
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 11:49 UTC (permalink / raw)
To: Gregory Haskins
Cc: Anthony Liguori, Ingo Molnar, Gregory Haskins, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8BAC750200005A00052A83@sinclair.provo.novell.com>
On 08/19/2009 02:40 PM, Gregory Haskins wrote:
>
>>>>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>>>>> connector, you are happy?
>>>>>
>>>>>
>>>>>
>>>> This doesn't improve virtio-net in any way.
>>>>
>>>>
>>> Any why not? (Did you notice I said "PCI compliant", i.e. over virtio-pci)
>>>
>>>
>> Because virtio-net will have gained nothing that it didn't have before.
>>
> ??
>
> *) ABI is virtio-pci compatible, as you like
>
That's not a gain, that's staying in the same place.
> *) fast-path is in-kernel, as we all like
>
That's not a gain as we have vhost-net (sure, in development, but your
proposed backend isn't even there yet).
> *) model is in vbus so it would work in all environments that vbus supports.
>
The ABI can be virtio-pci compatible or it can be vbus-comaptible. How
can it be both? The ABIs are different.
Note that if you had submitted a virtio-net backend I'd have asked you
to strip away all the management / bus layers and we'd have ended up
with vhost-net.
>>>> virtio already supports this model; see lguest and s390. Transporting
>>>> virtio over vbus and vbus over something else doesn't gain anything over
>>>> directly transporting virtio over that something else.
>>>>
>>>>
>>> This is not what I am advocating.
>>>
>>>
>>>
>> What are you advocating? As far as I can tell your virtio-vbus
>> connector plus the vbus-kvm connector is just that.
>>
> I wouldn't classify it anything like that, no. Its just virtio over vbus.
>
We're in a loop. Doesn't virtio over vbus need a virtio-vbus
connector? and doesn't vbus need a connector to talk to the hypervisor?
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: mlx4 2.6.31-rc5: SW2HW_EQ failed.
From: Christoph Lameter @ 2009-08-19 11:46 UTC (permalink / raw)
To: Roland Dreier; +Cc: netdev, Yevgeny Petrilin
In-Reply-To: <adaljlg1e0x.fsf@cisco.com>
On Wed, 19 Aug 2009, Roland Dreier wrote:
> By the way, my dual-socket nehalem EP system says:
>
> SMP: Allowing 16 CPUs, 0 hotplug CPUs
>
> which I think (haven't checked but the code sure looks that way) means
> that num_possible_cpus() is 16. This is a supermicro workstation board,
> forget the exact model.
Correct. My Dell R620 may have some weird ACPI info.
SMP: Allowing 32 CPUs, 16 hotplug CPUs
although the box does not have any slots for "hotplugging".
> And the fact that your system is different is not really a bug -- it
> just points to slightly incorrect data somewhere, most likely ACPI
> tables; having 32 possible CPUs on a system that can only ever really
> have 16 CPUs just leads to some overallocations, and tickles the mlx4
> bug where it can't handle more than 32 interrupts).
The patch does not fix the issue so far. We are having various hangs and
are trying to figure out what is gong on.
We ran an old 2.6.26 debian kernel on this and it worked fine.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 11:40 UTC (permalink / raw)
To: Avi Kivity
Cc: Anthony Liguori, Ingo Molnar, Gregory Haskins, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8BA635.9010902@redhat.com>
>>> On 8/19/2009 at 3:13 AM, in message <4A8BA635.9010902@redhat.com>, Avi Kivity
<avi@redhat.com> wrote:
> On 08/19/2009 09:40 AM, Gregory Haskins wrote:
>>
>>
>>>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>>>> connector, you are happy?
>>>>
>>>>
>>> This doesn't improve virtio-net in any way.
>>>
>> Any why not? (Did you notice I said "PCI compliant", i.e. over virtio-pci)
>>
>
> Because virtio-net will have gained nothing that it didn't have before.
??
*) ABI is virtio-pci compatible, as you like
*) fast-path is in-kernel, as we all like
*) model is in vbus so it would work in all environments that vbus supports.
>
>
>
>
>>> virtio already supports this model; see lguest and s390. Transporting
>>> virtio over vbus and vbus over something else doesn't gain anything over
>>> directly transporting virtio over that something else.
>>>
>> This is not what I am advocating.
>>
>>
>
> What are you advocating? As far as I can tell your virtio-vbus
> connector plus the vbus-kvm connector is just that.
I wouldn't classify it anything like that, no. Its just virtio over vbus.
-Greg
^ permalink raw reply
* [PATCH] TI DaVinci EMAC: delay DaVinci EMAC initialization
From: Sudhakar Rajashekhara @ 2009-08-19 20:39 UTC (permalink / raw)
To: netdev
Cc: davem, chaithrika, anantgole, davinci-linux-open-source,
Sudhakar Rajashekhara
On TI's DA850/OMAP-L138 EVM, MAC address is stored in SPI
flash which is accessed using MTD interface.
This patch delays the initialization of DaVinci EMAC driver
by changing module_init to late_initcall. This helps SPI and
MTD drivers to get initialized before EMAC thereby enabling
EMAC driver to read the MAC address while booting and use it.
Tested with NFS on DM644x, DM6467, DA830/OMAP-L137 and
DA850/OMAP-L138 EVMs.
Signed-off-by: Sudhakar Rajashekhara <sudhakar.raj@ti.com>
Reviewed-by: Chaithrika U S <chaithrika@ti.com>
---
drivers/net/davinci_emac.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
index 12fd446..5e6652b 100644
--- a/drivers/net/davinci_emac.c
+++ b/drivers/net/davinci_emac.c
@@ -2817,7 +2817,7 @@ static int __init davinci_emac_init(void)
{
return platform_driver_register(&davinci_emac_driver);
}
-module_init(davinci_emac_init);
+late_initcall(davinci_emac_init);
/**
* davinci_emac_exit: EMAC driver module exit
--
1.5.6
^ permalink raw reply related
* Re: [PATCH] revert TCP retransmission backoff on ICMP destination unreachable
From: Ilpo Järvinen @ 2009-08-19 10:55 UTC (permalink / raw)
To: Damian Lukowski; +Cc: Netdev, David Miller
In-Reply-To: <4A8B3FC8.9080707@tvk.rwth-aachen.de>
On Wed, 19 Aug 2009, Damian Lukowski wrote:
> Ok, I will fully quote previous messages now. Just wanted to avoid
> redundancy.
Full quoting is not what I meant, Dave explained what I meant in his mail.
> Anyway, as long as the draft is not accepted as RFC, the
> patch will violate current standards, so I doubt, Dave would like to
> apply it soon. :)
...Don't be too sure on that ;-).
> >>>> + icsk->icsk_backoff--;
> >>>> + icsk->icsk_rto >>= 1;
> >>> Are you absolute sure that we don't go to zero here when phase of the
> >>> moon is just right? I'd put a max(..., 1) guard there.
> >> Well, the retransmission timer doubles icsk_rto whenever it increases
> >> icsk_backoff, so we should reach the original icsk_rto, when
> >> icsk_backoff becomes zero.
> >
> > This is a false assumption. Take a look into the formula, there's a
> > cap how far icsk_rto goes but icsk_backoff keeps increasing. In fact,
> > there would probably be a need to set some other lower bound than 1,
> > TCP_RTO_MIN would probably be more appropriate for this.
>
> *sigh* I should have seen that. I would like to preserve the initial rto
> value and not work with RTO_MIN. First, I have thought about storing the
> initial rto in another state variable. But what, if we simply
> recalculate the value?:
> ###
> icsk->icsk_backoff--;
> - icsk->icsk_rto >>= 1;
> + tcp_set_rto(sk);
> + icsk->icsk_rto <<= icsk->icsk_backoff;
> + tcp_bound_rto(sk);
> ###
Yes, this is sort of a problem. I'd recommend that you read through all
icsk_rto readers and writers and see if recalculation can be done without
getting into some problems.
> >>>> + BUG_ON(!skb_r);
> >>>> +
> >>>> + if (sock_owned_by_user(sk)) {
> >>>> + /* Deferring retransmission clocked by ICMP
> >>>> + * due to locked socket. */
> >>>> + inet_csk_reset_xmit_timer(sk, ICSK_TIME_RETRANS,
> >>>> + min(icsk->icsk_rto, TCP_RESOURCE_PROBE_INTERVAL),
> >>> ??? (besides having indent wrong). What makes you think HZ/2 is right
> >>> value if icsk_rto is large?
> >> To be honest, I have taken over the code from my predecessor and didn't
> >> question this part.
> >
> > But you did Signoff that regardless, so you should know ;-).
>
> Only because checkpatch.pl didn't like it otherwise. :)
...And you dare to admit :-) ...It certainly has some other meaning beyond
checkpatch complain, for future, please check and understand what it
means that you add it there.
> >>>> + inet_csk_reset_xmit_timer(sk, ICSK_TIME_RETRANS,
> >>>> + icsk->icsk_rto, TCP_RTO_MAX);
> >>> RFC 2988, step 5.5 missing? Have you verified this patch for real?
> >> Thats the whole point of the draft. If consecutive retransmissions
> >> trigger fitting ICMPs, the RTO value is not doubled, but remains
> >> constant.
> >
> > No, either the draft is wrong or the implementation. 5.5 step should be
> > applied. Now you end up decreasing RTO per icmp which causes imminent
> > retransmission instead of keeping it "constant". Effectively, step 4, 5, 6
> > (halves like you do), 7 -> R, R is 5.4-5.6 and 5.5 increases RTO again
> > because the timer _expired_ here even though we did short-circuit it
> > instead of setting a zero timer and using the normal procedure (which in
> > itself is ok approach, no need to force timer to expire).
> >
> >> So you can have a retransmission schedule of
> >> t, t+r, t+2r, t+3r, t+4r, t+5r ...
> >> instead of
> >> t, t+r, t+3r, t+7r, t+15r, t+31r ...
> >
> > Sure, but that's not what happens, instead you will get this series:
> >
> > t, t+r, (t+r)/2, (t+r)/4, ...
> >
> > ...disclaimer: it might be off-by-something somewhere but I hope you get
> > my point anyway.
>
> It seems to me, that we focus on different things. From the TCP code's
> point of view, RFC2988 step 5.5 is of course executed, because timeouts
> still trigger tcp_retransmit_timer(), which doubles the RTO at the end.
No and yes. For no: you artificially made tcp_retransmit_timer to not be
called whenever the ICMP arrives at a point where RTO/2 step causes the
timer to expire right away. But yes: if you change it to call
tcp_retransmit_timer as we have discussed, 5.5 should be there (but that's
not in original patch which I was commenting).
> But from the network's point of view, there is no doubling, but constant
> intervals between retransmissions.
Plain wrong, you would only end up halving as per icmp when the timing is
right to produce the scenario I'm talking. Please show from the original
patch
> The following happens with patched TCP (not considering RTO_MAX):
> t: Connectivity breaks, rto is r
> t+r: First RTO fires, SND.UNA is retransmitted by
> tcp_retransmit_timer() and rto := 2rto
> Now it is rto == 2r
> t+r+epsilon: ICMP unreach arrives from a router.
> The patch sets rto := rto/2
> Now it is rto == r
> t+2r: Second RTO fires, SND.UNA is retransmitted by
> tcp_retransmit_timer() and rto := 2rto
> Now it is rto == 2r
> Assume, that this retransmission did not trigger
> any ICMP message.
> t+4r: Third RTO fires, SND.UNA is retransmitted by
> tcp_retransmit_timer() and rto := 2rto
> Now it is rto == 4r
> Assume, that this retransmission did not trigger
> any ICMP message.
> t+8r: Fourth RTO fires, SND.UNA is retransmitted by
> tcp_retransmit_timer() and rto := 2rto
> Now it is rto == 8r
> t+8r+epsilon: ICMP unreach arrives from a router.
> The patch sets rto := rto/2
> Now it is rto == 4r
> t+12r: Fifth RTO fires ...
No, that was not the scenario I'm talking about. Let me try to explain:
> t: Connectivity breaks, rto is r
> t+r: First RTO fires, SND.UNA is retransmitted by
> tcp_retransmit_timer() and rto := 2rto
> Now it is rto == 2r
t+2r+epsilon: ICMP unreach arrives from a router.
> The patch sets rto := rto/2
t+2r+epsilon > t+r+rto => call for retransmit in the icmp code
set rto = r
t+5/2r+epsilon ICMP unreach arrives from a router.
The patch sets rto := rto/2
t+5/2r+epsilon > t+something => call for retransmit
set rto = r/2
...
See, it's a downward spiral because nothing in the original patch takes
care of the rto*2 (ie., step 5.5) when RTO is artificially _caused by_ the
ICMP (ie., the halving does make expiry to happen right away, see draft
step 7).
> >> I can show you trace plots of patched an unpatched TCP, if you like.
> >> Should I attach files in the mails, or better post hyperlinks to them?
> >
> > Sure, but they must not be from a trivial case but the rto must
> > "flunctuate"
>
> How should they fluctuate? RTO is doubled by tcp_retransmit_timer() no
> matter what happens, and it is halved by the the patch, if an
> appropriate ICMP packet has been received.
The original patch doesn't agree. If you'd have used tcp_retransmit_timer
in the original I'd agree but you called for tcp_retransmit_skb(skb_r)
and set the next rto yourself which wouldn't let tcp_retransmit_timer to
run at all in the scenario I was talking about. But lets see the next
version of the patch first and only then continue the discussion on this
particular point about 5.5 (probably not then necessary anymore because
of a change we've discussed).
> > and the expirys must be using the "imminent" branch for it
> > to count for me as a proof that you did it right.
>
> I'm not sure what you mean.
The case I'm talking about is the branch where we notice that after doing
rto/2 the RTO is to be scheduled right away (see step 7 in the draft).
> Still, hyperlinks or attachments?
Either is fine, but large stuff as links. However, honestly I'm not that
interest in them as I know that if you have the traces from the particular
case I'm talking about, they would show the misbehavior (other cases
work but they do not interest me at all). It would have been mostly just
to convince you that this misbehavior is there, since my words didn't seem
to make it :-). I propose we put this "proving" aside too for now since
the changes we've already discussed should make this a non-issue anyway,
and therefore no need to waste time on that.
--
i.
^ permalink raw reply
* Dear Account User
From: Webmail Technical Support Team. @ 2009-08-19 9:35 UTC (permalink / raw)
Dear Account User,
We are currently upgrading our data base and e-mail account center i.e
homepage view. We shall be deleting old email accounts which are no longer
active to create more space for new accounts users.we have also
investigated a system wide security audit to improve and enhance our
current security.
In order to continue using our services you are require to update and
re-comfirmed your email account details as requested below.
To complete your account re-comfirmation,you must reply to this email
immediately and enter your account details as requested below.
Username : ..................
E-mail Login ID..............
Password : ..................
confirm password:............
Date of Birth :..............
Future Password :............
Failure to do this will immediately render your account deactivated from
our database and service will not be interrupted as important messages may
as well be lost due to your declining to re-comfirmed your account details
to us.
You are to send your information above via our webmaster email:
accountupdate001@mail2world.com
We apologise for the inconvenience that this will cause you during this
period,but trusting that we are here to serve you better and providing
more technology which revolves around email and internet.
It is also pertinent,you understand that our primary concern is for our
customers, and for the security of their files and data.
COMFIRMATION CODE: -/93-1A388-480 Webmail Technical Support Team.
^ permalink raw reply
* Re: [PATCHv3 2/2] vhost_net: a kernel-level virtio server
From: Arnd Bergmann @ 2009-08-19 9:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtualization, netdev, kvm, linux-kernel, mingo, linux-mm, akpm,
hpa, gregory.haskins, Or Gerlitz
In-Reply-To: <20090816065110.GA3008@redhat.com>
On Sunday 16 August 2009, Michael S. Tsirkin wrote:
> On Fri, Aug 14, 2009 at 01:40:36PM +0200, Arnd Bergmann wrote:
> >
> > * most of the transports are sockets, tap uses a character device.
> > This could be dealt with by having both a struct socket * in
> > struct vhost_net *and* a struct file *, or by always keeping the
> > struct file and calling vfs_readv/vfs_writev for the data transport
> > in both cases.
>
> I am concerned that character devices might have weird side effects with
> read/write operations and that calling them from kernel thread the way I
> do might have security implications. Can't point at anything specific
> though at the moment.
I understand your feelings about passing a chardev fd into your driver
and I agree that we need to be very careful if we want to allow it.
Maybe we could instead extend the 'splice' system call to work on a
vhost_net file descriptor. If we do that, we can put the access back
into a user thread (or two) that stays in splice indefinetely to
avoid some of the implications of kernel threads like the missing
ability to handle transfer errors in user space.
> I wonder - can we expose the underlying socket used by tap, or will that
> create complex lifetime issues?
I think this could get more messy in the long run than calling vfs_readv
on a random fd. It would mean deep internal knowledge of the tap driver
in vhost_net, which I really would prefer to avoid.
> > * Each transport has a slightly different header, we have
> > - raw ethernet frames (raw, udp multicast, tap)
> > - 32-bit length + raw frames, possibly fragmented (tcp)
> > - 80-bit header + raw frames, possibly fragmented (tap with vnet_hdr)
> > To handle these three cases, we need either different ioctl numbers
> > so that vhost_net can choose the right one, or a flags field in
> > VHOST_NET_SET_SOCKET, like
> >
> > #define VHOST_NET_RAW 1
> > #define VHOST_NET_LEN_HDR 2
> > #define VHOST_NET_VNET_HDR 4
> >
> > struct vhost_net_socket {
> > unsigned int flags;
> > int fd;
> > };
> > #define VHOST_NET_SET_SOCKET _IOW(VHOST_VIRTIO, 0x30, struct vhost_net_socket)
>
> It seems we can query the socket to find out the type,
yes, I understand that you can do that, but I still think that decision
should be left to user space. Adding a length header for TCP streams but
not for UDP is something that we would normally want to do, but IMHO
vhost_net should not need to know about this.
> or use the features ioctl.
Right, I had forgotten about that one. It's probably equivalent
to the flags I suggested, except that one allows you to set features
after starting the communication, while the other one prevents
you from doing that.
> > Qemu could then automatically try to use vhost_net, if it's available
> > in the kernel, or just fall back on software vlan otherwise.
> > Does that make sense?
>
> I agree, long term it should be enabled automatically when possible.
So how about making the qemu command line interface an extension to
what Or Gerlitz has done for the raw packet sockets?
Arnd <><
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: bad nat connection tracking performance with ip_gre
From: Timo Teräs @ 2009-08-19 8:40 UTC (permalink / raw)
To: Patrick McHardy; +Cc: netfilter-devel, netdev
In-Reply-To: <4A8B02D2.2090400@iki.fi>
Timo Teräs wrote:
> Ok, finally figured out the difference. Looks like depending
> on the sendto() / local route / forward route / my patched mrt
> the skb that gets passed to ipgre_tunnel_xmit() seems to have
> nfctinfo either 0 or 2. This value is not modified; nf_reset()
> is called just before ip_local_out(). Looks like nf_reset()
> clears nfct to NULL, but does not touch nfctinfo.
>
> So when LOCAL_OUT hook for the GRE packet is hit, depending
> where the packet came: it has nfct=NULL and nfctinfo=ESTABLISHED
> or NEW. This also seems to affect if that specific skb gets
> the nat/OUTPUT hook called.
>
> Is this behaviour for nf_reset() intentional?
Apparently it does not matter.
The real problem seems to be bug that did not account for
irq times properly. It was fixed by "sched: account system time
properly" f5f293a4e3d0a0c52cec31de6762c95050156516 and that
caused biased CPU usage measurements.
Testing with kernel having that, I'm getting the same CPU
readings. And oprofile in timer mode didn't help noticing this.
Sorry for the noise.
- Timo
^ permalink raw reply
* [PATCH v2] Speed-up pfifo_fast lookup using a private bitmap
From: Krishna Kumar @ 2009-08-19 7:55 UTC (permalink / raw)
To: davem; +Cc: Jarek Poplawski, netdev, herbert, Krishna Kumar, kaber
From: Krishna Kumar <krkumar2@in.ibm.com>
Maintain a per-qdisc bitmap for pfifo_fast giving availability
of skbs for each band. This allows faster lookup for a skb when
there are no high priority skbs. Also, it helps in (rare) cases
when there are no skbs on the list, where an immediate lookup is
faster than iterating through the three bands.
Changelog v1 -> v2
1. Moved the bitmap from Qdisc to the private area of pfifo_fast
2. pfifo_fast_reset sets bitmap to zero
3. Undo a line-wrap in initializing a variable
4. Fixed some comments
5. Readability and checkpatch fixes
Thanks,
- KK
Signed-off-by: Krishna Kumar <krkumar2@in.ibm.com>
---
net/sched/sch_generic.c | 70 ++++++++++++++++++++++++++------------
1 file changed, 48 insertions(+), 22 deletions(-)
diff -ruNp org/net/sched/sch_generic.c new/net/sched/sch_generic.c
--- org/net/sched/sch_generic.c 2009-08-07 12:05:43.000000000 +0530
+++ new/net/sched/sch_generic.c 2009-08-14 18:23:16.000000000 +0530
@@ -406,18 +406,38 @@ static const u8 prio2band[TC_PRIO_MAX+1]
#define PFIFO_FAST_BANDS 3
-static inline struct sk_buff_head *prio2list(struct sk_buff *skb,
- struct Qdisc *qdisc)
+/*
+ * Private data for a pfifo_fast scheduler containing:
+ * - queues for the three band
+ * - bitmap indicating which of the bands contain skbs
+ */
+struct pfifo_fast_priv {
+ u32 bitmap;
+ struct sk_buff_head q[PFIFO_FAST_BANDS];
+};
+
+/*
+ * Convert a bitmap to the first band number where an skb is queued, where:
+ * bitmap=0 means there are no skbs on any band.
+ * bitmap=1 means there is an skb on band 0.
+ * bitmap=7 means there are skbs on all 3 bands, etc.
+ */
+static const int bitmap2band[] = {-1, 0, 1, 0, 2, 0, 1, 0};
+
+static inline struct sk_buff_head *band2list(struct pfifo_fast_priv *priv,
+ int band)
{
- struct sk_buff_head *list = qdisc_priv(qdisc);
- return list + prio2band[skb->priority & TC_PRIO_MAX];
+ return priv->q + band;
}
static int pfifo_fast_enqueue(struct sk_buff *skb, struct Qdisc* qdisc)
{
- struct sk_buff_head *list = prio2list(skb, qdisc);
+ int band = prio2band[skb->priority & TC_PRIO_MAX];
+ struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
+ struct sk_buff_head *list = band2list(priv, band);
if (skb_queue_len(list) < qdisc_dev(qdisc)->tx_queue_len) {
+ priv->bitmap |= (1 << band);
qdisc->q.qlen++;
return __qdisc_enqueue_tail(skb, qdisc, list);
}
@@ -427,14 +447,18 @@ static int pfifo_fast_enqueue(struct sk_
static struct sk_buff *pfifo_fast_dequeue(struct Qdisc* qdisc)
{
- int prio;
- struct sk_buff_head *list = qdisc_priv(qdisc);
+ struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
+ int band = bitmap2band[priv->bitmap];
- for (prio = 0; prio < PFIFO_FAST_BANDS; prio++) {
- if (!skb_queue_empty(list + prio)) {
- qdisc->q.qlen--;
- return __qdisc_dequeue_head(qdisc, list + prio);
- }
+ if (likely(band >= 0)) {
+ struct sk_buff_head *list = band2list(priv, band);
+ struct sk_buff *skb = __qdisc_dequeue_head(qdisc, list);
+
+ qdisc->q.qlen--;
+ if (skb_queue_empty(list))
+ priv->bitmap &= ~(1 << band);
+
+ return skb;
}
return NULL;
@@ -442,12 +466,13 @@ static struct sk_buff *pfifo_fast_dequeu
static struct sk_buff *pfifo_fast_peek(struct Qdisc* qdisc)
{
- int prio;
- struct sk_buff_head *list = qdisc_priv(qdisc);
+ struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
+ int band = bitmap2band[priv->bitmap];
+
+ if (band >= 0) {
+ struct sk_buff_head *list = band2list(priv, band);
- for (prio = 0; prio < PFIFO_FAST_BANDS; prio++) {
- if (!skb_queue_empty(list + prio))
- return skb_peek(list + prio);
+ return skb_peek(list);
}
return NULL;
@@ -456,11 +481,12 @@ static struct sk_buff *pfifo_fast_peek(s
static void pfifo_fast_reset(struct Qdisc* qdisc)
{
int prio;
- struct sk_buff_head *list = qdisc_priv(qdisc);
+ struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
for (prio = 0; prio < PFIFO_FAST_BANDS; prio++)
- __qdisc_reset_queue(qdisc, list + prio);
+ __qdisc_reset_queue(qdisc, band2list(priv, prio));
+ priv->bitmap = 0;
qdisc->qstats.backlog = 0;
qdisc->q.qlen = 0;
}
@@ -480,17 +506,17 @@ nla_put_failure:
static int pfifo_fast_init(struct Qdisc *qdisc, struct nlattr *opt)
{
int prio;
- struct sk_buff_head *list = qdisc_priv(qdisc);
+ struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
for (prio = 0; prio < PFIFO_FAST_BANDS; prio++)
- skb_queue_head_init(list + prio);
+ skb_queue_head_init(band2list(priv, prio));
return 0;
}
static struct Qdisc_ops pfifo_fast_ops __read_mostly = {
.id = "pfifo_fast",
- .priv_size = PFIFO_FAST_BANDS * sizeof(struct sk_buff_head),
+ .priv_size = sizeof(struct pfifo_fast_priv),
.enqueue = pfifo_fast_enqueue,
.dequeue = pfifo_fast_dequeue,
.peek = pfifo_fast_peek,
^ permalink raw reply
* RE: [PATCH] net: add Xilinx emac lite device driver
From: John Linn @ 2009-08-19 7:20 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Michal Simek, netdev, Sadanand Mutyala, linuxppc-dev, jgarzik,
davem, John Williams
In-Reply-To: <20090818140418.347cb8ac@nehalam>
> -----Original Message-----
> From: Stephen Hemminger [mailto:shemminger@vyatta.com]
> Sent: Tuesday, August 18, 2009 10:04 PM
> To: John Linn
> Cc: netdev@vger.kernel.org; linuxppc-dev@ozlabs.org;
jgarzik@pobox.com; davem@davemloft.net; John
> Linn; grant.likely@secretlab.ca; Josh Boyer; John Williams; Michal
Simek; Sadanand Mutyala
> Subject: Re: [PATCH] net: add Xilinx emac lite device driver
>
> On Tue, 18 Aug 2009 09:30:41 -0600
> John Linn <john.linn@xilinx.com> wrote:
>
> > +/**
> > + * xemaclite_enable_interrupts - Enable the interrupts for the
EmacLite device
> > + * @drvdata: Pointer to the Emaclite device private data
> > + *
> > + * This function enables the Tx and Rx interrupts for the Emaclite
device along
> > + * with the Global Interrupt Enable.
> > + */
> > +static void xemaclite_enable_interrupts(struct net_local *drvdata)
>
> Docbook format is really a not necessary on local functions that
> are only used in the driver. It is fine if you want to use it, as
> long as the file isn't processed by kernel make docs but
> the docbook is intended for automatic generation of kernel API
manuals.
Thanks, we realize that and have been consistent with our drivers in
that area. We aren't adding them to the make for the docs so it doesn't
seem like it should be a problem.
>
> --
This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.
^ permalink raw reply
* RE: [PATCH] net: add Xilinx emac lite device driver
From: John Linn @ 2009-08-19 7:13 UTC (permalink / raw)
To: David Miller
Cc: netdev, linuxppc-dev, jgarzik, grant.likely, jwboyer,
john.williams, michal.simek, Sadanand Mutyala
In-Reply-To: <20090818.232815.73515404.davem@davemloft.net>
Thanks David, I may have got over zealous in my clean up to get it ready
for mainline.
I'll take care of that.
-- John
> -----Original Message-----
> From: David Miller [mailto:davem@davemloft.net]
> Sent: Wednesday, August 19, 2009 7:28 AM
> To: John Linn
> Cc: netdev@vger.kernel.org; linuxppc-dev@ozlabs.org;
jgarzik@pobox.com; grant.likely@secretlab.ca;
> jwboyer@linux.vnet.ibm.com; john.williams@petalogix.com;
michal.simek@petalogix.com; Sadanand Mutyala
> Subject: Re: [PATCH] net: add Xilinx emac lite device driver
>
> From: John Linn <john.linn@xilinx.com>
> Date: Tue, 18 Aug 2009 09:30:41 -0600
>
> > @@ -1926,6 +1926,11 @@ config ATL2
> > To compile this driver as a module, choose M here. The module
> > will be called atl2.
> >
> > +config XILINX_EMACLITE
> > + tristate "Xilinx 10/100 Ethernet Lite support"
> > + help
> > + This driver supports the 10/100 Ethernet Lite from Xilinx.
> > +
> > source "drivers/net/fs_enet/Kconfig"
> >
> > endif # NET_ETHERNET
>
> You can't unconditionally enable this driver everywhere.
> It depends upon things like openfirmware support, etc.
> so the driver will fail to link if the architecture doesn't
> have those support routines.
>
> Please fix this up, thanks.
This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 7:13 UTC (permalink / raw)
To: Gregory Haskins
Cc: Gregory Haskins, Anthony Liguori, Ingo Molnar, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8B66280200005A00052A54@sinclair.provo.novell.com>
On 08/19/2009 09:40 AM, Gregory Haskins wrote:
>
>
>>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>>> connector, you are happy?
>>>
>>>
>> This doesn't improve virtio-net in any way.
>>
> Any why not? (Did you notice I said "PCI compliant", i.e. over virtio-pci)
>
Because virtio-net will have gained nothing that it didn't have before.
>> virtio already supports this model; see lguest and s390. Transporting
>> virtio over vbus and vbus over something else doesn't gain anything over
>> directly transporting virtio over that something else.
>>
> This is not what I am advocating.
>
>
What are you advocating? As far as I can tell your virtio-vbus
connector plus the vbus-kvm connector is just that.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Avi Kivity @ 2009-08-19 7:11 UTC (permalink / raw)
To: Gregory Haskins
Cc: Ingo Molnar, kvm, alacrityvm-devel, linux-kernel, netdev,
Michael S. Tsirkin
In-Reply-To: <4A8B9B79.9050004@gmail.com>
On 08/19/2009 09:28 AM, Gregory Haskins wrote:
> Avi Kivity wrote:
>
>> On 08/18/2009 05:46 PM, Gregory Haskins wrote:
>>
>>>
>>>> Can you explain how vbus achieves RDMA?
>>>>
>>>> I also don't see the connection to real time guests.
>>>>
>>>>
>>> Both of these are still in development. Trying to stay true to the
>>> "release early and often" mantra, the core vbus technology is being
>>> pushed now so it can be reviewed. Stay tuned for these other
>>> developments.
>>>
>>>
>> Hopefully you can outline how it works. AFAICT, RDMA and kernel bypass
>> will need device assignment. If you're bypassing the call into the host
>> kernel, it doesn't really matter how that call is made, does it?
>>
> This is for things like the setup of queue-pairs, and the transport of
> door-bells, and ib-verbs. I am not on the team doing that work, so I am
> not an expert in this area. What I do know is having a flexible and
> low-latency signal-path was deemed a key requirement.
>
That's not a full bypass, then. AFAIK kernel bypass has userspace
talking directly to the device.
Given that both virtio and vbus can use ioeventfds, I don't see how one
can perform better than the other.
> For real-time, a big part of it is relaying the guest scheduler state to
> the host, but in a smart way. For instance, the cpu priority for each
> vcpu is in a shared-table. When the priority is raised, we can simply
> update the table without taking a VMEXIT. When it is lowered, we need
> to inform the host of the change in case the underlying task needs to
> reschedule.
>
This is best done using cr8/tpr so you don't have to exit at all. See
also my vtpr support for Windows which does this in software, generally
avoiding the exit even when lowering priority.
> This is where the really fast call() type mechanism is important.
>
> Its also about having the priority flow-end to end, and having the vcpu
> interrupt state affect the task-priority, etc (e.g. pending interrupts
> affect the vcpu task prio).
>
> etc, etc.
>
> I can go on and on (as you know ;), but will wait till this work is more
> concrete and proven.
>
Generally cpu state shouldn't flow through a device but rather through
MSRs, hypercalls, and cpu registers.
> Basically, what it comes down to is both vbus and vhost need
> configuration/management. Vbus does it with sysfs/configfs, and vhost
> does it with ioctls. I ultimately decided to go with sysfs/configfs
> because, at least that the time I looked, it seemed like the "blessed"
> way to do user->kernel interfaces.
>
I really dislike that trend but that's an unrelated discussion.
>> They need to be connected to the real world somehow. What about
>> security? can any user create a container and devices and link them to
>> real interfaces? If not, do you need to run the VM as root?
>>
> Today it has to be root as a result of weak mode support in configfs, so
> you have me there. I am looking for help patching this limitation, though.
>
>
Well, do you plan to address this before submission for inclusion?
>> I hope everyone agrees that it's an important issue for me and that I
>> have to consider non-Linux guests. I also hope that you're considering
>> non-Linux guests since they have considerable market share.
>>
> I didn't mean non-Linux guests are not important. I was disagreeing
> with your assertion that it only works if its PCI. There are numerous
> examples of IHV/ISV "bridge" implementations deployed in Windows, no?
>
I don't know.
> If vbus is exposed as a PCI-BRIDGE, how is this different?
>
Technically it would work, but given you're not interested in Windows,
who would write a driver?
>> Given I'm not the gateway to inclusion of vbus/venet, you don't need to
>> ask me anything. I'm still free to give my opinion.
>>
> Agreed, and I didn't mean to suggest otherwise. It not clear if you are
> wearing the "kvm maintainer" hat, or the "lkml community member" hat at
> times, so its important to make that distinction. Otherwise, its not
> clear if this is edict as my superior, or input as my peer. ;)
>
When I wear a hat, it is a Red Hat. However I am bareheaded most often.
(that is, look at the contents of my message, not who wrote it or his role).
>> With virtio, the number is 1 (or less if you amortize). Set up the ring
>> entries and kick.
>>
> Again, I am just talking about basic PCI here, not the things we build
> on top.
>
Whatever that means, it isn't interesting. Performance is measure for
the whole stack.
> The point is: the things we build on top have costs associated with
> them, and I aim to minimize it. For instance, to do a "call()" kind of
> interface, you generally need to pre-setup some per-cpu mappings so that
> you can just do a single iowrite32() to kick the call off. Those
> per-cpu mappings have a cost if you want them to be high-performance, so
> my argument is that you ideally want to limit the number of times you
> have to do this. My current design reduces this to "once".
>
Do you mean minimizing the setup cost? Seriously?
>> There's no such thing as raw PCI. Every PCI device has a protocol. The
>> protocol virtio chose is optimized for virtualization.
>>
> And its a question of how that protocol scales, more than how the
> protocol works.
>
> Obviously the general idea of the protocol works, as vbus itself is
> implemented as a PCI-BRIDGE and is therefore limited to the underlying
> characteristics that I can get out of PCI (like PIO latency).
>
I thought we agreed that was insignificant?
>> As I've mentioned before, prioritization is available on x86
>>
> But as Ive mentioned, it doesn't work very well.
>
I guess it isn't that important then. I note that clever prioritization
in a guest is pointless if you can't do the same prioritization in the host.
>> , and coalescing scales badly.
>>
> Depends on what is scaling. Scaling vcpus? Yes, you are right.
> Scaling the number of devices? No, this is where it improves.
>
If you queue pending messages instead of walking the device list, you
may be right. Still, if hard interrupt processing takes 10% of your
time you'll only have coalesced 10% of interrupts on average.
>> irq window exits ought to be pretty rare, so we're only left with
>> injection vmexits. At around 1us/vmexit, even 100,000 interrupts/vcpu
>> (which is excessive) will only cost you 10% cpu time.
>>
> 1us is too much for what I am building, IMHO.
You can't use current hardware then.
>> You're free to demultiplex an MSI to however many consumers you want,
>> there's no need for a new bus for that.
>>
> Hmmm...can you elaborate?
>
Point all those MSIs at one vector. Its handler will have to poll all
the attached devices though.
>> Do you use DNS. We use PCI-SIG. If Novell is a PCI-SIG member you can
>> get a vendor ID and control your own virtio space.
>>
> Yeah, we have our own id. I am more concerned about making this design
> make sense outside of PCI oriented environments.
>
IIRC we reuse the PCI IDs for non-PCI.
>>>> That's a bug, not a feature. It means poor scaling as the number of
>>>> vcpus increases and as the number of devices increases.
>>>>
> vcpu increases, I agree (and am ok with, as I expect low vcpu count
> machines to be typical).
I'm not okay with it. If you wish people to adopt vbus over virtio
you'll have to address all concerns, not just yours.
> nr of devices, I disagree. can you elaborate?
>
With message queueing, I retract my remark.
>> Windows,
>>
> Work in progress.
>
Interesting. Do you plan to open source the code? If not, will the
binaries be freely available?
>
>> large guests
>>
> Can you elaborate? I am not familiar with the term.
>
Many vcpus.
>
>> and multiqueue out of your design.
>>
> AFAICT, multiqueue should work quite nicely with vbus. Can you
> elaborate on where you see the problem?
>
You said you aren't interested in it previously IIRC.
>>>> x86 APIC is priority aware.
>>>>
>>>>
>>> Have you ever tried to use it?
>>>
>>>
>> I haven't, but Windows does.
>>
> Yeah, it doesn't really work well. Its an extremely rigid model that
> (IIRC) only lets you prioritize in 16 groups spaced by IDT (0-15 are one
> level, 16-31 are another, etc). Most of the embedded PICs I have worked
> with supported direct remapping, etc. But in any case, Linux doesn't
> support it so we are hosed no matter how good it is.
>
I agree that it isn't very clever (not that I am a real time expert) but
I disagree about dismissing Linux support so easily. If prioritization
is such a win it should be a win on the host as well and we should make
it work on the host as well. Further I don't see how priorities on the
guest can work if they don't on the host.
>>>
>>>
>> They had to build connectors just like you propose to do.
>>
> More importantly, they had to build back-end busses too, no?
>
They had to write 414 lines in drivers/s390/kvm/kvm_virtio.c and
something similar for lguest.
>> But you still need vbus-connector-lguest and vbus-connector-s390 because
>> they all talk to the host differently. So what's changed? the names?
>>
> The fact that they don't need to redo most of the in-kernel backend
> stuff. Just the connector.
>
So they save 414 lines but have to write a connector which is... how large?
>> Well, venet doesn't complement virtio-net, and virtio-pci doesn't
>> complement vbus-connector.
>>
> Agreed, but virtio complements vbus by virtue of virtio-vbus.
>
I don't see what vbus adds to virtio-net.
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
^ permalink raw reply
* Re: mlx4 2.6.31-rc5: SW2HW_EQ failed.
From: Roland Dreier @ 2009-08-19 7:03 UTC (permalink / raw)
To: Christoph Lameter; +Cc: netdev, Yevgeny Petrilin
In-Reply-To: <ada3a7p3o6f.fsf@cisco.com>
By the way, my dual-socket nehalem EP system says:
SMP: Allowing 16 CPUs, 0 hotplug CPUs
which I think (haven't checked but the code sure looks that way) means
that num_possible_cpus() is 16. This is a supermicro workstation board,
forget the exact model.
And the fact that your system is different is not really a bug -- it
just points to slightly incorrect data somewhere, most likely ACPI
tables; having 32 possible CPUs on a system that can only ever really
have 16 CPUs just leads to some overallocations, and tickles the mlx4
bug where it can't handle more than 32 interrupts).
Just FWIW.
- R.
^ permalink raw reply
* Re: Kernel oops on setting sky2 interfaces down
From: Rene Mayrhofer @ 2009-08-19 7:01 UTC (permalink / raw)
To: Mike McCormack; +Cc: netdev, Richard Leitner, Stephen Hemminger
In-Reply-To: <4A8131DD.7010700@mayrhofer.eu.org>
[-- Attachment #1: Type: Text/Plain, Size: 501 bytes --]
Hi everybody,
On Tuesday 11 August 2009 10:54:53 am Rene Mayrhofer wrote:
> Thus, there really seems to be an uncaught case in sky2.c. When
> sky2_phy_power_down is not called, chip should not go down, right? But
> still sky2_poll seems to be called (maybe by an interrupt belonging to
> another network interface but the same chip)?
Is there anything else I could try? We still have this issue, making one range
of hardware appliances unusable with 2.6 kernels...
best regards,
Rene
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* [PATCH 2/2] sctp: fix heartbeat count of path failure
From: Chunbo Luo @ 2009-08-19 7:01 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-sctp, chunbo.luo
In-Reply-To: <1250665268-29770-1-git-send-email-chunbo.luo@windriver.com>
RFC4960 Section 8.2 defined that the transport should enter INACTIVE
state only when the value in the error counter exceeds the protocol
parameter 'Path.Max.Retrans' of that destination address. This means
that the transport should enter INACTIVE state after pathmaxrxt+1
heartbeats are not acknowledged.
Signed-off-by: Chunbo Luo <chunbo.luo@windriver.com>
---
net/sctp/sm_sideeffect.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c
index 86426aa..0e2e269 100644
--- a/net/sctp/sm_sideeffect.c
+++ b/net/sctp/sm_sideeffect.c
@@ -447,7 +447,7 @@ static void sctp_do_8_2_transport_strike(struct sctp_association *asoc,
asoc->overall_error_count++;
if (transport->state != SCTP_INACTIVE &&
- (transport->error_count++ >= transport->pathmaxrxt)) {
+ (transport->error_count++ > transport->pathmaxrxt)) {
SCTP_DEBUG_PRINTK_IPADDR("transport_strike:association %p",
" transport IP: port:%d failed.\n",
asoc,
--
1.6.4
^ permalink raw reply related
* [PATCH 1/2] sctp: fix heartbeat count of association failure
From: Chunbo Luo @ 2009-08-19 7:01 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-sctp, chunbo.luo
RFC4960 Section 8.1 defined that the association should enter CLOSE
state when the value of association error counter exceeds the limit
indicated in the protocol parameter 'Association.Max.Retrans'. This
means that the association should enter CLOSE state after max_retrans+1
heartbeats are not acknowledged.
Signed-off-by: Chunbo Luo <chunbo.luo@windriver.com>
---
net/sctp/sm_statefuns.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index 7288192..90e4f06 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -5190,7 +5190,7 @@ sctp_disposition_t sctp_sf_do_6_3_3_rtx(const struct sctp_endpoint *ep,
SCTP_INC_STATS(SCTP_MIB_T3_RTX_EXPIREDS);
- if (asoc->overall_error_count >= asoc->max_retrans) {
+ if (asoc->overall_error_count > asoc->max_retrans) {
sctp_add_cmd_sf(commands, SCTP_CMD_SET_SK_ERR,
SCTP_ERROR(ETIMEDOUT));
/* CMD_ASSOC_FAILED calls CMD_DELETE_TCB. */
@@ -5404,7 +5404,7 @@ sctp_disposition_t sctp_sf_t2_timer_expire(const struct sctp_endpoint *ep,
((struct sctp_association *)asoc)->shutdown_retries++;
- if (asoc->overall_error_count >= asoc->max_retrans) {
+ if (asoc->overall_error_count > asoc->max_retrans) {
sctp_add_cmd_sf(commands, SCTP_CMD_SET_SK_ERR,
SCTP_ERROR(ETIMEDOUT));
/* Note: CMD_ASSOC_FAILED calls CMD_DELETE_TCB. */
@@ -5487,7 +5487,7 @@ sctp_disposition_t sctp_sf_t4_timer_expire(
* RFC2960 [5] section 8.1 and 8.2.
* association error counter is incremented in SCTP_CMD_STRIKE.
*/
- if (asoc->overall_error_count >= asoc->max_retrans) {
+ if (asoc->overall_error_count > asoc->max_retrans) {
sctp_add_cmd_sf(commands, SCTP_CMD_TIMER_STOP,
SCTP_TO(SCTP_EVENT_TIMEOUT_T4_RTO));
sctp_add_cmd_sf(commands, SCTP_CMD_SET_SK_ERR,
--
1.6.4
^ permalink raw reply related
* Re: [PATCH v3 3/6] vbus: add a "vbus-proxy" bus model for vbus_driver objects
From: Gregory Haskins @ 2009-08-19 6:40 UTC (permalink / raw)
To: Gregory Haskins, Avi Kivity
Cc: Anthony Liguori, Ingo Molnar, alacrityvm-devel,
Michael S. Tsirkin, kvm, linux-kernel, netdev
In-Reply-To: <4A8B9241.20300@redhat.com>
>>> On 8/19/2009 at 1:48 AM, in message <4A8B9241.20300@redhat.com>, Avi Kivity
<avi@redhat.com> wrote:
> On 08/19/2009 08:36 AM, Gregory Haskins wrote:
>>> If virtio net in guest could be improved instead, everyone would
>>> benefit.
>>>
>> So if I whip up a virtio-net backend for vbus with a PCI compliant
>> connector, you are happy?
>>
>
> This doesn't improve virtio-net in any way.
Any why not? (Did you notice I said "PCI compliant", i.e. over virtio-pci)
>
>>> I am doing this, and I wish more people would join. Instead,
>>> you change ABI in a incompatible way.
>>>
>> Only by choice of my particular connector. The ABI is a function of the
>> connector design. So one such model is to terminate the connector in
>> qemu, and surface the resulting objects as PCI devices. I choose not to
>> use this particular design for my connector that I am pushing upstream
>> because I am of the opinion that I can do better by terminating it in
>> the guest directly as a PV optimized bus. However, both connectors can
>> theoretically coexist peacefully.
>>
>
> virtio already supports this model; see lguest and s390. Transporting
> virtio over vbus and vbus over something else doesn't gain anything over
> directly transporting virtio over that something else.
This is not what I am advocating.
Kind Regards,
-Greg
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox