* Paris Hilton & Nicole Richie
From: office @ 2005-12-21 14:46 UTC (permalink / raw)
To: ex-smtp
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: Integrating IXP4xx NPE support in kernel (with IXP4xx accesslibrary)
From: Lennert Buytenhek @ 2005-12-21 13:52 UTC (permalink / raw)
To: Stefan Roese; +Cc: netdev, linux-arm-kernel
In-Reply-To: <200512211300.38163.sr@denx.de>
On Wed, Dec 21, 2005 at 01:00:34PM +0100, Stefan Roese wrote:
> The main question I have is, where should the IXP4xx access-library
> be located in the kernel directory structure?
Maybe you can explain to the list readers what it is and what it does?
If it's ixp4xx-specific, arch/arm/mach-ixp4xx might just be the best
place. The ixp2000 microengine loader was put into arch/arm/mach-ixp2000
for mostly that reason (being ixp2000-specific), while the ethernet driver
that uses that microengine loader will live under drivers/net/.
> Please comment on this and let me know if such an effort has any chance
> of getting accepted into the official kernel.
Assuming that the license issues have all been worked out now, it'll
mostly depend on the quality of the code. (If this is the same Intel
code that I saw a while ago, I think it'll need a fair amount of work
before it can go in.)
cheers,
Lennert
-------------------------------------------------------------------
List admin: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ: http://www.arm.linux.org.uk/mailinglists/faq.php
Etiquette: http://www.arm.linux.org.uk/mailinglists/etiquette.php
^ permalink raw reply
* Registration Confirmation
From: Admin @ 2005-12-21 13:45 UTC (permalink / raw)
To: x-Recipient
[-- Attachment #1: Type: text/plain, Size: 101 bytes --]
Protected message is attached!
***** Go to: http://www.hotmail.com
***** Email: postman@hotmail.com
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* [2.6 patch] drivers/net/sungem.c: gem_remove_one mustn't be __devexit
From: Adrian Bunk @ 2005-12-21 13:25 UTC (permalink / raw)
To: Nils Meyer, Andrew Morton, David S. Miller, Linus Torvalds
Cc: linux-kernel, jgarzik, netdev
In-Reply-To: <200512211323.19242.meyer@work.de>
On Wed, Dec 21, 2005 at 01:23:18PM +0100, Nils Meyer wrote:
> Hi,
Hi Nils,
> When compiling 2.6.15-rc6 or 2.6.14.4 on my sparc I get the following error:
>
> local symbol 0: discarded in section `.exit.text' from drivers/built-in.o [*]
>
> After calling scripts/reference_discarded.pl it gives me the following error:
>
> Error: ./drivers/net/sungem.o .init.text refers to 0000000000000448
> R_SPARC_WDISP30 .exit.text
>...
thanks for this report.
The untested patch below should fix this bug (I'm assuming your .config
hasn't set CONFIG_HOTPLUG).
It's a real problem that these bugs have become runtime errors in
kernel 2.6 on i386 instead of link errors as they were in kernel 2.4 on
i386 (and are still in kernel kernel 2.6 on some other architectures
like sparc64) resulting in fewer people noticing them. :-(
> kind regards
> Nils Meyer
cu
Adrian
<-- snip -->
gem_remove_one() is called from the __devinit gem_init_one().
Therefore, gem_remove_one() mustn't be __devexit.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
--- linux-2.6.15-rc6/drivers/net/sungem.c.old 2005-12-21 14:02:51.000000000 +0100
+++ linux-2.6.15-rc6/drivers/net/sungem.c 2005-12-21 14:03:37.000000000 +0100
@@ -2907,7 +2907,7 @@
return 0;
}
-static void __devexit gem_remove_one(struct pci_dev *pdev)
+static void gem_remove_one(struct pci_dev *pdev)
{
struct net_device *dev = pci_get_drvdata(pdev);
@@ -3181,7 +3181,7 @@
.name = GEM_MODULE_NAME,
.id_table = gem_pci_tbl,
.probe = gem_init_one,
- .remove = __devexit_p(gem_remove_one),
+ .remove = gem_remove_one,
#ifdef CONFIG_PM
.suspend = gem_suspend,
.resume = gem_resume,
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: hostmaster @ 2005-12-21 13:02 UTC (permalink / raw)
To: ralf
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David Stevens @ 2005-12-21 9:39 UTC (permalink / raw)
To: Pavel Machek
Cc: ak, David S. Miller, linux-kernel, mpm, netdev, netdev-owner,
Stephen Hemminger, sri
In-Reply-To: <20051221091114.GA8495@elf.ucw.cz>
> Why not do it the other way? "If you don't hear from me for 2 minutes,
> do a switchover". Then all you have to do is _not_ to send a packet --
> easier to do.
>
> Anything else seems overkill.
> Pavel
Because in some of the scenarios, including ours, it isn't a
simple failover to a known alternate device or configuration --
it is reconfiguring dynamically with information received on a
socket from a remote machine (while the swap device is unavailable).
Limited socket communication without allocating new memory
that may not be available is the problem definition. Avoiding the
problem in the first place (your solution) is effective if you
can do it, of course. The trick is to solve the problem when you
can't avoid it. :-)
+-DLS
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Pavel Machek @ 2005-12-21 9:11 UTC (permalink / raw)
To: Sridhar Samudrala
Cc: Stephen Hemminger, David S. Miller, mpm, ak, linux-kernel, netdev
In-Reply-To: <1134758299.10691.28.camel@w-sridhar2.beaverton.ibm.com>
Hi!
> > If it is only one place, why not pre-allocate one "I'm sick now"
> > skb and hold onto it. Any bigger solution seems to snowball into
> > a huge mess.
>
> But the problem is even sending/receiving a single packet can cause
> multiple dynamic allocations in the networking path all the way from
> the sockets layer->transport->ip->driver.
> To successfully send a packet, we may have to do arp, send acks and
> create cached routes etc. So my patch tried to identify the allocations
> that are needed to succesfully send/receive packets over a pre-established
> socket and adds a new flag GFP_CRITICAL to those calls.
> This doesn't make any difference when we are not in emergency. But when
> we go into emergency, VM will try to satisfy these allocations from a
> critical pool if the normal path leads to failure.
>
> We go into emergency when some management app detects that a swap device
> is about to fail(we are not yet in OOM, but will enter OOM soon). In order
> to avoid entering OOM, we need to send a message over a critical socket to
> a remote server that can initiate failover and switch to a different swap
> device. The switchover will happen within 2 minutes after it is initiated.
> In a cluster environment, the remote server also sends a message to other
> nodes which are also running the management app so that they also enter
> emergency. Once we successfully switch to a different swap device, the remote
> server sends a message to all the nodes and they come out of emergency.
>
> During the period of emergency, all other communications can block. But
> guranteeing the successful delivery of the critical messages will help
> in making sure that we do not enter OOM situation.
Why not do it the other way? "If you don't hear from me for 2 minutes,
do a switchover". Then all you have to do is _not_ to send a packet --
easier to do.
Anything else seems overkill.
Pavel
--
Thanks, Sharp!
^ permalink raw reply
* Re: [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
From: David S. Miller @ 2005-12-21 7:37 UTC (permalink / raw)
To: chris.leech; +Cc: linux-kernel, netdev
In-Reply-To: <41b516cb0512202310q78d5a25apab3c9d6fbb17089e@mail.gmail.com>
From: Chris Leech <chris.leech@gmail.com>
Date: Tue, 20 Dec 2005 23:10:07 -0800
> That could be a good way to deal with it. Actually, I should double
> check the length of tcp_skb_cb. I took a quick look and thought that
> there might be some room left there anyway, even though the comment in
> tcp.h says otherwise.
There isn't, it's basically full on 64-bit systems with ipv6 enabled.
Just put the DMA cookie object explicitly into struct sk_buff,
protected by CONFIG_NET_DMA.
^ permalink raw reply
* Re: [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
From: David S. Miller @ 2005-12-21 7:35 UTC (permalink / raw)
To: dada1; +Cc: christopher.leech, linux-kernel, netdev, andrew.grover,
john.ronciak
In-Reply-To: <43A8F43B.6020307@cosmosbay.com>
From: Eric Dumazet <dada1@cosmosbay.com>
Date: Wed, 21 Dec 2005 07:20:43 +0100
> Please consider not enlarging cb[] if not CONFIG_NET_DMA ?
I also disagree with putting this thing in the cb[] at all.
Just put the dma_cookie_t explicitly into struct sk_buff,
protected by CONFIG_NET_DMA.
^ permalink raw reply
* Re: [RFC][PATCH 1/5] I/OAT DMA support and TCP acceleration
From: Evgeniy Polyakov @ 2005-12-21 7:28 UTC (permalink / raw)
To: Chris Leech; +Cc: lkml, netdev, Grover, Andrew, Ronciak, John
In-Reply-To: <1135142254.13781.18.camel@cleech-mobl>
On Tue, Dec 20, 2005 at 09:17:34PM -0800, Chris Leech (christopher.leech@intel.com) wrote:
> DMA memcpy subsystem
> Provides an API for offloading memory copies to DMA devices.
> Along with client registration and DMA channel allocation, the main APIs are:
> dma_async_memcpy_buf_to_buf()
> dma_async_memcpy_buf_to_pg()
> dma_async_memcpy_pg_to_pg()
> dma_async_memcpy_complete()
Is here at least some locking?
All dma chain/engine list manipulations seem very suspicious.
--
Evgeniy Polyakov
^ permalink raw reply
* Your Password
From: postman @ 2005-12-21 7:21 UTC (permalink / raw)
To: mailserver
[-- Attachment #1: Type: text/plain, Size: 125 bytes --]
Account and Password Information are attached!
***** Go to: http://www.truespectra.com
***** Email: postman@truespectra.com
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Fwd: [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 7:10 UTC (permalink / raw)
To: lkml, netdev
In-Reply-To: <41b516cb0512202305p45439464o6b7ba6c2c88062bc@mail.gmail.com>
Sorry, bounced from vger the first time due to formatting
---------- Forwarded message ----------
From: Chris Leech <christopher.leech@intel.com>
Date: Dec 20, 2005 11:05 PM
Subject: Re: [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
To: Eric Dumazet <dada1@cosmosbay.com>
Cc: lkml <linux-kernel@vger.kernel.org>, netdev
<netdev@vger.kernel.org>, "Grover, Andrew" <andrew.grover@intel.com>,
"Ronciak, John" <john.ronciak@intel.com>
On 12/20/05, Eric Dumazet <dada1@cosmosbay.com> wrote:
> Please consider not enlarging cb[] if not CONFIG_NET_DMA ?
>
> I mean, most machines wont have a compatable NIC, so why should they pay the
> price (memory, cpu) in a critical structure named sk_buff ?
>
> #ifdef CONFIG_NET_DMA
> typedef dma_cookie_t net_dma_cookie_t;
> #else
> typedef struct {} net_dma_cookie_t;
> #endif
>
> ...
>
> char cb[40+sizeof(net_dma_cookie_t)];
>
That could be a good way to deal with it. Actually, I should double
check the length of tcp_skb_cb. I took a quick look and thought that
there might be some room left there anyway, even though the comment in
tcp.h says otherwise.
-Chris
^ permalink raw reply
* Registration Confirmation
From: webmaster @ 2005-12-21 6:33 UTC (permalink / raw)
To: majordomo
[-- Attachment #1: Type: text/plain, Size: 115 bytes --]
Account and Password Information are attached!
***** Go to: http://www.ynmail.com
***** Email: postman@ynmail.com
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
From: Eric Dumazet @ 2005-12-21 6:20 UTC (permalink / raw)
To: Chris Leech; +Cc: lkml, netdev, Grover, Andrew, Ronciak, John
In-Reply-To: <1135142263.13781.21.camel@cleech-mobl>
Chris Leech a écrit :
> Structure changes for TCP recv offload to I/OAT
>
> Adds an async_wait_queue and some additional fields to tcp_sock, a
> copied_early flag to sb_buff and a dma_cookie_t to tcp_skb_cb
>
> Renames cleanup_rbuf to tcp_cleanup_rbuf and makes it non-static so we
> can call it from tcp_input.c
>
> ---
> include/linux/skbuff.h | 5 +++--
> include/linux/tcp.h | 9 +++++++++
> include/net/tcp.h | 10 ++++++++++
> net/core/skbuff.c | 1 +
> net/ipv4/tcp.c | 11 ++++++-----
> net/ipv4/tcp_ipv4.c | 4 ++++
> net/ipv4/tcp_minisocks.c | 1 +
> net/ipv6/tcp_ipv6.c | 1 +
> 8 files changed, 35 insertions(+), 7 deletions(-)
> diff -urp a/include/linux/skbuff.h b/include/linux/skbuff.h
> --- a/include/linux/skbuff.h 2005-12-21 12:05:09.000000000 -0800
> +++ b/include/linux/skbuff.h 2005-12-21 12:10:14.000000000 -0800
> @@ -248,7 +248,7 @@ struct sk_buff {
> * want to keep them across layers you have to do a skb_clone()
> * first. This is owned by whoever has the skb queued ATM.
> */
> - char cb[40];
> + char cb[44];
Hi Chris
Please consider not enlarging cb[] if not CONFIG_NET_DMA ?
I mean, most machines wont have a compatable NIC, so why should they pay the
price (memory, cpu) in a critical structure named sk_buff ?
#ifdef CONFIG_NET_DMA
typedef dma_cookie_t net_dma_cookie_t;
#else
typedef struct {} net_dma_cookie_t;
#endif
...
char cb[40+sizeof(net_dma_cookie_t)];
Same remark apply for the rest of your patch : Please consider to make added
fields and code conditional to CONFIG_NET_DMA
Eric
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: info @ 2005-12-21 6:14 UTC (permalink / raw)
To: majordomo
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* deane Private e consultation and big savin's on our health products. barre
From: Raya @ 2005-12-21 6:08 UTC (permalink / raw)
To: Haywood Brockney; +Cc: knight, ogl-sample-cvs, schultz, daniel, netdev
Hey you, swear to me you'll swing by this e-store- I adore it!
You have to see this great deal I've found on the internet.
Since you are always helping me out, I want to let you know about this
great chance that you can save on health supplements.
With this esite, I got a gratis consultation to find the proper medical
products.
The prices of these performance enhancers are up to 55% off the retail
price.
And the orders are delivered quick to my apartment and I can even check the
progress online.
Not only can I order at the convenience of my house, I also feel more
relaxed cause I have the the correct health pills at the right time.
cgarqynmg http://au.geocities.com/amedzephyrgoodau
`But why? and she's so nice.'
dikas fieldhouse fjuang xm05 fabrication-related flkalt
'Your manner, no less than your actions, assures me that you have never
forgotten it, or ceased to think of it with bitterness,' returned Mr.
Brownlow. 'I speak of fifteen years ago, when you were not more than eleven
years old, and your father but one-and-thirty--for he was, I repeat, a boy,
when HIS father ordered him to marry. Must I go back to events which cast a
shade upon the memory of your parent, or will you spare it, and disclose to
me the truth?'
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: Admin @ 2005-12-21 6:07 UTC (permalink / raw)
To: Z-Account
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* [RFC][PATCH 5/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
TCP recv offload to I/OAT
Locks down user pages and sets up for the DMA in tcp_recvmsg, then calls
dma_async_try_early_copy in tcp_v4_do_rcv
---
tcp.c | 166 +++++++++++++++++++++++++++++++++++++++++++++---------------
tcp_input.c | 63 +++++++++++++++++++++-
tcp_ipv4.c | 16 +++++
3 files changed, 202 insertions(+), 43 deletions(-)
diff -rup a/net/ipv4/tcp.c b/net/ipv4/tcp.c
--- a/net/ipv4/tcp.c 2005-12-21 12:10:49.000000000 -0800
+++ b/net/ipv4/tcp.c 2005-12-21 12:14:42.000000000 -0800
@@ -257,6 +257,7 @@
#include <linux/fs.h>
#include <linux/random.h>
#include <linux/bootmem.h>
+#include <linux/dmaengine.h>
#include <net/icmp.h>
#include <net/tcp.h>
@@ -941,7 +942,7 @@ void tcp_cleanup_rbuf(struct sock *sk, i
struct tcp_sock *tp = tcp_sk(sk);
int time_to_ack = 0;
-#if TCP_DEBUG
+#if TCP_DEBUG && !defined(CONFIG_NET_DMA)
struct sk_buff *skb = skb_peek(&sk->sk_receive_queue);
BUG_TRAP(!skb || before(tp->copied_seq, TCP_SKB_CB(skb)->end_seq));
@@ -1132,6 +1133,21 @@ int tcp_recvmsg(struct kiocb *iocb, stru
target = sock_rcvlowat(sk, flags & MSG_WAITALL, len);
+#ifdef CONFIG_NET_DMA
+ if ((len > 1024) && !(flags & MSG_PEEK) && !sysctl_tcp_low_latency) {
+ preempt_disable();
+ tp->ucopy.dma_chan = __get_cpu_var(net_dma);
+ preempt_enable();
+ } else {
+ tp->ucopy.dma_chan = NULL;
+ }
+
+ if (tp->ucopy.dma_chan)
+ if (dma_lock_iovec_pages(msg->msg_iov, len,
+ &tp->ucopy.locked_list))
+ tp->ucopy.dma_chan = NULL;
+#endif
+
do {
struct sk_buff *skb;
u32 offset;
@@ -1161,7 +1177,12 @@ int tcp_recvmsg(struct kiocb *iocb, stru
"seq %X\n", *seq, TCP_SKB_CB(skb)->seq);
break;
}
- offset = *seq - TCP_SKB_CB(skb)->seq;
+
+ if (!skb->copied_early)
+ offset = *seq - TCP_SKB_CB(skb)->seq;
+ else
+ offset = 0;
+
if (skb->h.th->syn)
offset--;
if (offset < skb->len)
@@ -1273,6 +1294,8 @@ int tcp_recvmsg(struct kiocb *iocb, stru
} else
sk_wait_data(sk, &timeo);
+ tp->ucopy.wakeup = 0;
+
if (user_recv) {
int chunk;
@@ -1304,45 +1327,69 @@ do_prequeue:
}
continue;
- found_ok_skb:
+found_ok_skb:
/* Ok so how much can we use? */
- used = skb->len - offset;
- if (len < used)
- used = len;
-
- /* Do we have urgent data here? */
- if (tp->urg_data) {
- u32 urg_offset = tp->urg_seq - *seq;
- if (urg_offset < used) {
- if (!urg_offset) {
- if (!sock_flag(sk, SOCK_URGINLINE)) {
- ++*seq;
- offset++;
- used--;
- if (!used)
- goto skip_copy;
- }
- } else
- used = urg_offset;
+ if (!skb->copied_early) {
+ used = skb->len - offset;
+ if (len < used)
+ used = len;
+
+ /* Do we have urgent data here? */
+ if (tp->urg_data) {
+ u32 urg_offset = tp->urg_seq - *seq;
+ if (urg_offset < used) {
+ if (!urg_offset) {
+ if (!sock_flag(sk, SOCK_URGINLINE)) {
+ ++*seq;
+ offset++;
+ used--;
+ if (!used)
+ goto skip_copy;
+ }
+ } else
+ used = urg_offset;
+ }
}
- }
- if (!(flags & MSG_TRUNC)) {
- err = skb_copy_datagram_iovec(skb, offset,
- msg->msg_iov, used);
- if (err) {
- /* Exception. Bailout! */
- if (!copied)
- copied = -EFAULT;
- break;
+ if (!(flags & MSG_TRUNC)) {
+ if (tp->ucopy.dma_chan) {
+ tp->ucopy.dma_cookie = dma_skb_copy_datagram_iovec(
+ tp->ucopy.dma_chan, skb, offset,
+ msg->msg_iov, used,
+ tp->ucopy.locked_list);
+
+ if (tp->ucopy.dma_cookie < 0) {
+
+ printk(KERN_ALERT "dma_cookie < 0\n");
+
+ /* Exception. Bailout! */
+ if (!copied)
+ copied = -EFAULT;
+ break;
+ }
+ if ((offset + used) == skb->len)
+ skb->copied_early = 1;
+
+ } else {
+ err = skb_copy_datagram_iovec(skb, offset,
+ msg->msg_iov, used);
+ if (err) {
+ /* Exception. Bailout! */
+ if (!copied)
+ copied = -EFAULT;
+ break;
+ }
+ }
}
- }
- *seq += used;
- copied += used;
- len -= used;
+ *seq += used;
+ copied += used;
+ len -= used;
- tcp_rcv_space_adjust(sk);
+ tcp_rcv_space_adjust(sk);
+ } else {
+ used = skb->len;
+ }
skip_copy:
if (tp->urg_data && after(tp->copied_seq, tp->urg_seq)) {
@@ -1354,15 +1401,27 @@ skip_copy:
if (skb->h.th->fin)
goto found_fin_ok;
- if (!(flags & MSG_PEEK))
- sk_eat_skb(sk, skb);
+ if (!(flags & MSG_PEEK)) {
+ if (!skb->copied_early)
+ sk_eat_skb(sk, skb);
+ else {
+ __skb_unlink(skb, &sk->sk_receive_queue);
+ __skb_queue_tail(&tp->async_wait_queue, skb);
+ }
+ }
continue;
found_fin_ok:
/* Process the FIN. */
++*seq;
- if (!(flags & MSG_PEEK))
- sk_eat_skb(sk, skb);
+ if (!(flags & MSG_PEEK)) {
+ if (!skb->copied_early)
+ sk_eat_skb(sk, skb);
+ else {
+ __skb_unlink(skb, &sk->sk_receive_queue);
+ __skb_queue_tail(&tp->async_wait_queue, skb);
+ }
+ }
break;
} while (len > 0);
@@ -1383,8 +1442,37 @@ skip_copy:
tp->ucopy.task = NULL;
tp->ucopy.len = 0;
+ tp->ucopy.bytes_early_copied = 0;
}
+
+#ifdef CONFIG_NET_DMA
+ if (tp->ucopy.dma_chan) {
+ struct sk_buff *skb;
+ dma_cookie_t done, used;
+
+ dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+
+ while (dma_async_memcpy_complete(tp->ucopy.dma_chan,
+ tp->ucopy.dma_cookie, &done,
+ &used) == DMA_IN_PROGRESS) {
+ /* do partial cleanup of async_wait_queue */
+ while ((skb = skb_peek(&tp->async_wait_queue)) &&
+ (dma_async_is_complete(TCP_SKB_CB(skb)->dma_cookie,
+ done, used) == DMA_SUCCESS)) {
+ __skb_dequeue(&tp->async_wait_queue);
+ kfree_skb(skb);
+ }
+ }
+
+ /* Safe to free early-copied skbs now */
+ __skb_queue_purge(&tp->async_wait_queue);
+ dma_unlock_iovec_pages(tp->ucopy.locked_list);
+ tp->ucopy.dma_chan = NULL;
+ tp->ucopy.locked_list = NULL;
+ }
+#endif
+
/* According to UNIX98, msg_name/msg_namelen are ignored
* on connected socket. I was just happy when found this 8) --ANK
*/
diff -rup a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
--- a/net/ipv4/tcp_input.c 2005-12-21 12:05:00.000000000 -0800
+++ b/net/ipv4/tcp_input.c 2005-12-21 12:14:42.000000000 -0800
@@ -70,6 +70,7 @@
#include <net/tcp.h>
#include <net/inet_common.h>
#include <linux/ipsec.h>
+#include <linux/dmaengine.h>
#include <asm/unaligned.h>
int sysctl_tcp_timestamps = 1;
@@ -3623,10 +3624,12 @@ tcp_checksum_complete_user(struct sock *
* the rest is checked inline. Fast processing is turned on in
* tcp_data_queue when everything is OK.
*/
+
int tcp_rcv_established(struct sock *sk, struct sk_buff *skb,
struct tcphdr *th, unsigned len)
{
struct tcp_sock *tp = tcp_sk(sk);
+ int eaten = 0;
/*
* Header prediction.
@@ -3715,12 +3718,11 @@ int tcp_rcv_established(struct sock *sk,
goto discard;
}
} else {
- int eaten = 0;
-
if (tp->ucopy.task == current &&
tp->copied_seq == tp->rcv_nxt &&
len - tcp_header_len <= tp->ucopy.len &&
- sock_owned_by_user(sk)) {
+ sock_owned_by_user(sk) &&
+ !skb->copied_early) {
__set_current_state(TASK_RUNNING);
if (!tcp_copy_to_iovec(sk, skb, tcp_header_len)) {
@@ -3767,6 +3769,9 @@ int tcp_rcv_established(struct sock *sk,
__skb_queue_tail(&sk->sk_receive_queue, skb);
sk_stream_set_owner_r(skb, sk);
tp->rcv_nxt = TCP_SKB_CB(skb)->end_seq;
+
+ if (skb->copied_early)
+ tcp_cleanup_rbuf(sk, skb->len);
}
tcp_event_data_recv(sk, tp, skb);
@@ -3781,6 +3786,9 @@ int tcp_rcv_established(struct sock *sk,
__tcp_ack_snd_check(sk, 0);
no_ack:
+ if (skb->copied_early)
+ return 0;
+
if (eaten)
__kfree_skb(skb);
else
@@ -3864,6 +3872,55 @@ discard:
return 0;
}
+#ifdef CONFIG_NET_DMA
+void dma_async_try_early_copy(struct sock *sk, struct sk_buff *skb)
+{
+ struct tcp_sock *tp = tcp_sk(sk);
+ int dma_cookie;
+ int chunk = skb->len - tp->tcp_header_len;
+ struct tcphdr *th = skb->h.th;
+
+ if ((tp->ucopy.task == NULL) || (tp->ucopy.dma_chan == NULL) || tp->ucopy.wakeup)
+ return;
+
+ if ((tcp_flag_word(th) & TCP_HP_BITS) != tp->pred_flags ||
+ TCP_SKB_CB(skb)->seq != tp->rcv_nxt)
+ return;
+
+ if (tp->ucopy.dma_chan &&
+ chunk > 0 &&
+ chunk <= (tp->ucopy.len) &&
+ tp->copied_seq == tp->rcv_nxt &&
+ skb->ip_summed == CHECKSUM_UNNECESSARY) {
+
+ dma_cookie = dma_skb_copy_datagram_iovec(tp->ucopy.dma_chan,
+ skb, tp->tcp_header_len,
+ tp->ucopy.iov, chunk, tp->ucopy.locked_list);
+
+ if (dma_cookie < 0)
+ return;
+
+ tp->ucopy.dma_cookie = dma_cookie;
+ skb->copied_early = 1;
+
+ tp->ucopy.bytes_early_copied += chunk;
+ tp->copied_seq += chunk;
+ tp->ucopy.len -= chunk;
+ tcp_rcv_space_adjust(sk);
+
+ if (tp->ucopy.len == 0) {
+ tp->ucopy.wakeup = 1;
+ wake_up_interruptible(sk->sk_sleep);
+ }
+ } else if (chunk > 0) {
+ tp->ucopy.wakeup = 1;
+ wake_up_interruptible(sk->sk_sleep);
+ }
+}
+
+EXPORT_SYMBOL(dma_async_try_early_copy);
+#endif /* CONFIG_NET_DMA */
+
static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
struct tcphdr *th, unsigned len)
{
diff -rup a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
--- a/net/ipv4/tcp_ipv4.c 2005-12-21 12:10:49.000000000 -0800
+++ b/net/ipv4/tcp_ipv4.c 2005-12-21 12:14:42.000000000 -0800
@@ -1148,6 +1148,11 @@ int tcp_v4_do_rcv(struct sock *sk, struc
{
if (sk->sk_state == TCP_ESTABLISHED) { /* Fast path */
TCP_CHECK_TIMER(sk);
+
+#ifdef CONFIG_NET_DMA
+ dma_async_try_early_copy(sk, skb);
+#endif
+
if (tcp_rcv_established(sk, skb, skb->h.th, skb->len))
goto reset;
TCP_CHECK_TIMER(sk);
@@ -1256,10 +1261,19 @@ process:
bh_lock_sock(sk);
ret = 0;
if (!sock_owned_by_user(sk)) {
- if (!tcp_prequeue(sk, skb))
+#ifdef CONFIG_NET_DMA
+ struct tcp_sock *tp = tcp_sk(sk);
+ if (tp->ucopy.dma_chan)
ret = tcp_v4_do_rcv(sk, skb);
+ else
+#endif
+ {
+ if (!tcp_prequeue(sk, skb))
+ ret = tcp_v4_do_rcv(sk, skb);
+ }
} else
sk_add_backlog(sk, skb);
+
bh_unlock_sock(sk);
sock_put(sk);
^ permalink raw reply
* [RFC][PATCH 4/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
Structure changes for TCP recv offload to I/OAT
Adds an async_wait_queue and some additional fields to tcp_sock, a
copied_early flag to sb_buff and a dma_cookie_t to tcp_skb_cb
Renames cleanup_rbuf to tcp_cleanup_rbuf and makes it non-static so we
can call it from tcp_input.c
---
include/linux/skbuff.h | 5 +++--
include/linux/tcp.h | 9 +++++++++
include/net/tcp.h | 10 ++++++++++
net/core/skbuff.c | 1 +
net/ipv4/tcp.c | 11 ++++++-----
net/ipv4/tcp_ipv4.c | 4 ++++
net/ipv4/tcp_minisocks.c | 1 +
net/ipv6/tcp_ipv6.c | 1 +
8 files changed, 35 insertions(+), 7 deletions(-)
diff -urp a/include/linux/skbuff.h b/include/linux/skbuff.h
--- a/include/linux/skbuff.h 2005-12-21 12:05:09.000000000 -0800
+++ b/include/linux/skbuff.h 2005-12-21 12:10:14.000000000 -0800
@@ -248,7 +248,7 @@ struct sk_buff {
* want to keep them across layers you have to do a skb_clone()
* first. This is owned by whoever has the skb queued ATM.
*/
- char cb[40];
+ char cb[44];
unsigned int len,
data_len,
@@ -261,7 +261,8 @@ struct sk_buff {
nohdr:1,
nfctinfo:3;
__u8 pkt_type:3,
- fclone:2;
+ fclone:2,
+ copied_early:1;
__be16 protocol;
void (*destructor)(struct sk_buff *skb);
diff -urp a/include/linux/tcp.h b/include/linux/tcp.h
--- a/include/linux/tcp.h 2005-12-21 12:05:09.000000000 -0800
+++ b/include/linux/tcp.h 2005-12-21 12:10:14.000000000 -0800
@@ -18,6 +18,7 @@
#define _LINUX_TCP_H
#include <linux/types.h>
+#include <linux/dmaengine.h>
#include <asm/byteorder.h>
struct tcphdr {
@@ -249,6 +250,13 @@ struct tcp_sock {
struct iovec *iov;
int memory;
int len;
+
+ /* members for async copy */
+ int wakeup;
+ struct dma_chan *dma_chan;
+ int bytes_early_copied;
+ struct dma_locked_list *locked_list;
+ dma_cookie_t dma_cookie;
} ucopy;
__u32 snd_wl1; /* Sequence for window update */
@@ -294,6 +302,7 @@ struct tcp_sock {
__u32 snd_cwnd_stamp;
struct sk_buff_head out_of_order_queue; /* Out of order segments go here */
+ struct sk_buff_head async_wait_queue; /* DMA unfinished segments go here */
struct tcp_func *af_specific; /* Operations which are AF_INET{4,6} specific */
diff -urp a/include/net/tcp.h b/include/net/tcp.h
--- a/include/net/tcp.h 2005-12-21 12:05:09.000000000 -0800
+++ b/include/net/tcp.h 2005-12-21 12:10:14.000000000 -0800
@@ -563,6 +563,9 @@ extern u32 __tcp_select_window(struct so
* 40 bytes on 64-bit machines, if this grows please adjust
* skbuff.h:skbuff->cb[xxx] size appropriately.
*/
+
+#include <linux/dmaengine.h>
+
struct tcp_skb_cb {
union {
struct inet_skb_parm h4;
@@ -602,6 +605,7 @@ struct tcp_skb_cb {
__u16 urg_ptr; /* Valid w/URG flags is set. */
__u32 ack_seq; /* Sequence number ACK'd */
+ dma_cookie_t dma_cookie; /* async copy token */
};
#define TCP_SKB_CB(__skb) ((struct tcp_skb_cb *)&((__skb)->cb[0]))
@@ -867,8 +871,14 @@ static __inline__ void tcp_prequeue_init
{
tp->ucopy.task = NULL;
tp->ucopy.len = 0;
+ tp->ucopy.wakeup = 0;
tp->ucopy.memory = 0;
skb_queue_head_init(&tp->ucopy.prequeue);
+
+ tp->ucopy.dma_chan = NULL;
+ tp->ucopy.bytes_early_copied = 0;
+ tp->ucopy.locked_list = NULL;
+ tp->ucopy.dma_cookie = 0;
}
/* Packet is added to VJ-style prequeue for processing in process
diff -urp a/net/core/skbuff.c b/net/core/skbuff.c
--- a/net/core/skbuff.c 2005-12-21 12:05:09.000000000 -0800
+++ b/net/core/skbuff.c 2005-12-21 12:10:14.000000000 -0800
@@ -400,6 +400,7 @@ struct sk_buff *skb_clone(struct sk_buff
C(local_df);
n->cloned = 1;
n->nohdr = 0;
+ C(copied_early);
C(pkt_type);
C(ip_summed);
C(priority);
diff -urp a/net/ipv4/tcp.c b/net/ipv4/tcp.c
--- a/net/ipv4/tcp.c 2005-12-21 12:05:09.000000000 -0800
+++ b/net/ipv4/tcp.c 2005-12-21 12:10:27.000000000 -0800
@@ -936,7 +936,7 @@ static int tcp_recv_urg(struct sock *sk,
* calculation of whether or not we must ACK for the sake of
* a window update.
*/
-static void cleanup_rbuf(struct sock *sk, int copied)
+void tcp_cleanup_rbuf(struct sock *sk, int copied)
{
struct tcp_sock *tp = tcp_sk(sk);
int time_to_ack = 0;
@@ -1085,7 +1085,7 @@ int tcp_read_sock(struct sock *sk, read_
/* Clean up data we have read: This will do ACK frames. */
if (copied)
- cleanup_rbuf(sk, copied);
+ tcp_cleanup_rbuf(sk, copied);
return copied;
}
@@ -1219,7 +1219,7 @@ int tcp_recvmsg(struct kiocb *iocb, stru
}
}
- cleanup_rbuf(sk, copied);
+ tcp_cleanup_rbuf(sk, copied);
if (!sysctl_tcp_low_latency && tp->ucopy.task == user_recv) {
/* Install new reader */
@@ -1390,7 +1390,7 @@ skip_copy:
*/
/* Clean up data we have read: This will do ACK frames. */
- cleanup_rbuf(sk, copied);
+ tcp_cleanup_rbuf(sk, copied);
TCP_CHECK_TIMER(sk);
release_sock(sk);
@@ -1652,6 +1652,7 @@ int tcp_disconnect(struct sock *sk, int
__skb_queue_purge(&sk->sk_receive_queue);
sk_stream_writequeue_purge(sk);
__skb_queue_purge(&tp->out_of_order_queue);
+ __skb_queue_purge(&tp->async_wait_queue);
inet->dport = 0;
@@ -1855,7 +1856,7 @@ int tcp_setsockopt(struct sock *sk, int
(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT) &&
inet_csk_ack_scheduled(sk)) {
icsk->icsk_ack.pending |= ICSK_ACK_PUSHED;
- cleanup_rbuf(sk, 1);
+ tcp_cleanup_rbuf(sk, 1);
if (!(val & 1))
icsk->icsk_ack.pingpong = 1;
}
diff -urp a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
--- a/net/ipv4/tcp_ipv4.c 2005-12-14 15:50:41.000000000 -0800
+++ b/net/ipv4/tcp_ipv4.c 2005-12-21 12:10:27.000000000 -0800
@@ -1414,6 +1414,7 @@ static int tcp_v4_init_sock(struct sock
struct tcp_sock *tp = tcp_sk(sk);
skb_queue_head_init(&tp->out_of_order_queue);
+ skb_queue_head_init(&tp->async_wait_queue);
tcp_init_xmit_timers(sk);
tcp_prequeue_init(tp);
@@ -1466,6 +1467,9 @@ int tcp_v4_destroy_sock(struct sock *sk)
/* Cleans up our, hopefully empty, out_of_order_queue. */
__skb_queue_purge(&tp->out_of_order_queue);
+ /* Cleans up our async_wait_queue */
+ __skb_queue_purge(&tp->async_wait_queue);
+
/* Clean prequeue, it must be empty really */
__skb_queue_purge(&tp->ucopy.prequeue);
diff -urp a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
--- a/net/ipv4/tcp_minisocks.c 2005-12-14 15:50:41.000000000 -0800
+++ b/net/ipv4/tcp_minisocks.c 2005-12-21 12:10:27.000000000 -0800
@@ -389,6 +389,7 @@ struct sock *tcp_create_openreq_child(st
tcp_set_ca_state(newsk, TCP_CA_Open);
tcp_init_xmit_timers(newsk);
skb_queue_head_init(&newtp->out_of_order_queue);
+ skb_queue_head_init(&newtp->async_wait_queue);
newtp->rcv_wup = treq->rcv_isn + 1;
newtp->write_seq = treq->snt_isn + 1;
newtp->pushed_seq = newtp->write_seq;
diff -urp a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
--- a/net/ipv6/tcp_ipv6.c 2005-12-14 15:50:41.000000000 -0800
+++ b/net/ipv6/tcp_ipv6.c 2005-12-21 12:10:27.000000000 -0800
@@ -1863,6 +1863,7 @@ static int tcp_v6_init_sock(struct sock
struct tcp_sock *tp = tcp_sk(sk);
skb_queue_head_init(&tp->out_of_order_queue);
+ skb_queue_head_init(&tp->async_wait_queue);
tcp_init_xmit_timers(sk);
tcp_prequeue_init(tp);
^ permalink raw reply
* [RFC][PATCH 3/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
Utility functions for offloading sk_buff to iovec copies
Provides for pinning user space pages in memory, copying to iovecs, and
copying from sk_buffs including fragmented and chained sk_buffs.
---
net/core/user_dma.c | 410 ++++++++++++++++++++++++++++++++++++++++++++++++++
net/core/Makefile | 3
2 files changed, 412 insertions(+), 1 deletion(-)
--- /dev/null
+++ b/net/core/user_dma.c
@@ -0,0 +1,410 @@
+/*
+ Copyright(c) 2004 - 2005 Intel Corporation
+ Portions based on net/core/datagram.c and copyrighted by their authors.
+
+ This code allows the net stack to make use of a DMA engine for
+ skb to iovec copies.
+*/
+
+#include <linux/dmaengine.h>
+#include <linux/pagemap.h>
+#include <linux/socket.h>
+#include <linux/rtnetlink.h> /* for BUG_TRAP */
+#include <net/tcp.h>
+#include <asm/io.h>
+#include <asm/uaccess.h>
+
+#ifdef CONFIG_NET_DMA
+
+#define NUM_PAGES_SPANNED(start, length) \
+ ((PAGE_ALIGN((unsigned long)start + length) - \
+ ((unsigned long)start & PAGE_MASK)) >> PAGE_SHIFT)
+
+/*
+ * Lock down all the iovec pages needed for len bytes.
+ * Return a struct dma_locked_list to keep track of pages locked down.
+ *
+ * We are allocating a single chunk of memory, and then carving it up into
+ * 3 sections, the latter 2 whose size depends on the number of iovecs and the
+ * total number of pages, respectively.
+ */
+int dma_lock_iovec_pages(struct iovec *iov, size_t len,
+ struct dma_locked_list **locked_list)
+{
+ struct dma_locked_list *local_list;
+ struct page **pages;
+ int i;
+ int ret;
+
+ int nr_iovecs = 0;
+ int iovec_len_used = 0;
+ int iovec_pages_used = 0;
+
+ /* don't lock down non-user-based iovecs */
+ if (segment_eq(get_fs(), KERNEL_DS)) {
+ *locked_list = NULL;
+ return 0;
+ }
+
+ /* determine how many iovecs/pages there are, up front */
+ do {
+ iovec_len_used += iov[nr_iovecs].iov_len;
+ iovec_pages_used += NUM_PAGES_SPANNED(iov[nr_iovecs].iov_base,
+ iov[nr_iovecs].iov_len);
+ nr_iovecs++;
+ } while (iovec_len_used < len);
+
+ /* single kmalloc for locked list, page_list[], and the page arrays */
+ local_list = kmalloc(sizeof(*local_list)
+ + (nr_iovecs * sizeof (struct dma_page_list))
+ + (iovec_pages_used * sizeof (struct page*)), GFP_KERNEL);
+ if (!local_list)
+ return -ENOMEM;
+
+ /* list of pages starts right after the page list array */
+ pages = (struct page **) &local_list->page_list[nr_iovecs];
+
+ /* it's a userspace pointer */
+ might_sleep();
+
+ for (i = 0; i < nr_iovecs; i++) {
+ struct dma_page_list *page_list = &local_list->page_list[i];
+
+ len -= iov[i].iov_len;
+
+ if (!access_ok(VERIFY_WRITE, iov[i].iov_base, iov[i].iov_len)) {
+ dma_unlock_iovec_pages(local_list);
+ return -EFAULT;
+ }
+
+ page_list->nr_pages = NUM_PAGES_SPANNED(iov[i].iov_base,
+ iov[i].iov_len);
+ page_list->base_address = iov[i].iov_base;
+
+ page_list->pages = pages;
+ pages += page_list->nr_pages;
+
+ /* lock pages down */
+ down_read(¤t->mm->mmap_sem);
+ ret = get_user_pages(
+ current,
+ current->mm,
+ (unsigned long) iov[i].iov_base,
+ page_list->nr_pages,
+ 1,
+ 0,
+ page_list->pages,
+ NULL);
+ up_read(¤t->mm->mmap_sem);
+
+ if (ret != page_list->nr_pages) {
+ goto mem_error;
+ }
+
+ local_list->nr_iovecs = i + 1;
+ }
+
+ *locked_list = local_list;
+ return 0;
+
+mem_error:
+ dma_unlock_iovec_pages(local_list);
+ return -ENOMEM;
+}
+
+void dma_unlock_iovec_pages(struct dma_locked_list *locked_list)
+{
+ int i, j;
+
+ if (!locked_list)
+ return;
+
+ for (i = 0; i < locked_list->nr_iovecs; i++) {
+ struct dma_page_list *page_list = &locked_list->page_list[i];
+ for (j = 0; j < page_list->nr_pages; j++) {
+ SetPageDirty(page_list->pages[j]);
+ page_cache_release(page_list->pages[j]);
+ }
+ }
+
+ kfree(locked_list);
+}
+
+static dma_cookie_t dma_memcpy_tokerneliovec(struct dma_chan *chan,
+ struct iovec *iov, unsigned char *kdata, size_t len)
+{
+ dma_cookie_t dma_cookie = 0;
+
+ while (len > 0) {
+ if (iov->iov_len) {
+ int copy = min_t(unsigned int, iov->iov_len, len);
+ dma_cookie = dma_async_memcpy_buf_to_buf(
+ chan,
+ iov->iov_base,
+ kdata,
+ copy);
+ kdata += copy;
+ len -= copy;
+ iov->iov_len -= copy;
+ iov->iov_base += copy;
+ }
+ iov++;
+ }
+
+ return dma_cookie;
+}
+
+/*
+ * We have already locked down the pages we will be using in the iovecs.
+ * Each entry in iov array has corresponding entry in locked_list->page_list.
+ * Using array indexing to keep iov[] and page_list[] in sync.
+ * Initial elements in iov array's iov->iov_len will be 0 if already copied into
+ * by another call.
+ * iov array length remaining guaranteed to be bigger than len.
+ */
+static dma_cookie_t dma_memcpy_toiovec(struct dma_chan *chan, struct iovec *iov,
+ struct dma_locked_list *locked_list,
+ unsigned char *kdata, size_t len)
+{
+ int iov_byte_offset;
+ int copy;
+ dma_cookie_t dma_cookie = 0;
+ int iovec_idx;
+ int page_idx;
+
+ if (!chan)
+ return memcpy_toiovec(iov, kdata, len);
+
+ /* -> kernel copies (e.g. smbfs) */
+ if (!locked_list)
+ return dma_memcpy_tokerneliovec(chan, iov, kdata, len);
+
+ iovec_idx = 0;
+ while (iovec_idx < locked_list->nr_iovecs) {
+ struct dma_page_list *page_list;
+
+ /* skip already used-up iovecs */
+ while (!iov[iovec_idx].iov_len)
+ iovec_idx++;
+
+ page_list = &locked_list->page_list[iovec_idx];
+
+ iov_byte_offset = ((unsigned long)iov[iovec_idx].iov_base & ~PAGE_MASK);
+ page_idx = (((unsigned long)iov[iovec_idx].iov_base & PAGE_MASK)
+ - ((unsigned long)page_list->base_address & PAGE_MASK)) >> PAGE_SHIFT;
+
+ /* break up copies to not cross page boundary */
+ while (iov[iovec_idx].iov_len) {
+ copy = min_t(int, PAGE_SIZE - iov_byte_offset, len);
+ copy = min_t(int, copy, iov[iovec_idx].iov_len);
+
+ dma_cookie = dma_async_memcpy_buf_to_pg(chan,
+ page_list->pages[page_idx],
+ iov_byte_offset,
+ kdata,
+ copy);
+
+ len -= copy;
+ iov[iovec_idx].iov_len -= copy;
+ iov[iovec_idx].iov_base += copy;
+
+ if (!len)
+ return dma_cookie;
+
+ kdata += copy;
+ iov_byte_offset = 0;
+ page_idx++;
+ }
+ iovec_idx++;
+ }
+
+ /* really bad if we ever run out of iovecs */
+ BUG();
+ return -EFAULT;
+}
+
+static dma_cookie_t dma_memcpy_pg_toiovec(struct dma_chan *chan,
+ struct iovec *iov, struct dma_locked_list *locked_list,
+ struct page *page, unsigned int offset, size_t len)
+{
+ int iov_byte_offset;
+ int copy;
+ dma_cookie_t dma_cookie = 0;
+ int iovec_idx;
+ int page_idx;
+ int err;
+
+ /* this needs as-yet-unimplemented buf-to-buff, so punt. */
+ /* TODO: use dma for this */
+ if (!chan || !locked_list) {
+ u8 *vaddr = kmap(page);
+ err = memcpy_toiovec(iov, vaddr + offset, len);
+ kunmap(page);
+ return err;
+ }
+
+ iovec_idx = 0;
+ while (iovec_idx < locked_list->nr_iovecs) {
+ struct dma_page_list *page_list;
+
+ /* skip already used-up iovecs */
+ while (!iov[iovec_idx].iov_len)
+ iovec_idx++;
+
+ page_list = &locked_list->page_list[iovec_idx];
+
+ iov_byte_offset = ((unsigned long)iov[iovec_idx].iov_base & ~PAGE_MASK);
+ page_idx = (((unsigned long)iov[iovec_idx].iov_base & PAGE_MASK)
+ - ((unsigned long)page_list->base_address & PAGE_MASK)) >> PAGE_SHIFT;
+
+ /* break up copies to not cross page boundary */
+ while (iov[iovec_idx].iov_len) {
+ copy = min_t(int, PAGE_SIZE - iov_byte_offset, len);
+ copy = min_t(int, copy, iov[iovec_idx].iov_len);
+
+ dma_cookie = dma_async_memcpy_pg_to_pg(chan,
+ page_list->pages[page_idx],
+ iov_byte_offset,
+ page,
+ offset,
+ copy);
+
+ len -= copy;
+ iov[iovec_idx].iov_len -= copy;
+ iov[iovec_idx].iov_base += copy;
+
+ if (!len)
+ return dma_cookie;
+
+ offset += copy;
+ iov_byte_offset = 0;
+ page_idx++;
+ }
+ iovec_idx++;
+ }
+
+ /* really bad if we ever run out of iovecs */
+ BUG();
+ return -EFAULT;
+}
+
+void dma_memcpy_toiovec_wait(struct dma_chan *chan, dma_cookie_t cookie)
+{
+ if (cookie <= 0)
+ return;
+
+ dma_async_wait_for_completion(chan, cookie);
+}
+
+/**
+ * dma_skb_copy_datagram_iovec - Copy a datagram to an iovec.
+ * @skb - buffer to copy
+ * @offset - offset in the buffer to start copying from
+ * @iovec - io vector to copy to
+ * @len - amount of data to copy from buffer to iovec
+ * @locked_list - locked iovec buffer data
+ *
+ * Note: the iovec is modified during the copy.
+ */
+int dma_skb_copy_datagram_iovec(struct dma_chan *chan,
+ const struct sk_buff *skb, int offset, struct iovec *to,
+ size_t len, struct dma_locked_list *locked_list)
+{
+ int start = skb_headlen(skb);
+ int i, copy = start - offset;
+ dma_cookie_t cookie = 0;
+
+ /* Copy header. */
+ if (copy > 0) {
+ if (copy > len)
+ copy = len;
+ if ((cookie = dma_memcpy_toiovec(chan, to, locked_list,
+ skb->data + offset, copy)) < 0)
+ goto fault;
+ if ((len -= copy) == 0)
+ goto end;
+ offset += copy;
+ }
+
+ /* Copy paged appendix. Hmm... why does this look so complicated? */
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ int end;
+
+ BUG_TRAP(start <= offset + len);
+
+ end = start + skb_shinfo(skb)->frags[i].size;
+ if ((copy = end - offset) > 0) {
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
+ struct page *page = frag->page;
+
+ if (copy > len)
+ copy = len;
+
+ cookie = dma_memcpy_pg_toiovec(chan, to, locked_list, page,
+ frag->page_offset + offset - start, copy);
+ if (cookie < 0)
+ goto fault;
+ if (!(len -= copy))
+ goto end;
+ offset += copy;
+ }
+ start = end;
+ }
+
+ if (skb_shinfo(skb)->frag_list) {
+ struct sk_buff *list = skb_shinfo(skb)->frag_list;
+
+ for (; list; list = list->next) {
+ int end;
+
+ BUG_TRAP(start <= offset + len);
+
+ end = start + list->len;
+ if ((copy = end - offset) > 0) {
+ if (copy > len)
+ copy = len;
+ if ((cookie = dma_skb_copy_datagram_iovec(chan, list,
+ offset - start, to, copy, locked_list)) < 0)
+ goto fault;
+ if ((len -= copy) == 0)
+ goto end;
+ offset += copy;
+ }
+ start = end;
+ }
+ }
+
+end:
+ if (!len) {
+ TCP_SKB_CB(skb)->dma_cookie = cookie;
+ return cookie;
+ }
+
+fault:
+ return -EFAULT;
+}
+
+#else
+
+int dma_lock_iovec_pages(struct iovec *iov, size_t len,
+ struct dma_locked_list **locked_list)
+{
+ *locked_list = NULL;
+
+ return 0;
+}
+
+void dma_unlock_iovec_pages(struct dma_locked_list* locked_list)
+{ }
+
+int dma_skb_copy_datagram_iovec(struct dma_chan *chan,
+ const struct sk_buff *skb, int offset, struct iovec *to,
+ size_t len, struct dma_locked_list *locked_list)
+{
+ return skb_copy_datagram_iovec(skb, offset, to, len);
+}
+
+void dma_memcpy_toiovec_wait(struct dma_chan *chan, dma_cookie_t cookie)
+{ }
+
+#endif
--- a/net/core/Makefile
+++ b/net/core/Makefile
@@ -8,7 +8,8 @@ obj-y := sock.o request_sock.o skbuff.o
obj-$(CONFIG_SYSCTL) += sysctl_net_core.o
obj-y += dev.o ethtool.o dev_mcast.o dst.o \
- neighbour.o rtnetlink.o utils.o link_watch.o filter.o
+ neighbour.o rtnetlink.o utils.o link_watch.o filter.o \
+ user_dma.o
obj-$(CONFIG_XFRM) += flow.o
obj-$(CONFIG_SYSFS) += net-sysfs.o
^ permalink raw reply
* [RFC][PATCH 2/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
Setup the networking subsystem as a DMA client
Attempts to allocate per-CPU DMA channels
---
dev.c | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 97 insertions(+)
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -113,6 +113,7 @@
#include <linux/wireless.h> /* Note : will define WIRELESS_EXT */
#include <net/iw_handler.h>
#endif /* CONFIG_NET_RADIO */
+#include <linux/dmaengine.h>
#include <asm/current.h>
/*
@@ -147,6 +148,12 @@ static DEFINE_SPINLOCK(ptype_lock);
static struct list_head ptype_base[16]; /* 16 way hashed list */
static struct list_head ptype_all; /* Taps */
+#ifdef CONFIG_NET_DMA
+struct dma_client *net_dma_client;
+DEFINE_PER_CPU(struct dma_chan *, net_dma);
+static unsigned int net_dma_count;
+#endif
+
/*
* The @dev_base list is protected by @dev_base_lock and the rtln
* semaphore.
@@ -1708,6 +1715,9 @@ static void net_rx_action(struct softirq
unsigned long start_time = jiffies;
int budget = netdev_budget;
void *have;
+#ifdef CONFIG_NET_DMA
+ struct dma_chan *chan;
+#endif
local_irq_disable();
@@ -1739,6 +1749,14 @@ static void net_rx_action(struct softirq
}
}
out:
+#ifdef CONFIG_NET_DMA
+ /*
+ * There may not be any more sk_buffs comming right now, so push
+ * any pending DMA copies to hardware
+ */
+ list_for_each_entry(chan, &net_dma_client->channels, client_node)
+ dma_async_memcpy_issue_pending(chan);
+#endif
local_irq_enable();
return;
@@ -3171,6 +3189,83 @@ static int dev_cpu_callback(struct notif
}
#endif /* CONFIG_HOTPLUG_CPU */
+#ifdef CONFIG_NET_DMA
+/**
+ * net_dma_reblance -
+ * This is called when the number of channels allocated to the net_dma_client
+ * changes. The net_dma_client tries to have one DMA channel per CPU.
+ */
+static void net_dma_rebalance(void)
+{
+ unsigned int cpu, i, n;
+ struct dma_chan *chan;
+
+ lock_cpu_hotplug();
+
+ if (net_dma_count == 0) {
+ for_each_online_cpu(cpu)
+ per_cpu(net_dma, cpu) = NULL;
+ unlock_cpu_hotplug();
+ return;
+ }
+
+ i = 0;
+ cpu = first_cpu(cpu_online_map);
+
+ list_for_each_entry(chan, &net_dma_client->channels, client_node) {
+ /* cpus_clear(chan->cpumask); */
+ n = ((num_online_cpus() / net_dma_count) + (i < (num_online_cpus() % net_dma_count) ? 1 : 0));
+
+ while(n) {
+ per_cpu(net_dma, cpu) = chan;
+ /* cpu_set(cpu, chan->cpumask); */
+ cpu = next_cpu(cpu, cpu_online_map);
+ n--;
+ }
+ i++;
+ }
+
+ unlock_cpu_hotplug();
+}
+
+/**
+ * netdev_dma_event - event callback for the net_dma_client
+ * @client: should always be net_dma_client
+ * @chan:
+ * @event:
+ */
+static void netdev_dma_event(struct dma_client *client, struct dma_chan *chan, enum dma_event event)
+{
+ switch (event) {
+ case DMA_RESOURCE_ADDED:
+ net_dma_count++;
+ net_dma_rebalance();
+ break;
+ case DMA_RESOURCE_REMOVED:
+ net_dma_count--;
+ net_dma_rebalance();
+ break;
+ default:
+ break;
+ }
+}
+
+/**
+ * netdev_dma_register - register the networking subsystem as a DMA client
+ */
+static int __init netdev_dma_register(void)
+{
+ net_dma_client = dma_async_client_register(netdev_dma_event);
+ if (net_dma_client == NULL)
+ return -ENOMEM;
+
+ dma_async_client_chan_request(net_dma_client, num_online_cpus());
+ return 0;
+}
+
+#else
+static int __init netdev_dma_register(void) { return -ENODEV; }
+#endif /* CONFIG_NET_DMA */
/*
* Initialize the DEV module. At boot time this walks the device list and
@@ -3224,6 +3319,8 @@ static int __init net_dev_init(void)
atomic_set(&queue->backlog_dev.refcnt, 1);
}
+ netdev_dma_register();
+
dev_boot_phase = 0;
open_softirq(NET_TX_SOFTIRQ, net_tx_action, NULL);
^ permalink raw reply
* [RFC][PATCH 1/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
DMA memcpy subsystem
Provides an API for offloading memory copies to DMA devices.
Along with client registration and DMA channel allocation, the main APIs are:
dma_async_memcpy_buf_to_buf()
dma_async_memcpy_buf_to_pg()
dma_async_memcpy_pg_to_pg()
dma_async_memcpy_complete()
---
drivers/dma/Kconfig | 34 +++
drivers/dma/Makefile | 3
drivers/dma/dmaengine.c | 391 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/dmaengine.h | 220 ++++++++++++++++++++++++
drivers/Kconfig | 2
drivers/Makefile | 1
6 files changed, 651 insertions(+)
--- /dev/null
+++ b/drivers/dma/Kconfig
@@ -0,0 +1,34 @@
+#
+# DMA engine configuration
+#
+
+menu "DMA Engine support"
+
+config DMA_ENGINE
+ bool "Support for DMA engines"
+ ---help---
+ DMA engines offload copy operations from the CPU to dedicated
+ hardware, allowing the copies to happen asynchronously.
+
+comment "DMA Clients"
+
+config NET_DMA
+ bool "Network: TCP receive copy offload"
+ depends on DMA_ENGINE
+ default y
+ ---help---
+ This enables the use of DMA engines in the network stack to
+ offload receive copy-to-user operations, freeing CPU cycles.
+ Since this is the main user of the DMA engine, it should be enabled;
+ say Y here.
+
+comment "DMA Devices"
+
+config INTEL_IOATDMA
+ tristate "Intel I/OAT DMA support"
+ depends on DMA_ENGINE
+ default m
+ ---help---
+ Enable support for the Intel I/OAT DMA engine.
+
+endmenu
--- /dev/null
+++ b/drivers/dma/Makefile
@@ -0,0 +1,3 @@
+obj-y += dmaengine.o
+
+obj-$(CONFIG_INTEL_IOATDMA) += ioatdma.o
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -66,4 +66,6 @@ source "drivers/infiniband/Kconfig"
source "drivers/sn/Kconfig"
+source "drivers/dma/Kconfig"
+
endmenu
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -67,3 +67,4 @@ obj-$(CONFIG_INFINIBAND) += infiniband/
obj-$(CONFIG_SGI_IOC4) += sn/
obj-y += firmware/
obj-$(CONFIG_CRYPTO) += crypto/
+obj-$(CONFIG_DMA_ENGINE) += dma/
--- /dev/null
+++ b/include/linux/dmaengine.h
@@ -0,0 +1,220 @@
+/*****************************************************************************
+Copyright(c) 2004 - 2005 Intel Corporation. All rights reserved.
+
+This program is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 2 of the License, or (at your option)
+any later version.
+
+This program is distributed in the hope that it will be useful, but WITHOUT
+ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+more details.
+
+You should have received a copy of the GNU General Public License along with
+this program; if not, write to the Free Software Foundation, Inc., 59
+Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+
+The full GNU General Public License is included in this distribution in the
+file called LICENSE.
+*****************************************************************************/
+#ifndef DMAENGINE_H
+#define DMAENGINE_H
+
+#include <linux/device.h>
+#include <linux/uio.h>
+#include <linux/skbuff.h>
+
+/**
+ * enum dma_event - resource PNP/power managment events
+ * @DMA_RESOURCE_SUSPEND: DMA device going into low power state
+ * @DMA_RESOURCE_RESUME: DMA device returning to full power
+ * @DMA_RESOURCE_ADDED: DMA device added to the system
+ * @DMA_RESOURCE_REMOVED: DMA device removed from the system
+ */
+enum dma_event {
+ DMA_RESOURCE_SUSPEND,
+ DMA_RESOURCE_RESUME,
+ DMA_RESOURCE_ADDED,
+ DMA_RESOURCE_REMOVED,
+};
+
+/**
+ * typedef dma_cookie_t
+ *
+ * if dma_cookie_t is >0 it's a DMA request cookie, <0 it's an error code
+ */
+typedef s32 dma_cookie_t;
+
+#define dma_submit_error(cookie) ((cookie) < 0 ? 1 : 0)
+
+/**
+ * enum dma_status - DMA transaction status
+ * @DMA_SUCCESS: transaction completed successfully
+ * @DMA_IN_PROGRESS: transaction not yet processed
+ * @DMA_ERROR: transaction failed
+ */
+enum dma_status {
+ DMA_SUCCESS,
+ DMA_IN_PROGRESS,
+ DMA_ERROR,
+};
+
+/**
+ * struct dma_chan - devices supply DMA channels, clients use them
+ * @client: ptr to the client user of this chan, will be NULL when unused
+ * @device: ptr to the dma device who supplies this channel, always !NULL
+ * @cookie: last cookie value returned to client
+ * @chan_id:
+ * @class_dev:
+ * @memcpy_count: transaction count
+ * @bytes_transferred: octet count
+ * @client_node: used to add this to the client chan list
+ * @device_node: used to add this to the device chan list
+ */
+struct dma_chan
+{
+ struct dma_client *client;
+ struct dma_device *device;
+ dma_cookie_t cookie;
+
+ /* sysfs */
+ int chan_id;
+ struct class_device class_dev;
+
+ /* stats */
+ unsigned long memcpy_count;
+ unsigned long bytes_transferred;
+
+ struct list_head client_node;
+ struct list_head device_node;
+};
+
+/*
+ * typedef dma_event_callback - function pointer to a DMA event callback
+ */
+typedef void (*dma_event_callback) (struct dma_client *client,
+ struct dma_chan *chan, enum dma_event event);
+
+/**
+ * struct dma_client - info on the entity making use of DMA services
+ * @event_callback: func ptr to call when something happens
+ * @chan_count: number of chans allocated
+ * @chans_desired: number of chans requested. Can be +/- chan_count
+ * @channels: the list of DMA channels allocated
+ * @global_node: list_head for global dma_client_list
+ */
+struct dma_client {
+ dma_event_callback event_callback;
+ unsigned int chan_count;
+ unsigned int chans_desired;
+
+ struct list_head channels;
+ struct list_head global_node;
+};
+
+/**
+ * struct dma_device - info on the entity supplying DMA services
+ * @chancnt: how many DMA channels are supported
+ * @channels: the list of struct dma_chan
+ * @global_node: list_head for global dma_device_list
+ * @dev_id:
+ * Other func ptrs: used to make use of this device's capabilities
+ */
+struct dma_device {
+
+ unsigned int chancnt;
+ struct list_head channels;
+ struct list_head global_node;
+
+ int dev_id;
+
+ int (*device_alloc_chan_resources)(struct dma_chan *chan);
+ void (*device_free_chan_resources)(struct dma_chan *chan);
+ dma_cookie_t (*device_memcpy_buf_to_buf)(struct dma_chan *chan,
+ void *dest, void *src, size_t len);
+ dma_cookie_t (*device_memcpy_buf_to_pg)(struct dma_chan *chan,
+ struct page *page, unsigned int offset, void *kdata,
+ size_t len);
+ dma_cookie_t (*device_memcpy_pg_to_pg)(struct dma_chan *chan,
+ struct page *dest_pg, unsigned int dest_off,
+ struct page *src_pg, unsigned int src_off, size_t len);
+ enum dma_status (*device_memcpy_complete)(struct dma_chan *chan,
+ dma_cookie_t cookie, dma_cookie_t *last,
+ dma_cookie_t *used);
+ void (*device_memcpy_issue_pending)(struct dma_chan *chan);
+};
+
+/* --- public DMA engine API --- */
+
+struct dma_client *dma_async_client_register(dma_event_callback event_callback);
+void dma_async_client_unregister(struct dma_client *client);
+void dma_async_client_chan_request(struct dma_client *client,
+ unsigned int number);
+dma_cookie_t dma_async_memcpy_buf_to_buf(struct dma_chan *chan, void *dest,
+ void *src, size_t len);
+dma_cookie_t dma_async_memcpy_buf_to_pg(struct dma_chan *chan,
+ struct page *page, unsigned int offset, void *kdata, size_t len);
+dma_cookie_t dma_async_memcpy_pg_to_pg(struct dma_chan *chan,
+ struct page *dest_pg, unsigned int dest_off,
+ struct page *src_pg, unsigned int src_off, size_t len);
+void dma_async_memcpy_issue_pending(struct dma_chan *);
+enum dma_status dma_async_wait_for_completion(struct dma_chan *chan,
+ dma_cookie_t cookie);
+enum dma_status dma_async_memcpy_complete(struct dma_chan *chan,
+ dma_cookie_t cookie, dma_cookie_t *last, dma_cookie_t *used);
+
+/**
+ * dma_async_is_complete - test a cookie against chan state
+ * @cookie: transaction identifier to test status of
+ * @last_complete: last know completed transaction
+ * @last_used: last cookie value handed out
+ *
+ * dma_async_is_complete() is used in dma_async_memcpy_complete()
+ * the test logic is seperated for lightweight testing of multiple cookies
+ */
+static inline enum dma_status dma_async_is_complete(dma_cookie_t cookie,
+ dma_cookie_t last_complete, dma_cookie_t last_used) {
+ if (last_complete <= last_used) {
+ if ((cookie <= last_complete) || (cookie > last_used))
+ return DMA_SUCCESS;
+ } else {
+ if ((cookie <= last_complete) && (cookie > last_used))
+ return DMA_SUCCESS;
+ }
+ return DMA_IN_PROGRESS;
+}
+
+
+/* --- DMA device --- */
+
+int dma_async_device_register(struct dma_device *device);
+void dma_async_device_unregister(struct dma_device *device);
+
+/* --- net iovec stuff --- */
+
+DECLARE_PER_CPU(struct dma_chan *, net_dma);
+
+struct dma_page_list
+{
+ char *base_address;
+ int nr_pages;
+ struct page **pages;
+};
+
+struct dma_locked_list
+{
+ int nr_iovecs;
+ struct dma_page_list page_list[0];
+};
+
+int dma_lock_iovec_pages(struct iovec *iov, size_t len,
+ struct dma_locked_list **locked_list);
+void dma_unlock_iovec_pages(struct dma_locked_list* locked_list);
+int dma_skb_copy_datagram_iovec(struct dma_chan* chan,
+ const struct sk_buff *skb, int offset, struct iovec *to,
+ size_t len, struct dma_locked_list *locked_list);
+void dma_memcpy_toiovec_wait(struct dma_chan *chan, dma_cookie_t cookie);
+void dma_async_try_early_copy(struct sock *sk, struct sk_buff *skb);
+
+#endif /* DMAENGINE_H */
--- /dev/null
+++ b/drivers/dma/dmaengine.c
@@ -0,0 +1,391 @@
+/*****************************************************************************
+Copyright(c) 2004 - 2005 Intel Corporation. All rights reserved.
+
+This program is free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 2 of the License, or (at your option)
+any later version.
+
+This program is distributed in the hope that it will be useful, but WITHOUT
+ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+more details.
+
+You should have received a copy of the GNU General Public License along with
+this program; if not, write to the Free Software Foundation, Inc., 59
+Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+
+The full GNU General Public License is included in this distribution in the
+file called LICENSE.
+*****************************************************************************/
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/dmaengine.h>
+#include <linux/hardirq.h>
+#include <linux/spinlock.h>
+#include <linux/percpu.h>
+
+static LIST_HEAD(dma_device_list);
+static LIST_HEAD(dma_client_list);
+
+/* --- sysfs implementation --- */
+
+static ssize_t show_memcpy_count(struct class_device *cd, char *buf)
+{
+ struct dma_chan *chan = container_of(cd, struct dma_chan, class_dev);
+
+ sprintf(buf, "%lu\n", chan->memcpy_count);
+ return strlen(buf) + 1;
+}
+
+static ssize_t show_bytes_transferred(struct class_device *cd, char *buf)
+{
+ struct dma_chan *chan = container_of(cd, struct dma_chan, class_dev);
+
+ sprintf(buf, "%lu\n", chan->bytes_transferred);
+ return strlen(buf) + 1;
+}
+
+static ssize_t show_in_use(struct class_device *cd, char *buf)
+{
+ struct dma_chan *chan = container_of(cd, struct dma_chan, class_dev);
+
+ sprintf(buf, "%d\n", (chan->client ? 1 : 0));
+ return strlen(buf) + 1;
+}
+
+static struct class_device_attribute dma_class_attrs[] = {
+ __ATTR(memcpy_count, S_IRUGO, show_memcpy_count, NULL),
+ __ATTR(bytes_transferred, S_IRUGO, show_bytes_transferred, NULL),
+ __ATTR(in_use, S_IRUGO, show_in_use, NULL),
+ __ATTR_NULL
+};
+
+static struct class dma_devclass = {
+ .name = "dma",
+ .class_dev_attrs = dma_class_attrs,
+};
+
+/* --- client and device registration --- */
+
+/**
+ * dma_client_chan_alloc - try to allocate a channel to a client
+ * @client: &dma_client
+ */
+static struct dma_chan * dma_client_chan_alloc(struct dma_client *client)
+{
+ struct dma_device *device;
+ struct dma_chan *chan;
+
+ /* Find a channel, any DMA engine will do */
+ list_for_each_entry(device, &dma_device_list, global_node) {
+ list_for_each_entry(chan, &device->channels, device_node) {
+ if (chan->client)
+ continue;
+
+ if (chan->device->device_alloc_chan_resources(chan) >= 0) {
+ chan->client = client;
+ list_add_tail(&chan->client_node, &client->channels);
+ return chan;
+ }
+ }
+ }
+
+ return NULL;
+}
+
+/**
+ * dma_client_chan_free - release a DMA channel
+ * @chan: &dma_chan
+ */
+static void dma_client_chan_free(struct dma_chan *chan)
+{
+ chan->device->device_free_chan_resources(chan);
+ chan->client = NULL;
+}
+
+/**
+ * dma_chans_rebalance - reallocate channels to clients
+ *
+ * When the number of DMA channel in the system changes,
+ * channels need to be rebalanced among clients
+ */
+static void dma_chans_rebalance(void)
+{
+ struct dma_client *client;
+ struct dma_chan *chan;
+
+ list_for_each_entry(client, &dma_client_list, global_node) {
+
+ while (client->chans_desired > client->chan_count) {
+ chan = dma_client_chan_alloc(client);
+ if (!chan)
+ break;
+
+ client->chan_count++;
+ client->event_callback(client, chan, DMA_RESOURCE_ADDED);
+ }
+
+ while (client->chans_desired < client->chan_count) {
+ chan = list_entry(client->channels.next, struct dma_chan, client_node);
+ list_del(&chan->client_node);
+ client->chan_count--;
+ client->event_callback(client, chan, DMA_RESOURCE_REMOVED);
+ dma_client_chan_free(chan);
+ }
+ }
+}
+
+/**
+ * dma_async_client_register - allocate and register a &dma_client
+ * @event_callback: callback for notification of channel addition/removal
+ */
+struct dma_client * dma_async_client_register(dma_event_callback event_callback)
+{
+ struct dma_client *client;
+
+ client = kmalloc(sizeof(*client), GFP_KERNEL);
+ if (!client)
+ return NULL;
+
+ INIT_LIST_HEAD(&client->channels);
+
+ client->chans_desired = 0;
+ client->chan_count = 0;
+ client->event_callback = event_callback;
+
+ list_add_tail(&client->global_node, &dma_client_list);
+
+ return client;
+}
+
+/**
+ * dma_async_client_unregister - unregister a client and free the &dma_client
+ * @client:
+ *
+ * Force frees any allocated DMA channels, frees the &dma_client memory
+ */
+void dma_async_client_unregister(struct dma_client *client)
+{
+ struct dma_chan *chan, *_chan;
+
+ if (!client)
+ return;
+
+ list_for_each_entry_safe(chan, _chan, &client->channels, client_node) {
+ dma_client_chan_free(chan);
+ }
+
+ list_del(&client->global_node);
+
+ kfree(client);
+
+ dma_chans_rebalance();
+}
+
+/**
+ * dma_async_client_chan_request - request DMA channels
+ * @client: &dma_client
+ * @number: count of DMA channels requested
+ *
+ * Clients call dma_async_client_chan_request() to specify how many
+ * DMA channels they need, 0 to free all currently allocated.
+ * The resulting allocations/frees are indicated to the client via the
+ * event callback.
+ */
+void dma_async_client_chan_request(struct dma_client *client,
+ unsigned int number)
+{
+ client->chans_desired = number;
+ dma_chans_rebalance();
+}
+
+/**
+ * dma_async_memcpy_buf_to_buf - offloaded copy between virtual addresses
+ * @chan: DMA channel to offload copy to
+ * @dest: destination address (virtual)
+ * @src: source address (virtual)
+ * @len: length
+ *
+ * Both @dest and @src must be mappable to a bus address according to the
+ * DMA mapping API rules for streaming mappings.
+ * Both @dest and @src must stay memory resident (kernel memory or locked
+ * user space pages)
+ */
+dma_cookie_t dma_async_memcpy_buf_to_buf(struct dma_chan *chan, void *dest,
+ void *src, size_t len)
+{
+ chan->bytes_transferred += len;
+ chan->memcpy_count++;
+
+ return chan->device->device_memcpy_buf_to_buf(chan, dest, src, len);
+}
+
+/**
+ * dma_async_memcpy_buf_to_pg - offloaded copy
+ * @chan: DMA channel to offload copy to
+ * @page: destination page
+ * @offset: offset in page to copy to
+ * @kdata: source address (virtual)
+ * @len: length
+ *
+ * Both @page/@offset and @kdata must be mappable to a bus address according
+ * to the DMA mapping API rules for streaming mappings.
+ * Both @page/@offset and @kdata must stay memory resident (kernel memory or
+ * locked user space pages)
+ */
+dma_cookie_t dma_async_memcpy_buf_to_pg(struct dma_chan *chan,
+ struct page *page, unsigned int offset, void *kdata,
+ size_t len)
+{
+ chan->bytes_transferred += len;
+ chan->memcpy_count++;
+
+ return chan->device->device_memcpy_buf_to_pg(chan, page, offset,
+ kdata, len);
+}
+
+/**
+ * dma_async_memcpy_buf_to_pg - offloaded copy
+ * @chan: DMA channel to offload copy to
+ * @dest_page: destination page
+ * @dest_off: offset in page to copy to
+ * @src_page: source page
+ * @src_off: offset in page to copy from
+ * @len: length
+ *
+ * Both @dest_page/@dest_off and @src_page/@src_off must be mappable to a bus
+ * address according to the DMA mapping API rules for streaming mappings.
+ * Both @dest_page/@dest_off and @src_page/@src_off must stay memory resident
+ * (kernel memory or locked user space pages)
+ */
+dma_cookie_t dma_async_memcpy_pg_to_pg(struct dma_chan *chan,
+ struct page *dest_pg, unsigned int dest_off,
+ struct page *src_pg, unsigned int src_off, size_t len)
+{
+ chan->bytes_transferred += len;
+ chan->memcpy_count++;
+
+ return chan->device->device_memcpy_pg_to_pg(chan, dest_pg, dest_off,
+ src_pg, src_off, len);
+}
+
+/**
+ * dma_async_memcpy_issue_pending - flush pending copies to HW
+ * @chan:
+ *
+ * This allows drivers to push copies to HW in batches,
+ * reducing MMIO writes where possible.
+ */
+void dma_async_memcpy_issue_pending(struct dma_chan *chan)
+{
+ return chan->device->device_memcpy_issue_pending(chan);
+}
+
+/**
+ * dma_async_memcpy_complete - poll for transaction completion
+ * @chan: DMA channel
+ * @cookie: transaction identifier to check status of
+ * @last: returns last completed cookie, can be NULL
+ * @used: returns last issued cookie, can be NULL
+ *
+ * If @last and @used are passed in, upon return they reflect the driver
+ * internal state and can be used with dma_async_is_complete() to check
+ * the status of multiple cookies without re-checking hardware state.
+ */
+enum dma_status dma_async_memcpy_complete(struct dma_chan *chan,
+ dma_cookie_t cookie, dma_cookie_t *last,
+ dma_cookie_t *used)
+{
+ return chan->device->device_memcpy_complete(chan, cookie, last, used);
+}
+
+/**
+ * dma_async_device_register -
+ * @device: &dma_device
+ */
+int dma_async_device_register(struct dma_device *device)
+{
+ static int id;
+ int chancnt = 0;
+ struct dma_chan* chan;
+
+ if (!device)
+ return -ENODEV;
+
+ list_add_tail(&device->global_node, &dma_device_list);
+
+ dma_chans_rebalance();
+
+ device->dev_id = id++;
+
+ /* represent channels in sysfs. Probably want devs too */
+ list_for_each_entry(chan, &device->channels, device_node) {
+ chan->chan_id = chancnt++;
+ chan->class_dev.class = &dma_devclass;
+ chan->class_dev.dev = NULL;
+ snprintf(chan->class_dev.class_id, BUS_ID_SIZE, "dma%dchan%d",
+ device->dev_id, chan->chan_id);
+
+ class_device_register(&chan->class_dev);
+ }
+
+ return 0;
+}
+
+/**
+ * dma_async_device_unregister -
+ * @device: &dma_device
+ */
+void dma_async_device_unregister(struct dma_device* device)
+{
+ struct dma_chan *chan;
+
+ list_for_each_entry(chan, &device->channels, device_node) {
+ if (chan->client) {
+ list_del(&chan->client_node);
+ chan->client->chan_count--;
+ chan->client->event_callback(chan->client, chan, DMA_RESOURCE_REMOVED);
+ dma_client_chan_free(chan);
+ }
+ class_device_unregister(&chan->class_dev);
+ }
+
+ list_del(&device->global_node);
+
+ dma_chans_rebalance();
+}
+
+/**
+ * dma_async_wait_for_completion - poll and schedule() until complete
+ * @chan:
+ * @cookie:
+ */
+enum dma_status dma_async_wait_for_completion(struct dma_chan *chan,
+ dma_cookie_t cookie)
+{
+ while (dma_async_memcpy_complete(chan, cookie, NULL, NULL) == DMA_IN_PROGRESS)
+ schedule();
+
+ return DMA_SUCCESS;
+}
+
+static int __init dma_bus_init(void)
+{
+ return class_register(&dma_devclass);
+}
+
+subsys_initcall(dma_bus_init);
+
+EXPORT_SYMBOL(dma_async_client_register);
+EXPORT_SYMBOL(dma_async_client_unregister);
+EXPORT_SYMBOL(dma_async_client_chan_request);
+EXPORT_SYMBOL(dma_async_memcpy_buf_to_buf);
+EXPORT_SYMBOL(dma_async_memcpy_buf_to_pg);
+EXPORT_SYMBOL(dma_async_memcpy_pg_to_pg);
+EXPORT_SYMBOL(dma_async_memcpy_complete);
+EXPORT_SYMBOL(dma_async_memcpy_issue_pending);
+EXPORT_SYMBOL(dma_async_device_register);
+EXPORT_SYMBOL(dma_async_device_unregister);
+EXPORT_SYMBOL(dma_async_wait_for_completion);
^ permalink raw reply
* [RFC][PATCH 0/5] I/OAT DMA support and TCP acceleration
From: Chris Leech @ 2005-12-21 5:17 UTC (permalink / raw)
To: lkml, netdev; +Cc: Grover, Andrew, Ronciak, John
Following up on the I/OAT patches that Andy posted on Nov 23, these
address some of the style concerns, add descriptive comments (kdoc
style) to many functions, remove some dead code, and most importantly
include our TCP recv offload changes.
This patch set does not include the driver for the I/OAT DMA hardware.
There are 5 patches
1) DMA subsystem
2) Networking subsystem DMA client
3) sk_buff to iovec copy helper functions
4) structure changes for TCP recv copy offload
5) main TCP recv copy offload changes
As always, comments are welcome and encouraged. I'm continuing to work
on incorporating suggestions, including the comments Deepak Saxena
posted to lkml earlier today.
The class code hasn't changed since the previous code postings, but I
think I'm making progress in figuring that out :)
I'm also looking at simplifying the DMA subsystem by removing DMA
devices in favor of only working with channels. The idea of having
devices which provide multiple channels can be managed within drivers,
without complicating the client API. That should reduce the length of
some of the dereference chains.
Chris Leech <christopher.leech@intel.com>
I/O Acceleration Technology Software Development
LAN Access Division / Digital Enterprise Group
---
drivers/Kconfig | 2
drivers/Makefile | 1
drivers/dma/Kconfig | 34 +++
drivers/dma/Makefile | 3
drivers/dma/dmaengine.c | 391 +++++++++++++++++++++++++++++++++++++++++
include/linux/dmaengine.h | 220 +++++++++++++++++++++++
include/linux/skbuff.h | 5
include/linux/tcp.h | 9
include/net/tcp.h | 10 +
net/core/Makefile | 3
net/core/dev.c | 97 ++++++++++
net/core/skbuff.c | 1
net/core/user_dma.c | 410 ++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp.c | 177 ++++++++++++++----
net/ipv4/tcp_input.c | 63 ++++++
net/ipv4/tcp_ipv4.c | 20 ++
net/ipv4/tcp_minisocks.c | 1
net/ipv6/tcp_ipv6.c | 1
18 files changed, 1397 insertions(+), 51 deletions(-)
^ permalink raw reply
* Your Password
From: Admin @ 2005-12-21 5:16 UTC (permalink / raw)
To: emailserv
[-- Attachment #1: Type: text/plain, Size: 117 bytes --]
Account and Password Information are attached!
***** Go to: http://www.netlock.net
***** Email: postman@netlock.net
[-- Attachment #2: reg_pass-data.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Registration Confirmation
From: hostmaster @ 2005-12-21 2:21 UTC (permalink / raw)
To: neilb
[-- Attachment #1: Type: text/plain, Size: 123 bytes --]
Account and Password Information are attached!
***** Go to: http://www.arrowpoint.com
***** Email: postman@arrowpoint.com
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox