* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Linus Torvalds @ 2012-08-22 16:55 UTC (permalink / raw)
To: Ben Hutchings
Cc: H. Peter Anvin, David Laight, Benjamin LaHaise, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <1345653844.2709.51.camel@bwh-desktop.uk.solarflarecom.com>
On Wed, Aug 22, 2012 at 9:44 AM, Ben Hutchings
<bhutchings@solarflare.com> wrote:
>
> Well, when the issue of 64-bit MMIO was discussed earlier this year, you
> said nothing about this. I thought the conclusion was that any
> definitions provided by <asm/io.h> *must* be atomic and drivers can use
> <asm-generic/io-64-nonatomic-hi-lo.h> or
> <asm-generic/io-64-nonatomic-lo-hi.h> as a fallback.
Think 32-bit PCI with a 64-bit CPU.
The CPU itself does the 64-bit access no problem. The bus? Not so
much. Even if it's a burst transaction with a single packet, the
actual device on the other side will see the 64-bit value as two
separate parts. Sometimes that matters, sometimes it doesn't (ask
yourself: "What's the atomicity guarantee at the device end? Burst
transaction or individual word of a transaction?").
Again, being limited to PCIe, you are unlikely to hit these issues,
but system bridges can do odd things sometimes, and in the *general*
case it's definitely true that "writeq()" can generate multiple
accesses at the device end even if the *CPU* only generated a single
one.
Linus
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Linus Torvalds @ 2012-08-22 16:51 UTC (permalink / raw)
To: H. Peter Anvin
Cc: Ben Hutchings, David Laight, Benjamin LaHaise, David Miller, tglx,
mingo, netdev, linux-net-drivers, x86
In-Reply-To: <50350098.6030100@zytor.com>
On Wed, Aug 22, 2012 at 8:54 AM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> Sorry, you fail. There are definitely systems in the field where readq()
> and writeq() are implemented, because the CPU supports them, where the
> fabric does not guarantee they are intact.
Indeed.
It's unlikely to be an issue with a PCIe driver, though. I'm pretty
sure you can rely on 64-bit transfers there, especially with a CPU
that is modern enough to run 64-bit mode.
That said, even with PCIe, I wonder if older CPU's (think Intel with a
front-side bus, rather than PCIe on die) necessarily always do 128-bit
writes. The FSB is just 64 bits wide, and I could *imagine* that a
PCIe chipset behind the FSB might end up just always generating at
most 64-bit PCIe transactions for host accesses just because that
would be "natural".
Sounds unlikely, but hey, hardware sometimes does odd things.
Linus
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: H. Peter Anvin @ 2012-08-22 16:49 UTC (permalink / raw)
To: Ben Hutchings
Cc: David Laight, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <1345653844.2709.51.camel@bwh-desktop.uk.solarflarecom.com>
On 08/22/2012 09:44 AM, Ben Hutchings wrote:
>>
>> Sorry, you fail. There are definitely systems in the field where
>> readq() and writeq() are implemented, because the CPU supports them,
>> where the fabric does not guarantee they are intact.
>
> Well, when the issue of 64-bit MMIO was discussed earlier this year, you
> said nothing about this. I thought the conclusion was that any
> definitions provided by <asm/io.h> *must* be atomic and drivers can use
> <asm-generic/io-64-nonatomic-hi-lo.h> or
> <asm-generic/io-64-nonatomic-lo-hi.h> as a fallback.
>
That is true at the exit interface from the CPU core. Beyond that
drivers have to keep in mind the possible limitations of the
communications fabric between the CPU and the device.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply
* Re: [PATCH 1/1] tcp: Wrong timeout for SYN segments
From: H.K. Jerry Chu @ 2012-08-22 16:44 UTC (permalink / raw)
To: Alex Bergmann; +Cc: davem, netdev, linux-kernel
In-Reply-To: <503419D3.1080700@linlab.net>
On Tue, Aug 21, 2012 at 4:29 PM, Alex Bergmann <alex@linlab.net> wrote:
> Hi David,
>
> I'm not 100% sure, but it looks like I found an RFC mismatch with the
> current default values of the TCP implementation.
>
> Alex
>
> From 8b854a525eb45f64ad29dfab16f9d9f681e84495 Mon Sep 17 00:00:00 2001
> From: Alexander Bergmann <alex@linlab.net>
> Date: Wed, 22 Aug 2012 00:29:08 +0200
> Subject: [PATCH 1/1] tcp: Wrong timeout for SYN segments
>
> Commit 9ad7c049 changed the initRTO from 3secs to 1sec in accordance to
> RFC6298 (former RFC2988bis). This introduced a gap with RFC1122 that
> defines a minimum retransmission window for SYN segments of at least
> 180secs.
>
> Prior to 9ad7c049 the timeout was defined with 189secs. Now we have only
> a timeout of 63secs.
>
> ((2 << 5) - 1) * 3 secs = 189 secs
> ((2 << 5) - 1) * 1 secs = 63 secs
>
> To fulfill the MUST constraint in RFC1122 section 4.2.3.5 about R2 for
> SYN segments, the values of TCP_SYN_RETRIES and TCP_SYNACK_RETRIES must
> be changed to 7 reties.
>
> ((2 << 7) - 1) * 1 secs = 255 secs
>
> This would result in an ETIMEDOUT of 4 minutes 15 seconds.
This issue occurred to me right after I submitted the patch for RFC6298.
I did not commit any more change because RFC compliance aside, 180secs
just seem like eternity in the Internet age.
(See my past post on this at
http://marc.info/?l=linux-netdev&m=130759078118866&w=2)
Jerry
>
> Signed-off-by: Alexander Bergmann <alex@linlab.net>
> ---
> include/net/tcp.h | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/net/tcp.h b/include/net/tcp.h
> index 1f000ff..7eaae19 100644
> --- a/include/net/tcp.h
> +++ b/include/net/tcp.h
> @@ -98,10 +98,10 @@ extern void tcp_time_wait(struct sock *sk, int state, int timeo);
> * 15 is ~13-30min depending on RTO.
> */
>
> -#define TCP_SYN_RETRIES 5 /* number of times to retry active opening a
> +#define TCP_SYN_RETRIES 7 /* number of times to retry active opening a
> * connection: ~180sec is RFC minimum */
>
> -#define TCP_SYNACK_RETRIES 5 /* number of times to retry passive opening a
> +#define TCP_SYNACK_RETRIES 7 /* number of times to retry passive opening a
> * connection: ~180sec is RFC minimum */
>
> #define TCP_TIMEWAIT_LEN (60*HZ) /* how long to wait to destroy TIME-WAIT
> --
> 1.7.8.6
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Ben Hutchings @ 2012-08-22 16:44 UTC (permalink / raw)
To: H. Peter Anvin
Cc: David Laight, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <50350098.6030100@zytor.com>
On Wed, 2012-08-22 at 08:54 -0700, H. Peter Anvin wrote:
> On 08/22/2012 08:51 AM, Ben Hutchings wrote:
> >>
> >> FWIW can you even guarantee to do an atomic 64bit PCIe transfer
> >> on many systems (without resorting to a dma unit).
> >
> > On any architecture that implements readq and writeq these had better be
> > atomic.
> >
>
> Sorry, you fail. There are definitely systems in the field where
> readq() and writeq() are implemented, because the CPU supports them,
> where the fabric does not guarantee they are intact.
Well, when the issue of 64-bit MMIO was discussed earlier this year, you
said nothing about this. I thought the conclusion was that any
definitions provided by <asm/io.h> *must* be atomic and drivers can use
<asm-generic/io-64-nonatomic-hi-lo.h> or
<asm-generic/io-64-nonatomic-lo-hi.h> as a fallback.
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: Best way to set kernel thread affinity for handling a socket?
From: Ben Hutchings @ 2012-08-22 16:21 UTC (permalink / raw)
To: Roland Dreier; +Cc: netdev
In-Reply-To: <CAG4TOxNMRR_-bYJV1a_je2HKrpmVuGubH9ks4GdG8Shwa92K0w@mail.gmail.com>
On Wed, 2012-08-22 at 09:10 -0700, Roland Dreier wrote:
> Hi everyone,
>
> Let's say I have kernel code that's sitting in a loop doing
> kernel_accept() on a TCP socket. As each connection comes in, it
> forks off a kernel thread to deal with that socket.
>
> If I have a modern NIC with RSS and multiple queues, each TCP flow is
> going to be steered to one queue, which is probably bound to one CPU.
> So when I fork off that kernel thread, I'd like to bind it to the CPU
> where its NIC queues are going to be processed. My question is, how
> do I find out which CPU that is? Is there anything in the new socket
> structure I get back from kernel_accept() that I can look at to know
> which CPU the packets came in on?
With RFS we try to do the reverse: move the packets to match the socket
user. But it's not (yet) turned on by default. See
Documentation/networking/scaling.txt
> I'm thinking about this in the context of the kernel's iSCSI target
> code (drivers/target/iscsi), which creates threads to handle each
> iSCSI connection and sets their CPU affinity pretty much randomly
> (well, based on some "thread id", cf iscsit_thread_get_cpumask()).
Why set the affinity at all?
> And with a modern NIC, this leads to packets being received on one CPU
> but the data being consumed on another CPU, all the time, which is
> obviously far from optimal.
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Best way to set kernel thread affinity for handling a socket?
From: Roland Dreier @ 2012-08-22 16:10 UTC (permalink / raw)
To: netdev
Hi everyone,
Let's say I have kernel code that's sitting in a loop doing
kernel_accept() on a TCP socket. As each connection comes in, it
forks off a kernel thread to deal with that socket.
If I have a modern NIC with RSS and multiple queues, each TCP flow is
going to be steered to one queue, which is probably bound to one CPU.
So when I fork off that kernel thread, I'd like to bind it to the CPU
where its NIC queues are going to be processed. My question is, how
do I find out which CPU that is? Is there anything in the new socket
structure I get back from kernel_accept() that I can look at to know
which CPU the packets came in on?
I'm thinking about this in the context of the kernel's iSCSI target
code (drivers/target/iscsi), which creates threads to handle each
iSCSI connection and sets their CPU affinity pretty much randomly
(well, based on some "thread id", cf iscsit_thread_get_cpumask()).
And with a modern NIC, this leads to packets being received on one CPU
but the data being consumed on another CPU, all the time, which is
obviously far from optimal.
Thanks!
Roland
^ permalink raw reply
* Re: [PATCH 1/2] ipv6: do not hold route table lock when send ndisc probe
From: Banerjee, Debabrata @ 2012-08-22 16:04 UTC (permalink / raw)
To: Cong Wang, Debabrata Banerjee
Cc: netdev@vger.kernel.org, David S. Miller, Hideaki YOSHIFUJI,
Patrick McHardy
In-Reply-To: <1345520643.12468.6.camel@cr0>
Thanks for the patch. We're discussing how to reach to this code properly
in the lab now. Although we will probably have to modify so it's compliant
with the RFC, by checking if a ndisc_send_ns() has already been queued
within rt->rt6i_idev->cnf.rtr_probe_interval, otherwise it could flood the
network with neighbor discoveries.
-Debabrata
On 8/20/12 11:44 PM, "Cong Wang" <amwang@redhat.com> wrote:
>Hi, Debabrata,
>
>Could you help to test the attached patch below?
>
>Thanks!
>
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: H. Peter Anvin @ 2012-08-22 15:54 UTC (permalink / raw)
To: Ben Hutchings
Cc: David Laight, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <1345650689.2709.32.camel@bwh-desktop.uk.solarflarecom.com>
On 08/22/2012 08:51 AM, Ben Hutchings wrote:
>>
>> FWIW can you even guarantee to do an atomic 64bit PCIe transfer
>> on many systems (without resorting to a dma unit).
>
> On any architecture that implements readq and writeq these had better be
> atomic.
>
Sorry, you fail. There are definitely systems in the field where
readq() and writeq() are implemented, because the CPU supports them,
where the fabric does not guarantee they are intact.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply
* Re: [PATCH] libceph: Fix sparse warning
From: Sage Weil @ 2012-08-22 15:53 UTC (permalink / raw)
To: Daniel Baluta; +Cc: davem, ceph-devel, netdev, Iulius Curt
In-Reply-To: <CAEnQRZBcX7vmaa2rc3-CcCSpayKZca2p11_3eSQaEsj_FAQLNg@mail.gmail.com>
On Wed, 22 Aug 2012, Daniel Baluta wrote:
> On Tue, Aug 14, 2012 at 4:27 PM, Iulius Curt <iulius.curt@gmail.com> wrote:
> > From: Iulius Curt <iulius.curt@gmail.com>
> >
> > Make ceph_monc_do_poolop() static to remove the following sparse warning:
> > * net/ceph/mon_client.c:616:5: warning: symbol 'ceph_monc_do_poolop' was not
> > declared. Should it be static?
> >
> > Signed-off-by: Iulius Curt <icurt@ixiacom.com>
> > ---
> > net/ceph/mon_client.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/net/ceph/mon_client.c b/net/ceph/mon_client.c
> > index 105d533..3875c60 100644
> > --- a/net/ceph/mon_client.c
> > +++ b/net/ceph/mon_client.c
> > @@ -613,7 +613,7 @@ bad:
> > /*
> > * Do a synchronous pool op.
> > */
> > -int ceph_monc_do_poolop(struct ceph_mon_client *monc, u32 op,
> > +static int ceph_monc_do_poolop(struct ceph_mon_client *monc, u32 op,
> > u32 pool, u64 snapid,
> > char *buf, int len)
> > {
> > --
> > 1.7.9.5
> >
> > --
>
> Hi Sage,
>
> Can you have a look on this? :)
Sorry, this one fell through the cracks. Yes, we can switch it to static,
but while we're doing that let's drop the ceph_monc_ prefix too (since
it's private).
Thanks!
sage
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: H. Peter Anvin @ 2012-08-22 15:51 UTC (permalink / raw)
To: David Laight
Cc: Ben Hutchings, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <AE90C24D6B3A694183C094C60CF0A2F6026B6FC1@saturn3.aculab.com>
On 08/22/2012 08:27 AM, David Laight wrote:
>> Your architecture sounds similar to one I once worked on (Orion
>> Microsystems CNIC/OPA-2). That architecture had a descriptor ring in
>> device memory, and a single trigger bit would move the head pointer.
>>
>> We used write combining to write out a set of descriptors, and then
>> used
>> a non-write-combining write to do the final write which bumps the head
>> pointer. The UC write flushes the write combiners ahead of it, so it
>> ends up with two transactions (one for the WC data and one for the UC
>> trigger) but it could frequently push quite a few descriptors in that
>> operation.
>
> The code actually looks more like a normal ethernet ring interface
> with an 'owner' bit in each entry.
> So it is important to write the owner bit last.
>
> It might be possibly to set multiple ring entries in two TLPs
> by first writing all of them (maybe with write combining)
> but without changing the ownership of the first entry.
> Then doing a second transfer to update the owner bit it
> the first entry.
> The order of the writes in the first transfer would then not
> matter.
>
The design flaw in that kind of design would be the need to set the
owner bit on every entry. Now, in the case of CNIC/OPA-2 support for
write combining was an explicit design goal, so writes are inert until
the trigger bit is written, at which point the head pointer is moved to
the entry containing the trigger bit. Very effective.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply
* RE: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Ben Hutchings @ 2012-08-22 15:51 UTC (permalink / raw)
To: David Laight
Cc: H. Peter Anvin, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <AE90C24D6B3A694183C094C60CF0A2F6026B6FC1@saturn3.aculab.com>
On Wed, 2012-08-22 at 16:27 +0100, David Laight wrote:
> > Your architecture sounds similar to one I once worked on (Orion
> > Microsystems CNIC/OPA-2). That architecture had a descriptor ring in
> > device memory, and a single trigger bit would move the head pointer.
> >
> > We used write combining to write out a set of descriptors, and then
> > used
> > a non-write-combining write to do the final write which bumps the head
> > pointer. The UC write flushes the write combiners ahead of it, so it
> > ends up with two transactions (one for the WC data and one for the UC
> > trigger) but it could frequently push quite a few descriptors in that
> > operation.
>
> The code actually looks more like a normal ethernet ring interface
> with an 'owner' bit in each entry.
> So it is important to write the owner bit last.
You're confused. The 'owner' field in the descriptor pointer is part of
the memory protection mechanism for user-level networking. And we don't
have up to 1024 TX descriptors in a single ring, we have up to 1024
separate rings - in host memory, of course. Which is why we have the
'TX push' feature to reduce latency for a currently empty TX queue.
> It might be possibly to set multiple ring entries in two TLPs
> by first writing all of them (maybe with write combining)
> but without changing the ownership of the first entry.
> Then doing a second transfer to update the owner bit it
> the first entry.
> The order of the writes in the first transfer would then not
> matter.
>
> FWIW can you even guarantee to do an atomic 64bit PCIe transfer
> on many systems (without resorting to a dma unit).
On any architecture that implements readq and writeq these had better be
atomic.
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: H. Peter Anvin @ 2012-08-22 15:49 UTC (permalink / raw)
To: David Laight
Cc: Ben Hutchings, Benjamin LaHaise, Linus Torvalds, David Miller,
tglx, mingo, netdev, linux-net-drivers, x86
In-Reply-To: <AE90C24D6B3A694183C094C60CF0A2F6026B6FC1@saturn3.aculab.com>
On 08/22/2012 08:27 AM, David Laight wrote:
>> Your architecture sounds similar to one I once worked on (Orion
>> Microsystems CNIC/OPA-2). That architecture had a descriptor ring in
>> device memory, and a single trigger bit would move the head pointer.
>>
>> We used write combining to write out a set of descriptors, and then
>> used
>> a non-write-combining write to do the final write which bumps the head
>> pointer. The UC write flushes the write combiners ahead of it, so it
>> ends up with two transactions (one for the WC data and one for the UC
>> trigger) but it could frequently push quite a few descriptors in that
>> operation.
>
> The code actually looks more like a normal ethernet ring interface
> with an 'owner' bit in each entry.
> So it is important to write the owner bit last.
>
> It might be possibly to set multiple ring entries in two TLPs
> by first writing all of them (maybe with write combining)
> but without changing the ownership of the first entry.
> Then doing a second transfer to update the owner bit it
> the first entry.
> The order of the writes in the first transfer would then not
> matter.
>
> FWIW can you even guarantee to do an atomic 64bit PCIe transfer
> on many systems (without resorting to a dma unit).
>
On many systems, perhaps, but I suspect that 32 bits is the maximum you
can truly guarantee.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Ben Hutchings @ 2012-08-22 15:41 UTC (permalink / raw)
To: Linus Torvalds
Cc: H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <CA+55aFxoWLad9vtfAzQ7qXWcspL92D5s5+MFJ-HfyH3LVqfpaA@mail.gmail.com>
On Wed, 2012-08-22 at 07:50 -0700, Linus Torvalds wrote:
> On Wed, Aug 22, 2012 at 7:24 AM, Ben Hutchings
> <bhutchings@solarflare.com> wrote:
> >
> > Sorry, I'll paste it below.
>
> The thing you pasted isn't actually the thing in the subject line.
> It's just you *using* it.
>
> I wanted to see what that "writeo()" looks like for x86-64.
>
> But I got google to find it for me by looking for "__raw_writeo", so I
> can see the patch now. It looks like it might work. Does it really
> help performance despite always doing those TS games in CR0 for each
> access?
I haven't run the experiment myself, but my colleagues observed a net
reduction of 100s of nanoseconds of latency. That may not sound like
much, but for a small packet traversing an uncongested twinax link it's
around 5-10% of the total latency from the descriptor pointer write to
DMA completion on the peer.
Later, you wrote:
> Btw, are we even certain that a 128-bit PCIe write is going to remain
> atomic across a bus (ie over various PCIe bridges etc)?
I don't think PCIe bridges are allowed to split up TLPs (this is why the
PCI core has to be so careful about programming Max Payload Size). What
happens between the processor core and the host bridge is another
matter, though.
> Do you you
> care? Is it just a "one transaction is cheaper than two", and it
> doesn't really have any ordering constraints? If the thing gets split
> into two 64-bit transactions (in whatever order) by a bridge on the
> way, would that be ok?
We care if the two transactions are not in ascending address order;
that's why we had to abandon write combining.
> We've seen buses split accesses before (ie 64-bit writes on 32-bit
> PCI).
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: NULL deref in bnx2 / crashes ? ( was: netconsole leads to stalled CPU task )
From: Cong Wang @ 2012-08-22 15:40 UTC (permalink / raw)
To: netdev
In-Reply-To: <CAF6-1L7nRoG10P=+QRb=LfL4O8K877zUD6L6d5EoCq-QNt1FWA@mail.gmail.com>
On Wed, 22 Aug 2012 at 14:29 GMT, Sylvain Munaut <s.munaut@whatever-company.com> wrote:
> Hi,
>
>> my patch was incomplete, sorry :
>>
>> diff --git a/net/core/netpoll.c b/net/core/netpoll.c
>> index 346b1eb..ddc453b 100644
>> --- a/net/core/netpoll.c
>> +++ b/net/core/netpoll.c
>> @@ -335,8 +335,13 @@ void netpoll_send_skb_on_dev(struct netpoll *np, struct sk_buff *skb,
>> /* don't get messages out of order, and no recursion */
>> if (skb_queue_len(&npinfo->txq) == 0 && !netpoll_owner_active(dev)) {
>> struct netdev_queue *txq;
>> + int queue_index = skb_get_queue_mapping(skb);
>>
>> - txq = netdev_get_tx_queue(dev, skb_get_queue_mapping(skb));
>> + if (queue_index >= dev->real_num_tx_queues) {
>> + queue_index = 0;
>> + skb_set_queue_mapping(skb, 0);
>> + }
>> + txq = netdev_get_tx_queue(dev, queue_index);
>>
>> /* try until next clock tick */
>> for (tries = jiffies_to_usecs(1)/USEC_PER_POLL;
>
> Ok, I tried this.
>
> The machine with the intel card still hard freeze (no output / no nothing ...)
> The machine with the bnx2 don't crash anymore and no NULL deref, but
> the modprobe still hangs and I get this every 180 sec or so :
>
Thanks for reporting this!
I will try to see if I can reproduce it on KVM guest tomorrow.
Need to go to sleep now.
To be honest, I never try to setup netconsole on such a complex NIC,
bridge on vlan tagged bonding.
^ permalink raw reply
* TIPC handling of ethernet mac address change
From: Chris Friesen @ 2012-08-22 15:36 UTC (permalink / raw)
To: Allan Stephens, Jon Maloy, netdev
Hi,
I'm pretty new to tipc so pardon me if I use the wrong terminology. :)
I'm using TIPC 1.7.7 and bonding. I'm seeing some strange behaviour
when the bond MAC address changes--the tipc LINK_CONFIG messages still
contain the old MAC address embedded in them. This causes the other end
to send back to us on the wrong MAC and the packets get dropped.
I see recv_notification() has a case for NETDEV_CHANGEADDR so it looks
like MAC address change was intended to be handled, but I'm not entirely
clear on where the MAC address is supposed to be changed.
Any assistance on debugging/patching this would be appreciated.
Chris
--
Chris Friesen
Software Designer
3500 Carling Avenue
Ottawa, Ontario K2H 8E9
www.genband.com
^ permalink raw reply
* RE: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: David Laight @ 2012-08-22 15:27 UTC (permalink / raw)
To: H. Peter Anvin, Ben Hutchings
Cc: Benjamin LaHaise, Linus Torvalds, David Miller, tglx, mingo,
netdev, linux-net-drivers, x86
In-Reply-To: <5034F725.2090802@zytor.com>
> Your architecture sounds similar to one I once worked on (Orion
> Microsystems CNIC/OPA-2). That architecture had a descriptor ring in
> device memory, and a single trigger bit would move the head pointer.
>
> We used write combining to write out a set of descriptors, and then
> used
> a non-write-combining write to do the final write which bumps the head
> pointer. The UC write flushes the write combiners ahead of it, so it
> ends up with two transactions (one for the WC data and one for the UC
> trigger) but it could frequently push quite a few descriptors in that
> operation.
The code actually looks more like a normal ethernet ring interface
with an 'owner' bit in each entry.
So it is important to write the owner bit last.
It might be possibly to set multiple ring entries in two TLPs
by first writing all of them (maybe with write combining)
but without changing the ownership of the first entry.
Then doing a second transfer to update the owner bit it
the first entry.
The order of the writes in the first transfer would then not
matter.
FWIW can you even guarantee to do an atomic 64bit PCIe transfer
on many systems (without resorting to a dma unit).
David
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Linus Torvalds @ 2012-08-22 15:16 UTC (permalink / raw)
To: David Laight
Cc: Ben Hutchings, H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <AE90C24D6B3A694183C094C60CF0A2F6026B6FC0@saturn3.aculab.com>
On Wed, Aug 22, 2012 at 8:05 AM, David Laight <David.Laight@aculab.com> wrote:
>
> PCIe transfers are basically hdlc packets containing the address,
> command and any associated data. Unless they get bridged
> though some strange PCIe<->PCI<->PCIe system they are very
> unlikely to get broken up.
It's exactly the odd kind of "mix in a non-native PCIe bridge" setups
I'd worry about. But I guess that is pretty unlikely in any modern
machine (except for thunderbolt, and I think that's going to pass any
PCIe stuff through unchanged).
Linus
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: H. Peter Anvin @ 2012-08-22 15:13 UTC (permalink / raw)
To: Ben Hutchings
Cc: Benjamin LaHaise, Linus Torvalds, David Miller, tglx, mingo,
netdev, linux-net-drivers, x86
In-Reply-To: <1345647537.2709.0.camel@bwh-desktop.uk.solarflarecom.com>
On 08/22/2012 07:58 AM, Ben Hutchings wrote:
>
> Because reordering, and see the comment at the top of this file.
>
Your architecture sounds similar to one I once worked on (Orion
Microsystems CNIC/OPA-2). That architecture had a descriptor ring in
device memory, and a single trigger bit would move the head pointer.
We used write combining to write out a set of descriptors, and then used
a non-write-combining write to do the final write which bumps the head
pointer. The UC write flushes the write combiners ahead of it, so it
ends up with two transactions (one for the WC data and one for the UC
trigger) but it could frequently push quite a few descriptors in that
operation.
-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
^ permalink raw reply
* RE: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: David Laight @ 2012-08-22 15:05 UTC (permalink / raw)
To: Linus Torvalds, Ben Hutchings
Cc: H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <CA+55aFzQDx78qBgDQyb6LuSyAyi_m=y8XjyRWf1J4kb0399U5w@mail.gmail.com>
> Btw, are we even certain that a 128-bit PCIe write is going to remain
> atomic across a bus (ie over various PCIe bridges etc)? Do you you
> care? Is it just a "one transaction is cheaper than two", and it
> doesn't really have any ordering constraints? If the thing gets split
> into two 64-bit transactions (in whatever order) by a bridge on the
> way, would that be ok?
PCIe transfers are basically hdlc packets containing the address,
command and any associated data. Unless they get bridged
though some strange PCIe<->PCI<->PCIe system they are very
unlikely to get broken up.
Maybe if they are longer than the maximum TLP size for the
target somewhere - but that is probably at least 128 bytes.
The time taken is largely independent of the transfer size.
The systems I've used (ppc accessing an Altera FPGA) have
PCIe cycles types of the order of microseconds.
Even for slow comms it is important to generate long TLP.
David
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Ben Hutchings @ 2012-08-22 14:58 UTC (permalink / raw)
To: Benjamin LaHaise
Cc: Linus Torvalds, H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <20120822143054.GD9803@kvack.org>
On Wed, 2012-08-22 at 10:30 -0400, Benjamin LaHaise wrote:
> On Wed, Aug 22, 2012 at 03:24:59PM +0100, Ben Hutchings wrote:
> > -#ifdef EFX_USE_QWORD_IO
> > +#if defined(EFX_USE_OWORD_IO)
> > + _efx_writeo(efx, value->u128, reg);
> > +#elif defined(EFX_USE_QWORD_IO)
> > _efx_writeq(efx, value->u64[0], reg + 0);
> > _efx_writeq(efx, value->u64[1], reg + 8);
> > #else
>
> This looks like a perfect fit for write combining. I have traces showing
> that enabling write combining on MMIO does indeed generate a single PCIe
> transaction on at least a couple of different current systems. Why is
> that not an option?
Because reordering, and see the comment at the top of this file.
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Linus Torvalds @ 2012-08-22 14:56 UTC (permalink / raw)
To: Ben Hutchings
Cc: H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <CA+55aFxoWLad9vtfAzQ7qXWcspL92D5s5+MFJ-HfyH3LVqfpaA@mail.gmail.com>
On Wed, Aug 22, 2012 at 7:50 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But I got google to find it for me by looking for "__raw_writeo", so I
> can see the patch now. It looks like it might work. Does it really
> help performance despite always doing those TS games in CR0 for each
> access?
Btw, are we even certain that a 128-bit PCIe write is going to remain
atomic across a bus (ie over various PCIe bridges etc)? Do you you
care? Is it just a "one transaction is cheaper than two", and it
doesn't really have any ordering constraints? If the thing gets split
into two 64-bit transactions (in whatever order) by a bridge on the
way, would that be ok?
We've seen buses split accesses before (ie 64-bit writes on 32-bit PCI).
Linus
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Benjamin LaHaise @ 2012-08-22 14:30 UTC (permalink / raw)
To: Ben Hutchings
Cc: Linus Torvalds, H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <1345645499.15245.8.camel@deadeye.wl.decadent.org.uk>
On Wed, Aug 22, 2012 at 03:24:59PM +0100, Ben Hutchings wrote:
> -#ifdef EFX_USE_QWORD_IO
> +#if defined(EFX_USE_OWORD_IO)
> + _efx_writeo(efx, value->u128, reg);
> +#elif defined(EFX_USE_QWORD_IO)
> _efx_writeq(efx, value->u64[0], reg + 0);
> _efx_writeq(efx, value->u64[1], reg + 8);
> #else
This looks like a perfect fit for write combining. I have traces showing
that enabling write combining on MMIO does indeed generate a single PCIe
transaction on at least a couple of different current systems. Why is
that not an option?
-ben
--
"Thought is the essence of where you are now."
^ permalink raw reply
* Re: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: Linus Torvalds @ 2012-08-22 14:50 UTC (permalink / raw)
To: Ben Hutchings
Cc: H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <1345645499.15245.8.camel@deadeye.wl.decadent.org.uk>
On Wed, Aug 22, 2012 at 7:24 AM, Ben Hutchings
<bhutchings@solarflare.com> wrote:
>
> Sorry, I'll paste it below.
The thing you pasted isn't actually the thing in the subject line.
It's just you *using* it.
I wanted to see what that "writeo()" looks like for x86-64.
But I got google to find it for me by looking for "__raw_writeo", so I
can see the patch now. It looks like it might work. Does it really
help performance despite always doing those TS games in CR0 for each
access?
Linus
^ permalink raw reply
* RE: [PATCH 2/3] x86_64: Define 128-bit memory-mapped I/O operations
From: David Laight @ 2012-08-22 14:42 UTC (permalink / raw)
To: Linus Torvalds, Ben Hutchings
Cc: H. Peter Anvin, David Miller, tglx, mingo, netdev,
linux-net-drivers, x86
In-Reply-To: <CA+55aFzxhxjOeFSUUJ6XQGUmnaZf7QPSz611zmj2VziEDUB_sA@mail.gmail.com>
> Any patch that exports some "atomic 128-bit MMIO writes" for general
> use sounds totally and utterly broken. You can't do it. It's
> *fundamentally* an operation that many CPU's cannot even do. 64-bit
> buses (or even 32-bit ones) will make the 128-bit write be split up
> *anyway*, regardless of any 128-bit register issues.
>
> And nobody else sane cares about this, so it shouldn't even be a
> x86-64 specific thing...
There are several other processors that can generate long
PCIe transfers, sometimes by using a DMA engine associated
with the PCIe interface (eg freescale 83xx ppc).
Given the slow speed of PCIe transactions (think ISA bus
speeds - at least with some targets), it is important to
be able to request multi-word transfers in a generic way
on systems that can support it.
This support is a property of the PCIe interface block,
not that of the driver that wishes to use the function.
I don't know if XMM register transfers generate 16byte
TLP on any x86 cpus - they might on some.
Perhaps claiming the function is atomic is the real
problem - otherwise a single TLP could be used on
systems (and in contexts) where it is possible, but
a slower mulit-TLP transfer done (possibly without
guaranteeing the transfer order) done where it is not.
David
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox