[PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

This patchset introduces preemption checks for __this_cpu operations.

First we add new raw_cpu operations that perform this cpu operations
without preempt checks in the future.

Then those raw_cpu operations are used in a number of locations to avoid
false positives.

The last patch then adds the preemption checks by modifying the
__this_cpu macros in include/linux/percpu.h

V3->v4:
	- Drop CONFIG_DEBUG_THIS_CPU_OPERATIONS
	- Detail operation triggering the log entry.
	- Use quilt 0.60/ [PATCH] prefix.
	- Clean up subject lines.
	- Include raw_cpu_ops conversion/fixes that were tested on
	  a Ubuntu 13.04 desktop.
	- Traces

V2->V3:
	- Subject line in the raw_cpu_ops patch had ; instead of :.
	  Guess I am getting old.
	- Improve descriptions and variable names.
	- Run tests again with kvm to verify that it still works.
		A) No warnings with just the patches applied
		B) Lots of warnings with CONFIG_DEBUG_THIS_CPU_OPERATIONS enabled
		C) No warnings with 3 core patches applied that simply convert
			__this_cpu operations to raw_cpu_ops.

V1->V2:
	- Reuse preemption check logic in lib/smp_processor_id.c
	- Add CONFIG_DEBUG_THIS_CPU_OPERATIONS
	- Remove conversions to the use of raw_cpu_ops since
	  these may require some discussion first.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[David Miller]

Most sites are rejecting your posting because the CC: list looked like this:

Cc:	akpm@linuxfoundation.org
Cc:	Steven Rostedt <srostedt@redhat.com>
Cc:	linux-kernel@vger.kernel.org
Cc:	Ingo Molnar <mingo@kernel.org>
Cc:	Peter Zijlstra <peterz@infradead.org>
Cc:	Thomas Gleixner <tglx@linutronix.de>

Please only specify one CC: list with a comma separated list of recipients.

Thanks.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Ingo Molnar]

* David Miller <davem@davemloft.net> wrote:

> Most sites are rejecting your posting because the CC: list looked like 
> this:
> 
> Cc:	akpm@linuxfoundation.org
> Cc:	Steven Rostedt <srostedt@redhat.com>
> Cc:	linux-kernel@vger.kernel.org
> Cc:	Ingo Molnar <mingo@kernel.org>
> Cc:	Peter Zijlstra <peterz@infradead.org>
> Cc:	Thomas Gleixner <tglx@linutronix.de>
> 
> Please only specify one CC: list with a comma separated list of 
> recipients.

Another problem is that the patch emails are not properly threaded to the 
0/6 patch and thus appear out of order and mixed up:

66216 N C Oct 11 Christoph Lamet (  36) [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4
66217 N C Oct 11 David Miller    (  13) └─>
66218 N C Oct 11 Christoph Lamet (  43) ┬─>[PATCH 1/6] net: ip4_datagram_connect: Use correct form of statistics update
66219 N C Oct 11 Eric Dumazet    (  17) │ └─>
66220 N C Oct 11 Christoph Lamet ( 121) ├─>[PATCH 2/6] percpu: Add raw_cpu_ops
66221 N C Oct 11 Christoph Lamet ( 189) ├─>[PATCH 6/6] percpu: Add preemption checks to __this_cpu ops
66222 N C Oct 11 Christoph Lamet (  64) ├─>[PATCH 5/6] net: __this_cpu_inc in route.c
66223 N C Oct 11 Christoph Lamet ( 103) ├─>[PATCH 3/6] mm: Use raw_cpu ops for determining current NUMA node
66224 N C Oct 11 Christoph Lamet (  43) └─>[PATCH 4/6] Use raw_cpu_write for initialization of per cpu refcount.

Note how the order is 1,2,6,5,3,4 with no threading instead of 1,2,3,4,5,6 
with proper threading.

That won't cause email servers to reject the mails, it just makes the 
patches a bit harder to review.

Most kernel developers tend to use 'git send-email' to send patches to 
lkml, and that method is working pretty reliably.

The relevant 'git send-email' line is pretty well described in Greg's "The 
newbie's guide to hacking the Linux kernel" book:

  http://www.tuxradar.com/content/newbies-guide-hacking-linux-kernel

Thanks,

	Ingo

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Peter Zijlstra]

On Sat, Oct 12, 2013 at 06:51:20PM +0200, Ingo Molnar wrote:
> Another problem is that the patch emails are not properly threaded to the 
> 0/6 patch and thus appear out of order and mixed up:
> 
> 66216 N C Oct 11 Christoph Lamet (  36) [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4
> 66217 N C Oct 11 David Miller    (  13) └─>
> 66218 N C Oct 11 Christoph Lamet (  43) ┬─>[PATCH 1/6] net: ip4_datagram_connect: Use correct form of statistics update
> 66219 N C Oct 11 Eric Dumazet    (  17) │ └─>
> 66220 N C Oct 11 Christoph Lamet ( 121) ├─>[PATCH 2/6] percpu: Add raw_cpu_ops
> 66221 N C Oct 11 Christoph Lamet ( 189) ├─>[PATCH 6/6] percpu: Add preemption checks to __this_cpu ops
> 66222 N C Oct 11 Christoph Lamet (  64) ├─>[PATCH 5/6] net: __this_cpu_inc in route.c
> 66223 N C Oct 11 Christoph Lamet ( 103) ├─>[PATCH 3/6] mm: Use raw_cpu ops for determining current NUMA node
> 66224 N C Oct 11 Christoph Lamet (  43) └─>[PATCH 4/6] Use raw_cpu_write for initialization of per cpu refcount.
> 
> Note how the order is 1,2,6,5,3,4 with no threading instead of 1,2,3,4,5,6 
> with proper threading.
> 
> That won't cause email servers to reject the mails, it just makes the 
> patches a bit harder to review.
> 
> Most kernel developers tend to use 'git send-email' to send patches to 
> lkml, and that method is working pretty reliably.

I always use quilt mail, it typically adds a reference to the 0/0 email
and ensures the time of each msg is one second ahead of the previous one
to ensure order is correct.

I've no idea how Christoph managed to wreck this, but normally quilt
mail DTRT (as opposed to git send-email which used to default to
endlessly deep threads -- got fixed in 1.6 or 1.7 or thereabouts).

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

On Mon, 14 Oct 2013, Peter Zijlstra wrote:

> > Most kernel developers tend to use 'git send-email' to send patches to
> > lkml, and that method is working pretty reliably.
>
> I always use quilt mail, it typically adds a reference to the 0/0 email
> and ensures the time of each msg is one second ahead of the previous one
> to ensure order is correct.

Yes that I also what I expected to happen and what always happened in the
past.

> I've no idea how Christoph managed to wreck this, but normally quilt
> mail DTRT (as opposed to git send-email which used to default to
> endlessly deep threads -- got fixed in 1.6 or 1.7 or thereabouts).

Well this is going through Amazon cloud email which may be wrecking
things. Had a number of issues already with messages being delayed etc. I
can try to revert to my home setup and see if that fixes things.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Peter Zijlstra]

On Mon, Oct 14, 2013 at 07:02:29PM +0000, Christoph Lameter wrote:
> On Mon, 14 Oct 2013, Peter Zijlstra wrote:
> 
> > > Most kernel developers tend to use 'git send-email' to send patches to
> > > lkml, and that method is working pretty reliably.
> >
> > I always use quilt mail, it typically adds a reference to the 0/0 email
> > and ensures the time of each msg is one second ahead of the previous one
> > to ensure order is correct.
> 
> Yes that I also what I expected to happen and what always happened in the
> past.
> 
> > I've no idea how Christoph managed to wreck this, but normally quilt
> > mail DTRT (as opposed to git send-email which used to default to
> > endlessly deep threads -- got fixed in 1.6 or 1.7 or thereabouts).
> 
> Well this is going through Amazon cloud email which may be wrecking
> things. Had a number of issues already with messages being delayed etc. I
> can try to revert to my home setup and see if that fixes things.

So your 0/x has:

Message-ID: <00000141a8a7ea27-4ab5f822-e15f-4789-a5cf-f313c402a901-000000@email.amazonses.com>
Date: Fri, 11 Oct 2013 17:54:38 +0000

1/x has:

Message-Id: <00000141a8a7eb8a-370733b2-d230-4567-96e1-74c028998bff-000000@email.amazonses.com>
References: <20131011175518.634285474@linux.com>
Date: Fri, 11 Oct 2013 17:54:39 +0000

2/x has:

Message-Id: <00000141a8a7ecfe-16f29a32-36eb-4588-a87e-6ba8e0b3a91c-000000@email.amazonses.com>
References: <20131011175518.634285474@linux.com>
Date: Fri, 11 Oct 2013 17:54:39 +0000

3/x has:

Message-Id: <00000141a8a7fa10-998dfc65-4d07-4d83-999e-31cb0056cfb3-000000@email.amazonses.com>
References: <20131011175518.634285474@linux.com>
Date: Fri, 11 Oct 2013 17:54:42 +0000


So yes, Amazon is fucking with your email, not only the msgids
(obviously) but also the date stamps are not what quilt generates.

If someone from Amazon is reading this; please comment.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2018 bytes --]

On Sat, 12 Oct 2013, Ingo Molnar wrote:

> Another problem is that the patch emails are not properly threaded to the
> 0/6 patch and thus appear out of order and mixed up:
>
> 66216 N C Oct 11 Christoph Lamet (  36) [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4
> 66217 N C Oct 11 David Miller    (  13) О©╫О©╫>
> 66218 N C Oct 11 Christoph Lamet (  43) О©╫О©╫>[PATCH 1/6] net: ip4_datagram_connect: Use correct form of statistics update
> 66219 N C Oct 11 Eric Dumazet    (  17) О©╫ О©╫О©╫>
> 66220 N C Oct 11 Christoph Lamet ( 121) О©╫О©╫>[PATCH 2/6] percpu: Add raw_cpu_ops
> 66221 N C Oct 11 Christoph Lamet ( 189) О©╫О©╫>[PATCH 6/6] percpu: Add preemption checks to __this_cpu ops
> 66222 N C Oct 11 Christoph Lamet (  64) О©╫О©╫>[PATCH 5/6] net: __this_cpu_inc in route.c
> 66223 N C Oct 11 Christoph Lamet ( 103) О©╫О©╫>[PATCH 3/6] mm: Use raw_cpu ops for determining current NUMA node
> 66224 N C Oct 11 Christoph Lamet (  43) О©╫О©╫>[PATCH 4/6] Use raw_cpu_write for initialization of per cpu refcount.
>
> Note how the order is 1,2,6,5,3,4 with no threading instead of 1,2,3,4,5,6
> with proper threading.

Threading is done by quilt and it has been doing that for a pretty long time.

> That won't cause email servers to reject the mails, it just makes the
> patches a bit harder to review.

I do not have any control over how my ISP sorts these emails. You
repeatedly asked me to use quilt 0.60 because I guess you believed that it
could do some magic. I upgraded specially for you. I really do not know
what else to do to appease you. I could chance ISPs hoping that another
will deliver them in sequence but that is not easy to do. Maybe Amazon in
some way borks the headers.

> Most kernel developers tend to use 'git send-email' to send patches to
> lkml, and that method is working pretty reliably.

People are not allowed to use quilt for patches submitted to you?

I just checked and git send-mail does the threading in the same way as
quilt. There would be no change.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Ingo Molnar]

* Christoph Lameter <cl@linux.com> wrote:

> On Sat, 12 Oct 2013, Ingo Molnar wrote:
> 
> > Another problem is that the patch emails are not properly threaded to the
> > 0/6 patch and thus appear out of order and mixed up:
> >
> > 66216 N C Oct 11 Christoph Lamet (  36) [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4
> > 66217 N C Oct 11 David Miller    (  13) О©╫О©╫>
> > 66218 N C Oct 11 Christoph Lamet (  43) О©╫О©╫>[PATCH 1/6] net: ip4_datagram_connect: Use correct form of statistics update
> > 66219 N C Oct 11 Eric Dumazet    (  17) О©╫ О©╫О©╫>
> > 66220 N C Oct 11 Christoph Lamet ( 121) О©╫О©╫>[PATCH 2/6] percpu: Add raw_cpu_ops
> > 66221 N C Oct 11 Christoph Lamet ( 189) О©╫О©╫>[PATCH 6/6] percpu: Add preemption checks to __this_cpu ops
> > 66222 N C Oct 11 Christoph Lamet (  64) О©╫О©╫>[PATCH 5/6] net: __this_cpu_inc in route.c
> > 66223 N C Oct 11 Christoph Lamet ( 103) О©╫О©╫>[PATCH 3/6] mm: Use raw_cpu ops for determining current NUMA node
> > 66224 N C Oct 11 Christoph Lamet (  43) О©╫О©╫>[PATCH 4/6] Use raw_cpu_write for initialization of per cpu refcount.
> >
> > Note how the order is 1,2,6,5,3,4 with no threading instead of 1,2,3,4,5,6
> > with proper threading.
> 
> Threading is done by quilt and it has been doing that for a pretty long time.

The point is that it's not done properly, and hasn't been done properly by 
your past few submissions.

That kind of mess increase the cost of reviewing and processing your 
patches and that is why there are rules and suggestions about how patches 
should be submitted to lkml.

> > That won't cause email servers to reject the mails, it just makes the 
> > patches a bit harder to review.
> 
> I do not have any control over how my ISP sorts these emails. [...]

If you are subscribed to linux-kernel you can double check the mails as 
they arrive.

You can also do test-sends to yourself or a test email address on gmail 
without Cc:-ing lkml, to make sure it's all sense.

> > Most kernel developers tend to use 'git send-email' to send patches to 
> > lkml, and that method is working pretty reliably.
> 
> People are not allowed to use quilt for patches submitted to you?

PeterZ is using Quilt to submit patches and his submissions are perfect.

My only requirement is that the submissions should not be broken and messy 
and your submissions repeatedly failed on that regard.

I have no idea what is broken about your email setup, you'll have to debug 
it yourself by doing a couple of test submissions without Cc:-ing others.

I test email sending scripts every time I change tooling - it's not rocket 
science and you should stop trying to blame me for your broken tool chain.

> I just checked and git send-mail does the threading in the same way as 
> quilt. There would be no change.

Yet the threading in your submissions is broken.

The thing is, I don't really care with what tools and methods you solve 
the problem - the vast majority of kernel developers who are submitting 
patch series are able to solve it.

The important thing is that it is _your_ responsibility to make sure the 
end result is sane.

Thanks,

	Ingo

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Ingo Molnar]

* Christoph Lameter <cl@linux.com> wrote:

> > That won't cause email servers to reject the mails, it just makes the 
> > patches a bit harder to review.
> 
> I do not have any control over how my ISP sorts these emails. [...]

Your series from today has proper threading - so after all it _was_ 
possible for you to send a patch series correctly?

Note that you did not address the other problem that David Miller pointed 
out, the Cc list still looks messed up:

Message-Id: <20131015180050.084352174@linux.com>
User-Agent: quilt/0.60-1
Date: Tue, 15 Oct 2013 12:58:53 -0500
From: Christoph Lameter <cl@linux.com>
To: Tejun Heo <tj@kernel.org>
Cc: akpm@linuxfoundation.org, "David S. Miller" <davem@davemloft.net>
Cc: linux-arch@vger.kernel.org
Cc: Steven Rostedt <srostedt@redhat.com>
Cc: linux-kernel@vger.kernel.org
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Subject: [PATCH 01/10] net: Replace __get_cpu_var uses

Thanks,

	Ingo

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

On Wed, 16 Oct 2013, Ingo Molnar wrote:

> > I do not have any control over how my ISP sorts these emails. [...]
>
> Your series from today has proper threading - so after all it _was_
> possible for you to send a patch series correctly?

Well yes with some effort I was able to get funnel this stuff through my
home connection. Problem is that this is a hassle to do. I am going to
have limited availability in the next couple of weeks due to conferences
and vacation. Then I guess I need to see if I can relocate my development
system elsewhere.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Steven Rostedt]

On Fri, 2013-10-11 at 14:08 -0400, David Miller wrote:
> 
> Most sites are rejecting your posting because the CC: list looked like this:
> 
> Cc:     akpm@linuxfoundation.org
> Cc:     Steven Rostedt <srostedt@redhat.com>

Also, please remove my Red Hat email address from your address book and
replace it with rostedt@goodmis.org instead.

I try not to do upstream work from my Red Hat account, and use that only
for Red Hat internal issues.

Thanks,

-- Steve



> Cc:     linux-kernel@vger.kernel.org
> Cc:     Ingo Molnar <mingo@kernel.org>
> Cc:     Peter Zijlstra <peterz@infradead.org>
> Cc:     Thomas Gleixner <tglx@linutronix.de>
> 
> Please only specify one CC: list with a comma separated list of recipients.
> 
> Thanks.

Re: [PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

On Mon, 14 Oct 2013, Steven Rostedt wrote:

> On Fri, 2013-10-11 at 14:08 -0400, David Miller wrote:
> >
> > Most sites are rejecting your posting because the CC: list looked like this:
> >
> > Cc:     akpm@linuxfoundation.org
> > Cc:     Steven Rostedt <srostedt@redhat.com>
>
> Also, please remove my Red Hat email address from your address book and
> replace it with rostedt@goodmis.org instead.
>
> I try not to do upstream work from my Red Hat account, and use that only
> for Red Hat internal issues.

Ok.

[PATCH 0/6] percpu: Implement Preemption checks for __this_cpu operations V4

[Christoph Lameter]

This patchset introduces preemption checks for __this_cpu operations.

First we add new raw_cpu operations that perform this cpu operations
without preempt checks in the future.

Then those raw_cpu operations are used in a number of locations to avoid
false positives.

The last patch then adds the preemption checks by modifying the
__this_cpu macros in include/linux/percpu.h

V3->v4:
	- Drop CONFIG_DEBUG_THIS_CPU_OPERATIONS
	- Detail operation triggering the log entry.
	- Use quilt 0.60/ [PATCH] prefix.
	- Clean up subject lines.
	- Include raw_cpu_ops conversion/fixes that were tested on
	  a Ubuntu 13.04 desktop.
	- Traces

V2->V3:
	- Subject line in the raw_cpu_ops patch had ; instead of :.
	  Guess I am getting old.
	- Improve descriptions and variable names.
	- Run tests again with kvm to verify that it still works.
		A) No warnings with just the patches applied
		B) Lots of warnings with CONFIG_DEBUG_THIS_CPU_OPERATIONS enabled
		C) No warnings with 3 core patches applied that simply convert
			__this_cpu operations to raw_cpu_ops.

V1->V2:
	- Reuse preemption check logic in lib/smp_processor_id.c
	- Add CONFIG_DEBUG_THIS_CPU_OPERATIONS
	- Remove conversions to the use of raw_cpu_ops since
	  these may require some discussion first.