[CFP] RT Users Micro-Conf at Linux Plumbers 2017

[CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Steven Rostedt]

[ Call For Participation ]

Hi folks,

Linux Plumbers Conference (LPC) will be held this year in Los Angeles
California on September 13-15, 2017. Usually we have a Real-Time
Micro-Conference but this year Thomas Gleixner may not be able to
attend. Instead of having a kernel development focused event on the
PREEMPT_RT patch, I would like to throw out the idea of a user
experience focused event.

A month later there will be a Real-Time get together in Prague, that
would be perfect to be able to bring together a list of issues that
users may have and present this to the PREEMPT_RT kernel developers.

Are people willing to participate and have a discussion on what works
and what doesn't work with PREEMPT_RT. Be able to share horror stories
and work arounds with each other and perhaps come up with better
solutions. I will take what comes out of this with me to Prague and
perhaps be able to influence the development of PREEMPT_RT to address
any issues that arise.

There's also talk about having this combined with a Safety Critical
session focused on PREEMPT_RT.

Are people interesting in attending such an event? If so, I'll submit a
proposal for having such a Micro-Conference at Linux Plumbers. If not,
I wont bother wasting any time on it.

Thanks!

-- Steve

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Julia Cartwright]

[-- Attachment #1: Type: text/plain, Size: 3062 bytes --]

On Mon, Feb 27, 2017 at 05:25:56PM -0500, Steven Rostedt wrote:
> Hi folks,
>
> Linux Plumbers Conference (LPC) will be held this year in Los Angeles
> California on September 13-15, 2017. Usually we have a Real-Time
> Micro-Conference but this year Thomas Gleixner may not be able to
> attend. Instead of having a kernel development focused event on the
> PREEMPT_RT patch, I would like to throw out the idea of a user
> experience focused event.

I think this is a great idea.  I know that in the past there have been
PREEMPT_RT workshops at RTLWS, but they stop at the "patch and boot an
-rt kernel, see cyclictest numbers change".  And, while these workshops
are useful for absolute beginners, there is a whole bunch of other
"stuff" that needs to happen between the initial integration and
shipping a final kernel in a production setting.

> A month later there will be a Real-Time get together in Prague, that
> would be perfect to be able to bring together a list of issues that
> users may have and present this to the PREEMPT_RT kernel developers.

Having LPC host this seems a bit strange, given your defined audience
(users vs. kernel developers).  Perhaps it would be more effective as a
session in the Open Source Summit track, or at the very least, if
attendees at the OSS Summit were able to attend the miniconference w/o
the purchase of a LPC badge.

> Are people willing to participate and have a discussion on what works
> and what doesn't work with PREEMPT_RT. Be able to share horror stories
> and work arounds with each other and perhaps come up with better
> solutions. I will take what comes out of this with me to Prague and
> perhaps be able to influence the development of PREEMPT_RT to address
> any issues that arise.

As a brainstorming of potential content, here's what I'd like to see as
an attendee (certainly willing to present where appropriate too!):

  1. Debugging case studies:

There are a ton of debugging/visualizing/tracing tools out there
available for finding latency problems.  Too many tools; some with
overlapping purpose, and some better than others.  It would be an
interesting exercise to have a few folks demonstrate the complete
identification, debugging, and fix of a performance problem on RT,
complete with a walkthrough of what tools were used.

  2. What to consider when designing the priority scheme of your system
     (application threads, irqthreads, RCU threads, ...)

  3. An enumeration of "knobs" which impact RT behavior

Where knobs are any of the compile-time or run-time configuration
parameters and how they impact determinism (I'm thinking things we've
run into like IRQSOFF tracing, timerslack, the bizarre
/dev/cpu_dma_latency, etc).

> There's also talk about having this combined with a Safety Critical
> session focused on PREEMPT_RT.
>
> Are people interesting in attending such an event? If so, I'll submit a
> proposal for having such a Micro-Conference at Linux Plumbers. If not,
> I wont bother wasting any time on it.

Sounds good to me!  Let me know where I can help.

   Julia

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Steven Rostedt]

On Tue, 28 Feb 2017 14:41:21 -0600
Julia Cartwright <julia@ni.com> wrote:


> > A month later there will be a Real-Time get together in Prague, that
> > would be perfect to be able to bring together a list of issues that
> > users may have and present this to the PREEMPT_RT kernel developers.  
> 
> Having LPC host this seems a bit strange, given your defined audience
> (users vs. kernel developers).  Perhaps it would be more effective as a

I don't find it strange at all. There should be more "users" at
plumbers. LPC is about discussion and not presentations. When users and
developers get together, it should be a discussion.

> session in the Open Source Summit track, or at the very least, if
> attendees at the OSS Summit were able to attend the miniconference w/o
> the purchase of a LPC badge.

I think a badge would still be worth purchasing. Having the ability to
help influence development is a worthy goal.

> 
> > Are people willing to participate and have a discussion on what works
> > and what doesn't work with PREEMPT_RT. Be able to share horror stories
> > and work arounds with each other and perhaps come up with better
> > solutions. I will take what comes out of this with me to Prague and
> > perhaps be able to influence the development of PREEMPT_RT to address
> > any issues that arise.  
> 
> As a brainstorming of potential content, here's what I'd like to see as
> an attendee (certainly willing to present where appropriate too!):
> 
>   1. Debugging case studies:
> 
> There are a ton of debugging/visualizing/tracing tools out there
> available for finding latency problems.  Too many tools; some with
> overlapping purpose, and some better than others.  It would be an
> interesting exercise to have a few folks demonstrate the complete
> identification, debugging, and fix of a performance problem on RT,
> complete with a walkthrough of what tools were used.

Note, LPC is about discussions and not presentations or tutorials.
Although, I could submit a talk that would hopefully be a good
introduction into what the plumbers track would be like. I could maybe
get it accepted for the "shared" day (talks for both plumbers and Open
Source Summit attendees).

If you can create a discussion out of debugging tools. "Here's what I
have, is there a better way to do this?"

> 
>   2. What to consider when designing the priority scheme of your system
>      (application threads, irqthreads, RCU threads, ...)

I would turn this into something like "here's my work arounds to my
problems, are there better ways to do this?"

> 
>   3. An enumeration of "knobs" which impact RT behavior
> 
> Where knobs are any of the compile-time or run-time configuration
> parameters and how they impact determinism (I'm thinking things we've
> run into like IRQSOFF tracing, timerslack, the bizarre
> /dev/cpu_dma_latency, etc).

"here's what I've done, are there better ways to do this?"

Hmm, I'm seeing a trend here ;-)

> 
> > There's also talk about having this combined with a Safety Critical
> > session focused on PREEMPT_RT.
> >
> > Are people interesting in attending such an event? If so, I'll submit a
> > proposal for having such a Micro-Conference at Linux Plumbers. If not,
> > I wont bother wasting any time on it.  
> 
> Sounds good to me!  Let me know where I can help.

I rather have you tell us were you want to help :)

I'm looking for ideas from others. This email is just to poke peoples
brains.

-- Steve

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Julia Cartwright]

[-- Attachment #1: Type: text/plain, Size: 4662 bytes --]

On Tue, Feb 28, 2017 at 04:04:32PM -0500, Steven Rostedt wrote:
> On Tue, 28 Feb 2017 14:41:21 -0600 Julia Cartwright <julia@ni.com> wrote:
> > > A month later there will be a Real-Time get together in Prague, that
> > > would be perfect to be able to bring together a list of issues that
> > > users may have and present this to the PREEMPT_RT kernel developers.  
> > 
> > Having LPC host this seems a bit strange, given your defined audience
> > (users vs. kernel developers).  Perhaps it would be more effective as a
> 
> I don't find it strange at all. There should be more "users" at
> plumbers. LPC is about discussion and not presentations. When users and
> developers get together, it should be a discussion.
> 
> > session in the Open Source Summit track, or at the very least, if
> > attendees at the OSS Summit were able to attend the miniconference w/o
> > the purchase of a LPC badge.
> 
> I think a badge would still be worth purchasing. Having the ability to
> help influence development is a worthy goal.

I didn't mean to make it sound like I didn't consider the LPC badge to
be worth it.  LPC is one of my favorite conferences to attend! Anyone
who asks me which Linux conferences to attend, LPC is always top on my
list.

However, from my experience, the attendees at LPC are largely already
kernel engineers, or other low-level usermode people.  I suppose it's a
question of what "users" you are looking for.

That doesn't necessarily include people who are domain-experts in RTOS
applications but are relatively new to Linux; or people who use a
distro-packaged RT kernels for professional audio but have run into a
few minor snags; or ...

> > > Are people willing to participate and have a discussion on what works
> > > and what doesn't work with PREEMPT_RT. Be able to share horror stories
> > > and work arounds with each other and perhaps come up with better
> > > solutions. I will take what comes out of this with me to Prague and
> > > perhaps be able to influence the development of PREEMPT_RT to address
> > > any issues that arise.  
> > 
> > As a brainstorming of potential content, here's what I'd like to see as
> > an attendee (certainly willing to present where appropriate too!):
> > 
> >   1. Debugging case studies:
> > 
> > There are a ton of debugging/visualizing/tracing tools out there
> > available for finding latency problems.  Too many tools; some with
> > overlapping purpose, and some better than others.  It would be an
> > interesting exercise to have a few folks demonstrate the complete
> > identification, debugging, and fix of a performance problem on RT,
> > complete with a walkthrough of what tools were used.
> 
> Note, LPC is about discussions and not presentations or tutorials.
> Although, I could submit a talk that would hopefully be a good
> introduction into what the plumbers track would be like. I could maybe
> get it accepted for the "shared" day (talks for both plumbers and Open
> Source Summit attendees).
> 
> If you can create a discussion out of debugging tools. "Here's what I
> have, is there a better way to do this?"

Yes, I didn't mean to say to suggest a session with a bunch of
presentations.  I'm only suggesting a presentation as a means to
facilitate meaningful discussions.  Having a ground to start with (a
real-life debugging scenario), I think is a good way to get there.

> >   2. What to consider when designing the priority scheme of your system
> >      (application threads, irqthreads, RCU threads, ...)
>
> I would turn this into something like "here's my work arounds to my
> problems, are there better ways to do this?"
>
> >
> >   3. An enumeration of "knobs" which impact RT behavior
> >
> > Where knobs are any of the compile-time or run-time configuration
> > parameters and how they impact determinism (I'm thinking things we've
> > run into like IRQSOFF tracing, timerslack, the bizarre
> > /dev/cpu_dma_latency, etc).
>
> "here's what I've done, are there better ways to do this?"
>
> Hmm, I'm seeing a trend here ;-)

Certainly.

> > > There's also talk about having this combined with a Safety Critical
> > > session focused on PREEMPT_RT.
> > >
> > > Are people interesting in attending such an event? If so, I'll submit a
> > > proposal for having such a Micro-Conference at Linux Plumbers. If not,
> > > I wont bother wasting any time on it.  
> > 
> > Sounds good to me!  Let me know where I can help.
> 
> I rather have you tell us were you want to help :)

I'd happily talk to/facilitate discussions on the topics I mentioned
above.

   Julia

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Steven Rostedt]

On Tue, 28 Feb 2017 15:40:12 -0600
Julia Cartwright <julia@ni.com> wrote:


> > I think a badge would still be worth purchasing. Having the ability to
> > help influence development is a worthy goal.  
> 
> I didn't mean to make it sound like I didn't consider the LPC badge to
> be worth it.  LPC is one of my favorite conferences to attend! Anyone
> who asks me which Linux conferences to attend, LPC is always top on my
> list.

No worries, I didn't take it that way.

> 
> However, from my experience, the attendees at LPC are largely already
> kernel engineers, or other low-level usermode people.  I suppose it's a
> question of what "users" you are looking for.

Actually, we are trying to have LPC be more cross the board and not so
kernel centric. More like the old days of OLS.

> 
> That doesn't necessarily include people who are domain-experts in RTOS
> applications but are relatively new to Linux; or people who use a
> distro-packaged RT kernels for professional audio but have run into a
> few minor snags; or ...

I think these people would be perfect for LPC!


> > I rather have you tell us were you want to help :)  
> 
> I'd happily talk to/facilitate discussions on the topics I mentioned
> above.

Hopefully we can get more people to reply to this. It would be useless
to have a Micro-Conf that consists of two people sharing war stories.

-- Steve

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Steven Rostedt]

On Thu, 2 Mar 2017 00:39:04 +0100
Frederic Weisbecker <fweisbec@gmail.com> wrote:


> If anybody is interested, I can do an overview of Full Nohz (and Isolation
> alike), with a user/sysadmin focus. Unfortunately I can only talk about its
> usage on mainline. Although I can certainly document on its specifics in
> -rt.

Remember, Linux Plumbers is not about presentations or overviews (One
slide maximum). If you want to discuss what's not working (did we get
rid of that once a second tick yet?), and get a discussion going about
what needs to be done and how to fix it, then yeah, THAT would be great!

-- Steve

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Staffan Tjernstrom]

On Wed, 2017-03-01 at 21:08 -0500, Steven Rostedt wrote:
> On Thu, 2 Mar 2017 00:39:04 +0100
> Frederic Weisbecker <fweisbec@gmail.com> wrote:
> 
> 
> > ...Although I can certainly document on its
> > specifics in
> > -rt.
> 
> Remember, Linux Plumbers is not about presentations or overviews (One
> slide maximum). If you want to discuss what's not working (did we get
> rid of that once a second tick yet?), and get a discussion going
> about
> what needs to be done and how to fix it, then yeah, THAT would be
> great!
> 
> -- Steve
+1 on that.

Staffan Tj.

Re: [CFP] RT Users Micro-Conf at Linux Plumbers 2017

[Frederic Weisbecker]

On Wed, Mar 01, 2017 at 09:08:29PM -0500, Steven Rostedt wrote:
> On Thu, 2 Mar 2017 00:39:04 +0100
> Frederic Weisbecker <fweisbec@gmail.com> wrote:
> 
> 
> > If anybody is interested, I can do an overview of Full Nohz (and Isolation
> > alike), with a user/sysadmin focus. Unfortunately I can only talk about its
> > usage on mainline. Although I can certainly document on its specifics in
> > -rt.
> 
> Remember, Linux Plumbers is not about presentations or overviews (One
> slide maximum). If you want to discuss what's not working (did we get
> rid of that once a second tick yet?), and get a discussion going about
> what needs to be done and how to fix it, then yeah, THAT would be great!

Ok I can do that :-)