Re: schedulers and topology exposing questions

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Jan 28, 2016 at 03:10:57PM +0000, Dario Faggioli wrote:
> On Wed, 2016-01-27 at 10:27 -0500, Konrad Rzeszutek Wilk wrote:
> > On Wed, Jan 27, 2016 at 03:10:01PM +0000, George Dunlap wrote:
> > > 
> > > I'm not sure I understand the situation right, but it sounds a bit
> > > like
> > > what you're seeing is just a quirk of the fact that Linux doesn't
> > > always
> > > send IPIs to wake other processes up (either by design or by
> > > accident),
> > 
> > It does and it does not :-)
> > 
> > > but relies on scheduling timers to check for work to
> > > do.  Presumably
> > 
> > It .. I am not explaining it well. The Linux kernel scheduler when
> > called for 'schedule' (from the UDP sendmsg) would either pick the
> > next
> > appliction and do a context swap - of if there were none - go to
> > sleep.
> > [Kind of - it also may do an IPI to the other CPU if requested ,but
> > that requires
> > some hints from underlaying layers]
> > Since there were only two apps on the runqueue - udp sender and udp
> > receiver
> > it would run them back-to back (this is on baremetal)
> > 
> > However if SMT was not exposed - the Linux kernel scheduler would put
> > those
> > on each CPU runqueue. Meaning each CPU only had one app on its
> > runqueue.
> > 
> > Hence no need to do an context switch.
> > [unless you modified the UDP message to have a timeout, then it would
> > send an IPI]
> >
> So, may I ask what piece of (Linux) code are we actually talking about?
> Because I had a quick look, and could not find where what you describe
> happens....

udp_recvmsg->__skb_recv_datagram->sock_rcvtimeo->schedule_timeout
The sk_rcvtimeo is MAX_SCHEDULE_TIMEOUT but you can alter the
UDP by having a diffrent timeout.

And MAX_SCHEDULE_TIMEOUT when it eventually calls 'schedule()' just
goes to sleep (HLT) and eventually gets woken up VIRQ_TIMER.

The other side - udp_sendmsg is more complex, and I don't seem
to have the stacktrace.
 
> 
> Thanks and Regards,
> Dario
> -- 
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>