when/how is the schedule() function actually called?

when/how is the schedule() function actually called?

[Dawei Li]

Greetings!

I am reading several resources regarding the linux kernel scheduling
(in kernel 2.6.34).
There seems to be the periodic scheduler (tick_periodic()), which
invokes the scheduler_tick() function, then the entity_tick()
function, and then the resched_task() function. However, eventually,
the resched_task() function doesn't invoke the schedule() function; it
only invokes the set_tsk_need_resched() function. So, it is only
setting the need_resched flag.
My question is, when/how is the schedule() function actually called?
Also, I notice that at many locations in the code, when there is the
need to do a scheduling, the code is just setting the need_resched
flag. Same question: when/how is the schedule() function actually
called?
(Although I am using the 2.6.34 version, I believe the question is generally
applicable to any kernel version.)

Not sure if this list is still active. Appreciate any response.
Thanks!

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[Valdis Klētnieks]

On Sat, 09 Dec 2023 00:16:32 +0800, Dawei Li said:
> Greetings!

> (Although I am using the 2.6.34 version, I believe the question is generally
> applicable to any kernel version.)

That is, in general, a bad assumption when you are looking at kernel versions
old enough that they count as digital archaeology....

[/usr/src/linux-next] git show v2.6.34
tag v2.6.34
Tagger: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Sun May 16 14:17:44 2010 -0700
[/usr/src/linux-next] git diff --shortstat  v2.6.34..next-20231205
 96965 files changed, 32056985 insertions(+), 7606202 deletions(-)

Given that next-20231205 has just over 33 million lines of code, we're well
into territory where there's a vanishing small percentage of code still
remaining unchanged from 2010.

And yes, that means that even basic functions schedule() and friends have been
reworked in he past decade and a half....


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[Dawei Li]

[-- Attachment #1.1: Type: text/plain, Size: 2187 bytes --]

Hi Valdis,

Thanks for getting back to me!
I should've looked into newer versions. (Was looking at 2.6.34 because the
latest book I have on linux kernel, Linux
Kernel Development, 3rd Edition by Robert Love, is based on that version.)

I do have the same question for the latest version.
Here:
https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L5632
the scheduler_tick() is calling curr->sched_class->task_tick(rq, curr, 0);
https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L5651
This is calling entity_tick(),
https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/fair.c#L5447
and then resched_curr(),
https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L1040

As the comment says, resched_curr() is still just setting the need_resched
flag on a uniprocessor.
It is not calling the schedule() function.
When/how is the schedule() function actually called?

Appreciate any explanation/guidance on this!
Thanks!


On Sat, Dec 9, 2023 at 1:45 AM Valdis Klētnieks <valdis.kletnieks@vt.edu>
wrote:

> On Sat, 09 Dec 2023 00:16:32 +0800, Dawei Li said:
> > Greetings!
>
> > (Although I am using the 2.6.34 version, I believe the question is
> generally
> > applicable to any kernel version.)
>
> That is, in general, a bad assumption when you are looking at kernel
> versions
> old enough that they count as digital archaeology....
>
> [/usr/src/linux-next] git show v2.6.34
> tag v2.6.34
> Tagger: Linus Torvalds <torvalds@linux-foundation.org>
> Date:   Sun May 16 14:17:44 2010 -0700
> [/usr/src/linux-next] git diff --shortstat  v2.6.34..next-20231205
>  96965 files changed, 32056985 insertions(+), 7606202 deletions(-)
>
> Given that next-20231205 has just over 33 million lines of code, we're well
> into territory where there's a vanishing small percentage of code still
> remaining unchanged from 2010.
>
> And yes, that means that even basic functions schedule() and friends have
> been
> reworked in he past decade and a half....
>
>

[-- Attachment #1.2: Type: text/html, Size: 3407 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[Billie Alsup (balsup)]

>From: Dawei Li <daweilics@gmail.com>
>When/how is the schedule() function actually called? 

I vaguely recall the actual switch taking place at the end of an interrupt, 
and/or during the return from a syscall, or on explicit request (e.g. yield).
Look at __schedule and context_switch functions.




_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[Dawei Li]

[-- Attachment #1.1: Type: text/plain, Size: 934 bytes --]

Hi Billie,

Thanks for getting back to me!
Many sources mentioned the same thing, but where do I find the code that
supports this?

I followed the comment here:
https://github.com/torvalds/linux/blob/c527f5606aa545233a4d2c6d5c636ed82b8633ef/kernel/sched/core.c#L6536C10-L6536C10
to check all the following files:
arch/x86/entry/entry_32.S
arch/x86/entry/entry_64_compat.S
arch/x86/entry/entry_64.S
arch/x86/entry/entry.S

But I don't see any code that is calling the schedule or __schedule
function.

On Sat, Dec 9, 2023 at 3:10 AM Billie Alsup (balsup) <balsup@cisco.com>
wrote:

> >From: Dawei Li <daweilics@gmail.com>
> >When/how is the schedule() function actually called?
>
> I vaguely recall the actual switch taking place at the end of an
> interrupt,
> and/or during the return from a syscall, or on explicit request (e.g.
> yield).
> Look at __schedule and context_switch functions.
>
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 1831 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[Billie Alsup (balsup)]

> But I don't see any code that is calling the schedule or __schedule function.

I think with a little investigation, you will be able to figure it out.  For example, arch/x86/entry/common.c
in function do_syscall_64 calls syscall_entr_from_user_mode and syscall_exit_to_user_mode.   I could guess
we eventually reach exit_to_user_mode_loop, which will conditionally call schedule() which calls __schedule
(although I didn't investigate this chain, so am just guessing).

I'm a newbie to this stuff as well, but it is not too difficult to follow this.  I would guess there are
IDEs available that can help you with diagramming the flow if you don't want to manually find things
one layer at a time.  The Documentation folder also has a lot of good information.  For example,
Documentation/trace/histogram.rst shows some backtraces with __schedule, and you can see
examples of how it is called from do_syscall_64 and ret_from_fork.  Other traces are available in
Documentation/trace/ftrace.rst

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: when/how is the schedule() function actually called?

[iso m]

On 08/12/23 11:51 pm, Dawei Li wrote:
> Hi Valdis,
> 
> Thanks for getting back to me!
> I should've looked into newer versions. (Was looking at 2.6.34 because the
> latest book I have on linux kernel, Linux
> Kernel Development, 3rd Edition by Robert Love, is based on that version.)
> 
> I do have the same question for the latest version.
> Here:
> https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L5632
> the scheduler_tick() is calling curr->sched_class->task_tick(rq, curr, 0);
> https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L5651
> This is calling entity_tick(),
> https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/fair.c#L5447
> and then resched_curr(),
> https://github.com/torvalds/linux/blob/4df7c5fde316820286dfa6d203a1005d7fbe007d/kernel/sched/core.c#L1040
> 
> As the comment says, resched_curr() is still just setting the need_resched
> flag on a uniprocessor.
> It is not calling the schedule() function.
> When/how is the schedule() function actually called?
I believe it's called at the code path (kernel -> user) and (kernel -> kernel)

Here, it'll be at return from the interrupt context.

Checkout point 2. in the following comment snippet found above __schedule()
method in kernel/sched/core.c

/*
 * __schedule() is the main scheduler function.
 *
 * The main means of driving the scheduler and thus entering this function are:
 *
 *   1. Explicit blocking: mutex, semaphore, waitqueue, etc.
 *
 *   2. TIF_NEED_RESCHED flag is checked on interrupt and userspace return
 *      paths. For example, see arch/x86/entry_64.S.
 *
 *      To drive preemption between tasks, the scheduler sets the flag in timer
 *      interrupt handler scheduler_tick().
 *
 *   3. Wakeups don't really cause entry into schedule(). They add a
 *      task to the run-queue and that's it.
 *
 *      Now, if the new task added to the run-queue preempts the current
 *      task, then the wakeup sets TIF_NEED_RESCHED and schedule() gets
 *      called on the nearest possible occasion:
 *
 *       - If the kernel is preemptible (CONFIG_PREEMPTION=y):
 *
 *         - in syscall or exception context, at the next outmost
 *           preempt_enable(). (this might be as soon as the wake_up()'s
 *           spin_unlock()!)
 *
 *         - in IRQ context, return from interrupt-handler to
 *           preemptible context
 *
 *       - If the kernel is not preemptible (CONFIG_PREEMPTION is not set)
 *         then at the next:
 *
 *          - cond_resched() call
 *          - explicit schedule() call
 *          - return from syscall or exception to user-space
 *          - return from interrupt-handler to user-space
 *
 * WARNING: must be called with preemption disabled!
 */
static void __sched notrace __schedule(unsigned int sched_mode)


> 
> Appreciate any explanation/guidance on this!
> Thanks!
> 
> 
> On Sat, Dec 9, 2023 at 1:45 AM Valdis Klētnieks <valdis.kletnieks@vt.edu>
> wrote:
> 
>> On Sat, 09 Dec 2023 00:16:32 +0800, Dawei Li said:
>>> Greetings!
>>
>>> (Although I am using the 2.6.34 version, I believe the question is
>> generally
>>> applicable to any kernel version.)
>>
>> That is, in general, a bad assumption when you are looking at kernel
>> versions
>> old enough that they count as digital archaeology....
>>
>> [/usr/src/linux-next] git show v2.6.34
>> tag v2.6.34
>> Tagger: Linus Torvalds <torvalds@linux-foundation.org>
>> Date:   Sun May 16 14:17:44 2010 -0700
>> [/usr/src/linux-next] git diff --shortstat  v2.6.34..next-20231205
>>  96965 files changed, 32056985 insertions(+), 7606202 deletions(-)
>>
>> Given that next-20231205 has just over 33 million lines of code, we're well
>> into territory where there's a vanishing small percentage of code still
>> remaining unchanged from 2010.
>>
>> And yes, that means that even basic functions schedule() and friends have
>> been
>> reworked in he past decade and a half....
>>
>>
> 
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies