RT scheduler config, suggestions and questions

RT scheduler config, suggestions and questions

[Max Krasnyanskiy]

Folks,

I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG:
	sched_rt_period
	sched_rt_ratio

I do not believe that is correct. I know that we do not want to expose scheduler knobs
in general but theses are not the heuristic kind of knobs. There is no way the scheduler 
can magically figure out what the correct setting should be here.

Also shouldn't those new RT features that recently went be configurable and _disabled_ 
by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable. 
SCHED_FIFO is clearly defined as
"
  A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted 
  by a higher priority process, or it calls sched_yield(2).
"

Both the watchdog and the throttling are clearly braking that rule. I think it's good to have 
those features but not enabled by default and certainly not with sysctls that disable them 
hidden under debugging.
How about this:
- We introduce Kconfig options for them ?
- Expose all rt sysctls outside of #ifdef DEBUG

I can kook up some patches if that sounds ok.

----

btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
But's it gotta be per thread. Single setting per user is not enough. Unless a use has a single 
RT task.

Max

Re: RT scheduler config, suggestions and questions

[Peter Zijlstra]

On Tue, 2008-02-05 at 15:37 -0800, Max Krasnyanskiy wrote:
> Folks,
> 
> I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG:
> 	sched_rt_period
> 	sched_rt_ratio
> 
> I do not believe that is correct. I know that we do not want to expose scheduler knobs
> in general but theses are not the heuristic kind of knobs. There is no way the scheduler 
> can magically figure out what the correct setting should be here.

Yeah, since fixed.

> Also shouldn't those new RT features that recently went be configurable and _disabled_ 
> by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable. 
> SCHED_FIFO is clearly defined as
> "
>   A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted 
>   by a higher priority process, or it calls sched_yield(2).
> "

The watchdog is disabled by default, the bandwidth is .95s every 1s,
which is mainly a safe-guard against run-away real-time tasks. As long
as real-time usage stays within those limits nothing happens. If you
don't like it set sched_rt_runtime [*] to -1.

[*] provided in the interface changes posted a few days ago.

> Both the watchdog and the throttling are clearly braking that rule. I think it's good to have 
> those features but not enabled by default and certainly not with sysctls that disable them 
> hidden under debugging.
> How about this:
> - We introduce Kconfig options for them ?

I don't see why this would be needed.

> - Expose all rt sysctls outside of #ifdef DEBUG

Already did this somewhere along the line.

> btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
> But's it gotta be per thread.

It is.

> Single setting per user is not enough. Unless a use has a single RT task.

?

Re: RT scheduler config, suggestions and questions

[Peter Zijlstra]

On Wed, 2008-02-06 at 07:36 +0100, Peter Zijlstra wrote:

> > btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
> > But's it gotta be per thread.
> 
> It is.
> 
> > Single setting per user is not enough. Unless a use has a single RT task.
> 
> ?

Ah, its starting to make sense, you want it configurable per thread, not
process wide. Yes, I see how that is useful, just no idea how to expose
that to user-space atm.

Re: RT scheduler config, suggestions and questions

[Max Krasnyanskiy]

Peter Zijlstra wrote:
> On Tue, 2008-02-05 at 15:37 -0800, Max Krasnyanskiy wrote:
>> Folks,
>>
>> I just realized that in latest Linus' tree following sysctls are under SCHED_DEBUG:
>> 	sched_rt_period
>> 	sched_rt_ratio
>>
>> I do not believe that is correct. I know that we do not want to expose scheduler knobs
>> in general but theses are not the heuristic kind of knobs. There is no way the scheduler 
>> can magically figure out what the correct setting should be here.
> 
> Yeah, since fixed.
Super. I guess it's not in mainline yet. Or it wasn't yesterday.
 
>> Also shouldn't those new RT features that recently went be configurable and _disabled_ 
>> by default ? For example "RT watchdog" and "RT throttling" actually seem very questionable. 
>> SCHED_FIFO is clearly defined as
>> "
>>   A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted 
>>   by a higher priority process, or it calls sched_yield(2).
>> "
> 
> The watchdog is disabled by default, the bandwidth is .95s every 1s,
> which is mainly a safe-guard against run-away real-time tasks. As long
> as real-time usage stays within those limits nothing happens. If you
> don't like it set sched_rt_runtime [*] to -1.
Oh. From looking at the code I assumed I need to set sched_rt_ratio to 65536.
I guess you changed things around a bit. I'll look at the latest changes.

>> Both the watchdog and the throttling are clearly braking that rule. I think it's good to have 
>> those features but not enabled by default and certainly not with sysctls that disable them 
>> hidden under debugging.
>> How about this:
>> - We introduce Kconfig options for them ?
> 
> I don't see why this would be needed.
Reduces code size and RT scheduler complexity for the cases when these features are not required.
For example my preference would for RT scheduler to be as lean as possible to avoid any kind of 
overhead and cache pollution. I know both watchdog and throttling end up being a single if() when
disabled via sysctl, but it all adds up.
 
>> - Expose all rt sysctls outside of #ifdef DEBUG
> 
> Already did this somewhere along the line.
> 
>> btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
>> But's it gotta be per thread.
> 
> It is.
> 
>> Single setting per user is not enough. Unless a use has a single RT task.
> 
> ?
Currently it's per process. Most RT apps will most likely have several RT threads.

Max

Re: RT scheduler config, suggestions and questions

[Max Krasnyanskiy]

Peter Zijlstra wrote:
> On Wed, 2008-02-06 at 07:36 +0100, Peter Zijlstra wrote:
> 
>>> btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
>>> But's it gotta be per thread.
>> It is.
>>
>>> Single setting per user is not enough. Unless a use has a single RT task.
>> ?
> 
> Ah, its starting to make sense, you want it configurable per thread, not
> process wide. Yes, I see how that is useful, just no idea how to expose
> that to user-space atm.
Yes. That's what I meant. I don't think overall per process setting is that useful.
Per thread though would be useful.

How to expose that to the user-space ? The best option in my opinion is to extend
struct sched_param. That way both sched_setparam() and pthread_attr_setschedparam() 
can be used to set new attributes and it's backwards compatible.
Something like:

struct sched_param {
	...
	unsigned int sched_rt_limit;
	unsigned int sched_rt_...;
};

Max

Re: RT scheduler config, suggestions and questions

[Peter Zijlstra]

On Wed, 2008-02-06 at 10:04 -0800, Max Krasnyanskiy wrote:
> Peter Zijlstra wrote:
> > On Wed, 2008-02-06 at 07:36 +0100, Peter Zijlstra wrote:
> > 
> >>> btw I can see "watchdog" being very useful to catch hard-RT tasks that exceed the deadline.
> >>> But's it gotta be per thread.
> >> It is.
> >>
> >>> Single setting per user is not enough. Unless a use has a single RT task.
> >> ?
> > 
> > Ah, its starting to make sense, you want it configurable per thread, not
> > process wide. Yes, I see how that is useful, just no idea how to expose
> > that to user-space atm.
> Yes. That's what I meant. I don't think overall per process setting is that useful.
> Per thread though would be useful.
> 
> How to expose that to the user-space ? The best option in my opinion is to extend
> struct sched_param. That way both sched_setparam() and pthread_attr_setschedparam() 
> can be used to set new attributes and it's backwards compatible.
> Something like:
> 
> struct sched_param {
> 	...
> 	unsigned int sched_rt_limit;
> 	unsigned int sched_rt_...;
> };

I'm somewhat afraid to touch sched_param, but it seems I need to change
it anyway soonish, so I'd better figure out how to do that.