dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Wed Jun 12, 2019 3:56 pm

hello,
which are the default thread priorities of common single-thread programs (C++ sources) ,
multithreading executables using pthread (C++ sources, no arbitrary pthread priorities, just defaults)
and which priorities have programs run by the kernel?

User avatar
Paeryn
Posts: 2604
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Wed Jun 12, 2019 10:42 pm

Normal user threads are scheduled with SCHED_OTHER and have a priority of 0 and a nice level between -20 and +19, the default nice level is 0. The higher its niceness is the lower its priority is, so a process with a niceness of 19 has the lowest priority. Only SCHED_OTHER uses the niceness level and the kernel tries to give threads some time even if they have a high niceness.

Threads scheduled with SCHED_FIFO or SCHED_RR are realtime threads and have priorities between 1 and 99. A priority of 99 is the highest priority (the reverse of the nice level). This ensures that realtime threads always have a higher priority than any other.

There are a few other schedulers, they all have a priority of 0 I think. e.g. SCHED_IDLE which is for threads that have the absolute lowest priority (even lower than a SCHED_OTHER at nice 19)

You can set the niceness of programs you run with nice e.g. the following will start my_program with a niceness of 5 (slightly lower priority than usual).

Code: Select all

$ nice 5 ./my_program
Or you can use renice to change a running thread's nice level. There may be limits on what levels you can change nice to without needing to be root, and you can can only change niceness for threads you own (unless you are root).
She who travels light — forgot something.

dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Wed Jun 12, 2019 11:22 pm

thank you Paeryn for your reply!
So IIUC, the scheduler rules (for FIFO, RR, OTHER) cannot directly be compared to each other by identical level numbers.
Having read some posts here in this forum I thought SCHED_RR might be the best for my purposes as far as I understood,
so I tested with prios of 40 or 60, and both have been smooth and stable as far as I could see (using my UART comm program to an Arduino).
At least already by pthread SCHED_RR prio=40 it was far more stable than just my former single thread program without any arbitrary prios.

Do you think that this is a good choice or shall I get even higher (or perhaps lower) to give it a sufficient stability and priority over kernel programs (if that would be possible at all)?

User avatar
Paeryn
Posts: 2604
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Thu Jun 13, 2019 2:10 am

I'm not sure how threads at the same priority but with different schedulers are ordered with respect to others. All threads at pri=40 sched_rr will get equal run time (when the first finishes it goes to the back of its queue) but I don't know how it decides between a pri=40 sched_fifo thread and a pri=40 sched_rr thread. However it manages it, a pri=50 thread of either would take preference.

If you are timing dependant or need quick responses then ideally you want to give it just enough priority that it won't get locked out by threads that aren't (as) timing dependant or that you don't mind being less responsive. Try different priorities and see which gives you the best compromise between your program's response and the system as a whole's response. Just be careful about giving it too high a priority if it does a lot of work without yielding else you could lock out other important threads for significant periods of time (more of a problem on a single core RPi than a multicore).
She who travels light — forgot something.

dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Thu Jun 13, 2019 7:15 am

thanks, yes, that is actually the crucial question I am trying to solve:
For my own program I can probably define and handle different SCHED_RR prios, but which prio should I choose to be safe over kernel threads, tasks, programs, and/or daemons so that they don't temporarily take over and disturb my own time-critical thread?

dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Thu Jun 13, 2019 5:58 pm

PS, forgotten to ask...:

if my pthreads have e.g. SCHED_RR prio=50 :

which prio has a perpetual loop in main()?

If a pthread by prio=50 hangs up completely, will it block then the main loop, too?

And will a 2nd thread by prio=30 then also be blocked totally?

Code: Select all

int main() {
    
    // threads    
    pthread_t  thread1, thread2;
    struct     sched_param  param;
    
    printf("starting ..."); printf("\n");   
    
    // open UART Serial com port    
    Serial = serialOpen (uart, UARTclock);       
        

    // start multithreading  
    
    pthread_create(&thread1, NULL, KBD_thr, NULL);     // low priority: keyboard monitor 
    param.sched_priority = 30;
    pthread_setschedparam(thread1, SCHED_RR, &param);
   
    pthread_create(&thread2, NULL, UART_thr, NULL);    // medium  priority: UART
    param.sched_priority = 50;
    pthread_setschedparam(thread2, SCHED_RR, &param);    
     
    
    while(_TASKS_ACTIVE_) {          // <~~~~~~~~~~ perpetual main loop
       // SNIP
       delay(10); 
    }                // 
     
    // wait for threads to join 
    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);
   
    serialClose( Serial);
    exit(0);
}

User avatar
Paeryn
Posts: 2604
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Thu Jun 13, 2019 8:50 pm

dsyleixa123 wrote:
Thu Jun 13, 2019 5:58 pm
PS, forgotten to ask...:

if my pthreads have e.g. SCHED_RR prio=50 :

which prio has a perpetual loop in main()?

If a pthread by prio=50 hangs up completely, will it block then the main loop, too?

And will a 2nd thread by prio=30 then also be blocked totally?

Code: Select all

int main() {
    
    // threads    
    pthread_t  thread1, thread2;
    struct     sched_param  param;
    
    printf("starting ..."); printf("\n");   
    
    // open UART Serial com port    
    Serial = serialOpen (uart, UARTclock);       
        

    // start multithreading  
    
    pthread_create(&thread1, NULL, KBD_thr, NULL);     // low priority: keyboard monitor 
    param.sched_priority = 30;
    pthread_setschedparam(thread1, SCHED_RR, &param);
   
    pthread_create(&thread2, NULL, UART_thr, NULL);    // medium  priority: UART
    param.sched_priority = 50;
    pthread_setschedparam(thread2, SCHED_RR, &param);    
     
    
    while(_TASKS_ACTIVE_) {          // <~~~~~~~~~~ perpetual main loop
       // SNIP
       delay(10); 
    }                // 
     
    // wait for threads to join 
    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);
   
    serialClose( Serial);
    exit(0);
}
Your main thread will be running as a normal (SCHED_OTHER, nice=0), only the threads you create will have the schedulers and priorities asked for. As long as the higher priority thread doesn't continuously work (e.g. it sleeps or waits on other events) then you are ok, but if your priority 50 thread sits in a loop working 100% then any threads with a priority of less than 50 won't get a look in. Although on a multi-core RPi (2 or 3) then that one thread could theoretically sit in a loop working and it would only tie up one of the four cores, the other three will still be able to run the other threads as normal.

Why are you starting the threads with default scheduler and then straight away changing their scheduler? Generally you'd set the scheduler and priority in a pthread_attr_t before you start the thread. Here I just create one pthread_attr_t variable, initialise it and use it in both thread creations (with just the change in priority between them) and then destroy it straight after the last thread is started (we don't need it any more, the data in it is copied to the threads when they are started).

Code: Select all

    pthread_t  thread1, thread2;
    pthread_attr_t thread_attr;
    struct     sched_param  param;
    
    pthread_attr_init(&thread_attr);  // Initialise the attributes
    pthread_attr_setschedpolicy(&thread_attr, SCHED_RR);  // Set attributes to RR policy
    param.sched_priority = 30;
    pthread_attr_setschedparam(&thread_attr, &param); // Set attributes to priority 30
    pthread_create(&thread1, &thread_attr, KBD_thr, NULL);     // low priority: keyboard monitor

    param.sched_priority = 50;
    pthread_attr_setschedparam(&thread_attr, &param); // Set attributes to priority 50 (policy is already RR)
    pthread_create(&thread2, &thread_attr, UART_thr, NULL);    // medium  priority: UART

    pthread_attr_destroy(&thread_attr); // We've done with the attributes
    
    // ...
She who travels light — forgot something.

dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Fri Jun 14, 2019 8:03 am

thank you for your hints and advices!
As long as the higher priority thread doesn't continuously work (e.g. it sleeps or waits on other events) then you are ok, but if your priority 50 thread sits in a loop working 100% then any threads with a priority of less than 50 won't get a look in.
but IIUC, a 2nd thread also running at prio 50 will have a chance though, right?
Why are you starting the threads with default scheduler and then straight away changing their scheduler?
simply, because I didn't know it better ;)
I didn't find reasonable beginner-friendly tutorials on pthread functionalities so I took what I found in examples in the web 8-)
I will better use then your example instead, but just out of curiosity: will my current setup fail?

User avatar
Paeryn
Posts: 2604
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Sat Jun 15, 2019 12:20 am

dsyleixa123 wrote:
Fri Jun 14, 2019 8:03 am
thank you for your hints and advices!
As long as the higher priority thread doesn't continuously work (e.g. it sleeps or waits on other events) then you are ok, but if your priority 50 thread sits in a loop working 100% then any threads with a priority of less than 50 won't get a look in.
but IIUC, a 2nd thread also running at prio 50 will have a chance though, right?
Why are you starting the threads with default scheduler and then straight away changing their scheduler?
simply, because I didn't know it better ;)
I didn't find reasonable beginner-friendly tutorials on pthread functionalities so I took what I found in examples in the web 8-)
I will better use then your example instead, but just out of curiosity: will my current setup fail?
Sorry if that question came over a bit harsh, didn't intend it to be. Changing the scheduler settings afterwards won't break anything, it's just cleaner to set everything up first (the attributes let you specify several other things like stack size as well) and it saves the kernel time moving it. Also, if there is any issue with your chosen scheduler settings (like you don't have the necessary permissions for it) the thread won't be started and left running whilst you deal with it.

Linux manpages sometimes give example code to show the basics, try typing man pthread_create at the command line.

If there are two (or more) threads running at priority 50, as long as they are SCHED_RR then the kernel will rotate between them when choosing which to run (if they are ready). If there is a SCHED_FIFO also at priority 50 then that will generally be chosen before a SCHED_RR.

Not exactly how it's implemented but a rough overview :-

Each schedule type RR, FIFO, OTHER etc. has lists of all the threads of that type (one list for each priority level). Each time slice (the kernel uses interrupts to make sure time slices are started regardless of whether a process is already doing anything)
  1. The scheduler starts searching with pri=99 (highest priority)
  2. It looks through the FIFO list of the current pri for a thread ready to run, if it finds one it then that is the thread to wake up, it is left in the same place in the list. Jump to step 6.
  3. It then looks through the RR list of the current pri for a thread ready to run. If it finds one in here then that is the thread to wake up, but first it gets taken off the top of its RR list and put at the end so next time the scheduler gets to this list then the thread will be the last to be checked. Jump to step 6.
  4. No FIFO or RR thread was ready so decrease pri. If pri > 0 jump to step 2.
  5. pri=0 so no realtime threads were ready so now search the OTHER list for a thread ready to run, this is dynamically ordered but roughly by niceness (so starting at -19, finishing at +20).
  6. If a thread is to be woken, wake it for this time slice.
  7. This round of scheduling is over, start back at step 1 for the next time slice.
She who travels light — forgot something.

dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Sat Jun 15, 2019 6:53 am

thank you,
1st, no, it didn't come over sort of harsh at all ;)

and 2nd, thanks, the scheduling rules really made it more clear now. 8-)
just to be sure, acc to
2. It then looks through the RR list of the current pri for a thread ready to run. If it finds one in here then that is the thread to wake up, but first it gets taken off the top of its RR list and put at the end so next time the scheduler gets to this list then the thread will be the last to be checked. Jump to step 6.
...
6. If a thread is to be woken, wake it for this time slice.
IIUC:
if there are 2 different RR prio 50 theads (thread1, thread2) , and thread1 is hung up without delays
( like hung up by while(1); )
it actually cannot happen though that the entire system hangs up too, because the scheduler will (relentless and merciless) switch the time slices from thread1 to thread2 and vice versa either way?

User avatar
Paeryn
Posts: 2604
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Sat Jun 15, 2019 3:50 pm

It should swap between them, the locked up one would ultimately prevent any lower priority thread from being run though. Again, it's a rough approximation of what happens, and with multi-core setups you have as many threads active as there are cores and threads will usually try to be kept on the same core they were on from their previous time slice.
She who travels light — forgot something.



dsyleixa123
Posts: 306
Joined: Mon Jun 11, 2018 11:22 am

Re: default priorities of user programs (single thread, pthread) and of programs run by the kernel ?

Mon Jun 17, 2019 8:42 am

thank you, that is really very sophisticated stuff... ;)

Return to “C/C++”