Limiting certain processes to CPU % - Linux

Question

I have the following problem: some processes, generated dynamically, have a tendency to eat 100% of CPU. I would like to limit all the process matching some criterion (e.g. process name) to a certain amount of CPU percentage.

The specific problem I'm trying to solve is harnessing folding@home worker processes. The best solution I could think of is a perl script that's executed periodically and uses the cpulimit utility to limit the processes (if you're interested in more details, check this blog post). It works, but it's a hack :/

Any ideas? I would like to leave the handling of processes to the OS :)

---

Thanks again for the suggestions, but we're still missing the point :)

The "slowDown" solution is essentially what the "cpulimit" utility does. I still have to take care about what processes to slow down, kill the "slowDown" process once the worker process is finished and start new ones for new worker processes. It's precisely what I did with the Perl script and a cron job.

The main problem is that I don't know beforehand what processes to limit. They are generated dynamically.

Maybe there's a way to limit all the processes of one user to a certain amount of CPU percentage? I already set up a user for executing the folding@home jobs, hoping that i could limit him with the /etc/security/limits.conf file. But the nearest I could get there is the total CPU time per user...

It would be cool if to have something that enables you to say: "The sum of all CPU % usage of this user's processes cannot exceed 50%". And then let the processes fight for that 50% of CPU regarding to their priorities...

---

Guys, thanks for your suggestions, but it's not about priorities - I want to limit the CPU % even when there's plenty of CPU time available. The processes are already low priority, so they don't cause any performance issues.

I would just like to prevent the CPU from running on 100% for extended periods...

Answer 1

Throwing some sleep calls in there should force the process off the CPU for a certain time. If you sleep 30 seconds once a minute, your process shouldn't average more than 50% CPU usage during that minute.

Answer 2

I had a slightly similar issue with gzip.

Assuming we want to decrease the CPU of a gzip process:

    gzip backup.tar & sleep 2 & cpulimit --limit 10 -e gzip -z

Options:

• I found sleep useful as the cpulimit sometimes didn't pick up the new gzip process immediately
• --limit 10 limits gzip CPU usage to 10%
• -z automatically closes cpulimit when gzip process finishes

Another option is to run the cpulimit daemon.

Answer 3

You could scale down the CPU frequency. Then you don't have to worry about the individual processes. When you need more cpu's, scale the frequency back up.

Answer 4

I think in Linux there is no solution to cap the cpu usage, but there is an acceptable way to limit any process to a certain CPU usage: http://ubuntuforums.org/showthread.php?t=992706

In case they remove the info, here is again

---

1. INSTALL PACKAGES

2. Install cpulimit package. Code:

   sudo apt-get install cpulimit

3. Install gawk package. Code:

   sudo apt-get install gawk

4. CREATE CPULIMIT DAEMON FILE

Open text editor with root privileges and save bellow daemon script text to new file /usr/bin/cpulimit_daemon.sh

Code:

#!/bin/bash
# ==============================================================
# CPU limit daemon - set PID's max. percentage CPU consumptions
# ==============================================================

# Variables
CPU_LIMIT=20        # Maximum percentage CPU consumption by each PID
DAEMON_INTERVAL=3   # Daemon check interval in seconds
BLACK_PROCESSES_LIST=   # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
WHITE_PROCESSES_LIST=   # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.

# Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
if [[ -n "$BLACK_PROCESSES_LIST" &&  -n "$WHITE_PROCESSES_LIST" ]] ; then    # If both variables are defined then error is produced.
   echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty."
   exit 1
elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
   NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
   NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
else
   NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
fi

# Search and limit violating PIDs
while sleep $DAEMON_INTERVAL
do
   NEW_PIDS=$(eval "$NEW_PIDS_COMMAND")                                                                    # Violating PIDs
   LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}')                                          # Already limited PIDs
   QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$')   # PIDs in queue

   for i in $QUEUE_PIDS
   do
       cpulimit -p $i -l $CPU_LIMIT -z &   # Limit new violating processes
   done
done

3. CHANGE VARIABLES TO YOUR ENVIRONMENT NEEDS

CPU_LIMIT Change this variable in above script if you would like to omit CPU consumption for every process to any other percentage then 20%. Please read "If using SMP computer" chapter bellow if you have SMP computer (more then 1 CPU or CPU with more then 1 core).

DAEMON_INTERVAL Change this variable in above script if you would like to have more/less regular checking. Interval is in seconds and default is set to 3 seconds.

BLACK_PROCESS_LIST and WHITE_PROCESSES_LIST Variable BLACK_PROCESSES_LIST limits only specified processes. If variable is empty (default) all violating processes are limited.

Variable WHITE_PROCESSES_LIST limits all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.

One or both of the variables BLACK_PROCESSES_LIST and WHITE_PROCESSES_LIST has to be empty - it is not logical that both variables are defined.

You can specify multiple processes in one of this two variables using delimiter characters "|" (without double quotes). Sample: if you would like to cpulimit all processes except mysql, firefox and gedit processes set variable: WHITE_PROCESSES_LIST="mysql|firefox|gedit"

4. PROCEDURE TO AUTOMATICALLY START DAEMON AT BOOT TIME

5. Set file permissions for root user: Code:

   sudo chmod 755 /usr/bin/cpulimit_daemon.sh

6. Open text editor with root privileges and save bellow script to new file /etc/init.d/cpulimit

Code:

#!/bin/sh
#
# Script to start CPU limit daemon
#
set -e

case "$1" in
start)
if [ $(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print $1}' | wc -l) -eq 0 ]; then
    nohup /usr/bin/cpulimit_daemon.sh >/dev/null 2>&1 &
    ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon started successfully"; else print " * cpulimit daemon can not be started" }'
else
    echo " * cpulimit daemon can't be started, because it is already running"
fi
;;
stop)
CPULIMIT_DAEMON=$(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print $1}' | wc -l)
CPULIMIT_INSTANCE=$(ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | wc -l)
CPULIMIT_ALL=$((CPULIMIT_DAEMON + CPULIMIT_INSTANCE))
if [ $CPULIMIT_ALL -gt 0 ]; then
    if [ $CPULIMIT_DAEMON -gt 0 ]; then
        ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print $1}' | xargs kill -9   # kill cpulimit daemon
    fi

    if [ $CPULIMIT_INSTANCE -gt 0 ]; then
        ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | xargs kill -9                    # release cpulimited process to normal priority
    fi
    ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon can not be stopped"; else print " * cpulimit daemon stopped successfully" }'
else
    echo " * cpulimit daemon can't be stopped, because it is not running"
fi
;;
restart)
$0 stop
sleep 3
$0 start
;;
status)
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh"  {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon is running"; else print " * cpulimit daemon is not running" }'
;;
esac
exit 0

3. Change file's owner to root:

Code:

sudo chown root:root /etc/init.d/cpulimit

4. Change permissions:

Code:

sudo chmod 755 /etc/init.d/cpulimit

5. Add script to boot-up procedure directories: Code:

   sudo update-rc.d cpulimit defaults

6. Reboot to check if script starts cpulimit daemon at boot time: Code:

   sudo reboot

7. MANUALLY CHECK, STOP, START AND RESTART DAEMON

Note: Daemon and service in this tutorial has equal meaning.

Note: For users using prior to Ubuntu 8.10 (like Ubuntu 8.04 LTS) instead of service command use "sudo /etc/init.d/cpulimit status/start/stop/restart" syntax or install sysvconfig package using command: sudo apt-get install sysvconfig

Check if cpulimit service is running Check command returns: "cpulimit daemon is running" if service is running, or "cpulimit daemon is not running" if service is not running. Code:

sudo service cpulimit status

Start cpulimit service You can manually start cpulimit daemon which will start to omit CPU consumption. Code:

sudo service cpulimit start

Stop cpulimit service Stop command stops cpulimit daemon (so no new process will be limited) and also sets to all existing limited processes to have full access to CPU, just like it was before cpulimit was not running. Code:

sudo service cpulimit stop

Restart cpulimit service If you change some variables settings in /usr/bin/cpulimit_daemon.sh like CPU_LIMIT, DAEMON_INTERVAL, BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST, then after changing settings you must restart service. Code:

sudo service cpulimit restart

6. CHECK CPU CONSUMPTION WITH OR WITHOUT CPULIMIT DAEMON

Without daemon 1. stop cpulimit daemon (sudo service cpulimit stop) 2. execute CPU intensive tasks in background 3. execute command: top and check for %CPU column Result of %CPU is probably more then 20% for each process.

With daemon turned on 1. start cpulimit daemon (sudo service cpulimit start) 2. execute the same CPU intensive tasks in background 3. execute command: top and check for %CPU column Result of %CPU should be maximum 20% for each process. Note: Don't forget at beginning %CPU can be more then 20%, because daemon has to catch violating process in interval of 3 seconds (set in script by default)

7. IF USING SMP COMPUTER

I have tested this code on Intel dual-core CPU computer - that behaves like SMP computer. Don't forget that top command and also cpulimit by default behaves in Irix mode, where 20% means 20% of one CPU. If there are two CPUs (or dual-core) then total %CPU can be 200%. In top command Irix mode can be turned off with command I (pressing +i when top command is running) and Solaris mode is turned on, where total amount of CPU is divided by number of CPUs, so %CPU can be no more then 100% on any number of CPU computer. Please read more info about top command in top man page (search for I command). Please also read more about how cpulimit is operating on SMP computer in cpulimit official page.

But how does cpulimit daemon operates on SMP computer? Always in Irix mode. So if you would like to spend 20% of CPU power on 2-CPU computer then 40% should be used for CPU_LIMIT variable in cpulimit daemon script.

8. UNINSTALL CPULIMIT DAEMON AND CPULIMIT PROGRAM

If you would like to get rid of cpulimit daemon you can clean up your system by removing cpulimit daemon and uninstalling cpulimit program.

1. Stop cpulimit daemon Code:

   sudo service cpulimit stop # Stop cpulimit daemon and all cpulimited processes

2. Remove daemon from boot-up procedure Code:

   sudo update-rc.d -f cpulimit remove # Remove symbolic links

3. Delete boot-up procedure Code:

   sudo rm /etc/init.d/cpulimit # Delete cpulimit boot-up script

4. Delete cpulimit daemon Code:

   sudo rm /usr/bin/cpulimit_daemon.sh # Delete cpulimit daemon script

5. Uninstall cpulimit program Code:

   sudo apt-get remove cpulimit

6. Uninstall gawk program If you don't need this program for any other script, you can remote it. Code:

   sudo apt-get remove gawk

7. NOTE ABOUT AUTHORS

I have just written daemon for cpulimit (bash scripts above). I am not the author of cpulimit project. If you need more info about cpulimit program, please read official cpulimit web page: http://cpulimit.sourceforge.net/.

Regards, Abcuser

Answer 5

I had a similar problem, and the other solutions presented in the thread don't address it at all. My solution works for me right now, but it is suboptimal, particularly for the cases where the process is owned by root.
My workaround for now is to try very hard to make sure that I don't have any long-running processes owned by root (like have backup be done only as a user)

I just installed the hardware sensors applet for gnome, and set up alarms for high and low temperatures on the CPU, and then set up the following commands for each alarm:

low:

mv /tmp/hogs.txt /tmp/hogs.txt.$$ && cat /tmp/hogs.txt.$$ | xargs -n1 kill -CONT

high:

touch /tmp/hogs.txt && ps -eo pcpu,pid | sort -n -r | head -1 | gawk '{ print $2 }' >> /tmp/hogs.txt && xargs -n1 kill -STOP < /tmp/hogs.txt

The good news is that my computer no longer overheats and crashes. The downside is that terminal processes get disconnected from the terminal when they get stopped, and don't get reconnected when they get the CONT signal. The other thing is that if it was an interactive program that caused the overheating (like a certain web browser plugin!) then it will freeze in the middle of what I'm doing while it waits for the CPU to cool off. It would be nicer to have CPU scaling take care of this at a global level, but the problem is that I only have two selectable settings on my CPU and the slow setting isn't slow enough to prevent overheating.

Just to re-iterate here, this has nothing at all to do with process priority, re-nicing,and obviously nothing to do with stopping jobs that run for a long time. This has to do with preventing CPU utilization from staying at 100% for too long, because the hardware is unable to dissipate the heat quickly enough when running at full capacity (idle CPU generates less heat than a fully loaded CPU).

Some other obvious possibilities that might help are:

Lower the CPU speed overall in the BIOS

Replace the heatsink or re-apply the thermal gel to see if that helps

Clean the heatsink with some compressed air

Replace the CPU fan

[edit] Note: no more overheating at 100% CPU when I disable variable fan speed in the bios (asus p5q pro turbo). With the CPU fully loaded, each core tops out at 49 celcius.

Answer 6

You can limit the amount of cpu time with cgroups. The OS will handle resource management just like you ask for in you question.

Here is a wiki with examples: https://wiki.archlinux.org/index.php/Cgroups

Since the processes already run as a separate user limiting all process from that user should be the easiest solution.