Docker timeout for container?

Question

For my dissertation at University, I'm working on a coding leaderboard system where users can compile / run untrusted code through temporary docker containers. The system seems to be working well so far, but one problem I'm facing is that when code for an infinite loop is submitted, E.g:

while True:
   print "infinite loop"

the system goes haywire. The problem is that when I'm creating a new docker container, the Python interpreter prevents docker from killing the child container as data is still being printed to STDOUT (forever). This leads to the huge vulnerability of docker eating up all available system resources until the machine using the system completely freezes (shown below):

So my question is, is there a better way of setting a timeout on a docker container than my current method that will actually kill the docker container and make my system secure (code originally taken from here)?

#!/bin/bash
set -e

to=$1
shift

cont=$(docker run --rm "$@")
code=$(timeout "$to" docker wait "$cont" || true)
docker kill $cont &> /dev/null
echo -n 'status: '
if [ -z "$code" ]; then
    echo timeout
else
    echo exited: $code
fi

echo output:
# pipe to sed simply for pretty nice indentation
docker logs $cont | sed 's/^/\t/'

docker rm $cont &> /dev/null

Edit: The default timeout in my application (passed to the $to variable) is "10s" / 10 seconds.

---

I've tried looking into adding a timer and sys.exit() to the python source directly, but this isn't really a viable option as it seems rather insecure because the user could submit code to prevent it from executing, meaning the problem would still persist. Oh the joys of being stuck on a dissertation... :(

Answer 1

If you want to run the containers without providing any protection inside them, you can use runtime constraints on resources.

In your case, -m 100M --cpu-quota 50000 might be reasonable.

That way it won't eat up the parent's system resources until you get around to killing it.

Answer 2

I guess, you can use signals in python like unix to set timeout. you can use alarm of specific time say 50 seconds and catch it. Following link might help you. signals in python

Answer 3

I have achieved a solution for this problem.

First you must kill docker container when time limit is achieved:

#!/bin/bash
set -e
did=$(docker run -it -d -v "/my_real_path/$1":/usercode virtual_machine ./usercode/compilerun.sh 2>> $1/error.txt)
sleep 10 && docker kill $did &> /dev/null && echo -n "timeout" >> $1/error.txt &
docker wait "$did" &> /dev/null
docker rm -f $ &> /dev/null

The container runs in detached mode (-d option), so it runs in the background. Then you run sleep also in the background. Then wait for the container to stop. If it doesnt stop in 10 seconds (sleep timer), the container will be killed.

As you can see, the docker run process calls a script named compilerun.sh:

#!/bin/bash
gcc -o /usercode/file /usercode/file.c 2> /usercode/error.txt && ./usercode/file < /usercode/input.txt | head -c 1M > /usercode/output.txt
maxsize=1048576
actualsize=$(wc -c <"/usercode/output.txt")
if [ $actualsize -ge $maxsize ]; then
    echo -e "1MB file size limit exceeded\n\n$(cat /usercode/output.txt)" > /usercode/output.txt
fi

It starts by compiling and running a C program (its my use case, I am sure the same can be done for python compiller).

This part:

command | head -c 1M > /usercode/output.txt

Is responsible for the max output size thing. It allows output to be 1MB maximum.

After that, I just check if file is 1MB. If true, write a message inside (at the beginning of) the output file.

Answer 4

You could set up your container with a ulimit on the max CPU time, which will kill the looping process. A malicious user can get around this, though, if they're root inside the container.

There's another S.O. question, "Setting absolute limits on CPU for Docker containers" that describes how to limit the CPU consumption of containers. This would allow you to reduce the effect of malicious users.

I agree with Abdullah, though, that you ought to be able to docker kill the runaway from your supervisor.