How to run OpenAI Gym .render() over a server

Question

I am running a python 2.7 script on a p2.xlarge AWS server through Jupyter (Ubuntu 14.04). I would like to be able to render my simulations.

Minimal working example

import gym
env = gym.make('CartPole-v0')
env.reset()
env.render()

env.render() makes (among other things) the following errors:

...
HINT: make sure you have OpenGL install. On Ubuntu, you can run 
'apt-get install python-opengl'. If you're running on a server, 
you may need a virtual frame buffer; something like this should work: 
'xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>'")
...
NoSuchDisplayException: Cannot connect to "None"

I would like to some how be able to see the simulations. It would be ideal if I could get it inline, but any display method would be nice.

Edit: This is only an issue with some environments, like classic control.

---

Update I

Inspired by this I tried the following, instead of the xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py> (which I couldn't get to work).

xvfb-run -a jupyter notebook

Running the original script I now get instead

GLXInfoException: pyglet requires an X server with GLX

---

Update II

Issue #154 seems relevant. I tried disabling the pop-up, and directly creating the RGB colors

import gym
env = gym.make('CartPole-v0')
env.reset()

img = env.render(mode='rgb_array', close=True)  
print(type(img)) # <--- <type 'NoneType'>

img = env.render(mode='rgb_array', close=False) # <--- ERROR
print(type(img)) 

I get ImportError: cannot import name gl_info.

---

Update III

With inspiration from @Torxed I tried creating a video file, and then rendering it (a fully satisfying solution).

Using the code from 'Recording and uploading results'

import gym

env = gym.make('CartPole-v0')
env.monitor.start('/tmp/cartpole-experiment-1', force=True)
observation = env.reset()
for t in range(100):
#    env.render()
    print(observation)
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done:
        print("Episode finished after {} timesteps".format(t+1))
        break

env.monitor.close()

I tried following your suggestions, but got ImportError: cannot import name gl_info from when running env.monitor.start(....

From my understanding the problem is that OpenAI uses pyglet, and pyglet 'needs' a screen in order to compute the RGB colors of the image that is to be rendered. It is therefore necessary to trick python to think that there is a monitor connected

---

Update IV

FYI there are solutions online using bumblebee that seem to work. This should work if you have control over the server, but since AWS run in a VM I don't think you can use this.

---

Update V

Just if you have this problem, and don't know what to do (like me) the state of most environments are simple enough that you can create your own rendering mechanism. Not very satisfying, but.. you know.

Answer 1

I ran into this myself. Using xvfb as X-server somehow clashes with the Nvidia drivers. But finally this post pointed me into the right direction. Xvfb works without any problems if you install the Nvidia driver with the -no-opengl-files option and CUDA with --no-opengl-libs option. If you know this, it should work. But as it took me quite some time till I figured this out and it seems like I'm not the only one running into problems with xvfb and the nvidia drivers.

I wrote down all necessary steps to set everything up on an AWS EC2 instance with Ubuntu 16.04 LTS here.

Answer 2

I avoided the issues with using matplotlib by simply using PIL, Python Image Library:

import gym, PIL
env = gym.make('SpaceInvaders-v0')
array = env.reset()
PIL.Image.fromarray(env.render(mode='rgb_array'))

I found that I didn't need to set the XV frame buffer.

Answer 3

Got a simple solution working:

If on a linux server, open jupyter with

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

In Jupyter

import matplotlib.pyplot as plt
%matplotlib inline
from IPython import display

After each step

def show_state(env, step=0, info=""):
    plt.figure(3)
    plt.clf()
    plt.imshow(env.render(mode='rgb_array'))
    plt.title("%s | Step: %d %s" % (env._spec.id,step, info))
    plt.axis('off')

    display.clear_output(wait=True)
    display.display(plt.gcf())

Note: if your environment is not unwrapped, pass env.env to show_state.

Answer 4

There's also this solution using pyvirtualdisplay (an Xvfb wrapper). One thing I like about this solution is you can launch it from inside your script, instead of having to wrap it at launch:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(1400, 900))
display.start()

Answer 5

I had the same problem and I_like_foxes solution to reinstall nvidia drivers with no opengl fixed things. Here are the commands I used for Ubuntu 16.04 and GTX 1080ti https://gist.github.com/8enmann/931ec2a9dc45fde871d2139a7d1f2d78

Answer 6

I was looking for a solution that works in Colaboratory and ended up with this

from IPython import display
import numpy as np
import time

import gym
env = gym.make('SpaceInvaders-v0')
env.reset()

import PIL.Image
import io


def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

while True:
    time.sleep(0.01)
    env.step(env.action_space.sample()) # take a random action
    display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

EDIT 1:

You could use xvfbwrapper for the Cartpole environment.

from IPython import display
from xvfbwrapper import Xvfb
import numpy as np
import time
import pyglet
import gym
import PIL.Image
import io    

vdisplay = Xvfb(width=1280, height=740)
vdisplay.start()

env = gym.make('CartPole-v0')
env.reset()

def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


for _ in range(1000):
  time.sleep(0.01)
  observation, reward, done, info = env.step(env.action_space.sample()) # take a random action
  display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


vdisplay.stop()

If you're working with standard Jupyter, there's a better solution though. You can use the CommManager to send messages with updated Data URLs to your HTML output.

IPython Inline Screen Example

In Colab the CommManager is not available. The more restrictive output module has a method called eval_js() which seems to be kind of slow.