Dask Distributed client takes to long to initializ

2019-07-11 03:11发布

问题:

Trying to initialize a client with local cluster in Jupyter lab but hangs. This behaviour happens for python 3.5 and jupyter lab 0.35.

import dask.dataframe as dd
from dask import delayed
from distributed import Client
from distributed import LocalCluster
import pandas as pd
import numpy as np
import json

cluster = LocalCluster()
client = Cluster(cluster)
client

versions of the tools :

Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

And the jupyter lab version is

pip list |grep jupyter
jupyter            1.0.0  
jupyter-client     5.2.3  
jupyter-console    6.0.0  
jupyter-core       4.4.0  
jupyterlab         0.35.3 
jupyterlab-server  0.2.0  

Dask version

pip list |grep dask
dask               0.20.0 

What could be wrong?

EDIT: No exception on the terminal while trying to initialize client Interrupting the Kernel gives the following exceptions: Inside jupyter:

distributed.nanny - ERROR - Failed to restart worker after its process exited
Traceback (most recent call last):
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 291, in _on_exit
    yield self.instantiate()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 226, in instantiate
    self.process.start()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/nanny.py", line 370, in start
    yield self.process.start()
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/process.py", line 35, in _call_and_set_future
    res = func(*args, **kwargs)
  File "/home/avlach/virtualenvs/staistics3/lib/python3.5/site-packages/distributed/process.py", line 184, in _start
    process.start()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.5/multiprocessing/context.py", line 281, in _Popen
    return Popen(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_forkserver.py", line 36, in __init__
        super().__init__(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.5/multiprocessing/popen_forkserver.py", line 52, in _launch
        self.sentinel, w = forkserver.connect_to_new_process(self._fds)
      File "/usr/lib/python3.5/multiprocessing/forkserver.py", line 66, in connect_to_new_process
    client.connect(self._forkserver_address)
    ConnectionRefusedError: [Errno 111] Connection refused

and on the terminal

Kernel interrupted: 1e7da6ed-2137-41ce-818e-3484c0a659cc
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt