joblib:Parallel always was stuck in the end

2019-07-11 17:08发布

My parallel uses:

sdf = Parallel(n_jobs=3, verbose=1, pre_dispatch='1.5*n_jobs')(delayed(char_etl)(x,k1,k2,k3) for x in X)

X is a list .char_etl is a sring match function. version : python2.7.11,centos6.7,joblib 0.11 Every time was stuck,like this

[Parallel(n_jobs=3)]: Done 89 tasks | elapsed: 10.9s
[Parallel(n_jobs=3)]: Done 258 tasks | elapsed: 46.8s
[Parallel(n_jobs=3)]: Done 552 tasks | elapsed: 1.7min
[Parallel(n_jobs=3)]: Done 902 tasks | elapsed: 2.7min
[Parallel(n_jobs=3)]: Done 997 out of 1000 | elapsed: 2.9min remaining: 0.5s
Thanks.

0条回答
登录 后发表回答