Storm WordCount error: Pipe to subprocess seems to

2019-07-27 16:24发布

问题:

Storm 0.10.0
my previous question (Apache storm : Could not load main class org.apache.storm.starter.ExclamationTopology) which was solved.

hello I have a single node cluster up and running on my machine, the storm config file is as follows:(storm.yaml)

storm.zookeeper.servers:
#     - "server1"
#     - "server2"
  - "localhost" 
storm.zookeeper.port: 2181
nimbus.host: "localhost"
storm.local.dir: "/var/stormtmp" 
java.library.path: "/usr/local"
supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"

and I ran this WordCount topology on the cluster (found the topology here and simply ran it) https://dl.dropboxusercontent.com/s/kc933u6vz2crqkb/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar (which is python)

but on of the bolt is throwing the following error at port 6703 localhost

java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read. Serializer Exception: at backtype.storm.utils.ShellProcess.readShellMs

So I figured something was wrong in the topology so checked my WordCount-3-1457017776-worker-6701.log (file) and found this:

b.s.d.executor [INFO] TRANSFERING tuple TASK: 8 TUPLE: source: split:18, stream: default, id: {}, ["moon"]
b.s.d.executor [INFO] BOLT ack TASK: 18 TIME:  TUPLE: source: spout:25, stream: default, id: {}, [the cow jumped over the moon]
b.s.t.ShellBolt [INFO] ShellLog pid:1714, name:split Traceback (most recent call last):
  File "/var/stormtmp/supervisor/stormdist/WordCount-3-1457017776/resources/storm.py", line 172, in run
    self.process(tup)
  File "splitsentence.py", line 5, in process
    words = tup.values[0].split(" ")
IndexError: list index out of range

b.s.t.ShellBolt [ERROR] Halting process: ShellBolt died. Command: [python, splitsentence.py], ProcessInfo pid:1714, name:split exitCode:0, errorString:
java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:

        at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:101) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:321) [storm-core-0.10.0.jar:0.10.0]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_95]
b.s.d.executor [ERROR]
java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:

        at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:101) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:321) [storm-core-0.10.0.jar:0.10.0]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_95]

So I belive the index out of range(occured due to line 5, tuple becomming 0) is causing the bolt to die and the pipe to it is broken, so I am not able to do futher processing of data, Is my understanding about the issue correct ? and is there a soulution to this ? or may be a different topology I can test on? Please help me out this is my first topology running on storm