Storm使用遇到的一些问题【持续更新】

1、执行某个topology的时候,某个worker出现如下错误:

  2014-08-14 21:57:16 b.s.d.worker [INFO] Launching worker for dec-1-1408067377 on a1eb513a-51ac-440d-8c0a-704614450712:6703 with id 525827ef-eab8-4209-99a9-73fe1155714e and conf {"dev.zookeeper.path" "/root/host/apache-storm-0.9.2-incubating/workdir-zookeeper", ..."storm.messaging.netty.transfer.batch.size" 262144}
    2014-08-14 21:57:17 b.s.util [DEBUG] Touching file at /tmp/workdir/workers/525827ef-eab8-4209-99a9-73fe1155714e/pids/4749
    2014-08-14 21:57:18 b.s.d.worker [ERROR] Error on initialization of server mk-worker
    java.io.IOException: No such file or directory
                    at java.io.UnixFileSystem.createFileExclusively(Native Method) ~[na:1.6.0_45]

 

就是说 worker启动失败,从而导致topology执行不了,找了很多资料,也不知道什么原因,还在stackoverflow上发了个贴:

http://stackoverflow.com/questions/25320528/apache-storm-0-9-2-b-s-d-worker-error-error-on-initialization-of-server-mk-wor

配置里面,我配置了3个slots, 后来我发现我改成1个slots就好了(还不知道什么原因,求帮助~~~ 上面链接里面有我的配置)。

关于这个问题,我发现在windows上面可以支持多个slots,而且windows上面配置相当容易,建议初学者可以先在windows上进行试验

 


2、某个supervisor挂了,尝试启动,supervisor出现如下错误

2013-10-08 00:55:56 b.s.d.supervisor [INFO] Launching worker with assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id "CommentTopology-56-1380865400", :executors ([2 2] [5 5] [8 8] [11 11] [14 14] [17 17])} for this supervisor 5538ef5c-73df-4926-9ce9-6255e020b02e on port 6706 with id ff64b203-2242-4564-9b22-3fefcbf5f4af

2013-10-08 00:55:56 b.s.event [ERROR] Error when processing event
java.io.FileNotFoundException: File '/var/storm/supervisor/stormdist/CommentTopology-56-1380865400/stormconf.ser' does not exist

 

路人的解释:https://groups.google.com/forum/#!topic/storm-user/clZXsiLPReY  ,说是因为 supervisor’s *local-state* 过期了, 貌似有2种解决办法:

1、重启zookeeper
2、删除storm.local.dir目录中的supervisor和workers重启supervisor
第二种方法貌似可以,大家可以都试试,有人说0.9.0就已经解决了,我发现我用0.9.2还是一样,悲剧…
 

3、集群环境之下运行jar包 ,报如下的错

org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document 'http://www.springframework.org/schema/beans/spring-beans-3.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.

 

是因为打包的工具有点问题,一般用maven的assembly插件就可以了,但是它有个bug:具体情况参:http://jira.codehaus.org/browse/MASSEMBLY-360该bug产生的原因如下:工程一般依赖了很多的jar包,而被依赖的jar又会依赖其他的jar包,这样,当工程中依赖到不同的版本的spring时,在使用assembly进行打包时,只能将某一个版本jar包下的spring.schemas文件放入最终打出的jar包里,这就有可能遗漏了一些版本的xsd的本地映射,所以会报错,最后大家可以用maven-shade-plugin插件解决这个问题
一些参考资料
http://blog.csdn.net/program_think/article/details/6687076  介绍zmq

http://kisseveryone.iteye.com/blog/1728187 安装zmq

http://www.cnblogs.com/panfeng412/archive/2012/11/30/how-to-install-and-deploy-storm-cluster.html 安装storm

 

 

发表评论

电子邮件地址不会被公开。 必填项已用*标注

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax

您可以使用这些HTML标签和属性: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>