storm-starter | Learn to use Storm! | Learning library
kandi X-RAY | storm-starter Summary
kandi X-RAY | storm-starter Summary
Learn to use Storm!
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Main method
- Construct a planology builder
- Submit a Topology
- Build topology topology
- Entry point for the reachology
- Build storm topology
- Entry point for testing only
- Returns a string representation of this object
- Compares this rankable with another
- Initializes the last modified times
- Generate a random sentence
- Returns a 16 - bit hash code
- Wrap the topology
- Main entry point for testing
- Submit a topology
- Executes a single join
- Entry point for testing
- Prepare topology
- Launch a topology
- Compares this object with the specified object
storm-starter Key Features
storm-starter Examples and Code Snippets
Community Discussions
Trending Discussions on storm-starter
QUESTION
Currently learning on how to use Storm (version 2.1.0), I am a bit confused on a specific aspect of this data streaming processing (DSP) engine: How is output data handled? Tutorials provide good explanations on system setup and running our first application. Unfortunately, I didn't find a page providing details on results generated by a topology.
With DSP applications, there are no final output because input data is a continuously incoming stream of data (or maybe we can say there is a final output when application is stopped). What I would like is to be able to see the state of current output (the actual output data generated at current time) of a running topology.
I'm able to run WordCountTopology. I understand the output of this topology is generated by the following snippet of code:
...ANSWER
Answered 2019-Dec-16 at 14:42A few days have passed since I posted this question. I am back to share with you what I have tried. Although I cannot tell if it is the right way of doing, the two following propositions answer my question.
Simple System.out.println()The first thing I've tried is to make a System.out.println("Hello World!") directly within the prepare() method of my BaseBasicBolt. This method is called only once at the beginning of each Bolt's thread execution.
QUESTION
I tried to create a small example in Trident. The goal was to see how tuples are replayed in Case of failures. Below is the topology definition
...ANSWER
Answered 2019-Nov-21 at 20:06Trident spouts don't ack or fail tuples at the individual tuple level. Instead, tuples are acked as a batch.
Trident spouts will often look something like this interface.
QUESTION
I want to use storm-starter in intellij
I config strict as the offical site https://github.com/apache/storm/tree/master/examples/storm-starter said
but the intellj tell me lots of dependency can't find
...ANSWER
Answered 2019-Feb-27 at 12:16You are using the storm-starter from Storm 2.0.1. Storm 2.0.1 hasn't been released yet, so you can't download finished jars via Maven. Either build Storm yourself, or use the storm-starter from one of the released versions.
QUESTION
I'm new to Apache Storm and trying to get my feet wet.
Right now I simply want to log or print incoming Kafka messages which are received as byte arrays of ProtoBuf objects.
I need to do this within a Java Spring application.
I'm using Kafka 0.11.0.2
I'm using Storm 1.1.2 and have storm-core, storm-kafka, and storm-starters in my pom.
Main service class example
...ANSWER
Answered 2018-Aug-08 at 06:15Either declare PrinterBolt in a new file, or make the class static. The problem you're running into is that PrinterBolt is a non-static inner class of MyService, which means it contains a reference to the outer MyService class. Since MyService isn't serializable, PrinterBolt isn't either. Storm requires bolts to be serializable.
Also unrelated to the error you're seeing, you might want to consider using storm-kafka-client over storm-kafka, since the latter is deprecated.
QUESTION
I pulled the latest docker image from Docker Hub for Storm 1.2.1. And packaged the storm-starter example project from Storm source code 1.2.1 according to the README documentation under the project, and renamed the package as "topology.jar". But when I submitted the jar file into Storm docker with
$ docker run -it -v $(pwd)/topology.jar:/topology.jar storm storm jar /topology.jar org.apache.storm.starter.ExclamationTopology,
the error occurred:
Running: /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/apache-storm-1.2.1 -Dstorm.log.dir=/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /apache-storm-1.2.1/:/apache-storm-1.2.1/lib/:/apache-storm-1.2.1/extlib/*:/topology.jar:/conf:/apache-storm-1.2.1/bin -Dstorm.jar=/topology.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.storm.starter.ExclamationTopology
Error: Could not find or load main class org.apache.storm.starter.ExclamationTopology
And I added a HelloWorld class under the same package as the ExclamationTopology is which just print "Hello world!" in the main method:
public static void main (String[] args) {
System.out.println("Hello world!");
}
When I submit this new package with $ docker run -it -v $(pwd)/topology.jar:/topology.jar storm storm jar /topology.jar org.apache.storm.starter.HelloWorld, it printed "Hello world!" as expected.
So what is the problem? Any help is appreciated.
PS.
/META-INFO/MANIFEST.MF in the topology.jar
...
ANSWER
Answered 2018-Mar-08 at 14:42Finally, I got it. The official docker image on Docker Hub for Storm 1.2.1 misses the class called ConfigurableTopology. Copy it from storm-client as the local one, then recompile the package. You can submit it successfully.
QUESTION
I'm really new to maven and storm so I'm trying to follow the instructions in https://github.com/apache/storm/tree/master/examples/storm-starter. My current path is /home/luc/theTest/storm/examples/storm-starter. Inside the target folder there is a storm-starter-2.0.0-SNAPSHOT.jar file. I'm getting stuck when running
storm jar target/storm-starter-*.jar org.apache.storm.starter.ExclamationTopology -local
I get this error
ionTopology -local Running: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/home/luc/stormTest/apache-storm-1.1.1 -Dstorm.log.dir=/home/luc/stormTest/apache-storm-1.1.1/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/luc/stormTest/apache-storm-1.1.1/lib/servlet-api-2.5.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/slf4j-api-1.7.21.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/objenesis-2.1.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/kryo-3.0.3.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/log4j-core-2.8.2.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/log4j-over-slf4j-1.6.6.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/storm-core-1.1.1.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/log4j-slf4j-impl-2.8.2.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/minlog-1.3.0.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/log4j-api-2.8.2.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/clojure-1.7.0.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/ring-cors-0.1.5.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/asm-5.0.3.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/reflectasm-1.10.1.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/disruptor-3.3.2.jar:/home/luc/stormTest/apache-storm-1.1.1/lib/storm-rename-hack-1.1.1.jar:target/storm-starter-2.0.0-SNAPSHOT.jar:/home/luc/stormTest/apache-storm-1.1.1/conf:/home/luc/stormTest/apache-storm-1.1.1/bin -Dstorm.jar=target/storm-starter-2.0.0-SNAPSHOT.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.storm.starter.ExclamationTopology -local Error: Could not find or load main class org.apache.storm.starter.ExclamationTopology
Am I doing something wrong? I'm also a bit confused on whether I have to run the nimbus and supervisor first. I tried with and without them and neither worked anyways. Been searching the web but nothing works. Not sure what else to try.
...ANSWER
Answered 2017-Oct-30 at 23:13This is usually caused by inconsistent versions of storm-client and storm-starter. Try following these steps to get the example working.
download the latest release from
http://storm.apache.org/downloads.html- in this example, we will use version 1.1.1
- in this example, we will use version 1.1.1
extract this to a folder, lets call it
${STORM_HOME}cd into
${STORM_HOME}/examples/storm-starterexecute
mvn package -DskipTests=true- this should build the storm-starter jar in target folder
${STORM_HOME}/examples/storm-starter/target/storm-starter-1.1.1.jar
- this should build the storm-starter jar in target folder
run the example from
${STORM_HOME}directory:
./bin/storm jar examples/storm-starter/target/storm-starter-1.1.1.jar org.apache.storm.starter.ExclamationTopology
don't add the -local flag since it seems like the ExclamationTopology is only deployed in a LocalCluster if no args are passed. You can check the source code here: ${STORM_HOME}/examples/storm-starter/src/jvm/org/apache/storm/starter/ExclamationTopology.java
QUESTION
General: I'm a student who want to run some performance tests (WordCount) on Storm / Kafka / Flink / MS Azure SA / Spark. I want to use the Kafka Broker as an input source.
I used the WordCount Example from the Storm-Starter project and added Kafka as a spout:
...ANSWER
Answered 2017-Feb-04 at 17:57Deleting "config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 2);" did the job!
QUESTION
I've just installed storm 1.0.2 and when I try to compile the storm-starter package, it results in a build failure:
...ANSWER
Answered 2017-Jan-26 at 05:13Moving comments to the answer.
Storm root pom handles some works so you might need to "build from root" (via executing mvn clean install with/without -DskipTests) first.
After that you can build storm-starter via mvn clean install from storm-starter directory.
"Root directory" means the directory which contains doap_Storm.rdf, LICENSE, NOTICE, etc.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install storm-starter
You can use storm-starter like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the storm-starter component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
