jvm-profiler | JVM Profiler Sending Metrics to Kafka , Console Output | Monitoring library

by uber-common Java Version: Current License: Non-SPDX

X-Ray Key Features Code Snippets Community Discussions(2)Vulnerabilities Install Support

kandi X-RAY | jvm-profiler Summary

jvm-profiler is a Java library typically used in Performance Management, Monitoring applications. jvm-profiler has no bugs, it has no vulnerabilities, it has build file available and it has medium support. However jvm-profiler has a Non-SPDX License. You can download it from GitHub.

Uber JVM Profiler provides a Java Agent to collect various metrics and stacktraces for Hadoop/Spark JVM processes in a distributed way, for example, CPU/Memory/IO metrics. Uber JVM Profiler also provides advanced profiling capabilities to trace arbitrary Java methods and arguments on the user code without user code change requirement. This feature could be used to trace HDFS name node call latency for each Spark application and identify bottleneck of name node. It could also trace the HDFS file paths each Spark application reads or writes and identify hot files for further optimization. This profiler is initially created to profile Spark applications which usually have dozens of or hundreds of processes/machines for a single application, so people could easily correlate metrics of these different processes/machines. It is also a generic Java Agent and could be used for any JVM process as well.

Support

Quality

Security

License

Reuse

Support

jvm-profiler has a medium active ecosystem.

It has 1697 star(s) with 335 fork(s). There are 104 watchers for this library.

It had no major release in the last 6 months.

There are 10 open issues and 27 have been closed. On average issues are closed in 135 days. There are 4 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of jvm-profiler is current.

Quality

jvm-profiler has 0 bugs and 0 code smells.

Security

jvm-profiler has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

jvm-profiler code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

jvm-profiler has a Non-SPDX License.

Non-SPDX licenses can be open source with a non SPDX compliant license, or non open source licenses, and you need to review them closely before use.

Reuse

jvm-profiler releases are not available. You will need to build from source code and install.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

jvm-profiler saves you 3763 person hours of effort in developing the same functionality from scratch.

It has 8067 lines of code, 480 functions and 97 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed jvm-profiler and discovered the below as its top functions. This is intended to give you an instant insight into jvm-profiler implemented functionality, and help decide if they suit your requirements.

Sample the cpu time
Get the attributes of the platform MBean
Get a long value from a map
Read the contents of the file as a map
Expect this instance into a profile
Execute a set of static methods
Extract the value at the start and index from a string
Get argument values from string
Report profiling data
Gets an object from the database
Queries a list of columns
Main entry point
Run the java agent
Updates connection properties from yaml file
Update connection properties from yaml file
Generate metrics
Gets an object by primary key
Updates connection properties from connection properties
Executes the given query
Update arguments
Generate profiling
Record profiling
Reports cpu and metrics
Dump all threads
Returns information about the process
Sets the profiling information

Get all kandi verified functions for this library.

jvm-profiler Key Features

No Key Features are available at this moment for jvm-profiler.

jvm-profiler Examples and Code Snippets

No Code Snippets are available at this moment for jvm-profiler.

Community Discussions

Trending Discussions on jvm-profiler

how to pass javaagent to emr spark applications?

spark.executor.extraJavaOptions ignored in spark-submit

QUESTION

how to pass javaagent to emr spark applications?

Asked 2019-Dec-08 at 10:41

I am trying to use uber jvm profiler to profile my spark application (spark 2.4, running on emr 5.21)

Following is my cluster configuration

...

ANSWER

Answered 2019-Dec-08 at 10:41

I haven't used Uber JVM Profiler but I think to add extra jars in spark-submit you should be using --jars option. And you can add them directly from S3 bucket when working on EMR.

Also, at bootstrap, you're copying the jar jvm-profiler-1.0.0.jar into /tmp folder but when you set Java options you didn't add the path. Try this :

Source https://stackoverflow.com/questions/59233394

QUESTION

spark.executor.extraJavaOptions ignored in spark-submit

Asked 2019-Sep-19 at 08:12

I am a newbie trying to profile a local spark job. Here is the command that I am trying to execute, but I am getting a warning stating my executor options are being ignored since they are non-spark config properties.

error:

Warning: Ignoring non-spark config property: “spark.executor.extraJavaOptions=javaagent:statsd-jvm-profiler-2.1.0-jar-with-dependencies.jar=server=localhost,port=8086,reporter=InfluxDBReporter,database=profiler,username=profiler,password=profiler,prefix=MyNamespace.MySparkApplication,tagMapping=namespace.application”

Command:

./bin/spark-submit --master local[2] --class org.apache.spark.examples.GroupByTest --conf “spark.executor.extraJavaOptions=-javaagent:statsd-jvm-profiler-2.1.0-jar-with-dependencies.jar=server=localhost,port=8086,reporter=InfluxDBReporter,database=profiler,username=profiler,password=profiler,prefix=MyNamespace.MySparkApplication,tagMapping=namespace.application” --name HdfsWordCount --jars /Users/shprin/statD/statsd-jvm-profiler-2.1.0-jar-with-dependencies.jar libexec/examples/jars/spark-examples_2.11-2.3.0.jar

Spark version : 2.0.3

Please let me know, how to solve this.

Thanks in Advance.

...

ANSWER

Answered 2018-Apr-24 at 14:02

I think the problem is the double quote you are using to specify the spark.executor.extraJavaOptions. It should have been a single quote.

./bin/spark-submit --master local[2] --conf 'spark.executor.extraJavaOptions=-javaagent:statsd-jvm-profiler-2.1.0-jar-with-dependencies.jar=server=localhost,port=8086,reporter=InfluxDBReporter,database=profiler,username=profiler,password=profiler,prefix=MyNamespace.MySparkApplication,tagMapping=namespace.application' --class org.apache.spark.examples.GroupByTest --name HdfsWordCount --jars /Users/shprin/statD/statsd-jvm-profiler-2.1.0-jar-with-dependencies.jar libexec/examples/jars/spark-examples_2.11-2.3.0.jar

Source https://stackoverflow.com/questions/50002027

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install jvm-profiler

This command creates jvm-profiler.jar file with the default reporters like ConsoleOutputReporter, FileOutputReporter and KafkaOutputReporter bundled in it. If you want to bundle the custom reporters like RedisOutputReporter or InfluxDBOutputReporter in the jar file then provide the maven profile id for that reporter in the build command. For example to build a jar file with RedisOutputReporter, you can execute mvn -P redis clean package command. Please check the pom.xml file for available custom reporters and their profile ids.
Make sure JDK 8+ and maven is installed on your machine.
Run: mvn clean package

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: