profiling | discriminatory profiling of Ruby code | Monitoring library
kandi X-RAY | profiling Summary
kandi X-RAY | profiling Summary
Non-discriminatory profiling for your MRI Ruby code. This gem is a small wrapper around the ruby-prof gem, which is its only dependency. It lets you do simple but powerful profiling of your friend's bad code.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of profiling
profiling Key Features
profiling Examples and Code Snippets
Community Discussions
Trending Discussions on profiling
QUESTION
In the kubernetes source code there is a block of code that handles the profiling part but I can not acces the endpoints:
...ANSWER
Answered 2021-Jun-11 at 13:29Try:
QUESTION
I have a microservice written in Haskell, the compiler is 8.8.3.
I built it with --profile
option and ran it with +RTS -p
. It is running about 30 minutes, there is .prof
file but it is empty (literally 0 bytes). Previously I did it on my local machine and I stop the service with CTRL-C and after the exit it produced .prof
file which was not empty.
So, I have 2 questions:
- How to collect profiling information when a Haskell microservice runs under Kubernetes in the most correct way (to be able to read this .prof file)?
- How to pass run time parameter to Haskell run-time where to save this .prof file (maybe some workaround if no such an option), for 8.8.3 - because I have feeling that the file may be big and I can hit disk space problem. Also I don't know how to flush/read/get this file while microservice is running. I suppose if I will be able to pass full path for this .prof file then I can save it somewhere else on some permanent volume, to "kill" the service with
INT
signal for example, and to get this .prof file from the volume.
What is the usual/convenient way to get this .prof file when the service runs in Kubernetes?
PS. I saw some relevant options in the documentation for newest versions, but I am with 8.8.3
...ANSWER
Answered 2021-Jun-09 at 16:24I think the only way to do live profiling with GHC is to use the eventlog. You can insert Debug.Trace.traceEvent
into your code at the functions that you want to measure and then compile with -eventlog
and run with +RTS -l -ol -RTS
. You can use ghc-events-analyze
to analyze and visualize the produced eventlog.
The official eventlog documentation for GHC 8.8.3 is here.
QUESTION
Full project at https://github.com/ysangkok/cabal-profiling-issue
The project contains scaffolding generated by cabal init
. I'll paste the most interesting source snippets now.
In Main.hs
I have:
ANSWER
Answered 2021-Jun-08 at 05:51It is a known problem about compile multi-module program that contains TH code for profiling, see related sections in the documentation:
This causes difficulties if you have a multi-module program containing Template Haskell code and you need to compile it for profiling, because GHC cannot load the profiled object code and use it when executing the splices.
As a workaround, just put TemplateHaskell
into the other-modules
in your test.cabal,
QUESTION
I'm building a java CLI utility application that processes some data from a file.
Apart from reading from a file, all the operations are done in-memory. The in-memory processing part is taking a surprisingly long time so I tried profiling it but could not pinpoint any specific function that performed particularly bad.
I was afraid that JIT was not able to optimize the program during a single run, so I benchmarked how the runtime changes between the consecutive executions of the function with all the program logic (including reading the input file) and sure enough, the runtime for the in-memory processing part goes down for several executions and becomes almost 10 times smaller already on the 5th run.
I tried shuffling the input data before every execution, but it doesn't have any visible effect on this. I'm not sure if some caching may be responsible for this improvement or the JIT optimizations done during the program run, but since usually the program is ran once at time, it always shows the worst performance.
Would it be possible to somehow get a good performance during the first run? Is there a generic way to optimize performance for a short-running java applications?
...ANSWER
Answered 2021-Jun-06 at 12:22You probably cannot optimize startup time and performance by changing your application1, 2. And especially for a small application3. And I certainly don't think there are "generic" ways to do it; i.e. optimizations that will work for all cases.
However, there are a couple of JVM features that should improve performance for a short-lived JVM.
Class Data Sharing (CDS) is a feature that allows JIT compiled classes to be cached in the file system (as a CDS archive) and which is then reused by later of runs of your application. This feature has been available since Java 5 (though with limitations in earlier Java releases).
The CDS feature is controlled using the -Xshare
JVM option.
-Xshare:dump
generates a CDS archive during the run-Xshare:off
-Xshare:on
and-Xshare:auto
control whether an existing CDS archive will be used.
The other way to improve startup times for a HotSpot JVM is (was) to use Ahead Of Time (AOT) compilation. Basically, you compile your application to a native code binary using the jaotc
command, and then run the executable it produces rather than the java
command. The jaotc
command is experimental and was introduced in Java 9.
It appears that jaotc
was not included in the Java 16 builds published by Oracle, and is scheduled for removal in Java 17. (See JEP 410: Remove the Experimental AOT and JIT Compiler).
The current recommended way to get AOT compilation for Java is to use the GraalVM AOT Java compiler.
1 - You could convert into a client-server application where the server "up" all of the time. However, that has other problems, and doesn't eliminate the startup time issue for the client ... assuming that is coded in Java.
2 - According to @apangin, there are some other application tweaks that may could make you code more JIT friendly, though it will depend on what you code is currently doing.
3 - It is conceivable that the startup time for a large (long running) monolithic application could be improved by refactoring it so that subsystems of the application can be loaded and initialized only when they are needed. However, it doesn't sound like this would work for your use-case.
QUESTION
I'm trying to use pandas profiling
on titanic dateset.
Under the overview section there are some features with caption "HIGH CORRELATION
"
- I know what is the meaning of correlation, but the caption doesn't tell which feature is correlated to this feature ?
- So what is the meaning of "
HIGH CORRELATION
" in the pandas profiling doc ?
ANSWER
Answered 2021-Jun-06 at 04:25QUESTION
I am going to write my own FORTH "engine" in GNU assembler (GAS) for Linux x86-64 (specifically for AMD Ryzen 9 3900X that is siting on my table).
(If it will be success, I may use similar idea for make firmware for retro 6502 and similar home-brewed computer)
I want to add some interesting debugging features, as saving comments with the compiled code in for of "NOP words" with attached strings, which would do nothing in runtime, but when disassembling/printing out already defined words it would print those comment too, so it would not loose all the headers ( a b -- c) and comments like ( here goes this particular little trick ) and I would be able try to define new words with documentation, and later print all definitions in some nice way and make new library from those, which I consider good. (And have switch to just ignore comments for "production release")
I had read too much of optimalization here and I am not able to understand all of that in few weeks, so I will put out microoptimalisation until it will suffer performance problems and then I will start with profiling.
But I want to start with at least decent architectural decisions.
What I understood yet:
- it would be nice, if the programs was run mainly from CPU cache, not from memory
- the cache is filled somehow "automagically", but having related data/code compact and as near as possible may help a lot
- I identified some areas, that would be good candidates for caching and some, that are not so good - I sorted it in order of importance:
- assembler code - the engine and basic words like "+" - used all the time (fixed size, .text section)
- both stacks - also used all the time (dynamic, I will probably use rsp for data stack and implement return stack independly - not sure yet, which will be "native" and which "emulated")
- forth bytecode - the defined and compiled words - used at runtime, when the speed matters (still growing size)
- variables, constants, strings, other memory allocations (used in runtime)
- names of words ("DUP", "DROP" - used only when defining new words in compilation phase)
- comments (used one daily or so)
As there is lot of "heaps" that grows up (well, there is not "free" used, so it may be also stack, or stack growing up) (and two stacks that grows down) I am unsure how to implement it, so the CPU cache will cover it somehow decently.
My idea is to use one "big heap" (and increse it with brk() when needed), and then allocate big chunks of alligned memory on it, implementing "smaller heaps" in each chunk and extend them to another big chunk when the old one is filled up.
I hope, that the cache would automagically get the most used blocks first keep it most of the time and the less used blocks would be mostly ignored by the cache (respective it would occupy only small parts and get read and kicked out all the time), but maybe I did not it correctly.
But maybe is there some better strategy for that?
...ANSWER
Answered 2021-Jun-04 at 23:53Your first stops for further reading should probably be:
- What Every Programmer Should Know About Memory? re: cache
- https://agner.org/optimize/ re: everything else about writing efficient asm.
- https://uops.info/ for a better version of Agner Fog's instruction tables.
- See also other links in https://stackoverflow.com/tags/x86/info
so I will put out microoptimalisation until it will suffer performance problems and then I will start with profiling.
Yes, probably good to start trying stuff so you have something to profile with HW performance counters, so you can correlate what you're reading about performance stuff with what actually happens. And so you get some ideas of possible details you hadn't thought of yet before you go too far into optimizing your overall design idea. You can learn a lot about asm micro-optimization by starting with something very small scale, like a single loop somewhere without any complicated branching.
Since modern CPUs use split L1i and L1d caches and first-level TLBs, it's not a good idea to place code and data next to each other. (Especially not read-write data; self-modifying code is handled by flushing the whole pipeline on any store too near any code that's in-flight anywhere in the pipeline.)
Related: Why do Compilers put data inside .text(code) section of the PE and ELF files and how does the CPU distinguish between data and code? - they don't, only obfuscated x86 programs do that. (ARM code does sometimes mix code/data because PC-relative loads have limited range on ARM.)
Yes, making sure all your data allocations are nearby should be good for TLB locality. Hardware normally uses a pseudo-LRU allocation/eviction algorithm which generally does a good job at keeping hot data in cache, and it's generally not worth trying to manually clflushopt
anything to help it. Software prefetch is also rarely useful, especially in linear traversal of arrays. It can sometimes be worth it if you know where you'll want to access quite a few instructions later, but the CPU couldn't predict that easily.
AMD's L3 cache may use adaptive replacement like Intel does, to try to keep more lines that get reused, not letting them get evicted as easily by lines that tend not to get reused. But Zen2's 512kiB L2 is relatively big by Forth standards; you probably won't have a significant amount of L2 cache misses. (And out-of-order exec can do a lot to hide L1 miss / L2 hit. And even hide some of the latency of an L3 hit.) Contemporary Intel CPUs typically use 256k L2 caches; if you're cache-blocking for generic modern x86, 128kiB is a good choice of block size to assume you can write and then loop over again while getting L2 hits.
The L1i and L1d caches (32k each), and even uop cache (up to 4096 uops, about 1 or 2 per instruction), on a modern x86 like Zen2 (https://en.wikichip.org/wiki/amd/microarchitectures/zen_2#Architecture) or Skylake, are pretty large compared to a Forth implementation; probably everything will hit in L1 cache most of the time, and certainly L2. Yes, code locality is generally good, but with more L2 cache than the whole memory of a typical 6502, you really don't have much to worry about :P
Of more concern for an interpreter is branch prediction, but fortunately Zen2 (and Intel since Haswell) have TAGE predictors that do well at learning patterns of indirect branches even with one "grand central dispatch" branch: Branch Prediction and the Performance of Interpreters - Don’t Trust Folklore
QUESTION
I have an NVidia GeForce GTX 770 and would like to use its CUDA capabilities for a project I am working on. My machine is running windows 10 64bit.
I have followed the provided CUDA Toolkit installation guide: https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/.
Once the drivers were installed I opened the samples solution (using Visual Studio 2019) and built the deviceQuery and bandwidthTest samples. Here is the output:
deviceQuery:
...ANSWER
Answered 2021-Jun-04 at 04:13Your GTX770 GPU is a "Kepler" architecture compute capability 3.0 device. These devices were deprecated during the CUDA 10 release cycle and support for them dropped from CUDA 11.0 onwards
The CUDA 10.2 release is the last toolkit with support for compute 3.0 devices. You will not be able to make CUDA 11.0 or newer work with your GPU. The query and bandwidth tests use APIs which don't attempt to run code on your GPU, that is why they work where any other example will not work.
QUESTION
I'm looking at profiling the score calculation in my Optaplanner project to find out if there are any hotspots that would benefit from being optimised. However, visualvm shows most of the time to be taken in the self time of org.drools.modelcompiler.constraints.ConstraintEvaluator$InnerEvaluator$_2.evaluate
. I therefore assume that this method is what actually runs a lot of the constraint's code. What is the best way to find out what specific pieces of code are taking the most time?
ANSWER
Answered 2021-Jun-01 at 13:54The thing to understand about Constraint Streams is that it is not imperative programming, and therefore traditional performance optimization techniques such as code profiling are not going to be very helpful. Instead, I suggest you think of Constraint Streams as SQL - the way to have fast SQL is to think about how your data flows, how you join and what gets indexed.
Recently I wrote a blog post explaining the tricks behind making CS run fast. However, CS is internally interpreted by the Drools engine, and therefore studying it may give you some insights too. Not all insights there are applicable to CS, but if you take a look at drools-metric
, you should be able to see which constraints are comparatively slow. And then it becomes a game of tweaking.
QUESTION
I have been trying to deploy a hyperledger fabric model with 3 CAs 1 orderer and 2 peer nodes. I am able to create the channel with OSADMIN command of fabric but when I try to join the channel with peer node, I get Error: error getting endorser client for channel: endorser client failed to connect to peer-govt:7051: failed to create new connection: context......
.
Here are the logs from terminal (local host machine):
...ANSWER
Answered 2021-Jun-01 at 10:33I have fixed it. The issue I was facing was because of not setting the CORE_PEER_TLS_ENABLED = true
for CLI pod.
One thing I have got learn from this whole model, whenever you see TLS issue, first to check for would be checking CORE_PEER_TLS_ENABLED
variable. Make sure you have set it for all the pods or containers you are trying to interact with. The case can be false(for no TLS) or true(for using TLS) depending on your deployment.
Other things to keep in mind is using the correct variables of fabric including FABRIC_CFG_PATH
, CORE_PEER_LOCALMSPID
, CORE_PEER_TLS_ROOTCERT_FILE
, CORE_PEER_MSPCONFIGPATH
and some others depending on your command.
QUESTION
When I want to use NVProf for NCCL problem with --metrics all, The profiling results always return me like
...ANSWER
Answered 2021-May-28 at 15:37That behavior is expected.
events, metrics, that are gathered by default pertain to CUDA device code activity. To see something that might be instructive, try profiling with --print-gpu-trace
switch (and remove --metrics all
).
The documented "metrics" don't apply to the operations (data copying) that NCCL is doing. They apply to CUDA kernels (i.e. CUDA device code activity).
nvprof
does seem to have metrics that can be collected for NVLink activity. To see these, on a system that is applicable (e.g. has NVLink), run a command such as:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install profiling
Profile slow code from your friend or colleague like this:. The next time you call the code it will be profiled and three files will be written into a directory called profiling.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page