Chronicle-Queue | Micro second messaging that stores everything to disk
kandi X-RAY | Chronicle-Queue Summary
kandi X-RAY | Chronicle-Queue Summary
Micro second messaging that stores everything to disk
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Writes bytes to the stream
- Adds a position to the stream
- Writes bytes to the queue
- Sets the position for a given sequence
- Write the given marshallable to the Writer
- Adds a position to the stream
- Writes bytes to the queue
- Sets the position for a given sequence
- Writes the bytes at the given index
- Adds a position to the stream
- Writes bytes to the queue
- Sets the position for a given sequence
- Increment the index
- Moves the scan to the given index
- Acquires a long value for the given key
- Runs the tailer thread
- Overrides this instance with the specified metadata
- Load the write position value
- Iterate over all entries in the table
- Runs the task
- Refreshes the cycle queue
- Parses a file into a long
- Closes the store
- Get a value from the cache
- Moves the iterator to the beginning of the stream
- Parses a queue offset specification
- Refresh the file store
- Initialise the mappings for deprecated test cycles
- Cleans the store files without data
- Process a message history
- Move backward to the tail of the document
- Release resources
Chronicle-Queue Key Features
Chronicle-Queue Examples and Code Snippets
Community Discussions
Trending Discussions on Chronicle-Queue
QUESTION
this is the version of Chronicle Queue:
...ANSWER
Answered 2021-Nov-18 at 08:28try the following: you can find the documentation in https://github.com/OpenHFT/Chronicle-Queue
QUESTION
I have an application using Boot Strap running with cassandra 4.0, Cassandra java drive 4.11.1, spark 3.1.1 into ubuntu 20.4 with jdk 8_292 and python 3.6.
When I run a function that it call CQL by spark, the tomcat gave me the error bellow.
Stack trace:
...ANSWER
Answered 2021-May-25 at 23:23I openned two JIRA to understand this problem. See the links below:
QUESTION
One of our system has a micro service architecture using Apache Kafka as a service bus. Low latency is a very important factor but reliability and consistency (exactly once) are even more important.
When we perform some load tests we noticed signifiant performance degradation and all investigations pointed to big increases in Kafka topics producer and consumer latencies. No matter how much configuration we changed or more resources we added we could not get rid of the symptoms.
At the moment our needs are processing 10 transactions per second (TPS) and the load test is exercising 20 TPS but as the system is evolving and adding more functionality we know we'll reach a stage when the need will be 500TPS so we started being worried if we can achieve this with Kafka.
As a proof of concept I tried to switch to one of our micro services to use a chronicle-queue instead of a Kafka topic. It was easy to migrate following the avro example as from Chronicle-Queue-Demo git hub repo
...ANSWER
Answered 2021-Apr-20 at 01:55Hand building the Avro object each time seems a bit of a code smell to me.
Can you create a predefined message -> avro serializer and use that to feed the queue?
Or, just for testing, create one avro object outside the loop and feed that one object into the queue many times. That way you can see if it is the building or the queuing which is the bottleneck.
More general advice:
Maybe attach a profiler and see if you are making an excessive amount of object allocations. Which is particularly bad if they are getting promoted to higher generations.
See if they are your objects or Chronicle Queue ones.
Is your code maxing out your ram or cpu (or network)?
QUESTION
We had a StreamCorruptedException from chroniclequeue(chronicle-queue-5.20.106, Red Hat Linux release 6.10) and we have pasted the stacktrace below. During that time, there was a very high IO/disk operation by completely different process which we believe caused the chroniclequeue to pause for more than 15seconds and caused this corruption.
Even after restart as the queue was corrupted and it couldn't come up. Only way is to delete and start fresh, meaning loosing millions of data
Please help with solution or any work around. Thanks
STACKTRACE
...ANSWER
Answered 2020-Nov-19 at 15:23If there's a possibility for your application to be stalled for 15 seconds there's no solution possible on the Chronicle Queue side - you should reconsider the way your software works as Chronicle's tools are developed with ultra-low-latency in mind and we cater for microsecond latencies, not seconds.
If the lock is forcibly unlocked which is the case here the data WILL be corrupted irreversibly.
A workaround however could be to increase the timeout - default is 15000ms but when creating a queue you can increase it by using builder#timeoutMS() to specify something that works in your environment.
QUESTION
ANSWER
Answered 2020-Jul-27 at 23:16This is a bug in Chronicle-Wire that's now been fixed.
QUESTION
I mean when I write to java.util.concurrent.LinkedBlockingQueue I have a visibility guarantee happens-before. Single object put to the queue happens-before pull of the same object from the queue.
So I can assume in the reading thread, that any other state of variable is not older then the time at which the object was put to the LBQueue.
Do we have the same visibility guarantee in Chronicle-Queue ?
Do we have the same visibility guarantee in any of the usage style ( methodReader/writer vs wire().read().object() ) ?
ANSWER
Answered 2020-Mar-13 at 10:39Indeed you have the same (in fact, slightly stronger) guarantee. Each write to Chronicle Queue (in any style) results in a sequence of volatile write/reads to the memory-mapped file, hence provides memory barriers.
Additionally, each write is guarded by a write lock, which is implemented as a CAS-driven update to a value stored in a memory-mapped file.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install Chronicle-Queue
You can use Chronicle-Queue like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the Chronicle-Queue component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
