node-kafka | See below : - NOT MAINTAINED
kandi X-RAY | node-kafka Summary
kandi X-RAY | node-kafka Summary
I’m no longer maintaining node-kafka.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of node-kafka
node-kafka Key Features
node-kafka Examples and Code Snippets
Community Discussions
Trending Discussions on node-kafka
QUESTION
I been looking at the docks and now wondering if this is even a feature of kafka or node-kafka.
So my consumer receives a message from kafka, and while I process the message, I like to mark it as if I didn't get it until I completed processing it. This way, if the server crashes while processing, I can reconnect and received the messages that my application received prior to the crash it was working on.
High level example.
- Kafka message is a user id;
- Once received, I get user info from the database.
- I send an email to the user.
- When the email is sent. I log it to the database.
- I mark message complete. (I'm not sure if this step is possible)
This way, if any of the steps crashes before step five, when the server is back up, it receives the failed message again.
I'm using node.js LTS 8.10.0
and the npm package kafka-node@2.4.1
ANSWER
Answered 2018-Apr-13 at 19:17The way to do this is to save your last read offset to a database that you prefer. This way you can use this offset to start your consumer from the next time you restart it.
Kafka provides a mechanism to do this inbuilt with versions > 0.8
. Kafka stores the latest offset read by a consumer group to an internal topic called consumer_offsets
. Most of the client libraries have a provision to auto-commit last processed offset or you may opt to disable auto-commit and commit the offset yourself once your entire processing is complete.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install node-kafka
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page