rebalance | Python code to aid in Exchange-Traded Funds | Cryptocurrency library

I fixed your algorithm. Really just needed a slightly different approach. I changed it to use Math.floor() for participation and then got the diff (100 % length) needed to make the value 100% and added that to the last value. Here is the updated stackblitz.

https://stackblitz.com/edit/create-a-basic-angular-component-7vj1rr?file=src/app/example/user.component.html

PS: never make method calls in the template due to the overhead the this add each time change detection runs; instead use pure pipes or calculate their value for presentation when the data changes. There was a decimal formatting method being called that I removed.

Following should work.

The rebalance in Kafka is a protocol and is used by various components (Kafka connect, Kafka streams, Schema registry etc.) for various purposes.

In the most simplest form, a rebalance is triggered whenever there is any change in the metadata.

Now, the word metadata can have many meanings - for example:

• In the case of a topic, it's metadata could be the topic partitions and/or replicas and where (which broker) they are stored
• In the case of a consumer group, it could be the number of consumers that are a part of the group and the partitions they are consuming the messages from etc.

The above examples are by no means exhaustive i.e. there is more metadata for topics and consumer groups but I wouldn't go into more details here.

So, if there is any change in:

• The number of partitions or replicas of a topic such as addition, removal or unavailability
• The number of consumers in a consumer group such as addition or removal
• Other similar changes...

A rebalance will be triggered. In the case of consumer group rebalancing, consumer applications need to be robust enough to cater for such scenarios.

So rebalances are a feature. However, in your case it appears that it is happening very frequently so you may need to investigate the logs on your client application and the cluster.

Following are a couple of references that might help:

1. Rebalance protocol - A very good article on medium on this subject
2. Consumer rebalancing - Another post on SO focusing on consumer rebalancing

Posting comment as the community wiki answer for better visibility

I would use JDBC_PING for discovery, so only nodes which are using the same DB will be able to discover each other

Flink's Kafka consumer does not support stopping consumption from a topic (without a restart), but it does support dynamic topic and partition discovery. See https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/datastream/kafka/#dynamic-partition-discovery for details.

Your Kafka and Zookeeper deployments are running in the kaf namespace according to your screenshots, presumably you have set this up manually and applied the configurations while in that namespace? Neither the Kafka or Zookeeper YAML files explicitly state a namespace in metadata, so will be deployed to the active namespace when created.

Anyway, the Kafka deployment YAML you have is hardcoded to assume Zookeeper is setup in the default namespace, with the following line:

Previously while retrying I was using the below approach:

If you are adding nodes to a cluster following the proper steps, the new node will not accept client requests until the join/streaming operations are complete.

The only time that would happen, is if auto_bootstrap was set to false on the new node. Then, the node would join without streaming data, and attempt to service requests before the data could be successfully streamed (via repair/rebuild).

I've taken your simplified configured with just 2 nodes using a non-wildcard queue with redistribution-delay of 0, and I reproduced the behavior you're seeing on my local machine (i.e. without Kubernetes). I believe I see why the behavior is such, but in order to understand the current behavior you first must understand how redistribution works in the first place.

In a cluster every time a consumer is created the node on which the consumer is created notifies every other node in the cluster about the consumer. If other nodes in the cluster have messages in their corresponding queue but don't have any consumers then those other nodes redistribute their messages to the node with the consumer (assuming the message-load-balancing is ON_DEMAND and the redistribution-delay is >= 0).

In your case however, the node with the messages is actually down when the consumer is created on the other node so it never actually receives the notification about the consumer. Therefore, once that node restarts it doesn't know about the other consumer and does not redistribute its messages.

I see you've opened ARTEMIS-3321 to enhance the broker to deal with this situation. However, that will take time to develop and release (assuming the change is approved). My recommendation to you in the mean-time would be to configure your client reconnection which is discussed in the documentation, e.g.: