zerg | Docker-based distributed crawler service

You never use T in IUnit and in Unit nor in the rest of the code you provided. So it is useless to use generics...

So you can simply write:

You send the data to the template as races, not company.

Did you created/used entity manager? If not, persistence module did not start. Make a insert using entity manager, or try:

Use eclipselink.deploy-on-startup to configure deployment on startup (at the creation of the EntityManagerFactory) instead of occurring the first time an EntityManager is created.

https://www.eclipse.org/eclipselink/documentation/2.4/jpa/extensions/p_deploy_on_startup.htm#delayonstartup

use aggregation function in filter

As you want to consider only multiple id ,so use below one

The BSON spec — MongoDB’s native binary extended JSON format / data type — only supports 32 bit (signed) and 64 bit (signed) integers — 8 bytes being 64 bits.

The maximum integer value that can be stored in a 64 bit int is: 9,223,372,036,854,775,807

In your example you appear to have larger ids, for example: 11,611,747,760,398,664,000

I’m guessing that the app generating this data is using uint64 types (unsigned can hold x2-1 values).

I would start by looking at either of these potential solutions, if possible:

• Changing the other side to use int64 (signed) types for the IDs.
• Replacing the incoming IDs using ObjectId() as you then get a 12 byte ~ GUID for your unique IDs.

Here's the jist of how you'd accomplish this in pure Javascript:

So the secret sauce was to use clearClusterMarkers() instead of clearMarkers() since you adjust the markers to cluster markers when you define the argument clusterOptions = markerClusterOptions.

Here is the long form of a working single file app.R

Because you are also bootstrapping Message component in your @NgModule

Is it possible to configure Spark with the spark-streaming-kafka-0-10 library to read multiple Kafka partitions or an entire Kafka topic with a single task instead of creating a different Spark task for every Kafka partition available?

You could alter the number of generated partitions by calling repartition on the stream, but then you lose the 1:1 correspondence between Kafka and RDD partition.

The number of tasks generated by Kafka partitions aren't related to the fact you have 16 executors. The number of executors depend on your settings and the resource manager you're using.

There is a 1:1 mapping between Kafka partitions and RDD partitions with the direct streaming API, each executor will get a subset of these partitions to consume from Kafka and process where each partition is independent and can be computed on it's own. This is unlike the receiver based API which creates a single receiver on an arbitrary executor and consumes the data itself via threads on the node.

If you have 206 topics and 10 partitions each, you better have a decent sized cluster which can handle the load of the generated tasks. You can control the max messages generated per partition, but you can alter the number of partitions unless you're will to call the shuffling effect of the repartition transformation.