nsq | A realtime distributed messaging platform | Architecture library

This question is more rather more about an option on how to scale certain services in Kubernetes. I think the best pattern for this is to have a Kubernetes Operator. Essentially, this would manage the lifecycle of your nsqd application. There are several operator tools that you can use to create your nsqd operator:

• Operator Framework
• KUDO
• Kubebuilder
• Metacontroller

A good example of a scale-out operator is Srimzi which is Kafka operator for Kubernetes.

You can also find other well-known operators in the Operator Hub.

You can use use to pass variables to a function callback like this:

I come across this problem often in webscraping. The key is to store the intermediate results in an environment where they are accessible if your function throws an error. The obvious place is the global environment, but this depends on how you are using your function. If it is part of a package, then you don't want to write to the global workspace. In that case you can have a "storage" environment as part of the package.

Perhaps the neatest way to do this is to delete the intermediate object after the loop is complete, so it will only ever be visible / accessible if the loop throws an error.

Here is a function that demonstrates the principle:

The issue here is most certainly the fact that You never call the open() method of the superclass. This will cause some of the variables to be uninitialized. This should be solved by calling the super.open() inside Your open() method.

TLDR;

Just look up the symbol (ie. AAPL or GOOGL). The IXIC is an index, Alpha Vantage currently does not cover indexes.

Further notes:

1. Quotes from Alpha Vantage are aggregated, so any price you're getting won't be the price from what it's traded on the NASDAQ, but a quote of what the price is across exchanges.

2. If you're looking for a specific symbol, you can check for what it's listed as using the search endpoint.

3. Example here

Based on the name of your variable, "pockinesisfirehose", it looks like you may be using Kinesis Firehose, not a regular Kinesis Stream. A regular Kinesis Stream is needed for this collector. Kinesis Firehose will take the data coming into the Kinesis Stream, and put it into another Amazon service (S3, Redshift, etc.)

Currently in Spring Data Elasticsearch the returned entity is populated from the values returned in the _source field. The returned sort is not part of that, but a part of the additional information return for each search hit.

For the next release (4.0) we are currently in the process of rewriting how returned hits are handled. The next version will have a SearchHit class where T is the entity domain class, and the SearchHit object contains this entity along with other information like score, highlight, scripted fields, sort etc.

It is planned that the repository methods then can for example return Lists or Pages of these SearchHit objects.

But currently, the sort from the query will lead to a sorted return, but you cannot retrieve the sort values themselves - which in your case is bad, because they are calculated dynamically.

Edit:

You can retrieve the value by defining a property for the geo_distance in your entity and annotate this with the @ScriptedField annotation, and then provide the script in your native query. An example can be found in the at tests for Spring Data Elasticsearch. This example works with the ElaticsearchTemplate, but it should make no difference passing this query into a repository search method.

Edit 25.12.2019:

The current master branch, which will become Spring Data Elasticsearch 4.0, has this implemented. The found documents are returned in a SearchHit object that also contains the sortValues.

I think there's a confusion here that newCache and newCacheIO are involved with the Cloud Shake feature. They aren't - in fact, newCache happens to be one of the things that has no impact on the cloud cache (coincidentally, rather than because of the Cache in the name). I see no reason your use case shouldn't work with Cloud Shake.

The exception /home/jefdaj/shortcut/sharedir/.shake.cache/2faae061b9976bed/0x134125AC: getPermissions:getFileStatus: does not exist (No such file or directory) seems to imply that Shake has recorded that the above file is in the cache, and can be used to satisfy the file. So the question is where did the file go? Is it possible when you cleaned the files that they deleted the files from the cache too? Shake can be run with both --share-copy and --share-symlink - I'd recommend the former to ensure there is less action-at-a-distance, which might also explain what is deleting the files. I'd also recommend using Shake HEAD, as there are some fixes that haven't yet been released.

If that still doesn't work, perhaps raise a ticket on the Shake GitHub?

As you point out in the comments, you do not know how many files you get as output. There are two options:

• Make use of output directories.
• Make use of checkpoints, however this is kinda tricky to properly understand (in my opinion).

Untested but might just work:

rule makeblastdb: input: reference = "/path/to/workdir/{sample}.fasta" output: out = "{sample}.fasta.{ext}" shell: /Tools/ncbi-blast-2.9.0+/bin/makeblastdb -in {input.reference} -out {output.out} -dbtype nucl"

Here we mark the output as an output directory named sample-ext, and the output of ncbi-blast will be in this directory called sample.ext.

edit:

If we do not want all these output directories what we could do is say that we expect at least 1 output, and if that one exists we assume everything went fine: