VESPA | scale Evolutionary and Selective Pressure Analyses | Awesome List library

Yes, the default tokenizer is OpenNLP and it works with no configuration needed. It will guess the language if you don't set it, but if you know the document language it is better to use set_language (and language=...) in queries, since language detection is unreliable on short text.

However, OpenNLP tokenization (not detecting) only supports Danish, Dutch, Finnish, French, German, Hungarian, Irish, Italian, Norwegian, Portugese, Romanian, Russian, Spanish, Swedish, Turkish and English (where we use kstem instead). So, no CJK.

To support CJK you need to plug in your own tokenizer as described in the linguistics doc, or else use ngram instead of tokenization, see https://docs.vespa.ai/documentation/reference/schema-reference.html#gram

n-gram is often a good choice with Vespa because it doesn't suffer from the recall problems of CJK tokenization, and by using a ranking model which incorporates proximity (such as e.g nativeRank) you'l still get good relevancy.

Vespa don't support this today but feature requests are welcome over at https://github.com/vespa-engine/vespa/issues

You can configure summary classes which is also more efficient then resolving this at query time. See https://docs.vespa.ai/en/document-summaries.html

There are many ways to accomplish this but what is best depends on if you need free text style matching (linguistic processing of the string including tokenization and stemming) or not. It also depends on if this is just a ranking signal for documents that are already retrieved or used to retrieve documents.

If you don't need free text style matching but instead can use exact matching without linguistics processing (e.g using a fixed vocabulary) and this color ranking is just another ranking signal you should consider looking at using tensor ranking instead. Tensors are useful for ranking documents that are retrieved by the query operators, you cannot retrieve using a tensor (except for dense single order tensors using approximate nearest neighbor search). See tensor guide https://docs.vespa.ai/en/tensor-user-guide.html.

If you need free text style matching there are also several approaches. In the below example I assume that you want to have text style matching and that a query term 'purple' should match the document with 'black and purple'. See matching documentation https://docs.vespa.ai/en/reference/schema-reference.html#match

If you define the field colors like this

If PyTorch can't export the OneHot operator to ONNX I think your best option is to ask them to fix that?

Or, if you can extract the conversion from your model, such that the one-hot-encoded tensor is an input to your network, you can do that conversion on the Vespa side by writing a function supplying the one-hot tensor by converting the source data to it, e.g

Add a reference from the VBA editor (Tools -> References...) to Microsoft ActiveX Data Objects; choose the latest version, usually 6.1

Then you could write VBA code like the following:

Seems like you want to use a sparse tensor field instead since some addresses does not have a value. x{} denotes a sparse tensor, x[128] is an example of a dense tensor. See https://docs.vespa.ai/documentation/tensor-user-guide.html for an intro to vespa tensor fields.

Yes, the stub returned by Execution.Context.createContextStub() is just for testing and doesn't provide all the information that is needed. Instead:

1. Get a com.yahoo.search.searchchain.ExecutionFactory injected in your component (by declaring it as a parameter in the constructor).

2. To get an execution, call executionFactory.newExecution(chain)

To get this working you should avoid having underscores in the network name, use the fully qualified name for the config server and name the containers to get DNS working.

Create the network on a manager swarm host:

The collect script/function expects that there is a field called id in your document schema. If you alter the script to use the uri field instead you should be able to retrieve the documents.

• https://github.com/vespa-engine/sample-apps/blob/master/text-search/src/python/collect_training_data.py#L15
• https://github.com/vespa-engine/sample-apps/blob/master/text-search/src/main/application/schemas/msmarco.sd#L6