datasources | plugin contains various datasources contributed by the core | Web Framework library

The fundamental issue here is that tableView(_:viewForHeaderInSection:) is a pull based method and Rx is designed for push based systems. Obviously it can be done. After all, the base library did it for tableView(_:cellForRowAt:) but it's quite a bit more complex. You can follow the same system that the base library uses for the latter function.

Below is such a system. It can be used like this:

Please have a look at the official deployment guideline here: https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html#deploying

Extracting the important part:

Thinking on this some more, I suspect you may be misunderstanding how an Application.cfc's this scope operates. Short answer: this isn't persistent, like the application scope, because the component gets instantiated on every request

From Defining the application and its event handlers in Application.cfc

When ColdFusion receives a request, it instantiates the Application CFC and runs the Application.cfc code ...

Your code actually does work. At least in the sense that the include successfully changes the value of this.datasource when invoked. However, since the Application.cfc gets instantiated anew on every request, the component's this scope also gets recreated. Essentially wiping out any previous changes made inside OnApplicationStart(). That's why it seems like the code never assigns a datasource value, when it actually does.

Bottom line, an Application.cfc's this scope isn't intended to be used that way.

FWIW, you can see the behavior in action using the test files below. Just load test.cfm in a browser (at least twice) then check the logs. The output shows a value IS assigned to this.datasource the very first time test.cfm is requested from the application. However, that value disappears on the next http request because CF creates a new Application.cfc instance.

Application.cfc

You should create a RDD from JSON string and pass that to spark.read.json method.

We encountered the same issue when we added a new scheduled job configuration

How it has been addressed:

1. Create the JobLaunchUtils (similar to yours)

ABFS is a "real" file system, so the S3A zero rename committers are not needed. Indeed, they won't work. And the client is entirely open source - look into the hadoop-azure module.

the ADLS gen2 store does have scale problems, but unless you are trying to commit 10,000 files, or clean up massively deep directory trees -you won't hit these. If you do get error messages about Elliott to rename individual files and you are doing Jobs of that scale (a) talk to Microsoft about increasing your allocated capacity and (b) pick this up https://github.com/apache/hadoop/pull/2971

This isn't it. I would guess that actually you have multiple jobs writing to the same output path, and one is cleaning up while the other is setting up. In particular -they both seem to have a job ID of "0". Because of the same job ID is being used, what only as task set up and task cleanup getting mixed up, it is possible that when an job one commits it includes the output from job 2 from all task attempts which have successfully been committed.

I believe that this has been a known problem with spark standalone deployments, though I can't find a relevant JIRA. SPARK-24552 is close, but should have been fixed in your version. SPARK-33402 Jobs launched in same second have duplicate MapReduce JobIDs. That is about job IDs just coming from the system current time, not 0. But: you can try upgrading your spark version to see if it goes away.

My suggestions

1. make sure your jobs are not writing to the same table simultaneously. Things will get in a mess.
2. grab the most recent version spark you are happy with

I think insecure provider parameter is what you are looking for:

(Optional) Trust self-signed SSL certificates. If omitted, the OS_INSECURE environment variable is used.

Try:

I actually figured it out, sorry for not posting the answer immediately.

I found myself forgot to call the stub for the setString method. Here is the code.

As I understand you have two options, either inject the dependencies needed for static Future> _fetchCards(_) async via parameters, or mock the object in the locator itself. I would go for the fist option, and have something like :