sqlc | SQL Composer for Go | SQL Database library

Your understanding is correct.

--files argument is uploading files to executors only.

See this in the spark documentation

file: - Absolute paths and file:/ URIs are served by the driver’s HTTP file server, and every executor pulls the file from the driver HTTP server.

You can read more about this at advanced-dependency-management

Now coming back to your second question

How can I upload to master?

There is a concept of bootstrap-action in EMR. From the official documentation it means the following:

You can use a bootstrap action to install additional software or customize the configuration of cluster instances. Bootstrap actions are scripts that run on cluster after Amazon EMR launches the instance using the Amazon Linux Amazon Machine Image (AMI). Bootstrap actions run before Amazon EMR installs the applications that you specify when you create the cluster and before cluster nodes begin processing data.

How do I use it in my case?

While spawning the cluster you can specify your script in BootstrapActions JSON Something like the following along with other custom configurations:

Do I understand correctly that the registration numbers in the database dò contain the hyphens ?

In that case, you should modify your query to hold wildcards (e.g. https://www.guru99.com/wildcards.html)

As for the case, either convert the case of the PHP variable before entering it in the query ( strtoupper(), strtolower() ), or try to solve your case in your table definition.

If the database entries do not hold the hyphens, you can filter them out the variable using a replace or preg_replace() to remove them.

Edit

if I assume your database holds very uniform inputs, like this :

The problem is that PHP doesn't know that it should only use the value of Model from the dropdown below the selected radio button. It uses the value of the last one. If the user leaves that one unselected, the value "" doesn't correspond to any of the ENUM values, so you get an error.

You can give them distinct names, and then select the appropriate one based on the radio button.

HTML:

A commit like "cdf7025: Add MySQL json test" (or "456fcb1 Add MySQL test for SELECT * JOIN") suggests joins are supported.

But it is true, as mentioned in issue 643, that queries with JOINs are for now not documented yet.

1. add a column base_folder that contains only the folder part without the file, that will be used for joinning
2. group by base_folder and calculate max timestamp
3. join with original dataframe using base_folder and get max timestamp for rows where it's null

This is a typo:

If you are not using the latest version, try updating to it. It looks similar to this issue that was fixed recently. I would imagine the error message still showing up, but the job now finishing means that the support team is still working on it and hopefully they will fix it in the next release.

I donot think Spark or Scala provides inbuilt support for interpreting Math expressions using variable names

But you can work around this using java library ScriptEngineManager

trying to join the thread on the destructor just causes the thing to spit a resource deadlock exception which I don't understand since it's not trying to join itself it's trying to join a member object