unique-filename | unique filename for use in temporary directories | File Utils library

My Spark cluster has the following setup:

I've built a Rails 6 app that uses React as a frontend and using Bootstrap React for my styling components. Everything works fine locally but when I deploy to Heroku and I try to create an 'outage', it throws the following error:

I have installed Spark 2.4.3 with Hadoop 3.2 on an AWS EC2 instance. I’ve been using spark (mainly pyspark) in local mode with great success. It is nice to be able to spin up something small and then resize it when I need power, and do it all very quickly. When I really need to scale I can switch to EMR and go to lunch. It all works smoothly apart from one issue: I can’t get the local spark to reliably write to S3 (I've been using local EBS space). This is clearly something to do with all the issues outlined in the docs about S3’s limitations as a file system. However, using the latest hadoop, my reading of the docs is that should be able to get it working.

Note that I'm aware of this other post, which asks a related question; there is some guidance here, but no solution that I can see. How to use new Hadoop parquet magic commiter to custom S3 server with Spark

I have the following settings (set in various places), following my best understanding of the documentation here: https://hadoop.apache.org/docs/r3.2.1/hadoop-aws/tools/hadoop-aws/index.html

I see this on two different machines. When I navigate to the folder that contains my package.json file and execute the command yarn list, it lists a bunch of packages that I haven't installed. If I execute the command yarn check then it complains that most of the packages aren't installed.

So, what changed since the last time this worked correctly? Where is yarn finding all of the extraneous packages, and how do I convince it that they really aren't there?

Here are all of the relevant files in my project directory:

package.json

After upgrading to angular7 the build in azure fails on the "npm install" step. Locally everything works fine.

I have tried:

1. Setting the clean option in the build definition to true and to "all directories".
2. Forcing an npm update "npm@latest -g" step before the install step
3. Setting the "WEBSITE_NODE_DEFAULT_VERSION" on the Application Settings in Portal to the same version as I have locally (10.14.1).

But I am still getting the following error output from the "npm install" step. (Notice that the agent is not using the same version as the default value is set to)

My last option (as I see it) is to delete the entire service and create a new one with the same name - but I would like to avoid that..

Regarding the last error in the output (npm ERR! 404 Not Found: har-validator@5.1.2), I have no idea what this is but it is not included in my package.json. I believe it might be something the cli packages is using?

[Note to those reaching for the duplicate button: It is July 2017. There are near duplicates to this question from the past, however part of the reason for raising the question is to illuminate whether the answer has changed over time as technology has changed. Also, this question is more about how to create reliably unique folders at speed on a SAN from multiple cloned web servers. I cannot find a duplicate of this question but would be happy to be pointed at a recent example.]

Question - is there a pragmatic potential for multiple cloned MS 2008+ servers to produce the same GUID value?

Context:

I am investigating the creation of unique folders by multiple servers on a shared SAN. Imagine a web farm with many web servers using the same file storage. [Please refrain from the 'Why do it like that' responses.]

This is a C# / Dot Net based solution running on MS Windows server 2008+.

The natural tool to reach for is the GUID as it gives minimal overhead assuming no physical check for a folder of the given GUID on disk (because that would introduce time-costly network and disk reading overhead).

However, a GUID is only viable IF the GUID is actually unique cross-servers.

Research:

In this question from 2008, the poster asked about the uniqueness of a UUID / GUID.

In this question from 2009 the performance of MS GUID's was discussed.

I know of old that the 'random' functions in computers are seeded and therefore not random. I read that the GUID was originally generated partly from the device MAC address and therefore would have a firm chance of uniqueness. But then I read that this use of the MAC address was retired, which leads me to go back to considering the old 'rnd()' seeding issue or its equivalent and ponder the impact of uniqueness in practical terms across multiple servers.

And for those proponents of the stats related to UUID's, a couple of responses I found during research mention the Birthday Paradox. I wonder what practical impact this has across multiple servers.

Potentially useful question on Path.GetRandomFileName v's Guid.NewGuid.

Dot Net Pearls has an interesting article on performance of the algorithm behind Path.GetRandomFileName.