workflow | minimalist file based workflow | BPM library

I have a following class that reads csv data into Spark's Dataset. Everything works fine if I just simply read and return the data.

However, if I apply a MapFunction to the data before returning from function, I get

Exception in thread "main" org.apache.spark.SparkException: Task not serializable

Caused by: java.io.NotSerializableException: com.Workflow.

I know Spark's working and its need to serialize objects for distributed processing, however, I'm NOT using any reference to Workflow class in my mapping logic. I'm not calling any Workflow class function in my mapping logic. So why is Spark trying to serialize Workflow class? Any help will be appreciated.

I have a workflow which creates a new branch with a name that I save as an env variable. the reason is I need the workflow to run on a new clean branch.

1 Job after that I want to check out the branch. the problem is I cant seem to use env variables on the "ref" in order to check it out.

is there a way to do this ? or does github not support this yet.

example code:

Base.html

I am trying to follow this tutorial here - https://juliasilge.com/blog/xgboost-tune-volleyball/

I am using it on the most recent Tidy Tuesday dataset about great lakes fishing - trying to predict agency based on many other values.

ALL of the code below works except the final row where I get the following error:

I have a Python Apache Beam streaming pipeline running in Dataflow. It's reading from PubSub and writing to GCS. Sometimes I get errors like "Error in _start_upload while inserting file ...", which comes from:

I have recently decided to try out Laravel Sail instead of my usual setup with Vagrant/Homestead. Everything seems to be beautifully and easily laid out but I cannot seem to find a workaround for changing domain names in the local environment.

I tried serving the application on say port 89 with the APP_PORT=89 sail up command which works fine using localhost:89 however it seems cumbersome to try and remember what port was which project before starting it up.

I am looking for a way to change the default port so that I don't have to specify what port to serve every time I want to sail up. Then I can use an alias like laravel.test for localhost:89 so I don't have to remember ports anymore I can just type the project names.

I tried changing the etc/hosts file but found out it doesn't actually help with different ports

I essentially am trying to access my project by simply typing 'laravel.test' on the browser for example.

Also open for any other suggestions to achieve this. Thanks

**Update ** After all this search I actually decided to change my workflow to only have one app running at a time so now I am just using localhost and my CPU and RAM loves it, so if you are here moving from homestead to docker, ask yourself do you really need to run all these apps at the same time. If answer is yes research on, if not just use localhost, there is nothing wrong with it

There is a Java 11 (SpringBoot 2.5.1) application with simple workflow:

1. Upload archives (as multipart files with size 50-100 Mb each)
2. Unpack them in memory
3. Send each unpacked file as a message to a queue via JMS

When I run the app locally java -jar app.jar its memory usage (in VisualVM) looks like a saw: high peaks (~ 400 Mb) over a stable baseline (~ 100 Mb).

When I run the same app in a Docker container memory consumption grows up to 700 Mb and higher until an OutOfMemoryError. It appears that GC does not work at all. Even when memory options are present (java -Xms400m -Xmx400m -jar app.jar) the container seems to completely ignore them still consuming much more memory.

So the behavior in the container and in OS are dramatically different. I tried this Docker image in DockerDesktop Windows 10 and in OpenShift 4.6 and got two similar pictures for the memory usage.

Dockerfile

I am new to Spark and am trying to run on a hadoop cluster a simple spark jar file built through maven in intellij. But I am getting classnotfoundexception in all the ways I tried to submit the application through spark-submit.

My pom.xml:

I am currently doing a course of Version control using git from coursera. There is a project which I have to complete and make a git workflow. Here is the final gitflow graph of my project

My graph has 3 "feature 2 wip" commits but the other project graph has only one "feature 2 wip" commit. Here is the gitflow graph of other project

I am not able to understand whether my graph is correct or the other project graph is correct. We have to follow the instructions given in the project. I am also attaching the images of the instructions as well. If I am wrong then please explain my mistake in detail.