MESO | USER-MESO package for LAMMPS | GPU library

I am using Mesos 1.3.1 and Chronos in my local. I currently have 100 jobs scheduled every 30 minutes for testing.

Sometimes the tasks get stuck in RUNNING status forever until I restart the Mesos agent that the task is stuck. No agent restarted during this time.

I have tried to KILL the task but its status never gets updated to KILLED while the logs in Chronos say that successfully received the request. I have checked in Chronos that it did update the task as successful and end time is also correct but duration is ongoing and the task is still in RUNNING state.

Also the executor container is running forever for the task that are stuck. I have the executor container that will sleep for 20 seconds and set the offer_timeout to 30 seconds and executor_registration_timeout to 2 minutes.

I have also included Mesos reconciliation every 10 minutes but it updates the task as RUNNING every time.

I have also tried to force the task status to update again as FINISHED before the reconciliation but still not getting updated as FINISHED. It seems like the Mesos leader is not picking up the correct status for the stuck task.

I have tried to run with different task resource allocations (cpu: 0.5,0.75,1...) but does not solve the issue. I changed the number of jobs to 70 for every 30 minute but still happening. This issue is seen once per day which is very random and can happen to any job.

How can I remove this stuck task from the active tasks without restarting the Mesos agent? Is there a way to prevent this issue from happening?

I have two Django websites that create a Spark Session to a Cluster which is running on Mesos.

The problem is that whatever Django starts first will create a framework and take 100% the resources permanently, it grabs them and doesn't let them go even if idle.

I am lost on how to make the two frameworks use only the neede resources and have them concurrently access the Spark cluster.

Looked into spark schedulres, dynamic resources for spark and mesos but nothing seems to work. Is it even possible or should I change the approach?

I'm trying to receive stock data for about 1000 stocks, to speed up the process I'm using multiprocessing, unfortunately due to the large amount of stock data I'm trying to receive python as a whole just crashes.

Is there a way to use multiprocessing without python crashing, I understand it would still take some time to do all of the 1000 stocks, but all I need is to do this process as fast as possible.

I try to run my code written in main2.py

So I've tried looking for the answers on this website and Google. I swear I have the right idea to "close" off each ordered list accordingly:

I have a flask API that using pyspark starts spark and sends the job to a mesos cluster.

The executor fails because it's taking part of the route where spark-class is located in the Flask API.

Logs:

In a Mesos ecosystem(master + scheduler + slave), with the master executing tasks on the slaves, is there a configuration that allows modifying number of tasks executed on each slave?

Say for example, currently a mesos master runs 4 tasks on one of the slaves(each task is using 1 cpu). Now, we have 4 slaves(4 cores each) and except for this one slave the other three are not being used.
So, instead of this execution scenario, I'd rather prefer the master running 1 task on each of the 4 slaves.

I found this stackoverflow question and these configurations relevant to this case, but still not clear on how to use the --isolation=VALUE or --resources=VALUE configuration here.

Thanks for the help!

Can't access Mesos UI on the master:5050.

Running a small cluster 1 master-3 slaves on a Linux dist.

Master seems to be picked OK:

I am trying to obtain all data inside dt/dd table structure on a website.

My current code looks like this: