datadog-sidekiq | Rust app to track Sidekiq | Job Scheduling library

I need to submit a slurm job that needs to have a core count divisible by 7 on a cluster with 64 core nodes. One solution is to run a 7 node/16 core job, which works well because the parallelization works extremely well between these 7 groups of cores (very little communication between the 7 groups).

Scheduling of this job becomes difficult however since its hard for 7 nodes to open up 16 cores at one time. Are there any ways to submit jobs in the following configurations?

• Explicitly request 2 nodes, one uses 64 cores and one uses 48 cores.

• Allow the job to combine the 7 node job to place multiple node allocations on a single node, allowing it to simply find 7 groups of 16 cores.

The only thing I cannot allow is the groups of 16 cores to be split over 2 nodes, as this will dramatically hurt performance.

This is running on slurm 20.11.8

I was going through JobScheduler API documentation, it states "This is an API for scheduling various types of jobs against the framework that will be executed in your application's own process." What does application's own process mean, does that mean application context such that when app is killed job will terminated as well?

I have a SLURM job script as follows:

Let's suppose I have the following bash script (bash.sh) to be run on a HPC using slurm:

I need to add a job to run in the background, which I can easily do by MyJob.perform_later(args).

But I need some wait time before the scheduling, say 1 minute.

So if I schedule the job, it should get enqueued after 1 minute.

One workaround I could think of is to add sleep to my job. For example:

I am trying to add an Endbeforestartconstraint to my contrained programming problem. However, I receive an error saying that my end beforestart is not an array type. I do not understand this as I almost copied the constraint and data from the sched_seq example in CPLEX, I only changed it to integers.

What I try to accomplish with the constraint, is that task 3 and task 1 will be performed before task 2 will start.

How I can fix the array error for this constraint?

Please find below the relevant parts of my code

My distance matrix in my no overlap constraint does not seem to work in my model outcome. I have formulated the distance matrix by means of a tuple set. I have tried this in 2 different ways as can be seen in the code. Both tuple sets seem to be correct and the distance matrix is added in the noOverlap constraint for the dvar sequence.

Nevertheless I do not see the added transition distance between products in the optimal results. Jobs seem to continue at the same time when a job is finished. Instead of waiting for a transition time. I would like this transition matrix to hold both for machine 1 and machine 2.

Could someone tell me what I did wrong in my model formulation? I have looked into the examples, but they seem to be constructed in the same way. So I do not know what I am doing wrong.

mod.

I want to try to add a distance matrix to a simple scheduling problem in CPLEX using CP however I cannot manage to get this in without an error in my new dvar sequence.

I am trying to include setup times between products 1,2 and 3 which depend on the sequence that the products are scheduled. The setup times for the different sequences are given in a distancematrix.I tried to define the distance matrix as a tuple triplet in the mod file and as a matrix in the dat file but both options do not work for me.

The new setup dvar sequence for this setup time is called setup, and should represent the sequence of every job J on a machine. However, I receive the error that 'the function noOverlap dvarsequence,[range][range] does not exists. I do not understand what I am doing wrong, since I defined the dvar seq and the matrix so in my understanding it should work

Could someone help me out here? Stuck with this problem for a while now.

Please find below the mod. and dat. files.

Thank you in advance! mod.

How do I implement a Scheduled Notification service that sends alerts to the app's users at their preferred times? For the time being, they are receiving immediate notification; however, can someone advise on how I might schedule it?