job-runner | Runs Cron based jobs define via Yaml | Cron Utils library

by sstarcher Python Version: Current License: MIT

X-Ray Key Features Code Snippets Community Discussions(3)Vulnerabilities Install Support

kandi X-RAY | job-runner Summary

job-runner is a Python library typically used in Utilities, Cron Utils, Symfony applications. job-runner has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. However job-runner build file is not available. You can download it from GitHub.

Runs Cron based jobs define via Yaml

Support

Quality

Security

License

Reuse

Support

job-runner has a low active ecosystem.

It has 12 star(s) with 6 fork(s). There are no watchers for this library.

It had no major release in the last 6 months.

There are 2 open issues and 1 have been closed. On average issues are closed in 119 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of job-runner is current.

Quality

job-runner has 0 bugs and 0 code smells.

Security

job-runner has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

job-runner code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

job-runner is licensed under the MIT License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

job-runner releases are not available. You will need to build from source code and install.

job-runner has no build file. You will be need to create the build yourself to build the component from source.

Installation instructions are not available. Examples and code snippets are available.

Top functions reviewed by kandi - BETA

kandi has reviewed job-runner and discovered the below as its top functions. This is intended to give you an instant insight into job-runner implemented functionality, and help decide if they suit your requirements.

Create a crontab file
Convert to yaml
Merge two dicts
Convert memory limit value to integer
Substitute values in a string
Validate a crontab file
Load yaml file

Get all kandi verified functions for this library.

job-runner Key Features

No Key Features are available at this moment for job-runner.

job-runner Examples and Code Snippets

No Code Snippets are available at this moment for job-runner.

Community Discussions

Trending Discussions on job-runner

EMR EKS unable to launch driver pod

What is the difference between service account and service agent in GCP

How to have "dependent" default values that can be overriden by the user?

QUESTION

EMR EKS unable to launch driver pod

Asked 2021-Dec-21 at 18:41

How does one go about setting resource limits in EMR on EKS? My driver pod is failing to launch because it is requesting more CPU than it is allowed. This doesn't make sense to me. I am running the getting started code from the docs below.

I have added --conf spark.driver.limit.cores=2 in order to try and make the limit higher than what is listed in the error message below. I got this idea from here https://spark.apache.org/docs/latest/running-on-kubernetes.html#spark-properties

This cluster does have istio running in it. I am not sure if that would cause issues.

Here is the code I am running to trigger the job

...

ANSWER

Answered 2021-Dec-21 at 18:41

I was able to figure it out.

Source https://stackoverflow.com/questions/70427566

QUESTION

What is the difference between service account and service agent in GCP

Asked 2020-Dec-03 at 03:55

Say I have this case where

I have to run some test with dataflow
inside this dataflow job I need to access a gcs bucket and save my output there.
I will need to run the dataflow job with my own SA instead of the default SA.

I created a Google Service Account to run my dataflow job. But after I enabled the dataflow API. I end up having 2 SA in front of me.

the service account agent --> 123456789@dataflow.gserviceaccount.com
the dataflow job runner service account --> dataflow-job-runner@MY-PROJECT-ID.iam.gserviceaccount.com

It got me really confused to see what the official document says

Some Google Cloud services have Google-managed service accounts that allow the services to access your resources. These service accounts are sometimes known as service agents.

If I create a dataflow job to run with the dataflow-job-runner@MY-PROJECT-ID.iam.gserviceaccount.com SA, I suppose I'd need to grant the roles/storage.objectAdmin for it.

The question is

Do I need to grant any permission to the service account agent?
What does the service account agent actually do, what does it has to access any resource?

...

ANSWER

Answered 2020-Dec-03 at 03:55

Several Google Cloud services such as Cloud Dataflow require two sets of permissions.

The program that you write uses a service account. You grant this service account IAM roles to access resources that require authorization that your program requires. For example, reading data from Cloud Storage or issuing queries to BigQuery.

The service agent applies to the service's runtime. For example when you launch a job on Cloud Dataflow, Cloud Dataflow needs to launch VMs to run your program on. Your program is not launching the VMs, the service is. Therefore the service requires its own set of permissions. This is what the service agent is for.

By using two different service accounts, separation of privilege is achieved.

Source https://stackoverflow.com/questions/65092092

QUESTION

How to have "dependent" default values that can be overriden by the user?

Asked 2020-May-05 at 16:59

I have the following function from my odd-jobs job-queue library. There are a bunch of configuration parameters where the default implementation depends on another config parameter. For example:

cfgJobToHtml depends on cfgJobType, which defaults to defaultJobType. However, after calling defaultConfig, the user may choose to override the value for cfgJobType without changing cfgJobToHtml. The expected behaviour is that cfgJobToHtml should now use the user-provided value instead of defaultJobType.
Similarly, cfgAllJobTypes depends on cfgJobTypeSql, which in-turn defauls to defaultJobTypeSql. Again, after calling defaultConfig, if the user overrides the value for cfgJobTypeSql, then cfgAllJobTypes should use the overridden value, not defaultJobTypeSql.

The code below does not work the way I'm expecting it to. If you change cfgJobType the change is not picked up by cfgJobToHtml. Similarly, for cfgJobTypeSql.

What is the best way to have these "dependent" default values?

...

ANSWER

Answered 2020-May-05 at 15:25

People often implement it using builder pattern.

In your example, you first fill the defaults and then let user override some fields if she wants. With builder it's other way around: you let user fill the data she wants to override, then you fill the rest.

Specifically, you make an intermediate data type to hold a partially filled config, ConfigUnderConstruction. All fields there are optional. User can specify all the fields she is interested in, then you assemble the config, filling all the defaults:

Source https://stackoverflow.com/questions/61612491

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install job-runner

You can download it from GitHub.
You can use job-runner like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: