orchestrated | a minimal Ruby workflow orchestration framework | BPM library

I am having some issues when try to execute a DataFlow job orchestrated by Airflow. After triggered the DAG, i receive this error:

module 'apache_beam.io' has no attribute 'ReadFromBigQuery''

I need to execute various batch scripts located in a VM within my Resource Group. The call of these scripts needs to be orchestrated by Datafactory.

I learned about Run Command in VM, and which can be used via PowerShell or Rest API. I was wondering if any of these ways is recommended to include in a Datafactory pipeline.

My approach at this point would be to use a Web Activity to do the API calls, one for each script.

My question: Is there a more recommended way to do this? Do you see any bottleneck or situation that may be a problem with this approach?

I need to prepare an AMI based on CentOS 8 with pre-installed SSM-agent. Trying to use Image Builder for this. According to the documentation:

Instances used to build images and run tests using Image Builder must have access to the Systems Manager service. All build activity is orchestrated by SSM Automation. The SSM Agent will be installed on the source image if it is not already present, and it will be removed before the image is created.

So the question is how to prevent removing of SSM-agent? I need to keep it installed. Unfortunately couldn't find a solution in the documentation.

Heroku currently provides the database credentials as one connection string i.e: postgres://foo:bar@baz/fubarDb.

My development environment consists of a PostgreSQL container and an app container orchestrated with a docker-compose file. The docker-compose file supplies environment variables from .env file which currently looks a little like this:

I have a requirement of customizing the email sent to the user, from AD B2C, when she/he resets the password.

I followed this documentation to set up the self-service password reset flow, and it works fine: https://docs.microsoft.com/en-us/azure/active-directory-b2c/add-password-reset-policy?pivots=b2c-custom-policy

To provide a branded email for the password reset, I'm following this code, since it looks like that the only other option is to use Display Controls, which are currently in public preview (so I cannot use them in production): https://github.com/azure-ad-b2c/samples/tree/master/policies/custom-email-verifcation

The readme clearly states that it can be used also for the password reset, but the code only provides an example for the sign in email verification.

I tried to add the verificationCode OutputClaim in various TechnicalProfiles, but I'm unable to visualize the custom verificationCode textbox that is needed by the provided javascript code.

I'm thinking that maybe I should use a specific ContentDefinition, but I'm really struggling to find the correct way to update the custom policy xml.

Update to clarify: In the sign up example, the verification code is added to the LocalAccountSignUpWithLogonEmail TechnicalProfile:

I'm having an issue performing the following SABRe hotel book in the CRT environment using an orchestrated workflow.

I have a linux AMI 2 AWS instance with some services orchestrated via docker-compose, and I am using docker-compose up or docker-compose start commands to start them all. Now I am in the process to start/stop my ec2 instance automatically every day, but once it is started, I want to run some ssh to be able to change to the directory where docker-compose.yml file is, and then start it.

something like:

I have orchestrated a data pipe line using AWS Step function. In last state I want to send a custom notification. I'm using an Intrinsic function States.Format to format my message and subject. It works fine for Context object element. Here, I have tested that in Message parameter. But it doesn't work with input JSON. This is my input JSON { "job-param":{ "pipe-line-name":"My pipe line name", "other-keys":"other values" } }

I have been using Google CLoud for a few weeks now and I am facing a big problem for my limited GCP knowledge.

I have a python project whos goal is to "scrap" datas from a website using it's API. My project run a few tens of thousands of requests during executions and it can take very long (few hours, maybe more)

I have 4 python scripts in my project and it's all orchestrated by a bash script.

The execution is as follow :

• The first script check a CSV file with all the instructions for the requests, and exeute the requests, save all the results from the requests in CSV files
• Second script check the previously created CSV files and recreate an other CSV instruction file
• The first script run again but with the new instructions and again save results in CSV files
• Second script checks again and do the same again ...
• ... and so on a few times
• Third script cleans the datas, delete duplicates and create an unique CSV file
• Fourth script upload the final CSV file to bucket storage

Now I want to get ride of that bash script and I would like to automatize execution of thos scripts approx. once a week.

The problem here is the execution time. Here is what I already tested :

Google App Engine : The timeout of a request on GAE is limited to 10 minutes, and my functions can run for few hours. GAE is not usable here.

Google Compute Engine : My scripts will run max. 10-15 hours a week, keeping a compute engine up during all that time would be too pricey.

What could I do to automatize the execution of my scripts in a cloud environment ? What could be solutions I didn't though about, without changing my code ?

Thank you