DAG | DAG tool supports traversal of a Directed Acyclic Graph

The proper syntax is for example:

By default, every task in Airflow should succeed for a next task to start running. So if your email-task is the last task in your DAG, that automatically means all previous tasks have succeeded.

Alternatively, you could configure on_success_callback and on_failure_callback on your DAG, which executes a given callable. This passes in arguments to determine whether the DAG run failed or succeeded:

In the last template, replace {{inputs.artifacts.result}} with ”/tmp/templates_lst.txt”.

inputs.artifacts.NAME has no meaning in the source field, so Argo leaves it as-is. Python tries to interpret it as code, which is why you get an exception.

The proper way to communicate an input artifact to Python in Argo is to specify the artifact destination (which you’ve done) in the templates input definition. Then in Python, use files from that path the same way you would do in any Python app.

Nest your code in a wrapper and use position: sticky;. Also, you were using a lot of padding which seemed unnecessary so I removed it. See the CSS changes I made below.

1. Fully define your output parameters

Your output parameter spec is incomplete. You need to specify where the output parameter comes from.

Since you have multiple output parameters, you can't just use standard out ({{tasks.prepare-lst.outputs.parameters.result}}). You have to write two files and derive an output parameter from each.

2. Load the JSON array so it's iterable

If you iterate over the string representation of the array, you'll just get one character at a time.

3. Use an environment variable to pass input to Python

Although it's not strictly necessary, I consider it best practice. If a malicious actor had the ability to set the message parameter, they could inject Python into your workflow. Pass the parameter as an environment variable so the string remains a string.

Changes:

I think you are looking for: https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_api/airflow/providers/google/cloud/operators/functions/index.html#airflow.providers.google.cloud.operators.functions.CloudFunctionInvokeFunctionOperator

Note that in order to use in 1.10 you need to have backport provider packages installed (but I believe they are installed by default) and version of the operator might be slightly different due to backport packages not being released for quite some time already.

In Airflow 2

I would like to know if release+acquire can violate happens-before.

Happens-before relationship cannot be "violated", as it is a guarantee. Meaning, if you established it in a correct way, it will be there, with all its implications (unless there is a bug in the compiler).

However, establishing just any happens-before relationship doesn't guarantee that you've avoided all possible race conditions. You need to establish carefully chosen relationships between relevant operations, that will eliminate all scenarios when data race is possible.

Let's review this code snippet:

The 'run_as_user' feature uses 'sudo' to switch to airflow user in non-interactive mode. The sudo comand will never (no matter what parameters you specify including -E) preserve PATH variable unless you do sudo in --interactive mode (logging in by the user). Only in the --interactive mode the user's .profile , .bashrc and other startup scripts are executed (and those are the scripts that set PATH for the user usually).

All non-interactive 'sudo' command will have path set to secure_path set in /etc/sudoers file.

My case here:

Posting this Community Wiki for better visibility.

As mentioned in the comment section by @LEC this configuration is compatible with Cloud Composer V1 which can be found in GCP Documentation Triggering DAGs with Cloud Functions.

At the moment there can be found two tabs Cloud Composer 1 Guides and Cloud Composer 2 Guides. Under Cloud Composer 1 is code used by the OP, but if you will check Cloud Composer 2 under Manage DAGs > Triggering DAGs with Cloud Functions you will get information that there is not proper documentation yet.

This documentation page for Cloud Composer 2 is not yet available. Please use the page for Cloud Composer 1.

As solution, please use Cloud Composer V1.