croniter | python module to provide iteration | Date Time Utils library

Read the error message, where it is clearly stated you are missing a header file:

fatal error: rocksdb/slice.h: No such file or directory 705 | #include "rocksdb/slice.h" | ^~~~~~~~~~~~~~~~~ compilation terminated. error: command '/usr/bin/gcc' failed with exit code 1

Accordingly, you'll need to build and install RocksDB. This is separate from the installation of faust[rocksdb] with pip. That simply installs python-rocksdb, the Python interface to the underlying libraries.

There is even a (third-party) RocksDB docker image based on Python 3.7 Slim.

You could use that directly or take some tricks from the Dockerfile for that image.

You've set schedule_interval in the default_args, which propagates the given dict as arguments to all operators in the DAG. However, schedule_interval is an argument on DAG, so you must set it there:

This has happened to me once when I downloaded numpy using pip, but I used a computer with windows, and the solution was to move a file from one folder to another. The solution might be different on a Linux pc. Maybe this post can help you: Python pip install module is not found. How to link python to pip location?

The Mamba is conda-like package manager and needs compatible packages. The tasks is a pip package and ATM nobody has created recipe for conda/mamba for the tasks package. Because there is no recipe for that package, you are told that the package is not provided in channels you searched in (those were "conda-forge", "pkgs/main"...).

Solution:

Correct (recomended): Write a conda recipe for tasks and either use it locally, or create your channel on anaconda cloud or upload in to some public channel (This may take a while, as there is usually review process involved).

Punk (not recomended): Install your requirements except the tasks and then use pip to install the tasks. Note, that the changes made by pip to the env are not tracked by the mamba package manager. More info on the topic https://www.anaconda.com/blog/using-pip-in-a-conda-environment.

Notes: Search in public conda packages on anaconda cloud like this: https://anaconda.org/search?q=tasks

The website you are referring to uses the Quartz Scheduler which supports the fields

It seems that you are confusing the use-cases for PythonVirtualenvOperator and PythonOperator.

If you simply want to run a Python callable in a task (callable_virtualenv() in your case) you can use PythonOperator. In this case, it does not matter if you installed Airflow in a virtual environment, system wide, or using Docker.

What happens in your code is the following: PythonVirtualenvOperator creates another virtual environment (which is completely unrelated to the one in which you run Airflow), installs Airflow into it, and tries to import Variable. But this another Airflow installation is not configured and that is why you get those exceptions. You could set the AIRFLOW_HOME environment variable for this second Airflow installation to the same directory as used by the first Airflow installation, and this should actually work, but it looks like an overkill to me.

So, what you can do is install colorama into the same environment in which you installed Airflow and replace PythonVirtualenvOperator by PythonOperator.

BTW, those print() inside the callable would be redirected into a log file and not printed to terminal, so it probably does not make much sense to use colorama with them.

Maybe I shouldn't have posted the question so early, since it was a very simple error ...

The OAUTH_PROVIDERS variable should be an array!

I am on MacOS 10.15.7 Python version 3.8.2

I'm guessing you used the Python 3 bundled/pre-installed with macOS Catalina.

Something like the below