python-edi | EDI message generator in Python | Form library

After carefully reading log file, here's what I recommend to do for clean deploy.

While venv is activated and you're in project directory (where manage.py live), do the following:

• Open requirements.txt
• Delete unneeded modules from requirements.txt
• Uninstall unneeded modules with pip uninstall module
• Now run pip freeze requirements.txt
• Deactivate venv
• Then add requirements.txt to source control git add -A
• Commit your changes git commit -m "deleted unneeded modules"
• If you have existed Heroku app, connect to it with heroku git:remote -a myapp, else pass this step
• Deploy, git push heroku master

If you are unsure which modules your project need and I can't determin exactly the project's requirements but if you have basic project, you only need 3 modules which are Django, psycopg2-binary, gunicorn, so do the following for clean deploy:

• Delete venv
• Outside project directory, create new one
• Activate venv
• Install the modules you need, probably Django, psycopg2-binary, gunicorn
• Now run pip freeze requirements.txt
• Deactivate venv
• Then add requirements.txt to source control git add -A
• Commit your changes git commit -m "deleted unneeded modules"
• If you have existed Heroku app, connect to it with heroku git:remote -a myapp, else pass this step
• Deploy, git push heroku master

You're using old mysqlclient 1.3.12 with new MySQL 8. Either you need to downgrade MySQL to version 5.6. Or you need to use later mysqlclient.

The incompatibility was fixed in commit a2ebbd2 on Dec 21, 2017 so you need a later version of mysqlclient.

mysqlclient 1.3.13 was released on Jun 27, 2018. Try it or any later version.

The error comes from pip-tools trying to access the editable attribute on the class ParsedRequirement, whereas the correct attribute name on that class is is_editable. With previous versions of pip, the object at ireq were of type InstallRequirement, which does have the attribute editable.

Try pip==20.0.2; that seems to be the last version that returned InstallRequirement instead of ParsedRequirement from the relevant method (parse_requirements).

I reached out to AWS support. Meghana was in charge of this case.

Here is the reply:

It seems that you are confusing the use-cases for PythonVirtualenvOperator and PythonOperator.

If you simply want to run a Python callable in a task (callable_virtualenv() in your case) you can use PythonOperator. In this case, it does not matter if you installed Airflow in a virtual environment, system wide, or using Docker.

What happens in your code is the following: PythonVirtualenvOperator creates another virtual environment (which is completely unrelated to the one in which you run Airflow), installs Airflow into it, and tries to import Variable. But this another Airflow installation is not configured and that is why you get those exceptions. You could set the AIRFLOW_HOME environment variable for this second Airflow installation to the same directory as used by the first Airflow installation, and this should actually work, but it looks like an overkill to me.

So, what you can do is install colorama into the same environment in which you installed Airflow and replace PythonVirtualenvOperator by PythonOperator.

BTW, those print() inside the callable would be redirected into a log file and not printed to terminal, so it probably does not make much sense to use colorama with them.

Figured it out.

What works:

Maybe I shouldn't have posted the question so early, since it was a very simple error ...

The OAUTH_PROVIDERS variable should be an array!

• conda only keeps track of the packages it installed
• pip freeze will give you the packages that were either installed using pip package manager or they used setuptools in their setup.py so conda build generated the egg information.

conda vs pip

Downgrading the pip may fix this issue, you can check this out: conda issues

I am on MacOS 10.15.7 Python version 3.8.2

I'm guessing you used the Python 3 bundled/pre-installed with macOS Catalina.