I am trying to setup apache airflow on aws ec2(ubuntu 18.04)
Python version: 3.6.9
Pip Version: 9.0.1
To Install Apache Airflow I have used below command.
pip3 install apache-airflow[postgres,celery,rabbitmq,ldap,crypto,aws]
while I execute airflow initdb it shows me error
File "/home/ubuntu/.local/bin/airflow", line 26, in <module>
from airflow.bin.cli import CLIFactory
File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/bin/cli.py", line 71, in <module>
from airflow.www_rbac.app import cached_app as cached_app_rbac
File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 28, in <module>
from flask_appbuilder import AppBuilder, SQLA
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/__init__.py", line 6, in <module>
from .base import AppBuilder # noqa: F401
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/base.py", line 8, in <module>
from .api.manager import OpenApiManager
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/api/manager.py", line 7, in <module>
from flask_appbuilder.baseviews import BaseView
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/baseviews.py", line 21, in <module>
from .forms import GeneralModelConverter
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/forms.py", line 17, in <module>
from .fieldwidgets import (
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask_appbuilder/fieldwidgets.py", line 3, in <module>
from wtforms.widgets import html_params, HTMLString
ImportError: cannot import name 'HTMLString'
can somebody help me with this?
Thanks, Mariusz for the help.
I followed the steps mentioned
https://github.com/apache/airflow/issues/8506#issuecomment-617515743
and add --constraint https://raw.githubusercontent.com/apache/airflow/1.10.10/requirements/requirements-python3.7.txt after my pip install command and it got working.
It was an error with WTForms new release version
Related
On Ubuntu 21.04 with python 3.9, trying to install airflow. When I ran airflow init db command, below error is thrown
File "", line 855, in
exec_module File "", line 228, in
call_with_frames_removed File "/home/user/USB/.venv/lib/python3.9/site-packages/airflow/migrations/versions/82b7c48c147f_remove_can_read_permission_on_config.py",
line 29, in
from airflow.www.app import create_app File "/home/user/USB/.venv/lib/python3.9/site-packages/airflow/www/app.py",
line 24, in
from flask_appbuilder import SQLA File "/home/user/USB/.venv/lib/python3.9/site-packages/flask_appbuilder/init.py",
line 6, in
from .base import AppBuilder # noqa: F401 File "/home/user/USB//.venv/lib/python3.9/site-packages/flask_appbuilder/base.py",
line 8, in
from .api.manager import OpenApiManager File "/home/user/USB//.venv/lib/python3.9/site-packages/flask_appbuilder/api/manager.py",
line 8, in
from flask_appbuilder.baseviews import BaseView File "/home/user/USB//.venv/lib/python3.9/site-packages/flask_appbuilder/baseviews.py",
line 21, in
from .forms import GeneralModelConverter File "/home/user/USB//.venv/lib/python3.9/site-packages/flask_appbuilder/forms.py",
line 16, in
from .fields import EnumField, QuerySelectField, QuerySelectMultipleField File
"/home/user/USB//.venv/lib/python3.9/site-packages/flask_appbuilder/fields.py",
line 6, in
from wtforms.compat import string_types, text_type
I tried installing wtforms separately.
airflow was installing with pipenv
Any idea?
thanks
Just install airflow including constraints files as explained in installation documentation.
Airflow has complex dependencies and specific way of installation and in order to install it.in consistent way, you need to use constraints.
https://airflow.apache.org/docs/apache-airflow/stable/installation.html#installation-script
Try to use version 2.3.3 for WTForms and its compatible version marshmallow for example 2.21.0. I added this at the begging of my requirements.txt
marshmallow==2.21.0
WTForms==2.3.3
when trying to run the command: airflow webserver - i get the below error message
i am having issues getting the application to launch.
python 3.8.0
postgress latest version
Airflow Latest Version
we are having troubles finding an answer to this in the forums / through basic google searching, thank you in advance!
~$ airflow webserver
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/usr/local/lib/python3.8/dist-packages/airflow/__init__.py", line 34, in <module>
from airflow import settings
File "/usr/local/lib/python3.8/dist-packages/airflow/settings.py", line 35, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401
File "/usr/local/lib/python3.8/dist-packages/airflow/configuration.py", line 1115, in <module>
conf = initialize_config()
File "/usr/local/lib/python3.8/dist-packages/airflow/configuration.py", line 829, in initialize_config
from cryptography.fernet import Fernet
File "/usr/lib/python3/dist-packages/cryptography/fernet.py", line 17, in <module>
from cryptography.hazmat.primitives import hashes, padding
File "/usr/lib/python3/dist-packages/cryptography/hazmat/primitives/padding.py", line 13, in <module>
from cryptography.hazmat.bindings._padding import lib
ModuleNotFoundError: No module named '_cffi_backend'
I got below error when running 'aws' on a ec2 instance. Any clues what this problem is about?
>>aws configure
Traceback (most recent call last):
File "/bin/aws", line 19, in <module>
import awscli.clidriver
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 17, in <module>
import botocore.session
File "/usr/lib/python2.7/site-packages/botocore/session.py", line 27, in <module>
import botocore.configloader
File "/usr/lib/python2.7/site-packages/botocore/configloader.py", line 19, in <module>
from botocore.compat import six
File "/usr/lib/python2.7/site-packages/botocore/compat.py", line 26, in <module>
from urllib3 import exceptions
File "/usr/lib/python2.7/site-packages/urllib3/__init__.py", line 10, in <module>
from .connectionpool import (
File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 31, in <module>
from .connection import (
File "/usr/lib/python2.7/site-packages/urllib3/connection.py", line 45, in <module>
from .util.ssl_ import (
File "/usr/lib/python2.7/site-packages/urllib3/util/__init__.py", line 4, in <module>
from .request import make_headers
File "/usr/lib/python2.7/site-packages/urllib3/util/request.py", line 5, in <module>
from ..exceptions import UnrewindableBodyError
ImportError: cannot import name UnrewindableBodyError
In order to install aws cli on rhel 8 follow these steps
install python3-pip
pip3 install awscli --upgrade --user
confirm it by this
aws --version
if you want to check out docs on installation.
However my recommendation will be to NOT store your aws credentials ( programmatic key) within your instance because if by any chance your instance is compromised so does your account too, or if instance is shared among other users, they will get access to your creds.
if it was local or personal computer i would have recommended to go with aws configure command but since its not, the best practice will be to
create IAM role,give permission according to your use case, and attach that role to your ec2 instance and then run aws cli commands.
I've got an issue when I want to init my Flask DB, here is Terminal result :
user#Pc:~/Documents/test$ flask db init
Traceback (most recent call last):
File "/home/user/.local/bin/flask", line 7, in <module>
from flask.cli import main
File "/home/user/.local/lib/python3.6/site-packages/flask/__init__.py", line 17, in <module>
from werkzeug.exceptions import abort
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/__init__.py", line 151, in <module>
__import__('werkzeug.exceptions')
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/exceptions.py", line 71, in <module>
from werkzeug.wrappers import Response
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/wrappers/__init__.py", line 30, in <module>
from .request import PlainRequest
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/wrappers/request.py", line 6, in <module>
from .user_agent import UserAgentMixin
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/wrappers/user_agent.py", line 1, in <module>
from ..useragents import UserAgent
File "/home/user/.local/lib/python3.6/site-packages/werkzeug/useragents.py", line 212, in <module>
from werkzeug.wrappers import UserAgentMixin # noqa
ImportError: cannot import name 'UserAgentMixin'
I'm working on an Ubuntu 18.04 LTS, using Flask, SQLAlchemy, Python and PostgreSQL. From where does this error occurs, and how can I solve it?
P.S : I followed the mega tutorial about Flask
EDIT 1 : Werkzeug is already installed
(venv) user#Pc:~/Documents/test$ pip install Werkzeug
Requirement already satisfied: Werkzeug in ./venv/lib/python3.6/site-packages
EDIT 2 : I don't know why I got this issue, but this error disappear after I removed spaces indentations
Try pip install Werkzeug
UserAgentMixin is from werkzeug!
ref: https://tedboy.github.io/flask/generated/generated/werkzeug.UserAgentMixin.html
in my case, I found a file and a folder, both named "wrappers" in the package path, so I reinstall the package:
pip uninstall Werkzeug
pip install Werkzeug
though a little late, it may still help
This is the hints,how can I resolve it?
I use Python 3.5.1 created a virtual envirement by virtualenv
The source code works well on my friend's computer machine
Error:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "A:\Python3.5\lib\site-packages\django\core\management\__init__.py", line 385, in execute_from_command_line
utility.execute()
File "A:\Python3.5\lib\site-packages\django\core\management\__init__.py", line 354, in execute
django.setup()
File "A:\Python3.5\lib\site-packages\django\__init__.py", line 18, in setup
from django.utils.log import configure_logging
File "A:\Python3.5\lib\site-packages\django\utils\log.py", line 13, in <module>
from django.views.debug import ExceptionReporter, get_exception_reporter_filter
File "A:\Python3.5\lib\site-packages\django\views\debug.py", line 10, in <module>
from django.http import (HttpResponse, HttpResponseServerError,
File "A:\Python3.5\lib\site-packages\django\http\__init__.py", line 4, in <module>
from django.http.response import (
File "A:\Python3.5\lib\site-packages\django\http\response.py", line 13, in <module>
from django.core.serializers.json import DjangoJSONEncoder
File "A:\Python3.5\lib\site-packages\django\core\serializers\__init__.py", line 23, in <module>
from django.core.serializers.base import SerializerDoesNotExist
File "A:\Python3.5\lib\site-packages\django\core\serializers\base.py", line 6, in <module>
from django.db import models
File "A:\Python3.5\lib\site-packages\django\db\models\__init__.py", line 6, in <module>
from django.db.models.query import Q, QuerySet, Prefetch # NOQA
File "A:\Python3.5\lib\site-packages\django\db\models\query.py", line 13, in <module>
from django.db.models.fields import AutoField, Empty
File "A:\Python3.5\lib\site-packages\django\db\models\fields\__init__.py", line 18, in <module>
from django import forms
File "A:\Python3.5\lib\site-packages\django\forms\__init__.py", line 6, in <module>
from django.forms.fields import * # NOQA
File "A:\Python3.5\lib\site-packages\django\forms\fields.py", line 18, in <module>
from django.forms.utils import from_current_timezone, to_current_timezone
File "A:\Python3.5\lib\site-packages\django\forms\utils.py", line 15, in <module>
from django.utils.html import format_html, format_html_join, escape
File "A:\Python3.5\lib\site-packages\django\utils\html.py", line 16, in <module>
from .html_parser import HTMLParser, HTMLParseError
File "A:\Python3.5\lib\site-packages\django\utils\html_parser.py", line 12, in <module>
HTMLParseError = _html_parser.HTMLParseError
AttributeError: module 'html.parser' has no attribute 'HTMLParseError'
As you can read here this error is raised...
because HTMLParseError is deprecated from Python 3.3 onwards and removed in Python 3.5.
What you can do is downgrade your Python version or upgrade your Django version.
I've just got the same error here. The Django version installed on my machine was 1.7.
Upgrading to Django 1.8.* solved the problem for me.
You can upgrade your Django using following command:
If you are using pip3:
sudo pip3 install django --upgrade
If pip:
sudo pip install django --upgrade
pip install Django --upgrade
python manage.py runserver