airflow startup failed due to gunicorn - python

Traceback (most recent call last):
File "./python_runtime/bin/airflow", line 15, in <module>
args.func(args)
File "/home/admin/airflow/python_runtime/lib/python2.7/site-packages/airflow/bin/cli.py", line 423, in webserver
'gunicorn', run_args
File "/home/admin/airflow/python_runtime/lib/python2.7/os.py", line 346, in execvp
_execvpe(file, args)
File "/home/admin/airflow/python_runtime/lib/python2.7/os.py", line 385, in _execvpe
func(fullname, *argrest)
OSError: [Errno 2] No such file or directory
I find out that this error was cause by no gunicorn avaiable,so I print all paths in os.py:
/home/admin/airflow/python_runtime/bin/gunicorn
/home/admin/python2.7.2/bin/gunicorn
/home/admin/tools/local/bin/gunicorn
/home/admin/tools/bin/gunicorn
/home/admin/tools/opsdb/gunicorn
/home/admin/odps_tools/dship/gunicorn
/home/admin/php/bin/gunicorn
/home/admin/cmake/bin/gunicorn
/home/admin/jdk1.6.0_24/bin/gunicorn
/home/admin/tools/bin/gunicorn
/usr/kerberos/bin/gunicorn
/usr/local/sbin/gunicorn
/usr/sbin/gunicorn
/sbin/gunicorn
/usr/local/bin/gunicorn
/bin/gunicorn
/usr/bin/gunicorn
/usr/X11R6/bin/gunicorn
while gunicorn exist in this path
/home/admin/airflow/python_runtime/bin/gunicorn
something wrong with my env?

Airflow seems to expect gunicorn in PATH. For my ubuntu instance this fixed it:
$ export PATH=$PATH:~/.local/bin
$ airflow webserver

On MacOS, it wasn't enough with the proposed solution. If I used relative path (with either ~ or $HOME) it didn't work, but using absolute path solved the issue: PATH="${PATH}:/path/to/.venv/bin"
As you can see, I'm using a hidden folder that I created with Python venv, maybe this is connected.
To test it, is not enough to run gunicorn alone, this worked to me with the relative path as well, but which gunicorn returned nothing, and this indicates that airflow webserver will fail to start.

Related

importlib.metadata.PackageNotFoundError: No package metadata was found for djoser pyinstaller

Context
I made a Django react app. Now I want to make it a desktop application so that the user does not have type python manage.py runserver and also activate the environment every time. I used pyinstaller. I did all the steps mentioned for django.
Problem
when I run my executable file made from pyinstaller, I got this error
File "manage.py", line 5, in <module>
File "PyInstaller\loader\pyimod02_importers.py", line 499, in exec_module
File "djoser\__init__.py", line 6, in <module>
File "importlib\metadata\__init__.py", line 955, in version
File "importlib\metadata\__init__.py", line 928, in distribution
File "importlib\metadata\__init__.py", line 518, in from_name
importlib.metadata.PackageNotFoundError: No package metadata was found for djoser
[2200] Failed to execute script 'manage' due to unhandled exception!
What I have done
I have already installed Djoser in the environment and the environment is also activated. I have also tried to add in manage.py file and also in hidden_import lists but nothing changed. I have also tried adding --copy-meta=djoser in the build command but it got even worse error.
How do I fix this error and If there are any better alternative solutions out there? Thanks
Well I have found for myself another solution . To run this django file by creating a batch file
#echo off
set "VENV_NAME=myenv"
set "PROJECT_DIR=C:\path\to\project"
cd /d %~dp0
cd /d %PROJECT_DIR%
call "%VENV_NAME%\Scripts\activate.bat"
python manage.py runserver
manage.exe runserver --norelaod

Virtual Python enviroment with WINSW

I am trying to create a Windows Service which uses a Virtual Python Environment. I create the environment as described here with venv. To create the Windows Service I use winsw.
While I can both run a program in the virtual environment and setup a windows service I cannot figure out how to do both at once.
My first attempt was to call "call win_env/Scripts/activate.bat" in the .bat file which is used by the windows service. This did not work, as he still used the main enviroment and therefore ignored this line of code. Also the second answer in this issue suggests an solution. This also did not work for me as the program still used the main enviroment.
Possible this is because I do not use the env variables in the code but I cannot figure out how I would start the virtual env from there.
When I write that it uses the "main environment" I track this because the program fails because of some missing dependencies (wrong versions). These dependencies are installed in the virtual environment but not in the main environment. If I call the bat files without going into the virtual environment the program fails with the same message.
Error Message:
Exception in thread {X}:
Traceback (most recent call last):
File "{Path}\scoop\apps\anaconda3\current\lib\threading.py", line 973, in _bootstrap_inner
self.run()
File "{Path}\scoop\apps\anaconda3\current\lib\threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "{Path}\Sources\{Project}\.\populate_cache.py", line 25, in query_failures_from_db
df = pd.read_sql(statement, con, params=[date_from, date_to])
File "{Path}\Sources\{Project}\win_env\lib\site-packages\pandas\io\sql.py", line 563, in read_sql
pandas_sql = pandasSQL_builder(con)
File "{Path}\Sources\{Project}\win_env\lib\site-packages\pandas\io\sql.py", line 744, in pandasSQL_builder
import sqlite3
File "{Path}\scoop\apps\anaconda3\current\lib\sqlite3\__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "{Path}\scoop\apps\anaconda3\current\lib\sqlite3\dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.
.bat file:
call uvicorn main:app --port 8590 --host 0.0.0.0
winsw file:
<?xml version="1.0"?>
<!--This configuration file should be placed near the WinSW executable, the name should be the same.E.g. for myapp.exe the configuration file name should be myapp.xmlYou can find more information about configuration options here: https://github.com/kohsuke/winsw/blob/master/doc/xmlConfigFile.md -->
<configuration>
<id>Test Programm</id>
<name>Test Programm</name>
<description> XXX. </description>
<executable>{PathToBatfile}</executable>
<onfailure delay="10 sec" action="restart"/>
<onfailure delay="20 sec" action="restart"/>
<onfailure action="none"/>
<resetfailure>1 hour</resetfailure>
<workingdirectory>{PathToWorkingdirectory}</workingdirectory> // .bat file is located here
<priority>Normal</priority>
<stoptimeout>15 sec</stoptimeout>
<stopparentprocessfirst>true</stopparentprocessfirst>
<startmode>Automatic</startmode>
<waithint>15 sec</waithint>
<sleeptime>1 sec</sleeptime>
<logpath>{PathToLog}</logpath>
<!--OPTION: logDefines logging mode for logs produced by the executable.Supported modes:* append - Rust update the existing log* none - Do not save executable logs to the disk* reset - Wipe the log files on startup* roll - Rotate logs based on size* roll-by-time - Rotate logs based on time* rotate - Rotate logs based on size, (8 logs, 10MB each). This mode is deprecated, use "roll"Default mode: appendEach mode has different settings.See https://github.com/kohsuke/winsw/blob/master/doc/loggingAndErrorReporting.md for more details -->
<log mode="roll"> </log>
</configuration>
Based on the input you provided, I think the following batch file should work:
call %~dp0win_env/Scripts/activate.bat
uvicorn main:app --port 8590 --host 0.0.0.0
(The %~dp0 resolves to the directory where the batch file is stored, so it works independent of your current working directory.)
Note that the activate.bat does nothing special. It just updates your PATH environment variable and changes the command prompt, which is convenient during development. However, you could also just call the Python executable from your virtual environment without "activating" it. I'm not familiar with uvicorn, but it seems like it can be called as a Python module as well. In that case you could also use the following batch file:
win_env/Scripts/python.exe -m uvicorn main:app --port 8590 --host 0.0.0.0

How to get into the python shell within Odoo 8 environment?

I would like to use the Odoo Framework from the shell.
I installed the module "Shell command backport" (technical name: shell), but I couldn't make it work.
$ ./odoo.py shell --addons-path=/opt/odoo_8/src/linked-addons -d database_name
Traceback (most recent call last):
File "./odoo.py", line 160, in <module>
main()
File "./odoo.py", line 157, in main
openerp.cli.main()
File "/opt/odoo_8/src/OCA/OCB/openerp/cli/__init__.py", line 58, in main
for m in module.get_modules():
File "/opt/odoo_8/src/OCA/OCB/openerp/modules/module.py", line 351, in get_modules
plist.extend(listdir(ad))
File "/opt/odoo_8/src/OCA/OCB/openerp/modules/module.py", line 346, in listdir
return map(clean, filter(is_really_module, os.listdir(dir)))
OSError: [Errno 2] No such file or directory: '/opt/odoo8/openerp/addons'
Where is defined the path /opt/odoo8/openerp/addons? I checked this similar question as well.
If I don't write the addons path in the command the error appears again.
I read the answer of this other question, I tried the module and the script option but they didn't work. What should I do to make it work? Any hint would help.
Check your .opererp_serverrc for the user you are executing the command as. In the users home directory you will find this file. There may be reference to the addons_path. The path it appears to be looking for /opt/odoo8/openerp/addons differs from what you have specified in your command. I would check your config files.

Django manage.py runserver fails with references to strange temporary pip build directories

I am trying to write a script to start the server in my production environment (using apache mod_wsgi):
#!/bin/bash
#activate virtualenv
source ~/.virtualenvs/myproj/bin/activate
# install possibly changed requirements
echo "Checking changed dependencies..."
pip3 install --no-cache-dir -r /opt/myDjangoProj/requirements.txt
deactivate
# run server
echo "Starting django server ..."
#export DJANGO_SETTINGS_MODULE='myDjangoProj.settings.production'
~/.virtualenvs/myproj/bin/python3 /opt/myDjangoProj/manage.py runserver 8000 --settings myDjangoProj.settings.production
However, I alwas get this stacktrace: (installing requirements works fine, so I omit it)
Starting django server ...
Traceback (most recent call last):
File "/opt/myDjangoProj/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/localadm/.virtualenvs/myproj/lib/python3.2/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "/home/localadm/.virtualenvs/myproj/lib/python3.2/site-packages/django/core/management/__init__.py", line 312, in execute
django.setup()
File "/home/localadm/.virtualenvs/myproj/lib/python3.2/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/localadm/.virtualenvs/myproj/lib/python3.2/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "/tmp/pip-build-w8vru4/Django/django/contrib/admin/apps.py", line 22, in ready
File "/tmp/pip-build-w8vru4/Django/django/contrib/admin/__init__.py", line 24, in autodiscover
The directory /tmp/pip-build-w8vru4/ does not actually exist. Do you have any ideas what the problem might be?
First thing is generally we don't need shell scripts when we are using mod_wsgi.
refer https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/modwsgi/ for setting up django with mod_wsgi
Second your working directory not seams to be django project root. switch to django project root or include it in PYTHONPATH.

Gunicorn+Django+Heroku. Python Path Issue

I'm running a Django app that I used to deploy to Heroku with Gunicorn without a hitch. By that I mean that I could run
foreman start
on my localhost and be able to understand whether the application would work in the production environment. I recently made the decision to move all of our project's applications into a subfolder named 'applications'. This meant editing my DJANGO_SETTINGS_MODULE environment variable among other files. After doing this, I experimented with foreman and gunicorn until I got the local server up and running. At this point I deployed to production. Needless to say, there was an error:
Running `gunicorn --workers=4 applications.rocketlistings.wsgi -b 0.0.0.0: -k gevent` attached to terminal... up, run.8052
Traceback (most recent call last):
File "/app/.heroku/python/bin/gunicorn", line 9, in <module>
load_entry_point('gunicorn==0.17.2', 'console_scripts', 'gunicorn')()
File "/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 343, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2307, in load_entry_point
return ep.load()
File "/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2013, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named app.wsgiapp
I'm not sure what I'm doing wrong, and it's very difficult to debug the situation given the obscure remote setup heroku runs. Other threads discussing this error suggest that this is a path problem. This sounds very plausible, but I can't think of what could be wrong with the path, given that I have an init.py file in my applications folder, and everything works locally.
Here is my production path/pythonpath anyway:
Running `echo $PATH` attached to terminal... up, run.2583
/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin
Running `echo $PYTHONPATH` attached to terminal... up, run.8026
/app/
Thanks in advance!
Sometimes, this occurs when Gunicorn finds another module called 'app' in the PYTHONPATH. Setting PYTHONPATH to a fake value resolves the issue:
$ heroku config:set PYTHONPATH=fakepath
https://github.com/heroku/heroku-buildpack-python/wiki/Troubleshooting#no-module-named-appwsgiapp

Categories