I have an issue testing my flask API app where it won't consistently start up. Because of requirements, I have to make sure it runs on CentOS6 (python 2.6 by default), but I use a custom python 2.7 virtualenv. My startup command is as follows:
/opt/mist/mist_base/env/bin/uwsgi --py-autoreload 1 --master --socket=/tmp/mist_app.sock --pidfile=/tmp/mist_app.pid --module=wsgi --honour-stdin --chdir=/opt/mist/mist_base/app --threads=1 --virtualenv=/opt/mist/mist_base/env --die-on-term --uid mist --gid mist
where the custom virtualenv is /opt/mist/mist_base/env.
I had someone help me package this software to be installable using RPM/Yum (he had done it before and packaging it this way was a requirement); during this process he insisted that he build another custom python virtualenv (/opt/mist/mist_base/.localpython). Unfortunately he has since left the company, and we don't have details as to what was done. I looked for RPM build/init commands that would call binaries/libraries in that other virtualenv but I didn't find any.
Using the instructions provided to me, I built the resulting RPM and installed it on my test server and everything runs just fine. I gave the same RPM to my customer to install in his test server and it does not start.
My working boot sequence is as follows:
*** Starting uWSGI 2.0.14 (64bit) on [Thu May 18 13:49:09 2017] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-17) on 20 March 2017 07:18:55
os: Linux-2.6.32-696.1.1.el6.x86_64 #1 SMP Tue Apr 11 17:13:24 UTC 2017
nodename: mistdb
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/mist/mist_base/app
writing pidfile to /tmp/mist_app.pid
detected binary path: /opt/mist/mist_base/env/bin/uwsgi
setgid() to 487
setuid() to 493
chdir() to /opt/mist/mist_base/app
your processes number limit is 7394
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/mist_app.sock fd 3
Python version: 2.7.5 (default, Mar 20 2017, 06:43:30) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
Set PythonHome to /opt/mist/mist_base/env
Python main interpreter initialized at 0x21c4aa0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x21c4aa0 pid: 24902 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 24902)
spawned uWSGI worker 1 (pid: 24907, cores: 1)
Python auto-reloader enabled
Meanwhile this is what my customer sees:
*** Starting uWSGI 2.0.14 (64bit) on [Tue May 16 13:12:26 2017] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-17) on 20 March 2017 07:18:55
os: Linux-2.6.32-696.1.1.el6.x86_64 #1 SMP Tue Mar 21 12:19:18 EDT 2017
nodename: mist-testbed
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
writing pidfile to /tmp/mist_app.pid
detected binary path: /opt/mist/mist_base/env/bin/uwsgi
setgid() to 497
setuid() to 497
chdir() to /opt/mist/mist_base/app
your processes number limit is 15208
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/mist_app.sock fd 3
Python version: 2.7.5 (default, Mar 20 2017, 06:43:30) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]
Set PythonHome to /opt/mist/mist_base/env
Python main interpreter initialized at 0x280c890
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
File "/opt/mist/mist_base/.localpython/lib/python2.7/hashlib.py", line 139, in <module>
globals()[__func_name] = __get_hash(__func_name)
File "/opt/mist/mist_base/.localpython/lib/python2.7/hashlib.py", line 103, in __get_openssl_constructor
return __get_builtin_constructor(name)
File "/opt/mist/mist_base/.localpython/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
Traceback (most recent call last):
File "./wsgi.py", line 1, in <module>
from mist_main import app as application
File "./mist_main.py", line 1, in <module>
from _app import return_app
File "./_app.py", line 1, in <module>
from flask import Flask, Blueprint, render_template, session, redirect, url_for, escape, request, abort
File "/opt/mist/mist_base/env/lib/python2.7/site-packages/flask/__init__.py", line 17, in <module>
from werkzeug.exceptions import abort
File "/opt/mist/mist_base/env/lib/python2.7/site-packages/werkzeug/__init__.py", line 151, in <module>
__import__('werkzeug.exceptions')
File "/opt/mist/mist_base/env/lib/python2.7/site-packages/werkzeug/exceptions.py", line 71, in <module>
from werkzeug.wrappers import Response
File "/opt/mist/mist_base/env/lib/python2.7/site-packages/werkzeug/wrappers.py", line 26, in <module>
from werkzeug.http import HTTP_STATUS_CODES, \
File "/opt/mist/mist_base/env/lib/python2.7/site-packages/werkzeug/http.py", line 32, in <module>
from hashlib import md5
ImportError: cannot import name md5
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 25071)
spawned uWSGI worker 1 (pid: 25072, cores: 1)
Python auto-reloader enabled
During startup, the app first tries to use the .localpython virtualenv, then it tries using the env virtualenv. Since I can't find any RPM/init code that uses .localpython, I can't explain why 1. my app instance doesn't use it 2. my customer's instance does.
It's entirely possible it's due to a difference in environments, and we're going to try and build a VM based on their specs and see if that resolves this issue. Could this also simply be a case of having a broken virtualenv and/or not having all the needed external libraries not included?
My customer came back today and it looks like they do something additional to their CentOS 6 servers that we don't do. Specifically, they run a script to FIPS-enable their servers and that somehow screws with how my custom python 2.7 environment interacts with the built-in encryption libraries.
Related
I've setup a python3.9 django project with uWSGI 2.0.12. It seems that every few seconds or so the service crashes and restarts due to some bug in python related to the random seed and a KeyboardInterrupt.
Does anyone have any idea what is going on here? In general the app works, but every so often its not responsive.
The following log lines repeat themselves over and over again.
*** Starting uWSGI 2.0.18 (64bit) on [Sat Jul 30 00:25:13 2022] ***
compiled with version: 11.2.1 20210728 (Red Hat 11.2.1-2) on 27 January 2021 00:00:00
os: Linux-5.14.0-80.el9.x86_64 #1 SMP PREEMPT Thu Apr 21 09:50:22 UTC 2022
nodename: abc.dev
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /etc/uwsgi.d
detected binary path: /usr/sbin/uwsgi
chdir() to /opt/abc-api/mysite
your processes number limit is 15518
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/sockets/api.abc.dev.sock fd 6
Python version: 3.9.13 (main, Jun 9 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]
PEP 405 virtualenv detected: /opt/abc-api/venv
Set PythonHome to /opt/abc-api/venv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xb32450
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145840 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0xb32450 pid: 219059 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 219059)
spawned uWSGI worker 1 (pid: 219060, cores: 1)
Sat Jul 30 00:25:13 2022 - received message 0 from emperor
SIGINT/SIGQUIT received...killing workers...
Traceback (most recent call last):
File "/usr/lib64/python3.9/random.py", line 126, in seed
def seed(self, a=None, version=2):
KeyboardInterrupt
Update:
I've tried many things - including going back python3.7 (couldn't get it to work, different issue...). Finally I went back to 3.9 setup, but instead of using a socket file, I decided on http socket at 127.0.0.1:8181. Now this issue is not happening. Zero clue what's going on, but looks like it has to do with the socket file.
I am trying to run one sample application with uWSGI file directive using two files for it testService.ini(config file) & testService.py (application file).
**testService.ini**
[uwsgi]
wsgi-file = '/opt/testService/testService.py'
callable = app
**testService.py**
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
return [b"Hello World"]
running the above file using : uwsgi --http :9090 --wsgi-file testService.ini
Output:
Starting uWSGI 2.0.17 (64bit) on [Thu Jun 21 15:52:33 2018]
compiled with version: 5.4.0 20160609 on 14 June 2018 10:09:32
os: Linux-4.13.0-45-generic #50~16.04.1-Ubuntu SMP Wed May 30 11:18:27 UTC 2018
nodename: ubuntu
machine: x86_64
clock source: unix
detected number of CPU cores: 8
current working directory: /opt/testService
detected binary path: /home/jmallick/.local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 15481
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :9090 fd 4
spawned uWSGI http 1 (pid: 7848)
uwsgi socket 0 bound to TCP address 127.0.0.1:45781 (port auto-assigned) fd 3
Python version: 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609]
Python threads support is disabled. You can enable it with --enable-threads
Python main interpreter initialized at 0x824400
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72920 bytes (71 KB) for 1 cores
Operational MODE: single process
File "testService.ini", line 2
**wsgi-file = '/opt/testService/testService.py'
SyntaxError: can't assign to operator
failed to compile python file testService.ini**
unable to load app 0 (mountpoint='') (callable not found or import error)
no app loaded. going in full dynamic mode
uWSGI is running in multiple interpreter mode
spawned uWSGI worker 1 (and the only) (pid: 7847, cores: 1)
And over the browser I am getting "Internal Server Error". Can someone help me understanding what I am doing wrong here ?
I'm trying to setup uwsgi (and nginx eventually), but right now am stuck on uwsgi. I'm getting the following error when I try to run the server normally:
could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
And this is the output from running command uwsgi --http :8000 --module Hellfish.wsgi:
*** Starting uWSGI 2.0.17 (64bit) on [Tue Jul 3 18:34:40 2018] ***
compiled with version: 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2) on 03 July 2018 20:44:01
os: Darwin-16.7.0 Darwin Kernel Version 16.7.0: Fri Apr 27 17:59:46 PDT 2018; root:xnu-3789.73.13~1/RELEASE_X86_64
nodename: Whodinis-MacBook-Pro.local
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /Users/whodini/Desktop/who/Web/Hellfish
detected binary path: /Users/whodini/Desktop/who/Web/Hellfish/env/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 709
your memory page size is 4096 bytes
detected max file descriptor number: 256
lock engine: OSX spinlocks
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :8000 fd 5
spawned uWSGI http 1 (pid: 70999)
uwsgi socket 0 bound to TCP address 127.0.0.1:59020 (port auto-assigned) fd 4
Python version: 3.6.2 (v3.6.2:5fd33b5926, Jul 16 2017, 20:11:06) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x7fd92082c800
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72888 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
base dir path
Traceback (most recent call last):
File "./Hellfish/wsgi.py", line 16, in <module>
application = get_wsgi_application()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
django.setup(set_prefix=False)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/django/apps/config.py", line 94, in create
module = import_module(entry)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'django_countries'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 70998, cores: 1)
Not sure why it can't import module Django_countries. I'm kind of operating blind here because this is the first time I've done this and having trouble following docs and tutorials. I've tried to kill the port and restart it but came up with this issue again. Also not sure if my .ini file is in the right place or if its configured correctly. I put it in the same directory as my Hellfish.wsgi file. Hellfish is the name of my project. Finally, here is the contents of my .ini file:
[uwsgi]
socket = 127.0.0.1:8000
chdir = /Users/whodini/Desktop/who/Web/Hellfish/
home = /Users/whodini/Desktop/who/Web/Hellfish/env
module = Hellfish.wsgi:application
wsgi-file = Hellfish/wsgi.py
master = true
pidfile=/tmp/project-master.pid
vacuum=True
max-requests=5000
processes = 4
threads = 2
stats = 127.0.0.1:8000
Some help here would be much appreciated.
When I use uwsgi to connect the
$ uwsgi --http :8000 --plugin python --module myProject.wsgi
I get the bellow error:
!!! UNABLE to load uWSGI plugin: dlopen(./python_plugin.so, 10): image not found !!!
There are whole traceback:
(venv) MacBook-Pro:myProject ldl$ uwsgi --http :8000 --plugin python --module myProject.wsgi
!!! UNABLE to load uWSGI plugin: dlopen(./python_plugin.so, 10): image not found !!!
*** Starting uWSGI 2.0.17 (64bit) on [Wed May 9 11:24:54 2018] ***
compiled with version: 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2) on 09 May 2018 02:32:54
os: Darwin-16.7.0 Darwin Kernel Version 16.7.0: Mon Nov 13 21:56:25 PST 2017; root:xnu-3789.72.11~1/RELEASE_X86_64
nodename: MacBook-Pro.local
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /Users/sss/Desktop/QIYUN/Project/myProject
detected binary path: /Users/sss/Desktop/myProject/Project/myProject/venv/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 709
your memory page size is 4096 bytes
detected max file descriptor number: 10240
lock engine: OSX spinlocks
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :8000 fd 4
spawned uWSGI http 1 (pid: 5266)
uwsgi socket 0 bound to TCP address 127.0.0.1:54904 (port auto-assigned) fd 3
Python version: 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x7f82d9e00590
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72888 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
File "./myProject/wsgi.py", line 18, in <module>
from 管理员后台.网站管理admin_website_chat.views import sio
File "./管理员后台/网站管理/admin_website_chat/views.py", line 2, in <module>
from 管理员后台.用户管理.admin_usermanage.models import User
File "./管理员后台/用户管理/admin_usermanage/models.py", line 6, in <module>
from django.contrib.auth.models import AbstractUser
File "/Users/sss/Desktop/QIYUN/Project/myProject/venv/lib/python3.5/site-packages/django/contrib/auth/models.py", line 4, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/Users/sss/Desktop/myProject/Project/myProject/venv/lib/python3.5/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
class AbstractBaseUser(models.Model):
File "/Users/sss/Desktop/myProject/Project/myProject/venv/lib/python3.5/site-packages/django/db/models/base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "/Users/sss/Desktop/myProject/Project/myProject/venv/lib/python3.5/site-packages/django/apps/registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "/Users/sss/Desktop/myProject/Project/myProject/venv/lib/python3.5/site-packages/django/apps/registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 5265, cores: 1)
My computer is Macbook Pro 2015.
remove the option --plugin python and the error should go away.
I want to be able to create dynamic graphs within my Flask application. I am running this application within a virtual environment on a Linux box.
I looked around for different libraries that would help me with this, and identified Pygal as the one which I would like to use.
So, I activated my virtual environment using source venv/bin/activate and installed using pip install pygal. Everything installed correctly and I can import it perfectly in using the regular Python interface.
However, when I put import pygal into my Flask application and run the application, I get the following error in the logs.
Traceback (most recent call last):
File "/usr/share/nginx/www/mydir/run.py", line 2, in <module>
from app import app
File "/usr/share/nginx/www/mydir/app/__init__.py", line 23, in <module>
from app import views, models
File "/usr/share/nginx/www/mydir/views.py", line 9, in <module>
import datetime, locale, pygal
ImportError: No module named pygal
Does anyone know how I can resolve this?
Many thanks.
-- EDIT --
RE: Sean Vieira
Results of adding from sys import path; print(path) to my run script:
*** Starting uWSGI 1.9.20 (32bit) on [Tue Feb 25 14:55:16 2014] ***
compiled with version: 4.6.3 on 04 December 2013 05:11:40
os: Linux-3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013
nodename: **hidden**
machine: i686
clock source: unix
detected number of CPU cores: 1
current working directory: /etc/uwsgi/vassals
detected binary path: /usr/share/nginx/www/mydir/venv/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 3840
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /usr/share/nginx/www/mydir/mydir_uwsgi.sock fd 3
Python version: 2.7.3 (default, Sep 26 2013, 20:26:19) [GCC 4.6.3]
Set PythonHome to /usr/share/nginx/www/mydir/venv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x93e80a8
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 169864 bytes (165 KB) for 1 cores
*** Operational MODE: single process ***
added /usr/share/nginx/www/mydir/ to pythonpath.
['/usr/share/nginx/www/mydir/', '.', '', '/usr/share/nginx/www/mydir/venv/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg', '/usr/share/nginx/www/mydir/venv/lib/python2.7/site-packages/pip-1.1-py2.7.egg', '/usr/share/nginx/www/mydir/venv/lib/python2.7', '/usr/share/nginx/www/mydir/venv/lib/python2.7/plat-linux2', '/usr/share/nginx/www/mydir/venv/lib/python2.7/lib-tk', '/usr/share/nginx/www/mydir/venv/lib/python2.7/lib-old', '/usr/share/nginx/www/mydir/venv/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/share/nginx/www/mydir/venv/lib/python2.7/site-packages']
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x93e80a8 pid: 20964 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 20964)
spawned uWSGI worker 1 (pid: 20981, cores: 1)
Issue resolved.
I had to add the path to the package into the sys.path as described in an answer for this question. As I couldn't get Pygal to install within my virtual environment for some reason.