Python Requests + Azure VM - python

I'm using python requests library to talk between two of my apps (django and ckan) inside Azure VM (clean ubuntu 12.04).
And very wired bug appears. It seems like request.post sent successfully, but on the middle of my code running I got 504 request timeout.
Apache2, Nginx and Gunicorn (only timeout appears) logs are clean. My CKAN app throws same error but with different ports:
CKAN error example 1:
Exception happened during processing of request from ('127.0.0.1', 53555)
Traceback (most recent call last):
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/httpserver.py", line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
self.wfile.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
CKAN error example 2:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 53344)
Traceback (most recent call last):
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/httpserver.py", line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
self.wfile.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
As you can see port Is always different. One of my thoughts -- Azure has all ports closed unless you specify which one you want to open.
I have same codebase working alright locally.

By default, Azure linux VMs disable all ports except 22 to prevent inbound requests from internet. you can try to specify your Django app's port and configure the endpoint of the other VM with ckan.
Also, you can refer to https://github.com/Azure/azure-content/blob/master/articles/virtual-machines/virtual-machines-python-django-web-app-linux.md to deploy your django app on apache and bind it on 80 port on your VM.

Related

how can fixed this problem: ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

after install django when will start browsing at http://127.0.0.1:8000/ then this massage will show in powershell.
Exception happened during processing of request from ('127.0.0.1', 3575)
[06/Feb/2020 09:48:03] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 200 85876
[06/Feb/2020 09:48:03] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 200 85692
[06/Feb/2020 09:48:03] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 86184
Traceback (most recent call last):
File "c:\users\fuel\appdata\local\programs\python\python38\Lib\socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "c:\users\fuel\appdata\local\programs\python\python38\Lib\socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "c:\users\fuel\appdata\local\programs\python\python38\Lib\socketserver.py", line 720, in __init__
self.handle()
File "C:\Users\Fuel\.virtualenvs\sample-DQH4k00R\lib\site-packages\django\core\servers\basehttp.py", line 174, in handle
self.handle_one_request()
File "C:\Users\Fuel\.virtualenvs\sample-DQH4k00R\lib\site-packages\django\core\servers\basehttp.py", line 182, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "c:\users\fuel\appdata\local\programs\python\python38\Lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
this is the photo:
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
how can i fixed it?
I found this link on another stack Overflow link, is it possible you are using an outdated version of Python? It was reportedly fixed 2019-07-22 according to python's bugs website

Gunicorn is failing with OSError: [Errno 107] Transport endpoint is not connected

I was facing this issue while running my application [Docker Image] with the gunicorn version 19.9.0 in Openshift. I'm not using the nginx to set [secure_scheme_headers]:https://github.com/benoitc/gunicorn/issues/1766.
Gunicorn Configuration
gunicorn --chdir /src/app wsgi:application --bind 0.0.0.0:8000 --workers 4 --timeout 180 -k gevent
error
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 66, in handle
six.reraise(*sys.exc_info())
File "/usr/local/lib/python3.7/site-packages/gunicorn/six.py", line 625, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 49, in handle
req = six.next(parser)
File "/usr/local/lib/python3.7/site-packages/gunicorn/http/parser.py", line 41, in __next__
self.mesg = self.mesg_class(self.cfg, self.unreader, self.req_count)
File "/usr/local/lib/python3.7/site-packages/gunicorn/http/message.py", line 181, in __init__
super(Request, self).__init__(cfg, unreader)
File "/usr/local/lib/python3.7/site-packages/gunicorn/http/message.py", line 54, in __init__
unused = self.parse(self.unreader)
File "/usr/local/lib/python3.7/site-packages/gunicorn/http/message.py", line 230, in parse
self.headers = self.parse_headers(data[:idx])
File "/usr/local/lib/python3.7/site-packages/gunicorn/http/message.py", line 74, in parse_headers
remote_addr = self.unreader.sock.getpeername()
OSError: [Errno 107] Transport endpoint is not connected```
# Fix
I was able to fix the issue by shifting the older version of gunicorn==19.7.1
and gevent didn't work with the older version of the gunicorn.
https://github.com/benoitc/gunicorn/issues/1913
As mentioned in the comments, this is the same issue as #1993 and it is fixed in the pull request #2277. It is merged and available in the latest version of Gunicorn, version 20.1.0
So by updating to this version of Gunicorn, the problem is solved for me.

Connect to mysql server in a docker container from another docker container

In a nutshell
Using Dockers Desktop Community Version 2.0.0.3 (31259) in macOS Mojave I have two containers:
Alpine
MySQL
From the Alpine docker I am not be able to access the mysql server instance that is running on MySQL docker.
In details
On Alpine I am running python2 and python package mysqlclient. This is the configuration file that I use in my python application to access the mysql databases:
dbtype MySQLdb
connect-args {"user":"abe","db":"abe","passwd":"123"}
upgrade
host docker-mysql
port 32779
When I launch python application I see that it is unable to connect to mysqld server, in fact I get following error log:
python -m Abe.abe --config abe-my.conf
Reconnecting after rollback error: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)")
Reconnecting to database.
Reconnecting after rollback error: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)")
Reconnecting to database.
Reconnecting after rollback error: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)")
Reconnecting to database.
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/bitcoin-abe/Abe/abe.py", line 2123, in <module>
sys.exit(main(sys.argv[1:]))
File "/bitcoin-abe/Abe/abe.py", line 2117, in main
store = make_store(args)
File "/bitcoin-abe/Abe/abe.py", line 156, in make_store
store = DataStore.new(args)
File "Abe/DataStore.py", line 3427, in new
return DataStore(args)
File "Abe/DataStore.py", line 191, in __init__
store.initialize()
File "Abe/DataStore.py", line 637, in initialize
store.configure()
File "Abe/DataStore.py", line 944, in configure
config = store._sql.configure()
File "Abe/SqlAbstraction.py", line 643, in configure
sql.configure_ddl_implicit_commit()
File "Abe/SqlAbstraction.py", line 734, in configure_ddl_implicit_commit
if sql._test_ddl():
File "Abe/SqlAbstraction.py", line 741, in _test_ddl
sql.drop_table_if_exists("%stest_1" % sql.prefix)
File "Abe/SqlAbstraction.py", line 712, in drop_table_if_exists
sql._drop_if_exists("TABLE", obj)
File "Abe/SqlAbstraction.py", line 709, in _drop_if_exists
sql.rollback()
File "Abe/SqlAbstraction.py", line 632, in rollback
sql.reconnect()
File "Abe/SqlAbstraction.py", line 305, in reconnect
return sql.conn()
File "Abe/SqlAbstraction.py", line 288, in conn
sql._conn = sql.connect()
File "Abe/SqlAbstraction.py", line 249, in connect
conn = sql._connect(cargs)
File "Abe/SqlAbstraction.py", line 281, in _connect
return sql.module.connect(**cargs)
File "/usr/lib/python2.7/site-packages/MySQLdb/__init__.py", line 84, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/MySQLdb/connections.py", line 164, in __init__
super(Connection, self).__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)")
How it could be "fixed"?
I saw that the access file is /run/mysqld/mysqld.sock and I found it in the mysql docker, I suppose it should not be a problem to be able to connect the two dockers so that they "see each other", but at the moment I have no idea.

Flask Google Cloud App Engine: OSError: [Errno 98] Address already in use

I am trying to deploy a flask app on google cloud app engine. It runs smooth in my virtual environment locally but I get an 502 error running it in the cloud.
Now I am trying to debug my code on the cloud server, using debug mode and SSH into my instance. Using docker exec -it [ID] /bin/bash I am able to to access the root of my application. Now I upon running python app.py I get the following error:
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
Traceback (most recent call last):
File "app.py", line 479, in <module>
app.run(port=8080)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 941, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 814, in run_simple
inner()
File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 774, in inner
fd=fd)
File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 660, in make_server
passthrough_errors, ssl_context, fd=fd)
File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 577, in __init__
self.address_family), handler)
File "/usr/local/lib/python3.6/socketserver.py", line 453, in __init__
self.server_bind()
File "/usr/local/lib/python3.6/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/local/lib/python3.6/socketserver.py", line 467, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
I've been trying to kill those processes listed when I run:
ps -fA | grep python
However, this does not solve the problem of the address being in use. Also changing the port in the app.run() does not solve the issue for me.
I had a similar problem, it was caused by the Flask app also being run when the module was loaded, because I had
if __name__ == "__main__":
app.run()
at the bottom. Note that the recent requirement to name your sever file "main.py" could cause this bug to emerge.
I think the problem is that you don't need to specialize the port for cloud. Google Cloud finds the port to run your app on its own. So instead of app.run(port=8080) just write app.run()
While I was not able to figure out how to "free" the running address, I solved the problem by starting another flask process by running it on a different port like so:
flask run --port=80

Start HTTP/HTTPS server, python -m SimpleHTTPServer

I am trying to start HTTPS server on my localhost, but getting below error. I have opened cmd as an administrator.
C:\Windows\system32>python -m SimpleHTTPServer
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Python27\lib\SimpleHTTPServer.py", line 224, in <module>
test()
File "C:\Python27\lib\SimpleHTTPServer.py", line 220, in test
BaseHTTPServer.test(HandlerClass, ServerClass)
File "C:\Python27\lib\BaseHTTPServer.py", line 595, in test
httpd = ServerClass(server_address, HandlerClass)
File "C:\Python27\lib\SocketServer.py", line 419, in __init__
self.server_bind()
File "C:\Python27\lib\BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "C:\Python27\lib\SocketServer.py", line 430, in server_bind
self.socket.bind(self.server_address)
File "C:\Python27\lib\socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 10013] An attempt was made to access a socket in a way forb
idden by its access permissions
Your windows firewall may block python from listening.
Additionaly, try adding a different port number higher than 1024 to the command line and see if it helps.
I was not able to start since some service is already running on 8000,
I was able to successfully start the service on another port 2000
C:\>python -m SimpleHTTPServer 2000
Serving HTTP on 0.0.0.0 port 2000 ..
One can get the PID/status for port 8000 by netstat -ano
C:\>netstat -ano
Active Connections
Proto Local Address Foreign Address State PID
TCP 0.0.0.0:912 0.0.0.0:0 LISTENING 2544
TCP 0.0.0.0:8000 0.0.0.0:0 LISTENING 4
TCP 0.0.0.0:19781 0.0.0.0:0 LISTENING 3100
shows PID 4 which is a system process.

Categories