I run my proyect on PyCharm last version.
When I run my main.py it runs good and behaivor is desired. But If I restart (or re-run) the main.py, I got error
OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
No other isntances running. Python is the only thing on computer now. No other usages of ports. The only way to skip this error is or restarting computer or deleting every python from task manager.
Also, after stopping service, Swagger is still accesible with browser, I think that this swagger is using the port for main (because they run together) and that is why I can't run my main.py.
One way to reproduce this error is run this MWE, then stop and close PyCharm or just close Pycharm, then try to restart it and the message error comes again, and swagger is still up!. This happen while I'm coding and try to run the new things I wrote.
MWE:
from tornado.ioloop import IOLoop
from tornado_swagger.setup import setup_swagger
from tornado.options import parse_command_line,define,options
from tornado.web import Application,url,RequestHandler
class TestRequests(RequestHandler):
def get(self,var1):
"""
---
tags:
- Testing
summary: minimal working example
description: some dummy text
produces:
- requestsBody:
- application/json
parameters:
- name: var1
in: path
description: dummy tet
required: true
type: string
responses:
200:
description: dummy response for dummy requests here and now
"""
print(f"Reveived GET tests for {var1}")
res = {"result": "hello",
"product": str(var1)+"-foo"}
self.write(res)
class Applications(Application):
urls = [
url(r"/api/(?P<var1>[a-zA-Z0-9]+)/test",TestRequests)
]
def __init__(self):
settings = {'debug': True}
setup_swagger(self.urls,
swagger_url = '/swagger',
api_base_url="/Base",
description = "test swagger",
title = "test title")
super(Applications,self).__init__(self.urls,**settings)
define('port', default='8088', help='Port to listen on')
if __name__ == "__main__":
print("running")
parse_command_line()
app = Applications()
app.listen(options.port)
IOLoop.current().start()
Related
I want to use Fabric and run a command on local, without having to establish any additional connections.
How do I do this in fabric 2? ... documentation seems to miss to give any example.
The design decision to drop the local command in Fabric 2 makes this more difficult, but I was able to simulate it by using Context from Invoke instead of Connection:
from fabric import Connection
from invoke.context import Context
#task
def hostname(c):
c.run('hostname')
#task
def test(c):
conn = Connection('user#host')
hostname(conn)
local_ctx = Context(c.config) # can be passed into #task;
# Connection is a subclass of Context
hostname(local_ctx)
After several different attemps and spending lots of time I found this elegant solution for starting a server (pty=True) and run local commands.
fabfile.py
from fabric import task
#task
def env_test(c):
c.run("env", replace_env=False)
#task
def go(c):
c.run("manage.py runserver", replace_env=False, pty=True)
Please be aware again, these two commands are only meant for local development tasks!
Further Reading: Fabric2 Connections, Upgrading from 1.x
run, sudo, and local are done the same:
from fabric import Connection
cn = Connection('scott#104.131.61.12') # presumes ssh keys were exchanged
cn.run('ls -al') # assuming ssh to linux server - as scott
cn.sudo('whoami') # as root
cn.local('echo ---------- now from local')
cn.local('dir /w') # assuming client is windows
I am adding #TheRealChx101's comment as an answer because I ran into troubles with Connection.local.
Not all environment variables got into the pty, so some of my scripts did not work properly.
With the import from invoke run as local stanza (Invoke's local instead of Fabric's), everything worked fine.
# -*- coding: utf-8 -*-
from fabric import task
from invoke import run as local
#task(default=True)
def testwxmsg(c):
local("pytest --reuse-db --no-migrations tests/weixin/test_release_accrual.py")
This is similar to the answer by #phoibos, but I wanted to show that #task is not needed.
import sys
from fabric import Connection
from invoke.context import Context
target_host=sys.argv[1]
if target_host == 'localhost':
ctx = Context()
else:
ctx = Connection(target_host)
ctx.run('hostname', echo=False, hide=None)
if isinstance(ctx, Connection):
ctx.close()
Local:
> python demo.py localhost
MyComputerName
Remote:
> python demo.py demo.example.com
demo.example.com
I'm trying to write some unit tests for a flask application i'm building, this is the file that contains the flask application
app.py
from flask import Flask
app = Flask(__name__)
#app.route('/')
def home():
return 'hello world'
and then the file that contains the tests
tests.py
from unittest import TestCase, main
from app import app
from multiprocessing import Process
import requests
class Test(TestCase):
#classmethod
def setUpClass(cls):
cls.hostname = 'http://localhost:8000'
with app.app_context():
p = Process(target=app.run, kwargs={'port': 8000})
p.start()
cls.p = p
def test_options(self):
# print(self.p.is_alive()) # returns True but requests doesn't connect
result = requests.post(self.hostname).text
self.assertEqual(result, 'hello world')
#classmethod
def tearDownClass(cls):
cls.p.terminate() # works fine if i comment it out
pass
if __name__ == '__main__':
main()
I came up with an idea to use the requests module to test the application instead of the test_client that comes with flask.
From my understanding of the unittest module,
The setUpClass method is called once, that creates a new process and starts the application.
After that i'm free to run various tests with request.post or request.get.
Then at the end of the day, when all the tests have been run tearDownClass is supposed to terminate the process.
When i try to run the tests this is what i get (The stacktrace is actually longer than this, but it all boils down to this)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7faeeba183c8>: Failed to establish a new connection: [Errno 111] Connection refused',))
Because the flask process doesn't start and requests is unable to connect to localhost via port 8000.
but the strange part is if i comment out cls.p.terminate everything works properly and the test passes, the only problem is that i won't be able to kill the flask process again.
I know i can make the process daemonic or use the test_client like this:
class Test(TestCase):
#classmethod
def setUpClass(cls):
cls.hostname = 'http://localhost:8000'
with app.app_context():
app.config['TESTING'] = True
cls.app = app.test_client()
def test_options(self):
# print(self.p.is_live())
result = self.app.get().data
self.assertEqual(result, b'hello world')
and just move on with my life, but i really want to understand why my code doesn't work when i try to kill the process when i'm tearing down the test suite.
I've tried to use pdb to debug, but for some reason i don't know my terminal gets messed up when i try to type anything in the pdb prompt (Some letters don't show up in the terminal as i type, but when i hit enter i discover that the letters were actually there but were not visible, E.g to type print i might end up having prrrinnnttt because r and t refused to show up when i was pressing them)
I don't know what's happening in your case, but in one of my projects I use a bash script like this to test:
#!/bin/bash
set -eux
python app.py &
sleep 3
set +e
python tests.py
result=$?
kill $(ps aux | grep app.py | grep -v grep | awk '{print $2}')
exit $result
This handles bringing up the server, running the tests, and then killing the server, and there is no mention of handling the server in the python tests.
I'm running a cherrypy based app on an openshift gear. Recently I've been getting a "503 service temporarily unavailable" error whenever I try to go to the site. Inspecting the logs, I see I'm getting an ImportError where I try to import CherryPy. This is strange - CherryPy is listed as a dependency in my requirements.txt and used to be imported just fine. I double checked to make sure I'm getting the right path to the openshift activate_this.py and it seems to be correct. I'm not quite sure where to look next; any help would be appreciated. Thanks!
The failed import is at line 14 of app.py:
import os
import files
virtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'], 'virtenv')
virtualenv = os.path.join(virtenv, 'bin', 'activate_this.py')
conf = os.path.join(files.get_root(), "conf", "server.conf")
try:
execfile(virtualenv, dict(__file__=virtualenv))
print virtualenv
except IOError:
pass
import cherrypy
import wsgi
def mount():
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = os.environ['OPENSHIFT_APP_DNS']
cherrypy.config.update({"tools.staticdir.root": files.get_root()})
cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
cherrypy.tree.mount(wsgi.application(), "/", conf)
def start():
cherrypy.engine.start()
def end():
cherrypy.engine.exit()
if __name__ == "__main__":
mount()
start()
UPDATE
I eventually saw (when pushing to the openshift repo using git bash CLI) that the dependency installation from requirements.txt was failing with some exceptions I haven't bothered to look into yet. It then goes on to try to install dependencies in setup.py, and that works just fine.
Regarding the port in use issue...I have no idea. I changed my startup from tree.mount and engine.start to quickstart, and everything worked when I pushed to openshift. Just for kicks (and because I need it to run my tests), I switched back to cherrypy.tree.mount, pushed it, and it worked just fine.
Go figure.
I use the app.py entry point for Openshift. Here are several examples on how I start my server using the pyramid framework on Openshift. I use waitress as the server but I have also used the cherrypy wsgi server. Just comment out the code you don't want.
app.py
#Openshift entry point
import os
from pyramid.paster import get_app
from pyramid.paster import get_appsettings
if __name__ == '__main__':
here = os.path.dirname(os.path.abspath(__file__))
if 'OPENSHIFT_APP_NAME' in os.environ: #are we on OPENSHIFT?
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
config = os.path.join(here, 'production.ini')
else:
ip = '0.0.0.0' #localhost
port = 6543
config = os.path.join(here, 'development.ini')
app = get_app(config, 'main') #find 'main' method in __init__.py. That is our wsgi app
settings = get_appsettings(config, 'main') #don't really need this but is an example on how to get settings from the '.ini' files
# Waitress (remember to include the waitress server in "install_requires" in the setup.py)
from waitress import serve
print("Starting Waitress.")
serve(app, host=ip, port=port, threads=50)
# Cherrypy server (remember to include the cherrypy server in "install_requires" in the setup.py)
# from cherrypy import wsgiserver
# print("Starting Cherrypy Server on http://{0}:{1}".format(ip, port))
# server = wsgiserver.CherryPyWSGIServer((ip, port), app, server_name='Server')
# server.start()
#Simple Server
# from wsgiref.simple_server import make_server
# print("Starting Simple Server on http://{0}:{1}".format(ip, port))
# server = make_server(ip, port, app)
# server.serve_forever()
#Running 'production.ini' method manually. I find this method the least compatible with Openshift since you can't
#easily start/stop/restart your app with the 'rhc' commands. Mabye somebody can suggest a better way :)
# #Don't forget to set the Host IP in 'production.ini'. Use 8080 for the port for Openshift
# You will need to use the 'pre_build' action hook(pkill python) so it stops the existing running instance of the server on OS
# You also will have to set up another custom action hook so rhc app-restart, stop works.
# See Openshifts Origin User's Guide ( I have not tried this yet)
#Method #1
# print('Running pserve production.ini')
# os.system("pserve production.ini &")
#Method #2
#import subprocess
#subprocess.Popen(['pserve', 'production.ini &'])
I'm rather new to Python and Pyro4. So I try to follow the second example of this page Pyro - Python Remote Objects - 4.41, but when I run the server throw this exception:
Traceback (most recent call last):
File "greeting-server.py", line 10, in <module>
ns = Pyro4.locateNS() # find the name server
File "/usr/lib/python2.7/dist-packages/Pyro4/naming.py", line 344, in locateNS
raise e
Pyro4.errors.NamingError: Failed to locate the nameserver
Code Server:
# saved as greeting-server.py
import Pyro4
class GreetingMaker(object):
def get_fortune(self, name):
return "Hello, {0}. Here is your fortune message:\n" \
"Tomorrow's lucky number is 12345678.".format(name)
daemon = Pyro4.Daemon() # make a Pyro daemon
ns = Pyro4.locateNS() # find the name server
uri = daemon.register(GreetingMaker) # register the greeting maker as a Pyro object
ns.register("example.greeting", uri) # register the object with a name in the name server
print("Ready.")
daemon.requestLoop() # start the event loop of the server to wait for calls
Run pyro-ns in another terminial first:
$pyro-ns
*** Pyro Name Server ***
Name server listening on: ('0.0.0.0', 9090)
WARNING: daemon bound on hostname that resolves to loopback address 127.0.x.x
URI is: PYRO://127.0.1.1:9090/7f0001011d2a21ca9fb63702dd216e1143
URI written to: /home/guille/Documents/pyro examples/Pyro4-master/examples/banks/Pyro_NS_URI
Name Server started.
Remark: I work on Debian 8 and I've installed:
sudo apt-get install pyro4
sudo apt-get install python2-pyro4
pip install https://pypi.python.org/packages/2.7/s/serpent/serpent-
1.7-py2.py3-none-any.whl
to run this example
Maybe I missed something. Any ideas why this is not working, or things that I'm doing wrong?
thanks in advance.
This work for me:
Run python -m Pyro4.naming in another terminial first:
Not starting broadcast server for localhost.
NS running on localhost:9090 (127.0.0.1)
URI = PYRO:Pyro.NameServer#localhost:9090
and not pyro-ns I've done before for pyro4 as you see this procedure change
While the URI method in the docs is great, another way to connect is register the domain / IP using Pyro4 SimpleServe
Edit:
This was written for use with Python 3, thanks to #Cyberguille for pointing out that raw_input should be used instead of input on the client code when using Python 2.x
Server
Note that 0.0.0.0 exposes it to the world
# saved as greeting-server.py
import Pyro4
#Pyro4.expose
class GreetingMaker(object):
def get_fortune(self, name):
return "Hello, {0}. Here is your fortune message:\n" \
"Behold the warranty -- the bold print giveth and the fine print taketh away.".format(name)
Pyro4.Daemon.serveSimple({
GreetingMaker: 'Greeting',
}, host="0.0.0.0", port=9090, ns=False, verbose=True)
Then running python greeting-server.py to start the script
Client
# saved as greeting-client.py
import Pyro4
ipAddressServer = "" # TODO add your server remote IP here
# Works for Python3, see edit above for notes on Python 2.x
name = input("What is your name? ").strip()
greetingMaker = Pyro4.core.Proxy('PYRO:Greeting#' + ipAddressServer + ':9090')
print(greetingMaker.get_fortune(name)) # call method normally
I think you are mixing python 3 and python 2 versions here, because you wrote you had to install both 'pyro4' and 'python2-pyro4' packages.
I suspect the former is for python 3 and the latter is the legacy python 2 version.
The 'pyro-ns' shell command seems to launch an older, incompatible version of the name server.
I am working my way through learning Twisted, and have stumbled across something I'm not sure I'm terribly fond of - the "Twisted Command Prompt". I am fiddling around with Twisted on my Windows machine, and tried running the "Chat" example:
from twisted.protocols import basic
class MyChat(basic.LineReceiver):
def connectionMade(self):
print "Got new client!"
self.factory.clients.append(self)
def connectionLost(self, reason):
print "Lost a client!"
self.factory.clients.remove(self)
def lineReceived(self, line):
print "received", repr(line)
for c in self.factory.clients:
c.message(line)
def message(self, message):
self.transport.write(message + '\n')
from twisted.internet import protocol
from twisted.application import service, internet
factory = protocol.ServerFactory()
factory.protocol = MyChat
factory.clients = []
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
However, to run this application as a Twisted server, I have to run it via the "Twisted Command Prompt", with the command:
twistd -y chatserver.py
Is there any way to change the code (set Twisted configuration settings, etc) so that I can simply run it via:
python chatserver.py
I've Googled, but the search terms seem to be too vague to return any meaningful responses.
Thanks.
I don't know if it's the best way to do this but what I do is instead of:
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
you can do:
from twisted.internet import reactor
reactor.listenTCP(1025, factory)
reactor.run()
Sumarized if you want to have the two options (twistd and python):
if __name__ == '__main__':
from twisted.internet import reactor
reactor.listenTCP(1025, factory)
reactor.run()
else:
application = service.Application("chatserver")
internet.TCPServer(1025, factory).setServiceParent(application)
Hope it helps!
Don't confuse "Twisted" with "twistd". When you use "twistd", you are running the program with Python. "twistd" is a Python program that, among other things, can load an application from a .tac file (as you're doing here).
The "Twisted Command Prompt" is a Twisted installer-provided convenience to help out people on Windows. All it is doing is setting %PATH% to include the directory containing the "twistd" program. You could run twistd from a normal command prompt if you set your %PATH% properly or invoke it with the full path.
If you're not satisfied with this, perhaps you can expand your question to include a description of the problems you're having when using "twistd".
On windows you can create .bat file with your command in it, use full paths, then just click on it to start up.
For example I use:
runfileserver.bat:
C:\program_files\python26\Scripts\twistd.py -y C:\source\python\twisted\fileserver.tac
Maybe one of run or runApp in twisted.scripts.twistd modules will work for you. Please let me know if it does, it will be nice to know!
I haven't used twisted myself. However, you may try seeing if the twistd is a python file itself. I would take a guess that it is simply managing loading the appropriate twisted libraries from the correct path.
I am successfully using the simple Twisted Web server on Windows for Flask web sites.
Are others also successfully using Twisted on Windows, to validate that configuration?
new_app.py
if __name__ == "__main__":
reactor_args = {}
def run_twisted_wsgi():
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)
reactor.listenTCP(5000, site)
reactor.run(**reactor_args)
if app.debug:
# Disable twisted signal handlers in development only.
reactor_args['installSignalHandlers'] = 0
# Turn on auto reload.
import werkzeug.serving
run_twisted_wsgi = werkzeug.serving.run_with_reloader(run_twisted_wsgi)
run_twisted_wsgi()
old_app.py
if __name__ == "__main__":
app.run()