I am trying to learn Twisted, a Python framework, and I want to put a basic application online that, when it receive a message sends it back. I decided to use Heroku to host it, and I followed the instructions on their docs.
import os
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
port = int(os.environ.get('PORT', 5000))
reactor.listenTCP(port, EchoFactory(), interface = '0.0.0.0')
reactor.run()
Its all working except (and I know this is a stupid question), how do I send a message to it? When I am working locally I just do telnet localhost <port>, but now I have no idea.
Also, since heroku connects to a random port how can I know what port it connects my app to?
Thanks.
I'm not very familiar with Twisted, but I'm not sure what you're trying to do is supported on Heroku. Heroku currently only supports HTTP[S] requests and not raw TCP. There are more details in the answers to this question.
If you wanted to connect to your app, you should use the myapp.herokuapp.com host name or any custom domain that you've added.
"Pure Python applications, such as headless processes and evented web frameworks like Twisted, are fully supported on Cedar."
Reference: https://devcenter.heroku.com/articles/python-support
Related
I have an existing Python system that receives messages using Rabbit MQ. What is the absolute easiest way to get these events pushed to a browser using WebSockets using Python? Bonus if the solution works in all major browsers too.
Thanks,
Virgil
Here https://github.com/Gsantomaggio/rabbitmqexample I wrote an complete example that uses tornado and RabbitMQ.
You can find all the instruction from the site:
anyway ..you need:
pip install pika
pip install tornado
First you register your rabbitmq:
def threaded_rmq():
channel.queue_declare(queue="my_queue")
logging.info('consumer ready, on my_queue')
channel.basic_consume(consumer_callback, queue="my_queue", no_ack=True)
channel.start_consuming()
then you register your web-sockets clients:
class SocketHandler(tornado.websocket.WebSocketHandler):
def open(self):
logging.info('WebSocket opened')
clients.append(self)
def on_close(self):
logging.info('WebSocket closed')
clients.remove(self)
When you get a message, you can redirect it to the web-socket page.
def consumer_callback(ch, method, properties, body):
logging.info("[x] Received %r" % (body,))
# The messagge is brodcast to the connected clients
for itm in clients:
itm.write_message(body)
You could use Twisted, txAMQP and Autobahn|Python on the server to write a bridge in probably 50 lines of code, and Autobahn|JS on the browser side. Autobahn implements WebSocket, and WAMP on top, which provides you with Publish & Subscribe (as well as Remote Procedure Calls) over WebSocket.
When using raw WebSocket you would have to invent your own Publish & Subscribe over WebSocket - since I guess that is what you are after: extending the AMQP PubSub to the Web. Or you could check out STOMP.
Disclaimer: I am original author of WAMP and Autobahn.
This question already has answers here:
Python: Binding Socket: "Address already in use"
(13 answers)
Closed 8 years ago.
I have a twisted python server which runs on port 8080, and i have written different API's which runs on this server. so i want all these API's to run on a single port no. but when i try to use same port all API's for ex : 8081 and run at the same time using python interpreter. at that time i am getting this error : twisted.internet.error.CannotListenError: Couldn't listen on any:8081: [Errno 98] Address already in use.
As i am new to twisted so don't know much of the things and there is no proper documentation on twisted . please someone guide me to solve this error :
Here is the Code snippets :
from twisted.internet import epollreactor
epollreactor.install()
from zope.interface import implements
from twisted.internet import reactor,interfaces
from functools import partial
from pymongo import Connection
import json
from bson.objectid import ObjectId
import server_1
import replacePlus_Space
global data, data_new, datadB, coll_auth, coll_person
class Login_user(server_1.HTTPEchoFactory):
def __init__(self):
server_1.HTTPEchoFactory.__init__(self,"testsite")
server_1.HTTPEchoFactory.initResources(self)
self.initResources()
def initResources(self):
print "in Login"
self.responses["/login_user"] = partial(self.user_login)
# To connect to DB and Check error cases and insert into mongoDB..!!!
def user_login(self, client):
# some functinality..!!!!
d = Login_user()
reactor.listenTCP(8081,d)
reactor.run()
Second code snippet is :
from twisted.internet import epollreactor
epollreactor.install()
from zope.interface import implements
from twisted.internet import reactor,interfaces
from functools import partial
from pymongo import Connection
import json
from bson.objectid import ObjectId
import server_1
import replacePlus_Space
class AddFriendDB(server_1.HTTPEchoFactory):
def __init__(self):
server_1.HTTPEchoFactory.__init__(self,"testsite")
server_1.HTTPEchoFactory.initResources(self)
self.initResources()
def initResources(self):
print "in add friend"
self.responses["/addFriend_DB"] = partial(self.addFriendDB)
# To connect to DB and Check error cases and insert into mongoDB..!!!
def addFriendDB(self, client):
#some functionality here..!!!
d = AddFriendDB()
reactor.listenTCP(8081,d)
reactor.run()
Translating the question
Many of the specifics in your question don't make sense, so I'm translating your question into:
I have a twisted python standalone program which runs on port 8080, and i have written a different standalone program which runs on this server. I want both programs to run on a single port no. When i try to use same port for all programs ex : using port 8081 for both programs. I am getting this error :twisted.internet.error.CannotListenError: Couldn't listen on any:8081: [Errno 98] Address already in use. As i am new to twisted i don't know much of the things and there is no proper documentation on twisted. please someone guide me to solve this error.
Twisted has great documentation
You commented that:
[...] there is no proper documentation on twisted
That statement is blatantly false.
Twisted certainly has proper documentation, and as compared to most frameworks it has excellent documentation. To name just a few of the many high quality documentation sources:
Dave Peticola's (krondo) Twisted Introduction - an excellent and deep introduction to twisted that begins by explaining the technology Twisted is built upon. If you really want to understand twisted, this (long) introduction is the place to start
The high quality and extensive documentation on on Twisted's primary website twistedmatrix.com
The source itself. Twisted has well commented and surprisingly understand source, if the above documentation doesn't teach you what you need figure it out from the code.
What causes "Address already in use"
As previously answered by Bart Friederichs:
Only one process can listen on a certain port. If you start process 1 to listen on port 8081, and then start another process on that same port, you get this error.
It is an error from the TCP stack, not from python or twisted.
This is a fundament truth of TCP/IP stacks across all operating systems (or at least all process based operating systems that could run Python).
The error is your operating system reminding you that when data comes to a IP port the OS/IP-stack was designed to only forward to one process, a process that is implementing an application level protocol on that port. The error happens when a second program attempts to re-register a port some other program has already registered.
Work-arounds in general
Upon running into a port reuse issue like this you have to ask yourself:
Are the two programs even running the same application-level protocol?
If they are the same protocol, does the protocol have routing such that for any given transaction the correct sub-program/routine be identified?
Are the two programs at a different location in the routing?
If they aren't the same protocol (I.E. one was HTTP and the other is FTP) or they are the same protocol but its a protocol that doesn't have routing (I.E. two instances of NTP) then there is no easy way to mix them because your trying to break the laws of IP (and the way that application protocol implementations are built). The only solution will be to encapsulate one (or both) of the protocol(s) into a common protocol that also has application-level routing (I.E. encapsulating FTP inside of HTTP and using the URI to route)
If they are the same protocol, and the protocol provides for a per-transaction routing (I.E. the URI inside of HTTP transactions) and they aren't at the same routing location, then there are easier solutions, namely:
Merge the two application into one.
If they are the same routable protocol but at a different location (I.E. HTTP protocol with the URI for the first program at /login and second program at /addfriend) it should be trivial to pull out the post-routing logic of the two programs/process and merge them into one program that does both functions
Front-end the programs with a redirector (This solution is only easy with HTTP because of the tools available).
If you have HTTP protocol programs that have routing to separate locations, but for some reason its hard to merge the logic together (I.E. one program is written in Java, the other in Python) then you can look at front-ending both programs with a redirector like nginx
The way you would use a redirector in a case like this is to run your two apps on two different unused ports (I.E. 12001, 12002), and then run the redirector on the port you want the service to be on, running it with a config file that will redirect to your two programs via their unique routing locations (via configs like SF: How to redirect requests to a different domain/url with nginx)
Code example
The following three programs illustrate the process of merging two programs into one, so both sets of logic can be accessed from the same port.
Two separate programs:
If you run the following code a web server will be started up on localhost:8081. If you then point your web browser at http://127.0.0.1:8081/blah the blah page will be displayed.
#!/usr/bin/python
from twisted.internet import defer, protocol, reactor # the reactor
from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource
class BlahPage(Resource):
idLeaf = True
def render_GET(self, request):
return "<html><body>Blah Page!</body></html>"
class ShutdownPage(Resource):
def render_GET(self, request):
reactor.stop()
webroot = Resource()
webroot.putChild("blah", BlahPage())
webroot.putChild("shutdown", ShutdownPage())
def main():
# Register the webserver (TCP server) into twisted
webfactory = Site(webroot)
reactor.listenTCP(8081, webfactory)
print ("Starting server")
reactor.run()
if __name__ == '__main__':
main()
This code will start a web server on localhost:8082. If you then point your web browser at http://127.0.0.1:8082/foo the foo page will be displayed.
#!/usr/bin/python
from twisted.internet import defer, protocol, reactor # the reactor
from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource
class FooPage(Resource):
idLeaf = True
def render_GET(self, request):
return "<html><body>Foo Page!</body></html>"
class ShutdownPage(Resource):
def render_GET(self, request):
reactor.stop()
webroot = Resource()
webroot.putChild("foo", FooPage())
webroot.putChild("shutdown", ShutdownPage())
def main():
# Register the webserver (TCP server) into twisted
webfactory = Site(webroot)
reactor.listenTCP(8082, webfactory)
print ("Starting server")
reactor.run()
if __name__ == '__main__':
main()
Merging the logic
This code is the merger of the two previous programs, as you can see it only required copying a small amount of code to glue both of the above into one that allows for access to http://127.0.0.1:8080/blah and http://127.0.0.1:8080/blah.
#!/usr/bin/python
from twisted.internet import defer, protocol, reactor # the reactor
from twisted.web.server import Site # make the webserver go
from twisted.web.resource import Resource
class BlahPage(Resource):
idLeaf = True
def render_GET(self, request):
return "<html><body>Blah Page!</body></html>"
class FooPage(Resource):
idLeaf = True
def render_GET(self, request):
return "<html><body>Foo Page!</body></html>"
class ShutdownPage(Resource):
def render_GET(self, request):
reactor.stop()
webroot = Resource()
webroot.putChild("foo", FooPage())
webroot.putChild("blah", BlahPage())
webroot.putChild("shutdown", ShutdownPage())
def main():
# Register the webserver (TCP server) into twisted
webfactory = Site(webroot)
reactor.listenTCP(8080, webfactory)
print ("Starting server")
reactor.run()
if __name__ == '__main__':
main()
Only one process can listen on a certain port. If you start process 1 to listen on port 8081, and then start another process on that same port, you get this error.
It is an error from the TCP stack, not from python or twisted.
Fix it either by choosing different ports for each process, or create a forking server.
I am trying to learn Twisted, a Python framework, and I want to put a basic application online that, when it receive a message sends it back. I decided to use Heroku to host it, and I followed the instructions on their docs.
import os
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
port = int(os.environ.get('PORT', 5000))
reactor.listenTCP(port, EchoFactory(), interface = '0.0.0.0')
reactor.run()
Its all working except (and I know this is a stupid question), how do I send a message to it? When I am working locally I just do telnet localhost <port>, but now I have no idea.
Also, since heroku connects to a random port how can I know what port it connects my app to?
Thanks.
I'm not very familiar with Twisted, but I'm not sure what you're trying to do is supported on Heroku. Heroku currently only supports HTTP[S] requests and not raw TCP. There are more details in the answers to this question.
If you wanted to connect to your app, you should use the myapp.herokuapp.com host name or any custom domain that you've added.
"Pure Python applications, such as headless processes and evented web frameworks like Twisted, are fully supported on Cedar."
Reference: https://devcenter.heroku.com/articles/python-support
I have a web server running on Django.
Users can create events postponed in time.
These events must be recorded in queue and processed on another server.
Initially I thought to take the Twisted. something like:
#client - django server
factory = pb.PBClientFactory()
reactor.connectTCP(server_ip, server_port, factory)
d = factory.login(credentials.UsernamePassword(login, paswd),)
d.addCallbacks(self.good_connected,self.bad_connected)
d.addCallback(self.add_to_queue)
reactor.run()
def add_to_queue(self, p)
p.callRemote("pickup", data)
#server - twisted server
def perspective_pickup(self, data)
reactor.callLater(timeout, self.pickup_from_queue)
But now I have big doubts about this approach. Maybe do not use twisted? or connect it with Django differently
Run twisted inside of Django is not a good idea anyway. So, try Celery or run HTTP server with twisted and use urllib on django side to send data to twisted server.
I am currently trying to pull together a basic SSL server in twisted. I pulled the following example right off their website:
from twisted.internet import ssl, reactor
from twisted.internet.protocol import Factory, Protocol
class Echo(Protocol):
def dataReceived(self, data):
"""As soon as any data is received, write it back."""
print "dataReceived: %s" % data
self.transport.write(data)
if __name__ == '__main__':
factory = Factory()
factory.protocol = Echo
print "running reactor"
reactor.listenSSL(8080, factory,
ssl.DefaultOpenSSLContextFactory(
"./test/privatekey.pem", "./test/cacert.pem"))
reactor.run()
I then tried to hit this server using firefox by setting the url to https://localhost:8080 yet I receive no response. I do, however, see the data arriving at the server. Any ideas why I'm not getting a response?
You're not sending an http header back to the browser, and you're not closing the connection
You've implemented an SSL echo server here, not an HTTPS server. Use the openssl s_client command to test it interactively, not firefox (or any other HTTP client, for that matter).