We Keep Coding

How does twisted read /etc/hosts?

By default, Twisted reads /etc/hosts when connecting to websites on Linux systems. I'm just wondering which twisted file chooses /etc/hosts, So I can go in and change it to a file of my choosing?

The code you're looking for is in the Resolver Object line 621. If you want to change the behavior of the resolver you pass in the required hosts file as follows.
from twisted.names import client
from twisted.internet import reactor
def do_lookup(domain):
hosts = "/tmp/example"
resolver = client.createResolver(servers=None, resolvconf=None, hosts="/tmp/example")
d = resolver.getHostByName(domain)
d.addBoth(lookup_done)
def lookup_done(result):
print(result)
reactor.stop()
domain = b'twistfoo.com'
reactor.callLater(0, do_lookup, domain)
reactor.run()

PyFtpdLib How to add a user while still running server

I'm running a script for an FTP server in Python with PyFTPdLib like this:
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
def main():
authorizer = DummyAuthorizer()
authorizer.add_user('user', '12345', '.', perm='elradfmwM')
handler = FTPHandler
handler.authorizer = authorizer
address = ('localhost', 2121)
server = FTPServer(address, handler)
# start ftp server
server.serve_forever()
if __name__ == '__main__':
main()
Whenever I call this code, it keeps the command line busy with running the "server.serve_forever()", it doesn't keep running through the if loop.
So my question is, how do you add a user while the server is running without shutting down the server?
Do I need to create another script to call this one and then another to add a user? Will it need to be threaded so both can run without locking up on this one?
I've tried to look through the tutorials on this module, but I haven't found any examples or seen any solutions. I'm willing to switch to a different module to run a FTP server if needed, so long as I can control everything.
I'm going to be adding users a lot from a database, then removing them. So I don't want to take the server down and restart it every time I need to add someone.

In case anyone was interested in this topic, I think I have found a solution.
What I did was save the above code in a file called "create_ftp_server.py" and changed "main" from a def to a class.
#First .py module
class ftp_server:
def __init__(self):
self.authorizer = DummyAuthorizer()
self.authorizer.add_user('admin', 'password', '.', perm='elradfmwM')
def run(self):
self.handler = FTPHandler
self.handler.authorizer = self.authorizer
self.address = ('localhost', 21)
self.server = FTPServer(self.address, self.handler)
logging.basicConfig(filename='pyftpd.log', level=logging.INFO)
self.server.serve_forever()
def add_user(self,user,passwd,loc,privi):
self.authorizer.add_user(str(user), str(passwd), str(loc), perm=str(privi))
Then I made another module to call and run this file (really this is just for my personal taste, as you could just add this to the same module)
#Second .py module
import thread
import create_ftp_server
this_ftp = create_ftp_server.ftp_server()
thread.start_new_thread(this_ftp.run,())
thread.start_new_thread(this_ftp.add_user,('user','password',".",'elradfmwM'))
All you have to do now is decide how you want to call the thread to add the user.
Note: I tried to use the newer Threading module but I couldn't get any real results with that. This option will work well for me, though.

You can do it, with the SITE Command like this:
proto_cmds = FTPHandler.proto_cmds.copy()
#Define SITE ADDUSER USERNAME PASSWORT HOME PRIVS
proto_cmds.update(
{'SITE ADDUSER': dict(perm='m', auth=True, arg=True, help='Syntax: SITE <SP> ADDUSER USERNAME PASSWORT HOME PRIVS <SP>.')}
)
class siteadd(TLS_FTPHandler):
proto_cmds = proto_cmds
def ftp_SITE_ADDUSER(self,user,passwd,loc,privi):
self.authorizer.add_user(str(user), str(passwd), str(loc), perm=str(privi))
and change the handle line from
self.handler = FTPHandler
to
self.handler = siteadd
Check if this works.
Connect to your FTP and run SITE HELP command you see all available commands.
You can run the FTPd in daemon mode, too. Look here for an example

twisted - get OS-chosen listen port

I am writing a twisted P2P client using the application framework. The listen port for incoming connections will be on a random (OS-determined) port. However, I need a way to determine what that port is after creating it:
import twisted... etc.
application = service.Application('vmesh')
peerservice = MyPeerService()
servicecollection = service.IServiceCollection(application)
factory = MyPeerFactory(peerservice)
server = internet.TCPServer(0, factory) # listen on random port
listen_port = server.getHost().port # ??? doesn't work...
server.setServiceParent(servicecollection)
I can't find anything in the docs about querying the port created by internet.TCPServer() or by reactor.listenTCP() which it forwards to. I can't simply wait for a connection to occur since the client has to announce its port in order for those connections to ever happen.

listenTCP returns an IListeningPort, which has a getHost() method that gives back an object with a port. For example:
>>> from twisted.internet import reactor
>>> from twisted.internet.protocol import Factory
>>> port = reactor.listenTCP(0, Factory())
>>> port.getHost().port
55791
However, TCPServer doesn't call listenTCP until it is started with privilegedStartService. Plus, the IListeningPort isn't actually exposed via a public API. So, you will need to write your own Service. Luckily, it's quite easy to do this; TCPServer doesn't do very much. You just need to write one that reports its port somewhere as soon as it starts listening. Here's an example:
from twisted.internet import reactor
from twisted.application.service import Service
class PortReporter(Service, object):
def __init__(self, factory, reportPort):
self.factory = factory
self.reportPort = reportPort
def privilegedStartService(self):
self.listeningPort = reactor.listenTCP(0, self.factory)
self.reportPort(self.listeningPort.getHost().port)
return super(PortReporter, self).privilegedStartService()
def stopService(self):
self.listeningPort.stopListening()
return super(PortReporter, self).stopService()
You can then use this in a tac file, like so:
from twisted.internet.protocol import Factory
from twisted.application.service import Application
application = Application("test")
def showPortNumber(n):
print("The port number is: %d" % (n,))
PortReporter(Factory(), showPortNumber).setServiceParent(application)

FWIW if you need to do this with endpoints here is my implementation with a slight tweak for my local setup (the callback option would work well here too):
class PortReporter(StreamServerEndpointService, object):
def __init__(self, endpoint, factory):
StreamServerEndpointService.__init__(self, endpoint, factory)
self._reportedPort = None
def privilegedStartService(self):
r = super(PortReporter, self).privilegedStartService()
self._waitingForPort.addCallback(self.port_cb)
return r
def port_cb(self, port):
self._reportedPort = port.getHost().port
return port
def getReportedPort(self):
return self._reportedPort

You can access the port bind to your server like so if you didn't start the server yet (didn't call startService yet):
>>> serv._getPort()._realPortNumber
Else you can also do:
>>> serv._port._realPortNumber

How to gracefully exit application started with twistd?

I have a jabber client that is reading from its stdin and posting PubSub messages. If I get EOF on stdin, I want to terminate the client.
I first tried sys.exit(), but this causes an exception and the client does not exit. I then did some searching and found out that I should call reactor.stop(), but I am unable to make this work. The following code in my client:
from twisted.internet import reactor
reactor.stop()
Results in exceptions.AttributeError: 'module' object has no attribute 'stop'
What do I need to do to cause twistd to shut my application down and exit?
EDIT 2
The original problem was caused by some symlinks messing up the module import. After fixing that problem, I get a new exception:
twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running.
After the exception, twistd shuts down. I think this may be caused by the call to MyClient.loop in MyClient.connectionInitialized. Perhaps I need to defer the call until later?
EDIT
Here's the .tac file for my client
import sys
from twisted.application import service
from twisted.words.protocols.jabber.jid import JID
from myApp.clients import MyClient
clientJID = JID('client#example.com')
serverJID = JID('pubsub.example.com')
password = 'secret'
application = service.Application('XMPP client')
xmppClient = client.XMPPClient(clientJID, password)
xmppClient.logTraffic = True
xmppClient.setServiceParent(application)
handler = MyClient(clientJID, serverJID, sys.stdin)
handler.setHandlerParent(xmppClient)
Which I'm invoking with
twistd -noy sentry/myclient.tac < input.txt
Here's the code for MyClient:
import os
import sys
import time
from datetime import datetime
from wokkel.pubsub import PubSubClient
class MyClient(PubSubClient):
def __init__(self, entity, server, file, sender=None):
self.entity = entity
self.server = server
self.sender = sender
self.file = file
def loop(self):
while True:
line = self.file.readline()
if line:
print line
else:
from twisted.internet import reactor
reactor.stop()
def connectionInitialized(self):
self.loop()

from twisted.internet import reactor
reactor.stop()
that should work. The fact that it doesn't means something else is wrong on your application. I can't figure out what's wrong from the information you provided.
Can you provide more (all) of the code?
EDIT:
Ok, now the problem is that you don't stop your own while True loop, so it will keep looping and eventually stop the reactor again.
Try this:
from twisted.internet import reactor
reactor.stop()
return
Now, I suspect your loop isn't very good thing for a event-driven framework. While you're just printing lines, it is fine, but depending on what you want to really do (I suspect you'll do more than just print lines) you'll have to refactor that loop to work with events.

Use reactor.callFromThread(reactor.stop) instead of reactor.stop. This should solve the issue.

I used to do this way (in sigint handler of a non-twistd called application):
reactor.removeAll()
reactor.iterate()
reactor.stop()
I'm not 100% sure it is the right way, but twisted is happy
the same application started in a tac is handled directly by twistd signal handler, I've found this question because I have some rpc client requests that I would to wait for and handle result before exiting and looks like twistd is just killing the reactor without letting the call finish

termination of a twistd application in python

I am trying to write an application with twistd library written for Python. The application file ends like the following:
factory = protocol.ServerFactory()
factory.protocol = EchoServer
application = service.Application("Echo")
internet.TCPServer(8001, factory).setServiceParent(application)
I want to run something before my appication terminates (e.g. close a file). Does anyone know how to do that? because this is an event-handler and I don't know where the clean-up function is called.

You need to add code to the startService and stopService methods of the Service.
One way would be something like this:
from twisted.application import service
from twisted.internet import protocol
class MyService(service.Service):
def __init__(self,port=8001):
self.port = port
def startService(self):
self.factory = protocol.ServerFactory()
self.factory.protocol = EchoServer
from twisted.internet import reactor
reactor.callWhenRunning(self.startListening)
def startListening(self):
from twisted.internet import reactor
self.listener = reactor.listenTCP(self.port,self.factory)
print "Started listening"
def stopService(self):
self.listener.stopListening()
# Any other tidying
application = service.Application("Echo")
MyService().setServiceParent(application)