I'm trying to write a client in python 2.7 using Twisted. My code works just fine in linux (debian squeeze), but when I tried it on windows (xp and 7) I got a constant stream of error messages. A screenshot of these messages is here.
I have narrowed down the bug and was able to write a very stripped down version of my client that still contains the bug:
from twisted.internet.protocol import Protocol,ClientFactory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
class TheClient(LineReceiver):
def lineReceived(self,line):
print line
def connectionLost(self,reason):
reactor.stop()
class TheFactory(ClientFactory):
protocol = TheClient
class Test(object):
def doRead(self):
pass
def fileno(self):
return 0
def connectionLost(self,reason):
print 'connection lost'
def logPrefix(self):
return 'Client'
def main():
print 'starting'
test = Test()
reactor.addReader(test)
reactor.run()
if __name__ == '__main__':
main()
If the line containing 'reactor.addReader(test)' is commented out, I do not get any error messages. If I run this code on linux without commenting out any lines, I do not get any error messages.
I found this question, I don't think its the same problem, but as expected, it did not function properly on windows.
Is this code correct, and this is a windows bug, or do I have to do things differently for it to work in windows?
The Windows implementation of select only supports sockets. Presumably file descriptor 0 in your process does not represent a socket. More likely it represents something related to standard I/O.
If you'd just like to use standard I/O, then there's twisted.internet.stdio, though you may run into some rough edges with it on Windows (bug reports and fixes appreciated!).
If you're not interested in standard I/O and 0 was just an arbitrary test, you'll probably need to decide on what kind of input you're trying to do in particular. Depending on what kind of file descriptor you have, there will probably be a different approach to successfully reading from it.
Related
Im not quite sure what to classify this problem as, but as far as i can tell the connhandler thread should be able to write to the dictionary stored in the queuemanager class that is defined in hostmain, however when i print it out from the terminal prompt it comes up as empty, and when the thread and hostmain print the class object they have different memory addresses. This is my first project with multithreading and socketserver so i may have done something wrong, but to my knowledge i cant see why the thread from socketserver is unable to access the same servermanager object from hostmain.
To clarify a bit this project is using the socketserver library with the threading mixin, and the the threading library for everything showed in the example.
Request handler class code as well as a pastebin with pared down versions of pertinent code, reproducing the issue:
https://pastebin.com/u/kadytoast/1/PPWfyCFT
import testhostmain as hmain
import multiprocessing as mp
def joinpacket(flag, data):
return f"{flag}{hmain.packetdelim}{data}"
def splitpacket(msg):
msg = msg.split(hmain.packetdelim)
flag, data = msg[0], msg[1]
return flag, data
class ConnHandler(soss.BaseRequestHandler):
def handle(self):
# request handler to spawn processes
print(hmain.servermanager, "in connhandler")
# collecting peritype and name
msg = self.request.recv(hmain.buffer).decode("utf-8")
flag, data = splitpacket(msg)
data = data.split(hmain.datadelim)
self.peritype, self.macid = data[0], data[1]
#print(data)
# checks the header flag
if flag == hmain.newconflag:
hmain.servermanager.procdict[self.macid] = "connected"
print(hmain.servermanager.procdict, "from connhandler")
in this image you can see the two class printouts with their memory addresses, i would assume they should be the same but im not sure on that, however the dictionary printout from connhandler is correct but doesnt show up from the current process dictionary printout.
is there some conflict trying to use the threading mixin as well as seperately called threads?
Thankyou for getting this far if you did, if i need to provide anymore information i am happy to oblige. Thankyou again!
Edit: i expect i may have to add usage of the threading.lock, but nothing is writing to the dictionary at all so i dont expect that to be the issue in this case
So, I am having this following snippet which attempts to start Microsoft Powerpoint through the win32api:
import threading
import win32com.client
import sys
class myDemo(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
try:
myObject = win32com.client.Dispatch("Powerpoint.Application")
print "OK"
except:
print "Failed to start Powerpoint!"
sys.exit(1)
print "Now attempting to shutdown..."
try:
myObject.quit()
except:
print "Error"
if __name__ == "__main__":
test = myDemo()
test.start()
The problem is that it fails and I have no clue why.
However, if I change the last line to test.run() it will launch successfully.
So again why is this failing with test.start()?
Why is this happening and how should I solve it considering I need Powerpoint to run on a seperate thread asynchronously?
Thanks in advance.
EDIT: Apparently my question is somehow related to this: http://python.6.x6.nabble.com/Dispatch-error-CoInitialize-has-not-been-called-td1951088.html
However apart from the proposed proper solution no one seems to answer why exactly COM is behaving this way.
I'm afraid your question likely can't be summed up in one or two sentences due to complexities in COM and threading and why they work the way they do. But for starters, here's some good information why COM behaves the way it does under threading:
http://msdn.microsoft.com/en-us/library/ms809971.aspx
Additionally, you should consider reviewing the book Python Programming on Win32. It contains useful information that sheds more light on COM threading. (Despite its age it is still useful.)
Finally, in case it wasn't clear from the reference you provided, whenever your program uses threads and COM, you must indicate in code that you're going to use COM within a thread:
import pythoncom
import win32com.client
### ... inside the thread function ...
x = win32com.client.Dispatch("someCOMobject")
win32com.CoInitialize()
# com calls here
win32com.CoUninitialize()
This type of call uses what's called single-apartment threading. It occurs when the threaded code itself instantiates COM objects.
If you find yourself instantiating a single COM object outside the threaded code (and using the instantiated object in the threaded code e.g. passing access to the COM object between threads), then this type of COM threading is called multithreaded-apartment threading:
import sys
sys.coinit_flags = 0
import pythoncom
import win32com.client
# ... outside the thread function ...
x = win32com.client.Dispatch("someCOMobject")
# ... inside the thread function ...
pythoncom.CoInitialize(pythoncom.COINIT_MULTITHREADED)
# com calls here for x
pythoncom.CoUninitialize()
Hope this helps.
OK, so I think I found an answer but I am not yet sure why it works..
If I cut and paste this line import win32com.client from the top of the page right inside the try block where I dispatch microsoft powerpoint, the app works successfully.
However, I still can't find out why.
There are at least two more ways to solve the issue:
Use run() method instead of start(), i.e. test.run()
Before myObject = win32com.client.Dispatch("Powerpoint.Application") insert the following lines: import pythoncom; CoInitialize()
Notice that using run() instead of start() has been tested in other scripts and it always worked for me!
I distribute a software package for windows via distutils and py2exe. For development purposes, I'd like to be able to have access to a python console inside the py2exe build. I see there is a python27.dll file in the py2exe build, so I hope I can leverage that to launch a python terminal.
Is it possible to take an existing, or modify distutils/py2exe to get end user access to a Python shell in the py2exe environment?
There's a really bare-bones way to accomplish this as documented by Matt Anderson from the pymntos google group. I've seen some variations on it, but this one came up first when I googled. :p
The juice is in the stdlib code module, leveraging code.InteractiveInterpeter. The only thing you'd have to do is add this in as a thread as the application starts. Then, when the app starts you can telnet 'localhost 7777' and you should drop into a Python interpreter.
The problem with doing it as a thread though - you can't very easily twiddle variables / data in the main thread without doing some sort of queue and passing things around.
You could alternatively have an async socket - that way you could twiddle stuff as a main-thread participant. Thats inherently dangerous for a host of reasons. But, we're talking bare metal.
If you use the Twisted library, you could use Twisted Conch, which allows you to create an SSH or Telnet server that can talk to the rest of your app. However, this might be a problem since you're using the event loop from the UI to process events - you can't have two event loops. If you're using Qt, there is a Twisted Qt Reactor event loop. If it's windows or something else.. I have no idea. But, this should at least give you a few things to consider.
Original link is: https://groups.google.com/forum/?fromgroups#!topic/pymntos/-Mjviu7R2bs
import socket
import code
import sys
class MyConsole(code.InteractiveConsole):
def __init__(self, rfile, wfile, locals=None):
self.rfile = rfile
self.wfile = wfile
code.InteractiveConsole.__init__(
self, locals=locals, filename='<MyConsole>')
def raw_input(self, prompt=''):
self.wfile.write(prompt)
return self.rfile.readline().rstrip()
def write(self, data):
self.wfile.write(data)
netloc = ('', 7777)
servsock = socket.socket()
servsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
servsock.bind(netloc)
servsock.listen(5)
print 'listening'
sock, _ = servsock.accept()
print 'accepted'
rfile = sock.makefile('r', 0)
sys.stdout = wfile = sock.makefile('w', 0)
console = MyConsole(rfile, wfile)
console.interact()
I'm having a bizarre issue. Basically, the problem I have right now is dealing with two different LineReceiver servers that are connected to each other. Essentially, if I were to input something into server A, then I want some output to appear in server B. And I would like to do this vice versa. I am running two servers on two different source files (also running them on different processes via & shellscript) ServerA.py and ServerB.py where the ports are (12650 and 12651) respectively. I am also connecting to each server using telnet.
from twisted.internet import protocol, reactor
from twisted.protocols.basic import LineReceiver
class ServerA(LineReceiver);
def connectionMade(self):
self.transport.write("Is Server A\n")
def dataReceived(self, data):
self.sendLine(data)
def lineReceived(self, line):
self.transport.write(line)
def main():
client = protocol.ClientFactory()
client.protocol = ServerA
reactor.connectTCP("localhost", 12650, client)
server = protocol.ServerFactory()
server.protocol = ServerA
reactor.listenTCP(12651, server)
reactor.run()
if __name__ == '__main__':
main()
My issue is the use of sendLine. When I try to do a sendLine call from serverA with some arbitrary string, serverA ends up spitting out the exact string instead of sending it down the connection which was done in main(). Exactly why is this happening? I've been looking around and tried each solution I came across and I can't seem to get it to work properly. The bizarre thing is my friend is essentially doing the same thing and he gets some working results but this is the simplest program I could think of to try to figure out the cause for this strange phenomenon.
In any case, the gist is, I'm expecting to get the input I put into serverA to appear in serverB.
Note: Server A and Server B have the exact same source code save for the class names and ports.
You have overridden dataReceived. That means that lineReceived will never be called, because it is LineReceiver's dataReceived implementation that eventually calls lineReceived, and you're never calling up to it.
You should only need to override lineReceived and then things should work as you expect.
I have Python 2.7 amd64 running on Windows 7 + latest Twisted 12
I have very simple programm which connects stdin/stdout with remote tcp server
from twisted.internet import stdio, reactor, protocol
from twisted.protocols import basic
class Echo(basic.LineReceiver):
def connectionMade(self):
self.factory = CFactory()
self.connector = reactor.connectTCP('remote', 8585, self.factory)
def lineReceived(self, line):
self.connector.transport.write("%s\n" % line)
class Client(basic.LineReceiver):
def lineReceived(self, line):
delimiter = '\n'
print "%s" % line
class CFactory(protocol.ClientFactory):
protocol = Client
def main():
stdio.StandardIO(Echo())
reactor.run()
if __name__ == '__main__':
main()
On Linux it works just fine. But on Windows my input to stdin is being ignored. So nothing being sent to remote side
Any idea why?
class Client(basic.LineReceiver):
def lineReceived(self, line):
delimiter = '\n'
print "%s" % line
The delimiter = '\n' line here has no useful consequences. It defines a new name in the local scope of the lineReceived function, and then ignores that local name.
What you intended to do, I think, is:
class Client(basic.LineReceiver):
delimiter = '\n'
def lineReceived(self, line):
print "%s" % line
This defines a new attribute of the Client class, delimiter, which controls how the base class, basic.LineReceiver, splits up lines.
However, your example program only defines Client, it doesn't actually use it. So this isn't the cause of the problem. The cause of the problem is very similar, though. Echo is used to handle standard input, and it does not override the default LineReceiver delimiter at all. The default is '\r\n', which is not the line delimiter used on Windows standard input.
Start off your Echo definition like this:
class Echo(basic.LineReceiver):
delimiter = '\n'
and I think the program will behave as you want, since Windows uses "\n" to delimit lines on standard input (no Windows machine around to verify that, though, so you might want to double check; if that's not the case, a very useful response to make would be to explain what the delimiter is; if it is the case, verifying that information would also be a good thing).
Try dataReceived, it might work.:
class Echo(basic.LineReceiver):
...
def dataReceived(self, data):
self.connector.transport.write("%s\n" % data)
...
On Windows a patch is required to fix the problem of receiving input from Std I/O.
Refer to solution posted to ticket # 2157 on twistedmatrix.com here: http://twistedmatrix.com/trac/ticket/2157
The solution has several attachments - for individual patches. And there is a consolidated patch file which is an unified diff.
I have a 64-bit Python 2.7.3. running on a Windows 7 64-bit machine with Twisted 12.3.0 and I applied the above unified patch to the _win32stdio.py, _pollingfile.py, conio.py. This allows the input and output to/from Std I/O for IProtocol implementations.