We Keep Coding

How to handle multiple publishers on same port in zmq?

This question has been asked before, here. I have the exact same problem. I want to publish from a bunch of different processes, and use the same port every time.
I tried the solution presented in the answer, but this did not work for me. I get the error
File "/usr/local/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/local/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/akay/afk/multi.py", line 18, in to_zmq
socket.connect("tcp://*:%s" % port)
File "zmq/backend/cython/socket.pyx", line 478, in zmq.backend.cython.socket.Socket.connect (zmq/backend/cython/socket.c:4308)
ZMQError: Invalid argument
My code is like this, essentially taken straight from the example in the zmq docs here and here:
# Socket to talk to server
port = '5556'
context = zmq.Context()
socket = context.socket(zmq.SUB)
print "Listening for stream...", m
socket.bind("tcp://localhost:%s" % port) #change connect to bind, as per answer above
socket.setsockopt(zmq.SUBSCRIBE, topicfilter)
I am using python 2.7, and the most recent version of zmq. Any idea what I might be doing wrong?

Well, the error is clear:
[...]
socket.connect("tcp://*:%s" % port)
[...]
ZMQError: Invalid argument
You can't connect to *, you must specify an IP address (the server IP address). If both the client and the server run on a single machine, try with localhost or 127.0.0.1.

Pika blocking_connection.py random timeout connecting to RabbitMQ

i have a rabbit mq running on machine
both client and rabbitMQ are running on the same network
rabbitMQ has many clients
i can ping client from rabbitMQ and back
longest latency measured between the machine is 12.1 ms
network details : Standard Switch network (network of virtual machines running on single physical machine - using vmware VC)
im getting random timeouts when initializing RPC connection
/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py
problem is that the timeout isn't consistent and happens from time to time.
when manually testing this issue and running the blocking_connection.py 1000 times from the same machine that it fails no timeout accrue.
this is the error i get when failing :
2013-04-23 08:24:23,396 runtest-trigger.24397 24397 DEBUG producer_rabbit initiate_rpc_connection Connecting to RabbitMQ RPC queue rpcqueue_java on host: auto-db1
2013-04-23 08:24:25,350 runtest-trigger.24397 24397 ERROR testrunner go Run 1354: cought exception: timed out
Traceback (most recent call last):
File "/testrunner.py", line 193, in go
self.set_runparams(jobid)
File "/testrunner.py", line 483, in set_runparams
self.runparams.producers_testrun = self.initialize_producers_testrun(self.runparams)
File "/basehandler.py", line 114, in initialize_producers_testrun
producer.set_testcase_checkout()
File "/baseproducer.py", line 73, in set_testcase_checkout
self.checkout_handler = pm_checkout.get_producer(self.testcasecheckout)
File "/producer_manager.py", line 101, in get_producer
producer = self.load_producer(plugin_dir, producer_name)
File "/producer_manager.py", line 20, in load_producer
producer = getattr(producer_module, 'Producer')(producer_name, self.runparams)
File "/producer_rabbit.py", line 13, in __init__
self.initiate_rpc_connection()
File "/producer_rabbit.py", line 67, in initiate_rpc_connection
self.connection = pika.BlockingConnection(pika.ConnectionParameters( host=self.conf.rpc_proxy))
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py", line 32, in __init__
BaseConnection.__init__(self, parameters, None, reconnection_strategy)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 50, in __init__
reconnection_strategy)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 170, in __init__
self._connect()
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 228, in _connect
self.parameters.port or spec.PORT)
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/blocking_connection.py", line 44, in _adapter_connect
self._handle_read()
File "/usr/lib/python2.6/site-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 151, in _handle_read
data = self.socket.recv(self._suggested_buffer_size)
timeout: timed out
please assist

I had a similar issue. If everything looks fine, then you most likely have some sort of miss configuration, e.g. bad binding. If miss configured, then you'll get a timeout because the script can't reach where it thinks it needs to go, so the error can be miss leading in this case.
For my problem, I specifically had issues with both my rabbitmq.config file and my bindings and had to use my python solution shown in: RabbitMQ creating queues and bindings from a command line over the command line example I showed. Once updated and configured properly, everything worked fine. Hopefully this gets you in the right direction.

Pika provides some time out issue when connecting different hosts.Solution is to pass a socket_timeout argument in connection parameter.Pika should upgrade to >=0.9.14
credentials = pika.PlainCredentials(RABBITMQ_USER, RABBITMQ_PASS)
connection = pika.BlockingConnection(pika.ConnectionParameters(
credentials=credentials,
host=RABBITMQ_HOST,
socket_timeout=300))
channel = connection.channel()

django-paypal IPN doesn't work

I'm using django-paypal as a payment solution inside my django application. I'm trying to implement a IPN handler.
What happens when I receive an IPN message at my IPN-handling URL the django server crashes:
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 281, in run
self.finish_response()
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 321, in finish_response
self.write(data)
File "/usr/local/lib/python2.6/dist-packages/django/core/servers/basehttp.py", line 417, in write
self._write(data)
File "/usr/lib/python2.6/socket.py", line 300, in write
self.flush()
File "/usr/lib/python2.6/socket.py", line 286, in flush
self._sock.sendall(buffer)
error: [Errno 104] Connection reset by peer
My payments applications urls.py looks like this:
urlpatterns = patterns('mysite.payment.views',
(r'^thank_you/', 'thank_you'),
(r'^canceled/', 'canceled'),
(r'^paypal-ipn/', include('paypal.standard.ipn.urls'))
)
To me the error message is pretty useless. Would be great if someone could help me.

I admit I'm an idiot :)
You don't need ssl for this. But what you need is to do a syncdb before you are able to use it,...
God sometimes it is so easy that you just don't see it.

Can you monitor precisely the packet that paypal is sending your server using tcpdump or wireshark? It looks like they may be terminating the connection early, but it's hard to tell much without a longer traceback and/or a packet dump.
Edit:
I had forgotten about the https messages. Paypal probably requires HTTPS for those callbacks. The dev server won't support that, so unfortunately you will probably need to flesh out your server configuration before you can test that functionality.

How do I change permissions to a socket?

I am trying to run a simple Python based web server given here.
And I get the following error message:
Traceback (most recent call last):
File "webserver.py", line 63, in <module>
main()
File "webserver.py", line 55, in main
server = HTTPServer(('', 80), MyHandler)
File "/usr/lib/python2.5/SocketServer.py", line 330, in __init__
self.server_bind()
File "/usr/lib/python2.5/BaseHTTPServer.py", line 101, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/usr/lib/python2.5/SocketServer.py", line 341, in server_bind
self.socket.bind(self.server_address)
File "<string>", line 1, in bind
socket.error: (13, 'Permission denied')
As far as I understand my firewall blocks access to a socket? Am I right? If it is the case, how can I change the permissions? Is it dangerous to change these permissions?

If you want to bind to port numbers < 1024, you need to be root. It's not a firewall
issue; it's enforced by the operating system. Here's a reference from w3.org,
and a FAQ entry specific to Unix.

If you want to run on a port under 1024, you'll need to be root. You can open the socket and drop root's permission for the rest of your program by switching to another user.
Most times it's easier to run a real webserver (say nginx) on port 80 and proxy the requests through to your program which you can run on a high numbered port (8080 for example). This way you don't need to worry about screwing something up during the time your process is running as root, as it never runs as root.
If it's just for testing, run the server on port 8080 and connect at http://localhost:8080/

setting XML value using xmlrpc & python

I need to set the value of a field in an XML file which exists on a remote Linux box. How do I find out which port I should connect to ?
But even a proper ping is not happening:
import xmlrpclib
server = xmlrpclib.ServerProxy('http://10.77.21.240:9000')
print server.ping()
print "I'm in hurray"
bUT instead I got:
Traceback (most recent call last):
File "ping.py", line 3, in <module>
print server.ping()
File "C:\Python26\lib\xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "C:\Python26\lib\xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "C:\Python26\lib\xmlrpclib.py", line 1235, in request
self.send_content(h, request_body)
File "C:\Python26\lib\xmlrpclib.py", line 1349, in send_content
connection.endheaders()
File "C:\Python26\lib\httplib.py", line 892, in endheaders
self._send_output()
File "C:\Python26\lib\httplib.py", line 764, in _send_output
self.send(msg)
File "C:\Python26\lib\httplib.py", line 723, in send
self.connect()
File "C:\Python26\lib\httplib.py", line 704, in connect
self.timeout)
File "C:\Python26\lib\socket.py", line 514, in create_connection
raise error, msg
socket.error: [Errno 10061] No connection could be made because the target machine actively refused it.
What did I do wrong?

A couple of things to try / think about:
Go to a command prompt on the remote host and type "netstat -nap | grep 9000". If you don't get back something interesting it means that nothing is running at port 9000.
You show the remote host at 10.77.21.240. This is an unroutable address on the net (AKA Private Network), so is the server itself (not just your app) pingable? If you are on windows, goto Start -> Run and type "cmd". At the prompt type, "ping 10.77.21.240" and see what you get.
One more thought: the process may be up and running at 9000 on a reachable host, but it may have opened the port as 127.0.0.1:9000 instead of 0.0.0.0:9000. The first address will only be reachable by processes on the same machine, the second one will open the port on all available IP addresses the machine has.
Update in response to comment: The fact that it shouldn't be a problem doesn't eliminate the possibility it is. When you are debugging something that should be working, but isn't, you need to get fairly pedantic about checking each step, allowing yourself no room for 'Oh, I know that couldn't be the problem.' -- this is a verbal 'handwave' (often accompanied by a real handwave). You'd be surprised how often the problem exists in exactly the area you are handwaving! It takes 3 seconds to do the ping test. If it works, you move on, if it doesn't work ...
The first three steps in dealing with any system problem are:
Is it plugged in?
Is it turned on?
Is it configured properly?
And you have to do this for each and every piece of hardware/software in the food chain from your keyboard to the app. I'd guess 80% of 'sudden' failures are items 1 or 2 -- yes, really. Cables are a huge pain in the ass.
When on the phone with novices I normally start by going for the long pass -- if they can get news.google.com in a browser and then click on a random story, then I know that in general the network is OK. Why the news and why a random story? To sidestep browser cache issues. I've lost count of the number of times my older sister has called me up and announced "The Internet is broken!" The first thing we do is the google news test. 99% of the time it works, so I have her fire up reverse-WinVNC (UltraVNC's SingleClick is a God-send), I get on her machine, and then we see what the real problem is.
If the long pass doesn't work, then I see if they can get to their router. Etc. etc.