How can I transfer binary data over xmlrpc (python)? - python

As the name xmlrpc implies, this transfer protocol relies on XML to carry data, and cannot transfer binary data, or non-printable ASCII-characters (\n, \b, chr(2),...) [or can it?].
I would like to know if there is a way to transfer a character string safely from a client to a server with minimal impact on the coding (i.e. ONLY on the client side). I tried the xmlrpclib.Binary class but this only seem to work with files.
Testcode, server.py:
def output(text):
print "-".join([str(ord(x)) for x in text])
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()
client.py:
import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(".\n."+chr(2))
Expected outcome:
46-10-46-2
Seen outcome (on server side):
xmlrpclib.Fault: <Fault 1: "<class 'xml.parsers.expat.ExpatError'>:not well-formed (invalid token): line 7, column 1">

I think the expected answer was using xml-rpc base64 type. In python, on client side, you have to manually specify that a string contains binary data, using the xmlrpclib.Binary type.
import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(xmlrpclib.Binary(".\n."+chr(2)))

You could try encoding your binary data in a text format in the client and decoding it back into binary in the server. One encoding you could use is base64.
In your client:
import xmlrpclib
import base64
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(base64.b64encode(".\n."+chr(2)))
In your server:
import base64
def output(text):
print "-".join([str(ord(x)) for x in base64.b64decode(text)])
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()

Related

Sending an array over UDP connection

I want to send an array using a UDP connection. When I use the sendto function, it complains that it must be a string. Is there any way around this?
Thanks
You must serialize your data (in this case is an array) before sending it. Then in receiver, you will deserialize to get the original data.
You can do it in Python, using pickle or cPickle module:
import cPickle as p
# Sender
data_str = p.dumps(array)
sock.sendto(data_str, addr)
# Receiver
data,addr = sock.recvfrom(buf)
data_origin = p.loads(data)

How to send the parameter with UDP protocol using twisted

How to connect to the following url using twisted and get the response from the tracker.
udp://tracker.publicbt.com:80/announce?uploaded=0&downloaded=0&compact=1&event=started&peer_id=kovid_agarwal1235467&port=6881&info_hash=3389809f0c9096819294a680beb4adb96a738419&left=763922958
I tried the following code but I am getting connection Id mismatch error
from twisted.internet.protocol import DatagramProtocol
from MakeUrlRequest import encoded_Value
class SendAndReceiveUDP(DatagramProtocol):
def startProtocol(self):
self.transport.connect("31.172.124.3",80)
self.transport.write(encoded_Value())
def datagramReceived(self, datagram, addr):
print "Received %r" %datagram
from twisted.internet import reactor
udpclient=SendAndReceiveUDP()
reactor.listenUDP(6881,udpclient)
reactor.run()
Above the encoded_Value is nothing but the value as follows:
uploaded=0&downloaded=0&compact=1&event=started&peer_id=kovid_agarwal1235467&port=6881&info_hash=3389809f0c9096819294a680beb4adb96a738419&left=763922958
According to http://en.wikipedia.org/wiki/UDP_tracker, the UDP-based protocol for exchanging data with Bittorrent trackers uses "a custom binary format". It doesn't look like the data you're sending conforms to this format. So the error you receive probably indicates that you need to format your request differently. See http://www.bittorrent.org/beps/bep_0015.html for further details about that format.

Client and Server unable to exchange data python

I'm using python 3.3.this is Server.py.Everything is fine both server and client are able to connect
something might be wrong in here 'tcpcli.send('[%s]%s'%(bytes(ctime(),'utf-8'),data))'.help me out
from socket import *
from time import ctime
HOST=''
PORT=21567
BUFSIZ=1024
ADDR=(HOST,PORT)
tcp=socket(AF_INET,SOCK_STREAM)
tcp.bind(ADDR)
tcp.listen(5)
while True:
print('waiting for connection')
tcpcli,addr=tcp.accept()
print('...connected from:',addr)
while True:
data=tcpcli.recv(BUFSIZ)
if not data:
break
tcpcli.send('[%s]%s'%(bytes(ctime(),'utf-8'),data))
tcpcli.close()
tcp.close()
This is CLient.py
from socket import *
HOST='127.0.0.1'
PORT=21567
BUFSIZ=1024
ADDR=(HOST,PORT)
tcpcli=socket(AF_INET,SOCK_STREAM)
tcpcli.connect(ADDR)
while True:
data=input('>')
if not data:
break
tcpcli.send(data)
data=tcpcli.recv(BUFSIZ)
if not data:
break
print (data.decode('utf-8'))
tcpcli.close()
When i'm running both they are working fine except I'm unable to send any data from client.
I'm getting this error message.
tcpcli.send(data)
TypeError: 'str' does not support the buffer interface
You are using Python3. This means that, when using the CLI, input() will return a str object (equivalent to python2 unicode). It contains an internal representation of the unicode codepoints of the characters you entered. To send the data over a byte stream interface (such as pipes, sockets, …), you have to convert it to a bytes object. This is easily done by picking an encoding, such as UTF-8, and doing something like this:
data_raw = data.encode("utf-8")
tcpcli.send(data_raw)
You will have to adapt your servers code similarily, by first decoding the data you received from the client and reencoding it after you did string operations on it:
data_decoded = data.decode("utf-8")
reply = '[%s]%s' % (ctime(), data_decoded)
tcpcli.send(reply.encode("utf-8"))
You are building unicode strings, not byte strings, and the socket interface doesn't support unicode strings. You'll need to encode the result of the string interpolation:
tcpcli.send(bytes('[%s]%s' % (ctime(),data), 'utf-8'))

Getting some response from a python socket server

I have just started learning python and i was wondering how i would get the client to execute a function on the server and get some response
Here is my server code
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('localhost', 8089))
serversocket.listen(5)
while True:
connection, address = serversocket.accept()
buf = connection.recv(64)
if len(buf)> 0:
print(buf)
break
input('press enter')
This is the client code
import socket
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect(('localhost', 8089))
data = 'lorem ipsum'
clientsocket.send(data.encode())
input('press enter')
and this is the function
def addme(x,y):
return x + y
print (addme(6,4))
Supposing i have the function addme() on the server,would it be possible to call it from the client and the response displayed to the client?.
If you simply want to call functions you should check out XMLRPC. Simple and easy, here's the example from the python documentation.
# Server code
import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer
def is_even(n):
return n%2 == 0
server = SimpleXMLRPCServer(("localhost", 8000))
print "Listening on port 8000..."
server.register_function(is_even, "is_even")
server.serve_forever()
# Client code
import xmlrpclib
proxy = xmlrpclib.ServerProxy("http://localhost:8000/")
print "3 is even: %s" % str(proxy.is_even(3))
print "100 is even: %s" % str(proxy.is_even(100))
You'd have to send it some sort of message telling the server to execute this. For example you could send it a string "ADDME", when the server receives this, it stores addme()'s result and sends it back to the client which then prints it.
You need to set up your own communication protocol. Invent a command that, when you send it, makes the server execute some function.
To send data over a socket (comparable to a file-like object) you need to serialize (encode) it into a set of bytes, and, after receiving these bytes on the other end, deserialize (decode) those.
Encode the function's return value to e.g. JSON in case it is dictionary, to str in case it is an integer, or invent your own binary protocol or, if you would like to be able to send almost any kind of Python object through "the wire", then pickle the return value. Send the encoded (pickled) return value to the client. It has to decode (unpickle) it then.
In any case, you will have to implement your own protocol, with its own set of commands, while each command might have arguments. You will have to find a way to separate the command from its argument and will have to (in)validate commands you receive.
For learning network communication, your task is great. For implementing a production software, you must have a look and rock-solid messaging libraries such as xmlrpclib as pointed out by others.
Sounds like you are trying to implement RPC. See here for a discussion on existing libraries: What is the current choice for doing RPC in Python?
This is how i did it
server.py
from xmlrpc.server import SimpleXMLRPCServer
def addme(x,y):
return x + y
server = SimpleXMLRPCServer(("localhost", 8000))
print("Listening on port 8000...")
server.register_function(addme, "addme")
server.serve_forever()
input('press enter')
client.py
import xmlrpc.client
proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")
print("the sum: %s" % str(proxy.addme(6,4)))
input('press enter')

Sending and Receiving arrays via Sockets

Is it possible to send an array through UDP Sockets using Python? I am using Python 2.5 and trying to send a simple array but it's not working. It can send the array successfully but when I try to print it with an item of the array the program crashes. I'm not sure what the error is as I take the precaution of converting the data into an array but it's not working. Hope I explained the problem as clearly as possible. I would appreciate the help!
# Client program
from socket import *
import numpy
from array import*
# Set the socket parameters
host = "localhost"
port = 21567
buf = 4096
addr = (host,port)
# Create socket
UDPSock = socket(AF_INET,SOCK_DGRAM)
def_msg = "===Enter message to send to server===";
print "\n",def_msg
a = array('i',[1,3,2])
# Send messages
while (1):
data = raw_input('yes or now')
if data!= "yes":
break
else:
if(UDPSock.sendto(a,addr)):
print "Sending message"
# Close socket
UDPSock.close()
# Server program
from socket import *
# Set the socket parameters
host = "localhost"
port = 21567
buf = 4096
addr = (host,port)
# Create socket and bind to address
UDPSock = socket(AF_INET,SOCK_DGRAM)
UDPSock.bind(addr)
# Receive messages
while 1:
data,addr = UDPSock.recvfrom(buf)
L = eval(data)
if not data:
print "Client has exited!"
break
else:
print "\nReceived message '", L[1],"'"
# Close socket
UDPSock.close()
eval is doing something completely different than what you think.
To send data over network, you need to serialize it into an array of bytes, then deserialize it back. In Python, serialization of most objects can be done via pickle module:
if (UDPSock.sendto( pickle.dumps(a), addr)):
Deserialization:
data,addr = UDPSock.recvfrom(buf)
L = pickle.loads(data)
print repr(L) # prints array('i', [1, 3, 2])
I would personally use tostring and fromstring since the built-in serialization methods are many times faster and pickle may not support NaN, Inf and other undefined values.
You're trying to send a python object through a socket, it is normal that it doesn't work, you can't send objects in a socket, objects are not data, they are the representation of some data in a given programming language. You need to "translate" your object to data and re-create the object from the data on the other socket's side. One way to do this would be with the pickle module.
On the client side, you "pickle" the object:
data = pickle.dumps(my_array)
And on the server side, you "unpickle" the received data:
my_array = pickle.loads(received_data)
You could try to pickle the array. Pickle is a python library to en- and decode python objects. It is able to do much more, but it is definitely sufficient to fulfill your task:
on the sender side you pickle the object to a string:
pickled_string = pickle.dumps(a)
on the receiver side you unpickle the object:
a = pickle.loads(received_string)
# a is now your sent array
It has been a while since this question was asked, but I thought it's worth sharing the jsonsocket library. It makes it really easy to send strings, lists and dictionaries over sockets. It can handle big amounts of data efficiently. And you don't need to do any manual serialization/deserialization. Under the hood, it serializes the data as JSON strings on the client, and deserializes it on the server.
If you don't need UDP specifically, try zmqObjectExchanger (https://github.com/ZdenekM/zmq_object_exchanger). It wraps pickle and zmq to transfer python objects over TCP.

Categories