We Keep Coding

Sending a file over TCP connection

I want to send i file over TCP but when i try to run this the connection fails, the server receives the file but it gives this error: ERROR: Client timed out before sending a file
import selectors
import sys
from socket import *
import sock
sel1 = selectors.DefaultSelector()
print(len(sys.argv), sys.argv[1], sys.argv[2], sys.argv[3])
host = sys.argv[1]
port = int(sys.argv[2])
file = sys.argv[3]
try:
# Instaniating socket object
s = socket(AF_INET, SOCK_STREAM)
# Getting ip_address through host name
host_address = gethostbyname(host)
# Connecting through host's ip address and port number using socket object
s.connect((host_address, port))
sel1.register(
sock,
selectors.EVENT_READ, data = None)
fileToSend = open("file.txt", "rb")
data = fileToSend.read(1024)
while data:
print("Sending...")
fileToSend.close()
s.send(b"Done")
print("Done Sending")
print(s.recv(1024))
s.shutdown(2)
s.close()
except:
# Returning False in case of an exception
sys.stderr.write("Connection Failed")

Do the writing in a loop. There's no particular reason to chop it into 1024-byte pieces; the network stack will handle that for you.
By the way, your "Done" signal is not a good idea, especially since you're writing a binary file that might very well contain the word "Done". Remember that TCP is a streaming protocol. The other end does not see the exact packets you're sending. That is, just because you send 1024 bytes and 4 bytes, the other end might see it as reads of 256 and 772 bytes.
# Instaniating socket object
s = socket(AF_INET, SOCK_STREAM)
# Getting ip_address through host name
host_address = gethostbyname(host)
# Connecting through host's ip address and port number using socket object
s.connect((host_address, port))
fileToSend = open("file.txt", "rb")
print("Sending...")
while True:
data = fileToSend.read(1024)
if not data:
break
s.send( data )
fileToSend.close()
s.send(b"Done")
print("Done Sending")
print(s.recv(1024))
s.close()

Sending a file over a socket with Python only working after force-closing socket

I'm trying to send a file over a socket in Python 2.7.x . My client and server code is below. It's working, but only after the client connection kills the socket. So I added raw_input('finished') to the end of both for debugging.
So if I start the server, then run the client... It looks like all but the last bit of the file sends, until I forcefully kill the client and then it's all there. So the problem is definitely in the server loop... I just don't know how to fix it. if not data: break isn't being triggered. But, if I do something like if len(data) < 1024: break it won't work for bigger files.
Any help is appreciated!
# client.py
import socket
conn = socket.socket()
conn.connect(('localhost', 1337))
f = open('test.jpg', 'rb')
data = f.read(1024)
while data:
conn.send(data)
data = f.read(1024)
f.close()
raw_input('finished')
# server.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('localhost', 1337))
s.listen(5)
conn, addr = s.accept()
f = open('test.jpg', 'wb')
while True:
data = conn.recv(1024)
if not data:
break
f.write(data)
f.close()
raw_input('finished')

From your posted code:
while data:
conn.send(data)
data = f.read(1024)
From the Python socket documentation:
socket.send(string[, flags])
[...]
Returns the number of bytes sent. Applications are responsible for checking
that all data has been sent; if only some of the data was transmitted, the
application needs to attempt delivery of the remaining data.
That should tell you what the problem is, but just to be explicit about it: send() may or may not accept all of the bytes you asked it to send before returning, and it's up to you to handle it correctly in the case where it only accepts the first (n) bytes rather than the entire buffer. If you don't check send()'s return value, then you will sometimes drop some of the bytes of your file without knowing it. You need to check send()'s return value and if it is less than len(data), call send() again (as many times as necessary) with the remaining bytes. Alternatively you could call conn.sendall() instead of conn.send(), since sendall() will perform that logic for you.

socket connection receive bytes python

I'm trying to send a stream of data via socket in Python. So far I manage to create a dummy_data_gen.py which sends a line containing 4 floats to the server.py. However, I'm still having issues in the stability of the all setup.
server.py:
import sys
import time
import socket
import numpy as np
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 5002)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
# Create a list to store the incoming data
data = []
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
while True:
incoming_data = connection.recv(48).split(',')
print incoming_data
event = float(incoming_data[0]), float(incoming_data[1]), float(incoming_data[2]), float(incoming_data[3])
data += [event]
time.sleep(0.01)
finally:
# Clean up the connection
connection.close()
dummy_data_gen.py
import sys
import time
import socket
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 5002)
sock.connect(server_address)
file = '../data/myfile.txt'
# Simulating a real-time data stream at 100 Hz
try:
with open(file) as f:
for line in f:
sock.sendall(line)
time.sleep(0.01)
finally:
print >>sys.stderr, 'closing socket'
sock.close()
My problem is that sometimes the communication is working properly, however, I have situations where I receive more data per line than I should. In the following output example the first 7 lines are correct, however the following lines are incorrect and therefore problematic:
['391910745379', '24.134277', '-1.9487305', '-117.373535', '\n']
['391920745379', '24.434082', '-1.3491211', '-117.373535', '\n']
['391930745379', '23.68457', '-0.5996094', '-116.62402', '\n']
['391940745379', '24.434082', '-1.0493164', '-115.57471', '\n']
['391950745379', '24.134277', '-1.0493164', '-116.47412', '\n']
['391960745379', '23.234863', '-1.0493164', '-116.47412', '\n']
['391970745379', '24.583984', '-0.89941406', '-116.92383', '\n']
['391980745379', '23.384766', '-0.2998047', '-116.62402', '\n39']
['1990745379', '23.68457', '-0.5996094', '-115.72461', '\n39200']
['0745379', '23.834473', '-0.44970703', '-115.87451', '\n392010']
['745379', '23.534668', '-1.0493164', '-114.9751', '\n392020745']
['379', '23.384766', '-1.7988281', '-115.72461', '\n39203074537']
['9', '22.935059', '-0.44970703', '-114.9751', '\n392040745379', '']
I tried to play with the connection.recv bytes but I'm still facing this issue.
EDIT1: Following some suggestions I modified the server.py as follows:
del_message = '\n'
del_stream = ','
while True:
_buffer += connection.recv(1)
if del_message in _buffer:
incoming_data = _buffer.split(del_stream)
event = float(incoming_data[0]), \
float(incoming_data[1]), \
float(incoming_data[2]), \
float(incoming_data[3])
This approach seems to solve my issue, however the performance is extremely slow. My files contains approximately 6300 lines that were actually sent in 70 seconds (time interval at which the socket was closed on my dummy data generator). However, I took almost 10 minutes to receive all of the 6300 lines. It seems also that I receive more samples per second on the beginning rather that on the end of the stream.

If you have a message protocol that terminates messages with a newline, you need to write some code to implement that protocol. It won't work by magic.
You need a "receive a message" function, where "message" is defined as "a sequence of bytes delimited by a newline". You've never written any such function, so you're not receiving messages but just the chunks of bytes you're sending.

How to create a timestamp for python

I currently have a TCP client code with logging which saves the sent data in a text file. I want the data sent to be saved in a folder which header is the time stamp of the sent data. Is that possible? I have tried using several methods but my code kept failing. Could someone give me a guide , would really be such a helping hand as I have been stuck on this for quite awhile.
This is how my code looks like now:
import socket
import thread
import sys
BUFF = 1024 # buffer size
HOST = '172.16.166.206'
PORT = 1234 # Port number for client & server to receive data
def response(key):
return 'Sent by client'
def logger(string, file_=open('logfile.txt', 'a'), lock=thread.allocate_lock()):
with lock:
file_.write(string)
file_.flush() # optional, makes data show up in the logfile more quickly, but is slower
sys.stdout.write(string)
def handler(clientsock, addr):
while 1:
data = clientsock.recv(BUFF) # receive data(buffer).
logger('data:' + repr(data) + '\n') #Server to receive data sent by client.
if not data:
break #If connection is closed by client, server will break and stop receiving data.
logger('sent:' + repr(response('')) + '\n') # respond by saying "Sent By Client".
if __name__=='__main__':
ADDR = (HOST, PORT) #Define Addr
serversock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversock.bind(ADDR) #Binds the ServerSocket to a specific address (IP address and port number)
serversock.listen(5)
while 1:
logger('waiting for connection...\n')
clientsock, addr = serversock.accept()
logger('...connected from: ' + str(addr) + '\n') #show its connected to which addr
thread.start_new_thread(handler, (clientsock, addr))

You just need to import time module to generate the time stamp and create the directory named by time stamp.
Suppose you want to cut the log file into different folds by hour. The code will somewhat like bellow:
import time
import os
def logger(string, file_name='logfile.txt', lock=thread.allocate_lock()):
with lock:
time_stamp = time.strftime("%Y%m%d%H",time.localtime())
if not os.path.isdir(time_stamp): os.mkdir(time_stamp)
with open(os.path.join(time_stamp, file_name), "a") as file_:
file_.write(string)
file_.flush()
sys.stdout.write(string)
The code is not tested.

Check out the Python logging module at:
http://docs.python.org/2/howto/logging.html#logging-basic-tutorial

sending/receiving file UDP in python

I've made this sending / receiving scripts but i corrupted file !
i have no idea why I'm getting this issue !
sender.py
#!/usr/bin/env python
from socket import *
import sys
s = socket(AF_INET,SOCK_DGRAM)
host =sys.argv[1]
port = 9999
buf =1024
addr = (host,port)
file_name=sys.argv[2]
f=open(file_name,"rb")
data = f.read(buf)
s.sendto(file_name,addr)
s.sendto(data,addr)
while (data):
if(s.sendto(data,addr)):
print "sending ..."
data = f.read(buf)
s.close()
f.close()
receiver.py
#!/usr/bin/env python
from socket import *
import sys
import select
host="0.0.0.0"
port = 9999
s = socket(AF_INET,SOCK_DGRAM)
s.bind((host,port))
addr = (host,port)
buf=1024
data,addr = s.recvfrom(buf)
print "Received File:",data.strip()
f = open(data.strip(),'wb')
data,addr = s.recvfrom(buf)
try:
while(data):
f.write(data)
s.settimeout(2)
data,addr = s.recvfrom(buf)
except timeout:
f.close()
s.close()
print "File Downloaded"
and this the original receiver that I've modify it (works fine 100%)
#!/usr/bin/env python
from socket import *
import sys
import select
host="0.0.0.0"
port = 9999
s = socket(AF_INET,SOCK_DGRAM)
s.bind((host,port))
addr = (host,port)
buf=1024
f = open("file.pdf",'wb')
data,addr = s.recvfrom(buf)
try:
while(data):
f.write(data)
s.settimeout(2)
data,addr = s.recvfrom(buf)
except timeout:
f.close()
s.close()
print "File Donwloaded"
as you notice it's making file at the beginning.
exacted:
client => send file (name.ext) => server:save it (name.ext)
my output :
corrupted file for pdf and empty for txt

The problem with your code:
When data is send through sockets, normally the lower layers will merge the data from multiple sendTo calls and send them together to reduce network load.
You are sending the first 1024 bytes of the file twice.
What you should do:
Use some kind of a delimiter string having couple of characters (like "**_$$") so that it won't exist in the actual file binary representation. Then append this delimiter to the end of the filename.
Read from file again before starting the while loop.
At receiver end, receive everything into a single stream and then split using the delimiter. You will have the filename and the file data.
Update:
Working Code (Ubuntu / Windows XP)
# ----- sender.py ------
#!/usr/bin/env python
from socket import *
import sys
s = socket(AF_INET,SOCK_DGRAM)
host =sys.argv[1]
port = 9999
buf =1024
addr = (host,port)
file_name=sys.argv[2]
s.sendto(file_name,addr)
f=open(file_name,"rb")
data = f.read(buf)
while (data):
if(s.sendto(data,addr)):
print "sending ..."
data = f.read(buf)
s.close()
f.close()
# ----- receiver.py -----
#!/usr/bin/env python
from socket import *
import sys
import select
host="0.0.0.0"
port = 9999
s = socket(AF_INET,SOCK_DGRAM)
s.bind((host,port))
addr = (host,port)
buf=1024
data,addr = s.recvfrom(buf)
print "Received File:",data.strip()
f = open(data.strip(),'wb')
data,addr = s.recvfrom(buf)
try:
while(data):
f.write(data)
s.settimeout(2)
data,addr = s.recvfrom(buf)
except timeout:
f.close()
s.close()
print "File Downloaded"
Usage
>> python recevier.py
>> python sender.py localhost filename.txt

There are two problems here:
Syntax errors:
You're using a from socket import *. It's not an error on its own, but it becomes one when you do except socket.timeout.
Using UDP:
Using UDP, corruption shouldn't be a surprise. You probably don't want to be using UDP here, you should switch to TCP.
Here's why UDP is not appropriate here:
Packets may be lost but others could still reach their destination.
Packets may be duplicated
Packets may arrive in the wrong order
Note that switching to TCP will involve some refactoring of your code (it's a bit more complicated that just replacing SOCK_DGRAM with SOCK_STREAM), but in your case, you have to do it.
I'm not saying UDP is bad, but it's not appropriate in your case.