I'm writting a FTP Server in Python 2.7 over Sockets.
In FTP protocol, connection has to be closed by sender to define the End Of the File (EOF).
The code below is a brief example of what I am doing. Server sends file.mp4, since the server is the sender of the file, he will have to close the connection to define the end of the file.
C->S : PASV
S->Bind Port()
S->C : Entering Passive Mode (ip1,ip2,ip3,ip4,port1,port2)
C->Connect PASV (ip, port)
C->S : RETR file.mp4
S->C : Transfer file content via PASV connection....
S->Close PASV connection() = End Of File.
S->C : 226 File successfully transferred.
So far so good, but here is the error I need to handle.
If the Server has more UPLOAD speed than Client's DOWNLOAD speed (most likely).
Then the server would have already sent the file and closed connection = EOF before Client could have finished downloading it and saving it. Result, Client gets a malformatted file.
How could I figure out this?
Related
I am trying to upload a file to an ftp server on my same wifi network to get a picture on to a digital picture frame. I succeeded in uploading through file explorer, but when uploading using a python script I get a 530 response.
Here is the code so far
import ftplib
ftp = ftplib.FTP()
ftp.connect("111.111.1.11", 1111) #dummy host and port
file = open('C:/path/to/file/test1.png','rb')
ftp.storbinary('test.png', file)
file.close()
ftp.quit()
The server does not requre me to log in with a username and password on file explorer, is there some sort of default I need?
530 error code means Authentication failed error so you are missing the authentication piece. Maybe you can do something like this:
ftp = FTP(source_address=("111.111.1.11", 1111))
ftp.login(user, password)
Note that if you don't provide a user and password it will login with:
user anonymous
password anonymous
as described here
Also I would recommend you reading about S-FTP (Secure FTP) because in FTP the credentials are passed in clear text in the login request.
S-FTP is a communication protocol similar to FTP but built on top of ssh.
Hope this helped you !
I am working on a program that backs up files like OneDrive or iCloud.
I made everything work but when I send a file trough the sockets it just gets stuck until I close the connection.
Here is the code sample:
The client:
def UpdateFileOnServer(file):
print("[!] Uploading file to server: ",file)
with open(clientFolder+file,"rb") as f:
s.sendall(file.encode())
file = f.read()
s.sendall(file)
print("[!] File has been sent.")
The server:
with open(fileLocation,"wb") as f:
print("[!] File recieved! Downloading...")
while True:
data = conn.recv(4096)
if not data:f.close();break
f.write(data)
So to summarize again:
The client sends the file and says that it has been sent.
The server receives about 90% of the file and indefinitely hangs until I CTRL+C the client
When I do that the server finishes the file transfer and the file is successfully received.
I have found the answer to my problem thanks to all the kind commenters.
Create a server socket.
Connect with the client socket to the server socket.
Send the file with the client to the server.
When the client confirms that the file has been sent you close the connection.
The server continues to listen for connections.
Rinse and repeat!
I'm downloading two files from a C-More industrial HMI FTP server. I don't know what OS the HMI is running but I suspect that its FTP server has some quirks. Using Jython 2.7, one file can be read without difficulty but the other has a space in the file name and the normal wrap-in-quotes solution doesn't work - yet.
The following works in the Windows 10 FTP client.
ftp> get NO_SPACES.csv
200 PORT command successful.
150 Opening ASCII mode data connection for NO_SPACES.csv.
226 Transfer complete.
ftp: 12774 bytes received in 0.27Seconds 47.66Kbytes/sec.
ftp> get "WITH SPACE.csv"
200 PORT command successful.
150 Opening ASCII mode data connection for WITH SPACE.csv.
226 Transfer complete.
ftp: 6328 bytes received in 0.02Seconds 316.40Kbytes/sec.
So far, so good. Now try it in Python:
ftp = FTP(myIP) # Connect.
ftp.login(userName, password) # Login.
ftp.set_pasv(False) # Required by the C-More panel for some reason.
with io.BytesIO() as binary_buffer:
# read all of products into a binary buffer
# ftp.retrbinary("RETR NO_SPACES.csv", binary_buffer.write) # This line works.
ftp.retrbinary('RETR "WITH SPACE.csv"', binary_buffer.write) # This one doesn't.
The script console in my development system reports:
ftplib.error_perm: 550 "WITH SPACE.csv": Requested action not taken.
Filenames have been changed to protect the innocent.
Windows FTP likes the get command. Python seems to favour RETR.
I've tried 'RETR "WITH SPACE.csv"' and "RETR 'WITH SPACE.csv'". Same result.
If I have to I can rename the files in the HMI but that will require some validation and paperwork and that's no fun.
I'm developing this on the latest version of Inductive Automation's Ignition! SCADA system which uses Jython 2.7.
Has anyone got any ideas for me to try?
The ftplib has no issue with spaces. The problem are the quotes you add to the RETR command. There should be no quotes:
ftp.retrbinary('RETR WITH SPACE.csv', binary_buffer.write)
If you enable the debug mode in ftp using the -d switch, you will see that it also sends no quotes to the FTP server in the RETR command:
ftp> get "WITH SPACE.csv"
---> PORT 127,0,0,1,15,145
200 Port command successful
---> RETR WITH SPACE.csv
150 Opening data channel for file download from server of "/WITH SPACE.csv"
226 Successfully transferred "/WITH SPACE.csv"
ftp: 12 bytes received in 0.00Seconds 12000.00Kbytes/sec.
Note that the get is a commandline ftp client user command that translates to the FTP protocol RETR command.
Client side:
data = b'\xff' * 1000000
ssock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
#context is created by ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
ssock = context.wrap_socket(ssock, server_hostname='xd1337sv')
ssock.connect((SERVERADDR, SERVERPORT))
ssock.sendall(data)
#time.sleep(3)
ssock.close()
If I just use regular non-SSL socket, everything works correctly with the server receiving exact amount of data. If I use TLS socket, the behavior then depends on the version.
If I run either the server or client on Python 3.6 and therefore the TLSv1.2 will be used, there's no problem.
Problem arises only when TLSv1.3 is used and depends on the size of data and how soon client ssocket.close() line is executed.
If I put a right amount of time.sleep before ssocket.close() depending on the size of data, then I get no error. Otherwise, the server will get ConnectionResetError [WinError 10054] An existing connection was forcibly closed by the remote host and receive only part of the data, or throw ConnectionAbortedError [WinError 10053] An established connection was aborted by the software in your host machine and receive no data.
I'm testing both the server and client on my local machine with local address 192.168.1.2.
The difference is caused by TLS 1.3 sending a session ticket after the TLS handshake while with previous TLS versions the session ticket is send inside the TLS handshake. Thus, with TLS 1.3 data from the server (the session ticket) will arrive after the ssock.connect(...) is done. Since your application does not read any data after the connect it closes the socket while unread data are still inside the socket buffer of the underlying TCP socket. This will cause RST send to the server and cause there the connection reset error.
This is a known problems with applications which never attempt to read from the server. If the application would expect a response from the server and use recv to get it this would implicitly also read the session ticket.
To fix this situation when you don't expect the server to return any application data do a proper SSL shutdown of the socket before closing it. Since this will read the servers SSL shutdown message it will also implicitly read the session ticket send before by the server.
try:
ssock = ssock.unwrap()
except:
True
ssock.close()
For more information see also this issue and this documentation.
I was getting a similar problem when the application was running through gunicorn with certificates. The jsondecodeerror problem randomly came to the client, i.e. the response was empty. The only thing that TLS 1.2 was used.
The solution was simple, I deployed the application on uwsgi and the problem went away
I'm using a library (sentry.io observer) that should connect to a remote server via https and upload some data. I do not control the server, but I can see that no data is uploaded. I set the urllib logger level to debug and I see two log messages
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <server_url>:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (2): <server_url>:443
but no exception is thrown. I used wireshark to sniff packets and I see two SSL handshakes are executed, but the FIN packet is sent right after server finishes the handshake. Packets exchange looks like:
< - client sends message
> - server sends message
< TCP handshake [syn, syn ack, ack]
< Client hello
> Server hello, certificate, server key exchange, server hello done
< Client key exchange, change cipher spec, finished
> New session ticket, change cipher spec, finished
< TCP connection termination [fin ack, fin ack, ack]
This packet exchange is done twice, as urllib tries to connect to the remote server twice. The server certificate is valid, but the connection is cancelled by client. I set the library and urllib loggers to debug, but no error messages or anything that could help me narrow the issue down appears.
The issue only appears when requests are done from docker (based on centos 7), but when launching the app on ubuntu host it works fine, connection is established and data is uploaded. What could be the cause of the issue?