We Keep Coding

Python paramiko library - Put files from local server to remote server (CSV file) [duplicate]

Aim: I am trying to use SFTP through Paramiko in Python to upload files on server pc.
What I've done: To test that functionality, I am using my localhost (127.0.0.1) IP. To achieve that I created the following code with the help of Stack Overflow suggestions.
Problem: The moment I run this code and enter the file name, I get the "IOError : Failure", despite handling that error. Here's a snapshot of the error:
import paramiko as pk
import os
userName = "sk"
ip = "127.0.0.1"
pwd = "1234"
client=""
try:
client = pk.SSHClient()
client.set_missing_host_key_policy(pk.AutoAddPolicy())
client.connect(hostname=ip, port=22, username=userName, password=pwd)
print '\nConnection Successful!'
# This exception takes care of Authentication error& exceptions
except pk.AuthenticationException:
print 'ERROR : Authentication failed because of irrelevant details!'
# This exception will take care of the rest of the error& exceptions
except:
print 'ERROR : Could not connect to %s.'%ip
local_path = '/home/sk'
remote_path = '/home/%s/Desktop'%userName
#File Upload
file_name = raw_input('Enter the name of the file to upload :')
local_path = os.path.join(local_path, file_name)
ftp_client = client.open_sftp()
try:
ftp_client.chdir(remote_path) #Test if remote path exists
except IOError:
ftp_client.mkdir(remote_path) #Create remote path
ftp_client.chdir(remote_path)
ftp_client.put(local_path, '.') #At this point, you are in remote_path in either case
ftp_client.close()
client.close()
Can you point out where's the problem and the method to resolve it?
Thanks in advance!

The second argument of SFTPClient.put (remotepath) is path to a file, not a folder.
So use file_name instead of '.':
ftp_client.put(local_path, file_name)
... assuming you are already in remote_path, as you call .chdir earlier.
To avoid a need for .chdir, you can use an absolute path:
ftp_client.put(local_path, remote_path + '/' + file_name)

Generating RSA and writing to file

why do i get exception in this code:
I get output:
[*] Error creating your key
[*] Error creating your key
import os, hashlib
from Crypto.Cipher import AES
from Crypto.PublicKey import RSA
raw_key = RSA.generate(2048)
private_key = raw_key.exportKey('PEM')
try:
with open('master_private.pem', 'w+') as keyfile:
keyfile.write(private_key)
keyfile.close()
print ("[*] Successfully created your MASTER RSA private key")
except:
print ("[*] Error creating your key")
make_public = raw_key.publickey()
public_key = make_public.exportKey('PEM')
try:
with open("master_public.pem", "w+") as keyfile:
keyfile.write(public_key)
keyfile.close()
print ("[*] Successfully created your MASTER RSA public key")
except:
print ("[*] Error creating your key")
File is created successfully but it is not filled with anything. I am just starting Python.

you should catch exeception and show to know the problem, but i think your problem is the write method, private_key its bytes but you must be pass a str to write method them you can try:
keyfile.write(private_key.decode())
other problem could be your permission permissions, mabe not have permission to create a file, try catching the excaption and printing to know what happen
try:
with open('master_private.pem', 'w+') as keyfile:
keyfile.write(private_key)
keyfile.close()
print ("[*] Successfully created your MASTER RSA private key")
except Exception as e:
print ("[*] Error creating your key", e)
also check your syntax why that code is not well tried

SSH/SCP through Paramiko with key in string

I got a great Paramiko Python script to transfer file over SCP protocol. But My need is a single file (script.py) without any other file dependencies so I don't want to have an external file for my SSH private key.
What I'm trying to do is to embed the private key into a string variable and use this variable as the file needed by the script to connect. I tried with the StringIO library but it doesn't seems to work :
hostname = '1.1.1.1' # remote hostname where SSH server is running
port = 22
username = 'user'
password = 'pswd'
dir_local='/home/paramikouser/local_data'
dir_remote = "remote_machine_folder/subfolder"
glob_pattern='*.*'
import os
import glob
import paramiko
import md5
import StringIO
private_key = StringIO.StringIO('-----BEGIN RSA PRIVATE KEY-----\n-----END RSA PRIVATE KEY-----')
def agent_auth(transport, username):
"""
Attempt to authenticate to the given transport using any of the private
keys available from an SSH agent or from a local private RSA key file (assumes no pass phrase).
"""
try:
ki = paramiko.RSAKey.from_private_key_file(private_key)
except Exception, e:
print 'Failed loading' % (private_key, e)
agent = paramiko.Agent()
agent_keys = agent.get_keys() + (ki,)
if len(agent_keys) == 0:
return
for key in agent_keys:
print 'Trying ssh-agent key %s' % key.get_fingerprint().encode('hex'),
try:
transport.auth_publickey(username, key)
print '... success!'
return
except paramiko.SSHException, e:
print '... failed!', e
# get host key, if we know one
hostkeytype = None
hostkey = None
files_copied = 0
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
# try ~/ssh/ too, e.g. on windows
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print '*** Unable to open host keys file'
host_keys = {}
if host_keys.has_key(hostname):
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print 'Using host key of type %s' % hostkeytype
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
try:
print 'Establishing SSH connection to:', hostname, port, '...'
t = paramiko.Transport((hostname, port))
t.start_client()
agent_auth(t, username)
if not t.is_authenticated():
print 'RSA key auth failed! Trying password login...'
t.connect(username=username, password=password, hostkey=hostkey)
else:
sftp = t.open_session()
sftp = paramiko.SFTPClient.from_transport(t)
# dirlist on remote host
# dirlist = sftp.listdir('.')
# print "Dirlist:", dirlist
How can I use this string as a key?

Use RSAKey.from_private_key:
ki = paramiko.RSAKey.from_private_key(private_key)
See How do use paramiko.RSAKey.from_private_key()?
The answer on the above question shows code for Python 3.
In Python 2.7, this works:
import os
import glob
import paramiko
import StringIO
private_key_file = StringIO.StringIO()
private_key_file.write('-----BEGIN RSA PRIVATE KEY-----\nMIIEoQIBAAKCAQEAvG9YlF2da0jJ5PvvlmVnVnYYFc7kkJuC0wvsACVuvep/sds5\nIEX0e+/rq9UBj/V3rzsvbHzb6IVulSjEqcM32NA4SyqR1m5jAj/WVDXQcxzruBDO\nZbdNhDS1T4+HckTWzttAE4o83bRju+3BhR9CtrDtt+7CSei4MccSMEH7yxo1BGuL\nONfkhB6qAWh55T6tamTyjLg9R9xqBkG6x3ZmoOB9j/11P5awuUoE1DfbqQ3KMLSR\nr64unavECfBaBYvuxlWbxRJFAN8C95oE+Kbc1g1bEL9du6FIeHZ/eaBbkcl84Fm7\nswdHBnd7+tqfo4TGzvbEW4H2ZQLiuiGK23ao0wIBJQKCAQEAiYGvV4KVd81VDuFb\nzp0GOCypyrmSCKjVFowowdYgYRLnj5/5QRB0IxbcaKJbFgYm56e60qBNclOIC/sn\nuiasNm5uRLBclY7SoMbM1aq6tN3AxJakc70c4+8chip3mJMZStdYRZw6QOtrX5+o\n5JpFcI7yqNDS96nShTBnN/jMf3K6yjQjvTv/DJi9SHJ6dTtrY1AuUNEoiO3cwgeH\nFks169756L+fpweL4VjQl4UyClL0bwHWpe579XzjBV0AlGu1tHaE5zslTPtGw1lg\nnZhj/7skZKAIGQxIzfmGv4QEcvePKYzM8EUhOr/0O3BHjLC0lp5hMwmsPJfaHlMb\nBak0JQKBgQD0cRu65WNkCcRlpuUvp5/kiMvu7PmcFUsY6dMeV0bL4oQ+PCqfwXFj\nhkyS7V2DJnllYPwi6E68soie+IL1blmY7hWcoznJ48PWJ0bJmqBgzhpC623RtTKS\ny/O0IbrGKPpaRGfD/PAvqJOpwx7Im2k6/UVQ0OYSurC8CB3BDRTCXQKBgQDFWEq7\ny2SntPFA9zu+31bW57lb26l8nNmUXmRLnXyvqomAkCGSadiW/i5nBEBDV/zJ/rXp\n0QWrmrpfvjnMF6g26m4sj6Pfs5zoSV1+FEidqYDcytUPJnpR55Ulpshf57TGuFbx\n1SCnda1dmm3TzAzzKTc6MbSPV0krMyAgCP7E7wKBgFXijTPUDioRRQEe9pQz+eiD\nFzhFbHUcPPrqXu78EfSbsexaVCpK4qZtdNmtWDT/rhyzX4Hi6zthUpi4LgM0nAVM\nu3w5WX5JG0s+O3dEKoLgoXF1UBk/qfw50iqIZDfJNV38W889McuOQbgvzIusObrH\nsJIENSks1k/nLQx6N7npAoGAUAEzDdzVx3LeWJuUwwCY06oM4Azxrw8nxochvco5\nd6YAZI11ZN7NbaVRFQG5MA7p8QZlbKDYyQdgUFQJl+3qP8bSuB6Oix9Ncu1Panbt\nAaWVGz14+E3ej+hDYkqIlZVJSaStoE978NyuETDEvaXAD40/5yjoVclwsKYGGtM2\n2jcCgYA6v1tvd2QdDeijiSRnXAeJ1hDLB8Jj2WJqnDZ7dQ5+XTIKfY4POIpHCPx8\n6Uk4NCnyJGmBHog1M7Bjb/o0c1UTid6CNBI4ciVaRyXXcy6Czup2EhkiNGom2883\n8+9pdxShKf0pJCqdZxJdVmg1NHZnr20PwN7PASbVcRg3t+wt2g==\n-----END RSA PRIVATE KEY-----')
private_key_file.seek(0)
ki = paramiko.RSAKey.from_private_key(private_key_file)

Python SSL certificate check hostname according to common name

I want to check if an hostname and a port according to a SSL certificate. I created this function :
#staticmethod
def common_name_check(hostname, port):
try:
ctx = ssl.create_default_context()
s = ctx.wrap_socket(socket.socket(), server_hostname=hostname)
s.connect((hostname, int(port)))
cert = s.getpeercert()
ssl.match_hostname(cert, hostname)
except Exception as e:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
print(exc_type, fname, exc_tb.tb_lineno)
return False
else:
return True
My problem is : When a certificate is expired the verification failed. But the exception is ambiguous :
<class 'ssl.SSLError'>
I can't know if error is due to certificate expired or bad common name
How can I only check if hostname/port is valid for certificate ?

Exactly. SSL Error is too generic. I struggled with this too. So you might want to checkout this link.
Verifying SSL certificates in python
Here is a code only answer.
from OpenSSL import SSL
from socket import socket
context = SSL.Context(SSL.TLSv1_METHOD) # Use TLS Method
context.set_options(SSL.OP_NO_SSLv2) # Don't accept SSLv2
context.set_verify(SSL.VERIFY_NONE, callback)
context.load_verify_locations(ca_file, ca_path)
host = "google.com"
port = 443 #ssl
sock = socket()
ssl_sock = SSL.Connection(context, sock)
ssl_sock.connect((host, port))
ssl_sock.do_handshake()
def callback(self, conn, certificate, err, depth, valid):
# err here is the error code ([List of error codes and what they mean][2].
if err == 62:
print("HOSTNAME MISMATCH!!!")

I'm using something like this. certfile is PEM format.
Not quite the same, as I'm comparing against a certificate file..
# Check if the certificate commonName is a match to MYSTRING
certpath = 'c:\cert.cer'
from cryptography import x509
from cryptography.hazmat.backends import default_backend
cfile = open(certpath,"r")
cert_file = cfile.read()
cert = x509.load_pem_x509_certificate(cert_file, default_backend())
for fields in cert.subject:
current = str(fields.oid)
if "commonName" in current:
if fields.value == MYSTRING:
print 'certificate (' + fields.value + ') matches'
else:
print 'certificate (' + fields.value + ') does NOT match'

python paramiko error in Django: 'str' object has no attribute 'get_name'

So I've heard that it's an incredibly bad idea to host media on the same server as your Django/Apache web framework. So to solve this problem with my iOS app and greatly improve server side performance in the long run would be to upload the images to the Django server and then right after that, transfer the newly uploaded image to a separate server dedicated to hosting user profile images.
So using Paramiko I can send files directly to the directory of my choosing, but this does not seem to be working in my Django view.
Here's an incredible simple python script using Paramiko that allows you to upload files to a remote server via SMTP:
import base64
import getpass
import os
import socket
import sys
import traceback
import paramiko
#from paramiko.py3compat import input
# setup logging
#paramiko.util.log_to_file('demo_sftp.log')
port = 22
hostname = '198.199.101.115'
password = 'XXXXXXXXX'
username = 'root'
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'XXXXXXXXX'
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
# try ~/ssh/ too, because windows can't have a folder named ~/.ssh/
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print('*** Unable to open host keys file')
host_keys = {}
if hostname in host_keys:
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print('Using host key of type %s' % hostkeytype)
# now, connect and use paramiko Transport to negotiate SSH2 across the connection
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
# dirlist on remote host
dirlist = sftp.listdir('.')
print("Dirlist: %s" % dirlist)
# copy this demo onto the server
target_directory = '3COOL'
sftp.put('test_image.jpg', target_directory+'/test_image.jpg')
t.close()
So this script works perfectly fine when I run the script on my remote server which successfully transfers the jpeg image to another remote server I own. So I'd like to just cut and paste this script into my Django view which will transfer all freshly uploaded images to a different server, here's how this looks:
def profile_picture(request):
if request.POST:
form = UserProfileForm(request.POST, request.FILES)
obj = form.save(commit=False)
obj.user_id = request.user.id
obj.profile_picture = obj.profile_picture
check = UserProfile.objects.filter(user_id=request.user.id)
if check:
oldup = UserProfile.objects.get(user_id=request.user.id)
oldup.delete()
obj.save()
formNew = UserProfileForm()
args = {}
args.update(csrf(request))
args['uid'] = request.user.id
args['form'] = formNew
# CONVERT THE IMAGE TO A SMALLER SIZE
basewidth = 256
img = Image.open('var/www/bitcraft/static/'+str(obj.profile_picture))
wpercent = (basewidth/float(img.size[0]))
hsize = int((float(img.size[1])*float(wpercent)))
img = img.resize((basewidth,hsize), PIL.Image.ANTIALIAS)
img.save('var/www/bitcraft/static/'+str(obj.profile_picture))
import getpass
import os
import socket
import sys
import traceback
import paramiko
from paramiko.py3compat import input
# TRANSFER THIS IMAGE TO MEDIA HOSTING SERVER
port = 22
hostname = '198.199.101.115'
password = 'pcorysatqwrw'
username = 'root'
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'pcorysatqwrw'
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
except IOError:
try:
host_keys = paramiko.util.load_host_keys(os.path.expanduser('~/ssh/known_hosts'))
except IOError:
print('*** Unable to open host keys file')
host_keys = {}
if hostname in host_keys:
hostkeytype = host_keys[hostname].keys()[0]
hostkey = host_keys[hostname][hostkeytype]
print('Using host key of type %s' % hostkeytype)
t = paramiko.Transport((hostname, port))
t.connect(username=username, password=password, hostkey=hostkey)
sftp = paramiko.SFTPClient.from_transport(t)
dirlist = sftp.listdir('.')
print("Dirlist: %s" % dirlist)
target_directory = '3COOL'
sftp.put('var/www/bitcraft/static/'+str(obj.profile_picture), target_directory+str(obj.profile_picture))
t.close()
return render_to_response('profile.html', args, RequestContext(request))
else:
formNew = UserProfileForm()
args = {}
args.update(csrf(request))
args['uid'] = request.user.id
args['form'] = formNew
return render_to_response('profile.html', args, RequestContext(request))
Ultimately though, adding the simple Python script has not worked due to a strange error:
AttributeError at /upload_profile/ 'str' object has no attribute 'get_name'
Exception Location: /usr/local/lib/python2.7/dist-packages/paramiko-1.14.0-py2.7.egg/paramiko/transport.py in connect, line 873
/srv/www/django/chatfeed/views.py in profile_picture
t.connect(username=username, password=password, hostkey=hostkey)
I don't understand what's causing this... is this related to directory permission denied??

The problem is that you're passing a string for hostkey. As the docs show, that's supposed to be a PKey—that is, an object that wraps a private key.
And if you click on PKey, you'll see that a PKey has a get_name method. A string obviously doesn't. Hence the error.
You have some code that's supposed to use Paramiko to load a hostkey out of ~/.ssh/known_hosts at the top of your script. But if that hostname in host_keys is false, it won't do anything, and you'll end up with the default values you stuck at the top of the script:
hostkeytype = 'ecdsa-sha2-nistp256'
hostkey = 'XXXXXXXXX'
I don't know what you have in place of that 'XXXXXXXXX' in your real code, but presumably it's a string, and therefore not a valid hostkey.
I don't understand what's causing this... is this related to directory permission denied??
You'd have to tell us what you're talking about. Exactly what error or warning or whatever are you getting, and where? If the server is running as a user who doesn't have access to its own ~/.ssh, so you're seeing a "directory permission denied" warning of some kind and your host keys aren't getting loaded, then yes, this problem is indirectly related to that one—as in, if the host actually was in known_keys, and you solved the permissions problem, that would mask the error in your code, so you wouldn't notice it. But there are plenty of other things you could mean where the answer would be no, that has nothing to do with anything.

key = paramiko.RSAKey.from_private_key_file(filename='openssh key f',password='passphrase')
The keys file does not support .ppk format.