We Keep Coding

pysftp throwing paramiko.ssh_exception.SSHException even though known_hosts file provided?

Getting error
paramiko.ssh_exception.SSHException: No hostkey for host target.org found.
when using pysftp (for a connection that requires a specific port), even though I am providing the same known_hosts file that was initially used to connect to that location (following the post here). Ie. did...
[airflow#airflowetl reporting]$ sftp -oPort=15259 my_user#target.org
The authenticity of host '[target.org]:15259 ([205.172.2.88]:15259)' can't be established.
RSA key fingerprint is SHA256:UM6OflG0rkcYohes7qOlYoJZ4TIqVd0JQSh7HXYZQVA.
RSA key fingerprint is MD5:33:c2:30:22:57:5b:57:98:2f:11:07:4d:a3:4a:10:0f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[target.org]:15259,[205.172.2.88]:15259' (RSA) to the list of known hosts.
password
Enter password for my_user
Password:
Connected to target.org.
sftp> ls
csv_drop test_results
sftp> exit
and then copied the /home/me/.ssh/known_hosts to another location
[airflow#airflowetl .ssh]$ ls -lha
total 8.0K
drwx------ 2 airflow airflows 25 Oct 19 17:31 .
drwx------. 32 airflow airflows 4.0K Oct 19 17:31 ..
-rw-r--r-- 1 airflow airflows 777 Oct 19 17:32 known_hosts
[airflow#airflowetl .ssh]$ ls -lha /home/airflow/projects/backups/reporting/configs/known_hosts
-rw-r--r-- 1 airflow airflows 777 Oct 19 17:34 /home/airflow/projects/backups/reporting/configs/known_hosts
(noting that the permissions are the name for the original and copy) that gets used like...
# connect to ftp location
assert os.path.isfile(os.path.join(PROJECT_HOME, "configs", "known_hosts"))
cnopts = pysftp.CnOpts(knownhosts=os.path.join(PROJECT_HOME, "configs", "known_hosts"))
#cnopts = pysftp.CnOpts(knownhosts="/home/airflow/.ssh/known_hosts")
HOSTNAME = "target.org"
PORT = 15259
sftp = pysftp.Connection(HOSTNAME,
port=PORT,
username=CREDS["sink"]["ftp_creds"]["username"],
password=CREDS["sink"]["ftp_creds"]["password"],
cnopts=cnopts)
print(sftp.pwd())
print(sftp.listdir())
sftp.cwd(CONF["reporting_configs"]["to"]["share_folder_path"])
print(sftp.pwd())
print(sftp.listdir())
Yet, when running the script I get the error saying it could not find the hostkey (Note that even using the original known_hosts file path throws the same error (and happens whether using hostname or the IP)). What could be causing this? Any more debugging steps I could try to get more info? Don't have much experience with this kind of thing.
Looking in the debugger while running, I see that the cnopts hostkeys entries does seem to contain the right hostkey...
<HostKeyEntry ['[target.org]:15259', '[205.172.2.88]:15259']: <paramiko.rsakey.RSAKey object at 0x7f8d752d4208>>
Note that the hostkey entry is '[target.org]:15259' (ie. it is combining the specified port), even though the name provided as the service name to the connection function is just 'target.org'
The full traceback looks like...
Traceback (most recent call last):
File "./source/local2ftp.2.py", line 52, in <module>
cnopts=cnopts)
File "/home/airflow/projects/backups/reporting/venv/lib/python3.6/site-packages/pysftp/__init__.py", line 132, in __init__
self._tconnect['hostkey'] = self._cnopts.get_hostkey(host)
File "/home/airflow/projects/backups/reporting/venv/lib/python3.6/site-packages/pysftp/__init__.py", line 71, in get_hostkey
raise SSHException("No hostkey for host %s found." % host)
paramiko.ssh_exception.SSHException: No hostkey for host target.org found.
Exception ignored in: <bound method Connection.__del__ of <pysftp.Connection object at 0x7f1a61df6f98>>
Traceback (most recent call last):
File "/home/airflow/projects/backups/reporting/venv/lib/python3.6/site-packages/pysftp/__init__.py", line 1013, in __del__
self.close()
File "/home/airflow/projects/backups/reporting/venv/lib/python3.6/site-packages/pysftp/__init__.py", line 784, in close
if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
Note that the user I used to initially connect via sftp in the command line and am using in the pysftp script is not the user running the script (the user I used to connect to the sftp server is a special set of credentials for connecting). I don't know if this is relevant.

The pysftp does not support the host key entries with the port.
Either skip pysftp and use Paramiko directly.
Or hack it by replacing the [target.org]:15259 with target.org in the known_hosts file.

TLDR: The code expects the pystfp.Connection hostname to match that in the hostkey file (eg. ~/.ssh/known_hosts), but if the rsa entry in the hostkey file entry was created with a connection that specified the port and thus caused the rsa entry in known_hosts to be formatted in a way that pysftp could not understand / handle,
like...
[airflow#airflowetl reporting]$ sftp -oPort=15259 my_user#target.org
The rsa entry that gets created looks like...
'[target.org]:15259,[205.172.2.88]:15259 ssh-rsa AAAAB3HJGVJGCTCKHGVYTVKUH===...'
so when you use this hostkey file for the knownhosts in...
cnopts = pysftp.CnOpts(knownhosts=os.path.join(path, to, hostkey, or, known_hosts, file))
hostname = "target.org"
port = 15259
sftp = pysftp.Connection(hostname, port, username=user, password=pass, cnopts=cnopts)
the code ends up checking the hostkey file for an entry that looks like
"target.org"
(what you entered as the hostname) but only finds
"[target.org]:15259"
(the entry in the hostkey / known_hosts file formatted for the specific port you connected on) so throws an error.
Note also that you can't just use "[target.org]:15259" as the hostname for the pysftp.Connection() function either just to get it to match when the pysftp code does this internal check because it will say that it does not recognize the service name since (apparently) that's not a valid servicename representation (ie. it won't just know to separate that string into the hostname and port components).
What I had to do was...
Copy the known_hosts file that was created when I initially connected to the target.org host
then in that file, manually edit it to look like
'target.org,205.172.2.88 ssh-rsa AAAAB3HJGVJGCTCKHGVYTVKUH===...'
Specifically in the code, the pysftp.Connection() function...
Tries to get the hostkey
and uses the underlying Paraminko module to try to find the hostkey entry from the knownhosts file given in the cnopt arg
which uses the function here to match the string literal entries from the hostkey file with the hostname that you entered as the pysftp.Connection arg (which, if you connected to using a specific port, created an rsa entry that when taken as a literal is not formatted the way you likely are passing in the hostname arg and is not even a valid host name), thus confusing the matching logic and causing it to not find any matches
thus causing an exception to be raised, here back in the pysftp code

Python & Mcpi (minecraft) - connection refused error

I use mcpi: https://github.com/AdventuresInMinecraft/AdventuresInMinecraft-Linux
Starting the local server.
After, run program:
import mcpi.minecraft as minecraft
mc = minecraft.Minecraft.create()
mc.postToChat("Hello Minecraft World")
I am facing the below error:
Traceback (most recent call last):
File "/home/home/AdventuresInMinecraft/MyAdventures/HelloMinecraftWorld.py", line 2, in mc = minecraft.Minecraft.create()
File "/home/home/.local/lib/python3.6/site-packages/mcpi/minecraft.py", line 376, in create return Minecraft(Connection(address, port))
File "/home/home/.local/lib/python3.6/site-packages/mcpi/connection.py", line 17, in init self.socket.connect((address, port))
ConnectionRefusedError: [Errno 111] Connection refused

A ConnectionRefusedError means that the address + port combination was unable to be secured for this particular Minecraft server and thus raised an exception. This could be because some other application is already using the port of interest, the port is unavailable because of the OS, or a handful of other networking configuration mishaps.
But perhaps a better series of questions to ask yourself is:
What is the default address and port that minecraft.Minecraft.create() will attempt to launch / listen at?
Do I have access to that server (address + port)?
If I do have access, are there any security issues (AKA Firewall)?
This post has already addressed the root issue of your question, and I hope it gives you a good start at understanding the foundation of your problem.
Notice how their question mentions s.connect((host,port)) and your stack trace has self.socket.connect((address, port)) Looks like the same thing to me!
Some more reading:
- localhost
- check if port is in use

I encountered the same issue. I looked into the code of mcpi and found that the default port is 4711. However, a Minecraft Server's default port is 25565. All you need to do is add 2 parameters on the create() function. Code(Python):
mc = minecraft.Minecraft.create(address="127.0.0.1", port=25565)
btw change "address" in the code to the host of the server (only if you modified the "server.properties" file).
Also, ConnectionRefusedError doesn't mean that it's not secured, I believe it means that either the server is not online, it doesn't exist, or the server refused it for some reason.
EDIT:
Oops sorry I just found out that mcpi actually connects to the RaspberryJam plugin which is hosted on another IP and port. The plugin runs on port 4711. So mcpi has the right port.
So check if you have the RaspberryJam plugin installed. If not, download it from
https://www.spigotmc.org/resources/raspberryjuice.22724/
And put the .jar file inside the plugins folder in your server directory.

implement CRCCheck during SFTP file transfer [duplicate]

I use Paramiko to put a file to an SFTP server:
import paramiko
transport = paramiko.Transport((host, port))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(local_path, remote_path)
Now, I would like to check if it worked. The idea is that I compare the checksum of the local file and the remote one (that is located on the SFTP server).
Does Paramiko functionality allows to do that? If it is the case, how exactly it works?

With the SFTP, running over an encrypted SSH session, there's no chance the file contents could get corrupted while transferring. So unless it gets corrupted when reading the local file or writing the remote file, you can be pretty sure that the file was uploaded correctly, if the .put does not throw any error.
try:
sftp.put(local_path, remote_path)
except:
# Something went wrong
If you want to test explicitly anyway:
While there's the check-file extension to the SFTP protocol to calculate a remote file checksum, it's not widely supported. Particularly it's not supported by the most widespread SFTP server implementation, the OpenSSH. See What SFTP server implementations support check-file extension.
If you are lucky to connect to another SFTP server that supports the extension, you can use the Paramiko's SFTPFile.check method.
If not, your only option is to download the file back and compare locally.
If you have a shell access to the server, you can of course try to run some shell checksum command (sha256sum) over a separate shell/SSH connection (or channel) and parse the results. But that's not an SFTP solution anymore. See Comparing MD5 of downloaded files against files on an SFTP server in Python.

if the file doesn't upload then the method will throw an error, so u can check for error
import paramiko
transport = paramiko.Transport((host, port))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
try:
sftp.put(local_path, remote_path)
except IOError:
#'Failure'
except OSError:
#'Failure'

Name or service not known

I have been trying to get the ipaddress of the person who logged into the machine using the below code but I get a error.
>>> import socket
>>> socket.gethostbyname_ex(socket.gethostname())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
socket.gaierror: [Errno -2] Name or service not known
The same code works in other linux box.
Not sure I fix it.

Error has occurred just because of not setting up hostname properly. Set the hostname at three different places, which are in -
/etc/hostname
/etc/hosts
run command $ hostname
then logout and login again. You are done.

Check what is being returned by socket.gethostname() and see if you can ping it. Basically this is a lookup failure. Check your /etc/hosts to see if it is listed. I know it seems strange, but I think if the hostname being returned does not have an entry, you'll get a name service failure which is what that is.

If you are working with IPv6 or with servers with multiple network interfaces, this command will not work correctly.
Instead, you can use this command that tries to connect to the Google DNS server at 8.8.8.8 at port 53, and return your ip:
import socket
print([(s.connect(('8.8.8.8', 53)), s.getsockname()[0], s.close()) for s in [socket.socket(socket.AF_INET, socket.SOCK_DGRAM)]][0][1])

Python Not Executing MySQLdb connect

I am having some issues connecting to my hosted mysql database using python. My code is as follows:
import MySQLdb
db = MySQLdb.connect(host="xxxxxx.db.1and1.com", user="xxxxxxx",passwd="xxxxxxx", db="xxxxxxx")
I get the following error:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
db = MySQLdb.connect(host="db518597727.db.1and1.com", user="dbo518597727", passwd="OilyOily123", db="db518597727")
File "C:\Python27\lib\site-packages\MySQLdb\__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (2003, "Can't connect to MySQL server on 'xxxxxxx.db.1and1.com' (10060)")
My sql database is hosted by 1and1. I am sure I have the right credentials, as when I read from this database on a php website, using the same credentials (I have triple checked) it works fine.
I have tried this both on a Raspberry Pi (where I intend to use it) and on my Windows 8.1 PC, so I am pretty sure that it is not the computer that is the problem.
Any help would be much appreciated.Many thanks.

In MySQL credentials are per client, so we distinguish a couple of client types:
- localhost means the same machine as MySQL server connecting via Unix Socket
- 127.0.0.1 means the same machine as MySQL server connecting via TCP/IP
- A.B.C.D where letters are replaced by numbers in range 0-255 which means IP address of client machine connecting via TCP/IP
- * is a wildcard which means that any client can connect with given credentials via TCP/IP
Every entry in MySQL users table consists of client specification (described above), username, password and some other columns (not relevant here).
So the problem you are facing is that PHP script on 1and1 server can connect to the database hosted on 1and1 since the hosting company sets up their database server to accept connections from their own servers with valid credentials.
But the same credentials are considered invalid for connections coming from client machines unknown to 1and1.
What you can do is to ask 1and1 to give your specific IP access rights to your database.
What you can't to is to overcome this problem on your own.