New to Stackoverflow, so first off, hello.
I'm working on a little project for my school which is supposed to be a custom gui (written in python as an educational challenge for me since I've never used python) for the open source Unison program. We're trying to allow students and staff to sync a folder at home and the school by launching this program with as little input as possible (idiot-proofing if you will). The interface is supposed to be just they're school username and password and the gui wrapper should just send the username and password to Unison and sync them.
The problem is Unison in turn launches SSh and prompts for the password but python subprocess.communicate(input) method won't let ssh take the password. I realized ssh will only accept input from the terminal and I can't figure out how to trick it. I've read some stuff about using a pseudo terminal but I'm still stumped. RSA keys would be the ideal solution, but generating them and then placing them on the remote server still involves me needing to login with a password at least once and that would require a solution to the above, or the terminal which is not idiot proof.
def startSync(self):
'''
'''
userName = self.userNameIn.get()
userPass = self.userPassIn.get()
localDir = "/Users/localuser/syncFolder/"
remoteDir = " ssh://schoolServer/remotesyncFolder" #for testing purposes, I set this to my own home machine which logs into my own account if I don't provide me#myserver.net
unisonExecRemotePath = " -servercmd /Users/RemoteMe/unison" #unison is the unix executable responsible for launching unison on the remote system
silenceCmdPrompts = " -silent" #keeps unison from displaying directory differences and asking if its okay to sync
executionString = "./unison" + localDir + remoteDir + unisonExecRemotePath + silenceCmdPrompts
mainProcess = subprocess.Popen(executionString,shell = True, stdin = subprocess.PIPE)
mainProcess.communicate(userPass)
The execution strings works fine in there terminal if I paste it in. And any general python tips would also be appreciated if you're so inclined.
Thanks!
Unison User Manual: http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html
Edit: I should also note that while I'm currently developing under OSX and Linux, I will eventually have to make this windows compatible since most of my school's students run windows as their primary (or only) machine.
Look into pexpect.
import pexpect
child = pexpect.spawn('ssh myname#host.example.com')
child.expect('Password:')
child.sendline(mypassword)
child.interact()
If you want to send a password to ssh you need to open pseudoterminal (a pty) and talk to it using that instead of just using stdin/stdout. Take a look at the pexpect module, which is designed to do exactly that.
An alternative solution would involve some sort of out-of-band mechanism for distributing public keys: for example, set up a simple web application where people can paste in a public key and have it manage the authorized_keys file on behalf of the user.
Related
Hi I'm new to the community and new to Python, experienced but rusty on other high level languages, so my question is simple.
I made a simple script to connect to a private ftp server, and retrieve daily information from it.
from ftplib import FTP
#Open ftp connection
#Connect to server to retrieve inventory
#Open ftp connection
def FTPconnection(file_name):
ftp = FTP('ftp.serveriuse.com')
ftp.login('mylogin', 'password')
#List the files in the current directory
print("Current File List:")
file = ftp.dir()
print(file)
# # #Get the latest csv file from server
# ftp.cwd("/pub")
gfile = open(file_name, "wb")
ftp.retrbinary('RETR '+ file_name, gfile.write)
gfile.close()
ftp.quit()
FTPconnection('test1.csv')
FTPconnection('test2.csv')
That's the whole script, it passes my credentials, and then calls the function FTPconnection on two different files I'm retrieving.
Then my other script that processes them has an import statement, as I tried to call this script as a module, what my import does it's just connect to the FTP server and fetch information.
import ftpconnect as ftpc
This is the on the other Python script, that does the processing.
It works but I want to improve it, so I need some guidance on best practices about how to do this, because in Spyder 4.1.5 I get an 'Module ftpconnect called but unused' warning ... so probably I am missing something here, I'm developing on MacOS using Anaconda and Python 3.8.5.
I'm trying to build an app, to automate some tasks, but I couldn't find anything about modules that guided me to better code, it simply says you have to import whatever .py file name you used and that will be considered a module ...
and my final question is how can you normally protect private information(ftp credentials) from being exposed? This has nothing to do to protect my code but the credentials.
There are a few options for storing passwords and other secrets that a Python program needs to use, particularly a program that needs to run in the background where it can't just ask the user to type in the password.
Problems to avoid:
Checking the password in to source control where other developers or even the public can see it.
Other users on the same server reading the password from a configuration file or source code.
Having the password in a source file where others can see it over your shoulder while you are editing it.
Option 1: SSH
This isn't always an option, but it's probably the best. Your private key is never transmitted over the network, SSH just runs mathematical calculations to prove that you have the right key.
In order to make it work, you need the following:
The database or whatever you are accessing needs to be accessible by SSH. Try searching for "SSH" plus whatever service you are accessing. For example, "ssh postgresql". If this isn't a feature on your database, move on to the next option.
Create an account to run the service that will make calls to the database, and generate an SSH key.
Either add the public key to the service you're going to call, or create a local account on that server, and install the public key there.
Option 2: Environment Variables
This one is the simplest, so it might be a good place to start. It's described well in the Twelve Factor App. The basic idea is that your source code just pulls the password or other secrets from environment variables, and then you configure those environment variables on each system where you run the program. It might also be a nice touch if you use default values that will work for most developers. You have to balance that against making your software "secure by default".
Here's an example that pulls the server, user name, and password from environment variables.
import os
server = os.getenv('MY_APP_DB_SERVER', 'localhost')
user = os.getenv('MY_APP_DB_USER', 'myapp')
password = os.getenv('MY_APP_DB_PASSWORD', '')
db_connect(server, user, password)
Look up how to set environment variables in your operating system, and consider running the service under its own account. That way you don't have sensitive data in environment variables when you run programs in your own account. When you do set up those environment variables, take extra care that other users can't read them. Check file permissions, for example. Of course any users with root permission will be able to read them, but that can't be helped. If you're using systemd, look at the service unit, and be careful to use EnvironmentFile instead of Environment for any secrets. Environment values can be viewed by any user with systemctl show.
Option 3: Configuration Files
This is very similar to the environment variables, but you read the secrets from a text file. I still find the environment variables more flexible for things like deployment tools and continuous integration servers. If you decide to use a configuration file, Python supports several formats in the standard library, like JSON, INI, netrc, and XML. You can also find external packages like PyYAML and TOML. Personally, I find JSON and YAML the simplest to use, and YAML allows comments.
Three things to consider with configuration files:
Where is the file? Maybe a default location like ~/.my_app, and a command-line option to use a different location.
Make sure other users can't read the file.
Obviously, don't commit the configuration file to source code. You might want to commit a template that users can copy to their home directory.
Option 4: Python Module
Some projects just put their secrets right into a Python module.
# settings.py
db_server = 'dbhost1'
db_user = 'my_app'
db_password = 'correcthorsebatterystaple'
Then import that module to get the values.
# my_app.py
from settings import db_server, db_user, db_password
db_connect(db_server, db_user, db_password)
One project that uses this technique is Django. Obviously, you shouldn't commit settings.py to source control, although you might want to commit a file called settings_template.py that users can copy and modify.
I see a few problems with this technique:
Developers might accidentally commit the file to source control. Adding it to .gitignore reduces that risk.
Some of your code is not under source control. If you're disciplined and only put strings and numbers in here, that won't be a problem. If you start writing logging filter classes in here, stop!
If your project already uses this technique, it's easy to transition to environment variables. Just move all the setting values to environment variables, and change the Python module to read from those environment variables.
Problem:
Customer would like to make sure that the script I've developed in Python, running under CentOS7, can sufficiently obscure the credentials required to access a protected web service (that only supports Basic Auth) such that someone with access to the CentOS login cannot determine those credentials.
Discussion:
I have a Python script that needs to run as a specific CentOS user, say "joe".
That script needs to access a protected Web Service.
In order to attempt to make the credentials external to the code I have put them into a configuration file.
I have hidden the file (name starts with a period "."), and base64 encoded the credentials for obscurity, but the requirement is to only allow the Python script to be able to "see" and open the file, vs anyone with access to the CentOS account.
Even though the file is hidden, "joe" can still do an ls -a and see the file, and then cat the contents.
As a possible solution, is there a way to set the file permissions in CentOS such that they will only allow that Python script to see and open the file, but still have the script run under the context of a CentOS account?
Naive solution
For this use-case I would probably create with a script (sh or zsh or whatever, but I guess u use the default one here) a temporal user iamtemporal-and-wontstayafterifinish. Then creating the config file for being able to read ONLY by specifically this user (and none permission for all the others). Read here for the how: https://www.thegeekdiary.com/understanding-basic-file-permissions-and-ownership-in-linux/
Getting harder
If the problem still raises in case someone would have root-rights (for any such reason), then just simply forget everything above, and start planning for a vacation, cuz' this will be a lot longer then anyone would think.
Is not anymore a simple python problem, but needs a different business logic. The best u could do is to implement (at least this credentials handling part) in a low-level language so could handle memory in a customized way and ask for them runtime only, don't store them...
Or maybe if u could limit the scope of this user accesses towards the protected Web Service as u say.
Bonus
Even tho it wasn't explicitly asked, I would discourage you from storing credentials with using a simple base64...
For this purpose a simple solution could be the following one at least (without the knowledge of the whole armada of cryptography):
encrypt the passw with a asymmetric cryptographic algorithm (probably RSA with a huge key
inject the key for decryption as a env var while you have an open ssh session to the remote terminal
ideally u use this key only while u decrypt and send it, afterwards make sure u delete the references to the variables
Sidenote: it's still filled with 'flaws'. If security is really a problem, I would consider changing technology or using some sort of lib that handles these stuff more securely. I would start probably here: Securely Erasing Password in Memory (Python)
Not to mention memory dumps can be read 'easily' (if u know what u are looking for...): https://cyberarms.wordpress.com/2011/11/04/memory-forensics-how-to-pull-passwords-from-a-memory-dump/
So yeah, having a key-server which sends you the private key to decrypt is not enough, if you read these last two web entries...
Up until now, whenever I have needed to store a "secret" for a simple python application, I have relied on environment variables. In Windows, I set the variables via the Computer Properties dialog and I access them in my Python code like this:
database_password = os.environ['DB_PASS']
The simplicity of this approach has served me well. Now I have a project that uses Oauth2 authentication and I have a need to store tokens to the environment that may change throughout program execution. I want them to persist the next time I execute the program. This is what I have come up with:
#fetch a new token
token = oauth.fetch_token('https://api.example.com/oauth/v2/token', code=secretcode)
access_token = token['access_token']
#make sure it persists in the current session
os.environ['TOKEN'] = access_token
#store to the system environment (Windows)
cmd = 'SETX /M TOKEN ' + access_token
os.system(cmd)
It gets the job done quickly for me today, but does not seem like the right approach to add to my toolbox. Does anyone have a more elegant way of doing what I am trying to do that does not add too many layers of complexity? If the solution worked across platforms that would be a bonus.
I have used the Python keyring module with great success. It's an interface to credential vaults provided by the operating system (e.g., Windows Credential Manager). I haven't used it on Linux, but it appears to be supported, as well.
Storing a password/token and then retrieving it can be as simple as:
import keyring
keyring.set_password("system", "username", "password")
keyring.get_password("system", "username")
I'm currently writing a script where I need to gain access to another computer on my LAN while using administrative credentials that differ from the account I am logged in as. I attempted to use the requests module.
Here is my code so far:
import requests
with requests.Session() as c:
location = ('file://computer/c$/')
USERNAME = 'notrealusername'
PASSWORD = 'notrealpassword'
c.get(location)
logindata = dict(username=USERNAME, password=PASSWORD, next='/')
c.post(location, data=logindata, headers{"Referer":"file://computer/c$/"})
Can someone tell me how I can edit my code to make it work properly according to the criteria specified above?
Impacket
This 3rd party library is pretty useful for Windows related networking tasks. In this situation i would use their wmiexec.py script:
wmiexec.py
A semi-interactive shell, used through Windows Management Instrumentation. It does not require to install any service/agent at the target server. Runs as Administrator. Highly stealthy.
If your not wanting any 3rd party dependencies, you could write your own solution. A wmi shell is mentioned in the BlackHat Python book.
I'm in a situation where I need to pass some texts to a prompt generate by a API (seems for API it's a pretty weird behavior, this is the first time I ran into this), like below:
kvm_cli = libvirt.open("qemu+ssh://han#10.0.10.8/system")
then a prompt shows up asking for the ssh password (password for 10.0.10.8 is:), I have to manually type it there in order to move on and yield the kvm_cli object I needed.
I tried to use the pexpect module to deal with this however it's for OS command line instead of API.
It's also possible to work around this by using ssh certification files but it's not a favorable authentication approach in our scenario.
Since our wrapper to the 'open' method is not interactive, we cannot ask the user to input the password, do you guys have any thought how could I address it?
I am not a libvirt user, but I believe that the problem is not in the library, but in the connection method. You seem to be connecting via ssh, so you need to authenticate yourself.
I've been reading the libvirt page on ArchWiki, and I think that you could try:
setting up the simple (TCP/IP socket) connection method, or
setting up key-based, password-less SSH login for your virtual host.