Hi I'm new to the community and new to Python, experienced but rusty on other high level languages, so my question is simple.
I made a simple script to connect to a private ftp server, and retrieve daily information from it.
from ftplib import FTP
#Open ftp connection
#Connect to server to retrieve inventory
#Open ftp connection
def FTPconnection(file_name):
ftp = FTP('ftp.serveriuse.com')
ftp.login('mylogin', 'password')
#List the files in the current directory
print("Current File List:")
file = ftp.dir()
print(file)
# # #Get the latest csv file from server
# ftp.cwd("/pub")
gfile = open(file_name, "wb")
ftp.retrbinary('RETR '+ file_name, gfile.write)
gfile.close()
ftp.quit()
FTPconnection('test1.csv')
FTPconnection('test2.csv')
That's the whole script, it passes my credentials, and then calls the function FTPconnection on two different files I'm retrieving.
Then my other script that processes them has an import statement, as I tried to call this script as a module, what my import does it's just connect to the FTP server and fetch information.
import ftpconnect as ftpc
This is the on the other Python script, that does the processing.
It works but I want to improve it, so I need some guidance on best practices about how to do this, because in Spyder 4.1.5 I get an 'Module ftpconnect called but unused' warning ... so probably I am missing something here, I'm developing on MacOS using Anaconda and Python 3.8.5.
I'm trying to build an app, to automate some tasks, but I couldn't find anything about modules that guided me to better code, it simply says you have to import whatever .py file name you used and that will be considered a module ...
and my final question is how can you normally protect private information(ftp credentials) from being exposed? This has nothing to do to protect my code but the credentials.
There are a few options for storing passwords and other secrets that a Python program needs to use, particularly a program that needs to run in the background where it can't just ask the user to type in the password.
Problems to avoid:
Checking the password in to source control where other developers or even the public can see it.
Other users on the same server reading the password from a configuration file or source code.
Having the password in a source file where others can see it over your shoulder while you are editing it.
Option 1: SSH
This isn't always an option, but it's probably the best. Your private key is never transmitted over the network, SSH just runs mathematical calculations to prove that you have the right key.
In order to make it work, you need the following:
The database or whatever you are accessing needs to be accessible by SSH. Try searching for "SSH" plus whatever service you are accessing. For example, "ssh postgresql". If this isn't a feature on your database, move on to the next option.
Create an account to run the service that will make calls to the database, and generate an SSH key.
Either add the public key to the service you're going to call, or create a local account on that server, and install the public key there.
Option 2: Environment Variables
This one is the simplest, so it might be a good place to start. It's described well in the Twelve Factor App. The basic idea is that your source code just pulls the password or other secrets from environment variables, and then you configure those environment variables on each system where you run the program. It might also be a nice touch if you use default values that will work for most developers. You have to balance that against making your software "secure by default".
Here's an example that pulls the server, user name, and password from environment variables.
import os
server = os.getenv('MY_APP_DB_SERVER', 'localhost')
user = os.getenv('MY_APP_DB_USER', 'myapp')
password = os.getenv('MY_APP_DB_PASSWORD', '')
db_connect(server, user, password)
Look up how to set environment variables in your operating system, and consider running the service under its own account. That way you don't have sensitive data in environment variables when you run programs in your own account. When you do set up those environment variables, take extra care that other users can't read them. Check file permissions, for example. Of course any users with root permission will be able to read them, but that can't be helped. If you're using systemd, look at the service unit, and be careful to use EnvironmentFile instead of Environment for any secrets. Environment values can be viewed by any user with systemctl show.
Option 3: Configuration Files
This is very similar to the environment variables, but you read the secrets from a text file. I still find the environment variables more flexible for things like deployment tools and continuous integration servers. If you decide to use a configuration file, Python supports several formats in the standard library, like JSON, INI, netrc, and XML. You can also find external packages like PyYAML and TOML. Personally, I find JSON and YAML the simplest to use, and YAML allows comments.
Three things to consider with configuration files:
Where is the file? Maybe a default location like ~/.my_app, and a command-line option to use a different location.
Make sure other users can't read the file.
Obviously, don't commit the configuration file to source code. You might want to commit a template that users can copy to their home directory.
Option 4: Python Module
Some projects just put their secrets right into a Python module.
# settings.py
db_server = 'dbhost1'
db_user = 'my_app'
db_password = 'correcthorsebatterystaple'
Then import that module to get the values.
# my_app.py
from settings import db_server, db_user, db_password
db_connect(db_server, db_user, db_password)
One project that uses this technique is Django. Obviously, you shouldn't commit settings.py to source control, although you might want to commit a file called settings_template.py that users can copy and modify.
I see a few problems with this technique:
Developers might accidentally commit the file to source control. Adding it to .gitignore reduces that risk.
Some of your code is not under source control. If you're disciplined and only put strings and numbers in here, that won't be a problem. If you start writing logging filter classes in here, stop!
If your project already uses this technique, it's easy to transition to environment variables. Just move all the setting values to environment variables, and change the Python module to read from those environment variables.
I'm dealing with a large existing python application and trying to set global defaults for all the boto3 resources and clients that don't have any specified. Since there are many of them, I don't want to update every place the resources are created to create a botocore Config object; so it seems to make sense to use the environment variable approach to configuration. I want to set 4 configurations related to timeout and retry, but of the 4, the documentation only indicates that 2 of them can be set via environment variables. Same for configuring using a config file.
botocore.Config supports connect_timeout, read_timeout, retry mode, and retry max_attempts.
But the environment variables only support AWS_MAX_ATTEMPTS and AWS_RETRY_MODE (at least according to the documentation). How to set the connect_timeout and read_timeout by environment variable?
I don't think this is possible at the moment.
It seems that the environment variable names are all explicitly defined in botocore's config provider.
I have some configuration in a json file and on the database and I want to load those configuration on Django startup (Apache server startup).. I will be using those global variable within all the application.
For Example: External server connection api or number of instances.
What is the best way to define the global variables. I want to load the json file when server starts and use the variable value util server stop. ?
It sounds like the thing you're probably looking for is environment variables - you can always use a small script to set the environment variables from the JSON that you have at present.
Setting these in your .bashrc file or, more preferably a virtualenv will let you:
Take sensitive settings, like SECRET_KEY out of version control.
Offer database settings, either by supplying them as a DB URL or as seperate environment variables.
Set both Django settings and other useful variables outside of the immediate Django project.
The django-environ docs have a useful tutorial on how to set it up. The Django Cookie-Cutter project makes extensive use of Environment Variables (including DB and mailserver settings), and is a great place to pick up hints and approaches.
I need to get all permissions for particular user in VisualSVN server by using the python WMI query.
Is it possible to get the permissions in a single query ?
Upgrade VisualSVN Server to version 3.4. The new release introduces PowerShell cmdlets for Subversion server and repositories administration and management. New cmdlets you are interested in are Get-SvnAccessRule and Select-SvnAccessRule. Depending on your task, you could use one of the cmdlets to
obtain a list of effective access rules on a particular repository path
Select-SvnAccessRule MyRepo -Path /MyProject/foo/bar
obtain a list of all access rules explicitly assigned for user account DOMAIN\username
Get-SvnAccessRule -AccountName DOMAIN\Username
obtain a list of access rules explicitly assigned for user account (its SID)
Get-SvnAccessRule -AccountId S-1-5-32-545
I’m new at making COM servers and working with COM from Python so I want to clarify a few things I could not find explicit answers for:
Creating GUID’s Properly for COM servers
Do I generate:
The GUID for my intended COM server manually, copy it and use that # for the server from then on in? Therefore, when I distribute the application the other users will use the same GUID I created during development.
A new GUID each time the application or COM server object is initialized?
A new GUID on a per computer basis, only once during the initial setup then have the app save the GUID # and on future loads it pulls that # from the users setup file?
Example Scenario 1:
i) print(pythoncom.CreateGuid()) #in interpreter
ii) _reg_clsid_ = copy above GUID into your app
Example Scenario 2:
i)_reg_clsid_ = pythoncom.CreateGuid()
Example Scenario 3:
if self.isfile = os.path.isfile(url):
load_previous_generated_GUID(url)
else:
#first time running application or setup file is missing
GUID = pythoncom.CreateGuid()
save_GUID_to_setup_file(GUID)
Can I use GUID’s for tracking program/COM server versions?
If scenario #1 above is correct then:
When I make an upgrade I can test for the old GUID so I can interact with it correctly?
TODO: How do I get the GUID of a COM server from Python and/or VBA?
You definitely want scenario 1. You should generate a CLSID once during development, and use that until you change the set of functions your class exports (to COM).
Not only can you use CLSIDs for tracking versions, you must change the CLSIDs whenever you release a new version, or else you break COM identity rules; breaking the rules usually results in obscure failure modes when invoking your object out-of-process.
Typically, however, you don't expose CLSIDs directly to code, you expose PROGIDs instead, which are human-readable strings with embedded versioning information, and use the Win32 API CLSIDFromProgID to convert between the two. (VBA will do this for you; Python may do this too.)