I have a script which has a global variable that must be set by the user once and for all, the variable is a string containing a pathname, and each time the script runs it needs it. I don't want to prompt the user each time for this pathname.
Currently, I am considering asking the user to a set an environment variable permanently, by adding it to his /etc/profile or .bash_profile, and access it with sys.environ dictionary. The other option would be to have a config file and ask the user to edit the relevant line, then use configparser to read it.
Is there a recommended method for doing this?
Use the Python ConfigParser module, or configparser in Python 3.
It follows the standard *.ini format and allows you to store information from one run to the next in an easily readable format. The format is essentially self-documenting because you can name your keys in the file, and you can add comments to the configuration file too.
It also provides more flexibility over the environment variable method because it is easier to modify a configuration file, and the file can easily be passed from one computer to the next along with your script regardless of platform or other environment settings.
Your use case is exactly what configuration files are intended for, and you could accomplish your task with only a handful of lines of code:
cfg_parser = ConfigParser.ConfigParser() # Python 2.x
if cfg_parser.read('config_file_name.ini'):
path = cfg_parser.get('SECTION_NAME', 'path')
else:
print("No config file found")
This gives you your path, and all you have to ask your user to do is edit one line of a text file instead of making any system changes.
Additionally, this gives you a lot of room to expand in the future. If you ever want more options added to your script, modifying a configuration file is a lot easier than coming up with new environment variables.
Lastly, the ConfigParser library allows you to edit configuration files programmatically as well. You could add a command line option (perhaps with argparse) that allows your user to specify a path, and have your script automagically write its own config file with the path. Now your user never has to touch the configuration file manually, and will never have to add the path on the command line again either. Even better, if the path ever changes, your user can just run it with the command line path option again and voila, the old path in the config file is overwritten and the new one is saved.
I would definitely recommend the configuration file approach due to its flexibility and user-friendliness.
Related
I want to change a file to be permanently read-only and tried the solutions provided creating-read-only-pdf-file-using-python and change-file-to-read-only-mode-in-python.
However, in both cases it was still possible to edit the file and manually change it back to a read-only file.
Is there a way to prevent that, so that nobody could edit the properties or content of the file?
I thought about encrypting it, i.e. using SHA256 with a randomly created key, but that would render the file unreadable.
Is there a way to prevent that, so that nobody could edit the properties or content of the file?
No. As long as the file is on a writable device, it's always possible for a user to delete the file and replace it with a modified copy.
(And even if the file is on an immutable device, like a CD-ROM, the user can still create a modified copy of the entire device.)
If you are one a Unix-like system you can use the chmod command in the terminal.
The chmod command has an equivalent in python
You might have to run your script is super-user to change some permissions.
I have some input data that is user configurable, so i do not want to hard code it. Like the data path, result path etc.
Can you please suggest the best way to handle this data? Should i keep them in an excel or notepad and then read at run time? Or is there a better way to handle it?
Thanks
There are a lot of ways to do it.
Configuration file
You can store configuration in separate file in YAML, JSON, INI or any other format. There are a lot of tools and libraries for parsing and loading such configurations. Take a look on this article. Such approach is good for rarely changed configuration like services credentials, but it's not really good for configuration that changes very often.
Environment variables
Also, you can store configuration inside environment variables. Take a look on py-env-config. You can hard-code default configuration values but allow user to override them using environment variables.
Script arguments
If you are writing a script, you can always pass all configuration as command-line arguments/options. Manuals. Such approach is good of configs that changes very often (almost every script execution).
EDIT
I'll suggest you to use configuration file for this constants.
Using Linux, while the program itself and some other config files are located in one of the user's home directory (and run from there), because of hardware restrictions I am forced to run the main Python code prefixed by sudo. Now I want to call a function that already exists in one other Python file inside that user directory.
Simply using import functionfile does not seem to work, because (I assume) the program looks for that file in the root directory.
For "normal" tasks such as handling a config file that exists in that user directory, I can recreate the complete path by getting the user that was logged in before launching the sudo command, something like:
import os
sudo_username = os.getenv("SUDO_USER")
home_dir = "/home/" + sudo_username
However, I cannot imagine a method to import functionfile from other user directory, especially if I want to avoid absolute reference to usernames.
Is this technically possible ?
It is possible, if you import sys and add the directory to sys.path (or alternatively if you modify the PYTHONPATH), and the file is readable by root.
But it may not be a good idea. It may leave a root-owned .pyc file in that directory that the user cannot overwrite, and it is a huge security hole for a random user to be able to change code that root will be running.
I was wondering what the best practices are on working with paths in the following scenario:
I can either choose to change the current directory to the desired folder and then generate a file using only the file name, or just use the full path directly.
Here is the code where I set the current directory os.chdir():
a=time.clock()
import os
for year in range(start,end):
os.chdir("C:/CO2/%s" % year)
with open("Table.csv",'r') as file:
content=file.read()
b=time.clock()
b-a
Out[55]: 0.002037443263361638
And that is slower than when using the full path directly:
a=time.clock()
for year in range(start,end):
with open("C:/CO2/%s/Table.csv" % year,'r') as file:
content=file.read()
b=time.clock()
b-a
Out[56]: 0.0014569102613677387
I still doubt though whether using the full path is good practice. Are both the methods cross platform? Should I be using os.path instead of %s?
What's the use case for the code in question? Is it a script invoked on the command line by a user? If so, I would usually take the path as a command-line argument (sys.argv), as a command-line option (argparse), or using some sort of configuration file.
Or is the file path part of a more general-purpose module? In that case, I might think about wrapping the path and related code in a class (class FooBar). Users of the module could pass in the needed file path information when instantiating a FooBar. If users tended to use the same path over and over, I would again lean toward a strategy based on a configuration file.
Either way, the file path would be separate from the code -- at least for real software projects.
If we're talking about a one-off script with very few users and almost zero likelihood of future evolution or code re-use, it does not matter too much what you do.
As #lutz-horn said, hardcoded path isn't good idea for any code, except single-run scripts.
Talking about design, choose the methods that seem to be more explicit and simple for further development, don't optimize your code until run time becomes an issue.
In particular case, I'd prefer second way. No need to chdir until you're writing consistent files. You should use explicit chdir in case you're writing many files with different name schemas.
I have a simple web-server written using Python Twisted. Users can log in and use it to generate certain reports (pdf-format), specific to that user. The report is made by having a .tex template file where I replace certain content depending on user, including embedding user-specific graphs (.png or similar), then use the command line program pdflatex to generate the pdf.
Currently the graphs are saved in a tmp folder, and that path is then put into the .tex template before calling pdflatex. But this probably opens up a whole pile of problems when the number of users increases, so I want to use temporary files (tempfile module) instead of a real tmp folder. Is there any way I can make pdflatex see these temporary files? Or am I doing this the wrong way?
without any code it's hard to tell you how, but
Is there any way I can make pdflatex see these temporary files?
yes you can print the path to the temporary file by using a named temporary file:
>>> with tempfile.NamedTemporaryFile() as temp:
... print temp.name
...
/tmp/tmp7gjBHU
As commented you can use tempfile.NamedTemporaryFile. The problem is that this will be deleted once it is closed. That means you have to run pdflatex while the file is still referenced within python.
As an alternative way you could just save the picture with a randomly generated name. The tempfile is designed to allow you to create temporary files on various platforms in a consistent way. This is not what you need, since you'll always run the script on the same webserver I guess.
You could generate random file names using the uuid module:
import uuid
for i in xrange(3):
print(str(uuid.uuid4()))
The you save the pictures explictly using the random name and pass insert it into the tex-file.
After running pdflatex you explicitly have to delete the file, which is the drawback of that approach.