What would be a neat way to share configuration parameters\settings\constants between various projects in Python?
Using a DB seems like an overkill. Using a file raises the question of which project should host the file in its source control...
I'm open for suggestions :)
UPDATE:
For clarification - assume the various projects are deployed differently on different systems. In some cases in different directories, in other cases some of the projects are there and some are not.
I find that in many cases, using a configuration file is really worth the (minor) hassle. The builtin ConfigParser module is very handy, especially the fact that it's really easy to parse multiple files and let the module merge them together, with values in files parsed later overriding values from files parsed earlier. This allows for easy use of a global file (e.g. /etc/yoursoftware/main.ini) and a per-user file (~/.yoursoftware/main.ini).
Each of your projects would then open the config file and read values from it.
Here's a small code example:
basefile.ini:
[sect1]
value1=1
value2=2
overridingfile.ini:
[sect1]
value1=3
configread.py:
#!/usr/bin/env python
from ConfigParser import ConfigParser
config = ConfigParser()
config.read(["basefile.ini", "overridingfile.ini"])
print config.get("sect1", "value1")
print config.get("sect1", "value2")
Running this would print out:
3
2
Why don't you just have a file named constants.py and just have CONSTANT = value
Create a Python package and import it in the various projects...
Why is a database overkill? You're describing sharing data across different projects located on different physical systems with different paths to each project's directory. Oh, and sometimes the projects just aren't there. I can't imagine a better means of communicating the data. It only has to be a single table, that's hardly overkill if it provides the consistent access you need across platforms, computers, and even networks.
Related
I'm looking to store some individual settings to each user's computer. Things like preferences and a license key. From what I know, saving to the registry could be one possibility. However, that won't work on Mac.
One of the easy but not so proper techniques are just saving it to a settings.txt file and reading that on load.
Is there a proper way to save this kind of data? I'm hoping to use my wx app on Windows and Mac.
There is no proper way. Use whatever works best for your particular scenario. Some common ways for storing user data include:
Text files (e.g. Windows INI, cfg files)
binary files (sometimes compressed)
Windows registry
system environment variables
online profiles
There's nothing wrong with using text files. A lot of proper applications uses them exactly for the reason that they are easy to implement, and additionally human readable. The only thing you need to worry about is to make sure you have some form of error handling in place, in case the user decides to replace you config file content with some rubbish.
Take a look at Data Persistence on python docs. One option a you said could be persist them to a simple text file. Or you can save your data using some serialization format as pickle (see previous link) or json but it will be pretty ineficient if you have several keys and values or it will be too complex.
Also, you could save user preferences in an .ini file using python's ConfigParser module as show in this SO answer.
Finally, you can use a database like sqlite3 which is simpler to handle from your code in order to save and retrieve preferences.
I am developing a program that has a settings window in which I can change various parameters for my program. What is the best way to read/save them in some kind of config file? I know that some software and games use .ini files or similar system. How can I achieve this in Python?
The Python standard library includes the ConfigParser module, which handles ini-style configuration files for you. It's more than adequate for most uses.
Another popular option for configuration files is JSON - it's a simple notation which has good support from a wide range of languages.
Python has the json module in the standard library, which makes it very easy.
Since you introduced the term config file in your question, the previous answers concentrated on means for creating plain text files, which also could be manipulated using a standard text editor. Depending on the sort of settings to store this might not be desired, since it requires strict plausibility checks after reading back the config file at the very least. So I add the proposal of the shelves module which is a straight-forward way to make information persistent in files.
I have a project with 10 different python files. It has classes and functions - pretty much the lot.
I do want to share specific data that will represent the settings in the project between all the project files.
I came up with creating a settings.py file:
settings = {}
settings['max_bitrate'] = 160000
settings['dl_dir'] = r"C:\Downloads"
and then I import the class from every file.
Is there a more suitable way to do it?
I'm probably a little old-school in this regard, but in my latest project, I created a config file in /etc, then created a config module that uses ConfigParser to read it in and make it available, and import that config module wherever I need to read settings.
Your method sounds good to me, and has the advantage that you can easily change the implementation of the settings module, for example to use configuration files or the windows registry, or to provided a read only API.
I'm writing a script to make backups of various different files. What I'd like to do is store meta information about the backup. Currently I'm using the file name, so for example:
backups/cool_file_bkp_c20120119_104955_d20120102
Where c represents the file creation datetime, and d represents "data time" which represents what the cool_file actually contains. The reason I currently use "data time" is that a later backup may be made of the same file, in which case, I know I can safely replace the previous backup of the same "data time" without loosing any information.
It seems like an awful way to do things, but it does seem to have the benefit of being non-os dependent. Is there a better way?
FYI: I am using Python to script my backup creation, and currently need to have this working on Windows XP, 2003, and Redhat Linux.
EDIT: Solution:
From the answers below, I've inferred that metadata on files is not widely supported in a standard way. Given my goal was to tightly couple the metadata with the file, it seems that archiving the file alongside a metadata textfile is the way to go.
I'd take one of two approaches there:
create a stand alone file, on the backub dir, that would contain the desired metadata - this could be somethnng in human readable form, just to make life easier, such as a json data structure, or "ini" like file.
The other one is to archive the copied files - possibily using "zip", and bundle along with it a textual file with the desired meta-data.
The idea of creating zip archives to group files that you want together is used in several places, like in java .jar files, Open Document Format (offfice files created by several office sutres), Office Open XML (Microsoft specific offic files), and even Python language own eggs.
The ziplib module in Python's standard library has all the toools necessary to acomplish this - you can just use a dictionary's representation in a file bundled with the original one to have as much metadata as you need.
In any of these approaches you will also need a helper script to letyou see and filter the metadata on the files, of course.
Different file systems (not different operating systems) have different capabilities for storing metadata. NTFS has plenty of possibilities, while FAT is very limited, and ext* are somewhere in between. None of widespread (subjective term, yes) filesystems support custom tags which you could use. Consequently there exists no standard way to work with such tags.
On Windows there was an attempt to introduce Extended Attributes, but these were implemented in such a tricky way that were almost unusable.
So putting whatever you can into the filename remains the only working approach. Remember that filesystems have limitations on file name and file path length, and with this approach you can exceed the limit, so be careful.
I want to use Python to make a configuration file generator. My roughly idea is feeding input with template files and some XML files with the real settings. Then use the program to generate the real configuration files.
Example:
[template file]
server_IP = %serverip%
server_name = %servername%
[XML file]
<serverinfo>
<server ip="x.x.x.x" name="host1" />
<server ip="x.x.x.x" name="host2" />
</serverinfo>
and then get output configuration file like this
[server.ini]
[server1]
server_IP = x.x.x.x
server_name = host1
[server2]
server_IP = x.x.x.x
server_name = host2
I got several questions:
Is there any open source configuration generator program? (what could be the keyword), I wonder if there's anything can be added/modified in the design.
Does Python have good XML parser module?
Is it good idea to use XML file to save the original settings? I've been thinking to use Excel since it's more intuitive to maintain, but harder for program to parse. Not sure how people deal with this.
Hope the community can give me some suggestions. Thanks a lot!
EDIT:
In scenario that there are dozens of these output ini files. I am concerning 2 things.
there are dozens of ip/hostname and related informations, that may requires to be managed by human, so XML format would be a bit inconvenient. What could be the most convenient structure to manage those information? (Excel would be a handy tool to batch modify and look up info)
In case of need to add some extra line into ini files, I need a efficient way by just modify the template file and add extra info into the source file (may be the Excel file or whatever), then whole bunches of ini files can be generated quickly.
I recommend using excellent ConfigObj library by Michael Foord. It can read/write configuration files, even with nested sections.
I don't know if there are any open source configuration generators.
Python has several xml parser modules, the newest (and perhaps most pythonic) of which is ElementTree. You can find additional documentation for it at the developer's site.
I recommend avoiding xml in your configuration files if possible. A simple flat file full of name/value pairs is much easier for humans to work with. If you need a level or two of structure, ini-style files ought to do nicely, and python comes with a built-in parser for them.
It's terrific to use xml for any configuration file. Best choice in any interpreted language is to use code file and simply exec or import it. Pickle is also good, but not human readable.
I've used the minidom module. That would probably work for you.
You need some template engine, look at string.Template