Using the ConfigParser module I am attempting to read in a configuration file so I can safely allow users to edit settings for an application and share these configurations between scripts.
This has been working perfectly for the main script. However there is a secondary script that is called and it reads the same configuration file in the same location but returns an error that the location cannot be found.
Both scripts are located within the same directory, application/bin/
The configuration file is located in application/conf/ To reference the config file successfully in the main script I use the following code which works perfectly.
config = ConfigParser.ConfigParser()
config.readfp(open('../conf/settings.conf'))
When the secondary script executes with the same code it reports that the location does not exist? I used the logger module and got it to log sys.path[0] which correctly returned the same bin folder as the main script. Is there possibly something simple I am missing here?
Also any troubleshooting tips for problems like these are welcome.
You can prefer to use dirname and __file__:
from os.path import dirname, join
config = ConfigParser.ConfigParser()
config_fn = join(dirname(__file__), '..', 'conf', 'settings.conf')
config.read(config_fn)
Depending how you're launching the app (python bin/app.py, or python app.py), the '..' will be incorrect. By starting from the directory of the .py file, you'll always be able to construct the path from the .py to the .conf using that method.
Related
I know there are a lot of answers on this subject, but no one works once you compile a script in an executable.
In my python script, I create a file within the same directory of the script.
to get the path of the current dir I use pathlib
basepath = Path(__file__).parent
filename='myfile'
filepath=os.path.join(basepath, filename)
if I print the directory I get the file wrote in the good directory and everything works fine within python
(i.e desktop/myname/myscriptdir/myfile)
but once I "compile" with pyinstaller with --onefile, if I launch the executable, the directory will be
like
/var/folders/nr/w0698dl96j39_fq33lqd8pk80000gn/T/_MEIP12KxC/myfile
believed me, I tried a lot of various method (abspath, os.realpath..)to get the current dir, no one worked fine once in an executable file.
When you compile an app using pyinstaller with the --onefile or -F flag, the file that it creates is actually an archive file, like a .zip file.
When you execute that file it launches a process that extracts itself into a temporary folder somewhere in your OS filesystem. This is the path that is reported when you use the __file__ variable in the compiled application.
It then continues to launch your application from there and the temporary directory becomes the runtime durectory for the duration of the apps life. When the app is finally closed it deletes the temporary runtime directory on it's way out.
Since this is the case there are alternatives.
To get the current working directory during runtime use:
path = '.'
#or
path = os.getcwd()
To get the path to the compiled executable file during runtime:
path = sys.executable
I have imported a python module from another directory in Linux by using sys.path.insert but when I run that module it fails. As the imported module is trying to open a file that is local to that module. Please let me know how to fix this.
users/adr/release/invoke.py #this one fails with can’t open file ./config.ini
import sys
sys.path.insert(0, ‘/user/cdw/audit/’)
import audit_main
The path where the original module file resides.
/user/cdw/audit/audit_main.py # this module opens config.ini file to read pplication credentials.
/user/cdw/audit/config.ini
/user/cdw/audit/__ini__.py
When python loads a module it adds metadata to its global namespace that can be used by the module. __file__ is the filename of its .py file - when such a name makes sense. You can use it to get your config file
import os
config_filename = os.path.join(os.path.dirname(__file__), "config.ini")
except for when you can't. If this is a C extension file or if the file was in some other packaging like zip or egg or pyinstaller bundle or exe, it may not work. A ini file, which I assume is user editable, may need to be some place other than the module directory. If you make this .py part of an installable package, it may end up in a system directory the user can't edit. This may not be a problem for this project, just laying out some of the considerations.
My tests are at:
src/com/xyz/tests/api_test/<Test File>
My test file calls my libraries at:
src/com/xyz/libs/api_libs/<Library File>
My library file has to open a JSON file at:
src/com/xyz/libs/api_libs/configs/<Config File>
In my library file, since its at the same parent directory as the JSON configs, I have used the following code to open the JSON.
with open('configs/sample_wlan_json'):
<Do Some action>
I tried various paths like:
.../libs/api_libs/configs/<ConfigFileName>
src/com/mist/libs/api_libs/configs/<ConfigFileName>.json
The whole path from /Users/...... but nothing seems to work.
A relative path is relative to the current working directory. Current working directory depends on how an application is started, and not where it is.
So, if you want to have a path relative to your source code, you should not rely on the current working directory, but construct the absolute path instead.
You can construct a path which is relative to your source code by using the __file__ variable, which is the path to the current py file.
Something like this should work:
configs_dir = os.path.join(__file__, '..', 'configs')
with open(os.path.join(configs_dir, 'sample_wlan.json'), 'rt') as f:
...
A script at /foo/bar.py tries to run a second script /main.py using the subprocess module. Although main.py runs fine with python main.py in the Windows command prompt, running bar.py which calls main.py causes an error
ConfigParser.NoSectionError: No section: 'user'
Why is there now a problem with the path to settings.ini, and how can we fix it?
~/settings.ini
[user]
id: helloworld
~/foo/bar.py
subprocess.Popen([sys.executable, "../main.py"])
~/main.py
Config = ConfigParser.ConfigParser()
Config.read("settings.ini")
userId = Config.get('user', 'id')
If settings.ini is presumed to be in the same directory as main.py you can deduce its full path from __file__ and read the settings using the full path.
main.py:
import os
ini_path = os.path.join(os.path.dirname(__file__), "settings.ini")
Config = ConfigParser.ConfigParser()
Config.read(ini_path)
Check that the read() method returns a non-empty list otherwise it means that settings.ini is not found. Relative paths are resolved relative to the current working directory (place where you've run python), not script's directory (where bar.py is stored).
You should probably use appdirs.user_config_dir(), to get the directory where to put user's configuration files. Different OSes such as Windows, macOS, Linux distributions may use different defaults. appdirs follows conventions that are appropriate for a given OS.
If you want to get data stored in a file that is packaged with your installed Python modules then you could use pkgutil.get_data() or setuptools' pkg_resources.resource_string() that work even if your package is inside an archive. It is not recommended to construct the paths manually but if you need the directory with the current script then you could put get_script_dir() function into your script—it is more general than os.path.dirname(os.path.abspath(__file__)). If the relative position is always the same then you could use:
#!/usr/bin/env python
import os
import subprocess
import sys
main_script_dir = os.path.join(get_script_dir(), os.pardir)
subprocess.check_call([sys.executable, '-m', 'main'], cwd=main_script_dir)
See Python : How to access file from different directory
In my home directory, I have a folder called local. In it, there are files called __init__.py and local_settings.py. My django app is in a completely different directory. When the app is NOT running in DEBUG mode, I want it to load the local_settings.py file. How can this be acheived? I read the below:
Import a module from a relative path
Importing files from different folder in Python
http://docs.python.org/tutorial/modules.html
Basically, those tutorials are allowing to import from another directory, but what about a completely different working tree? I don't want to keep doing .., .., .. etc. Is there a way to goto the home directory?
I tried the following code:
import os, sys
os.chdir(os.path.join(os.getenv("HOME"), 'local'))
from local_settings import *
But i keep seeing errors in my apache error.log for it...
os.chdir just affects the current working directory, which has nothing whatsoever to do with where Python imports modules from.
What you need to do is to add the the local directory to the Pythonpath. You can either do this from the shell by modifying PYTHONPATH, or from inside Python by modifying sys.path:
import sys
import os
sys.path.append(os.path.expanduser("~/local"))
import local_settings
In response to your concerns about source control, you can just set the source control to ignore that file, and/or have a symlink installed as part of your deploy script to link the file on the os into another. I do both , though not in django. but it's a trivial task.
your deploy layout could look like this:
/current ( symlink to /releases/v3 )
/settings/local_settings.py
/releases/v1
/releases/v2
/releases/v3
and a task runs as part of your deploy:
cd /current
ln -s /settings/local_settings.py local_settings.py
if you're deploying with fab or capistrano, it's a few lines of configuration. i'm sure you could do this with puppet/chef simply too.