I have imported a python module from another directory in Linux by using sys.path.insert but when I run that module it fails. As the imported module is trying to open a file that is local to that module. Please let me know how to fix this.
users/adr/release/invoke.py #this one fails with can’t open file ./config.ini
import sys
sys.path.insert(0, ‘/user/cdw/audit/’)
import audit_main
The path where the original module file resides.
/user/cdw/audit/audit_main.py # this module opens config.ini file to read pplication credentials.
/user/cdw/audit/config.ini
/user/cdw/audit/__ini__.py
When python loads a module it adds metadata to its global namespace that can be used by the module. __file__ is the filename of its .py file - when such a name makes sense. You can use it to get your config file
import os
config_filename = os.path.join(os.path.dirname(__file__), "config.ini")
except for when you can't. If this is a C extension file or if the file was in some other packaging like zip or egg or pyinstaller bundle or exe, it may not work. A ini file, which I assume is user editable, may need to be some place other than the module directory. If you make this .py part of an installable package, it may end up in a system directory the user can't edit. This may not be a problem for this project, just laying out some of the considerations.
Related
I have a custom module that I am trying to read from a folder under a hierarchy:
> project-source
/tests
/provider
my_provider.py
settings_mock.py
__init__.py
I am trying to call, from my_provider.py
import tests.settings_mock as settings
Example from command line:
project-source> python tests/provider/my_provider.py
Error:
... ImportError: No module named settings_mock
I keep getting No module named settings_mock as error. I have already exported project_source path to PYTHONPATH. I have made tests into a package by creating a __init__.py file in its root, but no change in the error then.
I can print the settings_mock.py attributes when cd'ing project source
>>> import tests.settings_mock as settings
>>> print settings.storage_provider
correct storage provider value
Is anyone able to point out my mistake here? Thanks!
You only have one small mistake. To use subfolders, you need __init__.py, not init.py as you stated in the question. The difference is that __init__ is a builtin function of python, whereas init is not. Having this file in each subfolder tells the pyhon interpreter that the folder is a "package" that needs to be initialized.
UPDATED: It should be noted that python usually runs from the current directory that the script is located. If your executable main script is my_provider.py, then it's not going to know what to import, since the main script is located in a lower directory than the object it is trying to import. Think of it as a hierarchy. Scripts can only import things that are beneath them. Try separating out the executable from everything else in that file, if there are things that settings_mock needs to import.
I'm totally new to Python, and I want to create my first Python library for peronal uses.
I'm using Python 2.7.5, and running the IDLE interface.
So far, what I understood from the documentation and related questions is that:
Python goes through a list of directories listed in sys.path to find scripts and libraries
The package directory must contain a __init__.py file, which can be empty
The module I want to create should be a modulename.py file with the code inside the package directory
(Sources: http://www.johnny-lin.com/cdat_tips/tips_pylang/path.html --- https://docs.python.org/2/tutorial/modules.html)
And here is what I tried that fails:
Created a personal package directory C:\....\pythonlibs
Created a subpackage dir C:\....\pythonlibs\package
Created the __init__.py file inside both folders
Created the mymodule.py file in the packacge dir
And then in the IDLE used this code:
import sys
sys.path.append(r'C:\....\pythonlibs')
First issue:
Currently I have to do this append every time I enter the IDLE. How can I keep the directory in sys.path permanently just as there are a lot of other directories there?
Then I tried importing my package:
import pythonlibs #fails!! why?
import pythonlibs.package #fails!! why?
import package #works
The error is: ImportError: No module named pythonlibs
Second issue?
This seems to be against the documentation, why can't I import from the root pythonlibs folder?
With line
sys.path.append(r'C:\....\pythonlibs')
you are instructing interpreter to start looking for modules (libraries) in this directory. Since this directory does not contain pythonlibs folder (the parent does), it can't import it.
Similarly - because it contains the module package, it can import it.
I busy with some calculations in Python and therefore i have some bunch of scripts.
I have tried to clean this up thru 2 folders named scripts and tests.
Now i have the problem that my main Python file don't recognize the scripts in the subfolders.
So my import filename don't work anny more. When i look in some git files it looks like the don mention paths and still it works.
I had looked at this SE question but that gave me a error (ImportError: No module named "filename")
What have i to do in my main script, subfolder or files in subfolders.
my scripts are no classes yet... Probably not all become classes. so a generic solution is best
You can do relative imports from where you are. Let's assume you're importing from the file /home/janbert/projects/test/test.py
If you want to import /home/janbert/projects/test/subdir/file.py you write:
from subdir import file
And if you want to import /home/janbert/projects/otherproject/subdir/file.py you write:
from ..otherproject.subdir import file
Just remember that each python package (ie folder) must have a file named __init__.py (which can be empty), otherwise you can not import from that package.
I am writing a simple scheduling service. I don't want to hard-code all of the tasks it can schedule and instead would like to support plugins that can be dropped in a folder and loaded dynamically at runtime.
My plan is to have a JSON file (or any configuration file) that maps a task name to the location of a Python file (a module) which will have a class called Plugin. Pretty simple I thought. When someone schedules a task to run, they pass the task name and the time to run it. When the time elapses, the plugin is loaded (or reloaded) and is ran with any additional arguments passed to the scheduler.
I have been looking at the imp module to see how to load modules at runtime. I am not sure whether I want to list plugins using their physical location (file system path) or to use their module names like you'd see in a import statement. It seems imp wants to use physical location.
I got two different versions of this code "working". Here is one that uses importlib:
pluginName = self.__pluginLookup[pluginName]
module = import_module(pluginName)
module = reload(module) # force reload
plugin = module.Plugin()
return plugin
This is one I wrote using imp:
path = self.__pluginLookup[pluginName]
path, moduleName = split(path)
moduleName, extension = splitext(moduleName)
file, path, description = find_module(moduleName, [path])
with file:
module = load_module(moduleName, file, path, description)
plugin = module.Plugin()
return plugin
The problem I am running into is handling dependencies. If I have a plugin.py file that depends on a dependency.py file in the same folder, saying import dependency doesn't seem to work. Instead, it looks up the dependency from the PYTHONPATH.
How can I make the imports relative to the plugins themselves?
You could append path to sys.path:
import sys
sys.path.append(path)
where path is the directory containing the dependency.py.
If you have a plugins directory with an __init__.py, you can add that directory to sys.path. Then modules inside there can do from . import dependency to import another plugin. Or, if the plugin is itself a subpackage (i.e., a directory with its own __init__.py) then it can do from . import dep to import a dependency within the same plugin, or from .. import dep to import a dependency from the global plugins directory. With this setup you don't even need to use imp or the like; you can use the __import__ function, which works by module name.
One possible wrinkle, though, is you say the plugins directory will be "under the current working directory". What does that mean exactly? You mean you want people to be able to add plugins anywhere on the drive and still use them? It would be better to have one central plugins directory for your app, and add that to sys.path, and tell people to put their plugins there.
i created an executable egg to make it as an single file executable.
setup.py:
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(
name='app',
version='0.5',
description='foo',
author='microo8',
author_email='xxx#email.com',
packages=["foo", "bar"],
install_requires=["sqlalchemy>=0.7", "paramiko>=1.7.7.1"],
entry_points = {
'setuptools.installation': [
'eggsecutable = foo.module:main',
]
}
)
I can now call it: ./app-0.5-py2.7.egg, but the relative paths are all in the egg.
when I call print __file__ in the main function it prints /home/user/app-0.5-py2.7.egg/foo/module.py. I want to read an config file that must be in the same dir as the egg. And the same script must be executable also as "non-egg", so the config file will be in the dir with the script.
how can i find out that the script is executed from an egg and also the egg path?
So I dont know how to check that the running script is in an egg, but the config file path can be obtained with this: os.path.realpath('config.cfg').
This suits me and when I have a dir with the executable egg and an config file this is a correct path.
Also when I have just the script in the dir with the config file it also is correct.
And also when i use pyinstaller to create an singlefile exe.
You should load your configuration file using the ResourceManager API. That way your eggified code can load the configuration file even when zipped.
import pkg_resources
configdata = pkg_resources.resource_string(__name__, "myconfigfile.cfg")
The API can check for existence, test for directories, read data or provide you with an open file object for any resource that is part of your egg.
The API takes care of all the details; it'll extract files to a temporary directory for example, if you absolutely must have a filename for a resource but are running from a zipped egg.
Most of all, it'll work regardless of whether or not you are in fact installed as an egg or stand-alone.