How to load config file in python from an external module - python

I have a config module (myConfig.py) present in a library for which i created a standard distribution package using setuptools.
--configPackage
|
| ---- myConfig.py
| ---- __init__.py
myConfig.py is a key value pair like this:
MY_NAME = 'myname'
MY_AGE = '99'
Now I have another python project where I import this config module like this
import configPackage.myConfig as customConfig
If this config file was native to my python project and had not come from an external project then I would have done something like this:
app = Flask(__name__)
app.config.from_object('app.configPackage.config')
where config is actually config.py file under configPackage.
and any key value pair in config.py could then be accessed as
myName = app.config['MY_NAME']
My problem is that I am not able to load the external config file in the above mentioned way for a native config file. What I have tried is this which doesn't work:
import configPackage.myConfig as customConfig
app = Flask(__name__)
app.config.from_object(customConfig)
myName = app.config['MY_NAME']
I get the following error
model_name = app.config['model_name']
KeyError: 'model_name'
Which means that it is not able to load the config file from external module correctly. Can anyone tell me the right way to accomplish this ?

Try this,
in your __init__.py of configPackage import myConfig as shown below
import configPackage.myConfig as myConfig
And in your other app try
app.config.from_object('configPackage.myConfig')

Related

Import variables from a config File

I have a script that have multiple files and a config file with all the variables store in one Python file.
Folder structure:
Config file:
If I try to run the main file which calls the head function imported, an error pops up saying that the config cannot be imported.
Imports:
Your Functions folder has a __init__.py file. If your app executes from Main.py (ie if Main.py satisfies __name__ == "__main__") therefore wherever you are in your app you could import the config like this:
from Functions.Config import *
Edit:
However,from module import * is not recommended with local import. You could give an alias to the import and call name.variable instead of calling name directly.
Example:
def head():
# import packages
import Function.Config as conf
print(conf.more_200)
head()
>>> 50
Your syntax are wrong.
It is supposed to be from Config import * and not import .Config import *

NameError: name xxxxxx is not defined in flask

Am having an Import Error issue in a flask restful application, below is the project structure.
project/
app/
views/
tasks.py
flask_celery.py
run.py
So as you have seen above that's the project structure, so my challenge is am importing a variable from run.py to tasks.py. But when I run the application I get an error NameError: name 'celery_access' is not defined
So this is my run.py file :
from app.starwars.star_wars import StarWars
from app.utils.flask_celery import make_celery
app = Flask(__name__)
# configure celery
app.config.update(
CELERY_BROKER_URL='redis://redis:6379/0',
CELERY_RESULT_BACKEND='redis://redis:6379/0'
)
celery_access = make_celery(app)
# secret key
app.secret_key = "xxxx"
api = Api(app)
Then, in the tasks.py file it's where I access the variable name celery_access as below :
from run import celery_access
#celery_access.task()
def paginate_requested_data():
return Hello World''
How can I resolve the import ?
Functions from .py file (can (of course) be in different directory) can be simply imported by writing directories first and then the file name without .py extension:
from directory_name.file_name import function_name
And later be used: function_name()

Application says module not found after moving to directory

I'm rather new to python and have this odd issue, which I can't seem to find an answer for.
When both app.py and mod_db were in the root directory, it works but when I shifted them to a sub directory
My directory structure:
demo_api
|
|-- demo-api
|
|-- __init__.py
|-- app.py
|-- mod_db.py
My main module
import json
from flask import Flask, request, Response
from .db_mod import insert_and_score
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
#app.route('/emotional_scoring', methods=['POST'])
def get_scoring():
json_obj = request.json
ret_json = insert_and_score(json_obj)
resp = Response(json.dumps(ret_json), mimetype='application/json', status=200)
return resp
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=False)
The error message is on this line
from .mod_db import insert_and_score
I've tried change the sub-directory name. I've tried doing a full path, i.e. from demo_app.mod_db import insert_and_score and the error is ModuleNotFoundError: No module named 'demo_app'
The issue is that it works find in my IDE (PyCharm) but when I do it on command line, these are the errors I encounter.
As stated by mfrackowiak, I just had to change it to
from db_mod import insert_and_score
and it worked. But PyCharm doesn't like this. So I think it might be a PyCharm issue.
Fix by removing relative import
It seems you want both the relative imports and Pycharm to be happy. Like mfrackowiak said, you want to use absolute importing.
from db_mod import insert_and_score
Tell PyCharm what the new "Sources Root" is
Now to make Pycharm happy, you will want to tell it that the demo_api subdirectory is a "Sources Root". You can do this by right clicking the directory and go to Mark Directory as > Sources Root. You can also find it in Preferences > Project Structure. You can do this for each subdirectory, as needed.
Why this is good to do
This is useful when you have a python project as a subdirectory of a repo with many configs, scripts, and other files and directories. You often don't want the python app to be mixed in with those, so you move it to an app/ folder. This confuses Pycharm, as by default, it uses the Content Root as the Sources Root as well. You can fix this by simply telling Pycharm explicitly what the Sources Root is.
Example:
my-awesome-project/ <---- Content Root
|--.venv/
| |--<venv stuff>
|--scripts/
| |--build.sh
| |--run.sh
|--docker/
| |--dev.Dockerfile
| |--prod.Dockerfile
|--app/ <---------------- Sources Root
| |--sub-mod1/
| | |--foo.py
| | |--bar.py
| |--sub-mod2/
| |--baz.py
|--.gitignore
|--.python-version
|--requirements.txt
|--dev.env
|--docker-compose.yml

Writing doctests for pyramid web app which depend on settings in ini file

I would like to write doctests for my pyramid web app, using the webtest module. I tried it like this:
from my_webapp import main
from webtest import TestApp
app = TestApp(main({}))
result = app.get('/')
This raises a KeyError (because some.url is not known) when my code reaches this line:
url = request.registry.settings['some.url']
The value of some.url is specified in the paster ini file of my application. Is there a simple way to use my development.ini when running my test code? I did not yet fully understand how/when the ini file is loaded during pyramid start up, so it's hard to figure out where to load it while testing.
main is invoked with the contents of your ini file. A simple way to load your app from an ini is:
from pyramid.paster import get_app
app = get_app('testing.ini#main')
test_app = TestApp(app)
This expects "testing.ini" to be in the current working directory, so you may need to tweak that. If you'd like it to be relative to a spot in your tree you can use:
import os.path
import some_module
here = os.path.dirname(some_module.__file__)
app = get_app(os.path.join(here, 'testing.ini'))

Django AttributeError in settings.py file

I've been stuck all day on what seems to be a very silly import problem. From my Django project directory, I can import a module and run a function just fime:
(msg-gw)slashingweapon:~/msg-gw/www$ python
>>> import snpp
>>> snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
{'host': 'server.name.com', 'pass': 'whatever', 'port': 1234, 'user': 'me'}
But when I try to run my app, either through manage.py or by gunicorn, I get an attribute error:
(msg-gw)slashingweapon:~/msg-gw/www$ python manage.py runserver 8000
File "/home/slashingweapon/msg-gw/www/project/settings.py", line 26, in <module>
SNPP = snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
AttributeError: 'module' object has no attribute 'config_url'
The two relevant lines in my settings.py file are exactly what you would expect. Notice that I can import the module just fine, but the config_url() function isn't found.
import snpp
SNPP = snpp.config_url('snpp://server.name.com:1234?user=me&pass=whatever')
The directory layout is exactly what you would expect:
www
|
+-project
| +-__init__.py
| +-settings.py
| +-urls.py
| +-views.py
| +-wsgi.py
|
+-snpp
+-__init__.py
+-protocol.py
+-views.py
+-urls.py
The config_url() function is defined inside snpp/__init__.py
I have tried all kinds of things:
from snpp import config_url
move config_url to the file snpp/config and then import with
import snpp.confg
from snpp.config import config_url
from snpp import config and then invoke through config.config_url()
The __init__.py file is nothing special. It just lets you encode some server information as a string, so you can stick your SNPP config into the environment:
import urlparse
def config_url(urlStr):
config = {
'host':None,
'port':444,
'user':None,
'pass':None,
}
url = urlparse.urlparse(urlStr)
if url.scheme == 'snpp':
locParts = url.netloc.split(":")
config['host'] = locParts[0]
if len(locParts) > 1:
port = int(locParts[1])
if port > 0:
config['port'] = port
args = urlparse.parse_qs(url.query)
config['user'] = args.get('user', [None])[0]
config['pass'] = args.get('pass', [None])[0]
return config
I am using Python 2.7, django 1.5.1, and virtualenv.
Other parts of my project work well. When I print out the path in my browser, it looks correct. Importing snpp should not be a problem, since snpp is in the www directory:
/home/slashingweapon/msg-gw/www
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg
/home/slashingweapon/msg-gw/lib/python2.7/site-packages/django_json_rpc-0.6.2-py2.7.egg
/home/slashingweapon/msg-gw/lib/python27.zip
/home/slashingweapon/msg-gw/lib/python2.7
... etc ...
It doesn't matter if the snpp module is in my INSTALLED_APPS list or not. I get the same result.
Solved
With the help of SO denizens, I found the problem.
I had refactored my application by moving some reusable code pieces from the project directory to the new snpp directory. When I did that, I neglected to move or delete the *.pyc files.
The value of snpp.__file__ was:
/home/slashingweapon/msg-gw/www/project/snpp.pyc
instead of the expected:
/home/slashingweapon/msg-gw/www/snpp/__init__.pyc
During the import process, Python was looking in project/ before snpp/ and finding an old snpp.pyc file. It would import the old pyc file and be satisfied, thus ignoring the entire snpp/ dir.
Had I been a little sharper (or a little more experienced with Python) I might have noticed that I was getting some strange import behavior in general whenever I tried to import anything from snpp/. It should have occurred to me that the whole module was wonky, and not just the one function I was trying to use at the moment.
Check what exactly is being imported by using
snpp.__file__
right after import snpp statement.
Actually import might not be from the path you are expecting to see.

Categories