I'm working on a Flask extension from which I want to create a directory in the project's root path on the file system.
Suppose we have this directory structure
/project
/app
/tests
/my_folder
manage.py
my_folder should be created dynamically by the extension, which is a test utility and wraps the application under test in the /tests directory. However, I'm struggling to determine the project's root path within my extension.
For now, I am trying to guess the path from the run file:
def root_path(self):
# Infer the root path from the run file in the project root (e.g. manage.py)
fn = getattr(sys.modules['__main__'], '__file__')
root_path = os.path.abspath(os.path.dirname(fn))
return root_path
This obviously breaks as soon as the tests are run from within the IDE instead of the manage.py. I could simply infer the project's root relative to the app or tests directory, but I don't want to make any assumptions regarding the name or structure of these directories (since multiple apps might be hosted as subpackages in a single package).
I was wondering if there is a best practice for this type of problem or an undocumented method which the Flask object provides (such as get_root_path).
app.root_path contains the root path for the application. This is determined based on the name passed to Flask. Typically, you should use the instance path (app.instance_path) not the root path, as the instance path will not be within the package code.
filename = os.path.join(app.instance_path, 'my_folder', 'my_file.txt')
app.root_path is the absolute path to the root directory containing your app code.
app.instance_path is the absolute path to the instance folder. os.path.dirname(app.instance_path) is the directory above the instance folder. During development, this is next to or the same as the root path, depending on your project layout.
Related
Here's what I have:
Project A
----data_I_need.p
Project B
----use_data.py
If data_I_need.p were a source file (and in my path), I could easily import it into Project B. Does such method exist for non-source files?
Right now, I'm having to accomplish this with
with open('C:/......./data_I_need.p', 'rb') as data:
Project A is in my path (defined as 'Content Root' in PyCharm). I can imput source files from Project A.
To recap, the problem is that any Python file in PYTHONPATH can be imported, but resources (non-python files) cannot be loaded without an explicit path.
Project_A
----some_module.py
----data_I_need.p
Project_B
----use_data.py
Here is a solution. In Project B/use_data.py, include the following.
import os
import pathlib
import pickle
# full path to Project_B
here = os.path.dirname(__file__)
# path to folder *above* Project_B ---- joined to Project_A
PROJECT_A = os.path.join(*pathlib.Path(here).parts[:-1], 'Project A')
with open(os.path.join(PROJECT_A, 'data_i_need.p')) as resource:
var = pickle.load resource
This will work even if you're working in a shared drive (e.g., OneDrive) from two different operating systems.
Optionally, you could just set up a "shared_resources" folder and include a python script in that folder to access the resources, again working from os.path.dirname(__file__) then navigating the relative path.
I want to create a multi-serivce app engine app, using the first diagram shown on this page.
https://cloud.google.com/appengine/docs/standard/python/configuration-files#an_example
I want to use third party libraries so I used have a lib folder under the root directory, one yaml under the root directory.
Then I want one microservice called predict. So I created a folder called predict under the root directory as well, then under this folder, I want to write py files using the packages in the lib as well.
What I'm doing is like this:
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
vendor.add(os.path.join(parentdir, 'lib'))
Which didn't work for me as the error says:
ValueError: virtualenv: cannot access /base/data/home/apps/f~project-name/lib: No such virtualenv or site directory
Obviously it didn't work for me, what's the right way to do this?
Update: Dan's solution worked for me! So when I'm deploying my microservice predict, I need to go inside predict directory and deploy it. I think that's why it can't find lib. By symlink the lib library using bash.
ln -s ../lib/ ./lib
I solved this problem.
The problem you're facing is that each service can only access files in its own service directory, which is the directory where its app.yaml file is located, they can't access the app's root directory.
My approach (all my services use standard environment):
created a lib dir in the top app dir and installed in it each package that I want shared across multiple services
each service has its own subdir in the app's dir and a lib directory in it where:
I installed libraries needed only by that service (if any)
I symlinked the needed shared libraries from the top lib dir
This is my app dir structure:
app_dir/
app_dir/dispatch.yaml
app_dir/cron.yaml
app_dir/index.yaml
app_dir/queue.yaml
app_dir/lib/
app_dir/lib/shared_lib_1/
app_dir/lib/shared_lib_2/
app_dir/lib/shared_lib_3/
app_dir/service1/
app_dir/service1/app.yaml
app_dir/service1/lib/shared_lib_1 -> ../../lib/shared_lib_1
app_dir/service1/lib/shared_lib_2 -> ../../lib/shared_lib_2
app_dir/service1/lib/service1_lib_1
app_dir/service2/
app_dir/service2/app.yaml
app_dir/service2/lib/shared_lib_2 -> ../../lib/shared_lib_2
app_dir/service2/lib/shared_lib_3 -> ../../lib/shared_lib_3
app_dir/service2/lib/service2_lib_1
No fumbling with the lib path is required, all I have in each service is
vendor.add('lib')
See related:
How do I access a vendored library from a module in Python Google App Engine?
Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
Check all the directories in the path and the final file to make sure they all exist first before code runs. If they don't, prepend the code you showed here with code to create them...
I've pulled my django repo from bitbucket onto my digital ocean server. My project folder name in my server is project whereas my initial app from my repo (the one with settings.py is called app. Does this cause any problems? Because I know when you create a django project offline, the first directory is always the same as the parent directory.
whereas my initial app from my repo (the one with settings.py
That's not an application, that's your project's configuration.
And no, having the same name for your project's root directory and it's configuration directory shouldn't cause any problem.
That's even how it's designed to be so it's perfectly normal.
Since the wsgi.py (which should be what you use for production) dynamically handles path as long as the internal structure of your project is preserved (don't rename the project's configuration directory).
The absolute path to your project isn't hardcoded, it's retrieved by getting parent directories of your configuration directory. You can move your project and even rename its root directory.
Here is how it's handled :
wsgi.py :
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "testproject.settings")
wsgi.py sets the right value to use the right settings.py file.
settings.py :
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
settings.py sets BASE_DIR dynamically. This value is then what's used for handling pathes to avoid having an hardcoded absolute path that wouldn't work as soon as you move your project.
There won't be any issue with having the different name of parent directory and the project directory (the one with settings.py).
You can try renaming the parent directory and it would still work the same way but the name of project directory should not be touched since the same is linked with project settings which are used to run the project.
I have the following folder structure
app/
app/helpers/
app/helpers/methodhelper.py
app/methods/
app/methods/method.py
and I'm trying to import a function from methodhelper.py inside method.py
so I tried the following:
import app.helpers.methodhelper
OR
from app.helpers.methodhelper import function1
OR
import helpers.methodhelper
and I get:
"No module named app.helpers.methodhelper"
Important to note: helpers/__init__.py already exists
How should this be done ?
Your Django project's default path is in the root directory of the project (where the manage.py file is). You can either add the sub directories below that to your PYTHONPATH (easily done by appending to sys.path) or you can import that function using the full module path:
from projectname.app.helpers.methodhelper import function1
When I start a Django project, I always add
PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
to my settings.py. This path looks similar to /home/kyle/django_project_name/. Inside that directly is manage.py.
From there, also in my settings.py, I include:
sys.path.append(os.path.join(PROJECT_ROOT, 'django_project_name'))
This makes my apps importable without the need to include my project name in the module path.
you need to add the module in settings.py file
I'm trying to write a test for my python program, and I've created a /tests directory, where / means the projects root, not system root.
I have all my source files in /myProjectName.
Both directories have a __init__.py file, but project root does not, and both files are empty (do I need a __init__.py in the tests directory ?)
I've tried importing /myProjectName/main.py in /tests/test_main.py, but it doesn't work.
What is the right way to either structure the project directories or import main.py in test_main.py?
I would suggest moving your tests directory to be inside your project directory. Then you can use the answers here to import from the parent directory when you're in tests: Importing modules from parent folder
Otherwise, you can simply set your $PYTHONPATH to point to your project directory when you run tests.
I would like to suggest nose here as it automatically detects the tests and runs them for you
To run the tests, simply do a
$ nosetests
in your project directory
Why not try this line in testmain.py:
import os
os.chdir("/myProjectName/main.py ")
Then execute the script.
Hope it works!