We Keep Coding

I need help importing modules into tests from sibling folder

I get an import error "unable to import module" when I try to import a module
File structure
src
modules
__init__.py
authenticate_developer
twttier_importer
notebooks
main.ipynb
unit_tests
test_authenticate_developer
In test_authenticate_developer
import requests
from nose.tools import assert_true
import os
import sys
sys.path.append(os.path.abspath('../modules'))
import settings
import twitter_importer #returns import error
import authenticate_developer #returns import error
However when I use the same syntax in my notebooks it is successful.
import sys
import os
sys.path.append(os.path.abspath('../modules'))
sys.path.append(os.path.abspath('../'))
import twitter_importer
import authenticate_developer
import settings
I have looked at existing answers and I tried them out i.e., removing init.py from the root folder and removing or adding init.py to the tests folder. None seems to work for me.

I managed to get someone to help me with these and this is how we tackled it.
src
modules
__init__.py
authenticate_developer
twttier_importer
notebooks
main.ipynb
unit_tests
__init__.py
test_authenticate_developer
In test_authenticate_developer
import os,sys,inspect
sys.path.append(os.path.abspath('../modules'))
#This is to access the parent folder in order to import config
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import config
from modules.authenticate_developer import Authenticate
I cannot explain why I have to do it differently in the test files but this is my work around

Python can't find module when running in linux

When I go to the file path cd to the file path ~/repo/analysis_tools/fresh_sales/ and then run python3 apicall.py it runs fine but when I try and add it to cron using python3 ~/repo/analysis_tools/fresh_sales/apicall.py the python code returns the error: No module named 'utils'.
My current project structure:
Analysis Tools:
- utils:
+ builders.py
+ load_config.py
- fresh_sales:
+ apicall.py
The start of my code:
import sys
import os
sys.path.append('..')
sys.path.append(os.path.dirname(os.path.realpath("..")))
sys.path.insert(0, '')
from utils.load_config import load_config
import requests
import json
from pandas.io.json import json_normalize
from utils.builders import build_local_db_from_config
from datetime import datetime
from sys import exit

Your path is never changed you should add the project root to the path:
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)
You could also try relative imports but this would be easier if you had something like main.py in the project root. Calling scripts higher in the directory tree is a potential sign of incorrect structure, but not always.

Is there a way to specify a BASE_DIR setting in python available to all scripts in a package?

I have a script that is writing output to files.
The problem is that it will write the file in different locations depending on where it is called from.
Is there a way to set a BASE_DIR setting so that all scripts in this package will write files to a single location?
I've attempted to intialise this to the root of the project in the root's __init__.py:
import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
In my tests, that code is not run as:
*** NameError: name 'BASE_DIR' is not defined

I believe you just need to import the BASE_DIR variable in every module you're using it.
If 'my_pkg' is the root directory of your package, then you should do something like that:
my_pkg/__init__.py:
BASE_DIR = ... # as you have
my_pkg/some_module.py:
from my_pkg import BASE_DIR

Let's say your package name is mypackage. After you add BASE_DIR into mypackage/__init__.py, you still need add this line on top of each script in mypackage:
from mypackage import BASE_DIR
This way, you can access BASE_DIR from every script in the package.

How to force import a lib from python site-package?

I am using custom django struct as below:
all settings file in conf, all app in src
and need use below manage.py:
if __name__ == "__main__":
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
SRC_PATH = os.path.join(ROOT_PATH, 'src')
CONF_PATH = os.path.join(ROOT_PATH, 'conf')
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
And I also need to use celery in django.
But django-celery need a celery.py file in same directory with settings.py.
When runserver ,it would raise ImportError: cannot import name Celery because below code:
sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)
It import itself! not from site-package, because CONF_PATH is before site-package.
But I can't change that to
sys.path.append(SRC_PATH)
sys.path.append(CONF_PATH)
This way would cause django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
For now, the only way I know is changing celery.py to celery_somename.py, and I have to type this name every time when I start a celery job.
How do I force import a lib from python site-package? Or is there another way to start celery?

You can add at the top of the celery.py module and other modules importing celery:
from __future__ import absolute_import
This will make imports absolute by default, now:
import celery
Will import the installed celery package and not this module.

Deploying Pyramid to dotcloud

What is the proper way to deploy a Pyramid project to dotcloud?
The contents of wsgi.py:
import os, sys
from paste.deploy import loadapp
current_dir = os.path.dirname(__file__)
application = loadapp('config:production.ini', relative_to=current_dir)
I'm currently getting the following error.
uWSGI Error
wsgi application not found

This could indicate that wsgi.py could not be imported successfully.
You can check the following:
output of dotcloud logs appname.servicename
log into the service with dotcloud ssh appname.servicename, then go to the current directory, start python and see what happens if you try to do from wsgi import application
If that can help, here is a super-simple Pyramid app:
https://github.com/jpetazzo/pyramid-on-dotcloud

I was able to get pass the uWSGI Error error using :
import os
from paste.deploy import loadapp
current_dir = os.getcwd()
application = loadapp('config:production.ini', relative_to=current_dir)
I still had a path problem with the static files so I changed:
config.add_static_view('static', 'static', cache_max_age=3600)
to
config.add_static_view('<myapp>/static', 'static', cache_max_age=3600)

try this:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'hellodjango.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
http://docs.dotcloud.com/tutorials/python/django/