We Keep Coding

Is there a way to fix the pythonpath when using aws sam?

I'm trying to import a custom module that I created, but it breaks my API just to import it.
data directory:
-src
--order
----__init__.py
----app.py
----validator.py
----requirments.txt
--__init__.py
on my app.py I have this code:
import json
from .validator import validate
def handler(event, context):
msg = ''
if event['httpMethod'] == 'GET':
msg = "GET"
elif event['httpMethod'] == 'POST':
pass #msg = validate(json.loads(event['body']))
return {
"statusCode": 200,
"body": json.dumps({
"message": msg,
}),
}
I get this error:
Unable to import module 'app': attempted relative import with no known parent package
However, if I remove line 2 (from .validator import validate) from my code, it works fine, so the problem is with that import, and honestly, I can't figure what is going on. I have tried to import using:
from src.order.validator import validate
but it doesn't work either.

was able to solve my issue by generating a build through the command: sam build, and zipping my file, and putting it on the root folder inside aws-sam, it's not a great solution because I have to rebuild at every small change, but at least it's a workaround for now

It seems app.py has not been loaded as part of the package hierarchy (i.e. src and order packages have not been loaded). You should be able to run
from src.order import app
from the parent directory of src and your code will work. If you run python app.py from the terminal — which I assume is what you did — app.py will be run as a standalone script — not as part of a package.
However, I do not believe you need the .validator in your case since both modules are in the same directory. You should be able to do
from validator import validate

Application says module not found after moving to directory

I'm rather new to python and have this odd issue, which I can't seem to find an answer for.
When both app.py and mod_db were in the root directory, it works but when I shifted them to a sub directory
My directory structure:
demo_api
|
|-- demo-api
|
|-- __init__.py
|-- app.py
|-- mod_db.py
My main module
import json
from flask import Flask, request, Response
from .db_mod import insert_and_score
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
#app.route('/emotional_scoring', methods=['POST'])
def get_scoring():
json_obj = request.json
ret_json = insert_and_score(json_obj)
resp = Response(json.dumps(ret_json), mimetype='application/json', status=200)
return resp
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=False)
The error message is on this line
from .mod_db import insert_and_score
I've tried change the sub-directory name. I've tried doing a full path, i.e. from demo_app.mod_db import insert_and_score and the error is ModuleNotFoundError: No module named 'demo_app'
The issue is that it works find in my IDE (PyCharm) but when I do it on command line, these are the errors I encounter.

As stated by mfrackowiak, I just had to change it to
from db_mod import insert_and_score
and it worked. But PyCharm doesn't like this. So I think it might be a PyCharm issue.

Fix by removing relative import
It seems you want both the relative imports and Pycharm to be happy. Like mfrackowiak said, you want to use absolute importing.
from db_mod import insert_and_score
Tell PyCharm what the new "Sources Root" is
Now to make Pycharm happy, you will want to tell it that the demo_api subdirectory is a "Sources Root". You can do this by right clicking the directory and go to Mark Directory as > Sources Root. You can also find it in Preferences > Project Structure. You can do this for each subdirectory, as needed.
Why this is good to do
This is useful when you have a python project as a subdirectory of a repo with many configs, scripts, and other files and directories. You often don't want the python app to be mixed in with those, so you move it to an app/ folder. This confuses Pycharm, as by default, it uses the Content Root as the Sources Root as well. You can fix this by simply telling Pycharm explicitly what the Sources Root is.
Example:
my-awesome-project/ <---- Content Root
|--.venv/
| |--<venv stuff>
|--scripts/
| |--build.sh
| |--run.sh
|--docker/
| |--dev.Dockerfile
| |--prod.Dockerfile
|--app/ <---------------- Sources Root
| |--sub-mod1/
| | |--foo.py
| | |--bar.py
| |--sub-mod2/
| |--baz.py
|--.gitignore
|--.python-version
|--requirements.txt
|--dev.env
|--docker-compose.yml

Django: where is settings.py looking for imports, and why?

I have a Django app with a common directory structure:
project
---manage.py
---app
---__init__.py
---settings.py
---settings_secret.py
---a bunch of other files for the app itself
That settings_secret.py file contains variables from secrets.py which I do not want to send to github. For some reason, I cannot seem to import it into settings.py. First 5 lines of settings.py:
# Django settings for project.
DEBUG = True
TEMPLATE_DEBUG = DEBUG
import os
from settings_secret import *
Which fails with the partial stacktrace:
File "/foo/bar/project/app/settings.py", line 5, in <module> from settings_secret import *
ImportError: No module named 'settings_secret'
To debug, I created a test file inside /project/ like so:
from settings_secret import *
print(VARIABLE_FROM_SETTINGS_SECRET)
Which worked like a charm. So clearly, settings.py isn't looking in the right place for settings_secret. So where is it looking?

In settings.py, you should do: from .settings_secret import *
It works with the . because the proper syntax is supposed to be:
from app.settings_secret import *
Removing the app is shorthand coding, but the same principle applies. You need to tell Django to look for the directory (app), and then you are specifying which file in that directory to look for.
If you just did, from settings_secret import *, you are telling Django to look for the directory settings_secret, which doesn't exist.
Does that make sense to you?

Importing a sub class from a sub folder

New to Python I'm trying to setup a simple OOP-structure of files, folders and classes. Here are the file paths:
C:\Users\Mc_Topaz\Programmering\Python\Terminal\Main.py
C:\Users\Mc_Topaz\Programmering\Python\Terminal\Connections\Connection.py
C:\Users\Mc_Topaz\Programmering\Python\Terminal\Connections\NoConnection.py
Notice Connection.py and NoConnection.py is loacted in sub folder Connections.
Connection.py
class Connection:
def __init__(self):
pass
def ToString(self):
pass
NoConnection.py
from Connection import Connection
class NoConnection(Connection):
def __init__(self):
pass
def ToString(self):
print("No connection")
In the Main.py file I would like to call the ToString() method from each class.
Main.py
from Connections.Connection import Connection
from Connections.NoConnection import NoConnection
connection = Connection()
print(connection.ToString())
noConnection = NoConnection()
print(noConnection.ToString())
When I run the Main.py file I get this error:
C:\Users\Mc_Topaz\Programmering\Python\Terminal>Main.py Traceback
(most recent call last): File
"C:\Users\Mc_Topaz\Programmering\Python\Terminal\Main.py", line 2, in
from Connections.NoConnection import NoConnection
File
"C:\Users\Mc_Topaz\Programmering\Python\Terminal\Connections\NoConnection.py",
line 1, in
from Connection import Connection
ImportError: No module named 'Connection'
It seems that the interpreter cannot import the NoConnection class in my Main.py file due to it cannot import the Connection class from the NoConnection.py file.
I can run Connection.py and NoConnection.py separately with no problems.
I don't understand why the Main.py don't run. I assume is something super simple and I cannot see it due to I'm to green to Python.

For python to recognize a directory is a module, or a collection of python files, that directory must contain a file named __init__.py. That file doesn't need to contain anything code whatsoever, though it can. If you add this file to your Connections directory, the interpreter should be able to import the contained files.

Just to add on to what mobiusklein mentioned: a common practice with __init__.py is to import the objects you use most frequently to avoid redundancy in your imports. In your example your Connections\__init__.py would likely contain:
import Connection
import NoConnection
Then your Main.py could use the following import statements successfully:
from Connections import Connection, NoConnection

I think it's down to relative imports in Python 3.
Change the import line in NoConnection to be explicit...
from Connections.Connection import Connection
and it works in Python3 (it works either way in Python2). Path to Terminal may have to be on you PYTHONPATH environment variable.
"Python 3 has disabled implicit relative imports altogether; imports are now always interpreted as absolute, meaning that in the above example import baz will always import the top-level module. You will have to use the explicit import syntax instead (from . import baz)." from here https://softwareengineering.stackexchange.com/questions/159503/whats-wrong-with-relative-imports-in-python
So when you import Connection from NoConnection it will be looking for it at the Terminal level, not at the Terminal/Connections level.

UPDTAE: Read comments for this post as it contain the solution.
Still don't get this to work. But I have made some changes:
Changed names of classes
> Class Connection = Foo inside Connection.py
> NoConnection = Bar inside NoConnection.py
This is ensure that the interpreter don't get confused if Connection is a module or a class.
Dropped ini.py file
I have dropped the Connections\__init_.py file as this don't seem to be necessary in Python version 3.4.2.
Files still run separately
I can run Connection.py and NoConnection.py separately. So they work.
Main.py
from Connections.NoConnection import Bar
noConnection = Bar()
print(noConnection.ToString())
When running Main.py I get the same error at line 1:
"Cannot find module 'Connection' in NoConnection.py at line 1".
The only logical reason I can see why this error happens is:
The interpreter looks for Connection.py inside my Terminal folder as Python was started from that folder. Even if it's importing the NoConnection.py file from the Connections folder where Connection.py is located.
Is this a bug?

ImportError and Django driving me crazy

OK, I have the following directory structure (it's a django project):
-> project
--> app
and within the app folder, there is a scraper.py file which needs to reference a class defined within models.py
I'm trying to do the following:
import urllib2
import os
import sys
import time
import datetime
import re
import BeautifulSoup
sys.path.append('/home/userspace/Development/')
os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
from project.app.models import ClassName
and this code just isn't working. I get an error of:
Traceback (most recent call last):
File "scraper.py", line 14, in
from project.app.models import ClassName
ImportError: No module named project.app.models
This code above used to work, but broke somewhere along the line and I'm extremely confused as to why I'm having problems. On SnowLeopard using python2.5.

import sys
sys.path.append ('/path/to/the/project')
from django.core.management import setup_environ
import settings
setup_environ(settings)
from app.models import MyModel

Whoa whoa whoa. You should never ever have to put your project name in any of your app code. You should be able to reuse app code across multiple projects with no changes. Pinax does this really well and I highly recommend checking it out for a lot of django best practices.
The worst thing you could do here is to hard code your absolute path into your app or settings. You shouldn't do this because it will break during deployment unless you do some import local_settings hacking.
If you have to access the project root directory, try what pinax has in settings.py...
import os.path
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
The thing is that it looks like you are trying to access the models module within the same app and this is waaay easier.
To import models.py inside scraper.py in the same directory just use import models or import models as app_models if you already have something named models in scraper.py (django.db.models for instance). Are you familiar with Python module conventions?
However, the best way is probably to stick with the django idiom, from ... import ... statement:
from app import models
If this doesn't work automatically, then something is wrong in your settings.py.

You don't indicate if project is located in /home/userspace/Development/. I'll assume that it is.
Make sure there's an (empty by default) file named __init__.py in project and another one in app.
EDIT: Next thing to try: Fire up the Python command line in the script's directory and try the following:
import project
import project.app as app
import project.app.models as models
models.__dict__.keys()
Do they all work? If so, what is the last line's output? If not, which dies first?