I have the following package structure (drastically simplified from a real use case):
mypackage/
├── __init__.py
├── mod1.py
│ ├── def func1(): return 1
│ └── ...
│
├── mod2.py
│ ├── def func2(): return 2
│ └── ...
└── mod3.py
with __init__.py like this
from .mod1 import *
from .mod2 import *
Now, in mod3.py I want to access the packages complete namespace with one alias like res = p.func1() + p.func2() and I want this to achieve this by one relative import statement. Is this possible?
I don't want an absolute import like import mypackage as p (because the code should be indifferent on renaming the package).
Note, this is related to but different from this unanswered question from 2009.
Generally you do not want to use asterisks while doing imports so you can use this while still using relative import:
from . import mod1
You can call the function like this:
mod1.func1()
PS: If you are using Python 3 you are no longer required to use __init__.py to define packages.
Edit:
If you want use common namespace for all those functions you have imported you could create a new module to use as a header file:
mypackage/
├── header.py
│ ├── from .mod1 import *
│ └── from .mod2 import *
├── mod1.py
│ ├── def func1(): return 1
│ └── ...
│
├── mod2.py
│ ├── def func2(): return 2
│ └── ...
└── mod3.py
And in mod3.py you can import header.py and use the functions in various ways:
from .header import *
val1 = func1()
val2 = func2()
from . import header
val1 = header.func1()
val2 = header.func2()
from . import header as p
val1 = p.func1()
val2 = p.func2()
I'm making my own python package with submodules to organize things. How can I create an object defined at the top level and pass the object to a lower submodule so that the submodule edits values?
File structure:
packageFolder
├── mypackage
│ ├── resources
│ │ └── MASTERLOOKUP.csv
│ ├── analysis
│ │ ├── __init__.py
│ │ ├── elffileextract.py
│ │ └── apkextraction.py
│ └── __init__.py
├── setup.py
└── README.md
Inside of the top level __init__.py I included quite a bit of code so I didn't have a long string when creating objects, ie x = mypackage.m_file() preferable as opposed to x = mypackage.mymodule.m_file()
####__init__.py
class m_file():
def __init__(self):
self.FilePath = "*"
self.FileSize = "*"
self.FileType = "*"
self.FileDependencies = "*"
def fileoperation(self):
analysis.elffileextract.process(self) #want to pass in 'm_file' object
Inside a submodule file elffileextract.py I'll have something like this
####analysis/elffileextract.py
from . import m_file
def elfextract(mFile:m_file): #take object as argument
filename = mFile.FileName
mFile.FileDependencies = "glibc.so"
return
Inside of the file analysis/__init__.py I have the following
#### analysis/__init__.py
import mypackage.analysis.elffileextract.py
import mypackage.analysis.apkextraction.py
How I want to be able to use the package is as follows:
>>> import mypackage
>>> x = mypackage.m_file()
>>> x.FileDependencies
'*'
>>> x.fileoperation()
>>> x.FileDependencies
'glibc.so'
I'm not sure what the technical term for passing objects into submodules is but any answers to my problem or pointers in the right direction would be appreciated. I've tried various variations of imports in all files, I can't seem to figure it out. I may be trying to overcomplicate this in a non-pythonic way of doing things.
I have an API that uses FastAPI. In a single file (main.py), I have the call to the function that creates the API
from fastapi import FastAPI
# ...
app = FastAPI()
As well as all the endpoints:
#app.post("/sum")
async def sum_two_numbers(number1: int, number2: int):
return {'result': number1 + number2}
But as the application gets larger, the file is becoming messy and hard to maintain. The obvious solution would be to keep function definitions in separate modules and just import them and use them in main.py, like this:
from mymodules.operations import sum_two_numbers
# ...
#app.post("/sum")
sum_two_numbers(number1: int, number2: int)
Only that doesn't work. I don't know if I'm doing it wrong or it can't be done, but I get this error from VSCode:
Expected function or class declaration after decorator | Pylance
(My program has so many errors that I haven't seen the actual interpreter complaint, but if that's important, I can try debug it and post it here)
So is this impossible to do and I have to stick to the one-file API, or it is possible to do, but I'm doing it wrong? If the second, what is the right way?
The common solution is to split your application into subrouters. Instead of using app directly when registering your views, you create an instance APIRouter (from fastapi import APIRouter) inside each of your modules, then you register these subrouters into your main application.
Inside a dedicated api module, such as api/pages.py:
from fastapi import APIRouter
router = APIRouter()
#router.get('')
async def get_pages():
return ...
from .api import (
pages,
posts,
users,
)
app.include_router(pages.router, prefix='/pages')
app.include_router(posts.router, prefix='/posts')
app.include_router(users.router, prefix='/users')
Another powerful construct you can use is to have two dedicated base routers, one that requires authentication and one that doesn't:
unauthenticated_router = APIRouter()
authenticated_router = APIRouter(dependencies=[Depends(get_authenticated_user)])
.. and you can then register the different routes under each router, depending on whether you want to guard the route with an authenticated user or not. You'd have two subrouters inside each module, one for endpoints that require authentication and one for those that doesn't, and name them appropriately (and if you have no public endpoints, just use authenticated_router as the single name).
unauthenticated_router.include_router(authentication.router, prefix='/authenticate')
unauthenticated_router.include_router(users.unauthenticated_router, prefix='/users', tags=['users'])
authenticated_router.include_router(users.router, prefix='/users')
Any sub router registered under authenticated_router will have the get_authenticated_user dependency evaluated first, which in this case would throw a 401 error if the user wasn't logged in. You can then authorized further based on roles etc. in the dependencies for the view function - but this makes it very explicit whether you want your endpoint to end up in a chain that requires authentication or not.
So is this impossible to do and I have to stick to the one-file API, or it is possible to do, but I'm doing it wrong?
One file api is just for demo/test purposes, in the real world you always do multi-file api especialy with framework like fastapi where you use different type of file like pydantic models, db models, etc.
If the second, what is the right way?
There is no "right way", there is ways that fit your needs.
You can follow the advenced user guide, to see a good example.
what the doc suggest:
.
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── dependencies.py
│ └── routers
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── internal
│ ├── __init__.py
│ └── admin.py
what i use when i have db, pydantic models, etc
.
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── dependencies.py
│ └── routers
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── models
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── schemas
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── internal
│ │ ├── __init__.py
│ │ └── admin.py
here models represent db models and schemas represent pydantic models
I am trying to import a sibling module that itself has imports from another module.
The tree is as follows
├── USA
├── NewYork
│ ├── __init__.py
│ ├── Manhattan.py
│ └── Queens.py
├── Atlanta
│ ├── __init__.py
│ ├── Buckhead.py
The code is as follows
#Queens.py
def myFunction():
print("I am in Queens")
#Manhattan.py
from Queens import myFunction
class Manhattan():
def __init__(self):
myFunction()
print("I am in Manhattan")
Now, I want to instantiate an object of class Manhattan in Buckhead
#Buckhead.py
from NewYork.Manhattan import Manhattan
man = Manhattan()
I am getting a
ModuleNotFoundError: No module named 'Queens'
How do I go about instantiating the Manhttan class that ITSELF imports a function from the Queen module?
I have read everything on SO and all the django docs looking for a solution to this. My models work absolutely fine but i cant import them to my 'getnews.py' file in the same directory as models.py, i can to the views.py and there is absolutely no circular imports!
models.py:
from django.db import models
import datetime
from django.utils import timezone
class newsurls(models.Model):
title = models.CharField(max_length=200)
def __unicode__(self):
return unicode(self.title)
pub_date = models.DateTimeField("date published")
class newsblock(models.Model):
news = models.ForeignKey(newsurls)
url = models.URLField(max_length=2000)
def __unicode__(self):
return unicode(self.url)
image = models.URLField(max_length=2000)
favi = models.URLField(max_length=2000)
bgcolor = models.CharField(max_length=20)
tcolor = models.CharField(max_length=20)
genre = models.CharField(max_length=200)
in views.py: (works)
from news.models import newsurls, newsblock
getnews.py: (dont work)
from news.models import newsurls, newsblock
traceback:
Traceback (most recent call last):
File "/home/skru/newsshuffle/news/getnews.py", line 3, in <module>
from news.models import newsurls, newsblock
ImportError: No module named news.models
views.py:
from django.shortcuts import render_to_response
from news.models import newsurls, newsblock
try:
import cPickle as pickle
except:
import pickle
import random
def news(request):
newsDatabase = open('/home/skru/newsshuffle/news/storyDb.p', 'r+')
openData = newsDatabase.read()
dstory = pickle.loads(openData)
count = dstory['count']['links']
story = []
outstorys = []
keys = dstory.keys()
for key in keys:
if key != 'count':
story.append(dstory[key]['genre'])
story.append(dstory[key]['title'])
story.append(dstory[key]['url'])
story.append(dstory[key]['image'])
story.append(dstory[key]['bgcolor'])
story.append(dstory[key]['via'])
story.append(dstory[key]['tcolor'])
outstorys.append(story)
story = []
random.shuffle(outstorys)
lists = newsurls.objects.order_by('-pub_date')[:100]
return render_to_response('news/news.html',
{
'story_list':outstorys,
'count':count,
'lists':lists,
})
filepath:
├── db.sqlite3
├── manage.py
├── news
│ ├── static
│ │ └── news
│ │ ├── news.css
│ │ └── ...
│ ├── templates
│ │ └── news
│ │ ├── allnews.html
│ │ └── ...
│ ├── __init__.py
│ ├── admin.py
│ ├── dd.py
│ ├── getPrevDate.py
│ ├── getnews.py
│ ├── models.py
│ ├── storyDb.p
│ ├── tests.py
│ ├── urls.py
│ ├── views.py
└── newsshuffle
├── __init__.py
├── settings.py
├── urls.py
└── wsgi.py
i have tried every sort of different import 'newsshuffle.news.models' etc..,
added the system path to the system path manually as suggested in other feeds please help!!
manage.py:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "newsshuffle.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
added this because most other posts on SO point to setting DJANGO_SETTINGS_MODULE having to be set manually bur as you can see it already is
Given that directory structure, your import should be working.
How about trying a relative import instead? In both views.py and getnews.py this should work:
from models import ...
have You tried that?
from .models import newsurls, newsblock
Please also read about PEP8 http://legacy.python.org/dev/peps/pep-0008/ cause Your code is not formatted correctly.
EDITED: I see that You trying to do that but without '.' sign. Please note that before models is '.' like that .models