I have the following package structure (drastically simplified from a real use case):
mypackage/
├── __init__.py
├── mod1.py
│ ├── def func1(): return 1
│ └── ...
│
├── mod2.py
│ ├── def func2(): return 2
│ └── ...
└── mod3.py
with __init__.py like this
from .mod1 import *
from .mod2 import *
Now, in mod3.py I want to access the packages complete namespace with one alias like res = p.func1() + p.func2() and I want this to achieve this by one relative import statement. Is this possible?
I don't want an absolute import like import mypackage as p (because the code should be indifferent on renaming the package).
Note, this is related to but different from this unanswered question from 2009.
Generally you do not want to use asterisks while doing imports so you can use this while still using relative import:
from . import mod1
You can call the function like this:
mod1.func1()
PS: If you are using Python 3 you are no longer required to use __init__.py to define packages.
Edit:
If you want use common namespace for all those functions you have imported you could create a new module to use as a header file:
mypackage/
├── header.py
│ ├── from .mod1 import *
│ └── from .mod2 import *
├── mod1.py
│ ├── def func1(): return 1
│ └── ...
│
├── mod2.py
│ ├── def func2(): return 2
│ └── ...
└── mod3.py
And in mod3.py you can import header.py and use the functions in various ways:
from .header import *
val1 = func1()
val2 = func2()
from . import header
val1 = header.func1()
val2 = header.func2()
from . import header as p
val1 = p.func1()
val2 = p.func2()
I'm making my own python package with submodules to organize things. How can I create an object defined at the top level and pass the object to a lower submodule so that the submodule edits values?
File structure:
packageFolder
├── mypackage
│ ├── resources
│ │ └── MASTERLOOKUP.csv
│ ├── analysis
│ │ ├── __init__.py
│ │ ├── elffileextract.py
│ │ └── apkextraction.py
│ └── __init__.py
├── setup.py
└── README.md
Inside of the top level __init__.py I included quite a bit of code so I didn't have a long string when creating objects, ie x = mypackage.m_file() preferable as opposed to x = mypackage.mymodule.m_file()
####__init__.py
class m_file():
def __init__(self):
self.FilePath = "*"
self.FileSize = "*"
self.FileType = "*"
self.FileDependencies = "*"
def fileoperation(self):
analysis.elffileextract.process(self) #want to pass in 'm_file' object
Inside a submodule file elffileextract.py I'll have something like this
####analysis/elffileextract.py
from . import m_file
def elfextract(mFile:m_file): #take object as argument
filename = mFile.FileName
mFile.FileDependencies = "glibc.so"
return
Inside of the file analysis/__init__.py I have the following
#### analysis/__init__.py
import mypackage.analysis.elffileextract.py
import mypackage.analysis.apkextraction.py
How I want to be able to use the package is as follows:
>>> import mypackage
>>> x = mypackage.m_file()
>>> x.FileDependencies
'*'
>>> x.fileoperation()
>>> x.FileDependencies
'glibc.so'
I'm not sure what the technical term for passing objects into submodules is but any answers to my problem or pointers in the right direction would be appreciated. I've tried various variations of imports in all files, I can't seem to figure it out. I may be trying to overcomplicate this in a non-pythonic way of doing things.
I have an API that uses FastAPI. In a single file (main.py), I have the call to the function that creates the API
from fastapi import FastAPI
# ...
app = FastAPI()
As well as all the endpoints:
#app.post("/sum")
async def sum_two_numbers(number1: int, number2: int):
return {'result': number1 + number2}
But as the application gets larger, the file is becoming messy and hard to maintain. The obvious solution would be to keep function definitions in separate modules and just import them and use them in main.py, like this:
from mymodules.operations import sum_two_numbers
# ...
#app.post("/sum")
sum_two_numbers(number1: int, number2: int)
Only that doesn't work. I don't know if I'm doing it wrong or it can't be done, but I get this error from VSCode:
Expected function or class declaration after decorator | Pylance
(My program has so many errors that I haven't seen the actual interpreter complaint, but if that's important, I can try debug it and post it here)
So is this impossible to do and I have to stick to the one-file API, or it is possible to do, but I'm doing it wrong? If the second, what is the right way?
The common solution is to split your application into subrouters. Instead of using app directly when registering your views, you create an instance APIRouter (from fastapi import APIRouter) inside each of your modules, then you register these subrouters into your main application.
Inside a dedicated api module, such as api/pages.py:
from fastapi import APIRouter
router = APIRouter()
#router.get('')
async def get_pages():
return ...
from .api import (
pages,
posts,
users,
)
app.include_router(pages.router, prefix='/pages')
app.include_router(posts.router, prefix='/posts')
app.include_router(users.router, prefix='/users')
Another powerful construct you can use is to have two dedicated base routers, one that requires authentication and one that doesn't:
unauthenticated_router = APIRouter()
authenticated_router = APIRouter(dependencies=[Depends(get_authenticated_user)])
.. and you can then register the different routes under each router, depending on whether you want to guard the route with an authenticated user or not. You'd have two subrouters inside each module, one for endpoints that require authentication and one for those that doesn't, and name them appropriately (and if you have no public endpoints, just use authenticated_router as the single name).
unauthenticated_router.include_router(authentication.router, prefix='/authenticate')
unauthenticated_router.include_router(users.unauthenticated_router, prefix='/users', tags=['users'])
authenticated_router.include_router(users.router, prefix='/users')
Any sub router registered under authenticated_router will have the get_authenticated_user dependency evaluated first, which in this case would throw a 401 error if the user wasn't logged in. You can then authorized further based on roles etc. in the dependencies for the view function - but this makes it very explicit whether you want your endpoint to end up in a chain that requires authentication or not.
So is this impossible to do and I have to stick to the one-file API, or it is possible to do, but I'm doing it wrong?
One file api is just for demo/test purposes, in the real world you always do multi-file api especialy with framework like fastapi where you use different type of file like pydantic models, db models, etc.
If the second, what is the right way?
There is no "right way", there is ways that fit your needs.
You can follow the advenced user guide, to see a good example.
what the doc suggest:
.
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── dependencies.py
│ └── routers
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── internal
│ ├── __init__.py
│ └── admin.py
what i use when i have db, pydantic models, etc
.
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── dependencies.py
│ └── routers
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── models
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── schemas
│ │ ├── __init__.py
│ │ ├── items.py
│ │ └── users.py
│ └── internal
│ │ ├── __init__.py
│ │ └── admin.py
here models represent db models and schemas represent pydantic models
I am making a scraper function with tidy folder structure. But when I try to import scraper class into views.py it's giving an error:
'module' object is not callable
This is the Tree:
├── api_services
│ ├── spiders
│ │ ├── spiderAtom.py
│ │ └── spiderEbis.py
│ └── views
│ └── viewApi.py
In spiders folder I have this class:
class spiderAtom:
def atom():
string = "return this method"
return string
and trying import it in viewApi
from ..spiders import spiderAtom
def atomApi(request):
spider = spiderAtom()
response = spider.atom()
return HttpResponse(response)
But with the the way I am doing is not working.
Just add __init__.py file to api_services and then call from api_services.spiders.spiderAtom import spiderAtom.
I'm currently writing a library in python. I have a package called Selectors as a sub-directory of the library. I am trying to implement a new module in the package, but when I try to import it I get the error:
NameError: name '_RaceSelector__ResultSelector' is not defined
My directory looks like this:
Selectors
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── __pycache__
│ │ ├── SeasonSelector.cpython-38.pyc
│ │ ├── Selector.cpython-38.pyc
│ │ ├── __init__.cpython-38.pyc
│ │ ├── race_selector.cpython-38.pyc
│ │ ├── result_selector.cpython-38.pyc
│ │ └── season_selector.cpython-38.pyc
│ ├── race_selector.py
│ ├── race_selector.pyc
│ ├── result_selector.py
│ ├── result_selector.pyc
│ ├── season_selector.py
│ ├── season_selector.pyc
│ ├── selector.py
│ └── selector.pyc
I want to use the modules in race_selector.py, here is that file:
from .selector import __Selector
from .result_selector import __ResultSelector
class RaceSelector(__Selector):
data = []
loaded_races = []
header = []
result_selector = __ResultSelector()
selector.py
import os
import csv
class __Selector:
def __init__(self, file_name):
self.data_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../data/' + file_name + '.csv')
self.raw_data = self.load_data()
self.data = self.get_data()
result_selector.py
import os
from .selector import __Selector
class __ResultSelector(__Selector):
def __init__(self):
super().__init__('results')
I am able to import selector just fine and works as intended, but result_selector produces the error.
Thanks
When you do the following:
result_selector = __ResultSelector()
Python searches for _RaceSelector__ResultSelector because there is 2 underscores.
As mentioned in PEP8:
If your class is intended to be subclassed, and you have attributes that you do not want subclasses to use, consider naming them with double leading underscores and no trailing underscores. This invokes Python's name mangling algorithm, where the name of the class is mangled into the attribute name. This helps avoid attribute name collisions should subclasses inadvertently contain attributes with the same name.