I'm not sure what the best way of phrasing this question, but I am noticing that if I have say this file in the project directory:
a.py
import logging
import track
from raven.contrib.django.raven_compat.models import client
from django.conf import settings
from .MetricLogger import log_metrics
_LOG = logging.getLogger('application')
class TrackingClass(track.SubClass):
def record(self, *args, **kwargs):
try:
metrics = {
"metric": 'something',
"trigger": 'request',
"element": 'request'
}
# log_metrics(None, metrics)
except Exception:
client.captureException()
track.tracker.register(EventsappTrackingPixel) # this line gets called at some point without importing this module
The print statement gets called. What is happening and what is triggering it? Is this part of the class indexing that needs to be done or how is this working?
EDIT:
Turns out the track package we depend on is specifically looking for this filename in the root directory and importing it!
Django does not automatically import all files within a directory. It only automatically imports specific predefined files (should they exist), such as models.py and apps.py.
If you have additional custom modules in the directory, like profanityDetector.py, you'll need to manually import them, perhaps within apps.py.
Note that in Django, all modules are imported only once per worker process. This means that if you are not manually importing a.py from another module, it must have been registered as a callback through Django's logging system, possibly due to a pathing or directory mixup.
The call to print is not part of the class definition. It's at the module level, therefore it gets executed when the module is imported.
You can figure out what is triggering the import by inspecting the call stack. Insert these two lines right after (or before) the call to print:
import traceback
traceback.print_stack()
Related
I have a flask app with the root folder called project_folder.
A code snippet from the __init__.py file of this project_folder package:
#jwt.token_in_blacklist_loader
def check_if_token_in_blacklist(decrypted_token):
jti = decrypted_token['jti']
return project_folder.Model.RevokedTokenModel.is_jti_blacklisted(jti)
from project_folder.Controller.root import root
from project_folder.Controller import auth_controller
from project_folder.Controller import item_controller
Now the interesting thing is, that the project_folder package naturally has other smaller packages itself, which I'm importing to use them (for REST resources in this example). These are the last 3 lines, nothing throws an error so far.
But, if you take a look at the annotated function (in this example it always runs before some kind of JWT Token is being used), I am returning some inner package's function. Now when the logic truly runs this part the code breaks:
PROJECT_ROUTE\project_folder\__init__.py", line 38, in check_if_token_in_blacklist
return project_folder.Model.RevokedTokenModel.is_jti_blacklisted(jti)
NameError: name 'project_folder' is not defined
After thinking about it, it seems understandable. Importing from project_folder does import from the __init__.py file of the package, which is the actual file the interpreter currently is. So removing the package name prefix form the
return project_folder.Model.RevokedTokenModel.is_jti_blacklisted(jti)
to
return Model.RevokedTokenModel.is_jti_blacklisted(jti)
does not throw an error anymore.
The question is: Why is it only a problem inside the callback function and not with the last 3 imports?
This has to do with circular imports in python. Circular import is a form of circular dependency, created at the module import level.
How it works:
When you launch your application, python keeps a register (a kind of table) in which it records all the imported modules. When you call somewhere in your code a module, python will see in its registry if it has already been registered and loads it from there. You can access this registry via sys.module, which is actually a dictionary containing all the modules that have been imported since Python was started.
Example of use:
>>> import sys
>>> print('\n'.join(sys.modules.keys()))
So, since Python is an interpreted language, reading and execution of code is done line by line from top to bottom.
In your code, you put your imports at the bottom of your __init__.py file.
While browsing it, when python arrives at the line return project_folder.Model.RevokedTokenModel.is_jti_blacklisted(jti), it will look if the module exists in its register. Which is clearly not yet the case. That's why he raises an NameError: name 'project_folder' is not defined exception.
I have a program consistring of several modules specifying the respective web application handlers and one, specifying the respective router.
The library I use can be found here.
Excerpt from webapp.service (there are more such modules):
from webapp.router import ROUTER
#ROUTER.route('/service/[id:int]')
class ServicePermissions(AuthenticatedService):
"""Handles service permissions."""
NODE = 'services'
NAME = 'services manager'
DESCRIPTION = 'Manages services permissions'
PROMOTE = False
webapp.router:
ROUTER = Router()
When I import the webapp.router module, the webapp.service module does obviously not run. Hence, the #ROUTER.route('/service/[id:int]') decorator is not run and my web aplication will fail with the message, that the respective route is not available.
What is the best practice in that case to run the code in webapp.service to "run" the decorators? I do not really need to import the module itself or any of its members.
As stated in the comments fot the question,
you simply have to import the modules. As for linter complaints, those are the lesser of your problems. Linters are there to help - if they get into the way, just don't listen to them.
So, the simple way just to get your things working is, at the end of your __main__.py or __init__.py, depending on your app structure, to import explicitly all the modules that make use of the view decorator.
If you have a linter, check how to silence it on the import lines - that is usually accomplished with a special comment on the import line.
Python's introspection is fantastic, but it can't find instances of a class, or subclasses, if those are defined in modules that are not imported: such a module is just a text file sitting on the disk, like any data file.
What some frameworks offer as an approach is to have a "discovery" utility that will silently import all "py" files in the project folders. That way your views can "come into existence" without explicit imports.
You could use a function like:
import os
def discover(caller_file):
caller_folder = os.path.dirname(caller_file)
for current, folders, files in os.walk(caller_folder):
if current == "__pycache__":
continue
for file in files:
if file.endswith(".py"):
__import__(os.path.join(current, file))
And call it on your main module with discover(__file__)
I have a Django app that is up and running, mostly in an easy-to-understand way. But I have one quirky problem.
My code sits one level below where urls.py and views.py sit. I have a number of Python files, some of which are used as imports across the actual code in the other impoPython files.
I have found that, as part of a Django app, I need to precede import filenames with a '.' for them to import. So, if I have a file of code called foo.py, and an import file called vars.py, this won't import right:
# foo.py
from vars import *
but this works:
# foo.py
from .vars import *
The problem comes if I try to be more Pythonic (and safe) and name the import, so if I import everything it gets a prefix. But neither of the following work at all:
# foo.py
import vars as v
import .vars as v
my presumption is Django (using virtualenv or through other means) defines certain paths to look up import files, and the '.' becomes necessary to point the Python interpreter to the same directory in which the importing file sits to locate the import files it needs to find. But, somehow, this 'breaks' the ability to give the imported file contents its own prefix (or namespace, I guess).
Any thoughts?
i need to call a function from from one python class to another which are at different directories.
I'm using Eclipse and PyDev for developing scripts.
sample.py
class Employee:
def meth1(self,arg):
self.arg=arg
print(arg)
ob=Employee()
ob.meth1("world")
main.py
class main:
def meth(self,arg):
self.arg=arg
print(arg)
obj1=main()
obj1.meth("hello")
I need to access meth1 in main.py
updated code
main.py
from samp.sample import Employee
class main:
def meth(self,arg):
self.arg=arg
print(arg)
obj1=main()
obj1.meth("hello")
After executing main.py it is printing "world" automatically without calling it.
my requirement is i need to call meth1 from main.py explicitly
please find my folder below
import is the concept you need here. main.py will need to import sample and then access symbols defined in that module such as sample.Employee
To ensure that sample.py can be found at import time, the path to its parent directory can be appended to sys.path (to get access to that, of course, you will first need to import sys). To manipulate paths (for example, to turn a relative path like '../samp' into an absolute path, you might want to import os as well and take a look at the standard library functions the sub-module os.path has to offer.
You will have to import the sample.py file as a module to your main.py file. Since the files are in different directories, you will need to use the __init__.py
From the documentation:
The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.
Here is a related stack overflow question which explains the solution in more detail.
It's generally not a good idea to alter the sys.path variable. This can make scripts non-portable. Better is to just place your extra modules in the user site directory. You can find out what that is with the following script.
import os
import sys
import site
print(os.path.join(site.USER_BASE, "lib", "python{}.{}".format(*sys.version_info[0:2]), "site-packages"))
Place your modules there. Then you can just import as any other module.
import sample
emp = sample.Employee()
Later you can package it using distutils or setuptools and the imports will still work the same.
I have 2 scripts.
Main.py
Update.py
I have a function in Main.py which basically does the following:
def log(message):
print(message)
os.system("echo " + message + " >> /logfile.txt")
And in the Update.py file I have a single function which basically does the update. However throughout the update function, it calls "log(message)" with whatever the message is at that point.
The problem now though is I'm getting a NameError: global name "log" is not defined whenever I try use the function outside of the Main.py script.
Any help? on how I would be able to use the function 'log' wherever?
*Code simplified for explanation.
EDIT:
Main.py imports Update from /Scripts/Update.py
Update.py imports log from Main.py
When i try this, it fails saying "cannot import name Update"
Don't import log from Main. That'll rerun the code in Main.py, since running Main.py as a script and importing it as a module aren't equivalent. Other modules should not depend on functions defined in the main script. Instead, put your log function in another module and import that, or have Main.py explicitly pass a logger to the other modules somehow.
Update: You can't import Update because Python can't find it. Python checks in 4 places for modules to import, but the ones you should be interested in are
the directory the script was from, and
the directories specified in the PYTHONPATH environment variable.
You'll either need to put Main.py and the things it imports in the same directory, or add /Scripts to your PYTHONPATH.
Just add in Update.py the line
from Main import log
and you will be able to call log() from Update.py.
You should import the function:
from Main import log
Or best:
import Main
Main.log()
For modules importing each other, refer How can I have modules that mutually import each other.