I am relatively new to Python having used C# for many years and I'm hoping someone can help me with this question. I have a module called actuators.py that contains a number of classes for defining properties and methods for the servos I use in a robot project. In another module called robot.py, I instantiate my actuators like this:
import actuators as Actuators
myActuators = Actuators.AllActuators()
This allows me to access the properties of my servos as long as I am in the robot.py module. For example, I can write:
print myActuators.HeadTilt.MinPosition
to get the minimum value allowed for the servo that tilts the robot's head. So far so good.
Now I want to access these same values in a separate thread that is defined by a different module called tilt_head.py. I assume I need to import a reference to the robot.py module, but doing this ends up re-executing all the code in robot.py whereas all I really want is a static reference to the myActuators object. And I can't use
from robot.py import myActuators
because myActuators is not a module.
In C#, I would do this using a declaration like this:
public static Actuators myActuators;
which then allows me to reference myActuators in any other file within my project. Is there a way to do something similar in Python? If you need my actual code, I will be happy to post it.
Thanks!
And I can't use
from robot.py import myActuators
because myActuators is not a module.
But myActuators doesn't need to be a module. You can do exactly that. (Though you'll want to use just robot rather than robot.py)
http://docs.python.org/reference/simple_stmts.html#import
As well:
http://docs.python.org/tutorial/modules.html#more-on-modules
"A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module is imported somewhere."
So using a script that imports robot and then tilt_head, the executable stuff in robot.py will NOT be run multiple times.
Of course, if robot.py is intended to be the main module of the program, then I'd suggest going with A. Levy's answer.
You should be able to simply use:
from robot import myActuators
You can't say "from robot.py" because the name there is a module name, not a file name.
BTW: I'm not sure why you are doing import actuators as Actuators. Why do you want to change the case? Modules are most commonly lowercase.
As others have been saying, you can actually import myActuators from the robot module. To solve your problem of the code in robot being re-executed, the standard Python approach is to wrap the stand-alone code in an if __name__ == '__main__': so that it will only be executed when the module is used as a standalone app, but not when it is imported.
E.G.
# Things that are defined when robot.py is used as a module
# and when used standalone.
import actuators as Actuators
myActuators = Actuators.AllActuators()
if __name__ == '__main__':
# Stuff that you only want executed when running robot.py
Then when you want to use robot.py in another module, you should be able to do this:
from robot import myActuators
EDIT:
Actually, as JAB pointed out, I was a little confused in my explanation of conditional execution. your code won't be re-executed if you reload the module. The module loading machinery handles reloading of modules to make sure that they only get loaded and defined one time. All subsequent loads are references to the previously loaded module. If, however, you do have some steps that you only want to happen when you are running the module as a script, then you can place them in the if block and they won't be executed when you load robots from another module.
I guess it doesn't really solve your problem, though it seems like your problem was really an incorrect import syntax.
I would guess that you can access that object as robot.myActuators because it already exists in the process, inside the robot.py namespace.
Many thanks for all the answers and pointers. As it turns out, my problem was a recursive import that ended up trying to create an object more than once. By sorting out the instance creation a little more logically, I was finally able to make the problem go away.
Related
I'm developing a physical model using Python. I right now have four files (I suppose I can call them modules but I really am no Python expert) in one folder. Right now it works just fine but after reading a bit it turned out the way I do it could lead to errors when further expanding my model (by using from ... import* statement).
(1) initialize: Does nothing except for calling the plot() module with some information about what to plot. Here I would like to set some initial parameters which won't change during the computation and pass them to the compute module. Imports plot
(2) compute: Contains initial values (which I actually want to set in initialize module) and all the computation algorithm. During computation, functions which are stored in functions module are being called a lot. Imports functions
(3) functions: Contains actual physical functions.
(4) plot: Contains all the plots. Imports everything from compute (from compute import *)
Originally I only had the compute and the functions module. When I started plotting however, the compute module became too big (in my eyes) so I wanted to split. I realized I would need to pass all the variables I calculated in compute (quite a few) as arguments to the plot module. Or alternatively import the compute module with a prefix (e.g. import compute as com --> com.variable). I didn't like both options too much. So i decided to use the from compute import * statement, which allows me to use the same variable names as originally defined in compute. This looks a lot cleaner to me and makes it easier to read to variable names.
The problem is, when importing compute into my plot module (to get all the variable names) the code in compute is executed. But I would like to call compute from my initialize module to set up the initial parameters. However If i do so i would need to call compute twice, which takes a lot more time. Besides that, from ... import * apparently is not a good choice.
I'm grateful for any suggestions.
On the import statements
The main reason to avoid from my_model import * statements is, that it obscures which variables are imported (and available in the scope after the importing) and where does the variables comes from. For example if you have:
from mod1 import *
from mod2 import *
...
print(my_var)
... it is not immediately clear if the my_var comes from mod1 or mod2. Furthermore, lets say, we define my_var in mod1 and then during the development we define my_var also in mod2 (for example even without exporting it at all), suddenly my_var is something else which may lead to weird errors. And if you are using chained imports, it is really hard to keep track which import gets executed first and which last, so generally you need to make sure you never define variables with clashing names.
This problem is somewhat (personally, I still think it is worth avoiding just for that reason) mitigated with usage of modern IDEs which can point out clashing names, however it is still subject to overlooking. However, speaking of the IDEs... if you know a function you need is in a specific module -- if you imported the module as import my_mod -- you can type my_mod. and the IDE shows you the options, if you import it via from my_mod import * its not so easy.
For these reason it is generally recommended to use plain import my_mod or using aliases import my_mod as mm which have none of these problems. I know you mentioned you don't like this approach but maybe it is worth reconsidering. Nevertheless, if you still don't like it there is a middle ground of from my_mod import var_1, var_2, func. Then you don't have to write mm. every time you need the variables. It has a drawback that every time you want to use a new variable defined in my_mod you'll need to add it to the import statement as well, however on the bright side, this approach would make you sure you don't import clashing variables (as while adding it to import statement you can cross-check with other imports, and also it will make sure that you don't end up importing some variables that you don't want to import.
Nonetheless, if you are building your own small project you might indeed end up using from my_mod import * just be aware of the drawbacks mentioned above.
On the execution while importing
The fact that some (non-trivial) code gets executed during import statement suggests bad code design. During the import call, the only code that gets "executed" is the code written in the "top scope". That is, if there are some functions etc. these function (naturally) don't get executed, just... defined. However if you have some "direct" code in the "top scope", for example:
print(42)
this code will get executed (and prints 42) when first encountered appropriate import statement.
In most cases in this "top scope" should be just only class/function definitions and occasionally some global variables. All other code should be enclosed in some function or class or method... exactly for the reason, so it wouldn't be executed at the import statement. So if you have some code, which you wish to execute at specific moment, wrap that part of the code in a function a call that function (from the other module), rather than relying on the import statement to execute the code.
Note that there are some exceptions to aforementioned "rule". Most notably, if you want some code to gets executed when you call the file via python3 my_file.py you need to put the code you wish to be executed in the "top level". However even in this case this code should be put within if __name__ == '__main__': guard for this reason -- not to execute the code on import (just when "calling the file" directly), additionally, it is a good practice to have a very short code within this guard (for example just calling some function)
for namespace you can check python document
and for module put command in terminal
pip install (module name)
example:-
pip install flask
#the flask module is automatically download in your system
What is the best way to put variables in python for multiple scripts?
I have the following scripts:
main.py: This script to declare common variables for one.py, two.py, and three.py. Also takes input from the user in interactive way and from the defined input file.
config.py: This script has all three scripts configurations inputs like ip-adress, ports details. etc.
one.py: I have written this script with class and it take inputs from main.py and config.py.
two.py: I have written this script with class and it take inputs from main.py and config.py.
three.py: I have written this script with class and it take inputs from main.py and config.py.
run.py: This script loads class objects for the one.py, two.py, and three.py and calls the functions defined in them.
main.py
list1 = []
a = "common for one.py,two.py and three.py"
one.py
import main
main.list1.append("1")
two.py
import main
main.list1.append("2")
three.py
import main
main.list1.append("3")
Running script:
run.py
import main
import one
import two
import three
# start performing
#call one script functions
#call two script functions
#call three script functions
I need some suggestions from you.
Am I am doing it in right way?
Will my scripts be easy for future maintenance?
Since I am putting variable list1 in main.py. It will first
appending some value in one.py and then two.py is going to access it.
My doubt is two.py is importing main.py in this case it does not
overwrite with empty list value? I have tested it, and it works fine.
But I still want to take your suggestions for best practices.
Yes, I think what you're doing is basically OK. I would rename a couple of the scripts to better indicate their roles: I would rename run.py and make it main.py, and make what you called main.py something like common.py. However, for the sake of discussion, I'll use the script names you have in your question.
You don't have to worry about what's in your main.py reinitializing list1 variable every time it's imported. That's because the results of importing modules are "cached" in sys.modules and the top-level code in them only executes the first time they're imported. If another script imports one of them again, the results of the first time saved in cached version is used. You can think of loaded modules as "singleton" instances of a module class (which in fact they are, see types.ModuleType).
If there's a dependency in your run.py that the modules one, two, and three are imported in a certain order, then I would import them all in your main module, so other scripts don't have to worry about it (the ordering). That way other scripts can just import whichever ones they want to use.
The above is not to say your design couldn't be improved. It looks like you're basically using global variables, which have long been considered a "bad thing" for a number of reasons (see Global Variables Are Bad). To avoid that, the direct use of list1 would need to be replaced with an instance of some new custom class with it's lifetime and access to it handled indirectly via methods defined in the class after explicitly creating an instance of it. This instance would then need to be passed as argument to any function or method that needed use or otherwise do something with or to the instance.
This doesn't look easy to maintain, since the values appended to the list only happen upon calling the import in main. Import them in the wrong order, or forget to import one of them, and you will get unexpected results.
In general, it is fine to share variables between scripts, but the various modules should not edit the values of the shared variables, rather, each module should be solely responsible for the values of its variables.
How does one get (finds the location of) the dynamically imported modules from a python script ?
so, python from my understanding can dynamically (at run time) load modules.
Be it using _import_(module_name), or using the exec "from x import y", either using imp.find_module("module_name") and then imp.load_module(param1, param2, param3, param4) .
Knowing that I want to get all the dependencies for a python file. This would include getting (or at least I tried to) the dynamically loaded modules, those loaded either by using hard coded string objects or those returned by a function/method.
For normal import module_name and from x import y you can do either a manual scanning of the code or use module_finder.
So if I want to copy one python script and all its dependencies (including the custom dynamically loaded modules) how should I do that ?
You can't; the very nature of programming (in any language) means that you cannot predict what code will be executed without actually executing it. So you have no way of telling which modules could be included.
This is further confused by user-input, consider: __import__(sys.argv[1]).
There's a lot of theoretical information about the first problem, which is normally described as the Halting problem, the second just obviously can't be done.
From a theoretical perspective, you can never know exactly what/where modules are being imported. From a practical perspective, if you simply want to know where the modules are, check the module.__file__ attribute or run the script under python -v to find files when modules are loaded. This won't give you every module that could possibly be loaded, but will get most modules with mostly sane code.
See also: How do I find the location of Python module sources?
This is not possible to do 100% accurately. I answered a similar question here: Dependency Testing with Python
Just an idea and I'm not sure that it will work:
You could write a module that contains a wrapper for __builtin__.__import__. This wrapper would save a reference to the old __import__and then assign a function to __builtin__.__import__ that does the following:
whenever called, get the current stacktrace and work out the calling function. Maybe the information in the globals parameter to __import__ is enough.
get the module of that calling functions and store the name of this module and what will get imported
redirect the call the real __import__
After you have done this you can call your application with python -m magic_module yourapp.py. The magic module must store the information somewhere where you can retrieve it later.
That's quite of a question.
Static analysis is about predicting all possible run-time execution paths and making sure the program halts for specific input at all.
Which is equivalent to Halting Problem and unfortunately there is no generic solution.
The only way to resolve dynamic dependencies is to run the code.
each unit test I'm running is writing python code out to a file, then importing it as a module. The problem is that the code changes but further import statements don't modify the module.
I think what I need is a way to ether force a reload on a module or clear the internal bytecode cache. Any ideas?
Thanks!
Reimporting modules is tricky to get all the edge cases right. The documentation for reload mentions some of them. Depending on what you are testing, you may be better off by testing the imports with separate invocations of the interpreter by running each via, say, subprocess. It will likely be slower but also likely safer and more accurate testing.
Use reload().
Reload a previously imported module.
The argument must be a module object,
so it must have been successfully
imported before. This is useful if you
have edited the module source file
using an external editor and want to
try out the new version without
leaving the Python interpreter. The
return value is the module object (the
same as the module argument).
However, the module needs to be already loaded. A workaround is to handle the resulting NameError:
try:
reload(math)
except NameError:
import math
Write your code to differently-named modules. Writing new code into an existing file, and trying to import it again will not work well.
Alternatively, you can clobber sys.modules. For example:
class MyTestCase(unittest.TestCase):
def setUp(self):
# Record sys.modules here so we can restore it in tearDown.
self.old_modules = dict(sys.modules)
def tearDown(self):
# Remove any new modules imported during the test run. This lets us
# import the same source files for more than one test.
for m in [m for m in sys.modules if m not in self.old_modules]:
del sys.modules[m]
Ran into a similar situation.
Later on found that the white space indentation technique used matters.
Especially on windows platforms, ensure that a uniform technique is adapted
throughout the module i.e., either use tab or spaces exclusively.
I'm developing a Python application for the GAE.
The application consists of a bunch of classes and functions which are at the moment all in the same file main.py.
The application is running without problems.
Now, I want to refactor the application and outsource all the classes. Every class should be in her own file. The files shall be arranged in directories like this:
main.py
/directory1/class1.py
/directory1/class2.py
/directory2/class1.py
My problem is that inside these outsourced classes, I cannot use the functions of main.py.
I tried this inside the class-files.
from main import name_of_function
But the compiler says
from main import name_of_function
ImportError: cannot import name name_of_function
What did I wrong?
The name of the funktion is login. Maybe this causes the problem?
Try moving the extra functions from main.py into a separate file.
main.py
library.py # contains login() and other functions from main
/directory1/class1.py
/directory1/class2.py
/directory2/class1.py
Sometimes it is good to leave classes in same module not separate without purpose if they belong together.
The problem of using function from main is sign that you should refactor one module say common_utils.py out of those functions and separate it from main. You can import that to your modules, which use those. Do not think classes only think whole use case.
If you could give pseudo code of your program's logic, we could check the refactoring better together.