I am trying to have a module that provides a framework of sorts to short declarative one-file scripts that I write. In the short scripts, I would like to define a number of variables and functions that are accessed or called back by the framework module.
I am stuck ... I have tried two approaches, and I am not crazy about either one. Is there a better way?
First approach:
Framework module:
import __main__
def test():
print __main__.var
print __main__.func()
Script:
import framework
var="variable"
def func():
print "function"
framework.test()
I like this approach and it works, but eclipse gives me an 'Undefined variable from import: var' error on any imported variable or function from main. Obviously, this is not correct as it works, but it clutters up eclipse with many false errors.
Second Approach:
Framework module:
def test(glob):
print glob['var']
print glob['func']()
Script:
import framework
var="variable"
def func():
print "function"
framework.test(globals())
This seems to work and does not give me errors, but I don't like the dictionary type notation used to call variables and functions ... especially for functions: func'funcName'.
Is there a better way to implement this that results in clean code? Can I pass in a module name (ie main) as an argument to a function? Can Eclipse errors on main for the first approach be turned off?
I went with the first approach and told Eclipse to ignore the Undefined Variable from Import error by pressing command+1 (for my Mac, for Windows it would be ctrl+1) on each line where this error occurred.
command+1 adds the following comment to the end of each line:
# #UndefinedVariable
Thanks to #ekhumoro for pointing this out in the comments.
Related
I have two python files. One is main.py which I execute. The other is test.py, from which I import a class in main.py.
Here is the sample code for main.py:
from test import Test
if __name__ == '__main__':
print("You're inside main.py")
test_object = Test()
And, here is the sample code for test.py:
class Test:
def __init__(self):
print("you're initializing the class.")
if __name__ == '__main__':
print('You executed test.py')
else:
print('You executed main.py')
Finally, here's the output, when you execute main.py:
You executed main.py
You're inside main.py
you're initializing the class.
From the order of outputs above, you can see that once you import a piece of a file, the whole file gets executed immediately. I am wondering why? what's the logic behind that?
I am coming from java language, where all files included a single class with the same name. I am just confused that why python behaves this way.
Any explanation would be appricated.
What is happening?
When you import the test-module, the interpreter runs through it, executing line by line. Since the if __name__ == '__main__' evaluates as false, it executes the else-clause. After this it continues beyond the from test import Test in main.py.
Why does python execute the imported file?
Python is an interpreted language. Being interpreted means that the program being read and evaluated one line at the time. Going through the imported module, the interpreter needs to evaluate each line, as it has no way to discern which lines are useful to the module or not. For instance, a module could have variables that need to be initialized.
Python is designed to support multiple paradigms. This behavior is used in some of the paradigms python supports, such as procedural programming.
Execution allows the designer of that module to account for different use cases. The module could be imported or run as a script. To accommodate this, some functions, classes or methods may need to be redefined. As an example, a script could output non-critical errors to the terminal, while an imported module to a log-file.
Why specify what to import?
Lets say you are importing two modules, both with a Test-class. If everything from those modules is imported, only one version of the Test-class can exist in our program. We can resolve this issue using different syntax.
import package1
import package2
package1.Test()
packade2.Test()
Alternatively, you can rename them with the as-keyword.
from package1 import Test
from package2 import Test as OtherTest
Test()
OtherTest()
Dumping everything into the global namepace (i.e from test import *) pollutes the namespace of your program with a lot of definitions you might not need and unintentionally overwrite/use.
where all files included a single class with the same name
There is not such requirement imposed in python, you can put multiple classes, functions, values in single .py file for example
class OneClass:
pass
class AnotherClass:
pass
def add(x,y):
return x+y
def diff(x,y):
return x-y
pi = 22/7
is legal python file.
According to interview with python's creator modules mechanism in python was influenced by Modula-2 and Modula-3 languages. So maybe right question is why creators of said languages elected to implement modules that way?
The question is pretty simple: I need to move a piece of my code to the another file. In the main file, I'll do something like import not_main. Is is possible to run imported code as a part of the main file? Here's a simplified example:
__code
\__main.py
|__not_main.py
main.py content:
a = 5
import not_main
not_main.py content:
print a
When I'm running main.py, there is a error : NameError: name 'a' is not defined. How can I make it work? Thanks for any tips.
It's not possible to directly reference a variable in one module that's defined in another module unless you import that module first. Doesn't matter that it's the main module, this error would happen if you tried to do the same between any two modules. An imported module does not gain the scope of the module it's imported into.
There are possible workarounds, though I would caution using them if the code you're trying to separate out is fairly complex. But if you're intent on doing it, I'd suggest separating out any variables needed in both modules into a third module that only contains those variables. So the simple example you gave would turn into this:
cross_module_variables.py:
a = 5
main.py:
import not_main
not_main.py:
import cross_module_variables as cmv
print cmv.a
For more complex code you might need to assign the value of the variable in main after doing executing some code to produce the value. In that case you'll want to import cross_module_variables into the main module and assign a value to it. Course that variable has to be instantiated before it can be used in main so you'll have define the variable in cross_module_variable with some default value. So it would look something more like this:
cross_module_variables.py:
a = 0
main.py:
import cross_module_variables as cmv
cmv.a = 5
import not_main
not_main.py:
import cross_module_variables as cmv
print cmv.a
See this answer for more info on cross module variables.
With all that said, I would highly suggest you look at restructuring your code in some other sane way. It sounds like you're running all your code straight in modules instead of defining functions around discrete sections of code. You should look into ways of designing coherent functional programs.
I'm teaching myself Python (I have experience in other languages).
I found a way to import a "module". In PHP, this would just be named an include file. But I guess Python names it a module. I'm looking for a simple, best-practices approach. I can get fancy later. But right now, I'm trying to keep it simple while not developing bad habits. Here is what I did:
I created a blank file named __init__.py, which I stored in Documents (the folder on the Mac)
I created a file named myModuleFile.py, which I stored in Documents
In myModuleFile.py, I created a function:
def myFunction()
print("hello world")
I created another file: myMainFile.py, which I stored in Documents
In this file, I typed the following:
import myModuleFile.py
myModuleFile.myFunction()
This successfully printed out "hello world" to the console when I ran it on the terminal.
Is this a best-practices way to do this for my simple current workflow?
I'm not sure the dot notation means I'm onto something good or something bad. It throws an error if I try to use myFunction() instead of myModuleFile.myFunction(). I kind of think it would be good. If there were a second imported module, it would know to call myFunction() from myModuleFile rather than the other one. So the dot notation makes everybody know exactly which file you are trying to call the function from.
I think there is some advanced stuff using sys or some sort of exotic configuration stuff. But I'm hoping my simple little way of doing things is ok for now.
Thanks for any clarification on this.
For your import you don't need the ".py" extension
You can use:
import myModuleFile
myModuleFile.myFunction()
Or
from myModuleFile import myFunction
myFunction()
Last syntax is common if you import several functions or globals of your module.
Besides to use the "main" function, I'd put this on your module:
from myModuleFile import myFunction
if __name__ == '__main__':
myFunction()
Otherwise the main code could be executed in imports or other cases.
I'd use just one module for myModuleFile.py and myMainFile.py, using the previous pattern let you know if your module is called from command line or as import.
Lastly, I'd change the name of your files to avoid the CamelCase, that is, I'd replace myModuleFile.py by my_module.py. Python loves the lowercase ;-)
You only need to have init.py if you are creating a package (a package in a simple sense is a subdirectory which has one or more modules in it, but I think it may be more complex than you need right now).
If you have just one folder which has MyModule.py and MyMainFile.py - you don't need the init.py.
In MyMainFile.py you can write :
import myModuleFile
and then use
myModuleFile.MyFunction()
The reason for including the module name is that you may reuse the same function name in more than one module and you need a way of saying which module your program is using.
Module Aliases
If you want to you can do this :
import myModuleFile as MyM
and then use
MyM.MyFunction()
Here you have created MyM as an alias for myModuleFile, and created less typing.
Here Lies Dragons
You will sometimes see one other forms of IMport, which can be dangerous, especially for the beginner.
from myModuleFile import MyFunction
if you do this you can use :
MyFunction()
but this has a problem if you have used the same function name in MyMainFile, or in any other library you have used, as you now can't get to any other definition of the name MyFunction. This is often termed Contaminating the namespace - and should really be avoided unless you are absolutely certain it is safe.
there is a final form which I will show for completeness :
from myModuleFile import *
While you will now be able to access every function defined in myModuleFile without using myModuleFile in front of it, you have also now prevented your MyMainFile from using any function in any library which matches any name defined in myModuleFile.
Using this form is generally not considered to be a good idea.
I hope this helps.
I understand how to actually link python files, however, i don't understand how to get variable's from these linked files. I've tried to grab them and I keep getting NameError.
How do I go about doing this? The reason i want to link files is to simply neaten up my script and not make it 10000000000 lines long. Also, in the imported python script, do i have to import everything again? Another question, do i use the self function when using another scripts functions?
ie;
Main Script:
import sys, os
import importedpyfile
Imported Py File
import sys, os
I understand how to actually link python files, however, i don't
understand how to get variable's from these linked files. I've tried
to grab them and I keep getting NameError.
How are you doing that? Post more code. For instance, the following works:
file1.py
#!/usr/bin/env python
from file2 import file2_func, file2_variable
file2_func()
print file2_variable
file2.py:
#!/usr/bin/env python
file2_variable = "I'm a variable!"
def file2_func():
print "Hello World!"
Also, in the imported python script, do i have to import everything
again?
Nope, modules should be imported when the python interpreter reads that file.
Another question, do i use the self function when using another
scripts functions?
Nope, that's usually to access class members. See python self explained.
There is also more than one way to import files. See the other answer for some explanations.
I think what you are trying to ask is how to get access to global vars from on .py file without having to deal with namespaces.
In your main script, replace the call to import importedpyfile to say this instead
from importedpyfile import *
Ideally, you keep the code the way you have it. But instead, just reference those global vars with the importedpyfile namespace.
e.g.
import importedpyfile
importedpyfile.MyFunction() # calls "MyFunction" that is defined in importedpyfile.py
Python modules are not "linked" in the sense of C/C++ linking libraries into an executable. A Python import operation creates a name that refers to the imported module; without this name there is no (direct) way to access another module.
I'm working on pypreprocessor which is a preprocessor that takes c-style directives and I've been able to make it work like a traditional preprocessor (it's self-consuming and executes postprocessed code on-the-fly) except that it breaks library imports.
The problem is: The preprocessor runs through the file, processes it, outputs to a temporary file, and exec() the temporary file. Libraries that are imported need to be handled a little different, because they aren't executed, but rather they are loaded and made accessible to the caller module.
What I need to be able to do is: Interrupt the import (since the preprocessor is being run in the middle of the import), load the postprocessed code as a tempModule, and replace the original import with the tempModule to trick the calling script with the import into believing that the tempModule is the original module.
I have searched everywhere and so far and have no solution.
This Stack Overflow question is the closest I've seen so far to providing an answer:
Override namespace in Python
Here's what I have.
# Remove the bytecode file created by the first import
os.remove(moduleName + '.pyc')
# Remove the first import
del sys.modules[moduleName]
# Import the postprocessed module
tmpModule = __import__(tmpModuleName)
# Set first module's reference to point to the preprocessed module
sys.modules[moduleName] = tmpModule
moduleName is the name of the original module, and tmpModuleName is the name of the postprocessed code file.
The strange part is this solution still runs completely normal as if the first module completed loaded normally; unless you remove the last line, then you get a module not found error.
Hopefully someone on Stack Overflow know a lot more about imports than I do, because this one has me stumped.
Note: I will only award a solution, or, if this is not possible in Python; the best, most detailed explanation of why this is not impossible.
Update: For anybody who is interested, here is the working code.
if imp.lock_held() is True:
del sys.modules[moduleName]
sys.modules[tmpModuleName] = __import__(tmpModuleName)
sys.modules[moduleName] = __import__(tmpModuleName)
The 'imp.lock_held' part detects whether the module is being loaded as a library. The following lines do the rest.
Does this answer your question? The second import does the trick.
Mod_1.py
def test_function():
print "Test Function -- Mod 1"
Mod_2.py
def test_function():
print "Test Function -- Mod 2"
Test.py
#!/usr/bin/python
import sys
import Mod_1
Mod_1.test_function()
del sys.modules['Mod_1']
sys.modules['Mod_1'] = __import__('Mod_2')
import Mod_1
Mod_1.test_function()
To define a different import behavior or to totally subvert the import process you will need to write import hooks. See PEP 302.
For example,
import sys
class MyImporter(object):
def find_module(self, module_name, package_path):
# Return a loader
return self
def load_module(self, module_name):
# Return a module
return self
sys.meta_path.append(MyImporter())
import now_you_can_import_any_name
print now_you_can_import_any_name
It outputs:
<__main__.MyImporter object at 0x009F85F0>
So basically it returns a new module (which can be any object), in this case itself. You may use it to alter the import behavior by returning processe_xxx on import of xxx.
IMO: Python doesn't need a preprocessor. Whatever you are accomplishing can be accomplished in Python itself due to it very dynamic nature, for example, taking the case of the debug example, what is wrong with having at top of file
debug = 1
and later
if debug:
print "wow"
?
In Python 2 there is the imputil module that seems to provide the functionality you are looking for, but has been removed in python 3. It's not very well documented but contains an example section that shows how you can replace the standard import functions.
For Python 3 there is the importlib module (introduced in Python 3.1) that contains functions and classes to modify the import functionality in all kinds of ways. It should be suitable to hook your preprocessor into the import system.