Load module from python string without executing code - python

I'm trying to do something similar with: Load module from string in python
Sometimes I read source code provided by the user:
module = imp.new_module('AppModule')
exec_(s, module.__dict__)
sys.modules['AppModule'] = module
return module
And sometimes import from a file
module = __import__(module_name, fromlist=_fromlist)
return module
These modules are loaded to map objects and it'll be used inside a IDE.
My problem is: If there is any method call outside if __name__ == '__main__': this code is being executed and can interact with the IDE.
How can I import modules ignoring methods from being executed?

The process of importing a module requires that its code be executed. The interpreter creates a new namespace and populates it by executing the module's code with the new namespace as the global namespace, after which you can access those values (remember that the def and class statements are executable).
So maybe you will have to educate your users not to write modules that interact with the IDE?

Related

Why do you run other lines of codes of a python file(say test.py) when you are just importing a piece of test.py from somewhere else (say main.py)?

I have two python files. One is main.py which I execute. The other is test.py, from which I import a class in main.py.
Here is the sample code for main.py:
from test import Test
if __name__ == '__main__':
print("You're inside main.py")
test_object = Test()
And, here is the sample code for test.py:
class Test:
def __init__(self):
print("you're initializing the class.")
if __name__ == '__main__':
print('You executed test.py')
else:
print('You executed main.py')
Finally, here's the output, when you execute main.py:
You executed main.py
You're inside main.py
you're initializing the class.
From the order of outputs above, you can see that once you import a piece of a file, the whole file gets executed immediately. I am wondering why? what's the logic behind that?
I am coming from java language, where all files included a single class with the same name. I am just confused that why python behaves this way.
Any explanation would be appricated.
What is happening?
When you import the test-module, the interpreter runs through it, executing line by line. Since the if __name__ == '__main__' evaluates as false, it executes the else-clause. After this it continues beyond the from test import Test in main.py.
Why does python execute the imported file?
Python is an interpreted language. Being interpreted means that the program being read and evaluated one line at the time. Going through the imported module, the interpreter needs to evaluate each line, as it has no way to discern which lines are useful to the module or not. For instance, a module could have variables that need to be initialized.
Python is designed to support multiple paradigms. This behavior is used in some of the paradigms python supports, such as procedural programming.
Execution allows the designer of that module to account for different use cases. The module could be imported or run as a script. To accommodate this, some functions, classes or methods may need to be redefined. As an example, a script could output non-critical errors to the terminal, while an imported module to a log-file.
Why specify what to import?
Lets say you are importing two modules, both with a Test-class. If everything from those modules is imported, only one version of the Test-class can exist in our program. We can resolve this issue using different syntax.
import package1
import package2
package1.Test()
packade2.Test()
Alternatively, you can rename them with the as-keyword.
from package1 import Test
from package2 import Test as OtherTest
Test()
OtherTest()
Dumping everything into the global namepace (i.e from test import *) pollutes the namespace of your program with a lot of definitions you might not need and unintentionally overwrite/use.
where all files included a single class with the same name
There is not such requirement imposed in python, you can put multiple classes, functions, values in single .py file for example
class OneClass:
pass
class AnotherClass:
pass
def add(x,y):
return x+y
def diff(x,y):
return x-y
pi = 22/7
is legal python file.
According to interview with python's creator modules mechanism in python was influenced by Modula-2 and Modula-3 languages. So maybe right question is why creators of said languages elected to implement modules that way?

Python C extension importer how to use import lock

Background
I'm building an importer to encrypt my code. When compiling, I encrypt the code object in pyc files. When loading, I use this customized importer to decrypt the code object before execution.
Since my code is bundled in zip format, I decided to modify zipimporter from the builtin zipimport package by adding the decrypt logic into it.
I'm using Python 3.7
Problem
I modified the zimpimport.c and make it into C extension. The encrypt & decrypt process works fine but I started to see ImportError & AttributeError. For example:
"""module: test.py"""
class Foo:
def bar():
pass
foo = Foo()
// Import Error: can not import name 'foo' from test
// Attribute Error: foo has no attribute bar
When server starts, this error only occurs during the first few minutes randomly. So I suspect that it's a multithreading problem and it's caused by a thread seeing a partially initialized module object.
What I tried
After checking the origin c code, the implementation for loading a module is:
mod = PyImport_AddModuleObject(fullname);
// ... some code
mod = PyImport_ExecCodeModuleObject(fullname, code, modpath, NULL);
It first add module to sys.modules, then execute some c code and execute the module code at last. If another thread read the sys.modules before it finishes executing module code, it can get the ImportError and AttributeError.
I just copy the original implementation and it doesn't use import lock. I guess it's handled by interpreter on outer level and I have to explicitly use import lock. So I wrap the code block with the import lock as following:
#include "import.h"
_PyImport_AcquireLock();
// wrapped code block
_PyImport_ReleaseLock();
My Question
After adding import lock, the error still happens. I'm not familiar with python c api. Have I used the lock in a correct way? Are there other possible reasons that can cause this error?
Code
To test, put the following files in same folder:
zzimporter.c lock added in func zipimport_zipimporter_load_module_impl
zzimporter.h
setup.py
Run python setup.py build_ext --inplace and it will generate zzimporter.so. And it works just like the buit-in zipimport module.

Python equivalent of lua's loadfile

In lua there is a function called loadfile, this function allows the program to parse a .lua file into the current project. All the functions and variables work as if they were written in the same file.
This is useful for loading plugins. I am trying to port a telegram bot over to python but cannot find a function that allows me to load a .py file and have the functions be in the context of the file.
I have tried python's execfile and importing the file but that doesn't allow for the functions of the loaded file to be in scope of the initial file.
(i.e. fileA.py loads fileB.py. fileA has function "doThis". fileB can't access "doThis" using execfile)
How can I achieve the same thing in python as loadfile for lua?
I am using python 2
You generally shouldn't do this, since wildcard imports can make code harder to debug, but:
from other_module import *
This imports everything defined in other_module into the current module's global namespace. It's not exactly the same, but it's as close as you'll get.
execfile("other_module.py", globals())
is very similar, except it won't cache the module in sys.modules.

Python Idle and Terminal Import Differences

I just started using Python and I have a question about idle vs terminal.
In idle, I made a file called Robot.py
I have a class called Robot
class Robot(object)
def __init__(self,x,y):
#some code here etc...
def HelloWorld()
print "Hello World!"
I have another file called testrobot.py, which looks like so:
import Robot
r = Robot(1,4)
In idle, I am able to successfully create a Robot object when I run testrobot.py. However in terminal, it gives an error Message NameError: Robot is not defined
I'm not sure how to run my program in terminal.
Also:
How can I call my HelloWorld() function which is in Robots.py but not of the class Robot in an external file (such as testrobot.py)?
Thanks in advance!
When you load and run scripts in IDLE, they are automatically loaded for the interpreter. That means that as soon as you run the script in IDLE, the Python shell already has those types defined.
When you want to run it from outside of IDLE, i.e. without running the module first, you need to import your Robot from that module. To do that, you import the module, not the type:
import Robot
myRobot = Robot.Robot(...)
Or, if you want to use Robot directly, you need to use the from ... import syntax:
from Robot import Robot
myRobot = Robot(...)
Similarily, you can call your function by using Robot.HelloWorld in the first case, or directly if you add HelloWorld to the import list in the second case:
from Robot import Robot, HelloWorld
myRobot = Robot(...)
HelloWorld()
As you can see, it is generally a good idea to name your files in lower case, as those are the module names (or “namespaces” in other languages).
You are trying to create a class of an import, not the class imported.
You are trying to import a file called Robot.py with that import statement. To import your robot class you will have to type import Robot and then write Robot.Robot(1,4) to create an object of it. Alternatively you can import all references from the file: from Robot import * and then write Robot(1,4).
The reason it will work with IDLE is that it basically imports everything from the file you run, allowing you use the methods and classes in that file.

Import class without executing .py it is in?

I defined a class called Prof in a script called AddPntCode90_27.py. It opens some files, does some math, creates output files and so. Now I want to re-use the class for another programme. But as AddPntCode90_27.py is not a module it always executes the script instead of just importing the class.
I did from AddPntCode90_27 import * as well as from AddPntCode90_27 import Prof.
I am familiar with this article. So my questions are:
is it bad practice to define a class within a script like this? Should I always keep them in a separated file?
is there, however, a way to import just the class and its methods without executing the script it is defined in?
Ah, I'm running Python 2.7.
The way to do what you want is to use an if __name__ == "__main__" block. See this question.
It's perfectly fine to define classes in scripts, but you cannot import the class without executing the script, because it is only by executing the script that you define the class. Class definitions are not a "compile-time declaration" in Python; they are executed in order just like everything else in the module. You should use an if __name__=="__main__" block to protect code that you don't want to be run when you import your file as a module.
You should the if __name__="__main__: idiom to check whether Python is running the code or the code is being imported as a module.

Categories