Include/embed python file "as is" - python

For better code readability, let's suppose that I've got my code logically splitted in different python files, my main main.py file and an included.py file. I'd like to embed included.py inside of the main file.
I know it's possible to import it on main.py (e.g. from included import *), but, in some cases, objects/classes/variables on the main.py file may be imported on the included.py file too (e.g. from main import *).
What I'd like to do is to import the included.py file "as is" (like PHP does, with the include statement) on a specific position of my main.py file, by forcing the interpreter to read the content of the file and to place it on the specified position.
Is it possible?

In Python 2, try the execfile() command:
execfile('included.py')
In Python 3, try
exec(compile(open('included.py', "rb").read(), 'included.py', 'exec'))
I performed a simple test and this seems to work. Does it help in your case?
For more 'traditional' import ways:
https://stackoverflow.com/a/20749411/5172579
For the relation of Python2's execfile() and Python3's exec():
https://stackoverflow.com/a/6357418/5172579
p.s. as noted in the comment: Cyclic imports should be avoided in general and can lead easily to infinite import recursions.

Related

Add a module when running a python program from terminal

I wrote a module that, if it is imported, automatically changes the error output of my program. It is quite handy to have it in almost any python code I write.
Thus I don't want to add the line import my_errorhook to every code I write but want to have this line added automatically.
I found this answer, stating that it should be avoided to change the behavior of python directly. So I thought about changing the command line, something like
python --importModule my_errorhook main.py
and defining an alias in the bashrc to overwrite the python command to automatically add the parameter. Is there any way I could achieve such a behavior?
There is no such thing like --importModule in python command line. The only way you can incept the code without explicitly importing is by putting your functions in builtins module. However, this is a practice that is discouraged because it makes your code hard to maintain without proper design.
Let's assume that your python file main.py is the entry point of the whole program. Now you can create another file bootstrap.py, and put below codes into the new file.
import main
__builtins__.func = lambda x: x>=0
main.main()
Then the function func() can be called from all modules without being imported. For example in main.py
def main():
...
print(func(1))
...

How can I import a .pyc compiled python file and use it

Im trying to figure out how to include a .pyc file in a python script.
For example my script is called:
myscript.py
and the script I would like to include is called:
included_script.pyc
So, do I just use:
import included_script
And will that automatically execute the included_script.pyc ? Or is there something further I need to do, to get my included_script.pyc to run inside the myscript.py?
Do I need to pass the variables used in included_script.pyc also? If so, how might this be achieved?
Unfortunately, no, this cannot be done automatically. You can, of course, do it manually in a gritty ugly way.
Setup:
For demonstration purposes, I'll first generate a .pyc file. In order to do that, we first need a .py file for it. Our sample test.py file will look like:
def foo():
print("In foo")
if __name__ == "__main__":
print("Hello World")
Super simple. Generating the .pyc file can done with the py_compile module found in the standard library. We simply pass in the name of the .py file and the name for our .pyc file in the following way:
py_compile.compile('test.py', 'mypyc.pyc')
This will place mypyc.pyc in our current working directory.
Getting the code from .pyc files:
Now, .pyc files contain bytes that are structured in the following way:
First 4 bytes signalling a 'magic number'
Next 4 bytes holding a modification timestamp
Rest of the contents are a marshalled code object.
What we're after is that marshalled code object, so we need to import marshal to un-marshall it and execute it. Additionally, we really don't care/need the 8 first bytes, and un-marshalling the .pyc file with them is disallowed, so we'll ignore them (seek past them):
import marshal
s = open('mypyc.pyc', 'rb')
s.seek(8) # go past first eight bytes
code_obj = marshal.load(s)
So, now we have our fancy code object for test.py which is valid and ready to be executed as we wish. We have two options here:
Execute it in the current global namespace. This will bind all definitions inside our .pyc file in the current namespace and will act as a sort of: from file import * statement.
Create a new module object and execute the code inside the module. This will be like the import file statement.
Emulating from file import * like behaviour:
Performing this is pretty simple, just do:
exec(code_obj)
This will execute the code contained inside code_obj in the current namespace and bind everything there. After the call we can call foo like any other funtion:
foo()
# prints: In foo!
Note: exec() is a built-in.
Emulating import file like behaviour:
This includes another requirement, the types module. This contains the type for ModuleType which we can use to create a new module object. It takes two arguments, the name for the module (mandatory) and the documentation for it (optional):
m = types.ModuleType("Fancy Name", "Fancy Documentation")
print(m)
<module 'Fancy Name' (built-in)>
Now that we have our module object, we can again use exec to execute the code contained in code_obj inside the module namespace (namely, m.__dict__):
exec(code_obj, m.__dict__)
Now, our module m has everything defined in code_obj, you can verify this by running:
m.foo()
# prints: In foo
These are the ways you can 'include' a .pyc file in your module. At least, the ways I can think of. I don't really see the practicality in this but hey, I'm not here to judge.

How to import a .py file in all files?

Is it possible to import a test.py file in whole project instead of importing it in each file?
for example calling it some where in start.py
No.
However, suppose you have three modules, a.py, b.py, and c.py. Further, assume that c.py imports b, and b.py imports a. Then in c.py, you can refer to items in a using
b.a.foo
since b imported a into its namespace.
even if it is ... (which in programming is almost always the case) a better question is why would you want to.
It would be difficult to do, and it adds nothing... in fact it can severely hinder development, and you may experience very strange bugs that become very hard to track down
[edit based on OP comment]
your use case sounds like you just want it to be in builtins
start.py
import __builtin__
import my_weird_test_class
__builtin__.tester = my_weird_test_class
from main_entry import main
main()
now in any file you can use
tester.do_something()
without importing tester or whatever
but as I said this tends to be a very bad idea... it is much better to explicitly import it in all your files ...

python module import syntax

I'm teaching myself Python (I have experience in other languages).
I found a way to import a "module". In PHP, this would just be named an include file. But I guess Python names it a module. I'm looking for a simple, best-practices approach. I can get fancy later. But right now, I'm trying to keep it simple while not developing bad habits. Here is what I did:
I created a blank file named __init__.py, which I stored in Documents (the folder on the Mac)
I created a file named myModuleFile.py, which I stored in Documents
In myModuleFile.py, I created a function:
def myFunction()
print("hello world")
I created another file: myMainFile.py, which I stored in Documents
In this file, I typed the following:
import myModuleFile.py
myModuleFile.myFunction()
This successfully printed out "hello world" to the console when I ran it on the terminal.
Is this a best-practices way to do this for my simple current workflow?
I'm not sure the dot notation means I'm onto something good or something bad. It throws an error if I try to use myFunction() instead of myModuleFile.myFunction(). I kind of think it would be good. If there were a second imported module, it would know to call myFunction() from myModuleFile rather than the other one. So the dot notation makes everybody know exactly which file you are trying to call the function from.
I think there is some advanced stuff using sys or some sort of exotic configuration stuff. But I'm hoping my simple little way of doing things is ok for now.
Thanks for any clarification on this.
For your import you don't need the ".py" extension
You can use:
import myModuleFile
myModuleFile.myFunction()
Or
from myModuleFile import myFunction
myFunction()
Last syntax is common if you import several functions or globals of your module.
Besides to use the "main" function, I'd put this on your module:
from myModuleFile import myFunction
if __name__ == '__main__':
myFunction()
Otherwise the main code could be executed in imports or other cases.
I'd use just one module for myModuleFile.py and myMainFile.py, using the previous pattern let you know if your module is called from command line or as import.
Lastly, I'd change the name of your files to avoid the CamelCase, that is, I'd replace myModuleFile.py by my_module.py. Python loves the lowercase ;-)
You only need to have init.py if you are creating a package (a package in a simple sense is a subdirectory which has one or more modules in it, but I think it may be more complex than you need right now).
If you have just one folder which has MyModule.py and MyMainFile.py - you don't need the init.py.
In MyMainFile.py you can write :
import myModuleFile
and then use
myModuleFile.MyFunction()
The reason for including the module name is that you may reuse the same function name in more than one module and you need a way of saying which module your program is using.
Module Aliases
If you want to you can do this :
import myModuleFile as MyM
and then use
MyM.MyFunction()
Here you have created MyM as an alias for myModuleFile, and created less typing.
Here Lies Dragons
You will sometimes see one other forms of IMport, which can be dangerous, especially for the beginner.
from myModuleFile import MyFunction
if you do this you can use :
MyFunction()
but this has a problem if you have used the same function name in MyMainFile, or in any other library you have used, as you now can't get to any other definition of the name MyFunction. This is often termed Contaminating the namespace - and should really be avoided unless you are absolutely certain it is safe.
there is a final form which I will show for completeness :
from myModuleFile import *
While you will now be able to access every function defined in myModuleFile without using myModuleFile in front of it, you have also now prevented your MyMainFile from using any function in any library which matches any name defined in myModuleFile.
Using this form is generally not considered to be a good idea.
I hope this helps.

how to execute nested python files

I have 3 python files.(first.py, second.py, third.py) I'm executing 2nd python file from the 1st python file. 2nd python file uses the 'import' statement to make use of 3rd python file. This is what I'm doing.
This is my code.
first.py
import os
file_path = "folder\second.py"
os.system(file_path)
second.py
import third
...
(rest of the code)
third.py (which contains ReportLab code for generating PDF )
....
canvas.drawImage('xyz.jpg',0.2*inch, 7.65*inch, width=w*scale, height=h*scale)
....
when I'm executing this code, it gives error
IOError: Cannot open resource "xyz.jpg"
But when i execute second.py file directly by writing python second.py , everything works fine..!!
Even i tried this code,
file_path = "folder\second.py"
execfile(file_path)
But it gives this error,
ImportError: No module named third
But as i stated everything works fine if i directly execute the second.py file. !!
why this is happening? Is there any better idea for executing such a kind of nested python files?
Any idea or suggestions would be greatly appreciated.
I used this three files just to give the basic idea of my structure. You can consider this flow of execution as a single process. There are too many processes like this and each file contains thousandth lines of codes. That's why i can't change the whole code to be modularize which can be used by import statement. :-(
So the question is how to make a single python file which will take care of executing all the other processes. (If we are executing each process individually, everything works fine )
This should be easy if you do it the right way. There's a couple steps that you can follow to set it up.
Step 1: Set your files up to be run or imported
#!/usr/bin/env python
def main():
do_stuff()
if __name__ == '__main__':
The __name__ special variable will contain __main__ when invoked as a script, and the module name if imported. You can use that to provide a file that can be used either way.
Step 2: Make your subdirectory a package
If you add an empty file called __init__.py to folder, it becomes a package that you can import.
Step 3: Import and run your scripts
from folder import first, second, third
first.main()
second.main()
third.main()
The way you are doing thing is invalid.
You should: create a main application, and import 1,2,3.
In 1,2,3: You should define the things as your functions. Then call them from the main application.
IMHO: I don't need that you have much code to put into separate files, you just also put them into one file with function definitions and call them properly.
I second S.Lott: You really should rethink your design.
But just to provide an answer to your specific problem:
From what I can guess so far, you have second.py and third.py in folder, along with xyz.jpg. To make this work, you will have to change your working directory first. Try it in this way in first.py:
import os
....
os.chdir('folder')
execfile('second.py')
Try reading about the os module.
Future readers:
Pradyumna's answer from here solved Moin Ahmed's second issue for me:
import sys, change "sys.path" by appending the path during run
time,then import the module that will help
[i.e. sys.path.append(execfile's directory)]

Categories