I'm writing a Python program to execute embedded Python code in Verilog scripts. I thought of using the eval and exec functions, but I came across a problem: I would like to have all the execs and evals run in the shared context, without changing the main program's environment.
I'm putting exec and eval inside a function to call in the parsing routine:
# some parsing code
for embedded_code_string in list_of_embedded_code_strings:
execute_embedded_code(embedded_code_string)
# more parsing code
def execute_embedded_code(embedded_code_string):
exec(embedded_code_string)
# other routines involving io.StringIO for the redirection of stdout which isn't the problem.
If the first embedded code string to be run is row_len = 1, and the second one is column_len = row_len * 2, then when running the second code snippet, row_len would be undefined. It's expected: after all, exec is running in the context of the function execute_embedded_code, and after the function finishes, the variable would disappear from both locals() and globals().
It appears that you can set the local and global namespace for exec. However, the changes after running exec wouldn't be preserved in-place. (Corrections: the global_dict and local_dict parameter MUST be a dictionary, or it would be ignored. If it were a dictionary, it would be updated in-place.) Running globals() and locals() after exec would capture the change, but it would also capture the objects in the parsing program, and I wouldn't want the embedded code to inadvertently mess up the parsing program.
So my question is, how would I run many exec calls in their shared context, yet isolated enough that they wouldn't have unexpected consequences. No need to think of security, as all the embedded code to the run would be trusted.
I would like to get the individual outputs of each embedded code string, so I don't think joining them together and run it all at once would work.
You should be able to define your own shared globals that you pass to exec which is then modified by the embedded code:
def execute_embedded_code(embedded_code_string, shared_globals):
exec(embedded_code_string, shared_globals)
shared_globals = dict()
shared_globals['result'] = 0
sample_string = 'result += 1'
execute_embedded_code(sample_string, shared_globals)
print(shared_globals['result'])
Output
1
Note
To address a comment below, the documentation for exec states
If only globals is provided, it must be a dictionary, which will be used for both the global and the local variables.
Related
I'm facing some problems trying to load a full python script from my pastebin/github pages.
I followed this link, trying to convert the raw into a temp file and use it like a module: How to load a python script from a raw link (such as Pastebin)?
And this is my test (Using a really simple python script as raw, my main program is not so simple unfortunately): https://trinket.io/python/0e95ba50c8
When I run the script (that now is creating a temp file in the current directory of the .py file) I get this error:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\BOT\\Images\\tempxm4xpwpz.py'
Otherwise I also treid the exec() function... No better results unfortunately.
With this code:
import requests as rq
import urllib.request
def main():
code = "https://pastebin.com/raw/MJmYEKqh"
response = urllib.request.urlopen(code)
data = response.read()
exec(data)
I get this error:
File "<string>", line 10, in <module>
File "<string>", line 5, in hola
NameError: name 'printest' is not defined
Since my program is more complex compared to this simple test, I don't know how to proceed...
Basically What I want to achieve is to write the full script of my program on GitHub and connect it to a .exe so if I upgrade the raw also my program is updated. Avoiding to generate and share (only with my friends) a new .exe everytime...
Do you think is possible? If so.. what am I doing wrong?
PS: I'm also open to other possibilities to let my friends update the program without downloading everytime the .exe, as soon as they don't have to install anything (that's why I'm using .exe).
Disclaimer: it is really not a good idea to run an unverified (let alone untrusted) code. That being said if you really want to do it...
Probably the easiest and "least-dirty" way would be to run whole new process. This can be done directly in python. Something like this should work (inspiration from the answer you linked in your question):
import urllib.request
import tempfile
import subprocess
code = "https://pastebin.com/raw/MJmYEKqh"
response = urllib.request.urlopen(code)
data = response.read()
with tempfile.NamedTemporaryFile(suffix='.py') as source_code_file:
source_code_file.write(data)
source_code_file.flush()
subprocess.run(['python3', source_code_file.name])
You can also make your code with exec run correctly:
What may work:
exec(data, {}) -- All you need to do, is to supply {} as second argument (that is use exec(data, {})). Function exec may receive two additional optional arguments -- globals and locals. If you supply just one, it will use the same directory for locals. That is the code within the exec would behave like sort-of "clean" environment, at the top-level. Which is something you aim for.
exec(data, globals()) -- Second option is to supply the globals from your current scope. This will also work, though you probably has no need to give the execucted code access to your globals, given that that code will set-up everything inside anyway
What does not work:
exec(data, {}, {}) -- In this case the executed code will have two different dictionaries (albeit both empty) for locals and globals. As such it will behavie "as-in" (I'm not really sure about this part, but as I tested it, it seams as such) the function. Meaning that it will add the printest and hola functions to the local scope instead of global scope. Regardless, I expected it to work -- I expected it will just query the printest in the hola function from the local scope instead of global. However, for some reason the hola function in this case gets compiled in such a way it expects printest to be in global scope and not local, which is not there. I really did not figured out why. So this will result in the NameError
exec(data, globals(), locals()) -- This will provide access to the state from the caller function. Nevertheless, it will crash for the very same reason as in the previous case
exec(data) -- This is just a shorthand for exec(data, globals(), locals()
I am new to using the python interactive window and I like it but it seems to clear local variables in between runs, so if I run something like
def main():
dates = '2012152'
# %%
print(dates) # want to just run this
# %%
if __name__ == '__main__':
main()
# or even
main()
all at once it works fine but then if I just run the middle cell I get "dates not defined" error. It works outside of the function because apparently a global variable is saved:
dates = '2012152'
# %%
print(dates) # this works if this cell is run
# %%
Is there any way to get a similar behavior inside a function? If not it doesn't seem useful to me at all (maybe I have designed my code badly?).
Cells are a great way to experiment with flat code, but they are limited when working nested inside functions.
One way to work around this is to use a regular built-in python debugger and set a breakpoint within the function.
Here is a process I use for experimenting with code inside a function:
Set a breakpoint in the function, after the code you want to experiment with and start debugging.
Make any necessary changes to the code.
Select the lines that you've changed and that you want to run again. Make sure to include all the indentations as well.
Right-click and select Evaluate in Debug Console.
This will allow you to run the code one line at a time, see results, and make any necessary adjustments as you go along.
The process could be further improved by binding a keyboard shortcut to the this command.
Yes, print(dates) will not run as the dates variable isn't in scope, unless the function main is called and even then dates will be only in the local scope of the function, not in global scope.
So to print it outside the function, you need to first define it.
To use logarithmic function, I used export to pass a variable $var1 from bash to python script. After the calculation, I used
os.environ['var1']=str(result)
to send the result back to bash script.
However, the bash still shows the unmodified value.
You can have a look at the os.putenv(key, value) function here maybe it could help you.
Although as noted on the doc :
When putenv() is supported, assignments to items in os.environ are automatically translated into corresponding calls to putenv(); however, calls to putenv() don’t update os.environ, so it is actually preferable to assign to items of os.environ.
EDIT :
moooeeeep thought about it just a minute before me, but the variable you change only applies to the current process. In such a case I can see two solutions :
- Write the data to a file and read it with your bash script
- Call your bash script directly from within python so that the bash process would inherit your modified variable.
I want to define some globals in some number crunching work I am doing, I am incrementally writing the script and don't want previous result to keep being loaded/recalculated. One approach is to split out mature code into a separate file and only python run interactively new code. However I just want to do it in a single file for speed of development.
I was under the assumption that a global defined in a file would persist between invocations of run, but they do not.
So my script has the following chunk if code :
if globals().has_key('all_post_freq') != True:
print "creating all post freq var"
global all_post_freq
all_post_freq = all_post_freq_("pickle/all_post_freq.pickle")
How do I retain all_post_freq between invocations of ipython run
edit
ok I have split stuff up into files, but I know there must be a way of doing what I need to do :D
When you %run a file, it is normally started in a blank namespace, and its globals are added to the interactive namespace when it finishes. There's a -i flag which will run it directly in the interactive namespace, so it will see variables you've already defined:
%run -i myscript.py
I am writing a Python interpreter and want to redirect the function's return values to stdout, like the Python Interpreter in Interactive Mode. Within this mode, when the user calls a function, its return value is printed on the screen. The same occurs with expressions.
E.g.
>>> foo()
'Foo return value'
>>> 2+4
6
>>> print('Hello!')
'Hello!'
Changing the sys.stdout only affects the print function. How do I redirect the other expressions to stdout?
Thank you
First, the interactive mode does not print the return value from any function called. Instead, it prints the result of whatever expression the user typed in. If that's not a function call, it still gets printed. If it has 3 function calls in it, it still prints one result, not 3 lines. And so on.
So, trying to redirect function return values to stdout is the wrong thing to do.
What the interactive interpreter does is something sort of like this:
line = raw_input(sys.ps1)
_ = eval(line)
if _ is not None:
print repr(_)
(You may notice that you can change sys.ps1 from the interactive prompt to change what the prompt looks like, access _ to get the last value, etc.)
However, that's not what it really does. And that's not how you should go about this yourself either. If you try, you'll have to deal with complexities like keeping your own globals separate from the user's, handling statements as well as expressions, handling multi-line statements and expressions (doing raw_input(sys.ps2) is easy, but how do you know when to do that?), interacting properly with readline and rlcomplete, etc.
There's a section of the documentation called Custom Python Interpreters which explains the easy way to do this:
The modules described in this chapter allow writing interfaces similar to Python’s interactive interpreter. If you want a Python interpreter that supports some special feature in addition to the Python language, you should look at the code module.
And code:
… provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.
The idea is that you let Python do all the hard stuff, up to whatever level you want to take over, and then you just write the part on top of that.
You may want to look at the source for IDLE, ipython, bpython, etc. for ideas.
Instead of using exec() to run the user input, try eval():
retval = eval(user_input)
sys.stdout.write(repr(retval) + "\n")