We Keep Coding

Executing an import statement string and using the import [duplicate]

How do I execute a string containing Python code in Python?
Do not ever use eval (or exec) on data that could possibly come from outside the program in any form. It is a critical security risk. You allow the author of the data to run arbitrary code on your computer. If you are here because you want to create multiple variables in your Python program following a pattern, you almost certainly have an XY problem. Do not create those variables at all - instead, use a list or dict appropriately.

For statements, use exec(string) (Python 2/3) or exec string (Python 2):
>>> my_code = 'print("hello world")'
>>> exec(my_code)
Hello world
When you need the value of an expression, use eval(string):
>>> x = eval("2+2")
>>> x
4
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.

In the example a string is executed as code using the exec function.
import sys
import StringIO
# create file-like string to capture output
codeOut = StringIO.StringIO()
codeErr = StringIO.StringIO()
code = """
def f(x):
x = x + 1
return x
print 'This is my output.'
"""
# capture output and errors
sys.stdout = codeOut
sys.stderr = codeErr
exec code
# restore stdout and stderr
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
print f(4)
s = codeErr.getvalue()
print "error:\n%s\n" % s
s = codeOut.getvalue()
print "output:\n%s" % s
codeOut.close()
codeErr.close()

eval and exec are the correct solution, and they can be used in a safer manner.
As discussed in Python's reference manual and clearly explained in this tutorial, the eval and exec functions take two extra parameters that allow a user to specify what global and local functions and variables are available.
For example:
public_variable = 10
private_variable = 2
def public_function():
return "public information"
def private_function():
return "super sensitive information"
# make a list of safe functions
safe_list = ['public_variable', 'public_function']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
# add any needed builtins back in
safe_dict['len'] = len
>>> eval("public_variable+2", {"__builtins__" : None }, safe_dict)
12
>>> eval("private_variable+2", {"__builtins__" : None }, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_variable' is not defined
>>> exec("print \"'%s' has %i characters\" % (public_function(), len(public_function()))", {"__builtins__" : None}, safe_dict)
'public information' has 18 characters
>>> exec("print \"'%s' has %i characters\" % (private_function(), len(private_function()))", {"__builtins__" : None}, safe_dict)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'private_function' is not defined
In essence you are defining the namespace in which the code will be executed.

Remember that from version 3 exec is a function!
so always use exec(mystring) instead of exec mystring.

Avoid exec and eval
Using exec and eval in Python is highly frowned upon.
There are better alternatives
From the top answer (emphasis mine):
For statements, use exec.
When you need the value of an expression, use eval.
However, the first step should be to ask yourself if you really need to. Executing code should generally be the position of last resort: It's slow, ugly and dangerous if it can contain user-entered code. You should always look at alternatives first, such as higher order functions, to see if these can better meet your needs.
From Alternatives to exec/eval?
set and get values of variables with the names in strings
[while eval] would work, it is generally not advised to use variable names bearing a meaning to the program itself.
Instead, better use a dict.
It is not idiomatic
From http://lucumr.pocoo.org/2011/2/1/exec-in-python/ (emphasis mine)
Python is not PHP
Don't try to circumvent Python idioms because some other language does it differently. Namespaces are in Python for a reason and just because it gives you the tool exec it does not mean you should use that tool.
It is dangerous
From http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html (emphasis mine)
So eval is not safe, even if you remove all the globals and the builtins!
The problem with all of these attempts to protect eval() is that they are blacklists. They explicitly remove things that could be dangerous. That is a losing battle because if there's just one item left off the list, you can attack the system.
So, can eval be made safe? Hard to say. At this point, my best guess is that you can't do any harm if you can't use any double underscores, so maybe if you exclude any string with double underscores you are safe. Maybe...
It is hard to read and understand
From http://stupidpythonideas.blogspot.it/2013/05/why-evalexec-is-bad.html (emphasis mine):
First, exec makes it harder to human beings to read your code. In order to figure out what's happening, I don't just have to read your code, I have to read your code, figure out what string it's going to generate, then read that virtual code. So, if you're working on a team, or publishing open source software, or asking for help somewhere like StackOverflow, you're making it harder for other people to help you. And if there's any chance that you're going to be debugging or expanding on this code 6 months from now, you're making it harder for yourself directly.

eval() is just for expressions, while eval('x+1') works, eval('x=1') won't work for example. In that case, it's better to use exec, or even better: try to find a better solution :)

It's worth mentioning that exec's brother exists as well, called execfile, if you want to call a Python file. That is sometimes good if you are working in a third party package which have terrible IDE's included and you want to code outside of their package.
Example:
execfile('/path/to/source.py')
or:
exec(open("/path/to/source.py").read())

You accomplish executing code using exec, as with the following IDLE session:
>>> kw = {}
>>> exec( "ret = 4" ) in kw
>>> kw['ret']
4

As the others mentioned, it's "exec" ..
but, in case your code contains variables, you can use "global" to access it, also to prevent the compiler to raise the following error:
NameError: name 'p_variable' is not defined
exec('p_variable = [1,2,3,4]')
global p_variable
print(p_variable)

I tried quite a few things, but the only thing that worked was the following:
temp_dict = {}
exec("temp_dict['val'] = 10")
print(temp_dict['val'])
output:
10

Use eval.

Check out eval:
x = 1
print eval('x+1')
->2

The most logical solution would be to use the built-in eval() function .Another solution is to write that string to a temporary python file and execute it.

Ok .. I know this isn't exactly an answer, but possibly a note for people looking at this as I was. I wanted to execute specific code for different users/customers but also wanted to avoid the exec/eval. I initially looked to storing the code in a database for each user and doing the above.
I ended up creating the files on the file system within a 'customer_filters' folder and using the 'imp' module, if no filter applied for that customer, it just carried on
import imp
def get_customer_module(customerName='default', name='filter'):
lm = None
try:
module_name = customerName+"_"+name;
m = imp.find_module(module_name, ['customer_filters'])
lm = imp.load_module(module_name, m[0], m[1], m[2])
except:
''
#ignore, if no module is found,
return lm
m = get_customer_module(customerName, "filter")
if m is not None:
m.apply_address_filter(myobj)
so customerName = "jj"
would execute apply_address_filter from the customer_filters\jj_filter.py file

PyCharm: regex string intentions for function arguments

I have a function that takes a string argument that will be compiled into a regular expression, like so:
class Pattern:
def __init__(self, pattern, **kwargs) -> None:
self.re = re.compile(pattern)
self.extras = dict(**kwargs)
... more methods ...
When I initialize a Pattern object, I'd love for PyCharm (actually, I'm using IntelliJ IDEA with the Python plugin, but I think that's essentially the same platform) to know that my pattern argument string should conform to Python's regular expression syntax, and can be syntax-highlighted, just like when using re.compile(pattern):
# This won't be recognized as a regex intention:
p1 = Pattern(r'^(?P<site>[A-Z]{4}).(?P<unit>\d{4})')
# But this will:
p2 = re.compile(r'^(?P<site>[A-Z]{4}).(?P<unit>\d{4})')
Is there any way to give the IDE a hint here, that whatever is passed as the pattern argument to Pattern should have regex syntax? Ideally in some kind of hint comment or type hint that I could put in the code so it would work for other developers too.
I do know that:
I could create a plugin for this - in my case that would probably be vast overkill.
I can do "Inject language or reference" on an ad-hoc basis for any particular call to Pattern(...) - but that's temporary, it's tedious when there are a lot of such calls, and it doesn't help other developers read the code more easily.
Edit:
This question is the direct analog of "Can one automatically get Intellij's regex assistance for one's own regex parameters", but for Python instead of Java. The IntelliLang solution in that answer seems to be only applicable to editing Java or XML files.

I found one way but it works not so smoothly as I expected.
File - Settings - Editor - Language Injections - Add:
Generic Python
ID: RegExp
Places Patterns:
+ pyLiteralExpression().and(pyMethodArgument("Pattern", 0, "foo.bar.Pattern"))
Now RegExp will be injected into arguments to Pattern.
Next step is to share this injection. The injection was created with IDE-level scope and it is stored in IDE settings that are impossible to share.
To put this injection into project settings you have to move injection to Project-level scope (corresponding button is below the button that adds injection) but then it stops working :( The only way is to Export-Import injection but it is not convenient :(
Feel free to create an issue here.

Format of responses to help() in Python shell

I have a question about how certain "help()" items show up in my Python 3.6.2 IDLE shell running on Windows. From the docs, I'd expect to see sum(iterable[, start]) and pow (x, y[, z]), but calling help on those yields sum(iterable, start=0, /) , and pow(x, y, z=None, /). There may be other functions that display the same way.
I'm curious about why they put the descriptor in keyword form (which you cannot use explicitly when calling the functions, as doing so throws a "x takes no keyword arguments" error), but mostly, what is the slash doing there?

[Adding a bit to the linked answer...]
In Python3, documentation of optional args in signatures was changed from brackets, as in '[, start]' (with the default hopefully given in the docstring) to directly giving the default, as in 'start=0'. Sometime later (perhaps 3.4), '/' was (gradually) added to the signature of C-coded functions that do not allow passing an argument by keyword. Before this, there was no easy way to discover the fact without trial and exception. (I believe that there are a few optional args that do not have a default. These have to continue with brackets.)
None of this has anything to do with IDLE, which prints help() output as received. That is why I removed the tag. However, IDLE for current 3.6 and 3.7 adds the following to tooltips when the signature contains '/'.
['/' marks preceding arguments as positional-only]
I don't know if help() should do the same.

Python magical main() signature like Perl 6

Does python have any way to easily and quickly make CLI utilities without lots of argument parsing boilerplate?
In Perl 6, the signature for the MAIN sub automagically parses command line arguments.
Is there any way to do something similar in Python without lots of boilerplate? If there is not, what would be the best way to do it? I'm thinking a function decorator that will perform some introspection and do the right thing. If there's nothing already like it, I'm thinking something like what I have below. Is this a good idea?
#MagicMain
def main(one, two=None, *args, **kwargs):
print one # Either --one or first non-dash argument
print two # Optional --arg with default value (None)
print args # Any other non-dash arguments
print kwargs # Any other --arguments
if __name__ == '__main__':
main(sys.argv)

The Baker library contains some convenient decorators to "automagically" create arg parsers from method signatures.
For example:
#baker.command
def test(start, end=None, sortby="time"):
print "start=", start, "end=", end, "sort=", sortby
$ script.py --sortby name 1
start= 1 end= sortby= name

I'm not really sure what you consider to be parsing boilerplate. The 'current' approach is to use the argparse system for python. The older system is getopt.

Simon Willison's optfunc module tries to provide the functionality you're looking for.

The opterator module handles this.
https://github.com/buchuki/opterator

Python has the getopts module for doing this.

Recently I came across the begins project for decorating and simplifying command line handling.
It seems to offer a lot of the same functions you are looking for.

Ruby equivalent to Python's help()?

When working in interactive Python, I tend to rely on the built-in help() function to tell me what something expects and/or returns, and print out any documentation that might help me. Is there a Ruby equivalent to this function?
I'm looking for something I could use in irb. For example, in interactive Python I could type:
>>> help(1)
which would then print
Help on int object:
class int(object) | int(x[, base])
-> integer | |
Convert a string or number to an integer, if possible. A ...

It's now late 2014 and here's the two ways to get the Python help() *similarity, as long as you have the Ruby Docs installed:
From inside irb, You can call the help method with a string describing what you're looking for.
Example 1: help 'Array' for the Array class
Example 2: help 'Array#each' for the Array class each method.
From the command line, outside of irb, you can use the ri program:
Example 1: ri 'Array' for the Array class
Example 2: ri 'Array#each' for the Array class each method.
* Not quite as good as Python's, but still better than nothing

It's definitely a poor cousin to iPython's help, and one of the main features I miss after moving to Ruby, but you can also use ri from within irb. I'd recommend the wirble gem as an easy way to set this up.

Try using ri from the command line.
It takes a class name, method, or module as an argument, and gives you appropriate documentation. Many popular gems come with this form of documentation, as well, so it should typically work even beyond the scope of core Ruby modules.

There's supposed to be irb_help. But like mentioned in that post, it's broken in my ruby install as well.

For quick shell access to ruby documentation, just type ri followed by the method you're wanting to learn more about (from your shell).
For example:
ri puts
This must be fired up in your shell, not your irb (interactive ruby environment)
If you're in your irb environment, then another way, is to simply type help followed by the method you want to learn more about as follows:
help puts
However, this assumes that you have configured your Ruby environment correctly for that (help) to work properly within irb. I usually just have another shell open, and just use the ri directly for quick access when I'm in doubt about a certain method or arguments to a method.