This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 3 years ago.
I noticed that my code has many statements like this:
var = "some_string"
var = some_func(var)
var = another_func(var)
print(var) # outputs "modified_string"
It's really annoying me, it just looks awful (in the opposite of whole Python).
How to avoid using that and start using it in a way like this:
var = "some_string"
modify(var, some_func)
modify(var, another_func)
print(var) # outputs "modified_string"
That might not be the most "pythonic" thing to do, but you could "wrap" your string in a list, since lists are mutable in Python.
For example:
var = "string"
var_wrapper = [var]
Now you can pass that list to functions and access its only element. When changed, it will be visible outside of the function:
def change_str(lst):
lst[0] = lst[0] + " changed!"
and you'll get:
>>> change_str(var_wrapper)
>>> var_wrapper[0]
"string changed!"
To make things a bit more readable, you could take it one step further and create a "wrapper" class:
class my_str:
def __init__(self, my_string):
self._str = my_string
def change_str(self, new_str):
self._str = new_str
def __repr__(self):
return self._str
Now let's run the same example:
>>> var = my_str("string")
>>> var
string
>>> var.change_str("new string!")
>>> var
new string!
* Thanks for #Error-SyntacticalRemorse for the remark of making a class.
The problem is that str, int and float (long too, if you're in Py 2.x (True and False are really ints, so them too)) are what you call 'immutable types' in Python. That means that you can't modify their internal states: all manipulations of an str (or int or float) will result in a "new" instance of the str (or whatever) while the old value will remain in Python's cache until the next garbage collection cycle.
Basically, there's nothing you can do. Sorry.
In fact, there's been at least one attempt to add a compose function to functools. I guess I understand why they didn't... But hey, that doesn't mean we can't make one ourselves:
def compose(f1, f2):
def composition(*args, **kwargs):
return f1(f2(*args, **kwargs))
return composition
def compose_many(*funcs):
if len(funcs) == 1:
return funcs[0]
if len(funcs) == 2:
return compose(funcs[0], funcs[1])
else:
return compose(funcs[0], compose_many(*funcs[1:]))
Tested:
>>> def append_foo(s):
... return s + ' foo'
...
>>> def append_bar(s):
... return s + ' bar'
...
>>> append_bar(append_foo('my'))
'my foo bar'
>>> compose(append_bar, append_foo)('my')
'my foo bar'
>>> def append_baz(s):
... return s + ' baz'
...
>>> compose_many(append_baz, append_bar, append_foo)('my')
'my foo bar baz'
Come to think of it, this probably isn't the best solution to your problem. But it was fun to write.
the others already explained why that's not possible, but you could:
for modify in some_func, other_func, yet_another_func:
var = modify(var)
or as pst said:
var = yet_another_func(other_func(some_func(var)))
There is a way to modify an immutable variable, by rewriting it in the local symbol table, however, I think that it's not very nice and should be avoided as much as possible.
def updatevar(obj, value, callingLocals=locals()):
callingLocals[next(k for k, o in callingLocals.items() if o is obj)] = value
Another way, even less pythonic, is to use exec with a formatted instruction. It gets the variable name as a string thanks to this solution:
def getname(obj, callingLocals=locals()):
"""
a quick function to print the name of input and value.
If not for the default-Valued callingLocals, the function would always
get the name as "obj", which is not what I want.
"""
return next(k for k, v in callingLocals.items() if v is obj)
def updatevar2(k, v, callingLocals=locals()):
n = getname(k, callingLocals)
exec('global {};{}={}'.format(n, n, repr(v)))
The result is as expected:
var = "some_string"
updatevar(var, "modified_string")
print(var) # outputs "modified_string"
updatevar2(var, var + '2')
print(var) # outputs "modified_string2"
Strings are immutable in python, so your second example can't work. In the first example you are binding the name var to a completely new object on each line.
Typically multiple assignments to a single name like that are a code smell. Perhaps if you posted a larger sample of code someone here could show you a better way?
I'm just gonna put this right here (since none of the answers seem to have addressed it yet)
If you're commonly repeating the same sequences of functions, consider wrapping them in a higher level function:
def metafunc(var):
var = somefunc(var)
var = otherfunc(var)
var = thirdfunc(var)
return lastfunc(var)
Then when you call the function metafunc you know exactly what's happening to your var: nothing. All you get out of the function call is whatever metafunc returns.
Additionally you can be certain that nothing is happening in parts of your program that you forgot about. This is really important especially in scripting languages where there's usually a lot going on behind the scenes that you don't know about/remember.
There are benefits and drawbacks to this, the theoretical discussion is under the category of pure functional programming. Some real-world interactions (such as i/o operations) require non-pure functions because they need real-world implications beyond the scope of your code's execution.
The principle behind this is defined briefly here:
http://en.wikipedia.org/wiki/Functional_programming#Pure_functions
Related
This question already has answers here:
How do I create variable variables?
(17 answers)
Closed 1 year ago.
This is probably a really novice question, so forgive me.
If I know the name of an instance/variable - let's say I have a string called "variable_name" and a variable with the same name, how would I go about writing a piece of code that takes that string and converts it into something I can actually use and use class methods on, etc? Is that a thing that can happen at all?
Edit: Added some code to better articulate my question. I've got a code setup kind of like this (simplified for space):
class Class_Name:
count = 0
def __init__(self, foo, bar):
self.__class__.count += 1
self.foo = foo
self.bar = bar
def find_variable_name(class_name, number):
variable_name = "variable" + str(number)
return variable_name
variable1 = Class_Name("foo", "bar")
variable2 = Class_Name("foo2", "bar2")
variable3 = Class_Name("foo3", "bar3")
for instances in range(Class_Name.count):
print (find_variable_name(Class_Name, instances+1).foo)
This would give me the error "AttributeError: 'str' object has no attribute 'foo'" - how would I turn the object from a string to something I can work with?
You can simply use exec() function:
k = "my_str" # a string
exec(k + " = k") # my_str = "my_str"
Then, the output is:
>>> print(my_str)
'my_str'
The exec() function executes the string given it.
So, when we do exec(k + " = k"), it means, exec("my_str = k"), which assigns the value of the variable k to my_str.
This makes my_str = "my_str".
NOTE:
Be a little wary of the exec() function, especially if the value of the variable is user-inputted. Then, it could be dangerous to use.
locals() returns a dictionary of the variable bindings of the current scope.
The keys of the dictionary are strs so you can do lookup using the variable name. E.g.
>>> somevariable = 1
>>> locals()["somevariable"]
1
Of course it may be that the variable you want is not in the current scope, then it will not be in locals(). However, if it is in the global scope, you can use globals() in the same way. E.g.
>>> somevariable = 1
>>> globals()["somevariable"]
1
So in your example above, you might use a function like:
def find_variable_name(number):
return globals()["variable" + str(number)]
Which you can use as:
>>> variable1 = Class_Name("foo", "bar")
>>> find_variable_name(1).foo
'foo'
NOTE THAT this type of thing is not good programming practice. Doing this makes code harder to read and understand and maintain. Depending on your application, it might be better just to keep a Dict of your objects, indexing with the counts. E.g.
>>> d = {i: Class_Name("foo", "bar") for i in range(10)}
>>> d[1].foo
'foo'
A function is receiving a number of values that are all strings but need to be parsed in various ways, e.g.
vote_count = int(input_1)
score = float(input_2)
person = Person(input_3)
This is all fine except the inputs can also be None and in this case, instead of parsing the values I would like to end up with None assigned to the left hand side. This can be done with
vote_count = int(input_1) if input_1 is not None else None
...
but this seems much less readable especially with many repeated lines like this one. I'm considering defining a function that simplifies this, something like
def whendefined(func, value):
return func(value) if value is not None else None
which could be used like
vote_count = whendefined(int, input_1)
...
My question is, is there a common idiom for this? Possibly using built-in Python functions? Even if not, is there a commonly used name for a function like this?
In other languages there's Option typing, which is a bit different (solves the problem with a type system), but has the same motivation (what do do about nulls).
In Python there's more of a focus on runtime detection of this kind of thing, so you can wrap the function with an None-detecting guard (rather the data which is what Option typing does).
You could write a decorator that only executes a function if the argument is not None:
def option(function):
def wrapper(*args, **kwargs):
if len(args) > 0 and args[0] is not None:
return function(*args, **kwargs)
return wrapper
You should probably adapt that third line to be more suitable to the kind of data you're working with.
In use:
#option
def optionprint(inp):
return inp + "!!"
>>> optionprint(None)
# Nothing
>>> optionprint("hello")
'hello!!'
and with a return value
#option
def optioninc(input):
return input + 1
>>> optioninc(None)
>>> # Nothing
>>> optioninc(100)
101
or wrap a type-constructing function
>>> int_or_none = option(int)
>>> int_or_none(None)
# Nothing
>>> int_or_none(12)
12
If you can safely treat falsy values (such as 0 and the empty string) as None, you can use boolean and:
vote_count = input_1 and int(input_1)
Since it looks like you're taking strings for input, this might work; you can't turn an empty string to an int or float (or person) anyway. It's not overly readable for some, though the idiom is commonly used in Lua.
For testing purposes I want to directly execute a function defined inside of another function.
I can get to the code object of the child function, through the code (func_code) of the parent function, but when I exec it, i get no return value.
Is there a way to get the return value from the exec'ed code?
Yes, you need to have the assignment within the exec statement:
>>> def foo():
... return 5
...
>>> exec("a = foo()")
>>> a
5
This probably isn't relevant for your case since its being used in controlled testing, but be careful with using exec with user defined input.
A few years later, but the following snippet helped me:
the_code = '''
a = 1
b = 2
return_me = a + b
'''
loc = {}
exec(the_code, globals(), loc)
return_workaround = loc['return_me']
print(return_workaround) # 3
exec() doesn't return anything itself, but you can pass a dict which has all the local variables stored in it after execution. By accessing it you have a something like a return.
I hope it helps someone.
While this is the ugliest beast ever seen by mankind, this is how you can do it by using a global variable inside your exec call:
def my_exec(code):
exec('global i; i = %s' % code)
global i
return i
This is misusing global variables to get your data across the border.
>>> my_exec('1 + 2')
3
Needless to say that you should never allow any user inputs for the input of this function in there, as it poses an extreme security risk.
use eval() instead of exec(), it returns result
Something like this can work:
def outer():
def inner(i):
return i + 10
for f in outer.func_code.co_consts:
if getattr(f, 'co_name', None) == 'inner':
inner = type(outer)(f, globals())
# can also use `types` module for readability:
# inner = types.FunctionType(f, globals())
print inner(42) # 52
The idea is to extract the code object from the inner function and create a new function based on it.
Additional work is required when an inner function can contain free variables. You'll have to extract them as well and pass to the function constructor in the last argument (closure).
Here's a way to return a value from exec'd code:
def exec_and_return(expression):
exec(f"""locals()['temp'] = {expression}""")
return locals()['temp']
I'd advise you to give an example of the problem you're trying to solve. Because I would only ever use this as a last resort.
This doesn't get the return value per say, but you can provide an empty dictionary when calling exec to retrieve any variables defined in the code.
# Python 3
ex_locals = {}
exec("a = 'Hello world!'", None, ex_locals)
print(ex_locals['a'])
# Output: Hello world!
From the Python 3 documentation on exec:
The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.
For more information, see How does exec work with locals?
Here's a solution with a simple code:
# -*- coding: utf-8 -*-
import math
x = [0]
exec("x[0] = 3*2")
print(x[0]) # 6
Since Python 3.7, dictionary are ordered. So you no longer need to agree on a name, you can just say "last item that got created":
>>> d = {}
>>> exec("def addone(i): return i + 1", d, d)
>>> list(d)
['__builtins__', 'addone']
>>> thefunction = d[list(d)[-1]]
>>> thefunction
<function addone at 0x7fd03123fe50>
if we need a function that is in a file in another directory, eg
we need the function1 in file my_py_file.py
located in /home/.../another_directory
we can use the following code:
def cl_import_function(a_func,py_file,in_Dir):
... import sys
... sys.path.insert(0, in_Dir)
... ax='from %s import %s'%(py_file,a_func)
... loc={}
... exec(ax, globals(), loc)
... getFx = loc[afunc]
... return getFx
test = cl_import_function('function1',r'my_py_file',r'/home/.../another_directory/')
test()
(a simple way for newbies...)
program = 'a = 5\nb=10\nprint("Sum =", a + b)'
program = exec(program)
print(program)
I usually use the following pattern (as mentioned in this question):
a=1
s= "{a}".format(**locals())
I think it's a great way to write easily readable code.
Sometimes it's useful to "chain" string formats, in order to "modularize" the creation of complex strings:
a="1"
b="2"
c="{a}+{b}".format(**locals())
d="{c} is a sum".format(**locals())
#d=="1+2 is a sum"
Pretty soon, the code is pestered with X.format(**locals()).
To solve this problem, I tried to create a lambda:
f= lambda x: x.format(**locals())
a="1"
b="2"
c= f("{a}+{b}")
d= f("{c} is a sum")
but this throws a KeyError, since locals() are the lambda's locals.
I also tried to apply the format only on the last string:
a="1"
b="2"
c="{a}+{b}"
d="{c} is a sum".format(**locals())
#d=="{a}+{b} is a sum"
But this doesn't work, since python only formats once.
Now, I could write a function that formats repeatedly until there's nothing more to do:
def my_format( string, vars ):
f= string.format(**vars)
return f if f==string else my_format(f, vars)
but I'm wondering: is there a better way to do this?
f = lambda x, l=locals(): x.format(**l) appears to work...
and if you wanted a version that is a little more all-encompassing (and probably a lot slower):
fg = lambda x, l=locals(), g=globals(): x.format(**dict(g.items() + l.items()))
will find the symbols in either locals or globals.
If you only need to do this within the function scope as a local shortcut, the following will work:
def formatter(fmt, loc=locals()):
return fmt.format(**loc)
However, this will bind the value returned by locals() at the time of function declaration, rather than execution, so it will not be updated as values change, nor will it be useful when called from any other scope.
If you want to get access to the calling method's locals, you need to inspect the call stack (http://docs.python.org/2/library/inspect.html)
import inspect
def formatter(fmt):
parent = inspect.stack()[1][0] # 1 = the previous frame context
# 0 = the frame object
return fmt.format(**parent.f_locals)
Note that this may not work for implementations of python that are not CPython.
Now you can do:
a = "1"
b = "2"
c = formatter("{a}+{b}")
d = formatter("{c} is a sum")
Starting with Python 3.6 the effect of **locals() is already included in string#format or rather "formatted string literals".
See also PEP 498
and Python 3.6 release notes.
It's not a one-liner, but it works:
def fmt(s, l=locals()):
while '{' in s:
s = s.format(**l)
return s
a="1"
b="2"
c="{a}+{b}"
d="{c} is a sum"
print fmt(d) # 1+2 is a sum
Here's a one line (and slightly less efficient) recursive version:
fmt = lambda s, l=locals(): fmt(s.format(**l), l=l) if '{' in s else s
I have this function:
def icaocode(code):
c.execute("SELECT ICAO, LAT, LON FROM airports WHERE ICAO = ?", (code,))
result = c.fetchone()
if result is None:
print("No airport found with ICAO code", code)
sys.exit()
else:
print("Found", code)
[...]
Lets say I call this function with
icaocode(x)
How do I get the function to overwrite x with the results?
In function def:
def icaocode(code):
...
return code # new value
When calling:
x = icaocode(x)
Btw if the argument is mutable (like a list), you can overwrite it without returning the new value. If it's immutable (like a string, integer), you can't.
E.g.
def f(some_list):
some_list.append("something")
In this case
my_list = []
f(my_list)
my_list will be ["something"]
You can't overwrite the value of the parameter. That is, you can't change it to refer to another object. You can, however, change the object. There is an old thread on pass-by-value and pass-by-reference semantics in Python that you may find illuminating: https://stackoverflow.com/a/986145/399047
For example, you can append elements to a list that is passed in as a parameter. The following code:
def func(a_list):
a_list.append("some value")
l = [1,2,3]
print l
func(l)
print l
would give:
[1,2,3]
[1,2,3,"some value"]
In contrast, a string, cannot be modified. The following code:
def func2(a_str):
a_str += "suffix"
s = "test"
print s
func2(s)
print s
would give:
"test"
"test"
My recommendation, unless you have a good reason, is to avoid mutating your input parameters, and return the modified object instead. Side-effects can make for messy code.
If, at the end of all this you really want to modify your input parameter, one technique would be to wrap the code parameter inside another object. e.g.
def icaocode(code_list):
input_code = code_list[0]
[...]
# do something to input_code, assign result to
# output_code
[...]
code_list[0] = output_code
Then you would call with:
code_list = [code]
icaocode(code_list)
That said, this code is ugly to me, smells something awful, and I don't recommend it.
You can, but it is a horrible way to conduct business. Return the value instead, remember that you can return more than one value if you like. Here is however one way to return a value through a parameter. But don't use it.
>>> def a(b):
... b[0] = 'hi'
....
>>> c = ['nono']
>>> a(c)
>>> print(c)
['hi']