In many languages (and places) there is a nice practice of creating local scopes by creating a block like this.
void foo()
{
... Do some stuff ...
if(TRUE)
{
char a;
int b;
... Do some more stuff ...
}
... Do even more stuff ...
}
How can I implement this in python without getting the unexpected indent error and without using some sort of if True: tricks
Why do you want to create new scopes in python anyway?
The normal reason for doing it in other languages is variable scoping, but that doesn't happen in python.
if True:
a = 10
print a
In Python, scoping is of three types : global, local and class. You can create specialized 'scope' dictionaries to pass to exec / eval(). In addition you can use nested scopes
(defining a function within another). I found these to be sufficient in all my code.
As Douglas Leeder said already, the main reason to use it in other languages is variable scoping and that doesn't really happen in Python. In addition, Python is the most readable language I have ever used. It would go against the grain of readability to do something like if-true tricks (Which you say you want to avoid). In that case, I think the best bet is to refactor your code into multiple functions, or use a single scope. I think that the available scopes in Python are sufficient to cover every eventuality, so local scoping shouldn't really be necessary.
If you just want to create temp variables and let them be garbage collected right after using them, you can use
del varname
when you don't want them anymore.
If its just for aesthetics, you could use comments or extra newlines, no extra indentation, though.
Python has exactly two scopes, local and global. Variables that are used in a function are in local scope no matter what indentation level they were created at. Calling a nested function will have the effect that you're looking for.
def foo():
a = 1
def bar():
b = 2
print a, b #will print "1 2"
bar()
Still like everyone else, I have to ask you why you want to create a limited scope inside a function.
variables in list comprehension (Python 3+) and generators are local:
>>> i = 0
>>> [i+1 for i in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> i
0
but why exactly do you need this?
A scope is a textual region of a
Python program where a namespace is
directly accessible. “Directly
accessible” here means that an
unqualified reference to a name
attempts to find the name in the
namespace...
Please, read the documentation and clarify your question.
btw, you don't need if(TRUE){} in C, a simple {} is sufficient.
As mentioned in the other answers, there is no analogous functionality in Python to creating a new scope with a block, but when writing a script or a Jupyter Notebook, I often (ab)use classes to introduce new namespaces for similar effect. For example, in a notebook where you might have a model "Foo", "Bar" etc. and related variables you might want to create a new scope to avoid having to reuse names like
model = FooModel()
optimizer = FooOptimizer()
...
model = BarModel()
optimizer = BarOptimizer()
or suffix names like
model_foo = ...
optimizer_foo = ...
model_bar = ...
optimizer_bar= ...
Instead you can introduce new namespaces with
class Foo:
model = ...
optimizer = ...
loss = ....
class Bar:
model = ...
optimizer = ...
loss = ...
and then access the variables as
Foo.model
Bar.optimizer
...
I find that using namespaces this way to create new scopes makes code more readable and less error-prone.
While the leaking scope is indeed a feature that is often useful,
I have created a package to simulate block scoping (with selective leaking of your choice, typically to get the results out) anyway.
from scoping import scoping
a = 2
with scoping():
assert(2 == a)
a = 3
b = 4
scoping.keep('b')
assert(3 == a)
assert(2 == a)
assert(4 == b)
https://pypi.org/project/scoping/
I would see this as a clear sign that it's time to create a new function and refactor the code. I can see no reason to create a new scope like that. Any reason in mind?
def a():
def b():
pass
b()
If I just want some extra indentation or am debugging, I'll use if True:
Like so, for arbitrary name t:
### at top of function / script / outer scope (maybe just big jupyter cell)
try: t
except NameError:
class t
pass
else:
raise NameError('please `del t` first')
#### Cut here -- you only need 1x of the above -- example usage below ###
t.tempone = 5 # make new temporary variable that definitely doesn't bother anything else.
# block of calls here...
t.temptwo = 'bar' # another one...
del t.tempone # you can have overlapping scopes this way
# more calls
t.tempthree = t.temptwo; del t.temptwo # done with that now too
print(t.tempthree)
# etc, etc -- any number of variables will fit into t.
### At end of outer scope, to return `t` to being 'unused'
del t
All the above could be in a function def, or just anyplace outside defs along a script.
You can add or del new elements to an arbitrary-named class like that at any point. You really only need one of these -- then manage your 'temporary' namespace as you like.
The del t statement isn't necessary if this is in a function body, but if you include it, then you can copy/paste chunks of code far apart from each other and have them work how you expect (with different uses of 't' being entirely separate, each use starting with the that try: t... block, and ending with del t).
This way if t had been used as a variable already, you'll find out, and it doesn't clobber t so you can find out what it was.
This is less error prone then using a series of random=named functions just to call them once -- since it avoids having to deal with their names, or remembering to call them after their definition, especially if you have to reorder long code.
This basically does exactly what you want: Make a temporary place to put things you know for sure won't collide with anything else, and which you are responsible for cleaning up inside as you go.
Yes, it's ugly, and probably discouraged -- you will be directed to decompose your work into a set of smaller, more reusable functions.
As others have suggested, the python way to execute code without polluting the enclosing namespace is to put it in a class or function. This presents a slight and usually harmless problem: defining the function puts its name in the enclosing namespace. If this causes harm to you, you can name your function using Python's conventional temporary variable "_":
def _():
polluting_variable = foo()
...
_() # Run the code before something overwrites the variable.
This can be done recursively as each local definition masks the definition from the enclosing scope.
This sort of thing should only be needed in very specific circumstances. An example where it is useful is when using Databricks' %run magic, which executes the contents of another notebook in the current notebook's global scope. Wrapping the child notebook's commands in temporary functions prevents them from polluting the global namespace.
Related
Is there a way to limit function so that it would only have access to local variable and passed arguments?
For example, consider this code
a = 1
def my_fun(x):
print(x)
print(a)
my_fun(2)
Normally the output will be
2
1
However, I want to limit my_fun to local scope so that print(x) would work but throw an error on print(a). Is that possible?
I feel like I should preface this with: Do not actually do this.
You (sort of) can with functions, but you will also disable calls to all other global methods and variables, which I do not imagine you would like to do.
You can use the following decorator to have the function act like there are no variables in the global namespace:
import types
noglobal = lambda f: types.FunctionType(f.__code__, {})
And then call your function:
a = 1
#noglobal
def my_fun(x):
print(x)
print(a)
my_fun(2)
However this actually results in a different error than you want, it results in:
NameError: name 'print' is not defined
By not allowing globals to be used, you cannot use print() either.
Now, you could pass in the functions that you want to use as parameters, which would allow you to use them inside the function, but this is not a good approach and it is much better to just keep your globals clean.
a = 1
#noglobal
def my_fun(x, p):
p(x)
p(a)
my_fun(2, print)
Output:
2
NameError: name 'a' is not defined
Nope. The scoping rules are part of a language's basic definition. To change this, you'd have to alter the compiler to exclude items higher on the context stack, but still within the user space. You obviously don't want to limit all symbols outside the function's context, as you've used one in your example: the external function print. :-)
I have had PyCharm 2017.3 extract some code inside a top-level function to another top-level function, and it does a good job.
However, sometimes I would like to not put the extracted function on top level, but rather it should become a function nested inside the existing function. The rationale is re-using code that is only used inside a function, but several times there. I think that this "sub-function" should ideally not be accessible outside of the original function.
How can I do this? I have not seen any options in the refactoring dialog.
Example
a) Original code:
def foo():
a = ''
if a == '':
b = 'empty'
else:
b = 'not empty'
return b
b) What extracting does:
def foo():
a = ''
b = bar(a)
return b
def bar(a):
if a == '':
b = 'empty'
else:
b = 'not empty'
return b
c) What I would like to have:
def foo():
def bar():
if a == '':
b = 'empty'
else:
b = 'not empty'
return b
a = ''
b = bar(a)
return b
I am aware that bar's b will shadow foo's b unless it is renamed in the process. I also thought about completely accepting the shadowing by not returning or requesting b and just modifying it inside bar.
Please also hint me if what I want is not a good thing for any reason.
It is considered good practice to keep function boundaries isolated: get data as parameters and spit data as return values with as little side-effects as possible. That said, there are a few special cases where you break this rule; many of them when using closures. Closures are not as idiomatic in Python as they are in Javascript - personally I think it is good but many people disagree.
There is one place were closures are absolutely idiomatic in Python: decorators. For other cases where you would use a closure in order to avoid use of global variables and provide some form of data hiding there are other alternatives in Python. Although some people advocates using closure instead of a class when it has just one method, a plain function combined with functools.partial can be even better.
This is my guess about why there is no such feature in Pycharm: we almost never do it in Python, instead we tend to keep the function signature as foo(x) even when we can get x from the enclosing scope. Hell, in Python our methods receive self explicitly where most languages have an implicit this. If you write code this way then Pycharm already does everything that is needed when refactoring: it fixes the indentation when you cut & paste.
If you catch yourself doing this kind of refactoring a lot I guess you are coming from a language where closures are more idiomatic like Javascript or Lisp.
So my point is: this "nested to global" or "global to nested" function refactoring feature does not exist in Pycharm because nested functions relying on the enclosing scopes are not idiomatic in Python unless for closures - and even closures are not that idiomatic outside of decorators.
If you care enough go ahead and fill a feature request at their issue tracker or upvote some related tickets like #PY-12802 and #PY-2701 - as you can see those have not attracted a lot of attention possibly because of the reasons above.
I would like to write a function which receives a local namespace dictionary and update it. Something like this:
def UpdateLocals(local_dict):
d = {'a':10, 'b':20, 'c':30}
local_dict.update(d)
When I call this function from the interactive python shell it works all right, like this:
a = 1
UpdateLocals(locals())
# prints 20
print a
However, when I call UpdateLocals from inside a function, it doesn't do what I expect:
def TestUpdateLocals():
a = 1
UpdateLocals(locals())
print a
# prints 1
TestUpdateLocals()
How can I make the second case work like the first?
UPDATE:
Aswin's explanation makes sense and is very helpful to me. However I still want a mechanism to update the local variables. Before I figure out a less ugly approach, I'm going to do the following:
def LoadDictionary():
return {'a': 10, 'b': 20, 'c': 30}
def TestUpdateLocals():
a = 1
for name, value in LoadDictionary().iteritems():
exec('%s = value' % name)
Of course the construction of the string statements can be automated, and the details can be hidden from the user.
You have asked a very good question. In fact, the ability to update local variables is very important and crucial in saving and loading datasets for machine learning or in games. However, most developers of Python language have not come to a realization of its importance. They focus too much on conformity and optimization which is nevertheless important too.
Imagine you are developing a game or running a deep neural network (DNN), if all local variables are serializable, saving the entire game or DNN can be simply put into one line as print(locals()), and loading the entire game or DNN can be simply put into one line as locals().update(eval(sys.stdin.read())).
Currently, globals().update(...) takes immediate effect but locals().update(...) does not work because Python documentation says:
The default locals act as described for function locals() below:
modifications to the default locals dictionary should not be
attempted. Pass an explicit locals dictionary if you need to see
effects of the code on locals after function exec() returns.
Why they design Python in such way is because of optimization and conforming the exec statement into a function:
To modify the locals of a function on the fly is not possible without
several consequences: normally, function locals are not stored in a
dictionary, but an array, whose indices are determined at compile time
from the known locales. This collides at least with new locals added
by exec. The old exec statement circumvented this, because the
compiler knew that if an exec without globals/locals args occurred in
a function, that namespace would be "unoptimized", i.e. not using the
locals array. Since exec() is now a normal function, the compiler does
not know what "exec" may be bound to, and therefore can not treat is
specially.
Since global().update(...) works, the following piece of code will work in root namespace (i.e., outside any function) because locals() is the same as globals() in root namespace:
locals().update({'a':3, 'b':4})
print(a, b)
But this will not work inside a function.
However, as hacker-level Python programmers, we can use sys._getframe(1).f_locals instead of locals(). From what I have tested so far, on Python 3, the following piece of code always works:
def f1():
sys._getframe(1).f_locals.update({'a':3, 'b':4})
print(a, b)
f1()
However, sys._getframe(1).f_locals does not work in root namespace.
The locals are not updated here because, in the first case, the variable declared has a global scope. But when declared inside a function, the variable loses scope outside it.
Thus, the original value of the locals() is not changed in the UpdateLocals function.
PS: This might not be related to your question, but using camel case is not a good practice in Python. Try using the other method.
update_locals() instead of UpdateLocals()
Edit To answer the question in your comment:
There is something called a System Stack. The main job of this system stack during the execution of a code is to manage local variables, make sure the control returns to the correct statement after the completion of execution of the called function etc.,
So, everytime a function call is made, a new entry is created in that stack,
which contains the line number (or instruction number) to which the control has to return after the return statement, and a set of fresh local variables.
The local variables when the control is inside the function, will be taken from the stack entry. Thus, the set of locals in both the functions are not the same. The entry in the stack is popped when the control exits from the function. Thus, the changes you made inside the function are erased, unless and until those variables have a global scope.
I just switched to PyCharm and I am very happy about all the warnings and hints it provides me to improve my code. Except for this one which I don't understand:
This inspection detects shadowing names defined in outer scopes.
I know it is bad practice to access variable from the outer scope, but what is the problem with shadowing the outer scope?
Here is one example, where PyCharm gives me the warning message:
data = [4, 5, 6]
def print_data(data): # <-- Warning: "Shadows 'data' from outer scope
print data
print_data(data)
There isn't any big deal in your above snippet, but imagine a function with a few more arguments and quite a few more lines of code. Then you decide to rename your data argument as yadda, but miss one of the places it is used in the function's body... Now data refers to the global, and you start having weird behaviour - where you would have a much more obvious NameError if you didn't have a global name data.
Also remember that in Python everything is an object (including modules, classes and functions), so there's no distinct namespaces for functions, modules or classes. Another scenario is that you import function foo at the top of your module, and use it somewhere in your function body. Then you add a new argument to your function and named it - bad luck - foo.
Finally, built-in functions and types also live in the same namespace and can be shadowed the same way.
None of this is much of a problem if you have short functions, good naming and a decent unit test coverage, but well, sometimes you have to maintain less than perfect code and being warned about such possible issues might help.
The currently most up-voted and accepted answer and most answers here miss the point.
It doesn't matter how long your function is, or how you name your variable descriptively (to hopefully minimize the chance of potential name collision).
The fact that your function's local variable or its parameter happens to share a name in the global scope is completely irrelevant. And in fact, no matter how carefully you choose you local variable name, your function can never foresee "whether my cool name yadda will also be used as a global variable in future?". The solution? Simply don't worry about that! The correct mindset is to design your function to consume input from and only from its parameters in signature. That way you don't need to care what is (or will be) in global scope, and then shadowing becomes not an issue at all.
In other words, the shadowing problem only matters when your function need to use the same name local variable and the global variable. But you should avoid such design in the first place. The OP's code does not really have such design problem. It is just that PyCharm is not smart enough and it gives out a warning just in case. So, just to make PyCharm happy, and also make our code clean, see this solution quoting from silyevsk's answer to remove the global variable completely.
def print_data(data):
print data
def main():
data = [4, 5, 6]
print_data(data)
main()
This is the proper way to "solve" this problem, by fixing/removing your global thing, not adjusting your current local function.
A good workaround in some cases may be to move the variables and code to another function:
def print_data(data):
print data
def main():
data = [4, 5, 6]
print_data(data)
main()
I like to see a green tick in the top right corner in PyCharm. I append the variable names with an underscore just to clear this warning so I can focus on the important warnings.
data = [4, 5, 6]
def print_data(data_):
print(data_)
print_data(data)
It depends how long the function is. The longer the function, the greater the chance that someone modifying it in future will write data thinking that it means the global. In fact, it means the local, but because the function is so long, it's not obvious to them that there exists a local with that name.
For your example function, I think that shadowing the global is not bad at all.
Do this:
data = [4, 5, 6]
def print_data():
global data
print(data)
print_data()
data = [4, 5, 6] # Your global variable
def print_data(data): # <-- Pass in a parameter called "data"
print data # <-- Note: You can access global variable inside your function, BUT for now, which is which? the parameter or the global variable? Confused, huh?
print_data(data)
It looks like it is 100% a pytest code pattern.
See:
pytest fixtures: explicit, modular, scalable
I had the same problem with it, and this is why I found this post ;)
# ./tests/test_twitter1.py
import os
import pytest
from mylib import db
# ...
#pytest.fixture
def twitter():
twitter_ = db.Twitter()
twitter_._debug = True
return twitter_
#pytest.mark.parametrize("query,expected", [
("BANCO PROVINCIAL", 8),
("name", 6),
("castlabs", 42),
])
def test_search(twitter: db.Twitter, query: str, expected: int):
for query in queries:
res = twitter.search(query)
print(res)
assert res
And it will warn with This inspection detects shadowing names defined in outer scopes.
To fix that, just move your twitter fixture into ./tests/conftest.py
# ./tests/conftest.py
import pytest
from syntropy import db
#pytest.fixture
def twitter():
twitter_ = db.Twitter()
twitter_._debug = True
return twitter_
And remove the twitter fixture, like in ./tests/test_twitter2.py:
# ./tests/test_twitter2.py
import os
import pytest
from mylib import db
# ...
#pytest.mark.parametrize("query,expected", [
("BANCO PROVINCIAL", 8),
("name", 6),
("castlabs", 42),
])
def test_search(twitter: db.Twitter, query: str, expected: int):
for query in queries:
res = twitter.search(query)
print(res)
assert res
This will be make happy for QA, PyCharm and everyone.
I think this rule doesn't help much. I simply disabled it by going to Settings -> Editor -> Inspections and then checking off this rule:
Shadowing names from outer scope
To ignore the warning, as Chistopher said in a comment, you can comment above it
# noinspection PyShadowingNames
how we can construct static effect on python instead of using class and global ?
not like that one :
global a
a = []
#simple ex ;
fonk ( a , b , d)
x = 1
a.append ( x)
EDIT:
I want to create temporary memory , if I exit the function namely fonk , I want to save change as list on temporary memory .
We can do that demand only put static keyword in front of data type but in python , we dont have static, so I want that effect in python . Therefore , how can I do ?
As above code say "a" represents temporary memory
Default values for function arguments are evaluated once, at function definition time, so if you put a mutable object there, it will live across calls. Can be a gotcha, but very useful for caches and similar things static is often used for in other languages. Of course callers can override your cache in this case - but that's not a bad thing, they won't unless they have good reasons and in that case you should allow them to.
Example (this one is usually found in "gotchas" question instead ^^):
def append_and_return_static_list(item, items=[]):
items.append(item)
return items
append_and_return_static_list(0)
append_and_return_static_list(1)
print append_and_return_static_list(2) #=> [0,1,2]
Now, if you absolutely don't want to go that way, you still have other possibilities: You can create a variable outside the function and put the object you want to share there. You should propably prefix the name with a single underscore if you want it to be considered private to that place (not compiler-enforced-private but convention-and-survival-instinct-enforced).
Example (not the best code, the above is better in almost all cases):
_items = []
def append_and_return_static_list(item):
_items.append(item)
return _items
append_and_return_static_list(0)
append_and_return_static_list(1)
print append_and_return_static_list(2) #=> [0,1,2]
Attributes defined in global scope or in class scope are effectively static since modules are treated as singletons and by extension class definitions are singletons within a particular module global scope. (This explanation glosses over the dynamic features of Python that can change this behavior)
The global keyword is used within a local scope to disambiguate assignment. It's a declaration that the attribute name belongs to the enclosing global scope.
In your (broken) example you don't even need to use the global keyword since you are not using assignment, you are calling the append() method of a - no disambiguation is required.
An illustrative example of what global is used for:
>>> a = []
>>> def fonk():
... a = [1]
...
>>> print a
[]
>>> fonk()
>>> print a
[]
>>>
>>> def fonk2():
... global a
... a = [2]
...
>>> print a
[]
>>> fonk2()
>>> print a
[2]
EDIT: I guess I missed the point of the question (do not do it via global and classes), but I don't understand the objection especially when it seems that the example code was broken.