We Keep Coding

Python 3.8: KeyError resulting from a referral to vars() within a function... Even though the variable is clearly there and accessible

context: I have global variables ids1, ids2, ids3, ids4 and ids5.
When I attempt to execute this function
def collect_id_sols(id_no):
ids = vars()["ids" + str(id_no)]
sols = vars()["sols" + str(id_no)]
for line in range(0,len(ids)):
#rest irrelevant...
The interpreter throws a:
File "sols_from_ids.py", line 112, in <module>
collect_id_sols(1)
File "sols_from_ids.py", line 78, in collect_id_sols
ids = vars()["ids" + str(id_no)]
KeyError: 'ids1'
i.e. what it's telling me is that there is no such key "ids1".
However, the variable is CLEARLY existing and completely accessible.
Right after this error is thrown, if I do a >>>ids1 or >>>vars()["ids1"] within the interpreter, everything shows and works just as it should be.
What's going on? :(
P.S. And, of course, the global variables are declared and assigned before the function definition and call.

vars returns a dictionary giving the local scope, so it wouldn't know about these non-local variables. While there are ways (e.g. with globals()) to do what you want to do, a much better solution is to use a proper data structure such as a list or dictionary for your ids. Anytime you find yourself trying to iterate over a collection of variables which differ only by a number tacked onto the end, there is a data structure waiting to be born.

According to Python docs, vars() used without an argument acts like locals(), i.e. :
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
So when you use 'vars()' in your function block, it will return the symbol table of within the function block, which will be empty in your case. So you get a Key Error.
Using vars()["something"] to access the variable something only works at module-level, not inside functions or classes. Plus it is not a viable thing to do in my opinion.
What you probably want to do here is to use an array to store your variables idsX, where ids[1] will be ids1 etc. (or a dictionary)
Or you can have another function like :
def getIds(number) :
if number == 1 :
return ids1
elif number == 2 :
return ids2
etc, and then in collect_id_sols you just do ids = getIds(id_no).

Making functions that only do one thing

I'm practicing writing functions that do only one thing. I'm getting stuck with the below functions. play_games is what the user calls to run the whole program. Ideally, it should only have the call to the play_many_games method which is in one of the program's classes.
But if I don't also add the line that calls the prepare_reports function, then the reports will never get taken care of. How do I "do reports and games" if I insist on only having functions that do one thing?
Is the bottom line that we need a main function that can do more than one thing (e.g. a main function that can do both games and reports calls)?
def play_games(number_of_games):
games_engine = prepare_reports(number_of_games)
games_engine.play_many_games(number_of_games)
def prepare_reports(number_of_games):
report_requests = []
if number_of_games <= 100:
report_on_game = GameReport()
report_requests.append(report_on_game)
report_on_many_games = ManyGamesReport()
report_requests.append(report_on_many_games)
return GamesEngine(*report_requests)

Having a main function that does nothing but drive the process and call other functions, is a function that only does one thing. I suggest you create one.

You could try this, since there's not much of a point in declaring games_engine:
def play_games(number_of_games):
prepare_reports(number_of_games).play_many_games(number_of_games)
def prepare_reports(number_of_games):
report_requests = []
if number_of_games <= 100:
report_on_game = GameReport()
report_requests.append(report_on_game)
report_on_many_games = ManyGamesReport()
report_requests.append(report_on_many_games)
return GamesEngine(*report_requests)
However, what you showed us is technically a function doing only one thing. You see, by declaring games_engine = prepare_reports(number_of_games), you are just making your code easier to read.

Is there an equivalent in Python of Fortran's "implicit none"?

In Fortran there is a statement Implicit none that throws a compilation error when a local variable is not declared but used. I understand that Python is a dynamically typed language and the scope of a variable may be determined at runtime.
But I would like to avoid certain unintended errors that happen when I forget to initialize a local variable but use it in the main code. For example, the variable x in the following code is global even though I did not intend that:
def test():
y=x+2 # intended this x to be a local variable but forgot
# x was not initialized
print y
x=3
test()
So my question is that: Is there any way to ensure all variables used in test() are local to it and that there are no side effects. I am using Python 2.7.x. In case there is a local variable, an error is printed.

So my question is that: Is there any way to ensure all variables used
in test() are local to it and that there are no side effects.
There is a technique to validate that globals aren't accessed.
Here's a decorator that scans a function's opcodes for a LOAD_GLOBAL.
import dis, sys, re, StringIO
def check_external(func):
'Validate that a function does not have global lookups'
saved_stdout = sys.stdout
sys.stdout = f = StringIO.StringIO()
try:
dis.dis(func)
result = f.getvalue()
finally:
sys.stdout = saved_stdout
externals = re.findall('^.*LOAD_GLOBAL.*$', result, re.MULTILINE)
if externals:
raise RuntimeError('Found globals: %r', externals)
return func
#check_external
def test():
y=x+2 # intended this x to be a local variable but forgot
# x was not initialized
print y
To make this practical, you will want a stop list of acceptable global references (i.e. modules). The technique can be extended to cover other opcodes such as STORE_GLOBAL and DELETE_GLOBAL.
All that said, I don't see straight-forward way to detect side-effects.

There is no implicit None in the sense you mean. Assignment will create a new variable, thus a typo might introduce a new name into your scope.
One way to get the effect you want is to use the following ugly-ish hack:
def no_globals(func):
if func.func_code.co_names:
raise TypeError(
'Function "%s" uses the following globals: %s' %
(func.__name__, ', '.join(func.func_code.co_names)))
return func
So when you declare your function test–with the no_globals wrapper–you'll get an error, like so:
>>> #no_globals
... def test():
... y = x + 2 # intended this x to be a local variable but forgot
... # x was not initialized
... print y
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in no_globals
TypeError: Function "test" uses the following globals: x
>>>
>>> x = 3
>>> test()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'test' is not defined

Just avoid using globally-scoped variables at all. And if you must, prefix their names with something you'll never use in a local variable name.

If you were really worried about this, you could try the following:
def test():
try:
x
except:
pass
else:
return
y = x+2
print y
But I'd recommend simply being mindful when writing a function that you don't try to reference things before assigning them. If possible, try to test each function separately, with a variety of carefully-defined inputs and intended outputs. There are a variety of testing suites and strategies, not to mention the simple assert keyword.

In Python, this is quite simply entirely legal. In fact, it is a strength of the language! This (lack) of error is the reason why you can do something like this:
def function1():
# stuff here
function2()
def function2():
pass
Whereas in C, you would need to "forward declare" function2.
There are static syntax checkers (like flake8) for Python that do plenty of work to catch errors and bad style, but this is not an error, and it is not caught by such a checker. Otherwise, something like this would be an error:
FILENAME = '/path/to/file'
HOSTNAME = 'example.com'
def main():
with open(FILENAME) as f:
f.write(HOSTNAME)
Or, something even more basic like this would be an error:
import sys
def main():
sys.stdout.write('blah')
The best thing you can do is use a different naming convention (like ALL_CAPS) for module level variable declarations. Also, make it a habit to put all of your code within a function (no module-level logic) in order to prevent variables from leaking into the global namespace.

Is there any way to ensure all variables used in test() are local to it and that there are no side effects.
No. The language offers no such functionality.
There is the built in locals() function. So you could write:
y = locals()['x'] + 2
but I cannot imagine anyone considering that to be an improvement.

To make sure the correct variable is used, you need to limit the scope of the lookup. Inside a function, Python will look to arguments defined in line, then to the args and kwargs. After those, its going to look outside the function. This can cause annoying bugs if the function depends on a global variable that gets changed elsewhere.
To avoid using a global variable by accident, you can define the function with a keyword argument for the variables your going to use:
def test(x=None):
y=x+2 # intended this x to be a local variable but forgot
# x was not initialized
print y
x=3
test()
I'm guessing you don't want to do this for lots of variables. However, it will stop the function from using globals.
Actually, even if you want to use a global variable in the function, I think its best to make it explicit:
x = 2
def test(x=x):
y=x+2 # intended this x to be a local variable but forgot
# x was not initialized
print y
x=3
test()
This example will use x=2 for the function no matter what happens to the global value of x afterwards. Inside the function, x is fixed to the value it had at compile time.
I started passing global variables as keyword arguments after getting burned a couple times. I think this is generally considered good practice?

The offered solutions are interesting, especially the one using dis.dis, but you are really thinking in the wrong direction. You don't want to write such a cumbersome code.
Are you afraid that you will reach a global accidentally? Then don't write globals. The purpose of module globals is mostly to be reached. (in a comment I have read that you have 50 globals in scope, which seems to me that you have some design errors).
If you still DO have to have globals, then either use a naming convention (UPPER_CASE is recommended for constants, which could cover your cases).
If a naming convention is not an option either, just put the functions you don't want to reach any global in a separate module, and do not define globals there. For instance, define pure_funcs and inside of that module, write your "pure" functions there, and then import this module. Since python has lexical scope, functions can only reach variables defined in outer scopes of the module they were written (and locals or built-ins, of course). Something like this:
# Define no globals here, just the functions (which are globals btw)
def pure1(arg1, arg2):
print x # This will raise an error, no way you can mix things up.

Call function from class without declaring name object

We have a Tree, each node is an object.
The function that this tree has are 3, add(x);getmin();getmax()
The tree works perfectly; for example if i write
a = Heap()
a.add(5)
a.add(15)
a.add(20)
a.getmin()
a.getmax()
the stack look like this [5,15,20], now if i call getmin() it will print min element = 5 and the stack will look like [15,20] and so on.
The problem comes now;
the professor asked us to submit two files which are already created: main.py and minmaxqueue.py
main.py starts like this from minmaxqueue import add, getmin, getmax, and then is has already a list of functions calls of the kind
add(5)
add(15)
add(20)
getmin()
getmax()
in order to make work my script i had to do a=Heap() and then call always a.add(x). Since the TA's are going to run the script from a common file, i cant modify main.py such that it creates an object a=Heap(). It should run directly with add(5) and not with a.add(5)
Is there a way to fix this?

You can modify your module to create a global Heap instance, and define functions that forward everything to that global instance. Like this:
class Heap(object):
# all of your existing code
_heap = Heap()
def add(n):
return _heap.add(n)
def getmin():
return _heap.getmin()
def getmax():
return _heap.getmax()
Or, slightly more briefly:
_heap = Heap()
add = _heap.add
getmin = _heap.getmin
getmax = _heap.getmax
If you look at the standard library, there are modules that do exactly this, like random. If you want to create multiple Random instances, you can; if you don't care about doing that, you can just call random.choice and it works on the hidden global instance.
Of course for Random it makes sense; for Heap, it's a lot more questionable. But if that's what the professor demands, what can you do?

You can use this function to do that more quickly:
def make_attrs_global(obj):
for attr in dir(obj):
if not attr.startswith('__'):
globals()[attr] = getattr(obj, attr)
It makes all attributes of obj defined in global scope.
Just put this code at the end of your minmaxqueue.py file:
a = Heap()
make_attrs_global(a)
Now you should be able to call add directly without a. This is ugly but well...

How to force local scope in Python?

In C++, you can do this to force local scope:
{
int i = 1;
// Do stuff
}
// local variable i is destroyed
{
int i = 7;
// Do more stuff
}
This has the benefit that by the end of a forced local scope, any variables declared in the bracket are gone. This can help prevent using a previously defined variable x in a place later on where you didn't intend to use x.
Can you do this in Python? If so, how?
==UPDATE==
I'm aware of functions - which is the obvious thing. I was wondering if there was a quick way to do the above when the code is simple and not worth creating separate a function for - just some quick notation to emphasize that the variables in this block are not to be used anywhere else in the function.
From what people have said so far the short answer is no.
(I understand that there are clever ways like "del", or that this desire to have blocks may suggest refactoring into a separate function anyway. However I would like to emphasize this is just for short snippets where you want to emphasize the variables in this small block are not to be used elsewhere.)

In Python, if you declare a variable inside a function, it is local and cannot be accessed outside the function
>>> def x():
i = 5
>>> x()
>>> i
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
i
NameError: name 'i' is not defined
>>>
Alternatively, you can delete the variable from the namespace at the end so that you cannot reuse it.
>>> i = 5
>>> del i
>>> i
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
i
NameError: name 'i' is not defined
>>>

I had this same question, and found out that you absolutely can!
It's not as clean as the c style blocks, but through two quirks of python we can make it serve our purposes.
The Quirks:
Whatever code is inside a class runs immediately, even if the class is never used.
You can reuse the name of a class as many times as you want.
Here's your example:
class DoStuff:
i = 1
# Do stuff
# local variable i is destroyed
class DoStuff:
i = 7
# Do more stuff
# local variable i is destroyed
To fully represent the flexibility here, see this example. I've named the class "Scope", because that's probably what I'd call it to differentiate from other named classes.
Note that "Scope" can of course be anything.
I'd recommend you stick with one name for the entire project and add that name to your documentation, so that there is an understanding that this is a special name that should never ever be instantiated.
outer = 1
class Scope:
inner = outer
print("runs first ---")
print("outer %d" % outer)
print("inner %d" % inner)
class Scope:
inner = outer + 1
print("runs second ---")
print("outer %d" % outer)
print("inner %d" % inner)
print("runs last ---")
print("outer %d" % outer)
print("inner %d" % inner) # This will give an error. Inner does not exist in this scope!
Output:
runs first ---
outer 1
inner 1
runs second ---
outer 1
inner 2
runs last ---
outer 1
Traceback (most recent call last):
File "test.py", line 18, in <module>
print("inner %d" % inner) # This will give an error. Inner does not exist in this scope!
NameError: name 'inner' is not defined
So it is doable - let's take a look at the benefits / downsides tradeoffs.
Benefits:
Code remains linear and no unnecessary leaps in logic are needed to follow the code flow. This linearity will make it easier for newcomers to read and understand what a section of code actually does.
Code is self-documenting to future coders that this code is only used in this one place, making it easier to edit, as the coder will not need to do an unnecessary search to find other instances.
Downsides:
We're using quirks of Python to make this work, and I sense that this very idea of limiting scope as opposed to creating new one-time-use functions is not something that Python programmers tend to do. This may cause tensions in the workplace, or result in complaints of using a hack as opposed to following conventions on creating small functions whether or not something is used more than once.
If you leave the project and new programmers come onboard and see this code, they will probably be confused initially. Some documentation will be needed in order to set expectations, and care must be taken to make sure the explanation in the documentation remains accurate.
I think this is a worthwhile effort for all code where you'd like to limit the scope but there are not multiple places this code is used, or it is not yet clear how to write a generic function to address all those situations.
If anyone reading this feels there are other tradeoffs, comment here and I'll make sure they're represented in the "Downsides" section.
Here's some more discussion around this convention, which has been preferred by John Carmack, Jonathan Blow, and Casey Muratori.
https://news.ycombinator.com/item?id=12120752

I have committed to solve this with trickery.
from scoping import scoping
a = 2
with scoping():
assert(2 == a)
a = 3
b = 4
scoping.keep('b')
assert(3 == a)
assert(2 == a)
assert(4 == b)
https://github.com/l74d/scoping
By the way, I found that the dummy class solution might result in memory leak. For example, large numpy arrays created in the overwritten class did not seem to be garbage collected by watching the memory statistics, which may be an implementation-dependent thing though.

If you don't like the del solution, you can nest function definitions:
def one_function():
x=0
def f():
x = 1
f()
print(x) # 0
Of course, I think the better approach is to just split things up into smaller functions, so there's no need for this manual scoping. In C++, the coolest thing about it is that the destructor is automatically called -- in Python, you can't really guarantee that the destructor will be called, so this scoping wouldn't be very useful even if it were possible.

In C++, you use local scope with brackets {} to avoid variable redefinitions or naming conflicts:
{
int var=3;
}
{
float var=1.0f;
}
While in python, there are no explicit variable definition , you just assign some objects to a var name when you want to use it, and rebind the same name to some new variable:
var=3
#do something
var=1.0 #no need to "del var", var refers to a float object now
#do more stuff
Note that the use of scope block in C++ might be indicating your code needs to be refactored to functions or methods, which can be named and reused. And it's the same with python.