We Keep Coding

Python 3.8: KeyError resulting from a referral to vars() within a function... Even though the variable is clearly there and accessible

context: I have global variables ids1, ids2, ids3, ids4 and ids5.
When I attempt to execute this function
def collect_id_sols(id_no):
ids = vars()["ids" + str(id_no)]
sols = vars()["sols" + str(id_no)]
for line in range(0,len(ids)):
#rest irrelevant...
The interpreter throws a:
File "sols_from_ids.py", line 112, in <module>
collect_id_sols(1)
File "sols_from_ids.py", line 78, in collect_id_sols
ids = vars()["ids" + str(id_no)]
KeyError: 'ids1'
i.e. what it's telling me is that there is no such key "ids1".
However, the variable is CLEARLY existing and completely accessible.
Right after this error is thrown, if I do a >>>ids1 or >>>vars()["ids1"] within the interpreter, everything shows and works just as it should be.
What's going on? :(
P.S. And, of course, the global variables are declared and assigned before the function definition and call.

vars returns a dictionary giving the local scope, so it wouldn't know about these non-local variables. While there are ways (e.g. with globals()) to do what you want to do, a much better solution is to use a proper data structure such as a list or dictionary for your ids. Anytime you find yourself trying to iterate over a collection of variables which differ only by a number tacked onto the end, there is a data structure waiting to be born.

According to Python docs, vars() used without an argument acts like locals(), i.e. :
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
So when you use 'vars()' in your function block, it will return the symbol table of within the function block, which will be empty in your case. So you get a Key Error.
Using vars()["something"] to access the variable something only works at module-level, not inside functions or classes. Plus it is not a viable thing to do in my opinion.
What you probably want to do here is to use an array to store your variables idsX, where ids[1] will be ids1 etc. (or a dictionary)
Or you can have another function like :
def getIds(number) :
if number == 1 :
return ids1
elif number == 2 :
return ids2
etc, and then in collect_id_sols you just do ids = getIds(id_no).

Modify a __main__.variable from inside a module given the name of the variable as string

I am writing a module, which contains a function f(string).
A substring of string is the name of a variable that is in main and that I need to modify it so that the modifications are visible in main.
How can I do this without actually passing the variable to the function?
(The string is a SQL-like query, and the substring is a table name).
For now, I have only found a very ugly way involving modifying the builtin vars…
I have tried a lot of stuff involving exec, but nothing was able to change the variable in main spacename.
I have looked at different discussions, but wasn't able to fix it.

Modules are available via sys.modules. You could set the variable there.
def set_module_var(module_name, variable_name, value):
#sys.modules["__main__"].__dict__[name] = value
setattr(sys.modules[module_name], variable_name, value)
def f(string):
variable_name, value = do_some_magic(string)
set_module_var('__main__', variable_name, value)

Python: List static variables used inside a function

I want to define a function random_func with some arguments like arg_1 and arg_2 and some variables inside this function such as arg_3, arg_4.
How can I get a list of these arguments and parameters inside this function that starts with arg_ and return their corresponding values.
For example:
I list all variables by using dir() and then join all values by calling eval() function.
def random_func(arg_1, arg_2):
arg_3 = "a"
arg_4 = "b"
list = [name for name in dir() if name.startswith("arg_")]
return "-".join(eval(x) for x in list)
However, If I run, I get this error:
random_func(arg_1="one", arg_2="two")
>>> NameError: name 'arg_1' is not defined
Ideally I would get:
one-two-a-b
What can I do?

Note: This is a terrible idea, and I suspect an XY problem. If you keep reading, understand you're wandering off into the weeds, and probably not solving your real problem.
You can use locals() to get a read only view of the function's locals as a dict; it will give you access to both the name and the value without relying on eval (which is slow and more dangerous):
def random_func(arg_1, arg_2):
arg_3 = "a"
arg_4 = "b"
return "-".join([val for name, val in locals().items() if name.startswith("arg_")])
print(random_func("one", "two"))
Try it online!
As you desired, this prints one-two-a-b, and it does it without relying on eval at all.

Alter string in void function

I stumbled upon a low-level Python problem that I cannot understand. I want to do something to a string inside a function and keep those alterations.
name = 'name'
def get_new_name(name):
name = 'new_name'
get_new_name(name)
print(name) # expected 'new_name', got 'name'
I would expect that I get new_name printed, however, the result is name.
I know I could get what I want by using return in the function, but can I somehow use a void-like function as described here to get what i want?

You cannot modify immutable objects in python, no. Any reassignment does not modify the original object, but simply changes the reference that the variable points to.
With mutable objects, you can modify it contents, and the changes will be seen when accessing the object via any variables/references that point to it.
Understanding the difference between objects and references is the key to understanding this.
Since you're curious to know if this is possible... it is! But only if you wrap your string inside something else, say, a dict.
def get_new_name(d):
d['name'] = 'new_name'
name = 'name'
d = {'name' : name}
get_new_name(d)
name = d['name']
print(name)
'new_name'
You'd also need to decide on some protocol that you and the function agree upon for smooth communication between each other.
Sure, you could also use a global variable, but your question wasn't really about that... and I wouldn't recommend doing it either - it's bad practice to create impure functions.

Don't pass name to the function, and change the value of the global one. But it's not recommended...
def get_new_name():
global name
name = 'new_name'

your name variable in your void function is only local and does not share the same scope as your global variable name. Once your function finishes running, it discards the local variable with 'new_name' in it. Unless you include the print statement inside the function, which is pretty redundant.

If you want to simply access a global variable you just use its name.
However to change its value you need to use the global keyword.
From https://stackoverflow.com/a/10588342/6225257

How to create module-wide variables in Python? [duplicate]

This question already has answers here:
Using global variables in a function
(25 answers)
Closed 3 years ago.
The community reviewed whether to reopen this question 4 months ago and left it closed:
Original close reason(s) were not resolved
Is there a way to set up a global variable inside of a module? When I tried to do it the most obvious way as appears below, the Python interpreter said the variable __DBNAME__ did not exist.
...
__DBNAME__ = None
def initDB(name):
if not __DBNAME__:
__DBNAME__ = name
else:
raise RuntimeError("Database name has already been set.")
...
And after importing the module in a different file
...
import mymodule
mymodule.initDB('mydb.sqlite')
...
And the traceback was:
...
UnboundLocalError: local variable 'DBNAME' referenced before assignment
...
Any ideas? I'm trying to set up a singleton by using a module, as per this fellow's recommendation.

Here is what is going on.
First, the only global variables Python really has are module-scoped variables. You cannot make a variable that is truly global; all you can do is make a variable in a particular scope. (If you make a variable inside the Python interpreter, and then import other modules, your variable is in the outermost scope and thus global within your Python session.)
All you have to do to make a module-global variable is just assign to a name.
Imagine a file called foo.py, containing this single line:
X = 1
Now imagine you import it.
import foo
print(foo.X) # prints 1
However, let's suppose you want to use one of your module-scope variables as a global inside a function, as in your example. Python's default is to assume that function variables are local. You simply add a global declaration in your function, before you try to use the global.
def initDB(name):
global __DBNAME__ # add this line!
if __DBNAME__ is None: # see notes below; explicit test for None
__DBNAME__ = name
else:
raise RuntimeError("Database name has already been set.")
By the way, for this example, the simple if not __DBNAME__ test is adequate, because any string value other than an empty string will evaluate true, so any actual database name will evaluate true. But for variables that might contain a number value that might be 0, you can't just say if not variablename; in that case, you should explicitly test for None using the is operator. I modified the example to add an explicit None test. The explicit test for None is never wrong, so I default to using it.
Finally, as others have noted on this page, two leading underscores signals to Python that you want the variable to be "private" to the module. If you ever do an import * from mymodule, Python will not import names with two leading underscores into your name space. But if you just do a simple import mymodule and then say dir(mymodule) you will see the "private" variables in the list, and if you explicitly refer to mymodule.__DBNAME__ Python won't care, it will just let you refer to it. The double leading underscores are a major clue to users of your module that you don't want them rebinding that name to some value of their own.
It is considered best practice in Python not to do import *, but to minimize the coupling and maximize explicitness by either using mymodule.something or by explicitly doing an import like from mymodule import something.
EDIT: If, for some reason, you need to do something like this in a very old version of Python that doesn't have the global keyword, there is an easy workaround. Instead of setting a module global variable directly, use a mutable type at the module global level, and store your values inside it.
In your functions, the global variable name will be read-only; you won't be able to rebind the actual global variable name. (If you assign to that variable name inside your function it will only affect the local variable name inside the function.) But you can use that local variable name to access the actual global object, and store data inside it.
You can use a list but your code will be ugly:
__DBNAME__ = [None] # use length-1 list as a mutable
# later, in code:
if __DBNAME__[0] is None:
__DBNAME__[0] = name
A dict is better. But the most convenient is a class instance, and you can just use a trivial class:
class Box:
pass
__m = Box() # m will contain all module-level values
__m.dbname = None # database name global in module
# later, in code:
if __m.dbname is None:
__m.dbname = name
(You don't really need to capitalize the database name variable.)
I like the syntactic sugar of just using __m.dbname rather than __m["DBNAME"]; it seems the most convenient solution in my opinion. But the dict solution works fine also.
With a dict you can use any hashable value as a key, but when you are happy with names that are valid identifiers, you can use a trivial class like Box in the above.

Explicit access to module level variables by accessing them explicity on the module
In short: The technique described here is the same as in steveha's answer, except, that no artificial helper object is created to explicitly scope variables. Instead the module object itself is given a variable pointer, and therefore provides explicit scoping upon access from everywhere. (like assignments in local function scope).
Think of it like self for the current module instead of the current instance !
# db.py
import sys
# this is a pointer to the module object instance itself.
this = sys.modules[__name__]
# we can explicitly make assignments on it
this.db_name = None
def initialize_db(name):
if (this.db_name is None):
# also in local function scope. no scope specifier like global is needed
this.db_name = name
# also the name remains free for local use
db_name = "Locally scoped db_name variable. Doesn't do anything here."
else:
msg = "Database is already initialized to {0}."
raise RuntimeError(msg.format(this.db_name))
As modules are cached and therefore import only once, you can import db.py as often on as many clients as you want, manipulating the same, universal state:
# client_a.py
import db
db.initialize_db('mongo')
# client_b.py
import db
if (db.db_name == 'mongo'):
db.db_name = None # this is the preferred way of usage, as it updates the value for all clients, because they access the same reference from the same module object
# client_c.py
from db import db_name
# be careful when importing like this, as a new reference "db_name" will
# be created in the module namespace of client_c, which points to the value
# that "db.db_name" has at import time of "client_c".
if (db_name == 'mongo'): # checking is fine if "db.db_name" doesn't change
db_name = None # be careful, because this only assigns the reference client_c.db_name to a new value, but leaves db.db_name pointing to its current value.
As an additional bonus I find it quite pythonic overall as it nicely fits Pythons policy of Explicit is better than implicit.

Steveha's answer was helpful to me, but omits an important point (one that I think wisty was getting at). The global keyword is not necessary if you only access but do not assign the variable in the function.
If you assign the variable without the global keyword then Python creates a new local var -- the module variable's value will now be hidden inside the function. Use the global keyword to assign the module var inside a function.
Pylint 1.3.1 under Python 2.7 enforces NOT using global if you don't assign the var.
module_var = '/dev/hello'
def readonly_access():
connect(module_var)
def readwrite_access():
global module_var
module_var = '/dev/hello2'
connect(module_var)

For this, you need to declare the variable as global. However, a global variable is also accessible from outside the module by using module_name.var_name. Add this as the first line of your module:
global __DBNAME__

You are falling for a subtle quirk. You cannot re-assign module-level variables inside a python function. I think this is there to stop people re-assigning stuff inside a function by accident.
You can access the module namespace, you just shouldn't try to re-assign. If your function assigns something, it automatically becomes a function variable - and python won't look in the module namespace.
You can do:
__DB_NAME__ = None
def func():
if __DB_NAME__:
connect(__DB_NAME__)
else:
connect(Default_value)
but you cannot re-assign __DB_NAME__ inside a function.
One workaround:
__DB_NAME__ = [None]
def func():
if __DB_NAME__[0]:
connect(__DB_NAME__[0])
else:
__DB_NAME__[0] = Default_value
Note, I'm not re-assigning __DB_NAME__, I'm just modifying its contents.