You have three files: main.py, second.py, and common.py
common.py
#!/usr/bin/python
GLOBAL_ONE = "Frank"
main.py
#!/usr/bin/python
from common import *
from second import secondTest
if __name__ == "__main__":
global GLOBAL_ONE
print GLOBAL_ONE #Prints "Frank"
GLOBAL_ONE = "Bob"
print GLOBAL_ONE #Prints "Bob"
secondTest()
print GLOBAL_ONE #Prints "Bob"
second.py
#!/usr/bin/python
from common import *
def secondTest():
global GLOBAL_ONE
print GLOBAL_ONE #Prints "Frank"
Why does secondTest not use the global variables of its calling program? What is the point of calling something 'global' if, in fact, it is not!?
What am I missing in order to get secondTest (or any external function I call from main) to recognize and use the correct variables?
global means global for this module, not for whole program. When you do
from lala import *
you add all definitions of lala as locals to this module.
So in your case you get two copies of GLOBAL_ONE
The first and obvious question is why?
There are a few situations in which global variables are necessary/useful, but those are indeed few.
Your issue is with namespaces. When you import common into second.py, GLOBAL_ONE comes from that namespace. When you import secondTest it still references GLOBAL_ONE from common.py.
Your real issue, however, is with design. I can't think of a single logical good reason to implement a global variable this way. Global variables are a tricky business in Python because there's no such thing as a constant variable. However, convention is that when you want to keep something constant in Python you name it WITH_ALL_CAPS. Ergo:
somevar = MY_GLOBAL_VAR # good!
MY_GLOBAL_VAR = somevar # What? You "can't" assign to a constant! Bad!
There are plenty of reasons that doing something like this:
earth = 6e24
def badfunction():
global earth
earth += 1e5
print '%.2e' % earth
is terrible.
Of course if you're just doing this as an exercise in understanding namespaces and the global call, carry on.
If not, some of the reasons that global variables are A Bad Thing™ are:
Namespace pollution
Functional integration - you want your functions to be compartmentalized
Functional side effects - what happens when you write a function that modifies the global variable balance and either you or someone else is reusing your function and don't take that into account? If you were calculating account balance, all of the sudden you either have too much, or not enough. Bugs like this are difficult to find.
If you have a function that needs a value, you should pass it that value as a parameter, unless you have a really good reason otherwise. One reason would be having a global of PI - depending on your precision needs you may want it to be 3.14, or you may want it 3.14159265... but that is one case where a global makes sense. There are probably only a handful or two of real-world cases that can use globals properly. One of the cases are constants in game programming. It's easier to import pygame.locals and use KP_UP than remember the integer value responding to that event. These are exceptions to the rule.
And (at least in pygame) these constants are stored in a separate file - just for the constants. Any module that needs those constants will import said constants.
When you program, you write functions to break your problem up into manageable chunks. Preferably a function should do one thing, and have no side effects. That means a function such as calculatetime() should calculate the time. It probably shouldn't go reading a file that contains the time, and forbid that it should do something like write the time somewhere. It can return the time, and take parameters if it needs them - both of these are good, acceptable things for functions to do. Functions are a sort of contract between you (the programmer of the function) and anyone (including you) who uses the function. Accessing and changing global variables are a violation of that contract because the function can modify the outside data in ways that are not defined or expected. When I use that calculatetime() function, I expect that it will calculate the time and probably return it, not modify the global variable time which responds to the module time that I just imported.
Modifying global variables break the contract and the logical distinction between actions that your program takes. They can introduce bugs into your program. They make it hard to upgrade and modify functions. When you use globals as variables instead of constant, death awaits you with sharp pointy teeth!
Compare the results of the following to yours. When you use the correct namespaces you will get the results you expect.
common.py
#!/usr/bin/python
GLOBAL_ONE = "Frank"
main.py
#!/usr/bin/python
from second import secondTest
import common
if __name__ == "__main__":
print common.GLOBAL_ONE # Prints "Frank"
common.GLOBAL_ONE = "Bob"
print common.GLOBAL_ONE # Prints "Bob"
secondTest()
print common.GLOBAL_ONE # Prints "Bob"
second.py
#!/usr/bin/python
import common
def secondTest():
print common.GLOBAL_ONE # Prints "Bob"
Let me first say that I agree with everybody else who answered before saying that this is probably not what you want to do. But in case you are really sure this is the way to go you can do the following. Instead of defining GLOBAL_ONE as a string in common.py, define it as a list, that is, GLOBAL_ONE = ["Frank"]. Then, you read and modify GLOBAL_ONE[0] instead of GLOBAL_ONE and everything works the way you want. Note that I do not think that this is good style and there are probably better ways to achieve what you really want.
Related
I know you can use the global keyword. But that's not...global. It's just a way to reference another variable outside of this function. If you have one variable that you want to reference it's no big deal I guess (although still troublesome). But what if you have lots of variables that you want to move around functions? Do I have to declare those 2 or 3 or 10 variables each and every time in each function that are of global scope? Isn't there a way to declare[1] a variable (someplace or somehow) so as to be truly global?
Ideally I would like to have one file main.py with my main code and one file common.py with all my global variables (declared/initialized[1]) and with a from common import * I would be able to use every variable declared in common.py.
I apologize if it's a duplicate. I did search around but I can't seem to find something. Maybe you could do it with classes(?), but I'm trying to avoid using them for this particular program (but if there's no other way I'll take it)
EDIT: Adding code to show the example of two files not working:
file main.py
from common import *
def func1():
x=2
print('func1', x)
def func2():
print('fuc2',x)
print('a',x)
func1()
func2()
file common.py
x=3
this prints:
a 3
func1 2
fuc2 3
but it should print:
a 3
func1 2
fuc2 2
because func2 even though it's called after func1 (where x is assigned a value) sees x as 3. Meaning that fuc1 didn't actually use the "global" variable x but a local variable x different from the "global" one. Correct me If I'm wrong
[1] I know that in python you don't really declare variables, you just initialize them but you get my point
Technically, every module-level variable is global, and you can mess with them from anywhere. A simple example you might not have realized is sys:
import sys
myfile = open('path/to/file.txt', 'w')
sys.stdout = myfile
sys.stdout is a global variable. Many things in various parts of the program - including parts you don't have direct access to - use it, and you'll notice that changing it here will change the behavior of the entire program. If anything, anywhere, uses print(), it will output to your file instead of standard output.
You can co-opt this behavior by simply making a common sourcefile that's accessible to your entire project:
common.py
var1 = 3
var2 = "Hello, World!"
sourcefile1.py
from . import common
print(common.var2)
# "Hello, World!"
common.newVar = [3, 6, 8]
fldr/sourcefile2.py
from .. import common
print(common.var2)
# "Hello, World!"
print(common.newVar)
# [3, 6, 8]
As you can see, you can even assign new properties that weren't there in the first place (common.newVar).
It might be better practice, however, to simply place a dict in common and store your various global values in that - pushing a new key to a dict is an easier-to-maintain operation than adding a new attribute to a module.
If you use this method, you're going to want to be wary of doing from .common import *. This locks you out of ever changing your global variables, because of namespaces - when you assign a new value, you're modifying only your local namespace.
In general, you shouldn't be doing import * for this reason, but this is particular symptom of that.
I'm not an expert at python, so bear with me while I try to understand the nuances of variable scopes.
As a simple example that describes the problem I'm facing, say I have the following three files.
The first file is outside_code.py. Due to certain restrictions I cannot modify this file. It must be taken as is. It contains some code that runs an eval at some point (yes, I know that eval is the spawn of satan but that's a discussion for a later day). For example, let's say that it contains the following lines of code:
def eval_string(x):
return eval(x)
The second file is a set of user defined functions. Let's call it functions.py. It contains some unknown number of function definitions, for example, let's say that functions.py contains one function, defined below:
def foo(x):
print("Your number is {}!".format(x))
Now I write a third file, let's call it main.py. Which contains the following code:
import outside_code
from functions import *
outside_code.eval_string("foo(4)")
I import all of the function definitions from functions.py with a *, so they should be accessible by main.py without needing to do something like functions.foo(). I also import outside_code.py so I can access its core functionality, the code that contains an eval. Finally I call the function in outside_code.py, passing a string that is related to a function defined in functions.py.
In the simplified example, I want the code to print out "Your number is 4!". However, I get an error stating that 'foo' is not defined. This obviously means that the code in outside_code.py cannot access the same foo function that exists in main.py. So somehow I need to make foo accessible to it. Could anyone tell me exactly what the scope of foo currently is, and how I could extend it to cover the space that I actually want to use it in? What is the best way to solve my problem?
You'd have to add those names to the scope of outside_code. If outside_code is a regular Python module, you can do so directly:
import outside_code
import functions
for name in getattr(functions, '__all__', (n for n in vars(functions) if not n[0] == '_')):
setattr(outside_code, name, getattr(functions, name))
This takes all names functions exports (which you'd import with from functions import *) and adds a reference to the corresponding object to outside_code so that eval() inside outside_code.eval_string() can find them.
You could use the ast.parse() function to produce a parse tree from the expression before passing it to eval_function() and then extract all global names from the expression and only add those names to outside_code to limit the damage, so to speak, but you'd still be clobbering the other module namespace to make this work.
Mind you, this is almost as evil as using eval() in the first place, but it's your only choice if you can't tell eval() in that other module to take a namespace parameter. That's because by default, eval() uses the global namespace of the module it is run in as the namespace.
If, however, your eval_string() function actually accepts more parameters, look for a namespace or globals option. If that exists, the function probably looks more like this:
def eval_string(x, namespace=None):
return eval(x, globals=namespace)
after which you could just do:
outside_code.eval_string('foo(4)', vars(functions))
where vars(functions) gives you the namespace of the functions module.
foo has been imported into main.py; its scope is restricted to that file (and to the file where it was originally defined, of course). It does not exist within outside_code.py.
The real eval function accepts locals and globals dicts to allow you to add elements to the namespace of the evaluted code. But you can't do anything if your eval_string doesn't already pass those on.
The relevant documentation: https://docs.python.org/3.5/library/functions.html#eval
eval takes an optional dictionary mapping global names to values
eval('foo(4)', {'foo': foo})
Will do what you need. It is mapping the string 'foo' to the function object foo.
EDIT
Rereading your question, it looks like this won't work for you. My only other thought is to try
eval_str('eval("foo(4)", {"foo": lambda x: print("Your number is {}!".format(x))})')
But that's a very hackish solution and doesn't scale well to functions that don't fit in lambdas.
I want to define a few variables in my main python script and use them in a function which is defined in a separate module. Here is an example code. Lets say the main script is named main.py and the module is called mod.py.
Mod.py
def fun():
print a
main.py
from mod import *
global a
a=3
fun()
Now, this code gives me an error
NameError: global name 'a' is not defined
Can anyone please explain why the error is generated (i mean, a variable defined as global should be available to all functions, right?) and what may be a work-around? I already know about these two options and don't want to take any of these
Define the variable in the module instead of the main script.
pass the variable as argument to the function.
If there is any other option, please suggest.
Edit
I dont want to take the above options because
currently these values are fixed for me. But I suspect they may change in future (for example, database name and host ip). So, I want to store them as variables in one place. So that it becomes easy to edit the script in future. If I define the variables in each module, I will have to edit all of them.
I don't want to pass them in the functions because there are too many of them, some 50 or so. I know I can pass them as **kwarg, but that doesn't look too nice.
Global variables shared among modules are generally a bad idea. If you need them though (for example for some configuration purposes), you can do it like this:
global_config.py
# define the variable
a = 3
main.py
import global_config
def fun():
# use the variable
print(global_config.a)
This:
a variable defined as global should be available to all functions, right?
is just not true. That's not how global variables work; they are available to all functions in the module where they are defined.
You don't explain what you're doing or why those solutions don't work for you, but generally speaking global variables are a bad idea; passing the value explicitly is almost always the way to go.
I've come across recently a number of places in our code which do things like this
...
globals()['machine'] = otherlib.Machine()
globals()['logger'] = otherlib.getLogger()
globals()['logfile'] = datetime.datetime.now().strftim('logfiles_%Y_%m_%d.log')
and I am more than a little confused as to why people would do that, rather than doing
global machine
machine = otherlib.Machine()
and so on.
Here is a slightly anonymised function which does this, in full:
def openlog(num)
log_file = '/log_dir/thisprogram.' + num
if os.path.exists(log_file):
os.rename(log_file, log_file + '.old')
try:
globals()["log"] = open(log_file, 'w')
return log
except:
print 'Unable to open ' + log_file
sys.exit(1)
It confuses the hell out of pylint (0.25) as well me.
Is there any reason for coding it that way? There's minimal usage of eval in our code, and this isn't in a library
PS I checked Reason for globals() in python but it doesn't really answer as to why you'd use this for setting globals in a program
Maybe the function uses a local variable with the same name as the global one, and the programmer didn't want to bother changing the variable name?
def foo(bar):
global bar # SyntaxError
bar = bar + 1
def foo(bar):
globals()['bar'] = bar + 1
foo(1)
print(bar) # prints 2
Another use case, albeit still a bit specious (and clearly not the case in the example function you gave), is for defining variable names dynamically. This is rarely, if ever, a good idea, but it does come up a lot in questions on this site, at least. For example:
>>> def new_variable():
... name = input("Give your new variable a name! ")
... value = input("Give your new variable a value! ")
... globals()[name] = value
...
>>> new_variable()
Give your new variable a name! foo
Give your new variable a value! bar
>>> print(foo)
bar
Otherwise, I can think of only one reason to do this: perhaps some supervising entity requires that all global variables be set this way, e.g. "in order to make it really, really clear that these variables are global". Or maybe that same supervising entity has placed a blanket ban on the global keyword, or docks programmer pay for each line.
I'm not saying that any of these would be a good reason, but then again, I truly can't conceive of a good reason to define variables this way if not for scoping purposes (and even then, it seems questionable...).
Just in case, I did a timing check, to see if maybe the globals() call is faster than using the keyword. I'd expect the function call + dictionary access to be significantly slower, and it is.
>>> import timeit
>>> timeit.timeit('foo()', 'def foo():\n\tglobals()["bar"] = 1',number=10000000)
2.733132876863408
>>> timeit.timeit('foo()', 'def foo():\n\tglobal bar\n\tbar = 1',number=10000000)
1.6613818077011615
Given the code you posted and my timing results, I can think of no legitimate reason for the code you're looking at to be written like this. Looks like either misguided management requirement, or simple incompetence.
Are the authors PHP converts? This is a valid code in PHP:
$GLOBALS['b'] = $GLOBALS['a'] + $GLOBALS['b'];
See this for more examples. If someone was used to this way of writing the code, maybe they just used the closest matching way of doing it in Python and didn't bother to check for alternatives.
You'd sometimes use a superglobal $GLOBAL variable to define something, because although global keyword exists in PHP, it will only import existing variables - it cannot create a new variable as far as I know.
Does python have an equivalent to Tcl's uplevel command? For those who don't know, the "uplevel" command lets you run code in the context of the caller. Here's how it might look in python:
def foo():
answer = 0
print "answer is", answer # should print 0
bar()
print "answer is", answer # should print 42
def bar():
uplevel("answer = 42")
It's more than just setting variables, however, so I'm not looking for a solution that merely alters a dictionary. I want to be able to execute any code.
In general, what you ask is not possible (with the results you no doubt expect). E.g., imagine the "any code" is x = 23. Will this add a new variable x to your caller's set of local variables, assuming you do find a black-magical way to execute this code "in the caller"? No it won't -- the crucial optimization performed by the Python compiler is to define once and for all, when def executes, the exact set of local variables (all the barenames that get assigned, or otherwise bound, in the function's body), and turn every access and setting to those barenames into very fast indexing into the stackframe. (You could systematically defeat that crucial optimization e.g. by having an exec '' at the start of every possible caller -- and see your system's performance crash through the floor in consequence).
Except for assigning to the caller's local barenames, exec thecode in thelocals, theglobals may do roughly what you want, and the inspect module lets you get the locals and globals of the caller in a semi-reasonable way (in as far as deep black magic -- which would make me go postal on any coworker suggesting it be perpetrated in production code -- can ever be honored with the undeserved praise of calling it "semi-reasonable", that is;-).
But you do specify "I want to be able to execute any code." and the only solution to that unambiguous specification (and thanks for being so precise, as it makes answering easier!) is: then, use a different programming language.
Is the third party library written in Python? If yes, you could rewrite and rebind the function "foo" at runtime with your own implementation. Like so:
import third_party
original_foo = third_party.foo
def my_foo(*args, **kwds):
# do your magic...
original_foo(*args, **kwds)
third_party.foo = my_foo
I guess monkey-patching is slighly better than rewriting frame locals. ;)