We Keep Coding

Why are different objects created when using globals in a file vs importing them?

Below is a simple code example that may help to explain my question.
file_1.py
from functools import lru_cache
from file_2 import add_stuff, add_stats
#lru_cache()
def add(x, y):
return x + y
if __name__ == "__main__":
add(1, 2)
add(1, 2)
add(3, 4)
print(add.cache_info)
print(add.cache_info())
add_stuff(1, 2)
add_stuff(3, 4)
add_stats()
file_2.py
def add_stuff(x, y):
from file_1 import add
add(x, y)
def add_stats():
from file_1 import add
print(add.cache_info)
print(add.cache_info())
And the output looks like this:
<built-in method cache_info of functools._lru_cache_wrapper object at 0x017E9E48>
CacheInfo(hits=1, misses=2, maxsize=128, currsize=2)
<built-in method cache_info of functools._lru_cache_wrapper object at 0x017E9D40>
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)
When I use the function inside of the file it was defined in, the function object is different from when another file imports it. Which means that for things like lru_cache, if you didn't realize this, you could be populating two caches inside of your process/threads if you don't keep the cached functions inside of a different file from where they are used.
My question is, is this a python gotcha to look out for? Or is there documentation somewhere that I just never read that explains this more in depth? I looked at the lru_cache documentation, and this was not called out there as anything to be aware of.

When I use the function inside of the file it was defined in, the function object is different from when another file imports it.
Yes; there are two separate caches, because each is decorating a separate function object. The reason there are two separate function objects is because there are two separate modules created from the same source code.
One of these modules was created by from file_1 import add, which causes a module to be cached in the sys.modules with the key 'file_1' and a __name__ attribute of file_1. (Subsequent uses of import will look up this module in the cache).
The other one is created by running file_1.py as the main script. This causes a module with a __name__ attribute of __main__ to be created.
This is why and how the if __name__ == '__main__': trick works. The global variables available to a module - i.e., what you get by using globals() - come from attributes of the module object. Top-level scripts are also represented with module objects - they just aren't imported using import (although they are created using much of the same machinery, and cached; a '__main__' key will appear in sys.modules). That's where the information comes from, and thus why __name__ exists as a global variable in normal circumstances.
is this a python gotcha to look out for? Or is there documentation somewhere that I just never read that explains this more in depth? I looked at the lru_cache documentation, and this was not called out there as anything to be aware of.
It isn't explained in the lru_cache documentation because it isn't lru_cache's fault. It would happen with any decorator. In fact, it would happen with any code that makes the separate identity of the function objects relevant. For example, if we create this module_example.py:
def example():
print(example.module)
example.module = __name__
if __name__ == '__main__':
example()
(The reoccurrence of __name__ should make it obvious what is going on - although, of course, we could even just use __name__ directly in the function)
Now we test the code in interactive mode - run it, use the global function, and then import the module and use the imported function:
$ python -i module_example.py
__main__
>>> example()
__main__
>>> import module_example
>>> module_example.example()
module_example
>>> quit()
$
This is only a gotcha insofar as expecting a module to work as both the top-level code and as something importable, imposes some design considerations. Normally, if the code is intended to be imported, the "driver" code block (if any) will just do an informal test; or offer a simple, one-off UI for the module's functionality that doesn't care about consistency with an imported-module version of the same code.
Alternately put: the real problem here is a circular import. file_1 is indirectly importing itself to get at its own functionality, and it only "works" because of the implicit renaming of the module to __main__ the first time.

Cannot call function from within imported function

I have 2 files, the first is named function_call_test.py and contains the following code;
from Strategy_File import strategy
def function_1():
print('This works')
strategy()
The second file is called Strategy_File.py and contains the following code;
def strategy():
print('got here')
function_1()
When running the first script I get 'NameError: name 'function_1' is not defined'.
I thought that when you imported a function that it was added to the importing modules namespace. If that is the case why can't strategy() see function_1()?
Just as importantly, how do I make this work. The above is for demo purposes only, I have reasons for wanting strategy() to be in a separate module.
Python 3.6, Windows 7-64, Visual Studio 2019 and IDLE

Python is statically scoped. Lookup for a free variable (such as function_1) proceeds though the scopes where strategy is defined, not where it is called. As strategy is defined in the global scope of the module Strategy_File, that means looking for Strategy_File.function_1, and that function is not defined.
If you want strategy to call something in the current global scope, you need to define it to accept a callable argument, and pass the desired function when you call strategy.
# Strategy_File.py
# f is not a free variable here; it's a local variable
# initialized when strategy is called.
def strategy(f):
print('got here')
f()
and
# function_call_test.py
from Strategy_File import strategy
def function_1():
print('This works')
# Assign f = function_1 in the body of strategy
strategy(function_1)

You have to import every name into the file where it is used. So you need to modify Strategy_File.py to this:
from function_call_test import function_1
def strategy():
print('got here')
function_1()
But now you encounter a new problem: circular imports. Python won't allow this. So you will have to figure out a different way to organize your functions.

This error caused by Strategy_File.py is missing definition of function_1().
adding this line in Strategy_File.py on the top will helps.
edit: circular imports will not help. sorry for the misinformation.

accessing and changing module level variable [duplicate]

I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.

Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()

As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.

This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'

A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.

The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()

Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())

Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12

The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.

Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.

TypeError: gettext() missing 1 required positional argument: 'self' - python

I'm new to python and I'm tring to make a class for a modul which checking curses in texts.
can someone help please?
import urllib
class Checktext:
def __init__(self, text):
self.text = text
def gettext(self):
file = open(self.text, "r")
filetext = open.read()
for word in filetext.split():
openurl = urllib.request.urlopen("http://www.wdylike.appspot.com/?q=" + word)
output = openurl.read()
truer = "true" in str(output)
print(truer)
s = Checktext(r"C:\Users\Tzach\.atom\Test\Training\readme.txt")
Checktext.gettext()

You declared s as a new Checktext object, so you need to call s.gettext() not an un-instantiated Checktext.gettext(), as that has no self to refer to

The urllib is a package. You have to import the module request that is located in the package:
import urllib.request
The open(filename) return a file object. You want to call the method of that object:
filetext = file.read()
And as G. Anderson wrote, you want to call s.gettext() instead of Checktext.gettext(). The self inside is actually equal to the s outside. If you want to be weird then you actually can use also:
Checktext.gettext(s)
Notice the s passed as your missing parameter. Here Python actually reveals how the Object Oriented things are implemented internally. In majority of OO languages, it is carefully hidden, but calling a method of an object is always internally translated as passing one more special argument that points to the instance of the class, that is the object. When defining a Python method, that special argument is explicitly named self (by convention; you can name it differently -- you can try as the lecture, but you should always keep that convention).
Thinking about it thoroughly, you can get the key idea of the hidden magic of an OO language syntax. The instance of the class (the object) is actually only a portion of memory that stores the data part, and that is passed to the functions that implement the methods. The Checktext.gettext is actually the function, the s is the object. The s.gettext() is actually only a different way to express exactly the same. AS s is the instance of the Checktext class, the fact is stored inside the s. Therefore, the s.gettext() creates the illusion that the rigth code will be called magically. It fits with the trained brain better than the function approach if the s is thought as a tangible something.

TypeError: function missing 1 required positional argument: 'path' Flask Python

I have a static folder and my main usage is a sub-directory there that is found in
root/static/images/monkeys
I have a flask app and I have a variable like so:
app = Flask(__name__)
monkeys_folder_path = os.path.join(app.static_folder, 'images', 'monkeys')
I use it in two functions, one function serves a static image in that folder, this function works:
#app.route('/monkey/<address>')
def serve_static(address):
# this creates an image in /static/images/monkeys
monkey_generator.create_monkey_from_address(address)
filename = address + ".png"
return send_from_directory(monkeys_folder_path,filename)
I also have another function that uses this path, this function deletes images after X seconds from the folder
def remove_monkey_images(path):
threading.Timer(5.0, remove_monkey_images).start()
# this function iterates in a loop over the files in the path and deletes them
helper_functions.delete_images(path)
This function does not work, when I run the server locally I get
File "C:\Users\user\AppData\Local\Programs\Python\Python36-32\lib\threading.py", line 1182, in run
self.function(*self.args, **self.kwargs)
TypeError: remove_monkey_images() missing 1 required positional argument: 'path'
I call the function like so:
remove_monkey_images(path=monkeys_folder_path)
Thanks.

When you create the Timer, you have to pass it the called function's arguments, like this:
threading.Timer(5.0, remove_monkey_images, (path,)).start()
Source
As for the rest of your code I don't really know if it's consistent, but at least that's the cause of the error you're getting.

Python function can have positional or keyword parameters. Your function definition
def remove_monkey_images(path)
describe function with one positional parameter. This function could be called only with one positional argument like
remove_monkey_images(monkeys_folder_path)
If you want to use keyword argument you needs
def remove_monkey_images(path='/some_default_path')
In this case you could call function both with
remove_monkey_images(monkeys_folder_path)
and
remove_monkey_images(path=monkeys_folder_path)
and
remove_monkey_images()
In latter case inside function argument path will have default value '/some_default_path'.

there is a syntax issue with your problem.
Either do this:
remove_monkey_images(monkeys_folder_path)
instead of
remove_monkey_images(path=monkeys_folder_path)
or
Update your function definition as:
def remove_monkey_images(path=None):
threading.Timer(5.0, remove_monkey_images).start()
# this function iterates in a loop over the files in the path and deletes them
helper_functions.delete_images(path)