So I have a dictionary with a bunch of names that I use to call functions. It works fine, but I prefer to put it in my settings file. If I do so, though, I will get errors from the settings file saying that there are no functions by that name(even though I'm not calling them at the time). Any workarounds?
def callfunct(id, time):
#stuff here
def callotherfunct(id, time):
#stuff here
dict = {"blah blah": callfunct, "blah blah blah": callfunct, "otherblah": callotherfunct}
dict[str(nameid)](id, time)
Hope this makes sense. Also open to other ideas, but basically I have about 50 iterations of these definitions and unique names that are passed by nameid that need to call specific functions, so that's why I do it the way I do, so that I can add new names quickly. It would obviously be even quicker if I could get the dictionary into the settings file seamlessly as well.
If you try
def f_one(id, time):
pass
def f_two(id, time):
pass
d = {"blah blah":"f_one", "blah blah blah":"f_one", "otherblah","f_two"
locals()[d[str(nameid)]](id, time)
(replacing the dictionary initialization with just loading the config file with the string name of the functions you want to call), does that work?
If not, there needs to be a little more info: What does the config file look like, and how are you loading it?
I'm guessing the reason that the config file part isn't working is that you're trying to reference the functions directly from the config file, which shouldn't work. This is using whatever's stored in the config file and looking it up in the locals() dictionary (if you're in a function, you'll have to use globals() instead)
You could initialise the dictionary with the looked up function only when you attempt to access it:
d = {}
d.setdefault('func1', globals()['func1'])()
Related
I found the following code snippet that I can't seem to make work for my scenario (or any scenario at all):
def load(code):
# Delete all local variables
globals()['code'] = code
del locals()['code']
# Run the code
exec(globals()['code'])
# Delete any global variables we've added
del globals()['load']
del globals()['code']
# Copy k so we can use it
if 'k' in locals():
globals()['k'] = locals()['k']
del locals()['k']
# Copy the rest of the variables
for k in locals().keys():
globals()[k] = locals()[k]
I created a file called "dynamic_module" and put this code in it, which I then used to try to execute the following code which is a placeholder for some dynamically created string I would like to execute.
import random
import datetime
class MyClass(object):
def main(self, a, b):
r = random.Random(datetime.datetime.now().microsecond)
a = r.randint(a, b)
return a
Then I tried executing the following:
import dynamic_module
dynamic_module.load(code_string)
return_value = dynamic_module.MyClass().main(1,100)
When this runs it should return a random number between 1 and 100. However, I can't seem to get the initial snippet I found to work for even the simplest of code strings. I think part of my confusion in doing this is that I may misunderstand how globals and locals work and therefore how to properly fix the problems I'm encountering. I need the code string to use its own imports and variables and not have access to the ones where it is being run from, which is the reason I am going through this somewhat over-complicated method.
You should not be using the code you found. It is has several big problems, not least that most of it doesn't actually do anything (locals() is a proxy, deleting from it has no effect on the actual locals, it puts any code you execute in the same shared globals, etc.)
Use the accepted answer in that post instead; recast as a function that becomes:
import sys, imp
def load_module_from_string(code, name='dynamic_module')
module = imp.new_module(name)
exec(code, mymodule.__dict__)
return module
then just use that:
dynamic_module = load_module_from_string(code_string)
return_value = dynamic_module.MyClass().main(1, 100)
The function produces a new, clean module object.
In general, this is not how you should dynamically import and use external modules. You should be using __import__ within your function to do this. Here's a simple example that worked for me:
plt = __import__('matplotlib.pyplot', fromlist = ['plt'])
plt.plot(np.arange(5), np.arange(5))
plt.show()
I imagine that for your specific application (loading from code string) it would be much easier to save the dynamically generated code string to a file (in a folder containing an __init__.py file) and then to call it using __import__. Then you could access all variables and functions of the code as parts of the imported module.
Unless I'm missing something?
I am trying to make generic config, and thus config parser. There are two config files say A and B. I want to parse sections and make global values from them according to hardcoded list.
Here is an example:
in config file:
[section]
var1 = value1
var2 = value2
In global scope:
some_global_list = [ ["var1","var2"], ["var3","var4"] ]
in function to unpack this values, by ConfigParser:
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
for variables in some_global_list:
globals()[section]=dict()
for element in configparser.items(section):
globals()[section].update({element[0]:element[1]})
This works nicely...however. Scope of globals() seem to be limited to function which is obviously not what I intended. I can access variable only while in that function.
Could someone share better yet simple idea?
I know that i might move code to main and not to worry, but I don't think it is a good idea.
I thought also about making some generator (sorry for pseudocode here):
in global scope:
for x in some_global_list:
globals()[x] = x
also tried adding this to function:
for x in some_global_list[0]:
global x
but got nowhere.
Edit :
After discussion below, here it is:
Problem solved like this:
removed whole configuration from main script to module
imported (from module import somefunction) config from module
removed globals() in fact didnt need them, since function was changed a little like so:
in function:
def somefunction:
#(...)
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
temp_list=[]
for variables in some_global_list:
tmp=dict()
for element in configparser.items(section):
tmp.update({element[0]:element[1]})
temp_list.append (tmp)
return temp_list #this is pack for one file.
now in main script
tmp=[]
for i,conf_file in enumerate([args.conf1,args.conf2,args.conf3]):
if conf_file:
try:
tmp.append([function(params...)])
except:
#handling here
#but since i needed those config names as global variables
for j,variable_set in enumerate(known_variable_names[i]):
globals()[variable_set] = tmp[i][j]
so unfortunate hack presists. But seems to work. Thx for Your help guys.
I'm accepting (if thats possible) below answer since it gave me good idea :)
A simple way to solve this issue is in your application package within the __init__.py you can do something similar to the following:
app_config = read_config()
def read_config():
configparser = ConfigParser.RawConfigParser()
configparser.read(some_config_filename)
return configparser.as_dict() #An imaginery method which returns the vals as dict.
The "app_config" variable can be imported into any other module within the application.
I have a dictionary called fsdata at module level (like a global variable).
The content gets read from the file system. It should load its data once on the first access. Up to now it loads the data during importing the module. This should be optimized.
If no code accesses fsdata, the content should not be read from the file system (save CPU/IO).
Loading should happen, if you check for the boolean value, too:
if mymodule.fsdata:
... do_something()
Update: Some code already uses mymodule.fsdata. I don't want to change the other places. It should be variable, not a function. And "mymodule" needs to be a module, since it gets already used in a lot of code.
I think you should use Future/Promise like this https://gist.github.com/2935416
Main point - you create not an object, but a 'promise' about object, that behave like an object.
You can replace your module with an object that has descriptor semantics:
class FooModule(object):
#property
def bar(self):
print "get"
import sys
sys.modules[__name__] = FooModule()
Take a look at http://pypi.python.org/pypi/apipkg for a packaged approach.
You could just create a simple function that memoizes the data:
fsdata = []
def get_fsdata:
if not fsdata:
fsdata.append(load_fsdata_from_file())
return fsdata[0]
(I'm using a list as that's an easy way to make a variable global without mucking around with the global keyword).
Now instead of referring to module.fsdata you can just call module.get_fsdata().
I have a problem similar to the first problem in this question, which as far as I can see went unanswered.
I have a file "config.py" which contains a lot of parameters to be used by a class (this config.py file will change), however I can't get these to propagate into the class via execfile.
In an example piece of code:
class Class():
def __init__(self):
execfile("config.py")
print x
# config.py
x = "foo"
>>> t = Class()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __init__
NameError: global name 'x' is not defined
Any help welcome, or any better methods of retrieving parameters from a file to be used in a class.
Many Thanks.
I don't get what you're trying to do (but i don't like it, and this is just me) but to fix your problem do (test in python2.6):
class Class():
def __init__(self):
execfile('config.py', locals()) # Not recommanded, maybe you want globals().
print x
But from the doc:
Note
The default locals act as described
for function locals() below:
modifications to the default locals
dictionary should not be attempted.
Pass an explicit locals dictionary if
you need to see effects of the code on
locals after function execfile()
returns. execfile() cannot be used
reliably to modify a function’s
locals.
and about :
Any help welcome, or any better
methods of retrieving parameters from
a file to be used in a class.
You can use import.
Even though it might be convenient to keep configuration settings in a Python file I would recommend against it. I think it opens up a whole set of problems that you don't really want to have do deal with. Anything could be placed in your configuration file, including malicious code.
I would use either the json module or the ConfigParser module to hold my configuration.
If you have trouble choosing between those two I would recommend the json module. Json is a simple yet flexible format for structured data.
I'd like to init a class from data stored in a simple python file specified while calling the script. The config file named myconfig.py is :
str='home'
val=2
flt=7.0
I'd like to call it during class initilization like so. One of the objectives is to define variable types as well in the file. I know of the configparser, but this method less verbose if it can be made to work.
class ClassInit(object):
def __init__(self, configFile):
fp, path, des = imp.find_module('',configFile)
imp.load_module(configFile, fp, path, des)
self.__dict__ = configFile.__dict__
fp.close()
def printVal(self):
print '%s %0.2f'%(self.str, self.val)
if __name__ == '__main__':
srcDir = 'src/'
config = osp.join(srcDir, argv[0]) # config for current run
ci = ClassInit(config)
ci.printVal()
Is anything like this possible?
Well, there are several ways to do this. The easiest way would be to use eval() or exec to evaluate this code within the class scope. But that's also the most dangerous way, especially if these files can be created by someone other than you. In that case, the creator can write malicious code that can pretty much do anything. You can override the __builtins__ key of the globals dictionary, but I'm not sure if this makes eval/exec entirely safe. For example:
class ClassInit(object):
def __init__(self, configFile):
f = open(configFile)
config = f.read()
f.close()
config_dic = { '__builtins__': None}
exec 'a = 4' in config_dic
for key, value in config_dic.iteritems():
if key != '__builtins__':
setattr(self, key, value)
This method kills the unsafe 'builtins' object, but it's still not quite safe. For instance, the file may be able to define a function which would override one of your class's functions with malicious code. So I really don't recommend it, unless you absolutely control thos .py files.
A safer but more complex way would be to create a custom interpreter that interprets this file but doesn't allow running any custom code.
You can read the following thread, to see some suggestions for parsing libraries or other safer alternatives to eval():
Python: make eval safe
Besides, if all you ever need your config.py file for is to initialize some variables in a nice way, and you don't need to be able to call fancy python functions from inside it, you should consider using JSON instead. Python 2.6 and up includes simplejson, which you can use to initialize an object from file. The syntax is Javascript and not Python, but for initializing variables there's little difference there.
Can you try self.__dict__.update(configFile.__dict__)? I don't see why that wouldn't work.