I'm calling a notebook like this:
dbutils.notebook.run(path, timeout, arguments)
where arguments is a dictionary containing many fields for the notebook's widgets.
I want to debug called notebook interactively: copy/pasting the widget parameters takes time and can cause hard-to-spot errors not done perfectly.
It would be nice to just take the arguments dictionary and use it directly. Perhaps copying it, then populating the widgets from the dictionary.
How can I do this, or something like it?
If we get some variables like this:
dbutils.widgets.text('myvar', '-1', '')
myvar = dbutils.widgets.get('myvar')
I can override them like this:
config = {'myvar': '42'}
import sys
module = sys.modules[__name__]
for k, v in config.items():
setattr(module, k, v)
Which means all the overriding happens in a cell I can later delete, leaving no edits in the real code.
Pass the values as a json like below in the widget let it be "Filters"
{"Type":"Fruit","Item":"Apple"]
To read the json you need to make use of json library
import json
filters = dbutils.widgets.get("Filters")
jsonfilter = json.loads(filters)
Now you can access individual items by
jsonfilter["Item"]
Related
I found the following code snippet that I can't seem to make work for my scenario (or any scenario at all):
def load(code):
# Delete all local variables
globals()['code'] = code
del locals()['code']
# Run the code
exec(globals()['code'])
# Delete any global variables we've added
del globals()['load']
del globals()['code']
# Copy k so we can use it
if 'k' in locals():
globals()['k'] = locals()['k']
del locals()['k']
# Copy the rest of the variables
for k in locals().keys():
globals()[k] = locals()[k]
I created a file called "dynamic_module" and put this code in it, which I then used to try to execute the following code which is a placeholder for some dynamically created string I would like to execute.
import random
import datetime
class MyClass(object):
def main(self, a, b):
r = random.Random(datetime.datetime.now().microsecond)
a = r.randint(a, b)
return a
Then I tried executing the following:
import dynamic_module
dynamic_module.load(code_string)
return_value = dynamic_module.MyClass().main(1,100)
When this runs it should return a random number between 1 and 100. However, I can't seem to get the initial snippet I found to work for even the simplest of code strings. I think part of my confusion in doing this is that I may misunderstand how globals and locals work and therefore how to properly fix the problems I'm encountering. I need the code string to use its own imports and variables and not have access to the ones where it is being run from, which is the reason I am going through this somewhat over-complicated method.
You should not be using the code you found. It is has several big problems, not least that most of it doesn't actually do anything (locals() is a proxy, deleting from it has no effect on the actual locals, it puts any code you execute in the same shared globals, etc.)
Use the accepted answer in that post instead; recast as a function that becomes:
import sys, imp
def load_module_from_string(code, name='dynamic_module')
module = imp.new_module(name)
exec(code, mymodule.__dict__)
return module
then just use that:
dynamic_module = load_module_from_string(code_string)
return_value = dynamic_module.MyClass().main(1, 100)
The function produces a new, clean module object.
In general, this is not how you should dynamically import and use external modules. You should be using __import__ within your function to do this. Here's a simple example that worked for me:
plt = __import__('matplotlib.pyplot', fromlist = ['plt'])
plt.plot(np.arange(5), np.arange(5))
plt.show()
I imagine that for your specific application (loading from code string) it would be much easier to save the dynamically generated code string to a file (in a folder containing an __init__.py file) and then to call it using __import__. Then you could access all variables and functions of the code as parts of the imported module.
Unless I'm missing something?
I am using iPython in command prompt, Windows 7.
I thought this would be easy to find, I searched and found directions on how to use the inspect package but it seems like the inspect package is meant to be used for functions that are created by the programmer rather than functions that are part of a package.
My main goal to to be able to use the help files from within command prompt of iPython, to be able to look up a function such as csv.reader() and figure out all the possible arguments for it AND all possible values for these arguements.
In R programming this would simply be args(csv.reader())
I have tried googling this but they all point me to the inspect package, perhaps I'm misunderstanding it's use?
For example,
If I wanted to see a list of all possible arguments and the corresponding possible values for these arguments for the csv.reader() function (from the import csv package), how would I go about doing that?
I've tried doing help(csv.reader) but this doesn't provide me a list of all possible arguments and their potential values. 'Dialect' shows up but it doesn't tell me the possible values of the dialect argument of the csv.reader function.
I can easily go to the site: https://docs.python.org/3/library/csv.html#csv-fmt-params and see that the dialect options are: delimiter, doublequote, escapechar, etc.. etc..but is there a way to see this in Python console?
I've also tried dir(csv.reader) but this isn't what I was looking for either.
Going bald trying to figure this out....
There is no way to do this generically, help(<function>) will at a minimum return you the function signature (including the argument names), Python is dynamically typed so you don't get any types and arguments by themselves don't tell you what the valid values are. This is where a good docstring comes in.
However, the csv module does have a specific function for listing the dialects:
>>> csv.list_dialects()
['excel', 'excel-tab', 'unix']
>>> help(csv.excel)
Help on class excel in module csv:
class excel(Dialect)
| Describe the usual properties of Excel-generated CSV files.
...
The inspect module is extremely powerful. To get a list of classes, for example in the csv module, you could go:
import inspect, csv
from pprint import pprint
module = csv
mod_string = 'csv'
module_classes = inspect.getmembers(module, inspect.isclass)
for i in range(len(module_classes)):
myclass = module_classes[i][0]
myclass = mod_string+'.'+myclass
myclass = eval(myclass)
# could construct whatever query you want about this class here...
# you'll need to play with this line to get what you want; it will failasis
#line = inspect.formatargspect(*inspect.getfullargspec(myclass))
pprint(myclass)
Hope this helps get you started!
i can't get the printing of a dict. to work, seen alot exsample codes but none have worked
from UI import ask_info
def main():
mydict = {}
def add(mydict):
info=ask_info()
mydict=info
for i in mydict:
print(mydict[i])
def ask_info():
info1=input("enter the info")
info2=input("enter the info2")
informations=mydict(info1,info2)
return informations
the ask_info is in an different module, if i run this code the print is nothing also if write the dictionary into a file the file is empty. I also have tryed using a class module for this where i have refered to the class module in the "informations=class(info1,info2)" part
As far as I know, dictionaries in Python are not callable and don't work like that. Instead of mydict(info1,info2) you probably want something like mydict[info1] = info2.
Check this link to see how dictionaries work. If you are creating a new dictionary, you can do something like dict([(info1,info2)]) but it does not seem you are trying to do that.
I have a dictionary of addresses with their usernames and passwords listed that looks something like this:
address_dict = {'address1':{'username':'abc', 'password':'123'}, 'address2':{'username':'xyz', 'password':'456'}}
Is there a way to make this dictionary accessible for multiple scripts to read from and possibly write to? Like save it as seperate python file and import it or something?
Yes, you can do just that:
# module.py
address_dict = {'address1':{'username':'abc', 'password':'123'}, 'address2':{'username':'xyz', 'password':'456'}}
# main.py
import module
print(module.address_dict)
If you don't like the module. prefix, you could import the dictionary like so:
from module import address_dict
print(address_dict)
To access it and modify it at runtime, you can just define it in a module and then import it. But if you want your changes to be persistent (i.e. see the changed version next time you run the script) you need something else, like a database.
The simplest to use in this case would probably be the shelve module, which is based on pickle. You can also use pickle itself if you wish.
take a look at pickle :)
http://docs.python.org/2/library/pickle.html
You can use it to dump objects to files and also to read them back in with any other python script.
So I have a dictionary with a bunch of names that I use to call functions. It works fine, but I prefer to put it in my settings file. If I do so, though, I will get errors from the settings file saying that there are no functions by that name(even though I'm not calling them at the time). Any workarounds?
def callfunct(id, time):
#stuff here
def callotherfunct(id, time):
#stuff here
dict = {"blah blah": callfunct, "blah blah blah": callfunct, "otherblah": callotherfunct}
dict[str(nameid)](id, time)
Hope this makes sense. Also open to other ideas, but basically I have about 50 iterations of these definitions and unique names that are passed by nameid that need to call specific functions, so that's why I do it the way I do, so that I can add new names quickly. It would obviously be even quicker if I could get the dictionary into the settings file seamlessly as well.
If you try
def f_one(id, time):
pass
def f_two(id, time):
pass
d = {"blah blah":"f_one", "blah blah blah":"f_one", "otherblah","f_two"
locals()[d[str(nameid)]](id, time)
(replacing the dictionary initialization with just loading the config file with the string name of the functions you want to call), does that work?
If not, there needs to be a little more info: What does the config file look like, and how are you loading it?
I'm guessing the reason that the config file part isn't working is that you're trying to reference the functions directly from the config file, which shouldn't work. This is using whatever's stored in the config file and looking it up in the locals() dictionary (if you're in a function, you'll have to use globals() instead)
You could initialise the dictionary with the looked up function only when you attempt to access it:
d = {}
d.setdefault('func1', globals()['func1'])()