First I do know about 'import'. When I try 'import' it doesn't work. What I'm trying to do is split a single module into two parts, one of which is editable by a group and the other of which is not. I want the group to write well-defined 'retrieval functions' without the temptation or ability to edit the backend code that runs them (even accidentally). The changes in namespace on an 'import' are getting in my way. I'm looking for a macro-style inclusion of File_A's text within File_B, to be run inline as if it were part of File_B.
This is what I'm doing:
I have some generalized code that is designed to call a list of information retrieval functions in turn, and store the info in a unified way. To do this I add its text name to a list:
DataTypes = ['TypeA','TypeB','TypeC']
... and then define a function that knows how to get each type, and returns a populated object class:
def Get_TypeA:
# do some stuff to retrieve info
InfoObj Data
# Populate Data with the info I got
return Data
def Get_TypeB:
# etc. etc.
return Data
def Get_TypeC:
# etc. etc.
return Data
# Backend code below this line, hopefully nobody touches it?
# (But really it would be best if this was in a different file
# that is harder to mess with and has locked-down permissions.)
class InfoObj:
# stuff stuff definitions methods etc.
These functions will be edited by people with only a basic knowledge of Python but often bad coding habits, but who need to be able to frequently customize what is gathered and displayed. I already have a backend that checks the list and calls the functions, but I just want to move those definitions into a separate file from the rest of the backend code, but work as if they were right there inline. (i.e. turn the "hopefully nobody touches it" into a "people have to go out of their way to touch it")
Does Python have this?
When I try using Import, the retrieval functions lose contact with the definition of InfoObj. This happens whether I use 'import otherFile' or 'from otherFile import *'.
EDIT: This is what I'm using to retrieve the various types in a standard way:
THISMODULE = sys.modules[__name__]
for type in DataTypes:
RetrievalFn = getattr( THISMODULE, 'Get_'+type )
Data = RetrievalFn()
StoreInDB(Data)
(though this is boiled down to basics, there are try/except clauses, validation steps to make sure Data was populated correctly and doesn't contain incorrect types or bad stuff, and code to notify the team if and where something breaks but still process the rest of the items. The goal is that someone making edits does not break the rest of the checks, even if their edits broke one of the checks.)
This can be done with execfile(), but you should consider using some other mechanism for this instead such as polymorphism or plugins.
Related
I am designing python assignments for a class. I define functions, write docstrings and then I implement them. Afterward, I'd like to remove all my implementations of the functions and replace only the code (not the doc-strings, function names, and arguments) with a raise NotImplementedError.
Is there any tool (e.g. IDE) which removes all the code for me automatically, so that I don't have to replace the implemented function by myself? I was thinking about writing a small script, but I thought I might ask here before I do this ...
If anyone has written something similar or knows of a quick way how to this, I would appreciate this a lot.
Here's a minimal example of what I'd like to achieve:
test.py
def add(a,b):
"""
Adds two numbers
"""
return a+b
def multiply(a,b):
"""
Multiplies two numbers
"""
return a*b
should become in an automated fashion (and of course for much larger files):
test.py
def add(a,b):
"""
Adds two numbers
"""
raise NotImplementedEror
def multiply(a,b):
"""
Multiplies two numbers
"""
raise NotImplementedEror
I don't know of a tool to do specifically this, but Python provides great AST manipulation tools within its own standard library via the ast module. You'll need a third party module to "unparse" the result after transformation back into regular Python code, and after a quick search I found this one seems to do the trick, although there do seem to be many others.
Here's a bit of sample code to get you in the right direction. Obviously, you'll need to tweak this to get the exact behavior you want, especially if you want to provide classes instead of just top-level functions (as I've written nothing to handle that use case). But with a bit of Python knowledge, it should be possible to automate.
Also, this is Python 3 (which, as of the start of 2020, is the only supported Python version). If you're still on Python 2, it may require some modifications.
import ast
import astunparse
# Read our file using the built-in Python AST module.
with open('filename.py') as f:
data = ast.parse(f.read(), 'filename.py')
# Loop through all declarations.
for decl in data.body:
# Only modify functions
if isinstance(decl, ast.FunctionDef):
# The docstring is the first statement of the body. So we don't
# want to change it. Instead, replace the rest of the body with
# our pre-built "raise" call. Note that I figured out what "raise"
# looked like in AST form by running
#
# ast.dump(ast.parse("raise NotImplementedError()"))
#
decl.body[1:] = [ast.Raise(ast.Call(ast.Name('NotImplementedError'), [], []), None)]
# Use astunparse to pretty print the result as Python code.
print(astunparse.unparse(data))
It's definitely possible to automate, if you're willing to take the time to do it. If you're planning to do this for several assignments, or even over several semesters, you may consider making a script for it. But if you're just doing it once, it may be more worth your time to just do it by hand.
There are bits of code that I'd like to customize. Say I want to assign a bunch of student applications to a summer program to various readers (so 100 apps, 3 readers, divide them roughly evenly, etc). In some cases, I want to take reader preferences into consideration (I only want to read applications from students in California, etc.). In other cases, I don't care who they are assigned to. Right now I've got something that looks roughly like this:
def assign_readers(preferences_flag):
if preferences_flag:
assign_readers_with_preferences()
assign_remaining
I've got multiple cases of similar features throughout my code that I would like to easily turn on/off, but it doesn't look like a necessarily clean way of doing it. The same flag is sometimes used in other parts of the code, so I'm passing around these flags left and right. For example:
def log_reader_stats(preferences_flag, other_flag):
if preferences_flag:
log_reader_stats_with_preferences()
if other_flag:
log_readers_stats_with_other_stuff()
log_remaining_stats
What is an alternative way of doing this? Passing flags around seems repetitive and inefficient, but other than this I'm not sure how I can "toggle" such features on and off.
Below is an example of how some of the actual code being used, and how the flags come into play.
USE_PREF = True
USE_SPEC_GRP = True
def main():
# Load and store config file information
fnames = {}
snames = {}
options = read_config_file()
validate_config_params(options, fnames, snames)
# Load the applications file
apps = pio.file_to_frame(fnames["apps"], snames["apps"])
# load target and max number of apps each reader can handle.
rdr_counts = pio.file_to_frame(fnames["rdr_counts"], snames["rdr_counts"])
# Assign applications depending on which options are enabled
if USE_SPEC_GRP:
assign_all_special_groups(apps, fnames["spec_grp"], snames["spec_grp"])
if USE_PREF:
assign_apps_with_prefs(apps, rdr_counts,
fnames["rdr_prefs"], snames["rdr_prefs"])
assign_remaining_apps(apps, rdr_counts, fnames, snames)
Although you didn't ask this, there was a code smell that warrants explanation. Whenever you find yourself using parallel data sources like fnames and snames in:
assign_all_special_groups(apps, fnames["spec_grp"], snames["spec_grp"])
you are usually making error prone code. Instead you could have
names['spec_grp'] = ('something', 'anotherthing')
which ensures that the elements of spec_grp always remain associated with each other. There exists a namedtuple type which makes access very readable:
names['spec_grp'].f_thing
names['spec_grp'].s_thing
but without getting into that slight complexity you'll need to access them with
names['spec_grp'][0]
names['spec_grp'][1]
If I am reading your intent properly, the code above could combine these values with the option flags so that
options['spec_grp'] = (fname_for_spec_grp, sname_for_spec_group)
if options['spec_grp']:
assign_all_special_groups(apps, options["spec_grp"][0], options["spec_grp"][1])
This makes initializing configuration elements that have no value to None important but it is also good practice.
But didn't I just make your calling code longer and harder to read? Kinda. Did it buy you flexibility, maintainability, and safety for a few extra characters? Yep. And it does turn three data structures (options, fnames, snames) into one dictionary which signals if an option is desired and if so, what its arguments are.
You can simply create a class ReadersManager with a property flags, make the functions methods of that class and access self.flags inside them.
The best way is to look at how others have done it, in this case in the ConfigParser standard module. This uses dictionaries to store and retrieve configuration data (well actually it uses dictionaries of dictionaries but we needn't). The key point is that a dictionary can associate a name with most everything, and using data to describe the location of data is far better than hardcoding it.
In your case the dictionary
options = {
'USE_SPEC_GROUP': False,
'USE_PREF': False,
}
but it is a dictionary so I can add to it as needed
options['available'] = False
or even do bulk initialization easily:
options = {}
for option in "car plane train boat".split():
options[option] = False
of course accessing them in conditionals is easy
if options['boat']:
# do boat things
And now you have one variable to pass around that contains all the configuration data:
some_function(options)
No need to use a class when a fundamental type like dict is so useful on its own.
so i know this is a bit of a workaround and theres probably a better way to do this, but heres the deal. Ive simplified the code from where tis gathering this info from and just given solid values.
curSel = nuke.selectedNodes()
knobToChange = "label"
codeIn = "[value in]"
kcPrefix = "x"
kcStart = "['"
kcEnd = "']"
changerString = kcPrefix+kcStart+knobToChange+kcEnd
for x in curSel:
changerString.setValue(codeIn)
But i get the error i figured i would - which is that a string has no attribute "setValue"
its because if i just type x['label'] instead of changerString, it works, but even though changer string says the exact same thing, its being read as a string instead of code.
Any ideas?
It looks like you're looking for something to evaluate the string into a python object based on your current namespace. One way to do that would be to use the globals dictionary:
globals()['x']['label'].setValue(...)
In other words, globals()['x']['label'] is the same thing as x['label'].
Or to spell it out explicitly for your case:
globals()[kcPrefix][knobToChange].setValue(codeIn)
Others might suggest eval:
eval('x["label"]').setValue(...) #insecure and inefficient
but globals is definitely a better idea here.
Finally, usually when you want to do something like this, you're better off using a dictionary or some other sort of data structure in the first place to keep your data more organized
Righto, there's two things you're falling afoul of. Firstly, in your original code where you are trying to do the setValue() call on a string you're right in that it won't work. Ideally use one of the two calls (x.knob('name_of_the_knob') or x['name_of_the_knob'], whichever is consistent with your project/facility/personal style) to get and set the value of the knob object.
From the comments, your code would look like this (my comments added for other people who aren't quite as au fait with Nuke):
# select all the nodes
curSel = nuke.selectedNodes()
# nuke.thisNode() returns the script's context
# i.e. the node from which the script was invoked
knobToChange = nuke.thisNode()['knobname'].getValue()
codeIn = nuke.thisNode()['codeinput'].getValue()
for x in curSel:
x.knob(knobToChange).setValue(codeIn)
Using this sample UI with the values in the two fields as shown and the button firing off the script...
...this code is going to give you an error message of 'Nothing is named "foo"' when you execute it because the .getValue() call is actually returning you the evaluated result of the knob - which is the error message as it tries to execute the TCL [value foo], and finds that there isn't any object named foo.
What you should ideally do is instead invoke .toScript() which returns the raw text.
# select all the nodes
curSel = nuke.selectedNodes()
# nuke.thisNode() returns the script's context
# i.e. the node from which the script was invoked
knobToChange = nuke.thisNode()['knobname'].toScript()
codeIn = nuke.thisNode()['codeinput'].toScript()
for x in curSel:
x.knob(knobToChange).setValue(codeIn)
You can sidestep this problem as you've noted by building up a string, adding in square brackets etc etc as per your original code, but yes, it's a pain, a maintenance nightmare, and starting to go down that route of building objects up from strings (which #mgilson explains how to do in both a globals() or eval() method)
For those who haven't had the joy of working with Nuke, here's a small screencap that may (or may not..) provide more context:
Is it common in Python to keep testing for type values when working in a OOP fashion?
class Foo():
def __init__(self,barObject):
self.bar = setBarObject(barObject)
def setBarObject(barObject);
if (isInstance(barObject,Bar):
self.bar = barObject
else:
# throw exception, log, etc.
class Bar():
pass
Or I can use a more loose approach, like:
class Foo():
def __init__(self,barObject):
self.bar = barObject
class Bar():
pass
Nope, in fact it's overwhelmingly common not to test for type values, as in your second approach. The idea is that a client of your code (i.e. some other programmer who uses your class) should be able to pass any kind of object that has all the appropriate methods or properties. If it doesn't happen to be an instance of some particular class, that's fine; your code never needs to know the difference. This is called duck typing, because of the adage "If it quacks like a duck and flies like a duck, it might as well be a duck" (well, that's not the actual adage but I got the gist of it I think)
One place you'll see this a lot is in the standard library, with any functions that handle file input or output. Instead of requiring an actual file object, they'll take anything that implements the read() or readline() method (depending on the function), or write() for writing. In fact you'll often see this in the documentation, e.g. with tokenize.generate_tokens, which I just happened to be looking at earlier today:
The generate_tokens() generator requires one argument, readline, which must be a callable object which provides the same interface as the readline() method of built-in file objects (see section File Objects). Each call to the function should return one line of input as a string.
This allows you to use a StringIO object (like an in-memory file), or something wackier like a dialog box, in place of a real file.
In your own code, just access whatever properties of an object you need, and if it's the wrong kind of object, one of the properties you need won't be there and it'll throw an exception.
I think that it's good practice to check input for type. It's reasonable to assume that if you asked a user to give one data type they might give you another, so you should code to defend against this.
However, it seems like a waste of time (both writing and running the program) to check the type of input that the program generates independent of input. As in a strongly-typed language, checking type isn't important to defend against programmer error.
So basically, check input but nothing else so that code can run smoothly and users don't have to wonder why they got an exception rather than a result.
If your alternative to the type check is an else containing exception handling, then you should really consider duck typing one tier up, supporting as many objects with the methods you require from the input, and working inside a try.
You can then except (and except as specifically as possible) that.
The final result wouldn't be unlike what you have there, but a lot more versatile and Pythonic.
Everything else that needed to be said about the actual question, whether it's common/good practice or not, I think has been answered excellently by David's.
I agree with some of the above answers, in that I generally never check for type from one function to another.
However, as someone else mentioned, anything accepted from a user should be checked, and for things like this I use regular expressions. The nice thing about using regular expressions to validate user input is that not only can you verify that the data is in the correct format, but you can parse the input into a more convenient form, like a string into a dictionary.
I'm coding a poker hand evaluator as my first programming project. I've made it through three classes, each of which accomplishes its narrowly-defined task very well:
HandRange = a string-like object (e.g. "AA"). getHands() returns a list of tuples for each specific hand within the string:
[(Ad,Ac),(Ad,Ah),(Ad,As),(Ac,Ah),(Ac,As),(Ah,As)]
Translation = a dictionary that maps the return list from getHands to values that are useful for a given evaluator (yes, this can probably be refactored into another class).
{'As':52, 'Ad':51, ...}
Evaluator = takes a list from HandRange (as translated by Translator), enumerates all possible hand matchups and provides win % for each.
My question: what should my "domain" class for using all these classes look like, given that I may want to connect to it via either a shell UI or a GUI? Right now, it looks like an assembly line process:
user_input = HandRange()
x = Translation.translateList(user_input)
y = Evaluator.getEquities(x)
This smells funny in that it feels like it's procedural when I ought to be using OO.
In a more general way: if I've spent so much time ensuring that my classes are well defined, narrowly focused, orthogonal, whatever ... how do I actually manage work flow in my program when I need to use all of them in a row?
Thanks,
Mike
Don't make a fetish of object orientation -- Python supports multiple paradigms, after all! Think of your user-defined types, AKA classes, as building blocks that gradually give you a "language" that's closer to your domain rather than to general purpose language / library primitives.
At some point you'll want to code "verbs" (actions) that use your building blocks to perform something (under command from whatever interface you'll supply -- command line, RPC, web, GUI, ...) -- and those may be module-level functions as well as methods within some encompassing class. You'll surely want a class if you need multiple instances, and most likely also if the actions involve updating "state" (instance variables of a class being much nicer than globals) or if inheritance and/or polomorphism come into play; but, there is no a priori reason to prefer classes to functions otherwise.
If you find yourself writing static methods, yearning for a singleton (or Borg) design pattern, writing a class with no state (just methods) -- these are all "code smells" that should prompt you to check whether you really need a class for that subset of your code, or rather whether you may be overcomplicating things and should use a module with functions for that part of your code. (Sometimes after due consideration you'll unearth some different reason for preferring a class, and that's allright too, but the point is, don't just pick a class over a module w/functions "by reflex", without critically thinking about it!).
You could create a Poker class that ties these all together and intialize all of that stuff in the __init__() method:
class Poker(object):
def __init__(self, user_input=HandRange()):
self.user_input = user_input
self.translation = Translation.translateList(user_input)
self.evaluator = Evaluator.getEquities(x)
# and so on...
p = Poker()
# etc, etc...