I have one script that calls another - let's call them master.py and fetch.py. I suppose the second script could be integrated into the first, but it does have distinct functionality - so keeping them separate seems like a good way to force myself to learn how to call outside scripts.
Here's the basic structure of fetch.py:
<import block>
infiles = <paths>
arcpy.env.workspace = os.path.dirname(infile)
ws = arcpy.env.workspace
newfile_list = []
def main():
name = <name>
if not arcpy.Exists(name + ".gdb"):
global ws
new_gdb = DM.CreateFileGDB(ws, name + ".gdb")
newfile_list.append(new_gdb)
other_func1()
other_func2()
print "\nNew files from fetch.py:"
for i in newfile_list:
print " " + i
def other_func1():
stuff
def other_func2():
stuff
if __name__ == '__main__':
main()
And master.py:
<import block>
infiles = <paths>
def f1():
stuff
def f2():
stuff
import fetch
fetch.main()
f1()
f2()
The problems, concerning placement of the import block & file definitions of fetch.py:
When I put them inside main() and run it as a standalone script, my arcpy functions don't work because my various imports haven't run yet. Putting them ahead of main() and making them global, solves this problem.
But when I put them outside of main() as you see here, I get an error saying the local variable ws is referenced before assignment. I think this may have to do with my calling fetch.main(), where the initial lines don't get read. (I was able to make it work by declaring global ws, but don't know if this is advisable.)
How do I structure fetch.py so that the import statements and file definitions get read, both when run as a standalone script and when called?
Instead of rewriting your code for you, I will try to point you to some ideas from the developer philosophy toolkit to get you on the way how to refine your approach.
My feeling is that you should contemplate YAGNI and KISS when thinking about your code.
In your comment you write:
I'm keeping the 'fetch' script separate because finding feature orientation seems like a useful stand-alone tool down the road.
Like I say: YAGNI and KISS: If you need it as a standalone tool in the future then split it out in the future. For now keep it simple and put the code that belongs together in one module and make it callable exactly the way you need it. When you work with it and understand the problem better and see what else is needed, you can then add other ways of calling your script. There is no reason why you shouldn't keep all the code in one module while it is still manageable and just add different ways to call it.
If the time comes that you want to organize your code into several modules you can refactor it in a way that you pull out the things you do on the module level at the moment (either into functions or maybe even classes). Then you can control exactly what should happen when and you can control the order of things better.
As further reading in regards to structuring a Python project I would recommend the pypa sample project and Starting A Python Project The Right Way. The problems you have with imports will likely cease to exist if you structure your code and your project in a sensible, pythonic way.
Related
I have a tool I wrote in python that works completely fine when running in the maya script editor. However, I want to be able to import the script from the script directory. Which should be simple, and I am shocked I can't find the solution while searching the web.
My script format is like this example:
import maya.cmds as cmds
# GUI code with buttons, they call the functions below.
#
#
def function1():
#commands that do things
def function2():
#commands that do things
#List of functions continues
Like I said, the program functions perfectly when run in the script editor. When saving the script to the directory and using this method:
import module
reload (module)
module.function()
The GUI loads fine, but then when pushing the gui buttons, it says the functions are not defined. I don't understand what I am missing? If the script was loaded, shouldn't the functions be defined? Any help would be greatly appreciated, thank you!
Just because the GUI loads doesn't mean that all of your functions loaded properly. You need to put your file, (module.py) in a directory that is visible to your PYTHONPATH. If you're in Maya, you can also put it in the MAYA_SCRIPT_PATH
The PYTHONPATH / MAYA_SCRIPT_PATH are environment variables that you set before launching Maya. In a default Maya installation, some places where you could put your module.py file would be:
(Windows) C:\Users\YOUR_USER_NAME\Documents\maya\scripts
(Linux) ~/maya/scripts
(Mac) - Not sure, put probably also ~/maya/scripts
If you want to know where else you can place it, you run this
import os
print(os.getenv('MAYA_SCRIPT_PATH', ''))
print(os.getenv('PYTHONPATH', ''))
Any location in that list that you have write permissions to is OK to add your module.py file.
Also, it's worth noting that in your example module.function() would fail. It'd need to be module.function1() or module.function2() but I assume you know that. Hope this helps
Sup guys
So i know this is an old question that has already been answered but I have some extra information that helps in regards to GUI functions not working. (same error)
and there's basically nothing on this anywhere.
so the script director only helps when loading in the module through the shelf but will still return the same error "fuction is not defined"
this has todo with how the function is called through the UI element.
example.
this is will allow you to call function in the script editor
but gives you the function not defined error when the module is imported
def GUI_function():
pm.button( command = "function()")
def function():
do stuff
this on the other hand works.
def GUI_function():
pm.button( command = function)
def function(*_):
do stuff
i don't know why but maya tends to think function() is a nodetype
so remove the brackets if you not using arguments and you good to go
Most of my Python scripts (mostly written for web-scraping/data science apps) follow this kind of format:
# import whatever packages
import x, y, z
# do some web-scraping and data manipulation
# write some niche function I need
# make some plots and basically end the script
This is all done via an interactive editor/console (like Eclipse). I basically write the code above, and then copy-paste the code below for testing.
Is there a more "standard" way to go about this? I know C has functions defined above the main function, and I see packages in Python with if __name__ == "__main__" conventions; is this the "appropriate" way to go about development? I suppose the core question is whether you want to be able to use the functions you write in other projects, as well.
The main reason to use if __name__ == "__main__" is so that code under it doesn't run if the file is imported from another file. When a file is imported in python, it simply run through the interpreter as if it had been pasted in at that point, but usually you just want the functions defined in the file, and not to run any code in the file.
For example, if you want to test the functions in your file, or use them in another file, you'd put the code that actually runs the functions under the if __name__ == "__main__".
Since you're writing code for your own consumption, it's totally up to you whether you want to put your code under if __name__ == "__main__", but it's recommended if you want to write your code in a reusable way.
There is no official, universally followed standard in Python with regards to how to organize your code. In relation to your specific question, for readability, and to follow common style patterns, I would recommend you put your functions at the top, and group your web scraping, data manipulation, and plots at the bottom.
I've seen Python packages with if __name__ == "__main__" written in different places within the file before, and a main() defined elsewhere in the same file. I believe, however, that it should go at the bottom of the file you intend to have as your main.
For example, it could look something like this:
import everything
define class1
define function1
define class2
define function2
define function3
define main()
call function3
call function4
do_things
define function4
define function5
if __name__ == "__main__":
run_main()
The sample above is a very generic version of something I've actually got running at work. When executed, it makes calls to a few other files, and works beautifully. It even has a check in the if __name__ == "__main__": block to check for a minimum version of Python.
Hope this helps.
I have a script, that uses a config file called config.py. Actually this is rather a configuration module then. Anyways: the configuration-module contains a lot of parameters and dictionaries and lists of dictionaries and so on.
In the script today it is used like this
import config
def main():
myParameter = config.myParameter
Now I have another application scenario for this script that uses a related config ('config_advanced.py', but the parameters and dictionaries have different values.
My goal is now, to chose the name of the used config-modul as a passed command-line argument:
myScript.py -configuration config_advanced.py
Since the configuration-module is in the same folder than the main script, I guess I have to rename the passed configuration file to 'config.py' first. Afterwards I can perform import config. Otherwise, if I used `import config_advanced, I wouldn't be able to use a call like
config.myParameter
in the main script.
Another possibility could be, to put the configuration-modules in subfolders and keep the name config.py. The passed command-line-argument will have to contain the subfolder then.
Either way I won't be able to perform the import at the top of the main file, since I have to do the argument parsing first. This isn't a technically problem, but someone said that this it at least bad pratice.
What do you think?
What is a better way to do the trick with not much effort?
Thanks a lot
Edit:
One working solution has been
import sys fullpath = "d:\\python\\scripts\\projectA\\configurationFiles\\"
sys.path.append(fullpath)
config = __import__('config_advanced')
Without syspath it does NOT work, so those following tries won't work:
config = __import__('d:\\python\\scripts\\projectA\\configurationFiles\\config_advanced')
config = __import__('d:\\python\\scripts\\projectA\\configurationFiles\\config_advanced.py')
Another possibility that's similar to what you suggest in the question, but which doesn't need you to hide things in subfolders, is to put config_advanced.py and config_plain.py in the same folder as the main script and then dynamically make config.py a link to the actual config file you want to use.
However, martineau's suggestion is much simpler.
OTOH, georg brings up a very valid point, especially if this script isn't just for your own personal use. While using Python itself for the config data is flexible and powerful, it's perhaps a little too powerful. Config data should just be data, not live executable code. If you make a minor mistake when modifying config data you could cause havoc if it's in an executable file. And if a malicious user gets to it, there's no limit to the damage they could cause.
Bad data in a plain old data file will at worst cause a ValueError if it does something weird that your config parsing code isn't suspecting. But bad data in a live Python file could throw all sorts of nasty errors. Or even worse, it could do something evil in complete silence...
In reply to your comments, here's some code to illustrate the first point:
#! /usr/bin/env python
import os
config_file = "config.py"
def link_config(mode):
if os.path.exists(config_file):
os.remove(config_file)
config_name = "config_%s.py" % mode
os.symlink(config_name, config_file)
#.... parse command line to determine config_mode string, then do
link_config(config_mode)
#Now import the newly-linked config file
import config
If config_mode == "plain" the above code will cause config_plain.py to be imported as 'config'
and if config_mode == "advanced" it will cause config_advanced.py to be imported as 'config'
But as I said before, martineau's method is much simpler. And IIRC, os.symlink may not work on non-unix systems.
...
As for your second point, check out the docs for the json module
I'm just finding out now that when importing a module, it seems to run through ALL the code, instead of just the one function that I want it to go through. I've been trying to find a way around this, but can't seem to get it. Here is what is happening.
#mainfile.py
from elsewhere import something_else
number = 0
def main():
print('What do you want to do? 1 - something else')
donow = input()
if donow == '1':
something_else()
while 1:
main()
#elsewhere.py
print('I dont know why this prints')
def something_else():
from mainfile import number
print('the variable number is',number)
Now, although this code KIND OF works the way I want it to, the first time when I initiate it, it will go to the main menu twice. For example: I start the program, press one, then it asks me what I want to do again. If I press one again, then it will print "the variable number is 0".
Once I get this working, I would like to be importing a lot of variables back and forth. The only issue is,if I add more import statements to "elsewhere.py" I think it will just initiate the program more and more. If I put "from mainfile import number" on line 1 of "elsewhere.py", I think this raises an error. Are there any workarounds to this? Can I make a different file? What if I made a class to store variables, if that is possible? I'm very new to programming, I would appreciate it if answers are easy to read for beginners. Thank you for the help.
As Jan notes, that's what import does. When you run import, it runs all of the code in the module. You might think: no it doesn't! What about the code inside something_else? That doesn't get run! Right, when the def statement is executed it creates a new function, but it doesn't run it. Basically, it saves the code for later.
The solution is that pretty much all interesting code should be in a function. There are a few cases which make sense to put at the top-level, but if in doubt, put it inside a function. In your particular case, you shouldn't be printing at the top level, if you need to print for some reason, put that into a function and call it when you need it. If you care when something happens, put it in a function.
On a second node, don't import your primary script in other scripts. I.e. if your mainfile.py directly, don't import that in other files. You can but it produces confusing results, and its really best to pretend that it doesn't work.
Don't try to import variables back and forth. Down that path lies only misery. You should only be importing things that don't change. Functions, classes, etc. In any other case, you'll have hard time making it do what you want.
If you want to move variables between places, you have other options:
Pass function arguments
Return values from a function
Use classes
I'll leave it is an exercise to the reader to learn how to do those things.
import executes imported code
import simply takes the Python source file and executes it. This is why it prints, because that instruction is in the code and with import all the instructions get exectued.
To prevent execution of part of imported package/module, you shall use the famous:
if __name__ == "__main__":
print("I do not print with `import`")
Note, that this behaviour is not new in Python 3, it works the same way in Python 2.x too.
I love being able to modify the arguments the get sent to a function, using settrace, like :
import sys
def trace_func(frame,event,arg):
value = frame.f_locals["a"]
if value % 2 == 0:
value += 1
frame.f_locals["a"] = value
def f(a):
print a
if __name__ == "__main__":
sys.settrace(trace_func)
for i in range(0,5):
f(i)
And this will print:
1
1
3
3
5
What other cool stuff can you do using settrace?
I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:
Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.
It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.
It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.
It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.
It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.
It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.
Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:
# Method decorator example
from peak.util.decorators import decorate
class Demo1(object):
decorate(classmethod) # equivalent to #classmethod
def example(cls):
print "hello from", cls
A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.
Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.
</rant>
I made a module called pycallgraph which generates call graphs using sys.settrace().
Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.
So for example, consider this function:
def foo(x):
if x:
y = 10
return y
if you test it with this call:
assert foo(1) == 10
then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.
Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.
For example, get the memory consumption of Python code line-by-line: http://pypi.python.org/pypi/memory_profiler
One latest project that uses settrace heavily is PySnooper
It helps new programmers to trace/log/monitor their program output. Cheers!
I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.
The python debugger Pdb uses sys.settrace to analyse lines to debug.
Here's an c optimization/extension for pdb that also uses sys.settrace
https://bitbucket.org/jagguli/cpdb