I'm just finding out now that when importing a module, it seems to run through ALL the code, instead of just the one function that I want it to go through. I've been trying to find a way around this, but can't seem to get it. Here is what is happening.
#mainfile.py
from elsewhere import something_else
number = 0
def main():
print('What do you want to do? 1 - something else')
donow = input()
if donow == '1':
something_else()
while 1:
main()
#elsewhere.py
print('I dont know why this prints')
def something_else():
from mainfile import number
print('the variable number is',number)
Now, although this code KIND OF works the way I want it to, the first time when I initiate it, it will go to the main menu twice. For example: I start the program, press one, then it asks me what I want to do again. If I press one again, then it will print "the variable number is 0".
Once I get this working, I would like to be importing a lot of variables back and forth. The only issue is,if I add more import statements to "elsewhere.py" I think it will just initiate the program more and more. If I put "from mainfile import number" on line 1 of "elsewhere.py", I think this raises an error. Are there any workarounds to this? Can I make a different file? What if I made a class to store variables, if that is possible? I'm very new to programming, I would appreciate it if answers are easy to read for beginners. Thank you for the help.
As Jan notes, that's what import does. When you run import, it runs all of the code in the module. You might think: no it doesn't! What about the code inside something_else? That doesn't get run! Right, when the def statement is executed it creates a new function, but it doesn't run it. Basically, it saves the code for later.
The solution is that pretty much all interesting code should be in a function. There are a few cases which make sense to put at the top-level, but if in doubt, put it inside a function. In your particular case, you shouldn't be printing at the top level, if you need to print for some reason, put that into a function and call it when you need it. If you care when something happens, put it in a function.
On a second node, don't import your primary script in other scripts. I.e. if your mainfile.py directly, don't import that in other files. You can but it produces confusing results, and its really best to pretend that it doesn't work.
Don't try to import variables back and forth. Down that path lies only misery. You should only be importing things that don't change. Functions, classes, etc. In any other case, you'll have hard time making it do what you want.
If you want to move variables between places, you have other options:
Pass function arguments
Return values from a function
Use classes
I'll leave it is an exercise to the reader to learn how to do those things.
import executes imported code
import simply takes the Python source file and executes it. This is why it prints, because that instruction is in the code and with import all the instructions get exectued.
To prevent execution of part of imported package/module, you shall use the famous:
if __name__ == "__main__":
print("I do not print with `import`")
Note, that this behaviour is not new in Python 3, it works the same way in Python 2.x too.
Related
This question already has answers here:
Sharing scope in Python between called and calling functions
(5 answers)
Closed 1 year ago.
e.g. I have 2 py script, namely main.py and stringhelper.py.
Since print(f'{student_{i}_{j}}') cannot be read in IDE if variable student_1_2 exist and if i=1,j=2,
I want to make a function that can print 'x{a{h}b{i}c{j}z}y' by changing the string to
exec('''temp_s = 'a{}b{}c{}z'.format(h,i,j)''') # where I suppose e.g. h can refer to main.h but cannot
exec('''print('x{}y'.format(f'{temp_s}')'''
In
stringhelper.py
import re
def execprintf(s):
if s.find('{')<0:
print(s)
exec('''print(s)''')
else:
# use re to do sth to recongize {...{...{ case
if conditionA: # {...{...{ case
print('warning for input s')
elif conditionB: # ... {...{ case
# not finished
# not writing as I meet some problem for simple '{h}' case below
pass
else:
ss = re.split('{|}',s)
ss_even = ss[0::2]
ss_odd = ss[1::2]
s_wo_args = '{}'.join(ss_even)
s_args = ', '.join(ss_odd)
exec('''print(s_wo_args.format('''+s_args+'''))''')
In
main.py
from stringhelper.py import execprintf
h = 1
execprintf('{h}') # NameError: name 'h' is not defined
If I copy the execprintf function into main.py and does not import from stringhelper.py, it can read h and display 1.
Is it possible to read the h in stringhelper.py?
I need at least exceprintf('a{h}b{i}c{j}d') can run correctly first.
You're making a mistake that's common with beginning programmers, confusing data and code. You write your code before anyone (including yourself) runs a program. The code is not (typically) visible to the application when it's running and shouldn't need to be. When your program is running, a user (yourself or whoever) can enter data and this may change the behaviour of the program, but it never affects the code, nor should the user need to know anything about the code you wrote.
If you need to select a specific value based on some input (file, keyboard, whatever), there are other ways of doing that than picking a variable you know to hold that value by name.
In most cases, using a dictionary is the way to go in Python:
values = {
'a': 'some value',
'b': 'some other value'
}
choice = input('Pick a or b (and hit Enter):')
print('You selected:', values[choice])
Here, the initial values are in the code, but of course you could even load values from a file and the program would still work, without the user ever needing to know about or interact with your code.
Also, running code through exec() is rarely the correct solution. This involves your program running code entered by a user, which could be literally anything if you're not careful, and thus presents major security risks, as well as many opportunities for your code to fail.
The main reason beginning programmers feel the need to reach over and grab exec() or eval() is because they confuse what should remain in the code with what the user has access to - once you fix that, you'll find you rarely need to think about exec() and eval().
I completely understand that I should have written the script right the first time, but the fact is I have a script that generates a data file based upon two values passed to it from the command line- like this:
[sinux1~]: ./sim_gen.py 100 .3
I need to call this script from within another script, iterating over a range of values. I searched around and after navigating through all of the "you shouldn't," I tried :
exec(open("./sim_gen.py 100 .3").read())
And this doesn't seem to work.
Help?
Let's break this down into pieces:
exec(open("./sim_gen.py 100 .3").read())
This is equivalent to:
f = open("./sim_gen.py 100 .3")
contents = f.read()
exec(contents)
That open is the same open you use for, say, reading a text file or a CSV. You're asking for a file named "sim_gen.py 100 .3" in the current directory. Do you have one? Of course not. So the open fails.
The best solution is, as you already know, to rewrite sim_gen.py so that you can import it and call a function and pass the arguments to it.
Failing that, the cleanest answer is probably to just run the Python script as a subprocess:
import subprocess
import sys
subprocess.run([sys.executable, "./sim_gen.py", "100", ".3"])
Notice that this is effectively the same thing you're doing when you run the script from your shell, so if it was OK there, it's almost surely OK here.
If you really need to exec for some reason, you will need to do something really hacky, and temporarily change argv for that script's code:
import sys
_argv = sys.argv
try:
sys.argv = ["./sim_gen.py", "100", ".3"]
with open("./sim_gen.py 100 .3"):
exec(f.read())
finally:
sys.argv = _argv
Although really, unless the point of running this is to silently modify your own module's globals or the like, you (a) almost certainly don't really need exec, and (b) want to pass an explicit globals argument even if you do really need it.
This may be a dumb question, but I want to add a line at the very start of the code like
print 'previous runtime' time.time()-tic
Is there a way to do it? Or can I somehow get the previous runtime other than keeping a logfile?
I have to disagree with the accepted answer. You can edit Python code during interpretation for future runtime use. The file itself isn't locked during execution and can be edited by itself. Try this simple example named test.py:
def main():
with open('test.py','a') as me:
me.write(' print "test"\n')
if __name__== '__main__':
main()
Every time you run this, it will add an additional "print test" at the end of the file and next runtime will execute the modified code.
Your example is more complex, but I see no reason why you couldn't insert a variable with the last runtime and call it for a print.
me.write('last_runtime = {0}'.format(time.time())
I will say that while this is possible it's probably(read 'almost certainly') not the best approach. It would be much easier to just have a last_run.pickle and pickle dump a datetime object or whatever to save the last runtime (overwriting the previous last runtime). Then have your code reference this reference file.
While you CAN edit your code from your code, the only reason I can imagine for doing it this way is if you 100% need everything for your program in one file (which seems an unlikely requirement). Otherwise best to save your data somewhere and reference it.
No. It wouldn't be possible unless written somewhere. Simple reason is that once the python process ends, GC cleans up everything.
I have one script that calls another - let's call them master.py and fetch.py. I suppose the second script could be integrated into the first, but it does have distinct functionality - so keeping them separate seems like a good way to force myself to learn how to call outside scripts.
Here's the basic structure of fetch.py:
<import block>
infiles = <paths>
arcpy.env.workspace = os.path.dirname(infile)
ws = arcpy.env.workspace
newfile_list = []
def main():
name = <name>
if not arcpy.Exists(name + ".gdb"):
global ws
new_gdb = DM.CreateFileGDB(ws, name + ".gdb")
newfile_list.append(new_gdb)
other_func1()
other_func2()
print "\nNew files from fetch.py:"
for i in newfile_list:
print " " + i
def other_func1():
stuff
def other_func2():
stuff
if __name__ == '__main__':
main()
And master.py:
<import block>
infiles = <paths>
def f1():
stuff
def f2():
stuff
import fetch
fetch.main()
f1()
f2()
The problems, concerning placement of the import block & file definitions of fetch.py:
When I put them inside main() and run it as a standalone script, my arcpy functions don't work because my various imports haven't run yet. Putting them ahead of main() and making them global, solves this problem.
But when I put them outside of main() as you see here, I get an error saying the local variable ws is referenced before assignment. I think this may have to do with my calling fetch.main(), where the initial lines don't get read. (I was able to make it work by declaring global ws, but don't know if this is advisable.)
How do I structure fetch.py so that the import statements and file definitions get read, both when run as a standalone script and when called?
Instead of rewriting your code for you, I will try to point you to some ideas from the developer philosophy toolkit to get you on the way how to refine your approach.
My feeling is that you should contemplate YAGNI and KISS when thinking about your code.
In your comment you write:
I'm keeping the 'fetch' script separate because finding feature orientation seems like a useful stand-alone tool down the road.
Like I say: YAGNI and KISS: If you need it as a standalone tool in the future then split it out in the future. For now keep it simple and put the code that belongs together in one module and make it callable exactly the way you need it. When you work with it and understand the problem better and see what else is needed, you can then add other ways of calling your script. There is no reason why you shouldn't keep all the code in one module while it is still manageable and just add different ways to call it.
If the time comes that you want to organize your code into several modules you can refactor it in a way that you pull out the things you do on the module level at the moment (either into functions or maybe even classes). Then you can control exactly what should happen when and you can control the order of things better.
As further reading in regards to structuring a Python project I would recommend the pypa sample project and Starting A Python Project The Right Way. The problems you have with imports will likely cease to exist if you structure your code and your project in a sensible, pythonic way.
I love being able to modify the arguments the get sent to a function, using settrace, like :
import sys
def trace_func(frame,event,arg):
value = frame.f_locals["a"]
if value % 2 == 0:
value += 1
frame.f_locals["a"] = value
def f(a):
print a
if __name__ == "__main__":
sys.settrace(trace_func)
for i in range(0,5):
f(i)
And this will print:
1
1
3
3
5
What other cool stuff can you do using settrace?
I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:
Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.
It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.
It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.
It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.
It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.
It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.
Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:
# Method decorator example
from peak.util.decorators import decorate
class Demo1(object):
decorate(classmethod) # equivalent to #classmethod
def example(cls):
print "hello from", cls
A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.
Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.
</rant>
I made a module called pycallgraph which generates call graphs using sys.settrace().
Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.
So for example, consider this function:
def foo(x):
if x:
y = 10
return y
if you test it with this call:
assert foo(1) == 10
then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.
Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.
For example, get the memory consumption of Python code line-by-line: http://pypi.python.org/pypi/memory_profiler
One latest project that uses settrace heavily is PySnooper
It helps new programmers to trace/log/monitor their program output. Cheers!
I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.
The python debugger Pdb uses sys.settrace to analyse lines to debug.
Here's an c optimization/extension for pdb that also uses sys.settrace
https://bitbucket.org/jagguli/cpdb