I'm trying to do some script reuse for some python build scripts. An abbreviated version of my "reusable" part looks like (_build.py):
Sources = []
Sources += glob('*.c')
Sources += glob('../FreeRTOS/*.c')
...
def addSources(directory, *rest):
for each in rest:
Sources += ['../'+directory+'/'+each+'.c']
def addSharedSources(*args):
addSources('Shared', *args)
Then in the customized part, I have something like (build.py):
#!/usr/bin/env python
from _build import *
...
#Additional source files from ../Shared, FreeRTOS and *.c are already in
addSharedSources('ccpwrite', 'discovery', 'radioToo', 'telemetry', 'utility')
Unfortunately, when I try to run build.py, I get a traceback that looks like:
Traceback (most recent call last):
File "./build.py", line 8, in <module>
addSharedSources('ccpwrite', 'discovery', 'radioToo', 'telemetry', 'utility')
File "/Users/travisg/Projects/treetoo/Twig/_build.py", line 49, in addSharedSources
addSources('Shared', *args)
File "/Users/travisg/Projects/treetoo/Twig/_build.py", line 46, in addSources
Sources += ['../'+directory+'/'+each+'.c']
UnboundLocalError: local variable 'Sources' referenced before assignment
So even though I did the wildcard import, it would appear that when the import function is called, it's not referencing my "global" variable imported from the original. Is there a way to make it work? I toyed around with global, but that didn't seem to do what I wanted.
This has nothing to do with the import. You'll have the same problem if you run _build.py directly. The problem is that the function addSources is modifying the global Sources without declaring it global. Insert a global declaration in the addSources function, and all should be well.
Explanation: It is very easy to write this kind of code by mistake. So python allows you to read a global variable without declaring it global, but not to modify it.
Related
I would like to ask:
Is there (in Python) any way, how to absolutelly stupidly include other file into source code? Without any dancings like modules etc,
i.e. equivalent #include from C ?
There were laid many questions about this, but all of them are turning on import modules. No, I want only insert some text lines from any file to my code without preprocess it.
For example:
Let's file mysum.py
c = a + b # there is just this one line
And I want to include to another source code:
file experiment.py:
#!/usr/bin/python
# -*- coding: UTF8 -*-
import importlib
import sys
import math
a = 1
b = 2
import **mysum**
print c
Well, it does not work,
Traceback (most recent call last):
File "./experiment.py", line 12, in <module>
import mysum
File "/home/rna/unix/mypok/mysum.py", line 5, in <module>
c = a + b
NameError: name 'a' is not defined
I really need one short header file include to more different scripts, there would be definitions variables about MySQL connection and similar data, common for more scripts.
execfile used to do this in Python 2, but in Python 3 it's been replaced with:
exec(open('myfile.py').read())
This literally just reads that file (at runtime) and exec's the result. Note that this is not quite the same as what C++ does, which is that the preprocessor does all of this prior to the file being run (or compiled) at all, but it's probably the closest thing there is to it.
If you want to make this some kind of reusable function, you can do something like
import sys
def include(filename):
with open(filename, 'r') as f:
exec(f.read(), sys._getframe(1).f_globals, sys._getframe(1).f_locals)
include("myfile.py") # literally reads the file and evaluates it at runtime
The call to exec(f.read(), sys._getframe(1).f_locals, sys._getframe(1).f_locals) means that the exec call is run in the parent function's scope. See also the answers at What is an alternative to execfile in Python 3?
When you import a module, anything in global scope of the imported module is executed, this includes statements, variable definitions, class definitions, etc.
Example, if I were to modify your mysum.py
a = 1
b = 4
def some_func(a,b):
return a+b
Then in another script run
import mysum
All of those statements would be executed. I would have access to mysum.a, mysum.b, and mysum.some_func because they were executed on import. Your c = a+b acts exactly the same way
Let's now examine the script as you have it:
c = a+b
It is important that you just look at this as a standalone script, even though you are importing it. There is no a or b defined in the entire scope of that script, so when you import, c=a+b gets evaluated, with none of those variables being defined.
I'm not sure the exact use case of what you are trying to do, but if you wanted some function to add two variables, I would do it like so:
#mysum.py
def some_add(a,b):
return a+b
#other_module.py
import mysum
a = 2
b = 3
c = mysum.some_add(a,b)
You'll notice I didn't use a global-style call of a and b like
def some_add():
return a+b
Because that again would require a and b be in the global scope of that module, and you'll again get NameErrors
I have two scripts, main and statistics. In one script, main, several functions from the other script, statistics, are called. The code is as follows.
if initial_action == "get_portfolio_statistics":
statistics.get_portfolio_status()
statistics.get_current_securities()
statistics.get_open_trades()
Of course, the functions match as far as calling names go, and my IDE (PyCharm) is not giving any warnings. Tests have, refactor after refactor, turned up the fundamentally same error report:
Traceback (most recent call last):
File "G:/stuff/dev/!projects+repositories/RYZ/main.py", line 2, in <module>
import statistics
File "G:\stuff\dev\!projects+repositories\RYZ\statistics.py", line 1, in
<module>
import main
File "G:\stuff\dev\!projects+repositories\RYZ\main.py", line 12, in <module>
statistics.get_portfolio_status()
AttributeError: module 'statistics' has no attribute 'get_portfolio_status'
There are, however, a few intriguing recurrences that have popped up. First of all, in some cases, for no identifiable reason, the tests check out when the function names are changed, however, the results are not consistent with further tests. Second of all, the use of back/forward slashes in the file paths are not consistent, with a G:/ with the first call, and then G:\ for the two latest calls, though the first and latest calls are referring to the same file.
My question is whether this error is a matter of function naming, a matter of file path inconsistencies, a matter of cross-imports, or due to the fact that the functions being called are not housed in a class.
Project Structure:
ryz server backend (RYZ directory)
- main.py
- statistics.py
Import Structure:
# in main.py
import statistics
# in statistics.py
import main
statistics.py Structure:
<imports...>
def get_portfolio_status():
<code>
def get_current_securities():
<code>
def get_open_trades():
<code>
I'll bet it's because of the cross-import. Try to move anything that statistics.py require to that file and then import into and call from main.py.
Your function naming is normal.
I wouldn't worry about the mix of slashes. However, if you're constructing a path yourself, use the os module:
import os
path = os.path.join(os.getcwd(), 'dir', 'filename')
This will ensure that you get a path suited to your platform.
I've got a script where I want to import a dict from a file and then use it to execute functions.
The file codes.py is as follows:
rf_433mhz = {
"0x471d5c" : sensor_LaundryDoor,
}
And the file it's using is as follows:
#!/usr/bin/python
import mosquitto
import json
import time
def sensor_LaundryDoor():
print "Laundry Door Opened"
mqttc.publish("actuators", json.dumps(["switch_HallLight", "on"]))
from codes import rf_433mhz
but I'm getting a NameError.
Traceback (most recent call last):
File "sensors.py", line 11, in <module>
from codes import rf_433mhz
File "/root/ha/modules/processing/codes.py", line 2, in <module>
"0x471d5c" : sensor_LaundryDoor,
NameError: name 'sensor_LaundryDoor' is not defined
Is there any way to do what I'm trying to do? It seems to be getting stuck on not having the function inside codes.py
I'm trying to call sensor_LaundryDoor() as follows
def on_message(msg):
inbound = json.loads(msg.payload)
medium = inbound[0]
content = inbound[1]
if str(medium) == "433mhz":
try:
rf_433mhz[str(content)]()
except:
print "Sorry code " + content + " is not setup"
import isn't include. It won't dump the source code of codes.py into your script; rather, it runs codes.py in its own namespace, almost like a separate script, and then assigns either the module object or specific module contents to names in the namespace the import is in. In the namespace of codes.py, there is no sensor_LaundryDoor variable.
The way you're dividing the code into modules isn't very useful. To understand codes.py, you need to understand the other file to know what sensor_LaundryDoor is. To understand the other file, you need to understand codes.py to know what you're importing. This circular dependency would negate most of the benefit of modularizing your code even if it wasn't an error. Reorganize your code to fix the circular dependency, and you'll probably fix the NameError as well.
The problem is that in your dictionary that you're importing, you're setting the value of 0x471d5c to a variable that was either not defined, or not defined in that scope.
An example of this would be like:
Codes.py
#!/usr/bin/python
sensor_LaundryDoor = 'foo'
rf_433mhz = {
"0x471d5c" : sensor_LaundryDoor,
}
Main files
#!/usr/bin/python
from test import rf_433mhz
print rf_433mhz["0x471d5c"]
There are hacky ways to solve this but it looks like the real issue is that you're trying to write C-style code in Python. The Python way to do things would be to import sensor_LaundryDoor in codes.py before using it, and if this introduces a circular reference then that's a design issue.
Maybe you need three modules, events.py with your main loop which imports the dict from codes.py which imports the functions from sensors.py.
I need to be able to run a large amount of python code from a string. Simply using exec doesn't seem to work, as, while the code runs perfectly in a normal setting, doing it this way seems to throw an error. I also don't think I can just import it as it it hosted on the internet. Here is the code:
import urllib.request
URL = "https://dl.dropboxusercontent.com/u/127476718/instructions.txt"
def main():
instructions = urllib.request.urlopen(URL)
exec(instructions.read().decode())
if __name__ == "__main__":
main()
This is the error I've been getting:
Traceback (most recent call last):
File "C:\Python33\rc.py", line 12, in <module>
main()
File "C:\Python33\rc.py", line 9, in main
exec(instructions.read().decode())
File "<string>", line 144, in <module>
File "<string>", line 120, in main
NameError: global name 'Player' is not defined
The code I'm trying to run is available in the link in the first code snippet.
If you have any questions I'll answer them. Thank you.
Without specifying globals, the exec function (Python/bltinmodule.c) uses PyEval_GetGlobals() and PyEval_GetLocals(). For the execution frame of a function, the latter creates a new f_locals dict, which will be the target for the IMPORT_NAME, STORE_NAME, LOAD_NAME ops in the compiled code.
At the module level in Python the normal state of affairs is globals() == locals(). In that case STORE_NAME is using the module's globals, which is what a function defined within the module will use as its global namespace. However, using separate dicts for globals and locals obviously breaks that assumption.
The solution is to to manually supply globals, which exec will also use as locals:
def main():
instructions = urllib.request.urlopen(URL)
exec(instructions.read().decode(), globals())
You could also use a new dict that has __name__ defined:
def main():
instructions = urllib.request.urlopen(URL)
g = {'__name__': '__main__'}
exec(instructions.read().decode(), g)
I see in the source that the current directory will need a sound file named "pickup.wav", else you'll just get another error.
Of course, the comments about the security problems with using exec like this still apply. I'm only addressing the namespace technicality.
First I thought you might try __import__ with a StringIO object. Might look something like StackOverflow: Local Import Statements in Python.
... but that's not right.
Then I thought of using the imp module but that doesn't seen to work either.
Then I looked at: Alex Martelli's answer to Use of Eval in Python --- and tried to use it on a silly piece of code myself.
I can get the ast object, and the results of the compile() from that (though it also seems that one can simply call compile(some_string_containing_python_source, 'SomeName', 'exec') without going through the ast.parse() intermediary step if you like. From what I gather you'd use ast if you wanted to then traverse the resulting syntax tree, inspecting and possibly modifying nodes, before you compiled it.
At the end it seems that you'll need to exec() the results of your compile() before you have resulting functions, classes or variables defined in your execution namespace.
You can use pipe to put all strings into a child process of python and get output result from it.
Google os.popen or subprocess.Popen
I am getting back into python and I'm having a really basic issue....
My source has the following...
def calrounds(rounds):
print rounds
When I run this through the shell and try to call calrounds(3) I get..
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
calrounds(3)
NameError: name 'calrounds' is not defined
Its been awhile since I've used python, humor me.
Did you import your source first?
It says that the first line of your program is calling calrounds with a parameter 3. Move that below your function definition. The definition needs to be before you call the function. If you are using python 3.0+ you need parenthesis for the print statement.
>>> def calrounds(rounds):
print(rounds)
>>> calrounds(3)
3
The first thing to do is to look at how you're calling the function. Assuming it's in myModule.py, did you import myModule or did you from myModule import calrounds? If you used the first one, you need to call it as myModule.calrounds().
Next thing I would do is to make sure that you're restarting your interpreter. If you have imported a module, importing it again will not reload the source, but use what is already in memory.
The next posibility is that you're importing a file other than the one you think you are. You might be in a different directory or loading something from the standard library. After you import myModule you should print myModule.__file__ and see if it is the file you think you're working on. After 20 years of programming, I still find myself doing this about once a year and it's incredibly frustrating.
Finally, there's the chance that Python is just acting up. Next to your myModule.py there will be a myModule.pyc - this is where Python puts the compiled code so it can load modules faster. Normally it's smart enough to tell if your source has been modified but, occassionally, it fails. Delete your .pyc file and restart the interpreter.