A variable AA is in aaa.py. I want to use this variable in my other python file bbb.py
How do I access this variable?
You're looking for modules!
In aaa.py:
AA = 'Foo'
In bbb.py:
import aaa
print aaa.AA # Or print(aaa.AA) for Python 3
# Prints Foo
Or this works as well:
from aaa import AA
print AA
# Prints Foo
You can import it; this will execute the whole script though.
from aaa import AA
In your file bbb.py, add the following:
import sys
sys.path.append("/path/to/aaa.py/folder/")
from aaa import AA
Also would suggest reading more about Python modules and how import works. Official Documentation on Modules.
Although importing is the best approach (like poke or Haidro wrote), here is another workaround, if you're generating data with one script and want to access them in another, without executing "bbb.py". So if you're dealing with large lists/dicts, this approach works well, although it's an overkill if you simply trying to interchange a string or a number…
Besides that, you should describe your problem more detailed, since importing variables from another script is kind of hacky and may not be the way to go.
So here are two functions, the first one (dumpobj) dumps a variable (string, number, tuple, list, dict sets whatever) to a file, the second one (loadobj) reads it in. I decided to use json, since the data files are human readable.
You may want to try that, if you need it often:
import json
def dumpobj(the_object, filename) :
"""
Dumps the_object to filename via json. Use loadobj(filename)
to get it back.
>>> from tgio import dumpobj, loadobj
>>> foo = {'bar':42, 'narf':'fjoord', 23:[1,2,3,4,5]}
>>> dumpobj(foo, 'foo.var')
>>> bar = loadobj('foo.var')
>>> bar
{u'narf': u'fjoord', u'bar': 42, u'23': [1, 2, 3, 4, 5]}
"""
try:
with open(filename):
print(filename + " exists, I'll will overwrite it.")
except IOError:
print(filename + ' does not exist. Creating it...')
f = open(filename, 'w')
json.dump(the_object, f)
f.close()
def loadobj(filename) :
"""
Retrieves dumped data (via json - see dumpobj) from filename.
>>> from tgio import loadobj, dumpobj
>>> foo = {'bar':42, 'narf':'fjoord', 23:[1,2,3,4,5]}
>>> dumpobj(foo, 'foo.var')
>>> bar = loadobj('foo.var')
>>> bar
{u'narf': u'fjoord', u'bar': 42, u'23': [1, 2, 3, 4, 5]}
"""
try:
with open(filename):
print("Reading object from file " + filename)
except IOError:
print(filename + ' does not exist. Returning None.')
return None
f = open(filename, 'r')
the_object = json.load(f)
f.close
return the_object
See the usage examples in the docstrings!
Related
I'm storing f-strings with function calls in a separate file (with lots of variables).
I am writing a script that has hundreds of variables which are then loaded into an HTML table. Some of the contents in the HTML table require function calls.
This works:
def add_one(a):
return a + 1
a = 1
s = f"A is {a} and next comes {add_one(a)}"
print(s)
When I store s in a file, I can use **locals() to format it and it works when I store variables in s.txt.
Contents of s.txt:
A is {a}
Contents of script that works:
a = 1
print(open('s.txt').read().format(**locals()))
However, when I try to call functions, it does not work:
Contents of s.txt:
A is {a} and next comes {add_one(a)}
Contents of script that does not work:
def add_one(a):
return a + 1
a = 1
print(open('s.txt').read().format(**locals()))
What can I do to make it work (given my actual case is hundreds of function calls and not this simple 2 variable example)?
In this example it should result in A is 1 and next comes 2.
You might want to consider using a templating language rather than f-strings if you have a complex HTML table with hundreds of variables. e.g. Jinja2.
For simplicity I've stored the a value in a dictionary as this then simplifies passing it to the Jinja2 render and also converting it to JSON for storing it in a file.
Here is your example using Jinja2 templates and storing the data to a json file:
import json
from pathlib import Path
import jinja2
json_file = Path('/tmp/test_store.json')
jinja_env = jinja2.Environment()
# Set variable values
values = {'a': 3}
# Save to json file
json_file.write_text(json.dumps(values))
# Read from json file to dictionary with new variable name
read_values = json.loads(json_file.read_text())
def add_one(a):
return a + 1
# Add custom filter to jinja environment
jinja_env.filters['add_one'] = add_one
# Define template
template = jinja_env.from_string("A is {{a}} and next comes {{a | add_one}}")
# Print rendered template
print(template.render(read_values))
This gave the output of:
A is 3 and next comes 4
The JSON file is the following:
{"a": 3}
As mentioned in the discussion e.g. here, what you want does not really work in any simple way. There is one obvious workaround: storing an f-string (e.g. f"A is {a} and next comes {add_one(a)}") in your text file and then eval'ing it:
with open('s.txt', 'r') as f:
print(eval(f.read())) # A is 1 and next comes 2
Of course, all the usual warnings about shooting yourself in the foot apply here, but your problem definition sounds exactly like this use case. You can try sandboxing your functions and whatnot, but it generally does not work well. I would say it is still a viable use case for homebrew automation, but it has a massive potential for backfiring, and the only reason I am suggesting it is because alternative solutions are likely to be about as dangerous.
Use serialization and deserialization to store data
import json
data = {
"a" : 1,
"b" : 2,
"name" : "Jack",
"bunch_of_numbers" : [1, 2, 3, 5, 6]
}
file_name = "s.txt"
with open(file_name, 'w') as file:
file.write(json.dumps(data)) #serialization
with open(file_name, 'rb') as file:
data = json.load(file) # de-serialization
print(data)
Output:
{'a': 1, 'b': 2, 'name': 'Jack', 'bunch_of_numbers': [1, 2, 3, 5, 6]}
I have two CSV files, one of which is likely to contain a few more records that the other. I am writing a function to iterate over each and determine which records are in dump but not liar.
My code is as follows:
def update_lib(x, y):
dump = open(x, newline='')
libr = open(y, newline='')
dump_reader = csv.reader(dump)
for dump_row in dump_reader:
libr_reader = csv.reader(libr)
for libr_row in libr_reader:
if dump_row[0] == libr_row[0]:
break
I am expecting this to take the first row in dump (dump_row) and iterate over each row in library (libr_row) to see if the first elements match. If they do then I want to move to the next row in dump and if not I will do something else eventually.
My issue is that libr_reader appears to "remember" where it is and I can't get it to go back to the first row in libr, even when the break has been reached and I would therefore expect libr_reader to be re-initiated. I have even tried del libr_row and del libr_reader but this doesn't appear to make a difference. I suspect I am misunderstanding iterators, any help gratefully received.
As it's pasted in your question, you'll be creating a libr_reader object every time you iterate over a row in dump_reader.
dump_reader = csv.reader(dump)
for dump_row in dump_reader:
libr_reader = csv.reader(libr)
dump_reader here is created once. Assuming there are 10 rows from dump_reader, you will be creating 10 libr_reader instances, all from the same file handle.
Per our discussion in the comments, you're aware of that, but what you're unaware of is that the reader object is working on the same file handle and thus, is still at the same cursor.
Consider this example:
>>> import io
>>> my_file = io.StringIO("""Line 1
... Another Line
... Finally, a third line.""")
This is creating a simulated file object. Now I'll create a "LineReader" class.
>>> class LineReader:
... def __init__(self, file):
... self.file = file
... def show_me_a_line(self):
... print(self.file.readline())
...
If I use three line readers on the same file, the file still remembers its place:
>>> line_reader = LineReader(my_file)
>>> line_reader.show_me_a_line()
Line 1
>>> second_line_reader = LineReader(my_file)
>>> second_line_reader.show_me_a_line()
Another Line
>>> third_line_reader = LineReader(my_file)
>>> third_line_reader.show_me_a_line()
Finally, a third line.
To the my_file object, there's no material difference between what I just did, and doing this directly. First, I'll "reset" the file to the beginning by calling seek(0):
>>> my_file.seek(0)
0
>>> my_file.readline()
'Line 1\n'
>>> my_file.readline()
'Another Line\n'
>>> my_file.readline()
'Finally, a third line.'
There you have it.
So TL/DR: Files have cursors and remember where they are. Think of the file handle as a thing that remembers where the file is, yes, but also remembers where in the file your program is.
I am writing a function that exports variables as a dictionary to an external file.
The problem comes when calling that function from another script. I think it has something to do with the globals() parameter.
import sys
import os
mydict = {} #'initialising" the an empty dictionary to be used locally in the function below
def writeToValues(name):
fileName = os.path.splitext(os.path.basename(sys.argv[0]))[0]
valuePrint=open("values.py","a")
def namestr(obj,namespace):
return[name for name in namespace if namespace[name] is obj]
b = namestr(name, globals())
c = "".join(str(x) for x in b)
mydict[(c)] = name
valuePrint.write(fileName)
valuePrint.write("=")
valuePrint.write(str(mydict))
valuePrint.write("\n")
valuePrint.close()
return mydict
a = 2
b = 3
writeToValues(a)
writeToValues(b)
I get the following result:
Main Junkfile={'a': 2, 'b': 3}
note the word Main Junkfile is the name of the script I ran as that is what the function first does, to get the name of the file and use that to name the dictionary.
Now help me as I cannot generate the same if I import the function from another script.
Another problem is that running the script twice generates the values in steps.
Main Junkfile={'a': 2}
Main Junkfile={'b': 3, 'a': 2}
I cannot change the file open mode from append to write since I want to store values from other scripts, too.
this is not perfect but might help as an example:
import sys
import os
mydict = {}
def namestr(obj,namespace):
return[name for name in namespace if namespace[name] is obj]
def writeto(name):
fout = 'values.py'
filename = os.path.splitext(os.path.basename(sys.argv[0]))[0]
with open (fout, 'a') as f:
b = namestr(name, globals())
c = "".join(str(x) for x in b)
mydict[(c)] = name
data = filename + '=' + str(mydict) + '\n'
f.write(data)
return mydict
a = 2
b = 3
if __name__ == '__main__':
writeto(a)
writeto(b)
First of all, to get the current executing script name, or rather the module that called your function you'll have to pick it up from the stack trace. Same goes for globals() - it will execute in the same context of writeToValues() function so it won't be picking up globals() from the 'caller'. To remedy that you can use the inspect module:
import inspect
import os
def writeToValues(name):
caller = inspect.getmodule(inspect.stack()[1][0])
caller_globals = caller.__dict__ # use this instead of globals()
fileName = os.path.splitext(os.path.basename(caller.__file__))[0]
# etc.
This will ensure that you get the name of the module that imported your script and is calling writeToValues() within it.
Keep in mind that this is a very bad idea if you intend to write usable Python files - if your script name has spaces (like in your example) it will write a variable name with spaces, which will further result in syntax error if you try to load the resulting file into a Python interpreter.
Second, why in the name of all things fluffy are you trying to do a reverse lookup to find a variable name? You are aware that:
a = 2
b = 2
ab = 5
writeToValues(b)
will write {"ab": 2}, and not {"b": 2} making it both incorrect in intent (saves the wrong var) as well as in state representation (saves a wrong value), right? You should pass a variable name you want to store/update instead to ensure you're picking up the right property.
The update part is more problematic - you need to update your file, not just merely append to it. That means that you need to find the line of your current script, remove it and then write a new dict with the same name in its place. If you don't expect your file to grow to huge proportions (i.e. you're comfortable having it partially in the working memory), you could do that with:
import os
import inspect
def writeToValues(name):
caller = inspect.getmodule(inspect.stack()[1][0])
caller_globals = caller.__dict__ # use this instead of globals()
caller_name = os.path.splitext(os.path.basename(caller.__file__))[0]
# keep 'mydict' list in the caller space so multiple callers can use this
target_dict = caller_globals['mydict'] = caller_globals.get('mydict', {})
if name not in caller_globals: # the updated value no longer exists, remove it
target_dict.pop(name, None)
else:
target_dict[name] = caller_globals[name]
# update the 'values.py':
# optionaly check if you should update - if values didn't change no need for slow I/O
with open("values.py", "a+") as f:
last_pos = 0 # keep the last non-update position
while True:
line = f.readline() # we need to use readline() for tell() accuracy
if not line or line.startswith(caller_name): # break at the matching line or EOF
break
last_pos = f.tell() # new non-update position
append_data = f.readlines() # store in memory the rest of the file content, if any
f.seek(last_pos) # rewind to the last non-update position
f.truncate() # truncate the rest of the file
f.write("".join((caller_name, " = ", str(target_dict), "\n"))) # write updated dict
if append_data: # write back the rest of the file, if truncated
f.writelines(append_data)
return target_dict
Otherwise use a temp file to write everything as you read it, except for the line matching your current script, append the new value for the current script, delete the original and rename the temp file to values.py.
So now if you store the above in, say, value_writter.py and use it in your script my_script.py as:
import value_writter
a = 2
b = 3
value_writter.write_to_values("a")
value_writter.write_to_values("b")
a = 5
value_writter.write_to_values("a")
# values.py contains: my_script = {"a": 5, "b": 3}
Same should go for any script you import it to. Now, having multiple scripts edit the same file without a locking mechanism is an accident waiting to happen, but that's a whole other story.
Also, if your values are complex the system will break (or rather printout of your dict will not look properly). Do yourself a favor and use some proper serialization, even the horrible pickle is better than this.
I'm trying to store many variables in a file. I've tried JSON, pickle and shelve but they all seem to only take one variable
import shelve
myShelve = shelve.open('my.shelve')
myShelve.update(aasd,
basd,
casd,
dasd,
easd,
fasd,
gasd,
hasd,
iasd,
jasd)
myShelve.close()
And pickle
import pickle
with open("vars.txt", "wb") as File:
pickle.dumps(aasd,
basd,
casd,
dasd,
easd,
fasd,
gasd,
hasd,
iasd,
jasd,
File)
The errors I'm getting are along the lines of
TypeError: update() takes at most 2 positional arguments (11 given)
and
TypeError: pickle.dumps() takes at most 2 positional argument (11 given)
I'm not sure if there's any other way of storing variables except using a database, but that's a bit over what I'm currently capable of I'd say.
You can only pickle one variable at a time, but it can be a dict or other Python object. You could store your many variables in one object and pickle that object.
import pickle
class Box:
pass
vars = Box()
vars.x = 1
vars.y = 2
vars.z = 3
with open("save_vars.pickle", "wb") as f:
f.write(pickle.dumps(vars))
with open("save_vars.pickle", "rb") as f:
v = pickle.load(f)
assert vars.__dict__ == v.__dict__
using pickle, you dump one object at a time. Each time you dump to the file, you add another "record".
import pickle
with open("vars.txt", "wb") as File:
for item in (aasd,
basd,
casd,
dasd,
easd,
fasd,
gasd,
hasd,
iasd,
jasd)
pickle.dump(item,File)
Now, on when you want to get your data back, you use pickle.load to read the next "record" from the file:
import pickle
with open('vars.txt') as fin:
aasd = pickle.load(fin)
basd = pickle.load(fin)
...
Alternatively, depending on the type of data, assuming the data is stuff that json is able to serialize, you can store it in a json list:
import json
# dump to a string, but you could use json.dump to dump it to a file.
json.dumps([aasd,
basd,
casd,
dasd,
easd,
fasd,
gasd,
hasd,
iasd,
jasd])
EDIT: I just thought of a different way to store your variables, but it is a little weird, and I wonder what the gurus think about this.
You can save a file that has the python code of your variable definitions in it, for example vars.py which consists of simple statements defining your values:
x = 30
y = [1,2,3]
Then to load that into your program, just do from vars import * and you will have x and y defined, as if you had typed them in.
Original normal answer below...
There is a way using JSON to get your variables back without redefining their names, but you do have to create a dictionary of variables first.
import json
vars={} # the dictionary we will save.
LoL = [ range(5), list("ABCDE"), range(5) ]
vars['LOList'] = LoL
vars['x'] = 24
vars['y'] = "abc"
with open('Jfile.txt','w') as myfile:
json.dump(vars,myfile,indent=2)
Now to load them back:
with open('Jfile.txt','r') as infile:
D = json.load(infile)
# The "trick" to get the variables in as x,y,etc:
globals().update(D)
Now x and y are defined from their dictionary entries:
print x,y
24 abc
There is also an alternative using variable-by-variable definitions. In this way, you don't have to create the dictionary up front, but you do have to re-name the variables in proper order when you load them back in.
z=26
w="def"
with open('Jfile2.txt','w') as myfile:
json.dump([z,w],myfile,indent=2)
with open('Jfile2.txt','r') as infile:
zz,ww = json.load(infile)
And the output:
print zz,ww
26 def
I have this class that consists of 3 functions. Each function is in charge of one part of the whole process.
.load() loads up two files, re-formats their content and writes them to two new files.
.compare() takes two files and prints out their differences in a specific format.
.final() takes the result of .compare() and creates a file for every set of values.
Please ignore the Frankenstein nature of the logic as it is not my main concern at the moment. I know it can be written a thousand times better and that's fine by me for now as i am still new to Python and programing in general. I do have some theoretical experience but very limited technical practice and that is something i am working on.
Here is the code:
from collections import defaultdict
from operator import itemgetter
from itertools import groupby
from collections import deque
import os
class avs_auto:
def load(self, fileIn1, fileIn2, fileOut1, fileOut2):
with open(fileIn1+'.txt') as fin1, open(fileIn2+'.txt') as fin2:
frame_rects = defaultdict(list)
for row in (map(str, line.split()) for line in fin1):
id, frame, rect = row[0], row[2], [row[3],row[4],row[5],row[6]]
frame_rects[frame].append(id)
frame_rects[frame].append(rect)
for row in (map(str, line.split()) for line in fin2):
id, frame, rect = row[0], row[2], [row[3],row[4],row[5],row[6]]
frame_rects[frame].append(id)
frame_rects[frame].append(rect)
with open(fileOut1+'.txt', 'w') as fout1, open(fileOut2+'.txt', 'w') as fout2:
for frame, rects in sorted(frame_rects.iteritems()):
fout1.write('{{{}:{}}}\n'.format(frame, rects))
fout2.write('{{{}:{}}}\n'.format(frame, rects))
def compare(self, f1, f2):
with open(f1+'.txt', 'r') as fin1:
with open(f2+'.txt', 'r') as fin2:
lines1 = fin1.readlines()
lines2 = fin2.readlines()
diff_lines = [l.strip() for l in lines1 if l not in lines2]
diffs = defaultdict(list)
with open(f1+'x'+f2+'Result.txt', 'w') as fout:
for line in diff_lines:
d = eval(line)
for k in d:
list_ids = d[k]
for i in range(0, len(d[k]), 2):
diffs[d[k][i]].append(k)
for id_ in diffs:
diffs[id_].sort()
for k, g in groupby(enumerate(diffs[id_]), lambda (i, x): i - x):
group = map(itemgetter(1), g)
fout.write('{0} {1} {2}\n'.format(id_, group[0], group[-1]))
def final(self):
with open('hw1load3xhw1load2Result.txt', 'r') as fin:
lines = (line.split() for line in fin)
for k, g in groupby(lines, itemgetter(0)):
fst = next(g)
lst = next(iter(deque(g, 1)), fst)
with open('final/{}.avs'.format(k), 'w') as fout:
fout.write('video0=ImageSource("MovieName\original\%06d.jpeg", {}, {}, 15)\n'.format(fst[1], lst[2]))
Now to my question, how do i make it so each of the functions passes it's output files as values to the next function and calls it?
So for an example:
running .load() should output two files, call the .compare() function passing it those two files.
Then when .compare() is done, it should pass .final() the output file and calls it.
So .final() will open whatever file is passed to it from .compare() and not "test123.txt" as it is defined above.
I hope this all makes sense. Let me know if you need clarification. Any criticism is welcome concerning the code itself. Thanks in advance.
There are a couple of ways to do this, but I would write a master function that calls the other three in sequence. Something like:
def load_and_compare(self, input_file1, input_file2, output_file1, output_file2, result_file):
self.load(input_file1, input_file2, output_file1, output_file2)
self.compare(output_file1, output_file2)
self.final(result_file)
Looking over your code, I think you have a problem in load. You only declare a single dictionary, then load the contents of both files into it and write those same contents out to two files. Because each file has the same content, compare won't do anything meaningful.
Also, do you really want to write out the file contents and then re-read it into memory? I would keep the frame definitions in memory for use in compare after loading rather than reading them back in.
I don't really see a reason for this to be a class at all rather than just a trio of functions, but maybe if you have to read multiple files with mildly varying formats you could get some benefit of using class attributes to define the format while inheriting the general logic.
Do you mean call with the name of the two files? Well you defined a class, so you can just do:
def load(self, fileIn1, fileIn2, fileOut1, fileOut2):
... // do stuff here
// when done
self.compare( fileOut1, fileOut2 )
And so on.
I might be totally off here, but why don't you do it exactly as you're saying?
Just call self.compare() out of your load() method.
You can also add return statements to load() and return a tuple with the files.
Then add a 4th method to your class, which then collects the returned files and pipes them to the compare() method.
Best Regards!
One of the more powerful aspects of Python is that you can return something called a tuple. To answer this in a more generic Python sense consider this code:
>>> def load(file1, file2):
return file1+'.txt',file2+'.txt'
>>> def convert(file1, file2):
return 'converted_'+file1,'converted_'+file2
>>> convert(*load("Java", "C#"))
('converted_Java.txt', 'converted_C#.txt')
Each function takes two named arguments, but the returned tuple of the first can be "unpacked" into the input arguments of the second by adding a * in front of it.