I'm trying to make a config file manager that stores dictionaries in a pickle file, in a config folder in the same directory. I'm having issues with a pesky AttributeError that claims I don't have an attribute '_name', which I defined in the first line of init().
The main program has 2 classes, one inheriting from Exceptions (the error class) and the other inheriting from dict (the main class). The init takes a name and finds if the file exists. If it doesn't, then an empty dictionary is written to the given file.
I receive the error when I:
1. open an existing file to print the keys
2. write a key and value pair to an existing file
3. try and get a key from the dictionary
I've tried not calling dict.setitem in init and nothing changed, I still get the error. I tried just loading the file while doing absolutely nothing else to it, and couldn't get it to work.
import os
import pickle
class ConfigDict(dict):
'''
This class is responsible for all main functions.
Pass a string as an argument without a file type.
Keywords such as 'database' and 'aiconstructer' work well.
'''
def __init__(self, name):
self._name = name+'.pickle'
if not os.path.isfile(self._name):
with open(self._name, 'wb') as f:
pickle.dump('{}', f)
with open(self._name,'rb') as f:
obj = pickle.load(f)
if len(obj) > 2:
for k in obj.keys():
dict.__setitem__(self, k, obj[k])
def __getitem__(self, key):
if not key in self.keys():
raise ConfigKeyError(self, key)
return dict.__getitem__(key)
def __setitem__(self, key, val):
dict.__setitem__(self, key, val)
with open(self._name, 'wb') as f:
pickle.dump(self, f)
The Traceback is as follows:
Traceback (most recent call last):
File "interface.py", line 12, in <module>
test = ConfigDict('alpha')
File "/home/bear/Desktop/Config/confdict.py", line 35, in __init__
obj = pickle.load(f)
File "/home/bear/Desktop/Config/confdict.py", line 47, in __setitem__
with open(self._name, 'wb') as f:
AttributeError: 'ConfigDict' object has no attribute '_name'
The code for interface.py that initiates this is:
import sys
from confdict import ConfigDict, ConfigKeyError
test = ConfigDict('alpha')
if len(sys.argv) == 3:
key = sys.argv[1]
val = sys.argv[2]
print('Writing')
test[key] = val
print('Done')
elif len(sys.argv) == 2:
key = sys.argv[1]
print('{}:{}'.format(key,test[key]))
else:
print('Keys : Values')
print('-----------------')
for k in test.keys():
print('{} : {}'.format(k,test[k]))
I expect to be able to load the contents of the pickle file into self, but instead I get the AttributeError. Is it something I'm doing grammatically wrong, or is there a rule that I forgot to follow? Thank you very much for any help in advance.
Related
My save code is written as so in a file called 'MachLearn.py':
Whilst looking for this code I find out I accidentally overwrote it with an old version :/. It was essentially structured like this:
class attibuteGenerator():
def __init__(self):
#more class stuff
def returnAttributes(self, rating, position):
#func stuff
if __name__ = "__main__":
ag = attributeGenerator():
with open('attributeGenerator_pickle', 'wb') as f:
pickle.dump(f, ag)
My open code is written as so in a file called "mainGame.py"
def main():
with open('attributeGenerator_pickle', 'rb') as f:
bob = pickle.load(f)
print(bob.returnAttributes(34, "LW"))
if __name__ == "__main__":
main()
Is there an issue with my code? It's giving:
This type of error arises when you attempt to unpickle an object, such as a class instance, and you don't have access to the relevant class in your current Python session. You should reimport the packages you were using to generate the objects that you pickled.
The following minimal script will reproduce this problem:
import pickle
class my_class():
def __init__(self):
self.x = 2
inst = my_class()
with open('file.dat', 'wb') as f:
pickle.dump(inst, f); f.close()
del my_class # Deletes the class and causes the problem
with open('file.dat', 'rb') as f:
new_inst = pickle.load(f); f.close()
with error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-10-6571eae10a31> in <module>
12
13 with open('file.dat', 'rb') as f:
---> 14 new_inst = pickle.load(f); f.close()
AttributeError: Can't get attribute 'my_class' on <module '__main__'>
This can be resolved by removing the line: del my_class.
So, I have class that I use in a Flask app. I use this class in multiple pages, which is why I would like to save the creates class object in a pickle, and unpack it when I need it again. It just keeps on giving me errors.. I have a class that looks similar to this:
class files(name):
def __init__(self, name):
self.name = name
self.settings = Settings()
self.files_directory = self.settings.files_directory
self.files = self.create_list()
def store_files_from_folder(self):
loaded_files = []
files = list_files()
for file in files:
file_path = os.path.join(self.files_directory, file)
print('Loading file: {}'.format(file))
loaded_file = function_reads_in_files_from_folder(file_path, self.name)
loaded_files.append(loaded_file)
print('Loaded {} files'.format(len(loaded_files)))
and I'm trying to create the jsonpickle like this:
creates_class = files("Mario")
jsonpickle_test = jsonpickle.encode(creates_class, unpicklable=False)
result = jsonpickle.decode(jsonpickle_test, files)
But I get the following error:
Traceback (most recent call last):
File "C:\Users\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-8-23e9b5d176ac>", line 1, in <module>
result = jsonpickle.decode(jsonpickle_test, files)
File "C:\Users\lib\site-packages\jsonpickle\unpickler.py", line 41, in decode
data = backend.decode(string)
AttributeError: type object 'files' has no attribute 'decode'
And I can't get to resolve it. Could someone help me?
The problem is in the passed argument unpickable=False
unpicklable – If set to False then the output will not contain the information necessary to turn the JSON data back into Python objects, but a simpler JSON stream is produced.
You can avoid unpickable=False or load the produced data with json.loads to a dict and then use de kwargs arguments for the object creation
creates_class = files("Mario")
jsonpickle_test = jsonpickle.encode(creates_class, unpicklable=False)
result_dict = json.loads(jsonpickle_test)
create_class = files(**result_dict)
I have a function that calls a sub-function to open up a file. I am trying to test the parent function, but I want to patch the sub-function and have it return the data I pass in (as if it read from a file).
tests.py
# Read in the sample data
__SAMPLE_LOG = os.path.join(settings.BASE_DIR, "apps/tests/log_viewer/sample_logs/sample_manager_log.log")
sample_data = []
for line in reversed_lines(open(__SAMPLE_LOG)):
sample_data.append(line)
sample_data = ('').join(sample_data)
class ReadLog(TestCase):
#patch('apps.log_viewer.utils.reversed_lines', new_callable = mock_open, read_data = sample_data)
def test_returnsDictionaryContainingListOfDictionaries(self, mock_file):
activity = read_log()
# Make sure the sample data was read ==> this fails.
self.assertEqual(open(settings.ACTIVITY_LOG_FILE).read(), sample_data)
utils.py
def read_log():
# This is the line I am trying to patch
for line in reversed_lines(open(settings.ACTIVITY_LOG_FILE)):
# process data
# see: https://stackoverflow.com/questions/260273/most-efficient-way-to-search-the-last-x-lines-of-a-file-in-python/260433#260433
def reversed_lines(file):
"Generate the lines of file in reverse order."
part = ''
for block in reversed_blocks(file):
for c in reversed(block):
if c == '\n' and part:
yield part[::-1]
part = ''
part += c
if part: yield part[::-1]
def reversed_blocks(file, blocksize=4096):
"Generate blocks of file's contents in reverse order."
file.seek(0, os.SEEK_END)
here = file.tell()
while 0 < here:
delta = min(blocksize, here)
here -= delta
file.seek(here, os.SEEK_SET)
yield file.read(delta)
The error
I am trying to patch reversed_lines() in utils.py within the read_log() method, but read_log() is still reading from the actual log, indicating that I am not patching reversed_lines() correctly.
When I change
#patch('apps.log_viewer.utils.reversed_lines', new_callable = mock_open, read_data = sample_data)
to
#patch('builtins.open', new_callable = mock_open, read_data = sample_data)
I get
======================================================================
ERROR: test_returnsDictionaryContainingListOfDictionaries
(tests.log_viewer.test_utils.ReadLog)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py", line 1209, in patched
return func(*args, **keywargs)
File "/webapp/apps/tests/log_viewer/test_utils.py", line 32, in test_returnsDictionaryContainingListOfDictionaries
activity = read_log()
File "/webapp/apps/log_viewer/utils.py", line 64, in read_log
for line in reversed_lines(open(settings.ACTIVITY_LOG_FILE)):
File "/webapp/apps/log_viewer/utils.py", line 173, in reversed_lines
for block in reversed_blocks(file):
File "/webapp/apps/log_viewer/utils.py", line 164, in reversed_blocks
while 0 < here:
TypeError: '<' not supported between instances of 'int' and 'MagicMock'
Where am I going wrong?
Following the example from the docs at https://docs.python.org/3.3/library/unittest.mock.html#mock-open I think you want
#patch('builtins.open', mock_open(read_data = sample_data), create=True)
However reading through the source of mock_open: https://github.com/python/cpython/blob/3.7/Lib/unittest/mock.py#L2350
It appears that the tell method for filehandles is not implemented by the mock. The only supported methods are read, readline, readlines, write and iterating over the contents. You'll need to manually set up the mock for the tell method. This is not a general implementation but will work in your specific case:
class ReadLog(TestCase):
#patch('builtins.open', mock_open(read_data = sample_data), create=True)
def test_returnsDictionaryContainingListOfDictionaries(self, mock_file):
mock_file.return_value.tell.return_value = len(sample_data)
...
My goal is to serialize a dictionary object to a specific file location, and read it back each time the program is run. The following works in Python2.7, but throws an error in Python3.4. What confuses me is that this works the first time an object is saved to disk, but not on subsequent executions.
The problem seems to be that in setitem an error is thrown that says that 'ConfigDict' object has no attribute '_config_file'. Why does it not have a value any time except the first time I run the script??
import os
import pickle
class ConfigDict(dict):
def __init__(self, config_name): # Name of a pickle file within configs directory
self._config_directory = 'C:\\Users\\myfilepath\\configs'
self._config_file = self._config_directory + '\\' + config_name + '.pickle'
# If file does not exist, write a blank pickle file.
if not os.path.isfile(self._config_file):
with open(self._config_file, 'wb') as fh:
pickle.dump({}, fh)
# Read the pickle file from disk.
with open(self._config_file, 'rb') as fh:
pkl = pickle.load(fh)
self.update(pkl)
def __setitem__(self, key, value):
dict.__setitem__(self, key, value)
with open(self._config_file, 'wb') as fh:
pickle.dump(self, fh)
cc = ConfigDict('DBConfig')
print()
print()
cc['config_val_1'] = '1'
cc['config_val_3'] = '3'
print(cc['config_val_3'])
Here's the full traceback:
Traceback (most recent call last):
File "C:/Users/filepath/test.py", line 25, in <module>
cc = ConfigDict('DBConfig')
File "C:/Users/filepath/test.py", line 16, in __init__
pkl = pickle.load(fh)
File "C:/Users/filepath/test.py", line 21, in __setitem__
with open(self._config_file, 'wb') as fh:
AttributeError: 'ConfigDict' object has no attribute '_config_file'
I want to create a class for storing attributes of the many data files that my script has to process. The attributes are values that are found in the datafiles, or values that are calculated from other values that are found in the data files.
Unfortunately, I'm not understanding the output of the code that I've written to accomplish that goal. What I think this should do is: print the name of the file being processed and a value seqlength from that file. The actual output is given below the code.
class SrcFile:
def __init__(self, which):
self.name = which
def seqlength(self):
with open(self.name) as file:
linecounter = 0
for line in file:
linecounter += 1
if linecounter == 3:
self.seqlength = int(line.split()[0])
break
for f in files:
file = SrcFile(f)
print(file.name, file.seqlength)
This prints file.name as expected, but for file.seqlength it returns a value that I don't understand.
../Testdata/12_indels.ss <bound method SrcFile.seqlength of <__main__.SrcFile object at 0x10066cad0>>
It's clear to me that I'm not understanding something fundamental about classes and functions. Is it clear to you what I'm missing here?
.seqlength is a method and needs (), but you are also not returning anything from it. Try this instead:
def seqlength(self):
with open(self.name) as file:
linecounter = 0
for line in file:
linecounter += 1
if linecounter == 3:
return int(line.split()[0])
And then calling it:
for f in files:
file = SrcFile(f)
print(file.name, file.seqlength())
Thats because .seqlength is a method.
Try doing
print(filename, file.seqlength())