I am having issues with getting basic function of the nptdms module working.
First, I am just trying to open a TDMS file and print the contents of specific channels within specific groups.
Using python 2.7 and the nptdms quick start here
Following this, I will be writing these specific pieces of data into a new TDMS file. Then, my ultimate goal is to be able to take a set of source files, open each, and write (append) to a new file. The source data files contain far more information that is needed, so I am breaking out the specifics into their own file.
The problem I have is that I cannot get past a basic error.
When running this code, I get:
Traceback (most recent call last):
File "PullTDMSdataIntoNewFile.py", line 27, in <module>
tdms_file = TdmsFile(r"C:\\Users\daniel.worts\Desktop\this_is_my_tdms_file.tdms","r")
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 94, in __init__
self._read_segments(f)
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 119, in _read_segments
object._initialise_data(memmap_dir=self.memmap_dir)
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 709, in _initialise_data
mode='w+b', prefix="nptdms_", dir=memmap_dir)
File "C:\Anaconda2\lib\tempfile.py", line 475, in NamedTemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
File "C:\Anaconda2\lib\tempfile.py", line 244, in _mkstemp_inner
fd = _os.open(file, flags, 0600)
OSError: [Errno 2] No such file or directory: 'r\\nptdms_yjfyam'
Here is my code:
from nptdms import TdmsFile
import numpy as np
import pandas as pd
#set Tdms file path
tdms_file = TdmsFile(r"C:\\Users\daniel.worts\Desktop\this_is_my_tdms_file.tdms","r")
# set variable for TDMS groups
group_nameone = '101'
group_nametwo = '752'
# set objects for TDMS channels
channel_dataone = tdms_file.object(group_nameone 'Payload_1')
channel_datatwo = tdms_file.object(group_nametwo, 'Payload_2')
# set data from channels
data_dataone = channel_dataone.data
data_datatwo = channel_datatwo.data
print data_dataone
print data_datatwo
Big thanks to anyone who may have encountered this before and can help point to what I am missing.
Best,
- Dan
edit:
Solved the read data issue by removing the 'r' argument from the file path.
Now I am having another error I can't trace when trying to write.
from nptdms import TdmsFile, TdmsWriter, RootObject, GroupObject, ChannelObject
import numpy as np
import pandas as pd
newfilepath = r"C:\\Users\daniel.worts\Desktop\Mined.tdms"
datetimegroup101_channel_object = ChannelObject('101', DateTime, data_datetimegroup101)
with TdmsWriter(newfilepath) as tdms_writer:
tdms_writer.write_segment([datetimegroup101_channel_object])
Returns error:
Traceback (most recent call last):
File "PullTDMSdataIntoNewFile.py", line 82, in <module>
tdms_writer.write_segment([datetimegroup101_channel_object])
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 68, in write_segment
segment = TdmsSegment(objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 88, in __init__
paths = set(obj.path for obj in objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 88, in <genexpr>
paths = set(obj.path for obj in objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 254, in path
self.channel.replace("'", "''"))
AttributeError: 'TdmsObject' object has no attribute 'replace'
Related
I'm trying to add keywords to the IPTC data in a JPG file and failing miserably. I'm able to read in the keywords using the iptcinfo3 library and, seemingly, append the keyword to the list of current keywords but I'm failing when trying to write those keywords back to the JPG file, if not sooner. The error message is a bit unclear to me and may actually reference the appending of the new keyword (although a print statement seems to indicate it took).
I've tried three different metadata libraries (there doesn't seem to be one standard) and this is the furthest I've gotten with any of them (failing to even install one and not being able to get a second one to run). This seems so basic but I can't figure it out and haven't been able to adapt the few other code examples I've seen online to work, including iptcinfo3's example code fragment.
The current Error message is:
| => pipenv run python editMetadata.py
WARNING: problems with charset recognition (b'\x1b')
[b'Gus']
[b'Gus', b'frog']
Traceback (most recent call last):
File "editMetadata.py", line 22, in <module>
info.save_as('Gus2.jpg')
File "/Users/Scott/.local/share/virtualenvs/editPhotoMetadata-tx0JAOmI/lib/python3.7/site-packages/iptcinfo3.py", line 635, in save_as
jpeg_parts = jpeg_collect_file_parts(fh)
File "/Users/Scott/.local/share/virtualenvs/editPhotoMetadata-tx0JAOmI/lib/python3.7/site-packages/iptcinfo3.py", line 324, in jpeg_collect_file_parts
adobeParts = collect_adobe_parts(partdata)
File "/Users/Scott/.local/share/virtualenvs/editPhotoMetadata-tx0JAOmI/lib/python3.7/site-packages/iptcinfo3.py", line 433, in collect_adobe_parts
out = [''.join(out)]
TypeError: sequence item 0: expected str instance, bytes found
Code:
from iptcinfo3 import IPTCInfo
import os
# Create new info object
info = IPTCInfo('Gus.jpg')
# Print list of keywords
print(info['keywords'])
# Append the keyword I want to add
info['keywords'].append(b'frog')
# Print to test keyword has been added
print(info['keywords'])
# Save new info to file
info.save()
info.save_as('Gus2.jpg')
Instead of appending use equal "="
from iptcinfo3 import IPTCInfo
info = IPTCInfo('Gus.jpg')
print(info['keywords'])
# add keyword
info['keywords'] = ['new keyword']
info.save()
info.save_as('Gus_2.jpg')
I have the same error. It seems to be an issue with the save depending on the file.
from iptcinfo3 import IPTCInfo
info = IPTCInfo('image.jpg', force=True)
info.save()
Which gives me the same error.
WARNING: problems with charset recognition (b'\x1b')
WARNING: problems with charset recognition (b'\x1b')
Traceback (most recent call last):
File "./searchimages.py", line 123, in <module>
main(sys.argv[1:])
File "./searchimages.py", line 119, in main
find_photos(str(sys.argv[1]))
File "./searchimages.py", line 46, in find_photos
write_keywords(image, current_keywords, new_keywords)
File "./searchimages.py", line 109, in write_keywords
info.save_as('out.jpg')
File "/usr/local/lib/python3.7/site-packages/iptcinfo3.py", line 635, in save_as
jpeg_parts = jpeg_collect_file_parts(fh)
File "/usr/local/lib/python3.7/site-packages/iptcinfo3.py", line 324, in jpeg_collect_file_parts
adobeParts = collect_adobe_parts(partdata)
File "/usr/local/lib/python3.7/site-packages/iptcinfo3.py", line 433, in collect_adobe_parts
out = [''.join(out)]
TypeError: sequence item 0: expected str instance, bytes found
I'm relatively new to Python and am trying to analyze an entire folder of netCDF files. When I try to run my code I get the key error: u'satcode'. Here is my code:
import netCDF4 as nc
import glob
import numpy as np
filenames = []
for name in glob.glob("/Users/my_name/Desktop/ISCCP/*"):
print name
filenames.append(name)
# below reads multiple files (file_list)
f = nc.MFDataset(filenames)
I know the error is with f = nc.MFDataset(filenames), but I don't know why.
My traceback says:
runfile('/Users/my_name/.spyder/West_Coast_Model.py', wdir='/Users/my_name/.spyder')
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGG.v01r00.GLOBAL.1983.07.01.0000.GPC.10KM.CS00.EA1.00 (5).nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.0300.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.0600.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.0900.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.1200.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.1500.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.1800.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.07.99.2100.GPC.10KM.CS00.EA1.00.nc
/Users/my_name/Desktop/ISCCP/ISCCP-Basic.HGH.v01r00.GLOBAL.1983.08.99.0000.GPC.10KM.CS00.EA1.00.nc
Traceback (most recent call last):
File "", line 1, in
runfile('/Users/my_name/.spyder/West_Coast_Model.py', wdir='/Users/my_name/.spyder')
File "/Users/my_name/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "/Users/my_name/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
builtins.execfile(filename, *where)
File "/Users/my_name/.spyder/West_Coast_Model.py", line 20, in
f = MFDataset(filenames)
File "netCDF4/_netCDF4.pyx", line 5888, in >netCDF4._netCDF4.MFDataset.init
KeyError: u'satcode'
I think I figured out the problem. The 0th term in the files was corrupted, so I redownloaded it and converted it into netCDF4_CLASSIC and it worked.
This is my codes. When I try to save a xlxs with comments, It failed. How can I know when to save again.
from openpyxl import load_workbook
import datetime
filename = u"large_table.xlsx"
model = load_workbook(filename)
model.properties.lastPrinted = datetime.datetime.now()
model.save(filename)
model.properties.lastPrinted = datetime.datetime.now()
model.save(filename)
Traceback: It seems that self.workbook.vba_archive is set to None unexpectedly.
Traceback (most recent call last):
File "D:/h32workspace/trunk/event_editor/eric6/model/test_file.py", line 31, in <module>
model.save(filename)
File "C:\Python27\lib\site-packages\openpyxl\workbook\workbook.py", line 342, in save
save_workbook(self, filename)
File "C:\Python27\lib\site-packages\openpyxl\writer\excel.py", line 269, in save_workbook
writer.save(filename)
File "C:\Python27\lib\site-packages\openpyxl\writer\excel.py", line 251, in save
self.write_data()
File "C:\Python27\lib\site-packages\openpyxl\writer\excel.py", line 81, in write_data
self._write_worksheets()
File "C:\Python27\lib\site-packages\openpyxl\writer\excel.py", line 214, in _write_worksheets
self._write_comment(ws)
File "C:\Python27\lib\site-packages\openpyxl\writer\excel.py", line 184, in _write_comment
vml = fromstring(self.workbook.vba_archive.read(ws.legacy_drawing))
AttributeError: 'NoneType' object has no attribute 'read'
I tried to use keep_vba=True to load workbook, but if failed to save file correctly. The saved file can not be opened.
I used your code to save a sample .xlsx file. It saved without any issues.
Do you have any macro within you .xlsx file?
If yes, you may want to open the xlsx file with macro enabled using
model = load_workbook(filename, keep_vba=True)
See here for details on openpyxl usage with macro.
Also, try to save to a different filename than trying to overwrite original to make sure it works correctly.
fileout = "test2.xlsx"
model.save(fileout)
Hope this helps.
I am trying to open a HDF5 file in order to read it with python, so that I can do more things with it later. There is an error when I run the program to read the file. The program is below:
import h5py # HDF5 support
import numpy
fileName = "C:/.../file.h5"
f = h5py.File(fileName, "r")
for item in f.attrs.keys():
print item + ":", f.attrs[item]
mr = f['/entry/mr_scan/mr']
i00 = f['/entry/mr_scan/I00']
print "%s\t%s\t%s" % ("#", "mr", "I00")
for i in range(len(mr)):
print "%d\t%g\t%d" % (i, mr[i], i00[i])
f.close()
If I run the program I end up seeing this error:
Traceback (most recent call last):
File "TestHD5.py", line 8, in <module>
mr = f['/entry/mr_scan/mr']
File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects.c:2587)
File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects.c:2546)
File "C:\programs\Python27\lib\site-packages\h5py\_hl\group.py", line 166, in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects.c:2587)
File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (C:\aroot\work\h5py\_objects.c:2546)
File "h5py\h5o.pyx", line 190, in h5py.h5o.open (C:\aroot\work\h5py\h5o.c:3417)
KeyError: 'Unable to open object (Component not found)'
Am I just missing some modules to read the file, or is this something else. It will open the .h5 file if I use an h5 file veiwer program. Thank you
Your string:
path = "C:\Users\312001\m2020\data\20170104_145626\doPoint_20170104_150016\dataset_XMIT data_20170104_150020.h5"
is full of broken/illegal escapes (thankfully they will be turned into SyntaxErrors, though you are using Python 2), and some that actually do work, so Python thinks path is really equal to: 'C:\\Users\xca001\\m2020\\data\x8170104_145626\\doPoint_20170104_150016\\dataset_XMIT data_20170104_150020.h5' (note those \x##'s).
Your options:
Use a raw string by prefixing the string literal with r
Don't use backslashes for paths. Python will convert forward slashes to backslashes for Windows paths.
Double-backslash.
The answer that #NickT posted fixed the orginal problem I had. The problem that is shown in the new version is due to the hd5 folder names in the hd5 file not matching the folder names that the code provided.
I have a problem in python. I'm using scipy, where i use scipy.io to load a .mat file. The .mat file was created using MATLAB.
listOfFiles = os.listdir(loadpathTrain)
for f in listOfFiles:
fullPath = loadpathTrain + '/' + f
mat_contents = sio.loadmat(fullPath)
print fullPath
Here's the error:
Traceback (most recent call last):
File "tryRankNet.py", line 1112, in <module>
demo()
File "tryRankNet.py", line 645, in demo
mat_contents = sio.loadmat(fullPath)
File "/usr/lib/python2.6/dist-packages/scipy/io/matlab/mio.py", line 111, in loadmat
matfile_dict = MR.get_variables()
File "/usr/lib/python2.6/dist-packages/scipy/io/matlab/miobase.py", line 356, in get_variables
getter = self.matrix_getter_factory()
File "/usr/lib/python2.6/dist-packages/scipy/io/matlab/mio5.py", line 602, in matrix_getter_factory
return self._array_reader.matrix_getter_factory()
File "/usr/lib/python2.6/dist-packages/scipy/io/matlab/mio5.py", line 274, in matrix_getter_factory
tag = self.read_dtype(self.dtypes['tag_full'])
File "/usr/lib/python2.6/dist-packages/scipy/io/matlab/miobase.py", line 171, in read_dtype
order='F')
TypeError: buffer is too small for requested array
The whole thing is in a loop, and I checked the size of the file where it gives the error by loading it interactively in IDLE.
The size is (9,521), which is not at all huge. I tried to find if I'm supposed to clear the buffer after each iteration of the loop, but I could not find anything.
Any help would be appreciated.
Thanks.