I'm trying to merge multiple .xls files into a single workbook, where each file is inserted into a sheet, named with the .xls filename.
While surfing on web, I've seen the documentation of Pyexcel and a specific module which, as written here, could do the job easly.
Here's the code.
from pyexcel.cookbook import merge_all_to_a_book
import glob
merge_all_to_a_book(glob.glob("Dir\*.xls"),"output.xls")
As expected, it doesn't work. Here's the console output.
File "..\Desktop\scripts\provaimport.py", line 48, in <module>
merge_all_to_a_book(glob.glob("C:\Users\Tesisti\Desktop\forpythonscript\*.xls"),"output.xls")
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\cookbook.py", line 148, in merge_all_to_a_book
merged.save_as(outfilename)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\internal\meta.py", line 339, in save_as
return save_book(self, file_name=filename, **keywords)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\internal\core.py", line 51, in save_book
return _save_any(a_source, book)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\internal\core.py", line 55, in _save_any
a_source.write_data(instance)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\plugins\sources\file_output.py", line 38, in write_data
**self._keywords)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel\plugins\renderers\excel.py", line 30, in render_book_to_file
save_data(file_name, book.to_dict(), **keywords)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel_io\io.py", line 119, in save_data
**keywords)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel_io\io.py", line 141, in store_data
writer.write(data)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel_io\book.py", line 58, in __exit__
self.close()
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\pyexcel_xls\xlsw.py", line 86, in close
self.work_book.save(self._file_alike_object)
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\xlwt\Workbook.py", line 710, in save
doc.save(filename_or_stream, self.get_biff_data())
File "C:\Users\Tesisti\Anaconda2\lib\site-packages\xlwt\Workbook.py", line 680, in get_biff_data
self.__worksheets[self.__active_sheet].selected = True
Any idea on how to fix?
It seems to me that glob.glob("Dir*.xls") returned an empty list of files. Hence pyexcel's plugin pyexcel-xls fails to create an empty file.
The current solution, I would recommend is to take the latest pyexcel-xls and use try-except statement around merge_all_to_a_book, catching empty file case.
Related
I am building automation for Excel a multi-tabbed excel document. When I try to close the document I get the error below (full traceback, minus the personal details at the top), which then is corrupted and I cannot open the xlsx document. Unfortunately I haven't found any clues to go off of. I am using xlsxwriter functions to set row and column formatting, from what I've found this could be an issue but I haven't been able to track it down. Any thoughts on possible solutions?
writer.close()
File "/opt/homebrew/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 1480, in close
self._save()
File "/opt/homebrew/lib/python3.10/site-packages/pandas/io/excel/_xlsxwriter.py", line 244, in _save
self.book.close()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/workbook.py", line 324, in close
self._store_workbook()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/workbook.py", line 709, in _store_workbook
xml_files = packager._create_package()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/packager.py", line 137, in _create_package
self._write_worksheet_files()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/packager.py", line 193, in _write_worksheet_files
worksheet._assemble_xml_file()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/worksheet.py", line 4221, in _assemble_xml_file
self._write_cols()
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/worksheet.py", line 5807, in _write_cols
self._write_col_info(self.colinfo[col])
File "/opt/homebrew/lib/python3.10/site-packages/xlsxwriter/worksheet.py", line 5836, in _write_col_info
if width > 0:
TypeError: '>' not supported between instances of 'Format' and 'int'
I have a folder with many xlsx files that I'd like to convert to csv files.
During my research, if found several threads about this topic, such as this or that one. Based on this, I formulated the following code using glob and pandas:
import glob
import pandas as pd
path = r'/Users/.../xlsx files'
excel_files = glob.glob(path + '/*.xlsx')
for excel in excel_files:
out = excel.split('.')[0]+'.csv'
df = pd.read_excel(excel) # error occurs here
df.to_csv(out)
But unfortunately, I got the following error message that I could not interpret in this context and I could not figure out how to solve this problem:
Traceback (most recent call last):
File "<input>", line 11, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/util/_decorators.py", line 299, in wrapper
return func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/excel/_base.py", line 336, in read_excel
io = ExcelFile(io, storage_options=storage_options, engine=engine)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/excel/_base.py", line 1131, in __init__
self._reader = self._engines[engine](self._io, storage_options=storage_options)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/excel/_openpyxl.py", line 475, in __init__
super().__init__(filepath_or_buffer, storage_options=storage_options)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/excel/_base.py", line 391, in __init__
self.book = self.load_workbook(self.handles.handle)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/excel/_openpyxl.py", line 486, in load_workbook
return load_workbook(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/reader/excel.py", line 317, in load_workbook
reader.read()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/reader/excel.py", line 281, in read
apply_stylesheet(self.archive, self.wb)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/styles/stylesheet.py", line 198, in apply_stylesheet
stylesheet = Stylesheet.from_tree(node)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/styles/stylesheet.py", line 103, in from_tree
return super(Stylesheet, cls).from_tree(node)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/descriptors/serialisable.py", line 87, in from_tree
obj = desc.expected_type.from_tree(el)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/descriptors/serialisable.py", line 87, in from_tree
obj = desc.expected_type.from_tree(el)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/openpyxl/descriptors/serialisable.py", line 103, in from_tree
return cls(**attrib)
TypeError: __init__() got an unexpected keyword argument 'xfid'
Does anyone know how to fix this? Thanks a lot for your help!
I had the same problem here. After some hours thinking and searching I realized the problem is, actually, the file. I opened it using MS Excel, and save. Alakazan, problem solved.
The file was downloaded, so i think it's a "security" error or just an error from how the file was created. xD
EDIT:
It's not a security problem, but actually an error from the generation of file. The correct has the double of kb the wrong file.
An solution is: if using xlrd==1.2.0 the file can be opened, you can, after doing this, call read_excel to the Book(file opened by xlrd).
import xlrd
# df = pd.read_excel('TabelaPrecos.xlsx')
# The line above is the same result
a = xlrd.open_workbook('TabelaPrecos.xlsx')
b = pd.read_excel(a)
I try to read a xlsx into a data frame:
itut_ir = pd.read_excel('C:\\Users\\Administrator\\Downloads\\reportdata.xlsx')
print(itut_ir.to_string())
I receive this:
Traceback (most recent call last):
File
"C:\Users\Administrator\eclipse-workspace\Reports\GOW\Report.py",
line 44, in
df = pd.read_excel('C:\Users\Administrator\Downloads\reportdata.xlsx')
File
"C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\excel_base.py",
line 304, in read_excel
io = ExcelFile(io, engine=engine) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\excel_base.py",
line 824, in init
self._reader = self.enginesengine File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\excel_xlrd.py",
line 21, in init
super().init(filepath_or_buffer) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\excel_base.py",
line 353, in init
self.book = self.load_workbook(filepath_or_buffer) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\io\excel_xlrd.py",
line 36, in load_workbook
return open_workbook(filepath_or_buffer) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\xlrd_init.py",
line 117, in open_workbook
zf = zipfile.ZipFile(filename) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\zipfile.py",
line 1222, in init
self._RealGetContents() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\zipfile.py",
line 1289, in _RealGetContents
raise BadZipFile("File is not a zip file") zipfile.BadZipFile: File is not a zip file
does anybody have an idea? the file does not seem to be broken, I can open it with Excel.
thanks!
*** UPDATE ***
the file producing the error is being downloaded from FTP. opening the original file works ... if that gives you a hint :) thanks
I had the same issue just a little bit ago with an XLSX that I created in LibreOffice.
The solution was to check the XLSX to make sure it wasn't corrupted. In my case, loading a previous version of the XLSX file corrected the problem.
I have a zipped folder containing 15 000 yaml files. I'd like to iterate through the folder using yaml.safe_load so that each file is in a dictionary format and I can extract information from each file that I need. I've written some code so far using zipfile.ZipFile and yaml.safe_load but it only works for the first file in the zipped folder. Would anyone please mind taking a look and explaining what I'm misunderstanding please?
zip_file = zipfile.ZipFile("D:/export.zip")
files = zip_file.namelist()
print(files)
for i in range(10):
with zip_file.open(files[i]) as yamlfile:
yamlreader = yaml.safe_load(yamlfile)
print(yamlreader["identifier"])
for now I'm just iterating through 10 files to make life easier. Eventually I'd like to do the whole 15 000. "identifier" is a key in the yaml file.
This is the error:
10.5281/zenodo.1014773
Traceback (most recent call last):
File "C:/Users/estho/PycharmProjects/GSOC3/testing_dataextraction.py", line 20, in <module>
yamlreader = yaml.safe_load(yamlfile)
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\__init__.py", line 162, in safe_load
return load(stream, SafeLoader)
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\__init__.py", line 114, in load
return loader.get_single_data()
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\constructor.py", line 41, in get_single_data
node = self.get_single_node()
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\composer.py", line 36, in get_single_node
document = self.compose_document()
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\composer.py", line 127, in compose_mapping_node
while not self.check_event(MappingEndEvent):
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\parser.py", line 98, in check_event
self.current_event = self.state()
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\parser.py", line 428, in parse_block_mapping_key
if self.check_token(KeyToken):
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\scanner.py", line 116, in check_token
self.fetch_more_tokens()
File "C:\Users\estho\PycharmProjects\GSOC3\lib\site-packages\yaml\scanner.py", line 260, in fetch_more_tokens
self.get_mark())
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
in "yamlfile_10_5281_zenodo_1745362.yaml", line 4, column 1
Thank you.
It seems to me like in the file "yamlfile_10_5281_zenodo_1745362.yaml" there is a bad token name. Try running it without this file. In python \t is representative of a tab and so cannot be included in a string ect normally without escaping it.
I'm getting a strange error when parsing a YAML:
yaml.scanner.ScannerError: mapping values are not allowed here
The YAML file I'm trying to read is valid according to YAML Lint
Another strange thing is that it works fine on my laptop (Arch Linux) but not on the Server (Ubuntu). The PyYAML version is the same though on both machines.
I have seen the other posts on stackoverflow where people were missing the space after the colon, but I'm not missing any spaces.
This is the complete YAML file:
pipeline:
- read:
input: /home/omnibrain/projects/company/data/data.csv
output: some_data
- filter:
input: some_data
filtername: latlng_filter
minlat: 32.5
maxlat: 32.9
minlng: -117.4
maxlng: -117.0
- enhance:
input: some_data
enhancername: geo_enhancer
fields: zip
- write:
input: some_data
writername: csv_writer
output_dir: /home/omnibrain/outputs
columns: [id, latitude, longitude, zip, networktype]
filename: example1 # the output filename
And this is the complete stack trace:
Traceback (most recent call last):
File "/usr/local/bin/someproject", line 9, in <module>
load_entry_point('someproject==0.0.1', 'console_scripts', 'someproject')()
File "/usr/local/lib/python3.4/dist-packages/someproject-0.0.1-py3.4.egg/someproject/__init__.py", line 19, in main
pipeline.Pipeline(parser.parse_args().scriptfile).start()
File "/usr/local/lib/python3.4/dist-packages/someproject-0.0.1-py3.4.egg/someproject/pipeline/pipeline.py", line 20, in __init__
self._raw_pipeline = self._parse_yaml(yamlscript)
File "/usr/local/lib/python3.4/dist-packages/someproject-0.0.1-py3.4.egg/someproject/pipeline/pipeline.py", line 55, in _parse_yaml
data = yaml.load(yamlscript)
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/__init__.py", line 72, in load
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/constructor.py", line 35, in get_single_data
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 36, in get_single_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 55, in compose_document
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 84, in compose_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 133, in compose_mapping_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 82, in compose_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 111, in compose_sequence_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 84, in compose_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 133, in compose_mapping_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 84, in compose_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/composer.py", line 127, in compose_mapping_node
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/parser.py", line 98, in check_event
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/parser.py", line 428, in parse_block_mapping_key
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/scanner.py", line 116, in check_token
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/scanner.py", line 220, in fetch_more_tokens
File "/usr/local/lib/python3.4/dist-packages/PyYAML-3.11-py3.4-linux-x86_64.egg/yaml/scanner.py", line 580, in fetch_value
yaml.scanner.ScannerError: mapping values are not allowed here
in "./test1.yaml", line 3, column 93
You are not missing any spaces after the colon, you have too many spaces in the line starting with input: /home/omnibrain/projects/company/data/data.csv. That is why you see line 3 column 93
That whole line reads something like:
input: /home/omnibrain/projects/company/data/data.csv output: some_data
It also should have some funny characters messing with your display as normally you would see a string
... output: some_data
below the mappings not allowed here.
That kind of differences normally occur if the files look the same, but in reality are not, e.g. after copy and paste from one terminal to another. Or after pasting into a website like YAMLlint.
Generate an md5sum on both systems for the file to check if they are really the same. Use od -c on the YAML file to inspect it for strange characters.