We Keep Coding

How to load multiple .mat files into a python script

I want to load 38 .mat files into a dictionary to hold them all.
the .mat files are named subject1 to subject38
The code I tried is a simple for loop
import scipy.io as sio
data = {}
for i in range(1, 38):
data["data{}".format(i)] = sio.loadmat('subject{}.mat'.format(i))
the error I'm getting is:
Traceback (most recent call last):
File "D:/senior project/python/dataAqu.py", line 7, in
data["data{0}".format(i)] = sio.loadmat('subject{0}.mat'.format(i))
File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio.py", line 208, in loadmat
matfile_dict = MR.get_variables(variable_names)
File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio5.py", line 292, in get_variables
res = self.read_var_array(hdr, process)
File "C:\Users\mamdo\AppData\Roaming\Python\Python27\site-packages\scipy\io\matlab\mio5.py", line 252, in read_var_array
return self._matrix_reader.array_from_header(header, process)
File "mio5_utils.pyx", line 675, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 705, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 778, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex
File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric
File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element
File "streams.pyx", line 194, in scipy.io.matlab.streams.ZlibInputStream.read_string
File "pyalloc.pxd", line 9, in scipy.io.matlab.pyalloc.pyalloc_v
MemoryError

So I found the problem. The mat files shouldnt be opened by any other program - like matlab - if there is an error restart the computer.
Also if there is a memory problem try to integrate the mat files seperatly and perform whatever code you need and then load the next file.

Parsing a json file in python create difficulties

I want to parse a json file in python. I don't know the content of the file. I downloaded this file from a website in json format.
As per my knowledge to parse a json file we need this code
import json
sourcefile=open("News_Category_Dataset_v2.json","r")
json_data=json.load(sourcefile)
print (json_data)
But I got this error as describe below. jsonparse.py is my file name which is save in my computer d:/algorithm
D:\python\envs\algorithms\python.exe D:/algorithms/jsonparse.py
Traceback (most recent call last):
File "D:/algorithms/jsonparse.py", line 4, in <module>
json_data=json.load(sourcefile)
File "D:\python\envs\algorithms\lib\json\__init__.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "D:\python\envs\algorithms\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "D:\python\envs\algorithms\lib\json\decoder.py", line 342, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 366)
Process finished with exit code 1
How could I fix the problem?

Your file is not json. but it has lines where each one of them is json.
This snippet should help you
import json
json_list = []
for i in open('test.json'):
json_line = json.loads(i)
json_list.append(json_line)
print(json_list)

Importing json files on GUI using python. getting JSONDecodeError

Link to the .json file -> This is the link to the .json file.
I am making a Python based project and what I am trying to do here is to make use of a .json file. However when I tried to import it to use it with the GUI I got the following error.
Thank you in advance.
Traceback (most recent call last):
File "<ipython-input-1-31816de2c4db>", line 1, in <module>
runfile('C:/Users/Akshita/pyex/Webmap/Gui.py', wdir='C:/Users/Akshita/pyex/Webmap')
File "C:\Users\Akshita\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\Akshita\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Akshita/pyex/Webmap/Gui.py", line 12, in <module>
data=json.load(open(r"C:\Users\Akshita\pyex\Webmap\world.json"))
File "C:\Users\Akshita\Anaconda3\lib\json\__init__.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Users\Akshita\Anaconda3\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Users\Akshita\Anaconda3\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Akshita\Anaconda3\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
# -*- coding: utf-8 -*-
"""
Created on Mon May 21 10:15:01 2018
#author: Akshita
"""
from tkinter import *
import json
import difflib
from difflib import get_close_matches
data=json.load(open(r"C:\Users\Akshita\pyex\Webmap\world.json"))
window=Tk()
window.title("Webmap")
l1=Label(window,text="Name_of_place")
l1.grid(row=0,column=0)
val=StringVar()
e=Entry(window,textvariable=val)
e.grid(row=0,column=1)
be=Button(window,text="Enter",command=entry)
be.grid(row=0,column=2)
def entry():
w= e_val.get()
w=w.title()
if w in data:
t.insert("Data matched")
elif len(get_close_matches(w,data.keys(),cutoff=0.8))>0:
t.insert("Did you mean %s instead?" %get_close_matches(w,data.keys())[0])
t=Text(window,height=3,width=20)
t.grid(row=1,column=1)
by=Button(window,text="Yes",command=yeah)
by.grid(row=2,column=1)
def yeah():
t.insert("Processing")
bn=Button(window,text="No",command=nah)
bn.grid(row=2,column=2)
def nah():
t.insert("No such Data")
window.mainloop()

Looks like your JSON file is invalid - the sequence of brackets at the end is incorrect.
The end should look like this:
... <previous data> ... ,[2.96361,36.802216]]]}}]}
You can use online or IDE JSON checkers to validate your file.
update
Got your full json file.
It turns out that it has UTF-8 BOM header. Try to load() it this way:
import json
import codecs
data = json.load(codecs.open('tst.json', 'r', 'utf-8-sig'))
print(data)
Related SO post: https://stackoverflow.com/a/13156715/2315573

Error trying to open JSON tweet file

I'm getting the following error when attempting to open a json file.
Traceback (most recent call last):
File "C:\Python34\test.py", line 5, in <module>
data = json.load(data_file)
File "C:\Python34\lib\json\__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Python34\lib\json\__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "C:\Python34\lib\json\decoder.py", line 346, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 2 column 1 - line 8300 column 1 (char 157 - 30292811)
This is what I"m doing to open the file in idle:
import json
with open('three_minutes_tweets.json','r', encoding="utf-8") as data_file:
data = json.load(data_file)
print(data_file)
The file is a tweet sample file and looks likes simple dictionaries of dictionaries. Thank you

The error message is telling you exactly what the problem is. There is extra data starting at character 157. In other words, you have invalid JSON data. There is nothing wrong with your code.

how to unpack dmoz urls from rdf dump with python and rdflib?

i tried to open rdf file (dmoz rdf dump), but a get this error message
Traceback (most recent call last):
File "/media/_dev_/ODP_RDF_get_links.py", line 4, in <module>
result = g.parse("data/content.rdf")
File "/usr/local/lib/python2.7/dist-packages/rdflib/graph.py", line 1033, in parse
parser.parse(source, self, **args)
File "/usr/local/lib/python2.7/dist-packages/rdflib/plugins/parsers/rdfxml.py", line 577, in parse
self._parser.parse(source)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.7/xml/sax/xmlreader.py", line 123, in parse
self.feed(buffer)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 210, in feed
self._parser.Parse(data, isFinal)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 352, in end_element_ns
self._cont_handler.endElementNS(pair, None)
File "/usr/local/lib/python2.7/dist-packages/rdflib/plugins/parsers/rdfxml.py", line 160, in endElementNS
self.current.end(name, qname)
File "/usr/local/lib/python2.7/dist-packages/rdflib/plugins/parsers/rdfxml.py", line 331, in node_element_end
self.error("Repeat node-elements inside property elements: %s"%"".join(name))
File "/usr/local/lib/python2.7/dist-packages/rdflib/plugins/parsers/rdfxml.py", line 185, in error
raise ParserError(info + message)
file:///media/_dev_/data/content.rdf:5:12: Repeat node-elements inside property elements: http://dmoz.org/rdf/catid
my simple code is as follow:
import rdflib
g = rdflib.Graph()
result = g.parse("data/content.rdf")
print("graph has %s statements." % len(g))
i need to be able to read the file.
extract all links in the world category.
thanks for any possible help
EDIT:
PS: found this wikipedia rdf_dumps, so developing custom scripts is necessary to use this dump