I am writing a function that converts a 2d python array into a matlab file. Here is my code so far...
def save_array(arr,fname):
import scipy.io
import numpy
out_dict={}
out_dict[fname]=arr
scipy.io.savemat(fname.mat,out_dict)`
I want fname to be a string, but I am not sure how I can get the savemat part to work.
import scipy.io
import numpy as np
def save_array(arr, arrname, fname):
"""
Save an array to a .mat file
Inputs:
arr: ndarray to save
arrname: name to save the array as (string)
fname: .mat filename (string)
"""
out_dict={arrname: arr}
scipy.io.savemat(fname,out_dict)
save_array(np.array([1,2,3]), 'arr', 'test.mat')
Might be worth doing a python tutorial or two. This is very basic stuff!
Related
I'm trying to save a large amount of Numpy arrays to a root file using uproot. I've read through the uproot documentation, and as far as I can tell this ability was remove in uproot3. Note I can't save this as a histogram because the order of the data is important (it's a waveform). Does anyone know of a work around, or thing I'm missing to do this?
e.g: I want something like this
'''
import numpy as np
import uproot as up
array = np.random.normal(0,1,1000)
file = up.recreate('./test.root')
file['Tree1'] = array
'''
I'm looking up in Numpy docs how to save an array, the provided this:
from tempfile import TemporaryFile
outfile = TemporaryFile()
np.save(outfile, array)
I tried doing it without the tempfile thing and it worked, so, I'm wondering what's the point of that?
I have .dat file that I want to use in my script which draws scatter graph with data input from that .dat file. I have been manually converting .dat files to .csv for this purpose but I find it not satisfactory.
This is what I am using currently.
import pandas as pd import matplotlib.pyplot as plt import numpy as np
filename=raw_input('Enter filename ')
csv = pd.read_csv(filename)
data=csv[['deformation','stress']]
data=data.astype(float)
x=data['deformation']
y=data['stress']
plt.scatter(x,y,s=0.5)
fit=np.polyfit(x,y,15)
p=np.poly1d(fit)
plt.plot(x,p(x),"r--")
plt.show()
Programmer friend told me that it would be more convenient to convert it to JSON and use it as such. How would I go about this?
try using the numpy read feature
import numpy as np
yourArray = np.fromfile('YourData.dat',dtype=dtype)
yourArray = np.loadtxt('YourData.dat')
loadtxt is more flexible than fromfile
I am trying to concatenate many numpy arrays, I put each array in one file, In fact the problem that I have a lot of files, Memory can't support to create a big array Data_Array = np.zeros((1000000,7000)), where I will put all my files. So, I found in this question Combining NumPy arrays that I can use np.concatenate:
file1= np.load('file1_Path.npy')
file2= np.load('file2_Path.npy')
file3= np.load('file3_Path.npy')
file4= np.load('file4_Path.npy')
dataArray=np.concatenate((file1, file2, file3, file4), axis=0)
test= dataArray.shape
print(test)
print (dataArray)
print (dataArray.shape)
plt.plot(dataArray.T)
plt.show()
This way gives me a very good result, but now, I need to replace file1, file2, file3, file4 by the path to the folder of my files:
import matplotlib.pyplot as plt
import numpy as np
import glob
import os, sys
fpath ="Path_To_Big_File"
npyfilespath =r'Path_To_Many_Numpy_Files'
os.chdir(npyfilespath)
npfiles= glob.glob("*.npy")
npfiles.sort()
for i,npfile in enumerate(npfiles):
dataArray=np.concatenate(npfile, axis=0)
np.save(fpath, all_arrays)
It gives me this error:
np.concatenate(npfile, axis=0)
ValueError: zero-dimensional arrays cannot be concatenated
Could you please help me to make this method np.concatenate works?
If you wish to use large arrays, just use np.memmap instead of loading the data into memory. The advantage of memmap is that data is always saved to disk when necessary. For example, you can create a memory mapped array in the following way:
import numpy as np
a=np.memmap('myFile',dtype=np.int,mode='w+',shape=(1000000,8000))
You can then use 'a' as a normal numpy array.
The limit is then your hard disk ! This creates a file on your hard disk that you can read later. You just change mode to 'r' and read data from the array.
More info about memmap here: https://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html
In order to fill that array from npy files of shape (1,8000), just write:
for i,npFile in enumerate(npfFiles):
a[i,:]=np.load(npFile)
a.flush()
The flush method insures everything has been written on disk
I have 166600 numpy files, I want to put them into one numpy file: file by file,
I mean that the creation of my new big file must from the begin: the first file must be read and written in the file, so the big file contains only the first file, after that I need to read and write the second file, so the big file contains the first two files.
import matplotlib.pyplot as plt
import numpy as np
import glob
import os, sys
fpath ="path_Of_my_final_Big_File"
npyfilespath ="path_of_my_numpy_files"
os.chdir(npyfilespath)
npfiles= glob.glob("*.npy")
npfiles.sort()
all_arrays = np.zeros((166601,8000))
for i,npfile in enumerate(npfiles):
all_arrays[i]=np.load(os.path.join(npyfilespath, npfile))
np.save(fpath, all_arrays)
If I understand your questions correctly, you can use numpy.concatenate for this:
import matplotlib.pyplot as plt
import numpy as np
import glob
import os, sys
fpath ="path_Of_my_final_Big_File"
npyfilespath ="path_of_my_numpy_files"
os.chdir(npyfilespath)
npfiles= glob.glob("*.npy")
npfiles.sort()
all_arrays = []
for i, npfile in enumerate(npfiles):
all_arrays.append(np.load(os.path.join(npyfilespath, npfile)))
np.save(fpath, np.concatenate(all_arrays))
Depending on the shape of your arrays and the intended concatenation, you might need to specify the axis parameter of concatenate.