I've created a program that retrieves data from a device on the serial port every half second or so. It then appends that data to the array that sets the data points and then updates the plot. Everything goes fine until it's been running for an hour or so, at which point the program stops responding.
Does anyone know if there is a size limit for this array? If anyone has any ideas on handling a data set that could be millions of points, I would love to hear your thoughts.
Using the code below I was able to get matplotlib to show a simple graph of ten million points. I suspect the problem isn't with the array size.
import numpy as np
import matplotlib.pyplot as plt
import random
nsteps = 10000000
draws = np.random.randint(0,2,size=nsteps)
steps = np.where(draws>0,1,-1)
walk = steps.cumsum()
plt.plot(np.arange(nsteps), np.array(walk), 'r-')
plt.title("Big Set Random Walk with $\pm1$ steps")
plt.show()
There seems to be a some limit. I just tried
import pylab
import numpy as np
n = 10000000 # my code works fine for n = 1000000
x = np.random.normal(0,1,n)
pylab.plot(x)
pylab.show()
And got the following error:
OverflowError: Agg rendering complexity exceeded. Consider downsampling or decimating your data.
Related
I would like to voxelise a .stl file and write it into an np.array. The resolution of the voxels should be adjustable.
Here is my code for this:
component_path = r"C:\Users\User\documents\components\Test_1.stl"
mesh = o3d.io.read_triangle_mesh(component_path)
voxel_grid = o3d.geometry.VoxelGrid.create_from_triangle_mesh(mesh, voxel_size = 3)
ply_path = "voxel.ply"
o3d.io.write_voxel_grid(ply_path, voxel_grid, True,True,True)
pcd = o3d.io.read_point_cloud(ply_path)
list_path = "list.xyz"
o3d.io.write_point_cloud(list_path, pcd)
Then I read the coordinate points from the list, write them into a 3D array and plot them. When plotting, the border is not displayed for certain voxel sizes, as can be seen in the image (although it is present in the original). Is there a solution for this so that it doesn't happen no matter what voxel size?
voxelized picture with missing border
In addition, the voxel size changes the maximum dimension. So the component originally has three times the length as it is shown here. How can this be adjusted? (If I just multiply a factor, the voxels stay small but pull the distances apart).
Is there perhaps a more reasonable way to write a voxelisation of a .stl file and put the centers of voxels into an np.array?
If anyone ever has the same problem and is looking for a solution:
This project worked for me: GitHub: stl-to-voxel
The model is then also filled. If the maximum dimension is known, you can determine the exact voxel size via the resolution.
Here is some code:
import stl_reader
import stltovoxel
import numpy as np
import copy
import os
import sys
input=r"C:\Users\user\Example.stl"
output=r"C:\Users\user\Test.xyz"
resolution = 50 #Resolution, into how many layers the model should be divided
stltovoxel.doExport(input, output, resolution)
Afterwards, you can read the coordinates from the list, write them into an array and process them further (quite normally).
So the following is a snippet of what I'm working on where I'm working on and it's meant to find the peaks from the data. I'm trying to add user choice into the mix and for each peak whether it's good or not. I want to have the figure constantly change throughout the script and the input will save the peak value for later use. But the figure simply doesn't respond and I can't seem to find why. Any help would be much appreciated!
import os
import numpy as np
import datetime
import matplotlib.pyplot as plt
for moving in range(len(indexes)):
parsedFile = 'TEST/parsedWVIAdata_%d-%d_%d' % (DT[1,0],int(DT[1,5]),moving + 1)
plt.figure(1)
plt.plot(HD,H2O)
plt.plot(HD[indexes[moving]],H2O[indexes[moving]],"rx")
plt.axis([HD[indexes[moving]]-xrang,HD[indexes[moving]]+xrang,H2O[indexes[moving]]-yrang,H2O[indexes[moving]]+yrang])
plt.show()
select = False
rawrchoice = raw_input("Is this point a peak (y/n): ")
if rawrchoice=='y': select = True
peaks = 4
plt.savefig(parsedFile+'.jpg',pdi=1000)
plt.clf()
plt.close()
Picture of figure not responding - http://i.imgur.com/I6YAtYF.png
I am trying to plot a simple 'merc' map with boundary from shape file. The total size of the shape file ne_10m_admin_0_countries_lakes.shp is just 8mb. The simple mslp surface plot from GFS data took about more than 28 sec which I think is too much. After investigation I found that it is the reading of shape file consuming around 10 sec. A simple code to demonstrate the issue is shown below:-
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
south = 0
north = 5
west = 70
east = 85
m = Basemap(projection='merc', llcrnrlat=south, urcrnrlat=north,
llcrnrlon=west, urcrnrlon=east, resolution='c')
m.readshapefile('data/gis-data/world_countries/'
'ne_10m_admin_0_countries_lakes', 'ne_10m_admin_0_countries_lakes', linewidth=0.7)
plt.savefig('map.png')
To show the problem I am facing, I have run the above code with and without commenting the path to shape file. Here is time the script took after reading from shape file:-
$ time python test.py
real 0m18.234s
user 0m17.832s
sys 0m1.020s
Here is the result without reading from shape file:-
$ time python test.py
real 0m2.506s
user 0m2.360s
sys 0m0.324s
Is there any way to read the shape file quickly? Is there any solution/trick for this issue?
I would put this question into Geographic Information Systems too, there are some people to help you. If basemap is slow, did you try cartopy, i saw this on the matplotlib site, see http://matplotlib.org/1.4.1/mpl_toolkits/index.html
I've been trying to get some data to display in a matplotlib graph and I'm having an issue that seems fairly unexpected. I was originally trying to plot a large number of data points (~500000) and was getting the
OverflowError: Agg rendering complexity exceeded. Consider downsampling or decimating your data.
So, I did just that. I decimated my data using both the signal.decimate function and using slice notation. None of these solved my issue, I still get the complexity exceeded error even when trying to plot only 60 data points. I've attempted to determine if my computer my have some bad settings but I am fully capable of plotting 500000 points in a straight line without a hiccup. I'll add some example code and maybe someone can help me spot the error of my ways.
import scikits.audiolab as audiolab
if __name__ == "__main__":
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import freqz
sound = audiolab.sndfile('exampleFile.wav', 'read')
sound_info = sound.read_frames(sound.get_nframes())
sound.close()
nsamples = sound_info.size
t = np.linspace(0, 5, nsamples, endpoint=False)
plt.figure()
plt.plot(t, sound_info, label='Filtered signal (600 Hz)')
plt.show()
i am rather new in python and before was a MATLAB user. I am sorry if my question is to obvious.
I have a huge 47MB file that contains 3D array (351:467:300). That is 300 images from a camera. I want to re-plot them as a kind of animation. Basically, just slice and plot all 300 images. Here is my code
import numpy as np
import time
import matplotlib.pyplot as plt
import scipy.io as c
datafile = c.loadmat('data.mat') # loading data
img = datafile['img'] # extracting the (351:467:300) array
imgShape = np.shape(img)
for i in range(0,imgShape(2)):
plt.imshow(img[:,:,i])
time.sleep(0.3)
plt.draw()
print('Done!')
the problem is: when it comes to imshow the figure window is black and NOT RESPONDING until it finishes the loop, so i can't see anything. How to solve this? How to force it to update "on stream" ?
And the plotting is veeeryyy slow compared to matlab (not because of time.sleep :) i tried without :) ) loop runs really slowly. I am using Spyder, is that a reason ?
Thanks a lot in advance !
Use the following corrections:
you need to make it interactive.
import numpy as np
#import time
#import matplotlib.pyplot as plt
import scipy.io as c
import pylab as pl #added
datafile = c.loadmat('data.mat') # loading data
img = datafile['img'] # extracting the (351:467:300) array
imgShape = np.shape(img)
pl.ion() #added
for i in range(imgShape(2)): #no need for 0
pl.cla() #added
pl.imshow(img[:,:,i])
# time.sleep(0.3)
pl.draw()
pl.pause(0.3) #added
pl.ioff() #added
print('Done!')