We Keep Coding

Trying to plot a random function but it seems that it's not quite random

I am trying to plot a random function from a textbook that looks like this:
What I did was to generate a random number between 1 to 20 for all numbers in a range.
import numpy as np
import random
import matplotlib.pyplot as plt
X = np.arange(1,20,0.2).tolist()
Random = [random.random() for x in X]
fig, axs = plt.subplots(1, 1)
axs.plot(X, Random,linestyle = 'dotted', color='r')
Then I plotted this But I got this:
Now I am wondering this is not quite random, We can see in some ranges that the function is increasing or decreasing, It looks like the values are not completely independent from each other.
My question is that how can I write a function that is completely random like the first picture.

I was able to do it well with the same code.
Why don't you try running it again?
I think you have to use scattar .
X = np.arange(1,20,0.2).tolist()
Random = [random.random() for x in X]
plt.scatter(X,Random)

The horizontal and vertical axes are the same in your first plot, the one you said you want to reproduce, leading me to believe that it is plotting pairs of random numbers against each other. (It actually looks like they are U(-6,6) values.) That's what the following code does:
import random
import matplotlib.pyplot as plt
randnums = [random.random() for _ in range(101)]
plt.scatter(randnums[0:100], randnums[1:101])
plt.show()
Results will vary from run to run, but look like this:
This plot indicates that adjacent values are equally balanced in all four quadrants of the unit square, which is a good indication that they are uncorrelated but nowhere near as good as doing rigorous testing.
It's worth mentioning that humans are notoriously bad at judging randomness. That's why the PRNGs used in most language libraries are thoroughly vetted by a large battery of statistical tests. See Wikipedia for details about the Diehard tests or the TestU01 suite. Python's random uses Mersenne Twister, which has been subjected to those tests and been deemed adequate for statistical (but not cryptographic) use. In other words, you don't need to do the testing, it's already been done.

How to manipulate figures while a script is running in Python?

Introduction
As I am coming from matlab, I am used to an interactive interface where a script can update figures while it is running. During the processing each figure can be re-sized or even closed. This probably means that each figure is running in its own thread which is obviously not the case with matplotlib.
IPython can imitate the Matlab behavior using the magic command %pylab or %matplotlib which does something that I don't understand yet and which is the very point of my question.
My goal is then to allow standalone Python scripts to work as Matlab does (or as IPython with %matplotlib does). In other words, I would like this script to be executed from the command line. I am expecting a new figure that pop-up every 3 seconds. During the execution I would be able to zoom, resize or even close the figure.
#!/usr/bin/python
import matplotlib.pyplot as plt
import time
def do_some_work():
time.sleep(3)
for i in range(10):
plt.plot([1,2,3,4])
plt.show() # this is way too boilerplate, I'd like to avoid it too.
do_some_work()
What alternative to %matplotlib I can use to manipulate figures while a script is running in Python (not IPython)?
What solutions I've already investigated?
I currently found 3 way to get a plot show.
1. %pylab / %matplotlib
As tom said, the use of %pylab should be avoided to prevent the namespace to be polluted.
>>> %pylab
>>> plot([1,2,3,4])
This solution is sweet, the plot is non-blocking, there is no need for an additionnal show(), I can still add a grid with grid() afterwards and I can close, resize or zoom on my figure with no additional issues.
Unfortunately the %matplotlib command is only available on IPython.
2. from pylab import * or from matplotlib.pyplot import plt
>>> from pylab import *
>>> plot([1,2,3,4])
Things are quite different here. I need to add the command show() to display my figure which is blocking. I cannot do anything but closing the figure to execute the next command such as grid() which will have no effect since the figure is now closed...
** 3. from pylab import * or from matplotlib.pyplot import plt + ion()**
Some suggestions recommend to use the ion() command as follow:
>>> from pylab import *
>>> ion()
>>> plot([1,2,3,4])
>>> draw()
>>> pause(0.0001)
Unfortunately, even if the plot shows, I cannot close the figure manually. I will need to execute close() on the terminal which is not very convenient. Moreover the need for two additional commands such as draw(); pause(0.0001) is not what I am expecting.
Summary
With %pylab, everything is wonderful, but I cannot use it outside of IPython
With from pylab import * followed by a plot, I get a blocking behavior and all the power of IPython is wasted.
from pylab import * followed by ion offers a nice alternative to the previous one, but I have to use the weird pause(0.0001) command that leads to a window that I cannot close manually (I know that the pause is not needed with some backends. I am using WxAgg which is the only one that works well on Cygwin x64.
This question advices to use matplotlib.interactive(True). Unfortunately it does not work and gives the same behavior as ion() does.

Change your do_some_work function to the following and it should work.
def do_some_work():
plt.pause(3)
For interactive backends plt.pause(3) starts the event loop for 3 seconds so that it can process your resize events. Note that the documentation says that it is an experimental function and that for complex animations you should use the animation module.
The, %pylab and %matplotlib magic commands also start an event loop, which is why user interaction with the plots is possible. Alternatively, you can start the event loop with %gui wx, and turn it off with %gui. You can use the IPython.lib.guisupport.is_event_loop_running_wx() function to test if it is running.
The reason for using ion() or ioff() is very well explained in the 'What is interactive mode' page. In principle, user interaction is possible without IPython. However, I could not get the interactive-example from that page to work with the Qt4Agg backend, only with the MacOSX backend (on my Mac). I didn't try with the WX backend.
Edit
I did manage to get the interactive-example to work with the Qt4Agg backend by using PyQt4 instead of PySide (so by setting backend.qt4 : PyQt4 in my ~/.config/matplotlibrc file). I think the example doesn't work with all backends. I submitted an issue here.
Edit 2
I'm afraid I can't think of a way of manipulating the figure while a long calculation is running, without using threads. As you mentioned: Matplotlib doesn't start a thread, and neither does IPython. The %pylab and %matplotlib commands alternate between processing commands from the read-eval-print loop and letting the GUI processing events for a short time. They do this sequentially.
In fact, I'm unable to reproduce your behavior, even with the %matplotlib or %pylab magic. (Just to be clear: in ipython I first call %matplotlib and then %run yourscript.py). The %matplotlib magic puts Matplotlib in interactive-mode, which makes the plt.show() call non-blocking so that the do_some_work function is executed immediately. However, during the time.sleep(3) call, the figure is unresponsive (this becomes even more apparent if I increase the sleeping period). I don't understand how this can work at your end.
Unless I'm wrong you'll have to break up your calculation in smaller parts and use plt.pause (or even better, the animation module) to update the figures.

My advice would be to keep using IPython, since it manages the GUI event loop for you (that's what pylab/pylot does).
I tried interactive plotting in a normal interpreter and it worked the way it is expected, even without calling ion() (Debian unstable, Python 3.4.3+, Matplotlib 1.4.2-3.1). If I recall it right, it's a fairly new feature in Matplotlib.
Alternatively, you can also use Matplotlib's animation capabilities to update a plot periodically:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import time
plt.ion()
tt = np.linspace(0, 1, 200)
freq = 1 # parameter for sine
t0 = time.time() # for measuring ellapsed time
fig, ax = plt.subplots()
def draw_func(i):
""" This function gets called repeated times """
global freq # needed because freq is changed in this function
xx = np.sin(2*np.pi*freq*tt)/freq
ax.set_title("Passed Time: %.1f s, " % (time.time()-t0) +
"Parameter i=%d" % i)
ax.plot(tt, xx, label="$f=%d$ Hz" % freq)
ax.legend()
freq += 1
# call draw_func every 3 seconds 1 + 4 times (first time is initialization):
ani = animation.FuncAnimation(fig, draw_func, np.arange(4), interval=3000,
repeat=False)
# plt.show()
Checkout matplotlib.animation.FuncAnimation for details. You'll find further examples in the examples section.

Creating movie from a series of matplotlib plots using matplotlib.animation

I have a script which generates a series of time dependent plots. I'd like to "stitch" these together to make a movie.
I would preferably like to use matplotlib.animation. I have looked at examples from matplotlib documentation but I can't understand how it works.
My script currently makes 20 plots at successive time values and saves these as 00001.png up to 000020.png:
from scipy.integrate import odeint
from numpy import *
from math import cos
import pylab
omega=1.4
delta=0.1
F=0.35
def f(initial,t):
x,v=initial
xdot=v
vdot=x-x**3-delta*v-F*cos(omega*t)
return array([xdot,vdot])
T=2*pi/omega
nperiods = 100
totalsteps= 1000
small=int((totalsteps)/nperiods)
ntransients= 10
initial=[-1,0]
kArray= linspace(0,1,20)
for g in range (0,20):
k=kArray[g]
x,v=initial
xpc=[]
vpc=[]
if k==0:
x,v=x,v
else:
for i in range(1,nperiods)
x,v=odeint(f,[x,v],linspace(0,k*T,small))[-1] )
for i in range (1,nperiods):
x,v=odeint(f,[x,v],linspace(k*T,T+k*T,small))[-1]
xpc.append(x)
vpc.append(v)
xpc=xpc[ntransients:]
vpc=vpc[ntransients:]
pylab.figure(17.8,10)
pylab.scatter(xpc,vpc,color='red',s=0.2)
pylab.ylim([-1.5,1.5])
pylab.xlim([-2,2])
pylab.savefig('0000{0}.png'.format(g), dpi=200)
I'd appreciate any help. Thank you.

I think matplotlib.animation.FuncAnimation is what you're looking for. Basically, it repeatedly calls a defined function, passing in (optional) arguments as needed. This is exactly what you're already doing in your for g in range(0,20): code. You can also define an init function to get things set up. Check out the base class matplotlib.animation.Animation for more info on formats, saving, the MovieWriter class, etc.

more efficient Python scripting in Blender3D

I am basically building a 3D scatter plot using primitive UV spheres and am running into memory issues when attempting to create more than a couple hundred points at one time. I am limited on my laptop with a 2.1Ghz processor but wanted to know if there is a better way to write this:
import bpy
import random
while count < 5:
bpy.ops.mesh.primitive_uv_sphere_add(size=.3,\
location=(random.randint(-9,9), random.randint(-9,9),\
random.randint(-9,9)), rotation=(0,0,0))
count += 1
I realize that with such a simple script any performance increase is likely negligible but wanted to give it a shot anyway.

Some possible suggestions
I would pre-calculate the x,y,z values, store them in a mathutil vector and add it to a dict to be iterated over.
Duplication should provide a smaller memory footprint than
instantiating new objects. bpy.ops.object.duplicate_move(OBJECT_OT_duplicate=(linked:false, TRANSFORM_OT_translate=(transform)
Edit:
Doing further research it appears each time a bpy.ops.* is called the redraw function . One user documentented exponential increase in time taken to genenerate UV sphere.
CoDEmanX provided the following code snippet to another user.
import bpy
bpy.ops.object.select_all(action='DESELECT')
bpy.ops.mesh.primitive_uv_sphere_add()
sphere = bpy.context.object
for i in range(-1000, 1000, 2):
ob = sphere.copy()
ob.location.y = i
#ob.data = sphere.data.copy() # uncomment this, if you want full copies and no linked duplicates
bpy.context.scene.objects.link(ob)
bpy.context.scene.update()
Then it is just a case of adapting the code to set the object locations
obj.location = location_dict[i]

How can I release memory after creating matplotlib figures

I have several matlpotlib functions rolled into some django-celery tasks.
Every time the tasks are called more RAM is dedicated to python. Before too long, python is taking up all of the RAM.
QUESTION: How can I release this memory?
UPDATE 2 - A Second Solution:
I asked a similar question specifically about the memory locked up when matplotlib errors, but I got a good answer to this question .clf(), .close(), and gc.collect() aren't needed if you use multiprocess to run the plotting function in a separate process whose memory will automatically be freed once the process ends.
Matplotlib errors result in a memory leak. How can I free up that memory?
UPDATE - The Solution:
These stackoverflow posts suggested that I can release the memory used by matplotlib objects with the following commands:
.clf(): Matplotlib runs out of memory when plotting in a loop
.close(): Python matplotlib: memory not being released when specifying figure size
import gc
gc.collect()
Here is the example I used to test the solution:
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from pylab import import figure, savefig
import numpy as np
import gc
a = np.arange(1000000)
b = np.random.randn(1000000)
fig = plt.figure(num=1, dpi=100, facecolor='w', edgecolor='w')
fig.set_size_inches(10,7)
ax = fig.add_subplot(111)
ax.plot(a, b)
fig.clf()
plt.close()
del a, b
gc.collect()

Did you try to run you task function several times (in a for) to be sure that not your function is leaking no matter of celery?
Make sure that django.settings.DEBUG is set False( The connection object holds all queries in memmory when DEBUG=True).

import matplotlib.pyplot as plt
from datetime import datetime
import gc
class MyClass:
def plotmanytimesandsave(self):
plt.plot([1, 2, 3])
ro2 = datetime.now()
f =ro2.second
name =str(f)+".jpg"
plt.savefig(name)
plt.draw()
plt.clf()
plt.close("all")
for y in range(1, 10):
k = MyClass()
k.plotmanytimesandsave()
del k
k = "now our class object is a string"
print(k)
del k
gc.collect
with this program you will save directly as many times you want without the plt.show() command. And the memory consumption will be low.