We Keep Coding

Trying to plot a random function but it seems that it's not quite random

I am trying to plot a random function from a textbook that looks like this:
What I did was to generate a random number between 1 to 20 for all numbers in a range.
import numpy as np
import random
import matplotlib.pyplot as plt
X = np.arange(1,20,0.2).tolist()
Random = [random.random() for x in X]
fig, axs = plt.subplots(1, 1)
axs.plot(X, Random,linestyle = 'dotted', color='r')
Then I plotted this But I got this:
Now I am wondering this is not quite random, We can see in some ranges that the function is increasing or decreasing, It looks like the values are not completely independent from each other.
My question is that how can I write a function that is completely random like the first picture.

I was able to do it well with the same code.
Why don't you try running it again?
I think you have to use scattar .
X = np.arange(1,20,0.2).tolist()
Random = [random.random() for x in X]
plt.scatter(X,Random)

The horizontal and vertical axes are the same in your first plot, the one you said you want to reproduce, leading me to believe that it is plotting pairs of random numbers against each other. (It actually looks like they are U(-6,6) values.) That's what the following code does:
import random
import matplotlib.pyplot as plt
randnums = [random.random() for _ in range(101)]
plt.scatter(randnums[0:100], randnums[1:101])
plt.show()
Results will vary from run to run, but look like this:
This plot indicates that adjacent values are equally balanced in all four quadrants of the unit square, which is a good indication that they are uncorrelated but nowhere near as good as doing rigorous testing.
It's worth mentioning that humans are notoriously bad at judging randomness. That's why the PRNGs used in most language libraries are thoroughly vetted by a large battery of statistical tests. See Wikipedia for details about the Diehard tests or the TestU01 suite. Python's random uses Mersenne Twister, which has been subjected to those tests and been deemed adequate for statistical (but not cryptographic) use. In other words, you don't need to do the testing, it's already been done.

When I run '''sns.histplot(df['price'])''' in pycharm I get the code output but no graph, why is this?

I'm using pycharm to run some code using Seaborn. I'm very new to python and am just trying to learn the ropes so I'm following a tutorial online. I've imported the necessary libraries and have run the below code
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
# import the data saved as a csv
df = pd.read_csv('summer-products-with-rating-and-performance_2020-08.csv')
df["has_urgency_banner"] = df["has_urgency_banner"].fillna(0)
df["discount"] = (df["retail_price"] -
df["price"])/df["retail_price"]
df["rating_five_percent"] = df["rating_five_count"]/df["rating_count"]
df["rating_four_percent"] = df["rating_four_count"]/df["rating_count"]
df["rating_three_percent"] = df["rating_three_count"]/df["rating_count"]
df["rating_two_percent"] = df["rating_two_count"]/df["rating_count"]
df["rating_one_percent"] = df["rating_one_count"]/df["rating_count"]
ratings = [
"rating_five_percent",
"rating_four_percent",
"rating_three_percent",
"rating_two_percent",
"rating_one_percent"
]
for rating in ratings:
df[rating] = df[rating].apply(lambda x: x if x>= 0 and x<= 1 else 0)
# Distribution plot on price
sns.histplot(df['price'])
My output is as follows:
Process finished with exit code 0
so I know there are no errors in the code but I don't see any graphs anywhere as I'm supposed to.
Ive found a way around this by using this at the end
plt.show()
which opens a new tab and uses matplotlib to show me a similar graph.
However in the code I'm using to follow along, matplotlib is not imported or used (I understand that seaborn has built in Matplotlib functionality) as in the plt.show statement is not used but the a visual graph is still achieved.
I've also used print which gives me the following
AxesSubplot(0.125,0.11;0.775x0.77)
Last point to mention is that the code im following along with uses the following
import seaborn as sns
# Distribution plot on price
sns.distplot(df['price'])
but distplot has now depreciated and I've now used histplot because I think that's the best alternative vs using displot, If that's incorrect please let me know.
I feel there is a simple solution as to why I'm not seeing a graph but I'm not sure if it's to do with pycharm or due to something within the code.

matplotlib is a dependency of seaborn. As such, importing matplotlib with import matplotlib.pyplot as plt and calling plt.show() does not add any overhead to your code.
While it is annoying that there is no sns.plt.show() at this time (see this similar question for discussion), I think this is the simplest solution to force plots to show when using PyCharm Community.
Importing matplotlib in this way will not affect how your exercises run as long as you use a namespace like plt.

Be aware the 'data' must be pandas DataFrame object, not: <class 'pandas.core.series.Series'>
I using this, work finely:
# Distribution plot on price
sns.histplot(df[['price']])
plt.show()

How do I use the detrending function for matplotlib.pyplot.acorr?

I am a Python beginner. I am trying to detrend a time-series before running an autocorrelation analysis by using acorr in matplotlib. But there is something about the syntax that I fail understand.
Matplotlib's website (https://matplotlib.org/3.1.0/api/_as_gen/matplotlib.pyplot.acorr.html) describes how to use detrending with the acorr function: "x is detrended by the detrend callable. This must be a function x = detrend(x) accepting and returning an numpy.array." I must be reading this wrong, because the code I use does not work.
Failed attempts:
plt.acorr(values, detrend=True)
plt.acorr(values, detrend="linear")
plt.acorr(values=detrend(values))
As you can see, some rudimentary fact about syntax or matplotlib escapes me. Please help.

In matplotlib.mlab you find functions which you can use for detrending. An example:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import mlab
wn = np.random.normal(size=10**3)
plt.figure()
plt.acorr(np.abs(wn), maxlags=200, detrend=mlab.detrend_none) #default detrend
plt.figure()
plt.acorr(np.abs(wn), maxlags=200, detrend=mlab.detrend) #subtract sample mean

How to avoid running out of memory in Python plotting?

I need to plot a great bunch of different objects (~10^5 filled ellipses and similar shapes). What I do is add them one at a time using the command plt.gcf().gca().add_artist(e) and then use plt.show() at the end. This requires more memory than what I have.
Is there a way to plot them one at a time (that is, without adding them as I did above), and thus reduce the amount of memory I consume? I would be fine even with a solution that significantly increases the amount of time required for the plotting.

To draw a large quantity of similar objects you have to use one of the different matplotlib.collections classes — alas, their usage is a bit arcane, at least when it is my understanding that is involved...
Anyway, starting from the docs and this official example
I was able to put together the following code
$ cat ellipses.py
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import EllipseCollection
N = 10**5
# centres of ellipses — uniform distribution, -5<=x<5, -3<=y<3
xy = np.random.random((N,2))*np.array((5*2,3*2))-np.array((5,3))
# width, height of ellipses
w, h = np.random.random(N)/10, np.random.random(N)/10
# rotation angles, anticlockwise
a = np.random.random(N)*180-90
# we need an axes object for the correct scaling of the ellipses
fig, ax = plt.subplots()
# create the collection
ec = EllipseCollection(w, h, a,
units='x',
offsets=xy,
transOffset=ax.transData)
ax.add_collection(ec)
ax.autoscale(tight=True)
plt.savefig('el10^5.png')
I timed it on my almost low-end notebook
$ time python -c 'import numpy; import matplotlib.pyplot as p; f, a = p.subplots()'
real 0m0.697s
user 0m0.620s
sys 0m0.072s
$ time python ellipses.py
real 0m5.704s
user 0m5.616s
sys 0m0.080s
$
As you can see, when you discount the staging required for every plot, it takes about
5 seconds — and what is the result?
I think that the details about eccentricity and angle are lost in such a dense representation, but I don't know the specifics of your task and won't comment further.

How to update matplotlib plot in while loop? [duplicate]

what I am trying to do is having a script compute something, prepare a plot and show the already obtained results as a pylab.figure - in python 2 (specifically python 2.7) with a stable matplotlib (which is 1.1.1).
In python 3 (python 3.2.3 with a matplotlib git build ... version 1.2.x), this works fine. As a simple example (simulating a lengthy computation by time.sleep()) consider
import pylab
import time
import random
dat=[0,1]
pylab.plot(dat)
pylab.ion()
pylab.draw()
for i in range (18):
dat.append(random.uniform(0,1))
pylab.plot(dat)
pylab.draw()
time.sleep(1)
In python 2 (version 2.7.3 vith matplotlib 1.1.1), the code runs cleanly without errors but does not show the figure. A little trial and error with the python2 interpreter seemed to suggest to replace pylab.draw() with pylab.show(); doing this once is apparently sufficient (not, as with draw calling it after every change/addition to the plot). Hence:
import pylab
import time
import random
dat=[0,1]
pylab.plot(dat)
pylab.ion()
pylab.show()
for i in range (18):
dat.append(random.uniform(0,1))
pylab.plot(dat)
#pylab.draw()
time.sleep(1)
However, this doesn't work either. Again, it runs cleanly but does not show the figure. It seems to do so only when waiting for user input. It is not clear to me why this is, but the plot is finally shown when a raw_input() is added to the loop
import pylab
import time
import random
dat=[0,1]
pylab.plot(dat)
pylab.ion()
pylab.show()
for i in range (18):
dat.append(random.uniform(0,1))
pylab.plot(dat)
#pylab.draw()
time.sleep(1)
raw_input()
With this, the script will of course wait for user input while showing the plot and will not continue computing the data before the user hits enter. This was, of course, not the intention.
This may be caused by different versions of matplotlib (1.1.1 and 1.2.x) or by different python versions (2.7.3 and 3.2.3).
Is there any way to accomplish with python 2 with a stable (1.1.1) matplotlib what the above script (the first one) does in python 3, matplotlib 1.2.x:
- computing data (which takes a while, in the above example simulated by time.sleep()) in a loop or iterated function and
- (while still computing) showing what has already been computed in previous iterations
- and not bothering the user to continually hit enter for the computation to continue
Thanks; I'd appreciate any help...

You want the pause function to give the gui framework a chance to re-draw the screen:
import pylab
import time
import random
import matplotlib.pyplot as plt
dat=[0,1]
fig = plt.figure()
ax = fig.add_subplot(111)
Ln, = ax.plot(dat)
ax.set_xlim([0,20])
plt.ion()
plt.show()
for i in range (18):
dat.append(random.uniform(0,1))
Ln.set_ydata(dat)
Ln.set_xdata(range(len(dat)))
plt.pause(1)
print 'done with loop'
You don't need to create a new Line2D object every pass through, you can just update the data in the existing one.
Documentation:
pause(interval)
Pause for *interval* seconds.
If there is an active figure it will be updated and displayed,
and the gui event loop will run during the pause.
If there is no active figure, or if a non-interactive backend
is in use, this executes time.sleep(interval).
This can be used for crude animation. For more complex
animation, see :mod:`matplotlib.animation`.
This function is experimental; its behavior may be changed
or extended in a future release.
A really over-kill method to is to use the matplotlib.animate module. On the flip side, it gives you a nice way to save the data if you want (ripped from my answer to Python- 1 second plots continous presentation).
example, api, tutorial

Some backends (in my experience "Qt4Agg") require the pause function, as #tcaswell suggested.
Other backends (in my experience "TkAgg") seem to just update on draw() without requiring a pause. So another solution is to switch your backend, for example with matplotlib.use('TkAgg').