Consider the following code:
import pandas as pd
import os
csv = """Country,Total Cases,New Cases,Total Deaths,New Deaths,Recovered,Serious
China,74187,1751,2006,138,14796,12017
Diamond Princess,621,79,,,17,20
Singapore,81,,,,29,4
Japan,80,6,1,,20,4
Hong Kong,63,1,2,1,5,6
S. Korea,51,20,,,16,
Thailand,35,,,,17,2
USA,29,,,,3,
Taiwan,23,1,1,,2,
Malaysia,22,,,,15,
Vietnam,16,,,,14,
Germany,16,,,,9,
Australia,15,,,,10,
France,12,,1,,7,
Macao,10,,,,5,
U.A.E.,9,,,,3,1
U.K.,9,,,,8,
Canada,8,,,,1,
Philippines,3,,1,,2,
Italy,3,,,,,2
India,3,,,,3,
Russia,2,,,,2,
Spain,2,,,,2,
Nepal,1,,,,1,
Belgium,1,,,,1,
Sri Lanka,1,,,,1,
Finland,1,,,,1,
Egypt,1,,,,,
Cambodia,1,,,,1,
Sweden,1,,,,,"""
with open("data/covid.csv", "w") as file:
file.write(csv)
df = pd.read_csv("data/covid.csv").fillna(0).astype(dtype = int, errors='ignore')\
.sort_values(by='Total Cases', ascending=False)
latest = df.to_dict('list')
some_var = latest.keys()
print(latest.keys()) # It's a notebook. Printing or calling a variable does not matter.
latest.keys()
Please see the attached image to see the issue
It works in a jupyter notebook on conda and if I run it via a normal python file. Which highlights that the issue lies with how VSCode displays notebook outputs
As you can see, VSCode does not display some outputs after running a cell, but does display others after running a cell.
If I try this in a conda environment, it works just fine:
Any suggestions for why this issue only happens in VSCode & How I might go about fixing it? Is it a kernel thing?
This issue was caused by an extension within vscode called Leaflet Map.
It changed the default renderer for notebook cells in vscode.
If anyone else has issues with VSCode outputs in future, you can click the elipsis on the output cell -> Click Change Presentation & ensure you're using an up to date renderer:
I am trying to run the python code using the R-Markdown file (RMarkdown to pdf).
What I achieved till now -
1- I am able to configure my python engine using knitr and reticulate library
2- I am able to execute my python codes.
What I tried -
1- I tried all the methods which are discussed in this forum, but nothing is working out.
2- I also tried to save the image,(as one of the posts here suggests), but that also is not working.
My problem -
1- When I am trying to plot a graph using matlplotlib and command plt.imshow() and plt.show(), it's not printing the image in the output. Rather it's showing the image in a separate window. You can see my results in the attached image.
Result_of_my_code
Here is my code
```{r setup, include=FALSE}
library(knitr)
library(reticulate)
knitr::knit_engines$set(python = reticulate::eng_python)
```
```{python}
import numpy as np
import os
import torch
import torchvision.datasets as dsets
import matplotlib.pyplot as plt
print(os.getcwd())
os.chdir('D:\\1st year\\Python codes\\CIFR Analysis\\self contained analysis')
print(os.getcwd())
train_mnist = dsets.MNIST("../data", train=True)
test_mnist = dsets.MNIST("../data", train= False)
print(len(train_mnist))
#print(train_mnist[0][0])
plt.imshow(train_mnist[0][0], cmap="gray")
#plt.savefig("trainzero.png")
plt.show()
```
Kindly, help me to fix this issue, as I want to compile my python codes using the R markdown file.
thanks
So with R Markdown, you have to do some things a little differently. In the following, I have a dataframe with two series created by concatenating them. The original plotting code in the Jupyter Notebook is as follows and just printed out the series.
# make a plot of model fit
train.plot(figsize=(16,8), legend=True)
backtest.plot(legend=True);
However, it does not work with way with R Markdown. Then with plotting, you always have to assign them, and with the code below, you get the same plot.
dfreg = pd.concat([reg, backtest], axis = 1)
ax = dfreg.plot(figsize=(16,8), legend = True)
ax1 = predictions.plot(legend=True)
plt.show()
This is common with other plotting functions like plot_acf() too.
I typically use MATLAB, but want to push myself to learn something about Python. I tried a code of linear regression that introduced by a youtuber. Here is the code:
import pandas as pd
from sklearn import linear_model
import matplotlib.pyplot as plt
#read data
dataframe = pd.read_fwf('brain_body.txt')
x_values = dataframe[['Brain']]
y_values = dataframe[['Body']]
#train model on data
body_reg = linear_model.LinearRegression()
body_reg.fit(x_values,y_values)
#visualize results
plt.scatter(x_values,y_values)
plt.plot(x_values,body_reg.predict(x_values))
plt.show()
But I ended up with a very strange plot (I use Python 3.6):
1
here is part of details:
2
Apparently, something is missing or wrong.
The data of brain_body.txt can be found in https://github.com/llSourcell/linear_regression_demo/blob/master/brain_body.txt
Any suggestion or advice is welcome.
Update
I tried sera's code, and here is what I get:
3
It's funny and weird. it occurred to me that something is wrong with my data file, or something missing in my Python, but I just copied and pasted the raw data into the notepad and saved as .txt; I tried Python 3.6 and 2.7 as well as Pycharm and Spyder...so I have no idea...
BTW, the youtube video is here
#sascha #Moritz #sera I asked my friend to run the same code and data file, and everything is fine. In other words, there is something wrong with my Python and I don't know why. Let me try another computer and/or try an earlier version of python.
I tried, but nothing changed. Here are two different approaches I used to install Python:
1. Install Python (e.g. ver. 3.6); install Pycharm; install packages Pandas, scikit-learn...
2. Install Anaconda
Solved
Thanks for #Marc Bataillou 's suggestion. This is a problem associated with different versions of matplotlib. The problem was found in version 2.1.0. I tried 2.0.2 and found that the original code works fine in the older version; apparently, some changes are made from 2.0.2 to 2.1.0. Thanks for all your efforts.
You should use
plt.scatter(x_values.values,y_values.values)
instead of
plt.scatter(x_values,y_values)
I hope it works !
You can visualize the results using the following code. I use cross validation for the predictions. If the model was perfect, then all the dots would be on the plotted line.
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import cross_val_predict
from sklearn import linear_model
#read data
dataframe = pd.read_fwf('brain_body.txt')
x_values = dataframe[['Brain']]
y_values = dataframe[['Body']]
#model on data
body_reg = linear_model.LinearRegression()
# cross_val_predict returns an array of the same size as `y` where each entry
# is a prediction obtained by cross validation:
predicted = cross_val_predict(body_reg, x_values, y_values, cv=10)
fig, ax = plt.subplots()
ax.scatter(y_values, predicted, edgecolors=(0, 0, 0))
ax.plot([y_values.min(), y_values.max()], [y_values.min(), y_values.max()], 'k--', lw=4)
ax.set_xlabel('Measured')
ax.set_ylabel('Predicted')
plt.show()
Results:
Data
https://ufile.io/p7x0r
I want to produce plots like this, except with many more particles. Matplotlib is woefully inadequate.
Right now I am using mayavi in python 3.5 running through a jupyter notebook. As I need to plot 5x10^5 spheres it will not be practical, since time is a limiting factor already at 2x10^4 spheres.
Here is my python code to produce the mayavi plot. I have a numpy array of values [a,r,x,y,z]. It's not relevant what the first quantity is for this problem.
"""VISUALIZATION WITH MAYAVI"""
#I think this is too slow to be practical.
#view particles with mayavi
import mayavi
from mayavi import mlab
%gui qt
def plot_sphere(p): #feed it p and get back a sphere \n",
t1,R,a,b,c = p
[phi,theta] = np.mgrid[0:2*np.pi:12j,0:np.pi:12j] #increase the numbers before j for better resolution but more time
x = R*np.cos(phi)*np.sin(theta) + a
y = R*np.sin(phi)*np.sin(theta) + b
z = R*np.cos(theta) + c
return mlab.mesh(x, y, z)
#run s over all particles and plot it
def view(particles):
for p in particles:
plot_sphere(p)
view(spheres)
This code produces plots like this:
I have been told I should look into writing my numpy arrays to .vtk files using evtk, then visualizing these in paraview. I downloaded paraview and read this, but perhaps my version of python is limiting me? First, install pyevtk-- okay:
I tried conda install -c opengeostat pyevtk=1.0.0, but it fails due to incompatibility with my python version. I looked for details but could not find any.
Next I downloaded the repository [here][https://pypi.python.org/pypi/PyEVTK/1.0.0], then used pip to install it successfully.
Next I put evtk.py, vtk.py, hl.py, and xml.py, and tried some examples in the repository-- none of them work. Seemingly there is some problem with
from .vtk import *
type commands. I tried replacing all of these in the four .py files with
from evtk import vtk
from vtk import *
and such, but no luck. Long story short, I can't get pyevtk working to export my numpy arrays as .vtk files. I could use some help in this regard, or better yet I would love a different option to get my numpy arrays rendered by paraview. Any help is appreciated !
Ok, I solved my own problem. This image is made using paraview, after converting numpy arrays to a .vtu object using pyevtk.
Out of the box, the repository did not work, there was some problem with importing the modules inside the four .py files, so I modified them all. Instead of from .vtk import *, I changed it to from vtk import *, and so on, in every module in the library. evtk.py was not able to import a class from xml.py, so I just copied it and pasted, then deleted xml.py. After some tinkering and clueless modifying to make the errors go away, eventually it worked.