Using PyCharm 2022.1.4 to develop Jupyter notebooks, I would like to see complete Pandas dataframes without vertical scrollbars. Using the following code, the entire dataframe can be seen if scrolled. I would rather have all the lines shown at once. Thank you
from IPython.core.display_functions import display
import numpy as np
with pd.option_context('display.max_rows', None, 'display.max_columns', None, "expand_frame_repr", False):
display(pd.DataFrame(np.random.randint(0,100,size=(15, 40))))
Related
I'm trying to create a hist plot in Power BI.
I got installed ANaconda, MS Vusial Code.
Screenshots with my settings:
I'm trying make hist with simple table with 1 column.
The following code to create a dataframe and remove duplicated rows is always executed and acts as a preamble for your script:
dataset = pandas.DataFrame(reg_min_ses_dt)
dataset = dataset.drop_duplicates()
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
sns.histplot(data=dataset['reg_min_ses_dt'])
plt.show()
But I get this error:
I think, I just didn't set up some python extension or something else.
I just want make Python visual like this.
You need to activate conda before you can use its packages in PBIDesktop.exe. Simple as that.
I only want certain cells and certain cell outputs to show up when I export my Juypter Notebook from VSCode. I have not been able to get an answer that works from Google, StackOverflow, and ChatGPT.
So when I export the .ipynb file to HTML in VSCode, how do I modify which cells are included in the HTML and which are not? For example, what would I do to include just the ouptut of the cell below and not the actual code?
import pandas as pd
import seaborn as sns
df = pd.read_csv(file.csv)
sns.histplot(df['Variable 1']
This post seems to indicate the best/only option is tagging cells then removing them with nbconvert. This seems inefficient in VSCode, especially compared to the easy output = FALSE or echo = FALSE in RStudio.
This seems like it should be an easy and common question but I am getting no good solutions from the internet. ChatGPT suggested include #hide-in-export to the cells I didn't want but that didn't work
The StackOverflow post I linked suggested using TagRemovePreprocessor with nbconvert and marking all the cells I want gone but that seems so clunky. Follow-up question: If tagging cells and removing them in export with nbconvert, what is the fastest way to tag cells in VSCode?
Although it is still a bit cumbersome, I think it is still a feasible method. Use F12 to open the web background, delete cells or output cells.
I still don't know if there is an easier way but here is what I have done with help from ChatGPT, this blog post, and this StackOverflow answer.
First, have a function that adds cell tags to the certain cells you want to hide:
import json
def add_cell_tag(nb_path, tag, cell_indices):
# Open the .ipynb file
with open(nb_path, 'r', encoding='utf-8') as f:
nb = json.load(f)
# Get the cells from the notebook
cells = nb['cells']
# Add the tag to the specified cells
for index in cell_indices:
cell = cells[index]
if 'metadata' not in cell:
cell['metadata'] = {}
if 'tags' not in cell['metadata']:
cell['metadata']['tags'] = []
cell['metadata']['tags'].append(tag)
# Save the modified notebook
with open(nb_path, 'w', encoding='utf-8') as f:
json.dump(nb, f)
Second, run the function and add a tag (can be any string) to the cells you want to hide in the HTML export:
add_cell_tag(nb_path, 'hide-code', [0, 1, 2])
Finally, use nbconvert in the terminal to export and filter the notebook:
jupyter nbconvert --to html --TagRemovePreprocessor.remove_cell_tags=hide-code path/to/notebook.ipynb
The cells made be entirely removed or just the output or just the input:
TagRemovePreprocessor.remove_input_tags
TagRemovePreprocessor.remove_single_output_tags
TagRemovePreprocessor.remove_all_outputs_tags
Not sure the difference between those last two. Additionally, I had a helper function to count the cells in the notebook and one to clear all tags in the notebook.
I am working on an app which will be able to show a graph of the company's performance in stocks, I wanted to turn the pandas plot of that company into an image without saving it. Can someone tell me what to do?
from fastquant import get_pse_data
import matplotlib.pyplot as plt
import pandas as pd
df = get_pse_data(symbol, '2019-01-01', '2020-01-01')
ma30 = df.close.rolling(30).mean()
close_ma30 = pd.concat([df.close, ma30], axis=1).dropna()
I am actually thinking of adding this plot derived from a pandas dataframe close_ma30 = pd.concat([df.close, ma30], axis=1).dropna() into my html code:
I want to create a python function that will allow me to return it as an image for a django code. Thank you for the help!
You can use Dataframe-image to convert a pandas plot into a image, you can Visit https://pypi.org/project/dataframe-image/.
dataframe_image has the ability to export both normal and styled DataFrames as images from within a Python script. Pass your normal or styled DataFrame to the export function along with a file location to save it as an image.
>>> import dataframe_image as dfi
>>> dfi.export(df_styled, 'df_styled.png')
You may also export directly from the DataFrame or styled DataFrame using the dfi.export and export_png methods, respectively.
>>> df.dfi.export('df.png')
>>> df_styled.export_png('df_styled.png)
As a Python Library
Dataframe_image can also be used outside of the notebook as a normal Python library. In a separate Python script, import the dataframe_image package and pass the file name of your notebook to the convert function.
>>> import dataframe_image as dfi
>>> dfi.convert('path/to/your_notebook.ipynb',
to='pdf',
use='latex',
center_df=True,
max_rows=30,
max_cols=10,
execute=False,
save_notebook=False,
limit=None,
document_name=None,
table_conversion='chrome'
chrome_path=None,
latex_command=None,
output_dir=None,
)
By default, the new file(s) will be saved in the same directory where the notebook resides. Do not run this command within the same notebook that is being converted.
From the Command Line
The command line tool dataframe_image will be available upon installation with the same options as the convert function from above.
dataframe_image --to=pdf "my notebook with dataframes.ipynb"
Finding Google Chrome
You must have Google Chrome (or Brave) installed in order for dataframe_image to work. The path to Chrome should automatically be found. If Chrome is not in a standard location, set it with the chrome_path parameter.
Using matplotlib instead of Chrome
If you do not have Chrome installed or cannot get it to work properly, you can alternatively use matplotlib to convert the DataFrames to images. Select this option by setting the table_conversion parameter to 'matplotlib'.
Publish to Medium
Closely related to this package is jupyter_to_medium, which publishes your notebooks directly and quickly as Medium blog posts.
Dependencies
You must have the following Python libraries installed.
A few days back, accidentally, I was able to plot technical indicators using these codes
ch.add_BBANDS()
ch.add_RSI(14)
But now I am not able to do so as I am having a problem installing quantmod.
I know we can plot candlestick chart by using plotly but with quantmod there seems to be a single line code which would be easier.
I realized that the problem is in Installing and importing quantmod as it says:
ImportError:
The plotly.plotly module is deprecated,
please install the chart-studio package and use the
chart_studio.plotly module instead.
I also tried to degrade my plotly version as mentioned here but nothing worked. I want to know if someone has been able to install and import quantmod in jupyter notebook and plot some of the charts as indicated by following codes. I would appreciate if you show me the correct way to get quantmod working in my jupyter notebook.
ch = qm.Chart(df)
ch.to_figure(type='ohlc', dimensions=(2560,1440))
ch = qm.Chart(df)
ch.add_BBANDS()
ch.add_RSI(14)
ch.to_figure(type='candlestick', title='EQUITY')
Try using QuantFig to get what you are trying to get. Chart studio now requires a sign in. QuantFig uses the index of the pandas dataframe as the x axis for the chart. So take care to format it. the columns shall be named 'open', 'high', 'low', close' as it is expected by QuantFig. you can use the following snippet for inspiration.
import chart_studio.plotly as py
import cufflinks as cf
import python as pd
py.sign_in('Python-Demo-Account', 'gwt101uhh0')
def plotData(df=None):
if df == None:
df=cf.datagen.ohlc()
qf=cf.QuantFig(df,title='First Quant Figure',legend='top',name='GS')
qf.add_bollinger_bands()
qf.add_rsi()
fig = qf.iplot(asFigure=True)
fig.show()
plotData()
I'm using tqdm combined with Pandas on Jupyter notebooks.
I have a Pandas dataframe df.
When i use df.progress_apply, new lines are printed instead of updating only one.
This is what I currently do :
tqdm.pandas(desc="Computing MONTH...")
df["MONTH"] = df.progress_apply(compute_month, axis=1)
My question is not a duplicate of this question : tqdm in Jupyter Notebook
because they said to use tqdm_notebook instead of tqdm.
I can't use tqdm_notebook as I need to implement df.progress_apply.
I can't reproduce this issue on a minimal example because my code is too heavy.
Here is an issue in Github related of this problem but couldn't help me : https://github.com/tqdm/tqdm/issues/375
You should do something like that:
from tqdm import tqdm_notebook
tqdm_notebook().pandas()
And that will do!