I currently have a script which reads a Google Sheet URL, and converts to a dataframe. To access this file, I need to authenticate each time I run the script. Is there an alternative which means I only have to do this once?
from google.colab import auth
auth.authenticate_user()
import gspread
from oauth2client.client import GoogleCredentials
gc = gspread.authorize(GoogleCredentials.get_application_default())
import pandas as pd
import matplotlib.pyplot as plt
wb = gc.open_by_url('...')
sheet = wb.worksheet('Sheet1')
data = sheet.get_all_values()
df = pd.DataFrame(data)
Thanks!
Related
I had one more problem with my app on stremalit. I am using PyDrive library to connect with google drive where I am uploading photos for educational purposes. The problem is that every week the app is throwing authentication error. The app is on the streamlit cloud. In my github directory I have clients_secrets.json file, creds.json file and also settings.yaml file.
Here is my code:
import streamlit as st
import pandas as pd
import requests
import sqlalchemy as db
#import matplotlib.pyplot as plt
import leafmap.foliumap as leafmap
from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import text
from plotly import graph_objects as go
from plotly.subplots import make_subplots
import branca
#import os
from pathlib import Path
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
import streamlit as st
#import logging
import numpy as np
from PIL import Image
#logging.getLogger('googleapicliet.discovery_cache').setLevel(logging.ERROR)
gauth = GoogleAuth()
gauth.CommandLineAuth()
drive = GoogleDrive(gauth)
What I am doing at the moment is changing creds.json file the moment my credentials are expired. But I wish that I won't need to do it all the time. Any idea how can I make authentication process automatic without the need to change the creds.json file every week?
I tried to change creds.json file every week
I just started using Google Colab a few horus ago and I'm trying to figure our how to read,write and save stuff etc.
I have this code on Jupyter notebook,and I'm having trouble at the last part where I save the file, I want to save it either on my local computer or Google Drive?
import pandas as pd
pd.set_option('display.max_columns', 999)
#load data
df = pd.read_csv('D:\\Project\\database\\Isolation Forest\\IF 15 PERCENT.csv')
df.shape
#data info
info = df.info()
print(info)
#data description
describe = df.describe() #print(describe)
f = open('D:\\Project\\database\\Isolation Forest\\Final Description IF TEST11.txt', "w+")
print(describe, file=f)
f.close()
and
Google Colab Code:
import pandas as pd
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
# Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
link = '......'
fluff, id = link.split('=')
print (id) # Verify that you have everything after '='
downloaded = drive.CreateFile({'id':id})
downloaded.GetContentFile('IF 15 PERCENT.csv')
df = pd.read_csv('IF 15 PERCENT.csv',index_col=None)
info = df.info()
print(info)
describe = df.describe()
I don't really know how to save it now as txt file and w+
Thank you.
This will save your dataframe in text format:
tfile = open('test.txt', 'w+')
tfile.write(describe.to_string())
tfile.close()
While uploading CSV file to Google drive, it automatically converting to Google Sheets. How to save it as CSV file in drive? or can I read google sheet through pandas data frame ?
Develop environment: Google Colab
Code Snippet:
Input
data = pd.read_csv("ner_dataset.desktop (3dec943a)",
encoding="latin1").fillna(method="ffill")
data.tail(10)
Output
[Desktop Entry]
0 Type=Link
1 Name=ner_dataset
2 URL=https://docs.google.com/spreadsheets/d/1w0...
WORKING CODE
from google.colab import auth
auth.authenticate_user()
import gspread
from oauth2client.client import GoogleCredentials
gc = gspread.authorize(GoogleCredentials.get_application_default())
worksheet = gc.open('Your spreadsheet name').sheet1
# get_all_values gives a list of rows.
rows = worksheet.get_all_values()
print(rows)
# Convert to a DataFrame and render.
import pandas as pd
pd.DataFrame.from_records(rows)
#Mount the Drive
from google.colab import drive
drive.mount('drive')
#Authenticate you need to do with your credentials, fill yourself
gauth = GoogleAuth()
#Create CSV and Copy
df.to_csv('data.csv')
!cp data.csv drive/'your drive'
I need to extract an image from a Google Sheets spreadsheet. It is not connected to a cell, and does not have an URL--it's just an inserted image:
How can I extract it using Python?
Thank you very much for an answer.
here is a part of a code:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import urllib
import numpy as numpy
scopes = ['spreadsheets.google.com/feeds']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json',scopes)
client = gspread.authorize(creds)
spreadsheet = client.open_by_url("docs.google.com/spreadsheets/d/…)
worksheet = spreadsheet.worksheet("Ark1")
list_of_lists = worksheet.range('A1:Q31')
print('{}\n'.format(list_of_lists))
Struggling to get xlxswriter to create multiple worksheets and use the pandas df.to_excel method to place the data in there.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import xlsxwriter
import os as os
import sqlite3 as sql
import xlrd
path = (r'C:\Users\test1\Downloads\category.xlsx') #folder
data = pd.DataFrame()
data = pd.read_excel(path,sheetname='1')
print (data.shape)
data.head()
#create excel workbook using xlsxwriter
#workbook = xlsxwriter.Workbook('hyperlink.xlsx')
writer = ExcelWriter('hyperlink.xlsx')
data.to_excel(writer,'Sheet1')
workbook.close()