I have been trying to figure this out for a while now and just dont seem to be able to break through so hopefully someone out there has done this before.
My issue is that I am trying to do a batch update of a google spreadsheet using the gdata python client libraries and authenticating via oauth2. I have found an example of how to do the batch update using the gdata.spreadsheet.service module here: https://code.google.com/p/gdata-python-client/wiki/UsingBatchOperations
However that does not seem to work when authenticating via oauth2 and so I am having to use the gdata.spreadsheets.client module instead as discussed in this post: https://code.google.com/p/gdata-python-client/issues/detail?id=549
Using the gdata.spreadsheets.client module works for authentication and for updating the sheet however batch commands does not seem to work. Below is my latest variation of the code which is about the closest I have got. It seems to work but the sheet is not updated and the batch_status returned is: 'Insert not supported on batch.' (Note: I did try modifying the batch_operation and batch_id parameters of the CellEntries in the commented out code but this did not work either.)
Thanks for any help you can provide.
import gdata
import gdata.gauth
import gdata.service
import gdata.spreadsheets
import gdata.spreadsheets.client
import gdata.spreadsheets.data
token = gdata.gauth.OAuth2Token(client_id=Client_id,client_secret=Client_secret,scope=Scope,
access_token=ACCESS_TOKEN, refresh_token=REFRESH_TOKEN,
user_agent=User_agent)
client = gdata.spreadsheets.client.SpreadsheetsClient()
token.authorize(client)
range = "D6:D13"
cellq = gdata.spreadsheets.client.CellQuery(range=range, return_empty='true')
cells = client.GetCells(file_id, 'od6', q=cellq)
objData = gdata.spreadsheets.data
batch = objData.BuildBatchCellsUpdate(file_id, 'od6')
n = 1
for cell in cells.entry:
cell.cell.input_value = str(n)
batch.add_batch_entry(cell, cell.id.text, batch_id_string=cell.title.text, operation_string='update')
n = n + 1
client.batch(batch, force=True)
Related
I've seen a bunch of issues related to this, but doesn't seem there is a strong fix. Basically, I'm trying to scrape all of the accounts an Instagram account follows, along with how many followers they have.
I have it now pushing to a spreadsheet, so that I can start to sort and analyze.
My only problem is the script keeps timing out. I've tried using time.sleep (off someone's recommendation) but it's only making the information load in slower & not fixing the issue.
Any suggestions? I could just be doing something entirely wrong - learning as I go.
import gspread
import instaloader
loader = instaloader.Instaloader()
gc = gspread.service_account(filename='creds.json')
sh = gc.open_by_key('1cD8mX8tR2iSQgSmpk6QxBVVHYVV8VxGs2uhtA8iBSpQ')
worksheet = sh.sheet1
loader.login("vcf1948", "VCF1948!")
profile = instaloader.Profile.from_username(loader.context, "michelleobama")
followees = profile.get_followees()
for followee in profile.get_followees():
print('{} has {} followees'.format(followee.username, followee.followers))
AddValue = [followee.username, int(followee.followees)]
worksheet.append_row(AddValue)
it's my second time to post my problem.
I need to look up if an url is safe by google using Pysafebrowsing it's a python package, so I get i different result Comparison of the original source :
https://transparencyreport.google.com/safe-browsing/search
My script is :
from pysafebrowsing import SafeBrowsing
s = SafeBrowsing(KEY)
r = s.lookup_urls([domain])
print(r)
Please if you have any idea help me!
I am working on a python script to retrieve and work with data from Google Sheets. So far, I can log in with no issues and read/write cell data.
However, as far as some cells containing formulas are concerned, the value python returns is #DIV/0!, instead of the result I see in the cell of the spreadsheet itself.
Why is that? I tried to specify the "Value Render Option" parameter as either FORMATTED_VALUE, UNFORMATTED_VALUE or FORMULA, but nothing works.
For example, one cell contains an IF formula, which shows up as "Positive" or "Negative" in the sheet itself. I want python to retrieve this value. Instead, it gets me #DIV/0! when I print out the values. Same goes for a calculation I do in the sheet itself (which generates a proper result when I view the sheet online).
My script resembles the example ones on the Google tutorial pages.
Thanks for helping out a begginer! Cheers!
Here's the script:
from __future__ import print_function
from pprint import pprint
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from googleapiclient import discovery
import pickle
SCOPES = ['https://www.googleapis.com/auth/spreadsheets.readonly']
spreadsheet_id = '1-V95KxzJCPglRa90KnvnHBvY2doffeAxuRrEBEA0mHg'
range_ = 'Finance_3!A1:D30'
credentials = None
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
credentials = pickle.load(token)
service = discovery.build('sheets', 'v4', credentials=credentials)
value_render_option = 'FORMATTED_VALUE'
date_time_render_option = 'FORMATTED_STRING'
request = service.spreadsheets().values().get(spreadsheetId=spreadsheet_id, range=range_, valueRenderOption=value_render_option, dateTimeRenderOption=date_time_render_option)
response = request.execute()
pprint(response)
Seems like I found out the cause. There's nothing wrong with the code, I tested it on different spreadsheets.
When trying to fetch or otherwise work with GOOGLEFINANCE historical price data, the supplied value is NONE (or NULL). That's what's causing the DIV/0 issue. Even if your formula is based on a formula that's based on historical price, it still doesn't work.
The "realtime" price gets returned just fine though.. It's only the historical price that's not supported.
For price history I am now using Alpha Vantage. Works fine with python.
Cheers!
import json
from os import path, makedirs
_default_dir = path.expanduser('~/.config/gspread_pandas')
_default_file = 'google_secret.json'
def ensure_path(pth):
if not path.exists(pth):
makedirs(pth)
hi, I'm currently working on data collection via selenium and pandas to parse the data and edit it with pandas to send the data to google spread
however, while I'm working on gspread-pandas module, the module needs to put google_secret json file to '~/.config/gspread_pandas'. which is fixed location as described in the link below
https://pypi.python.org/pypi/gspread-pandas/0.15.1
I want to make some function to set the custom location to achieve independent working app environment.
for example, I want to locate the file to here
default_folder = os.getcwd()
the default_folder will be where my project is located(the same folder)
what can I do with it?
If you see the source https://github.com/aiguofer/gspread-pandas/blob/master/gspread_pandas/conf.py you can notice, that you can create your own config and pass it to Spread object constructor.
But yes, this part is really badly documented.
So, this code works well for me:
from gspread_pandas import Spread, conf
c = conf.get_config('[Your Path]', '[Your filename]')
spread = Spread('username', 'spreadname', config=c)
Thank you for this. It really should be documented better. I was getting so frustrated trying to get this to work with heroku, but it worked perfectly. I had to change to the following:
c = gspread_pandas.conf.get_config('/app/', 'google_secret.json')
spread = gspread_pandas.Spread('google_sheet_key_here_that_is_a_long_string', config=c)
I am new to Google Datastore and following this google documentation.
https://cloud.google.com/datastore/docs/datastore-api-tutorial
I have done authorization for calling Google APIs by reading this doc.
https://developers.google.com/identity/protocols/application-default-credentials#callingpython
I have only two files:
1. client_secret.json
2. datastoreConnection.py
and this is my datastoreConnection.py till now:
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/path/to/client_secret.json'
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
from google.cloud import datastore
client = datastore.Client(project ="xyz",namespace = "staging")
def list_tasks(client):
query = client.query(kind='testData')
query.projection = ['patientName']
return list(query.fetch(limit=10))
print(list_tasks(client))
This code is working fine and returning me the required data.
Problem is arising when I apply Projection with multiple properties.
e.g
query.projection = ['patientName','age']
code is giving me the error:
google.cloud.exceptions.PreconditionFailed: 412 no matching index found. recommended index is:<br/>- kind: testData<br/> properties:<br/> - name: age<br/> - name: patientName<br/>
For making projection query I read this.
https://cloud.google.com/datastore/docs/concepts/queries#datastore-projection-query-python
I have cross checked the property names but still having the same error.
How can I solve this ?
I have seen index.yaml file in other questions related to this kind of problem. Is this necessary to use and what are its benefit?
Any time you need to retrieve more than one property, but not all properties (*), you need to have a matching composite index. In this case you need a composite index on patient name and age properties. You can create a composite index using index.yaml file or gcloud command line tool. The documentation below should help you:
https://cloud.google.com/datastore/docs/concepts/indexes.