We Keep Coding

how to use offical google translate api on local runtime via python

after reading the offical google translate api document, it provide us with the following sample code:
from google.cloud import translate
def translate_text(text="Hello, world!", project_id="weighty-site-333613"):
client = translate.TranslationServiceClient().from_service_account_json('key.json')
location = "global"
parent = f"projects/{project_id}/locations/{location}"
response = client.translate_text(
request={
"parent": parent,
"contents": [text],
"mime_type": "text/plain",
"source_language_code": "en-US",
"target_language_code": "zh-CN",
}
)
for translation in response.translations:
print("Translated text: {}".format(translation.translated_text))
translate_text()
These code worked properly on google cloud terminal.
However, even if i put the "key.json" file in the same folder, an error like this is shown:
/usr/local/bin/python3.6 /Users/jiajunmao/Documents/GitHub/translator_of_excel/google_trans.py
Traceback (most recent call last):
File "/Users/jiajunmao/Documents/GitHub/translator_of_excel/google_trans.py", line 37, in <module>
translate_text()
File "/Users/jiajunmao/Documents/GitHub/translator_of_excel/google_trans.py", line 22, in translate_text
client = translate.TranslationServiceClient().from_service_account_json('key.json')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/translate_v3/services/translation_service/client.py", line 354, in __init__
always_use_jwt_access=True,
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/translate_v3/services/translation_service/transports/grpc.py", line 158, in __init__
always_use_jwt_access=always_use_jwt_access,
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/translate_v3/services/translation_service/transports/base.py", line 110, in __init__
**scopes_kwargs, quota_project_id=quota_project_id
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_default.py", line 488, in default
raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
Process finished with exit code 1
can someone tell me what should i do at this step? thank you so much

You need service account json file with correct permissions from GCP under IAM & Service Accounts.
Then you need to implement command,
GOOGLE_APPLICATION_CREDENTIALS = "/path/to/your/service_account.json"

Firestore client in python (as user) using firebase_admin or google.cloud.firestore

I am building a python client-side application that uses Firestore. I have successfully used Google Identity Platform to sign up and sign in to the Firebase project, and created a working Firestore client using google.cloud.firestore.Client which is authenticated as a user:
import json
import requests
import google.oauth2.credentials
from google.cloud import firestore
request_url = f"https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key={self.__api_key}"
headers = {"Content-Type": "application/json; charset=UTF-8"}
data = json.dumps({"email": self.__email, "password": self.__password, "returnSecureToken": True})
response = requests.post(request_url, headers=headers, data=data)
try:
response.raise_for_status()
except (HTTPError, Exception):
content = response.json()
error = f"error: {content['error']['message']}"
raise AuthError(error)
json_response = response.json()
self.__token = json_response["idToken"]
self.__refresh_token = json_response["refreshToken"]
credentials = google.oauth2.credentials.Credentials(self.__token,
self.__refresh_token,
client_id="",
client_secret="",
token_uri=f"https://securetoken.googleapis.com/v1/token?key={self.__api_key}"
)
self.__db = firestore.Client(self.__project_id, credentials)
I have the problem, however, that when the token has expired, I get the following error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAUTHENTICATED
details = "Missing or invalid authentication."
debug_error_string = "{"created":"#1613043524.699081937","description":"Error received from peer ipv4:172.217.16.74:443","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Missing or invalid authentication.","grpc_status":16}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/home/my_app/src/controllers/im_alive.py", line 20, in run
self.__device_api.set_last_updated(utils.device_id())
File "/home/my_app/src/api/firestore/firestore_device_api.py", line 21, in set_last_updated
"lastUpdatedTime": self.__firestore.SERVER_TIMESTAMP
File "/home/my_app/src/api/firestore/firestore.py", line 100, in update
ref.update(data)
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/document.py", line 382, in update
write_results = batch.commit()
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/batch.py", line 147, in commit
metadata=self._client._rpc_metadata,
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/gapic/firestore_client.py", line 1121, in commit
request, retry=retry, timeout=timeout, metadata=metadata
File "/usr/local/lib/python3.7/dist-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/usr/local/lib/python3.7/dist-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.Unauthenticated: 401 Missing or invalid authentication.
I have tried omitting the token and only specifying the refresh token, and then calling credentials.refresh(), but the expires_in in the response from the https://securetoken.googleapis.com/v1/token endpoint is a string instead of a number (docs here), which makes _parse_expiry(response_data) in google.oauth2._client.py:257 raise an exception.
Is there any way to use the firestore.Client from either google.cloud or firebase_admin and have it automatically handle refreshing tokens, or do I need to switch to the manually calling the Firestore RPC API and refreshing tokens at the correct time?
Note: There are no users interacting with the python app, so the solution must not require user interaction.

Can't you just pass the string cast as integer _parse_expiry(int(float(response_data))) ?
If it is not working you could try to make a call and refresh token after getting and error 401, see my answer for the general idea on how to handle tokens.

As mentioned by #Marco, it is recommended that you use a service account if it's going to be used in an environment without user. When you use service account, you can just set GOOGLE_APPLICATION_CREDENTIALS environment variable to location of service account json file and just instantiate the firestore Client without any credentials (The credentials will be picked up automatically):
import firestore
client = firestore.Client()
and run it as (assuming Linux):
$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
$ python file.py
Still, if you really want to use user credentials for the script, you can install the Google Cloud SDK, then:
$ gcloud auth application-default login
This will open browser and for you to select account and login. After logging in, it creates a "virtual" service account file corresponding to your user account (that will also be loaded automatically by clients). Here too, you don't need to pass any parameters to your client.
See also: Difference between “gcloud auth application-default login” and “gcloud auth login”

Accessing google cloud storage bucket from cloud functions throws 500 error

I'm trying to access google cloud storage bucket from cloud functions (python) instance and it's throwing mystic 500 error.
I have given the service account editor role too. It didn't make any change.
I also checked if any of the quota is going off limit. The limits were not even close.
Please, anyone can help me find cause of this error?
here is the code
from google.cloud import storage
import os
import base64
storage_client = storage.Client()
def init_analysis(event, context):
print("event", event)
pubsub_message = base64.b64decode(event['data']).decode('utf-8')
print(pubsub_message)
bucket_name = 'my-bucket'
bucket = storage_client.get_bucket(bucket_name)
blobs = bucket.list_blobs()
for blob in blobs:
print(blob.name)
Error:
Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 99, in refresh service_account=self._service_account_email) File "/env/local/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 208, in get_service_account_token 'instance/service-accounts/{0}/token'.format(service_account)) File "/env/local/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 140, in get url, response.status, response.data), response) google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token from the Google Compute Enginemetadata service. Status: 500 Response:\nb'Could not fetch URI /computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token\\n'", <google.auth.transport.requests._Response object at 0x2b0ef9edf438>) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function _function_handler.invoke_user_function(event_object) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function event_context.Context(**request_or_event.context)) File "/user_code/main.py", line 21, in init_analysis bucket = storage_client.get_bucket(bucket_name) File "/env/local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 227, in get_bucket bucket.reload(client=self) File "/env/local/lib/python3.7/site-packages/google/cloud/storage/_helpers.py", line 130, in reload _target_object=self, File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 315, in api_request target_object=_target_object, File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 192, in _make_request return self._do_request(method, url, headers, data, target_object) File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 221, in _do_request return self.http.request(url=url, method=method, headers=headers, data=data) File "/env/local/lib/python3.7/site-packages/google/auth/transport/requests.py", line 205, in request self._auth_request, method, url, request_headers) File "/env/local/lib/python3.7/site-packages/google/auth/credentials.py", line 122, in before_request self.refresh(request) File "/env/local/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 102, in refresh six.raise_from(new_exc, caught_exc) File "<string>", line 3, in raise_from google.auth.exceptions.RefreshError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token from the Google Compute Enginemetadata service. Status: 500 Response:\nb'Could not fetch URI /computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token\\n'", <google.auth.transport.requests._Response object at 0x2b0ef9edf438>)
google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token from the Google Compute Enginemetadata service. Status: 500 Response:\nb'Could not fetch URI /computeMetadata/v1/instance/service-accounts/my-project#appspot.gserviceaccount.com/token\\n'"

The error you are getting it's because your Cloud Functions service account doesn't have the cloudfunctions.serviceAgent role. As you can see on the documentation:
Authenticating as the runtime service account from inside your function may fail if you change the Cloud Functions service account's permissions.
However, I found that sometimes you can not add this role as it doesn't show up as an option. I have reported this issue to the Google Cloud Functions engineering team and they are working to solve it.
Nevertheless, you can add the role again using this gcloud command:
gcloud projects add-iam-policy-binding <project_name> --role=roles/cloudfunctions.serviceAgent --member=serviceAccount:service-<project_number>#gcf-admin-robot.iam.gserviceaccount.com

Google ML Engine and Python Datastore API, 'Forbidden: 403 Request had insufficient authentication scopes.'

I'm running a TensorFlow model on Google ML Engine. When the model training is finished, I want to store a JSON string with the results to Datastore. For this, I am using the following:
from gcloud import datastore
def put_json_into_datastore(json_str, project_id, entity_type):
"""
Store json string in Datastore
"""
# Instantiate the client to the project
datastore_client = datastore.Client(project_id)
# The name/ID for the new entity
name = str(datetime.datetime.now())
# The Cloud Datastore key for the new entity
entity_key = datastore_client.key(entity_type, name)
# Prepare the new entity
entity = datastore.Entity(key=entity_key)
# Get the json string into the entity
entity.update(json_str)
# Put the entity into Datastore
datastore_client.put(entity)
Although, I am getting the error 'Forbidden: 403 Request had insufficient authentication scopes.' Here's the full error trace:
Traceback (most recent call last): File
"/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals File "/root/.local/lib/python2.7/site-packages/trainer/train.py", line 243,
in
FLAGS.entity_type) File "/root/.local/lib/python2.7/site-packages/trainer/data_helpers.py",
line 253, in put_json_into_datastore
datastore_client.put(entity) File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/client.py",
line 329, in put
self.put_multi(entities=[entity]) File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/client.py",
line 355, in put_multi
current.commit() File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/batch.py",
line 260, in commit
self._commit() File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/batch.py",
line 243, in _commit
self.project, self._commit_request, self._id) File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/connection.py",
line 342, in commit
_datastore_pb2.CommitResponse) File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/connection.py",
line 124, in _rpc
data=request_pb.SerializeToString()) File "/usr/local/lib/python2.7/dist-packages/gcloud/datastore/connection.py",
line 98, in _request
raise make_exception(headers, error_status.message, use_json=False) Forbidden: 403 Request had insufficient authentication
scopes.
Do I need to grant access somewhere for the ML engine to access Datastore?

The Cloud ML service doesn't execute with permissions sufficient to access Datastore. One way around this would be to upload credentials (e.g a json service account key file) for a service account with access to Cloud Datastore. You could then use that to obtain credentials capable of accessing Datastore.

Google Documents List API v3 (Python) Update document

I'm trying to update a document on Google Docs/Drive after I created it, using the GData helpers for Python.
The new version of the API lacks documentation for Py.
client = gdata.docs.client.DocsClient(source=PluginConfig.APP_NAME)
client.http_client.debug = PluginConfig.DEBUG
client.client_login(
PluginConfig.EMAIL,
PluginConfig.PASSWORD,
source=PluginConfig.APP_NAME,
service=client.auth_service
)
[...]
# Upload the text file
ms = gdata.data.MediaSource()
ms.SetFileHandle(file_path, content_type)
doc = gdata.docs.data.Resource(type='document', title=title)
doc.description = gdata.docs.data.Description(description)
doc = client.CreateResource(doc, media=ms)
doc = client.UpdateResource(doc, media=ms, new_revision=True)
Login and document creation work fine, but the Update() receives 400 Bad Request
Traceback (most recent call last):
File "coll.py", line 301, in <module>
main()
File "coll.py", line 293, in main
doc = client.UpdateResource(doc, media=ms, new_revision=True)
File "/usr/lib/python2.7/dist-packages/gdata/docs/client.py", line 344, in update_resource
uri_params=uri_params, **kwargs)
File "/usr/lib/python2.7/dist-packages/gdata/client.py", line 1151, in update_file
auth_token=auth_token, method='PUT')
File "/usr/lib/python2.7/dist-packages/gdata/client.py", line 1085, in upload_file
start_byte, self.file_handle.read(self.chunk_size))
File "/usr/lib/python2.7/dist-packages/gdata/client.py", line 1044, in upload_chunk
raise error
gdata.client.RequestError: Server responded with: 400, Invalid Request
More output: http://pastebin.com/LZL3qV0N
Any help is appreciated.

Try using the newer Drive API, its documentation includes Python samples in the reference guide, a Python quickstart and a complete sample application written in Python on App Engine.