Im trying to launch an ml-engine jobs submit training using a cloud composer, i'm using this guide for instructions recommendation-system-tensorflow-deploy.
Im using a plugin which google created (see the implementation here)
Im trying to make it work on python version 3.5, this by changing line 206 from:
training_request = {
'jobId': job_id,
'trainingInput': {
'scaleTier': self._scale_tier,
'packageUris': self._package_uris,
'pythonModule': self._training_python_module,
'region': self._region,
'args': self._training_args,
'masterType': self._master_type
}
To:
training_request = {
'jobId': job_id,
'trainingInput': {
'scaleTier': self._scale_tier,
'packageUris': self._package_uris,
'pythonModule': self._training_python_module,
'region': self._region,
'args': self._training_args,
'masterType': self._master_type,
'python-version': '3.5' #self._python_version
}
I also tried to add to it the run time version (runtime-version='1.12') but i keep on getting the following error:
[2019-01-20 11:58:36,331] {models.py:1594} ERROR - <HttpError 400 when requesting https://ml.googleapis.com/v1/projects/hallowed-forge-577/jobs?alt=json returned "Invalid JSON payload received. Unknown name "python-version" at 'job.training_input': Cannot find field.">
Traceback (most recent call last)
File "/usr/local/lib/airflow/airflow/models.py", line 1492, in _run_raw_tas
result = task_copy.execute(context=context
File "/home/airflow/gcs/plugins/ml_engine_plugin.py", line 241, in execut
self._project_id, training_request, check_existing_job
File "/home/airflow/gcs/plugins/ml_engine_plugin.py", line 79, in create_jo
request.execute(
File "/usr/local/lib/python3.6/site-packages/oauth2client/util.py", line 135, in positional_wrappe
return wrapped(*args, **kwargs
File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 838, in execut
raise HttpError(resp, content, uri=self.uri
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://ml.googleapis.com/v1/projects/hallowed-forge-577/jobs?alt=json returned "Invalid JSON payload received. Unknown name "python-version" at 'job.training_input': Cannot find field."
[2019-01-20 11:58:36,334] {models.py:1623} INFO - Marking task as FAILED.
[2019-01-20 11:58:36,513] {models.py:1627} ERROR - Failed to send email to: ['airflow#example.com']
[2019-01-20 11:58:36,516] {models.py:1628} ERROR - HTTP Error 401: Unauthorized
Traceback (most recent call last)
File "/usr/local/lib/airflow/airflow/models.py", line 1625, in handle_failur
self.email_alert(error, is_retry=False
File "/usr/local/lib/airflow/airflow/models.py", line 1778, in email_aler
send_email(task.email, title, body
File "/usr/local/lib/airflow/airflow/utils/email.py", line 44, in send_emai
return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype
File "/usr/local/lib/airflow/airflow/contrib/utils/sendgrid.py", line 116, in send_emai
_post_sendgrid_mail(mail.get()
File "/usr/local/lib/airflow/airflow/contrib/utils/sendgrid.py", line 122, in _post_sendgrid_mai
response = sg.client.mail.send.post(request_body=mail_data
File "/usr/local/lib/python3.6/site-packages/python_http_client/client.py", line 252, in http_reques
return Response(self._make_request(opener, request, timeout=timeout)
File "/usr/local/lib/python3.6/site-packages/python_http_client/client.py", line 176, in _make_reques
raise ex
python_http_client.exceptions.UnauthorizedError: HTTP Error 401: Unauthorize
Notice that the python version actually changes (to 3.6 from the original 2.7) so changing the python version, does something, but then gets stuck
Any help on what i'm missing here will be awesome!
It seems like the example uses an old version of airflow MLEngineTrainingOperator.
The last version implements the runtime-version/python-version training params.
Use the current version:
mlengine_operator.py
Related
EDIT 2: As #ximzend said, spotipy has moved to playlist_items instead of playlist_tracks
EDIT 1: In the process of moving to a curses interface, I changed the progress bar to a tqdm one (rather than rich), and that seemed to solve my problem. I did also upgrade spotipy (after it started working) but no change was needed in the API code ¯( ツ )/¯ so idk if rich was breaking the API req somehow or whatever
Thought I would put this here to help anyone in the future ツ
Spotipy throws this error when trying to retrieve data about tracks in a playlist
requests.exceptions.HTTPError: 404 Client Error: Not Found for url:
The error is thrown on the line below, specifically the self.sp.playlist_tracks(playlist_uri)['items'] part (The progress.track is a progress bar for the loop from rich)
for song in progress.track(self.sp.playlist_tracks(playlist_uri)['items'], description='Listing songs...'):
Full traceback is:
HTTP Error for GET to https://api.spotify.com/v1/playlists/4wi8hpuh4oyjtivbzk3k9n/tracks with Params: {'limit': 100, 'offset': 0, 'fields': None, 'market': None, 'additional_types': 'track'} returned 404 due to Not found.
Traceback (most recent call last):
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\spotipy\client.py", line 245, in _internal_call
response.raise_for_status()
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.spotify.com/v1/playlists/4wi8hpuh4oyjtivbzk3k9n/tracks?limit=100&offset=0&additional_types=track
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "t:\projects\youtusic\main.py", line 132, in <module>
main()
File "t:\projects\youtusic\main.py", line 119, in main
track_list = obj.sp_get_tracks(playlist_uri)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "t:\projects\youtusic\youtusic.py", line 104, in sp_get_tracks
for song in progress.track(self.sp.playlist_tracks(playlist_uri)['items'], description='Listing songs...'):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\spotipy\client.py", line 657, in playlist_tracks
return self.playlist_items(playlist_id, fields, limit, offset,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\spotipy\client.py", line 681, in playlist_items
return self._get(
^^^^^^^^^^
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\spotipy\client.py", line 297, in _get
return self._internal_call("GET", url, payload, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "T:\projects\youtusic\youtusic-env\Lib\site-packages\spotipy\client.py", line 267, in _internal_call
raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 404, code:-1 - https://api.spotify.com/v1/playlists/4wi8hpuh4oyjtivbzk3k9n/tracks?limit=100&offset=0&additional_types=track:
Not found., reason: None
The code I was using was as follows:
def sp_get_tracks(self, playlist_link: str) -> list:
'''
Uses [spotipy](https://pypi.org/project/spotipy/) to retrieve
a list of songs in the playlist provided. Returns a list to be
used with `grab_yt_links`
'''
playlist_uri = playlist_link.split('/')[-1].split('?')[0]
song_titles = []
for song in progress.track(self.sp.playlist_tracks(playlist_uri)['items'], description='Listing songs...'):
track_name: str = song['track']['name']
artist_name: str = song['track']['artists'][0]['name']
track_name = track_name.replace(' ', '+')
artist_name = artist_name.replace(' ', '+')
song_titles.append(f'{artist_name}+{track_name}')
return song_titles
It should returns a list of all the songs (and artists) in the playlist. (The progress.track is a progress bar for the loop from rich). The 404 confuses me because my network connection is fine and I am able to perform Google searches etc
Additionally, the playlist only has 19 songs in so should be resticted by the limits I have read elsewhere
You get this error, because Spotify made a code breaking change to its API.
SpotiPy has fixed this issue by releasing an upgrade.
To install this upgrade, open a command prompt and run pip install spotipy --upgrade.
After the upgrade, you need to use playlist_items(playlist_id, fields=None, limit=100, offset=0, market=None, additional_types=('track', 'episode')). playlist_tracks is deprecated.
python code.
import google.cloud.logging
client = google.cloud.logging.Client.from_service_account_json("file.config")
client.setup_logging()
import logging
loggin.info("error")
traceback:
Traceback (most recent call last):
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/grpc/_channel.py", line 923, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/grpc/_channel.py", line 826, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.PERMISSION_DENIED
details = "The caller does not have permission"
debug_error_string = "{"created":"#1612798593.245379000","description":"Error received from peer ipv4:142.250.71.42:443","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"The caller does not have permission","grpc_status":7}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/cloud/logging/handlers/transports/background_thread.py", line 123, in _safely_commit_batch
batch.commit()
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/cloud/logging/logger.py", line 383, in commit
client.logging_api.write_entries(entries, **kwargs)
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/cloud/logging/_gapic.py", line 121, in write_entries
self._gapic_api.write_log_entries(
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/cloud/logging_v2/gapic/logging_service_v2_client.py", line 476, in write_log_entries
return self._inner_api_calls["write_log_entries"](
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/retry.py", line 281, in retry_wrapped_func
return retry_target(
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/Users/soubhagyapradhan/Desktop/upwork/baby/data-science/env/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 The caller does not have permission
Here i am trying to use google logging, But i am getting above error.
Please take a look
I am doing this using python. Is there any issue on generating service account creation
As John said, have you checked if the Service Account has the proper role assigned? As the official documentation says: "Using Cloud Logging library for Python requires the IAM Logs Writer role on Google Cloud. Most Google Cloud environments provide this role by default".
On the other hand, I am bit curious you used the "google-cloud-storage" tag, however you are not mention something related to it. Have you checked if the issue is because you do not have the enough permissions (As Storage Admin) to access your bucket?
I was using :
client = google.cloud.logging.Client()
(not ".from_service_account_json")
and I was getting the exact same error message. I also had the proper role roles/logging.logWriter. I am using using cloud-logging 2.6.0
I found that when I ran the code above in Vertex AI training job, if I don't create a client using my project_id in the following way:
client = google.cloud.logging.Client(project=<user project number>)
then google.cloud.logging use some kind of dummy project_id i.e "i285ca2410679d8f1p-tp" which don't have the necessay access. Putting my project_id and all the error messages are then gone.
Can anybody help me i want to create a solrcloud on aws using this code https://github.com/LucidWorks/solr-scale-tk i try to build using cmd [fab demo:demo1,n=1] getting below error I'm getting this while pulling instances after connection to amazon server.
ERROR: boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
Appreciate your help
thanks in advance
root#adminuser-VirtualBox:/opt/febric/solr-scale-tk# fab demo:demo1,n=1
Going to launch 1 new EC2 m3.medium instances using AMI ami-8d52b9e6
Setup Instance store BlockDeviceMapping: /dev/sdb -> ephemeral0
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main
*args, **kwargs
File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 427, in execute
results['<local-only>'] = task.run(*args, **new_kwargs)
File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 174, in run
return self.wrapped(*args, **kwargs)
File "/opt/febric/solr-scale-tk/fabfile.py", line 1701, in demo
ec2hosts = new_ec2_instances(cluster=demoCluster, n=n, instance_type=instance_type)
File "/opt/febric/solr-scale-tk/fabfile.py", line 1163, in new_ec2_instances
placement_group=placement_group)
File "/usr/local/lib/python2.7/dist-packages/boto/ec2/connection.py", line 973, in run_instances
verb='POST')
File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1208, in get_object
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidParameterValue</Code><Message>Value () for parameter groupId is invalid. The value cannot be empty</Message></Error></Errors><RequestID>ca03b6d4-ce0e-46d3-99e3-ccad4a43c4ff</RequestID></Response>
You need to create a security group at your region with the name exactly called "solr-scale-tk" and has the required ports opened up. Please refer to this blog and follow the instructions in Setup Amazon Account part.
I've seen a couple of questions that already asked this but there were no responses, so I'll give it a try. When I use the following code:
import praw, time
r = praw.Reddit(user_agent="Bot experiment by redacted")
r.login('redacted', 'redacted')
I get a connection error that has the following traceback:
Traceback (most recent call last):
File "redacted", line 5, in <module>
r.login('redacted', 'redacted')
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/praw/__init__.py", line 1263, in login
self.request_json(self.config['login'], data=data)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/praw/decorators.py", line 161, in wrapped
return_value = function(reddit_session, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/praw/__init__.py", line 519, in request_json
response = self._request(url, params, data)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/praw/__init__.py", line 383, in _request
_raise_response_exceptions(response)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/praw/internal.py", line 172, in _raise_response_exceptions
response.raise_for_status()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/models.py", line 831, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden
I have tried this from python 3.4 and 2.7, I've tried running from IDLE and from the terminal. I've tried leaving my username and password out and logging in when prompted. I've tried from my Mac in my hotel room and a Windows machine from work and I get the same error everytime. I've tried from my bot account that I just made and my normal account. Does anyone have any ideas?
The issue was that I had the word 'bot' in my user_agent string. After it was removed, there were no problems.
I'm getting an error when trying to upload a custom module for a trigger app. I'm not sure what's going on; it looks like maybe a bad response a badly formatted json.
Error in remote call to api.post_files: Forge API call to module/admob/version went wrong: Expecting object: line 9 column 1 (char 174)
Traceback (most recent call last):
File "/Users/kevzettler/Library/Trigger Toolkit/build-tools/forge/async.py", line 106, in run
result = self._target(*self._args, **self._kwargs)
File "/Users/kevzettler/Library/Trigger Toolkit/trigger/api/api.py", line 20, in post_files
return forge_tool.singleton.remote._api_post(method, data=data, cookies=cookies, files=opened_files)
File "/Users/kevzettler/Library/Trigger Toolkit/build-tools/forge/remote.py", line 475, in _api_post
_check_api_response_for_error(url, 'POST', resp)
File "/Users/kevzettler/Library/Trigger Toolkit/build-tools/forge/remote.py", line 121, in _check_api_response_for_error
_raise_if_json_error(content_dict, error_template, resp, url)
File "/Users/kevzettler/Library/Trigger Toolkit/build-tools/forge/remote.py", line 81, in _raise_if_json_error
raise RequestError(resp, error_message, errors=errors)
RequestError: Forge API call to module/admob/version went wrong: Expecting object: line 9 column 1 (char 174)
It looks like the content of your config_schema.json file is invalid.
Unfortunately the error message you see isn't very useful. The good news is newer platform versions have better validation of local files before upload, so if you develop your module against a newer platform version errors like this should be listed in the Toolkit before upload.