Trouble with Etherscan API - python

I'm trying to connect to the Etherscan API in order download my transaction history across my cryptocurrency wallets. I've used the following Python code, which works just fine with the sample Etherscan gives in their API documentation.
import pandas as pd
import requests
address = '0xfbb1b73c4f0bda4f67dca266ce6ef42f520fbb98' #wallet address
url = 'http://api.etherscan.io/api?module=account&action=txlist&address='+address+'&startblock=0&endblock=99999999&sort=asc&apikey=my_api_key'
r = requests.get(url)
r.status_code #200 indicates success
json_output =r.json()
json_output["result"]
data = pd.DataFrame(json_output["result"])
I consistently get no data returned although the status code tells me the query is a success. Any thought? How stupid am I being?

Related

Python fails to parse JSON in API calls

I want to write a python script that retrieves data from a website using it's API. Using the code below (which comes from this [tutorial][1] - click [here for more context][2]), I am able to successfully submit a query to the website but not retrieve the data I need.
import requests, sys, json
WEBSITE_API = "https://rest.uniprot.org/beta"
def get_url(url, **kwargs):
response = requests.get(url, **kwargs);
if not response.ok:
print(response.text)
response.raise_for_status()
sys.exit()
return(response)
r = get_url(f"{WEBSITE_API}/uniprotkb/search?query=parkin AND (taxonomy_id:9606)&fields=id,accession,length,ft_site", headers={"Accept": "text/tsv"})
print(r.text)
Instead of printing the data I've queried, I get an error message:
{"url":"http://rest.uniprot.org/uniprotkb/search","messages":["Invalid request received. Requested media type/format not accepted: 'text/tsv'."]}
I figured out that I can print the data as a JSON-formatted string if I change the second-to-last line of my code to this:
r = get_url(f"{WEBSITE_API}/uniprotkb/search?query=parkin AND (taxonomy_id:9606)&fields=id,accession,length,ft_site")
In other words, I simply remove the following code snippet:
headers={"Accept": "text/tsv"}
I highly doubt this is an issue with the website because the tutorial the non-working code comes from was from a November 2021 webinar put on by an affiliated institution. I think a more likely explanation is that my python environment lacks some library and would appreciate any advice about how to investigate this possibility. Another possible solution I would appreciate is code and/or library suggestions that would put me on the path to simply parsing the JSON-string into a tab-delimited file.
[1]: https://colab.research.google.com/drive/1SU3j4VmXHYrYxOlrdK_NCBhajXb0iGes#scrollTo=EI91gRfPYfbc
[2]: https://drive.google.com/file/d/1qZXLl19apibjszCXroC1Jg2BGjcychqX/view

Getting The Data of a POST Request in Python

I am trying to get the skills returned after entering a Job title (such as "Process Analyst") on this website: https://match.emsiskills.com/skills/job.
After entering the job title, I observed that skills are called by a POST request. I tried matching the data required by this POST request but I keep getting a 415 error.
Here's my code:
import requests
skills_url = 'https://match.emsiskills.com/api/emsi-services/profiles/rankings/skills'
data = '{"filter":{"title":["15.74"]},"rank":{"by":"profiles","limit":60,"min_profiles":1}}'
r = requests.post(skills_url, data=data, json=True)
This returns a 415 error as mentioned earlier.
Any help is appreciated.
Thanks to #JustinEzequiel comment, it turns out I was using the calling post method incorrectly.
Here is the updated and working code:
import requests
skills_url = 'https://match.emsiskills.com/api/emsi-services/profiles/rankings/skills'
data = {"filter":{"title":["15.74"]},"rank":{"by":"profiles","limit":60,"min_profiles":1}}
r = requests.post(skills_url, json=data)

Trying to write Python to request API from 'nlm.nih.gov'

I am trying to run my csv data thru "https://rxnav.nlm.nih.gov/REST/interaction" to identify any drug interactions using python. What else do I need in order to have the program be ready?
I got 200 when print status_code is that mean my code is up and ready?
import requests
response = requests.get("https://rxnav.nlm.nih.gov/REST/interaction")
print(response.status_code)
Here's how you'd hit this API, using requests and the details in their example:
import requests
uri = "https://rxnav.nlm.nih.gov/REST/interaction/interaction.json"
params = {'rxcui': 341248}
r = requests.get(uri, params)
Now you can check that r.status_code is 200, and get at the result of the request. For example:
r.json()
As you may realize, this returns a Python dictionary.
The general idea is that requsts.get() takes the base URL, followed by the query parameters, given as a dictionary. What you get back depends on the API endpoint you're querying, and/or on the parameters. In this, it's giving you JSON. Others might give you text (see r.text for this representation), or bytes (r.content).

Iinvalid subscription key when using Microsoft Emotion API for Video in API Testing Console and in Python 2.7

I only manage to use the Emotion API subscription key for pictures but never for videos. It makes no difference whether I use the API Testing Console or try to call the Emotion API by Pathon 2.7. In both cases I get a response status 202 Accepted, however when opening the Operation-Location it says
{ "error": { "code": "Unauthorized", "message": "Access denied due to
invalid subscription key. Make sure you are subscribed to an API you are
trying to call and provide the right key." } }
On the Emotion API explanatory page it says that Response 202 means that
The service has accepted the request and will start the process later.
In the response, there is a "Operation-Location" header. Client side should further query the operation status from the URL specified in this header.
Then there is Response 401, which is exactly what my Operation-Location contains. I do not understand why I'm getting a response 202 which looks like response 401.
I have tried to call the API with Python using at least three code versions that I found on the Internet that
all amount to the same, I found the code here :
Microsoft Emotion API for Python - upload video from memory
python-upload-video-from-memory
import httplib
import urllib
import base64
import json
import pandas as pd
import numpy as np
import requests
_url = 'https://api.projectoxford.ai/emotion/v1.0/recognizeInVideo'
_key = '**********************'
_maxNumRetries = 10
paramsPost = urllib.urlencode({'outputStyle' : 'perFrame', \
'file':'C:/path/to/file/file.mp4'})
headersPost = dict()
headersPost['Ocp-Apim-Subscription-Key'] = _key
headersPost['content-type'] = 'application/octet-stream'
jsonGet = {}
headersGet = dict()
headersGet['Ocp-Apim-Subscription-Key'] = _key
paramsGet = urllib.urlencode({})
responsePost = requests.request('post', _url + "?" + paramsPost, \
data=open('C:/path/to/file/file.mp4','rb').read(), \
headers = headersPost)
print responsePost.status_code
videoIDLocation = responsePost.headers['Operation-Location']
print videoIDLocation
Note that changing _url = 'https://api.projectoxford.ai/emotion/v1.0/recognizeInVideo' to _url =
'https://westus.api.cognitive.microsoft.com/emotion/v1.0/recognizeInVideo' doesn't help.
However, afterwards I wait and run every half an hour:
getResponse = requests.request('get', videoIDLocation, json = jsonGet,\
data = None, headers = headersGet, params = paramsGet)
print json.loads(getResponse.text)['status']
The outcome has been 'Running' for hours and my video is only about half an hour long.
Here is what my Testing Console looks like Testing Console for Emotion API, Emotion Recognition in Video
Here I used another video that is about 5 minutes long and available on the internet. I found the video in a different usage example
https://benheubl.github.io/data%20analysis/fr/
that uses a very similar code, which again gets me a response status 202 Accepted and when opening the Operation-Location the subscription key is wrong
Here the code:
import httplib
import urllib
import base64
import json
import pandas as pd
import numpy as np
import requests
# you have to sign up for an API key, which has some allowances. Check the
API documentation for further details:
_url = 'https://api.projectoxford.ai/emotion/v1.0/recognizeinvideo'
_key = '*********************' #Here you have to paste your
primary key
_maxNumRetries = 10
# URL direction: I hosted this on my domain
urlVideo = 'http://datacandy.co.uk/blog2.mp4'
# Computer Vision parameters
paramsPost = { 'outputStyle' : 'perFrame'}
headersPost = dict()
headersPost['Ocp-Apim-Subscription-Key'] = _key
headersPost['Content-Type'] = 'application/json'
jsonPost = { 'url': urlVideo }
responsePost = requests.request( 'post', _url, json = jsonPost, data = None,
headers = headersPost, params = paramsPost )
if responsePost.status_code == 202: # everything went well!
videoIDLocation = responsePost.headers['Operation-Location']
print videoIDLocation
There are further examples on the internet and they all seem to work but replicating any of them never worked for me. Does anyone have any idea what could be wrong?
The Video Feature of Emotion API retires October 30th, so maybe you should change your procedure to screenshots anyways.
But for your question: The API returns you an URL where your results are accessible. You cannot open this URL in your browser, this will give you the notice of "invalid key", instead you need to call over python again this URL including your key.
I will post you my code how to get the score, I am using Python 3, so there might be some adjustments necessary. Only "tricky" point is getting the Operation ID, which is just the ID in the URL ( =location in my case) which leads to your request. Rest of the parameters like subscription key etc. is as before.
#extract operation ID from location-string
OID = location[67:]
bod = ""
try:
conn =
http.client.HTTPSConnection('westus.api.cognitive.microsoft.com')
conn.request("GET", "/emotion/v1.0/operations/"+OID+"?%s" %params, bod, headers)
response = conn.getresponse()
data = response.read()
print(data)
conn.close()
except Exception as e:
print("[Errno {0}] {1}".format(e.errno, e.strerror))
Did you verify your API call is working using curl? Always prototype calls using curl first. If it works in curl but not in Python, use Fiddler to observe the API request and response.
I also found an answer in the following link, all steps are explained:
https://gigaom.com/2017/04/10/discover-your-customers-deepest-feelings-using-microsoft-facial-recognition/

Python Error in getting or downloading a picture from facebook

I'm having a problem getting a picture in facebook.
I'm using python plus facebook graph api. I research getting or how to download a picture in facebook so here's what i got.
import os
import json
import urllib
import pprint
ACCESS_TOKEN = 'ACCESS_TOKEN_SAMPLE'
host = "https://graph.facebook.com/v2.8"
path = "/me?fields=id,name,picture"
params = urllib.urlencode({"access_token": ACCESS_TOKEN})
url = "{host}{path}?{params}".format(host=host, path=path, params=params)
resp = urllib.urlopen(url).read()
getDATA = json.loads(resp)
os.system("echo " + str(getDATA) | tee >log.txt)
I tried to understand it but I'm having an error running this. Here's the error i got:
{uerror: {umessage: uAn active access token must be used to query information about the current user., ucode: 2500, utype: uOAuthException, ufbtrace_id: uGLoKMknFrR1}}
Also I'm trying to put the downloaded data to a log file. I need an output which has like a downloading progress or bar? Like when you run a speedtest in a terminal it shows the size of the data that it downloads.

Categories