Hey I am having problems accessing some lists.
I can access Items using this code:
data = session.get(BASE_URL + 'tori_market.php',params={'format': 'json'}).json()
except ValueError:
for item in data['items']:
print(item['price'])
But I can not access User using the same code:
data = session.get(BASE_URL + 'tori_market.php',params={'format': 'json'}).json()
except ValueError:
for users in data['user']:
print(user['max'])
Edit: I've posted the wrong code,here is the one i'm using.
data = session.get(BASE_URL + 'tori_market.php',params={'format': 'json'}).json()
except ValueError:
for users in data['user']:
print(users['balance'])
What is wrong with it?
You can check how the API directory are build in this link.
The full traceback is:
Traceback (most recent call last):
File "/Users/Cristi/Desktop/RealBot/prova.py", line 34, in <module>
data = session.get(BASE_URL + 'tori_market.php',params={'format': 'json'}).json()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/Cristi/Desktop/RealBot/prova.py", line 37, in <module>
print(users['balance'])
TypeError: string indices must be integers
As soon as it's a passworded page i can give you a screenshot,here.
Edited answer
the user in above is a key in json data, so when you do
for users in data["user"]
you are already iterating over its keys.
Instead, for sake of brevity, do,
for key in data["user"]:
print key, data["user"][key]
This will print all the data within the user dict for you. So now key can take the values of "balance" etc.
Original answer
This is a typo between users and user, you use:
for users in data['user']:
but access it as:
print(user['max'])
Instead access it as:
print(users['max'])
Related
I have tested around 1000 bitcoin addresses and there were no problems, however after trying to test around 40,000 addresses it gives me this error
from requests import get
import pandas as pd
import json
def make_api_url():
data = pd.read_csv('bitcoinaddr.csv')
Wallet_Address = (data.loc[:,"Address"])
BASE_URL = "https://blockchain.info/balance"
for addresses in Wallet_Address:
addresses = '|'.join(Wallet_Address)
url = BASE_URL + f"?active={addresses}" #error
return url
get_balance_url = make_api_url()
response = get(get_balance_url)
j_data = response.json()
finalbalance=[]
tx=[]
totalreceived=[]
new_dataset = pd.DataFrame.from_dict(j_data, orient='index')
print(new_dataset)
finalbalance=new_dataset['final_balance'].to_list()
tx=new_dataset['n_tx'].to_list()
totalreceived=new_dataset['total_received'].to_list()
data['Final Balance']=finalbalance
data['n_tx']=tx
data['Total Received']=totalreceived
new_dataset.to_csv('CheckedBTCAddress.csv',index=True,header=True)
# https://blockchain.info/balance?active=$address
The error code shows
Traceback (most recent call last): File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py",
line 971, in json
return complexjson.loads(self.text, **kwargs) File "C:\Users\AppData\Local\Programs\Python\Python310\lib\json_init_.py",
line 346, in loads
return _default_decoder.decode(s) File "C:\Users\AppData\Local\Programs\Python\Python310\lib\json\decoder.py",
line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\Forensic\AppData\Local\Programs\Python\Python310\lib\json\decoder.py",
line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"c:\Users
Scripting\BTCAddressBulkChecker.py", line 21, in
data = response.json() File "C:\UsersPython\Python310\lib\site-packages\requests\models.py",
line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1
(char 0)
i'm using replits database but when i try to load it in it returns an error
any ideas?
cookie = db[name]
cookiepc = db[name + "cookiepc"]
increase = db[name + "increase"]
the error is
Traceback (most recent call last):
File "main.py", line 25, in <module>
cookiepc = db[name + "cookiepc"]
File "/home/runner/Cookie-clicker/venv/lib/python3.8/site-packages/replit/database/database.py", line 439, in __getitem__
raw_val = self.get_raw(key)
File "/home/runner/Cookie-clicker/venv/lib/python3.8/site-packages/replit/database/database.py", line 479, in get_raw
raise KeyError(key)
KeyError: 'shdfgwbdhfbadwcookiepc'
According to the documentation for replit-py, a KeyError is raised when an attempt is made to read from a key that doesn't exist in the database.
You can use db.get to specify a default value for if the key doesn't exist:
print(db.get("b", "default")) # default
db["b"] = "pie"
print(db.get("b", "default")) # pie
I'm having trouble using the Python email module to parse emails where the FROM header has parentheses in it. This only seems to be the problem when using email.policy.default as opposed to email.policy.compat32.
Is there a solution to this problem, other than switching policies?
A minimum working example is below, for Python 3.6.5:
import email
import email.policy as email_policy
raw_mime_msg=b"from: James Mishra \\(says hi\\) <james#example.com>"
compat32_obj = email.message_from_bytes(
raw_mime_msg, policy=email_policy.compat32)
default_obj = email.message_from_bytes(
raw_mime_msg, policy=email_policy.default)
print(compat32_obj['from'])
print(default_obj['from'])
The first print statement returns:
James Mishra \(says hi\) <james#example.com>
and the second print statement returns:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1908, in get_address
token, value = get_group(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1867, in get_group
"display name but found '{}'".format(value))
email.errors.HeaderParseError: expected ':' at end of group display name but found '\(says hi\) <james#example.com>'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1734, in get_mailbox
token, value = get_name_addr(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1720, in get_name_addr
token, value = get_angle_addr(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1646, in get_angle_addr
"expected angle-addr but found '{}'".format(value))
email.errors.HeaderParseError: expected angle-addr but found '\(says hi\) <james#example.com>'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_email.py", line 12, in <module>
print(default_obj['from'])
File "/usr/local/lib/python3.6/email/message.py", line 391, in __getitem__
return self.get(name)
File "/usr/local/lib/python3.6/email/message.py", line 471, in get
return self.policy.header_fetch_parse(k, v)
File "/usr/local/lib/python3.6/email/policy.py", line 162, in header_fetch_parse
return self.header_factory(name, value)
File "/usr/local/lib/python3.6/email/headerregistry.py", line 589, in __call__
return self[name](name, value)
File "/usr/local/lib/python3.6/email/headerregistry.py", line 197, in __new__
cls.parse(value, kwds)
File "/usr/local/lib/python3.6/email/headerregistry.py", line 340, in parse
kwds['parse_tree'] = address_list = cls.value_parser(value)
File "/usr/local/lib/python3.6/email/headerregistry.py", line 331, in value_parser
address_list, value = parser.get_address_list(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1931, in get_address_list
token, value = get_address(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1911, in get_address
token, value = get_mailbox(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1737, in get_mailbox
token, value = get_addr_spec(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1583, in get_addr_spec
token, value = get_local_part(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1413, in get_local_part
obs_local_part, value = get_obs_local_part(str(local_part) + value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1454, in get_obs_local_part
token, value = get_word(value)
File "/usr/local/lib/python3.6/email/_header_value_parser.py", line 1340, in get_word
if value[0]=='"':
IndexError: string index out of range
email.policy.default is intended to be compliant with the email RFCs, and your message is not compliant with RFC 5322. If the parenthesized part is supposed to be a comment, then the message should look like
raw_mime_msg=b"from: James Mishra (says hi) <james#example.com>"
to be compliant. If it is not supposed to be a comment, then the parentheses should appear inside a quoted string. That might look something like
raw_mime_msg=b'from: "James Mishra (says hi)" <james#example.com>'
Since your message is not compliant, using the policy that expects compliance is a poor fit. If you want to handle non-compliant messages, email.policy.compat32 is a better choice than email.policy.default.
Ok, so what I need is just CVS file with basic stocks data with additional sector column for each company. Getting sectors and tickers from wikipedia works fine.
But Google/Yahoo data doesn't want to work anymore, so I've tried to use Quandl.I've recently made a few technical indicators based on it so I thought it would be great.But there is following error.
I am still in doubt if I can format my request like that so I assume that there is a problem but I can't figure a way to make it work.
Thank you for any advice and sorry for misspelings.
from bs4 import BeautifulSoup
import requests
import pandas as pd
import lxml
import quandl as qdl
def get_ticker_and_sector(url='https://en.wikipedia.org/wiki/List_of_S%26P_500_companies'):
r = requests.get(url)
data = r.text
soup = BeautifulSoup(data, 'lxml')
table = soup.find('table')
sp500 = {}
for tr in table.find_all('tr')[1:]:
tds = tr.find_all('td')
ticker = tds[0].text
sector = tds[3].text
sp500[ticker] = sector
return sp500
if __name__ == '__main__':
sp500 = get_ticker_and_sector()
for i, (ticker, sector) in enumerate(sp500.items()):
stock_df = qdl.get('WIKI/%s', start_date="2010-12-11", end_date="2011-12-31")%(ticker)
stock_df['Name'] = ticker
stock_df['Sector'] = sector
if i == 0:
all_df = stock_df
else:
all_df = all_df.append(stock_df)
all_df.to_csv('all_sp500_data_2.csv')
Error.
Traceback (most recent call last):
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/connection.py", line 55, in parse
return response.json()
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/requests/models.py", line 886, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tomek/PycharmProjects/untitled4/get_file.py", line 30, in <module>
stock_df = qdl.get('WIKI/%s', start_date="2010-12-11", end_date="2011-12-31")%('ticker')
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/get.py", line 48, in get
data = Dataset(dataset_args['code']).data(params=kwargs, handle_column_not_found=True)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/model/dataset.py", line 47, in data
return Data.all(**updated_options)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/operations/list.py", line 14, in all
r = Connection.request('get', path, **options)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/connection.py", line 36, in request
return cls.execute_request(http_verb, abs_url, **options)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/connection.py", line 44, in execute_request
cls.handle_api_error(response)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/connection.py", line 61, in handle_api_error
error_body = cls.parse(resp)
File "/home/tomek/tomek-workspace/pythons/udemy/lib/python3.5/site-packages/quandl/connection.py", line 57, in parse
raise QuandlError(http_status=response.status_code, http_body=response.text)
quandl.errors.quandl_error.QuandlError: (Status 400) Something went wrong. Please try again. If you continue to have problems, please contact us at connect#quandl.com.
The traceback is giving you an 400 status code which means something is wrong with the request to the API.
More specifically, Quandl's documentation says the following regarding a 400 status code:
We could not recognize your API key. Please check your API key and try again. You can find your API key under your account settings.
As a good starting point, have you entered your API key after installing the Quandl module? Steps are here for more information on how to do it.
Please note, I only glanced at the rest of your code, and didn't test it out fully to see if there are other errors!
I am trying to load json file availalble here : https://gist.githubusercontent.com/anonymous/e5ef9cb96acb98e1f813d5166d472c70/raw/eabf219c51ace122ad82b7037bbf93d347fb4a9b/data.json
with open('data.json') as data_file:
data = json.load(data_file)
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/home/karimk/python/lib/python2.7/json/__init__.py", line 291, in load
**kw)
File "/home/karimk/python/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/home/karimk/python/lib/python2.7/json/decoder.py", line 367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 3 column 1 - line 6 column 740 (char 2826 - 16384)
What wrong am I doing here?
What wrong : json is not valid
Solution : update data to be a valid json, by example remove end of line inside string item.