I am trying to read data from a server like this:
with requests.Session() as s:
data = {}
r = s.get('https://something.com' , json = data ).json()
training_set1 = np.empty([-1,4])
training_set1[:,0] = r["o"]
training_set1[:,1] = r["h"]
training_set1[:,2] = r["l"]
training_set1[:,3] = r["c"]
But I don't know the length of arrays, so I used -1 then got this error message:
ValueError: negative dimensions are not allowed
How can I fix this code? The response r is a JSON object:
{"t":[1322352000,1322438400],
"o":[123,123],
"h":[123,123],
"l":[123,123],
"c":[123,123]}
that I am trying to rearrange it to a numpy array.
Numpy arrays have fixed sizes. You cannot initialize a dynamic sized array. What you can do is use a list of lists and later convert the list to a numpy array.
Something like this should work assuming r["x"] is a list. (Untested code)
with requests.Session() as s:
data = {}
r = s.get('https://something.com' , json = data ).json()
t_set1 = []
t_set1.append(r["o"])
t_set1.append(r["h"])
t_set1.append(r["l"])
t_set1.append(r["c"])
training_set1 = np.array(t_set1)
Edit: Edited for the order "o","h","l",""c after OP edited the question
You cannot declare a numpy array with an unknown dimension. But you can declare it in one single operation:
training_set1 = np.array([r["o"], r["o"], r["h"], r["l"]])
or even better:
training_set1 = np.array([r[i] for i in "oohl"])
Related
import requests
import json
response = requests.get('http://dataservice.accuweather.com/currentconditions/v1/2807435?apikey=secret')
print(response)
x = response.json()
y = json.dumps(x)
z = json.loads(y)
a = z['WeatherText']
print(a)
yt = input("Press Enter to close")
This code is for retrieving weather data from accuweather. When I run the file, i get an error:
TypeError: list indices must be integers or slices, not str
A sample image of the initial json file is:
This image is the output
Please help me with the error
You only need x, but you also need to recognise that it is a list:
a = x[0]['WeatherText']
Variable z is a list of dict. Therefore you need to go to the first position by:
a = z[0]['WeatherText']
I am trying to iterate through a CSV file and create a numpy array for each row in the file, where the first column represents the x-coordinates and the second column represents the y-coordinates. I then am trying to append each array into a master array and return it.
import numpy as np
thedoc = open("data.csv")
headers = thedoc.readline()
def generatingArray(thedoc):
masterArray = np.array([])
for numbers in thedoc:
editDocument = numbers.strip().split(",")
x = editDocument[0]
y = editDocument[1]
createdArray = np.array((x, y))
masterArray = np.append([createdArray])
return masterArray
print(generatingArray(thedoc))
I am hoping to see an array with all the CSV info in it. Instead, I receive an error: "append() missing 1 required positional argument: 'values'
Any help on where my error is and how to fix it is greatly appreciated!
Numpy arrays don't magically grow in the same way that python lists do. You need to allocate the space for the array in your "masterArray = np.array([])" function call before you add everything to it.
The best answer is to import directly to a numpy array using something like genfromtxt (https://docs.scipy.org/doc/numpy-1.10.1/user/basics.io.genfromtxt.html) but...
If you know the number of lines you're reading in, or you can get it using something like this.
file_length = len(open("data.csv").readlines())
Then you can preallocate the numpy array to do something like this:
masterArray = np.empty((file_length, 2))
for i, numbers in enumerate(thedoc):
editDocument = numbers.strip().split(",")
x = editDocument[0]
y = editDocument[1]
masterArray[i] = [x, y]
I would recommend the first method but if you're lazy then you can always just build a python list and then make a numpy array.
masterArray = []
for numbers in thedoc:
editDocument = numbers.strip().split(",")
x = editDocument[0]
y = editDocument[1]
createdArray = [x, y]
masterArray.append(createdArray)
return np.array(masterArray)
I want take data from API from polish http by json format. But, I have problem take data from array in array.
From "normal" json I can took data, but this json have struture as 'krs_podmioty.id' => 'blabla' <= I have problem with . (dot) and array in array.
I try get data from https://api-v3.mojepanstwo.pl/dane/krs_podmioty/10186.json?layers[]=dzialalnosci&layers[]=reprezentacja.
You can decode on: http://freeonlinetools24.com/json-decode (and past text from http).
It's public website and data.
If you will look it, I want data from segment:
'krs_podmioty'.person_id' => array ( 0 => '14439' .... 11 => '1233301' )
import urllib.request
import json
res = urllib.request.urlopen('https://api-v3.mojepanstwo.pl/dane/krs_podmioty/10186.json?layers[]=dzialalnosci&layers[]=reprezentacja')
res_body = res.read()
j = json.loads(res_body.decode("utf-8"))
for item in j['data']:
ucmdbId = (item['krs_podmioty'])
print('Id podmioty: '.format(ucmdbId))
exit(0)
In perfect situation I need print list of all "krs_podmioty.person_id"
Thank you very much!
import requests
import json
result = requests.get('https://api-v3.mojepanstwo.pl/dane/krs_podmioty/10186.json?layers[]=dzialalnosci&layers[]=reprezentacja').json()
ids = result['data']['krs_podmioty.person_id']
for id in ids:
print('Id podmioty: ' + id)
Try this:
for item in j['data']['krs_podmioty.person_id']:
ucmdbId = item
print('Id podmioty: {0} '.format(ucmdbId))
j['data'] contained all of the objects in the 'data' array in which you could call for the krs_podmioty.person_id key to get its corresponding value array.
I am new in python coding and I would like to get XML file from a server, parse it and save to csv file.
2 parts are ok, I am able to get the file and parse it, but there is an issue with saving as a csv.
The code:
import requests
import numpy as np
hu = requests.get('https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml', stream=True)
from xml.etree import ElementTree as ET
tree = ET.parse(hu.raw)
root = tree.getroot()
namespaces = {'ex': 'http://www.ecb.int/vocabulary/2002-08-01/eurofxref'}
for cube in root.findall('.//ex:Cube[#currency]', namespaces=namespaces):
np.savetxt('data.csv', (cube.attrib['currency'], cube.attrib['rate']), delimiter=',')
Error I get is: mismatch between array dtype and format specifier.
It probably means I get data and try to save it as array, and there appears a mismatch.
But i am not sure how to fix the problem and to not have a mismatch.
Thank you
from the docs, your second argument in np.savetext should be a tuple of equal sized arrays. What you are providing are strings:
>>> x = y = z = np.arange(0.0,5.0,1.0)
>>> np.savetxt('test.out', x, delimiter=',') # X is an array
>>> np.savetxt('test.out', (x,y,z)) # x,y,z equal sized 1D arrays
>>> np.savetxt('test.out', x, fmt='%1.4e') # use exponential notation
You'll need to gather all of the concurrency and rate values into arrays, then save as csv:
concurrency, rate = [], []
for cube in root.findall('.//ex:Cube[#currency]', namespaces=namespaces):
concurrency.append(cube.attrib['concurrency'])
rate.append(cube.attrib['rate'])
np.savetext('file.csv', (concurrency, rate), delimeter='c')
I was trying to hot-encode data.
Data is list of vocabulary_size = 17005207.
To hot-encode, I made a list of inputs of num_labels = 100.
Following code:
inputs = []
for i in range(vocabulary_size):
inputs.append(np.arange(num_labels) == data[i]).astype(np.float32)
Throws me an Error:
AttributeError: 'NoneType' object has no attribute 'astype'
I tried dtype = np.float32 inside append function but again erroneous.
When I try this :
inputs = []
for i in range(vocabulary_size):
inputs.append(np.arange(num_labels) == data[i])
inputs = np.array(inputs,dtype=np.float32)
I get correct answer : Hot-Encoded Input Sequence of vocabulary_size x num_labels.
Any Alternative Solution Of One Line Without Using Numpy?
Solved :Can I be done directly using numpy array(input) with list(data)?
Info about data : data = np.ndarray(len(words), dtype=np.int32)
Reformat function:
def reformat(data):
num_labels = vocabulary_size
print (type(data))
data = (np.arange(num_labels) == data[:,None]).astype(np.int32)
return data
print (data,len(data))
return data
New Question : The dimension of data is (vocabulary_size,)...How to convert data using ravel or reshape into dimension of (1,vocabulary_size)?
Not sure whether I've understood correctly what you're asking for, but if what you want is a oneliner, you could transform you're already working code into this:
inputs = np.array([np.arange(num_labels) == data[i] for i in range(vocabulary_size)], dtype=np.float32)