IndexError: list index out of range in loop - python

I am using Python 3 / Tweepy to create a list that contains the user names associated with various Twitter handles.
My code creates an empty dictionary, loops through the handles in the list to get the user name, saves this info in a dictionary and then appends the dictionary to a new list.
I am getting IndexError: list index out of range when I run the code. When I remove the 4th line of the for loop I do not get errors. Any thoughts on how I can resolve the issue? Why is this line of code causing errors? Thanks!
Here is my code:
def analyzer():
handles = ['#Nasdaq', '#Apple', '#Microsoft', '#amazon', '#Google', '#facebook', '#GileadSciences', '#intel']
data = []
# Grab twitter handles and append the name to data
for handle in handles:
data_dict = {}
tweets = api.user_timeline(handle)
data_dict['Handle'] = handle
data_dict['Name'] = tweets[0]['user']['name']
data.append(data_dict)

i guess main issue in below code
tweets = api.user_timeline(handle)
api.user_timeline() may returns you empty list and you are trying to access
first element from this empty list.
tweets[0]
that's why you are getting 'index out of range' issue.
you can modify your code somthing like this -
for handle in handles:
data_dict = {}
tweets = api.user_timeline(handle)
data_dict['Handle'] = handle
if tweets:
data_dict['Name'] = tweets[0]['user']['name']
data.append(data_dict)

the error is occuring beacause of the empty list which you are trying to access with index 0. you can control this by checking if list is empty or not:
def analyzer():
handles = ['#Nasdaq', '#Apple', '#Microsoft', '#amazon', '#Google', '#facebook', '#GileadSciences', '#intel']
data = []
# Grab twitter handles and append the name to data
for handle in handles:
data_dict = {}
tweets = []
tweets = api.user_timeline(handle)
if tweets:
data_dict['Handle'] = handle
data_dict['Name'] = tweets[0]['user']['name']
data.append(data_dict)

Related

Unable to append to list

if sample_collected == 'true':
b_id = list(Booking.objects.filter(center__isnull=False,org_type='homedx').values_list('id', flat=True))
for booki_id in b_id:
print("Booking ID", booki_id)
if booki_id in list(ReceivedPackageTubes.objects.values_list('booking_id', flat=True)):
packages = list(Booking.objects.filter(id=booki_id).values_list('packages__name', flat=True))
print("Packages", packages)
count_tube = []
for i in packages:
pt = package_models.PackageTubes.objects.filter(package__name=i).values_list('tubequantity__tube__name', flat=True).count()
count_tube.append(pt)
total_tubes = sum(count_tube)
print("Total Tubes", total_tubes)
collected = list(ReceivedPackageTubes.objects.filter(booking_id=booki_id).values_list('booking_id', flat=True))
print("Collected", collected)
collected_ids = []
if len(collected) == total_tubes:
collected_ids.append(booki_id)
print("Collected ID", collected_ids)
In the last line if len(collected) == total_tubes. I am trying to append **collected_id**. in collected_ids = []. Every time it is appending one. I know I need to use for loop but I am unable to do that. Can somebody help. I am learning Python and Django. Thank you !!
You're using a for loop to iterate over booki_id elements in b_id, and adding them to the collected_ids list. However, each time you process the next booki_id, you're resetting collected_ids to an empty list. Try moving the line:
collected_ids = []
outside the for loop, so that you only initialize the list once and avoid overwriting it.

I need to fill a List with API call's results with Multithreading

So I've been trying with many different methods but I can't get around it. Basically this happens:
API function call returns a Dict inside of a list.
I have a list of arguments that need to be passed to the function above one by one.
I don't care about order.
Last step is to append that list to a Pandas.DataFrame which will remove duplicates and order and etc.
Examples (btw, the API is Python-Binance):
symbols = ['ADAUSDT', 'ETHUSDT', 'BTCUSDT']
orders = pd.DataFrame()
for s in symbols:
orders = orders.append(client.get_all_orders(symbol=s)) # This returns the Dict
I tried using Queue() and Thread(), both with Lock(). I tried ThreadPoolExecutor() as well but I cannot make it work. The furthest I reached was with the last method but the amount of lines where different after each execution:
orders = pd.DataFrame()
temp = []
with ThreadPoolExecutor() as executor:
executor.map(get_orders, symbols)
for x in temp:
orders = orders.append([x])
Any ideas?
Thanks
On the sidelines, used alone orders.append([x])
this can help you
from binance import Client
client = Client(api_key, api_secret)
symbols = ['ADAUSDT', 'ETHUSDT', 'BTCUSDT']
orders_symbol =[]
orders = {}
for s in symbols:
orders.setdefault(s,{})
orders_symbol = client.get_all_orders(symbol=s)
for i in orders_symbol:
orders[s][i['orderId']] = i
print (s,i['orderId'],orders[s][i['orderId']])
print ()

How to append dictionaries to a list without overwriting data

I'm having a bit of trouble with a function I'm trying to write. What it is supposed to do is 1) go to a particular URL and get a list of financial sectors stored in a particular div; 2) visit each sector's respective page and get 3 particular pieces of information from there; 3) put the gathered collection into a dictionary; and 4) append that dictionary to another dictionary.
The desired output is a dictionary containing a list of dictionaries for all the sectors.
Here is my function:
def fsr():
fidelity_sector_report = dict()
url = "https://eresearch.fidelity.com/eresearch/goto/markets_sectors/landing.jhtml"
import requests
from bs4 import BeautifulSoup
# scrape the url page and locate links for each sector
try:
response = requests.get(url)
if not response.status_code == 200:
return 'Main page error'
page = BeautifulSoup(response.content, "lxml")
sectors = page.find_all('a',class_="heading1")
for sector in sectors:
link = 'https://eresearch.fidelity.com/' + sector['href']
name = sector.text
sect = dict()
lst = []
# scrape target pages for required information
try:
details = requests.get(link)
if not details.status_code == 200:
return 'Details page error'
details_soup = BeautifulSoup(details.content,'lxml')
fundamentals = details_soup.find('div',class_='sec-fundamentals')
values = dict()
#locate required values by addressing <tr> text and put them in a dictionary
values['Enterprise Value'] = fundamentals.select_one('th:contains("Enterprise Value") + td').text.strip()
values['Return on Equity (TTM)'] = fundamentals.select_one('th:contains("Return on Equity (TTM)") + td').text.strip()
values['Dividend Yield'] = fundamentals.select_one('th:contains("Dividend Yield") + td').text.strip()
#add values to the sector dictionary
sect[name] = values
# add the dictionary to the list
lst.append(dict(sect))
# for a dictionary using the list
fidelity_sector_report['results'] = lst
except:
return 'Something is wrong with details request'
return fidelity_sector_report
except:
return "Something is horribly wrong"
AS far as I can tell, it performs the main taks wonderfully, and the problem appears at the stage of appending a formed dictionary to a list - instead of adding new piece, it gets overwritten completely. I figured that out by putting print(lst) right after the fidelity_sector_report['results'] = lst line.
What should I change so that list (and, correspondingly, dictionary) gets formed as planned?
You should move the lst=[] outside of the sectors loop.
Your problem appears since for each sector, you reset lst and you append the current sector data to an empty list.
The following code causes the value of fidelity_sector_report['results'] to be replaced with lst.
fidelity_sector_report['results'] = lst
I presume you would want to access the respective values using a key, you can add the following line below fidelity_sector_report = dict() to initialize a dictionary:
fidelity_sector_report['results'] = {}
Then, create a key for each sector using the sector name and set the value with your values dictionary by replacing fidelity_sector_report['results'] = lst with:
fidelity_sector_report['results'][name] = dict(values)
You can access the data by using the relevant keys, i.e fidelity_sector_report['results']['Financials']['Dividend Yield'] for the dividend yield of the financials sector.

How do I extract a list a list of name from a json file in python?

I'm trying to extract list of names from json file called WW_trend, which contains a dict called trends, with a list of all the trending topics from X period of time
world_trends = set([... for trend in ...])
this code chunk is what im supposed to use (start from basically i dont know what im supposed to add)
world_trends = set([name for trend in WW_trends:
name = trend['name']])
I tried this but I get back invalid syntax.
Correct code would be:
world_trends = set()
for trend in WW_trends:
try:
world_trends.add(trend['name'])
except:
pass

Nested "for" in Django view won´t work

I want to generate a JSON type object for a HttpResponse and in order to build it i´m using a nested "for" structure. I wrote down some code, tried it with my python interpreter but when I used it on my django view it refuses to work correctly.
My structure is something like this:
tarifas = ['2.0A','2.0DHA','2.0DHSA']
terminos = ['Dia','Hora','GEN','NOC','VHC','COFGEN','COFNOC','COFVHC','PMHGEN','PMHNOC','PMHVHC','SAHGEN','SAHNOC','SAHVHC','FOMGEN','FOMNOC','FOMVHC','FOSGEN','FOSNOC','FOSVHC','INTGEN','INTNOC','INTVHC','PCAPGEN','PCAPNOC','PCAPVHC','TEUGEN','TEUNOC','TEUVHC']
data_json = {}
data_json['datos_TOT'] = []
data_json['datos_TEU'] = []
data_json['fecha'] = fecha
for i in range(3):
data_json['datos_TOT'].append({})
data_json['datos_TEU'].append({})
data_json['datos_TOT'][i]['tarifa'] = tarifas[i]
data_json['datos_TEU'][i]['tarifa'] = tarifas[i]
for j in range(0,24):
data_json['datos_TEU'][i]['values'] = []
data_json['datos_TEU'][i]['values'].append({})
data_json['datos_TEU'][i]['values'][j]['periodo'] = "{0}-{1}".format(j,j+1)
return HttpResponse(json.dumps(data_json), content_type="application/json")
In fact it has one more depth level but as the second don´t work I didn´t put it here.
With this nested structure I expected a JSON object with (b-a) entries in the first level with (d-c) entries each one. But what I see is that the second loop only returns the last value! So if the "j" loop goes from 0 to 24 it will just return "23" and nothing more. Seems like it just works one "lap".
Is there any limit in nesting loops in the views? If there is, where could I place them? I´m trying to keep the models.py free from logic.
Your problem is that you reset data_json['datos_TEU'][i]['values'] to an empty list at the beginning of every iteration of the j loop, so it will only ever have one element. Move that line to before the nested loop.
Note that your code could be written much more Pythonically:
for tarifa in tarifas:
tot = {'tarifa': tarifa}
data_json['datos_TOT'].append(tot)
teu = {'tarifa': tarifa}
values = []
for j, termino in enumerate(terminos):
value = {'termino': termino, 'periodo': "{0}-{1}".format(j,j+1)}
values.append(value)
teu['values'] = values
data_json['datos_TEU'].append(teu)

Categories