I would like to grab satellite positions from the page(s) below, but I'm not sure if scraping is appropriate because the page appears to be updating itself every second using some internal code (it keeps updating after I disconnect from the internet). Background information can be found in my question at Space Stackexchange: A nicer way to download the positions of the Orbcomm-2 satellites.
I need a "snapshot" of four items simultaneously:
UTC time
latitude
longitude
altitude
Right now I use screen shots and manual typing. Since these values are being updated by the page - is conventional web-scraping going to work here? I found a "screen-scraping" tag, should I try to learn about that instead?
I'm looking for the simplest solution to get those four values, I wonder if I can just use urllib or urllib2 and avoid installing something new?
example page: http://www.satview.org/?sat_id=41186U I need to do 41179U through 41189U (the eleven Orbcomm-2 satellites that SpaceX just put in orbit)
One option would be to fire up a real browser and continuously poll the position in an endless loop:
import time
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.satview.org/?sat_id=41186U")
while True:
location = driver.find_element_by_css_selector("#sat_latlon .texto_track2").text
latitude, longitude = location.split("\n")[:2]
print(latitude, longitude)
time.sleep(1)
Sample output:
(u'-16.57', u'66.63')
(u'-16.61', u'66.67')
...
Here we are using selenium and Firefox - there are multiple drivers for different browsers including headless, like PhantomJS.
no need to scrape. Just look at the source html of that page and copy/paste the javascript code. None of the positions are fetched remotely...they're all calculated on the fly in the page. So just grab the code and run it yourself!
Space-Track.org's REST API seems built to handle this type of request. Once you have an account there, you can even download a sample script (updated here) to download TLES:
# STTest.py
#
# Simple Python app to extract resident space object history data from www.space-track.org into a spreadsheet
# (prior to executing, register for a free personal account at https://www.space-track.org/auth/createAccount)
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the Free Software Foundation,
# either version 3 of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU General Public License for more details.
#
# For full licencing terms, please refer to the GNU General Public License (gpl-3_0.txt) distributed with this release,
# or see http://www.gnu.org/licenses/gpl-3.0.html.
import requests
import json
import xlsxwriter
import time
import datetime
import getpass
import sys
class MyError(Exception):
def __init___(self, args):
Exception.__init__(self, "my exception was raised with arguments {0}".format(args))
self.args = args
# See https://www.space-track.org/documentation for details on REST queries
# "Find Starlinks" query finds all satellites w/ NORAD_CAT_ID > 40000 & OBJECT_NAME matching STARLINK*, 1 line per sat;
# the "OMM Starlink" query gets all Orbital Mean-Elements Messages (OMM) for a specific NORAD_CAT_ID in JSON format.
uriBase = "https://www.space-track.org"
requestLogin = "/ajaxauth/login"
requestCmdAction = "/basicspacedata/query"
requestFindStarlinks = "/class/tle_latest/NORAD_CAT_ID/>40000/ORDINAL/1/OBJECT_NAME/STARLINK~~/format/json/orderby/NORAD_CAT_ID%20asc"
requestOMMStarlink1 = "/class/omm/NORAD_CAT_ID/"
requestOMMStarlink2 = "/orderby/EPOCH%20asc/format/json"
# Parameters to derive apoapsis and periapsis from mean motion (see https://en.wikipedia.org/wiki/Mean_motion)
GM = 398600441800000.0
GM13 = GM ** (1.0 / 3.0)
MRAD = 6378.137
PI = 3.14159265358979
TPI86 = 2.0 * PI / 86400.0
# Log in to personal account obtained by registering for free at https://www.space-track.org/auth/createAccount
print('\nEnter your personal Space-Track.org username (usually your email address for registration): ')
configUsr = input()
print('Username capture complete.\n')
configPwd = getpass.getpass(prompt='Securely enter your Space-Track.org password (minimum of 15 characters): ')
# Excel Output file name - e.g. starlink-track.xlsx (note: make it an .xlsx file)
configOut = 'STText.xslx'
siteCred = {'identity': configUsr, 'password': configPwd}
# User xlsxwriter package to write the .xlsx file
print('Creating Microsoft Excel (.xlsx) file to contain outputs...')
workbook = xlsxwriter.Workbook(configOut)
worksheet = workbook.add_worksheet()
z0_format = workbook.add_format({'num_format': '#,##0'})
z1_format = workbook.add_format({'num_format': '#,##0.0'})
z2_format = workbook.add_format({'num_format': '#,##0.00'})
z3_format = workbook.add_format({'num_format': '#,##0.000'})
# write the headers on the spreadsheet
print('Starting to write outputs to Excel file create...')
now = datetime.datetime.now()
nowStr = now.strftime("%m/%d/%Y %H:%M:%S")
worksheet.write('A1', 'Starlink data from' + uriBase + " on " + nowStr)
worksheet.write('A3', 'NORAD_CAT_ID')
worksheet.write('B3', 'SATNAME')
worksheet.write('C3', 'EPOCH')
worksheet.write('D3', 'Orb')
worksheet.write('E3', 'Inc')
worksheet.write('F3', 'Ecc')
worksheet.write('G3', 'MnM')
worksheet.write('H3', 'ApA')
worksheet.write('I3', 'PeA')
worksheet.write('J3', 'AvA')
worksheet.write('K3', 'LAN')
worksheet.write('L3', 'AgP')
worksheet.write('M3', 'MnA')
worksheet.write('N3', 'SMa')
worksheet.write('O3', 'T')
worksheet.write('P3', 'Vel')
wsline = 3
def countdown(t, step=1, msg='Sleeping...'): # in seconds
pad_str = ' ' * len('%d' % step)
for i in range(t, 0, -step):
sys.stdout.write('{} for the next {} seconds {}\r'.format(msg, i, pad_str))
sys.stdout.flush()
time.sleep(step)
print('Done {} for {} seconds! {}'.format(msg, t, pad_str))
# use requests package to drive the RESTful session with space-track.org
print('Interfacing with SpaceTrack.org to obtain data...')
with requests.Session() as session:
# Need to log in first. NOTE: we get a 200 to say the web site got the data, not that we are logged in.
resp = session.post(uriBase + requestLogin, data=siteCred)
if resp.status_code != 200:
raise MyError(resp, "POST fail on login.")
# This query picks up all Starlink satellites from the catalog. NOTE: a 401 failure shows you have bad credentials.
resp = session.get(uriBase + requestCmdAction + requestFindStarlinks)
if resp.status_code != 200:
print(resp)
raise MyError(resp, "GET fail on request for resident space objects.")
# Use json package to break json-formatted response into a Python structure (a list of dictionaries)
retData = json.loads(resp.text)
satCount = len(retData)
satIds = []
for e in retData:
# each e describes the latest elements for one resident space object. We just need the NORAD_CAT_ID...
catId = e['NORAD_CAT_ID']
satIds.append(catId)
# Using our new list of resident space object NORAD_CAT_IDs, we can now get the OMM message
maxs = 1 # counter for number of sessions we have established without a pause in querying space-track.org
for s in satIds:
resp = session.get(uriBase + requestCmdAction + requestOMMStarlink1 + s + requestOMMStarlink2)
if resp.status_code != 200:
# If you are getting error 500's here, its probably the rate throttle on the site (20/min and 200/hr)
# wait a while and retry
print(resp)
raise MyError(resp, "GET fail on request for resident space object number " + s + '.')
# the data here can be quite large, as it's all the elements for every entry for one resident space object
retData = json.loads(resp.text)
for e in retData:
# each element is one reading of the orbital elements for one resident space object
print("Scanning satellite " + e['OBJECT_NAME'] + " at epoch " + e['EPOCH'] + '...')
mmoti = float(e['MEAN_MOTION'])
ecc = float(e['ECCENTRICITY'])
worksheet.write(wsline, 0, int(e['NORAD_CAT_ID']))
worksheet.write(wsline, 1, e['OBJECT_NAME'])
worksheet.write(wsline, 2, e['EPOCH'])
worksheet.write(wsline, 3, float(e['REV_AT_EPOCH']))
worksheet.write(wsline, 4, float(e['INCLINATION']), z1_format)
worksheet.write(wsline, 5, ecc, z3_format)
worksheet.write(wsline, 6, mmoti, z1_format)
# do some ninja-fu to flip Mean Motion into Apoapsis and Periapsis, and to get orbital period and velocity
sma = GM13 / ((TPI86 * mmoti) ** (2.0 / 3.0)) / 1000.0
apo = sma * (1.0 + ecc) - MRAD
per = sma * (1.0 - ecc) - MRAD
smak = sma * 1000.0
orbT = 2.0 * PI * ((smak ** 3.0) / GM) ** (0.5)
orbV = (GM / smak) ** (0.5)
worksheet.write(wsline, 7, apo, z1_format)
worksheet.write(wsline, 8, per, z1_format)
worksheet.write(wsline, 9, (apo + per) / 2.0, z1_format)
worksheet.write(wsline, 10, float(e['RA_OF_ASC_NODE']), z1_format)
worksheet.write(wsline, 11, float(e['ARG_OF_PERICENTER']), z1_format)
worksheet.write(wsline, 12, float(e['MEAN_ANOMALY']), z1_format)
worksheet.write(wsline, 13, sma, z1_format)
worksheet.write(wsline, 14, orbT, z0_format)
worksheet.write(wsline, 15, orbV, z0_format)
wsline = wsline + 1
maxs = maxs + 1
print(str(maxs))
if maxs > 18:
print('\nSnoozing for 60 secs for rate limit reasons (max 20/min and 200/hr).')
countdown(60)
maxs = 1
session.close()
workbook.close()
print('\nCompleted session.')
Related
Please can anyone advise how I may retrieve FX Forwards NDF's outright bid / outright ask or indeed any price data for USD/KRW for the 1W, 1M, 3M etc tenors.
I have attempted to follow the DAPI instructions as well as attempting to find answers via Stackoverflow to no avail. I can however succesfully retrieve live bid asks for SPOT USD KRW or even Equities such as AAPL no problem
I have tried using different combinations of the tickers, although I see no error codes no actual live prices come back? Please does anyone have any ideas to get live ticking NDF outright prices:
Any & all help is greatly appreciated :) as Bloomberg seemingly don't provide any assistance
['USD/KRW N 2M Curncy'], ['USD/KRW N 3M Curncy'] , ['USD/KRW N 3M ICAP Curncy']
p.s the Excel Bloomberg formula such as =BFxForward("USDKRW","3M","BidOutright") is essentially what I'm trying to replicate via python, attempting to follow the DAPI instructions seems to not work.
I have used the C++ BLPAPI pdf examples to attempt to get this working however no NDF examples seemingly exist.
def main_subscribe():
tickers = ['USD/KRW N 2M Curncy', 'USD/KRW N 6M Curncy', 'USD/KRW N 9M Curncy']
fields = ['BID', 'LAST_BID_TIME_TODAY_REALTIME', 'ASK','MID']
interval = 2
options = parseCmdLine()
# Fill SessionOptions
sessionOptions = blpapi.SessionOptions()
sessionOptions.setServerHost(options.host)
sessionOptions.setServerPort(options.port)
print("Connecting to %s:%s" % (options.host, options.port))
# Create a Session
session = blpapi.Session(sessionOptions)
# Start a Session
if not session.start():
print("Failed to start session.")
return
try:
# Open service to get subscription data from
if not session.openService('//blp/mktdata'):
print("Failed to open '//blp/mktdata")
return
# init subscriptions
subs = blpapi.SubscriptionList()
flds = ','.join(fields)
istr = interval and 'interval=%.1f' % interval or ''
for ticker in tickers:
subs.add(ticker, flds, istr, blpapi.CorrelationId(ticker))
session.subscribe(subs)
# Process received events
while(True):
# We provide timeout to give the chance for Ctrl+C handling:
ev = session.nextEvent(900)
for msg in ev:
print(msg)
# if ev.eventType() == blpapi.Event.SUBSCRIPTION_DATA:
# try:
# for msg in ev:
# #print(msg)
# print(f"{fields[0]}:{msg.getElementAsString(fields[0])} , {fields[3]}:{msg.getElementAsString(fields[3])} , {fields[2]}:{msg.getElementAsString(fields[2])} , {fields[1]}:{msg.getElementAsString(fields[1])}")
# except Exception as e:
# print(e)
# #print(msg)
# None
finally:
# Stop the session
session.stop()
This is the output when main-subscribe is run:
CID: {[ valueType=POINTER classId=0 value=0000024DBF510CB0 ]}
RequestId: -----------------------------
MarketDataEvents = {
MKTDATA_EVENT_TYPE = SUMMARY
MKTDATA_EVENT_SUBTYPE = INITPAINT
API_RULES_VERSION = 201411210
SIMP_LAST_PX_ALL_SESS_DIR_RT = 1
SMART_FIELDS_METADATA_VERSION_RT = "21.10.08.02 "
IS_DELAYED_STREAM = false
MID = 1.000000
RT_API_MACHINE = "apipubx0#----------"
RT_YLD_CHG_NET_1D = 0.000000
IND_BID_FLAG = false
IND_ASK_FLAG = false
BASE_PRICE_ENABLED_RT = false
EVT_DELTA_TIMES_RT = 0
ALL_PRICE_COND_CODE = ""}
This is the KRW <Curncy> FRD <Go> screen in the Bloomi Terminal:
If you hover the mouse over the 3M outright Bid (in the circle), the pop-up shows the underlying ticker to be KWN+3M BGN Curncy.
When I put this ticker in Excel as:
=BDP("KWN+3M BGN Curncy","BID","UpdateFrequency",500) then I get updating bid side pricing which matches the Terminal screen.
Since the underlying DAPI for Excel and Python is the same, I would guess that this ticker will work with the blpapi too. I usually find it is quicker to test tickers and fields in Excel.
I'm developing a Django project that will use BankID for authorization and digitally sign. I am using pybankid, and I have nothing but nice things to say about that project. My problem lies with trying to use the code provided by bankIDs documentation.
QRCode Docs
import hashlib
import hmac
import time
qr_start_token = rp_response["qrStartToken"]
# "67df3917-fa0d-44e5-b327-edcc928297f8"
qr_start_secret = rp_response["qrStartSecret"]
# "d28db9a7-4cde-429e-a983-359be676944c"
order_time = time.time()
# (The time in seconds when the response from the BankID service was delivered)
qr_time = str(int(time.time() - order_time))
# ("0" or another string with a higher number depending on order_time and current time)
qr_auth_code = hmac.new(qr_start_secret, qr_time, hashlib.sha256).hexdigest()
# "dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8" (qr_time="0")
# "949d559bf23403952a94d103e67743126381eda00f0b3cbddbf7c96b1adcbce2" (qr_time="1")
# "a9e5ec59cb4eee4ef4117150abc58fad7a85439a6a96ccbecc3668b41795b3f3" (qr_time="2")
# (64 chars hex)
qr_data = str.join(".", "bankid", qr_start_token, qr_time, qr_auth_code)
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.0.dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8" (qr_time="0")
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.1.949d559bf23403952a94d103e67743126381eda00f0b3cbddbf7c96b1adcbce2" (qr_time="1")
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.2.a9e5ec59cb4eee4ef4117150abc58fad7a85439a6a96ccbecc3668b41795b3f3" (qr_time="2")
I get TypeError: key: expected bytes or bytearray, but got 'str', when I try to convert qr_start_secret to bytes I get Unicode-objects must be encoded before hashing. I'm at a loss. Does anyone have any ideas?
EDIT: Here is my current code, and it works. Currently struggling with how to render a constantly changing QR code value client side, as context will only transfer static values.
if request.META['HTTP_USER_AGENT']:
ua_string = request.META['HTTP_USER_AGENT']
user_agent = parse(ua_string)
if user_agent.is_pc:
status=client.collect(order_ref=auth["orderRef"])["status"]
order_time = time.time()
while status == "pending":
qr_start_token = auth["qrStartToken"]
qr_start_secret = auth["qrStartSecret"]
qr_time = str(int(time.time() - order_time))
qr_auth_code = hmac.new(qr_start_secret.encode(), qr_time.encode(), hashlib.sha256).hexdigest()
qr_data = ".".join(["bankid", qr_start_token, qr_time, qr_auth_code])
print(f'qr_data: {qr_data}')
status=client.collect(order_ref=auth["orderRef"])["status"]
print(status)
qr = segno.make(qr_data)
qr.save('media/img/temp/' + personal_number + '.svg')
if status == "complete":
print("Logged on")
dj_login(request, user)
return render(request, 'home/auth-login-Success.html')
time.sleep(1)
There are multiple issues with their sample code
It looks like qr_start_token and qr_start_secret are strings.
str.join will raise error for receiving 4 arguments
Try:
import hashlib
import hmac
import time
qr_start_token = "67df3917-fa0d-44e5-b327-edcc928297f8"
qr_start_secret = "d28db9a7-4cde-429e-a983-359be676944c"
order_time = time.time()
# (The time in seconds when the response from the BankID service was delivered)
qr_time = str(int(time.time() - order_time))
# ("0" or another string with a higher number depending on order_time and current time)
print(f'qr_time: {qr_time}')
qr_auth_code = hmac.new(qr_start_secret.encode(), qr_time.encode(), hashlib.sha256).hexdigest()
# "dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8" (qr_time="0")
# "949d559bf23403952a94d103e67743126381eda00f0b3cbddbf7c96b1adcbce2" (qr_time="1")
# "a9e5ec59cb4eee4ef4117150abc58fad7a85439a6a96ccbecc3668b41795b3f3" (qr_time="2")
# (64 chars hex)
print(f'qr_auth_code: {qr_auth_code}')
print(qr_auth_code == "dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8")
qr_data = str.join(".", ["bankid", qr_start_token, qr_time, qr_auth_code])
# or better
# qr_data = ".".join(["bankid", qr_start_token, qr_time, qr_auth_code])
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.0.dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8" (qr_time="0")
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.1.949d559bf23403952a94d103e67743126381eda00f0b3cbddbf7c96b1adcbce2" (qr_time="1")
# "bankid.67df3917-fa0d-44e5-b327-edcc928297f8.2.a9e5ec59cb4eee4ef4117150abc58fad7a85439a6a96ccbecc3668b41795b3f3" (qr_time="2")
print(f'qr_data: {qr_data}')
output:
qr_time: 0
qr_auth_code: dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8
True
qr_data: bankid.67df3917-fa0d-44e5-b327-edcc928297f8.0.dc69358e712458a66a7525beef148ae8526b1c71610eff2c16cdffb4cdac9bf8
Python 3.8 , windows 10
Hi All, im pretty new to Python so please be nice :)
I am having problems on the final stage of a piece of code writing that interacts with a busAPI. (no I am not a bus spotter)
So far I have taken in data from a user
Validated the service is one we support
Retrieved pertinent data about the service
Retrieved route information for the service, and passed this to the final part where I'm meant to map it.
Unfortunately, it is only plotting one point, the end stop
I can see my functions interacting as I have copious amounts of debug logging going on so I can see data flowing between functions etc.
....
....
1
DEBUG 4: Alyssum Walk , 51.89186 , 0.93168
DEBUG 5: Alyssum Walk , 51.89186 , 0.93168
DEBUG 5.1: map_it_dict{'stop': 'Alyssum Walk', 'latitude': 51.89186, 'longitude': 0.93168}
stop latitude longitude
0 Alyssum Walk 51.89186 0.93168
1
DEBUG 4: Azalea Court , 51.89278 , 0.93334
DEBUG 5: Azalea Court , 51.89278 , 0.93334
DEBUG 5.1: map_it_dict{'stop': 'Azalea Court', 'latitude': 51.89278, 'longitude': 0.93334}
stop latitude longitude
0 Azalea Court 51.89278 0.93334
1
DEBUG 4: Library , 51.89165 , 0.93706
DEBUG 5: Library , 51.89165 , 0.93706
DEBUG 5.1: map_it_dict{'stop': 'Library', 'latitude': 51.89165, 'longitude': 0.93706}
stop latitude longitude
0 Library 51.89165 0.93706
1
DEBUG 0: Returned from bus_route(). Last Stop is Library , 51.89165 , 0.93706
As can be seen above I am getting consistent data across all my functions and my Pandas dataframe.
DEBUG 0 is the main funcntion
DEBUG 4 -comes from my function busroute. This queries a URL and returns the bust stops on the route. This is delivered with the bus stand name, and the lat and long of the bus stop. The final part of the busroute function passes the busstop, lat and long to the Map_it function
DEBUG 5 - is showing us that it is receiving the same data from the Bus_route function. finally , on line 19 you can see the pandas dataframe, again with the same information.
the ) is the index in my pandas data fram
the 1 is the number of items in the dictionary at that time ( total route is 53 items)
The data that comes in to my map_it function should plot the52 bus stops across a folium map. but it is only mapping the last one.
I think this is happening as the data that is passed over to the map_it function is being passed line by line and I suspect the dictionary is over written each time. hence the index of 0 against the datagram.
I get all the data in my bus_route function'
What bus do you want?: 64
The number 64 is a bus service we support
DEBUG 0: From validate_bus() we got 64
DEBUG 0: From bus_service(): Bus Number 64 , Traveling from: Greenstead, Essex, Traveling to: Shrub End, Essex
DEBUG 4: Hazell Avenue , 51.87154 , 0.86659
DEBUG 4: Paxman Avenue , 51.87186 , 0.86827
DEBUG 4: Alderman Blaxhill School , 51.87236 , 0.87038
...
...
...
DEBUG 4: Alyssum Walk , 51.89186 , 0.93168
DEBUG 4: Azalea Court , 51.89278 , 0.93334
DEBUG 4: Library , 51.89165 , 0.93706
DEBUG 0: Returned from bus_route(). Last Stop is Library , 51.89165 , 0.93706
Here is my code for the two functions.
def bus_route(bus_number):
# This function queries the transport API for route data for a specific bus service number
# It receives input as bus_number from main()
# it provides bus_stand, lat and long of each of the stops along the specific buses route
# and passes them to map_it() for route mapping.
bus = bus_number
# Retrieve a URL via urllib3
# This could be tidied up using data from URL but for expediancy it is coded in here
# Use %s to pass in the Constants and Variables to make up the URL
if bus == "64":
url = BASE_URL + '/route/FESX/%s/inbound/1500IM2349B/2020-05-22/06:40/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
elif bus == "65":
url = BASE_URL + '/route/FESX/%s/inbound/1500IM2456B/2020-05-22/19:27/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
elif bus == "67":
url = BASE_URL + '/route/FESX/%s/inbound/150033038003/2020-05-22/06:55/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
elif bus == "70":
url = BASE_URL + '/route/FESX/%s/inbound/1500IM77A/2020-05-22/06:51/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
elif bus == "74B":
url = BASE_URL + '/route/FESX/%s/inbound/15003303800B/2020-05-22/20:10/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
elif bus == "88":
url = BASE_URL + '/route/FESX/%s/inbound/1500IM77A/2020-05-22/05:50/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
else:
url = BASE_URL + '/route/FESX/%s/inbound/1500IM52/2020-05-22/06:00/timetable.json?app_id=%s&app_key=%s' \
'&edge_geometry=false&stops=ALL' % (bus, APP_ID, API_KEY)
http = urllib3.PoolManager()
# Request our data, and decode the json data returned
response = http.request('GET', url)
bus_route_dict = json.loads(response.data.decode('utf-8'))
x = 0
# iterate through our dictionary giving us the bus stop names and their
# lat and long so we can plot them on a map.
#while x < len(bus_route_dict['stops']):
# print(x)
for stop in bus_route_dict['stops']:
bus_stand = stop['stop_name']
lat = stop['latitude']
long = stop['longitude']
print("DEBUG 4: " + bus_stand + " , " + str(lat) + " , " + str(long))
#map_it(bus_stand, lat, long)
return bus_stand, lat, long
def map_it(bus_stand, lat, long):
# This function maps teh bus route on a folium map
# It receives input as bus_stand, lat, and long from bus_route
# Folium mapping
stop = bus_stand
latitude = lat
longitude = long
# DEBUG code to show we are receiving code from get_route()
print("DEBUG 5: " + stop + " , " + str(latitude) + " , " + str(longitude))
# Setup a dictionary to store the information we need to build
# a folium map
map_it_dict = {}
map_it_dict['stop'] = stop
map_it_dict['latitude'] = latitude
map_it_dict['longitude'] = longitude
print("DEBUG 5.1: map_it_dict" + str(map_it_dict))
# lets get the dict into pandas
map_it_df = pd.DataFrame([map_it_dict])
# Check we got data - we get it. tw 22/05/2020
#print(map_it_df.head())
# Prep data for the map
locations = map_it_df[['latitude', 'longitude']]
locationlist = locations.values.tolist()
print(len(locationlist))
# Now build the map
# the Location is the Lat/Long for Colchester
route_64 = folium.Map(location=[51.8959,0.8919] , zoom_start=14)
for point in range(0, len(locationlist)):
folium.Marker((locationlist[point]) , popup=map_it_dict['stop']).add_to(route_64)
route_64.save("route_maps/route_64.html ")
I cant see where to create my dictionary as all i seem to get out in the bus_route() function is single lines. I realise if i could get the print statement to append to the dictionary then that would be my problem solved.
I'm writing a program to toggle the lights at my house based on my iPhone's GPS coordinates. Below is what I have so far. However, I feel like there must be a better way to do this. Is there a way to get GPS data without pinging my phone every five minutes?
So far I've tried the following with no joy:
Using Shortcuts and Scriptable I tried to write some JavaScript that would trigger when I got close to home. However, I could not figure out how to use await require('wemo-client') using scriptablify. I kept getting an error, "ReferenceError: Can't find variable: require".
IFTTT does not have a variable timed trigger so the lights won't turn off after 15 minutes. Also, I plan on adding a motion sensor trigger that is unsupported.
Pythonista is $10. Yes, I am that cheap.
Apple HomeKit does not support the model I'm using, Wemo Smart Light Switch F7C030.
The code below works, but I hate that I have to ping my phone every five minutes. I'd rather save battery life by firing this code once or twice a day, as needed.
Any suggestions would be greatly appreciated.
Code:
import sys
import time
import datetime
import os
from pyicloud import PyiCloudService
import pywemo
APPLE_ID = os.getenv('APPLE_ID') # Apple ID username
APPLE_ID_PASSWORD = os.getenv('APPLE_ID_PASSWORD') # Apple ID password
API = PyiCloudService(APPLE_ID, APPLE_ID_PASSWORD)
IPHONE = API.devices[1]
LOCATION = IPHONE.location()
FIVE = 300 # 5 * 60 seconds
FIFTEEN = 900 # 15 * 60 seconds
ONEMILE = 0.01449275362318840579710144927536 # one mile is 1/69 degrees lat or long
HOMELAT = # my home's latitude
HOMELONG = # my home's longitude
WEMOS = pywemo.discover_devices()
LEN_WEMOS = range(len(WEMOS))
# Two factor authentication to retrieve iPhone data
if API.requires_2fa:
import click
print("Two-step authentication required. Your trusted devices are:")
DEVICES = API.devices
for i, device in enumerate(DEVICES):
print(" %s: %s" % (i, device.get('deviceName', "SMS to %s" % device.get('phoneNumber'))))
DEF_DEVICE = click.prompt('Which device would you like to use?', default=0)
DEVICE = DEVICES[DEF_DEVICE]
if not API.send_verification_code(DEVICE):
print("Failed to send verification code")
sys.exit(1)
CODE = click.prompt('Please enter validation code')
if not API.validate_verification_code(DEVICE, CODE):
print("Failed to verify verification code")
sys.exit(1)
# Turn off the lights when I leave
def leavehome():
timenow = datetime.datetime.now()
print("Left home on {}".format(timenow.strftime("%B %d, %Y at %H:%M:%S")))
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Turn on the lights for 15 minutes when I get home
def arrivehome():
timenow = datetime.datetime.now()
print("Arrived home on {}".format(timenow.strftime("%B %d, %Y at %H:%M:%S")))
# Loop through all Wemo devices
for wemo in LEN_WEMOS:
WEMOS[wemo].on()
time.sleep(FIFTEEN)
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Automatically turn off the lights after 15 minutes - save electricity
def timeoff():
time.sleep(FIFTEEN)
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Ping my phone for GPS data
def pingphone(prev):
mylat = LOCATION["latitude"]
mylong = LOCATION["longitude"]
logic(prev, mylat, mylong)
time.sleep(FIVE)
# Perform logic to determine if I'm home, out, arriving, or leaving
def logic(prev, lat, long):
inrange = (HOMELAT+ONEMILE >= lat >= HOMELAT-ONEMILE and HOMELONG+ONEMILE >= long >= HOMELONG-ONEMILE)
current = bool(inrange)
previous = prev
if current and not previous:
arrivehome()
elif previous and not current:
leavehome()
else:
timeoff()
pingphone(current)
# Run the script
pingphone(False)
I use a template of a python script (running on Raspberry Pi) to send sensor data (i2c) via WiFi to my PC. The problem is, the values are not static. If I start the Web application, it reads the data from the sensor only once. So, if I check the values from my PC, I can see it sent the data correctly, but they won't change.
How can I modify the script to refresh the i2c_output value, without starting the script over and over again?
Here is what I have tried so far:
import web
import sys, os
import smbus
import math
#
# Lot of initialisation... forget that part
#
accel_xout = read_word_2c(0x3b)
accel_yout = read_word_2c(0x3d)
accel_zout = read_word_2c(0x3f)
afs_sel = read_word_2c(0x28)
LSB_afs_sel = 16384.0
accel_xout_sc = accel_xout / LSB_afs_sel
accel_yout_sc = accel_yout / LSB_afs_sel
accel_zout_sc = accel_zout / LSB_afs_sel
i2c_output = str(accel_xout_sc) + str(accel_yout_sc) + str(accel_zout_sc)
urls = ( '/','Index',
)
class Index:
def GET(self):
return i2c_output
if __name__=="__main__":
app=web.application(urls,globals())
app.run()
Move the code that retrieves the sensor data into a method and invoke that method each time the index is called.
def get_sensor_output():
#
# Lot of initialisation... forget that part
#
accel_xout = read_word_2c(0x3b)
accel_yout = read_word_2c(0x3d)
accel_zout = read_word_2c(0x3f)
afs_sel = read_word_2c(0x28)
LSB_afs_sel = 16384.0
accel_xout_sc = accel_xout / LSB_afs_sel
accel_yout_sc = accel_yout / LSB_afs_sel
accel_zout_sc = accel_zout / LSB_afs_sel
i2c_output = str(accel_xout_sc) + str(accel_yout_sc) + str(accel_zout_sc)
return i2c_output
class Index:
def GET(self):
return get_sensor_output()
Note: You may want to implement some sort of cache depending on how often this get method is called. currently each call will retrieve the sensor data, which may or may not be an expensive operation that will drain the battery on your pi