I have created a windows internet speed test, I'd like to improve it and make the code more presentable as well as better define my functions.
When the computer reaches initialise, due to the variable being in another function, it cannot call it. How can I rectify this as I have various variables being called in different functions.
Feel free to use this speedtester as well, I will be working on developing a useful phone app to run the code as well.
The code prints the current date and time, searches for the connected SSID, initialises the speedtest module, scans for servers, selects the best server, initiates ping test, then download speed test, then upload speed test, followed by printing the results on screen and writing it to a simple txt file for viewing later.
Each function shows its run time using the time module and lastly total execution time with date and time also.
It works perfectly without the functions, and on android without find_ssid(): but I keep running into the trouble of localised variables.
import speedtest
from datetime import datetime
import subprocess
import re
import time
def main():
def date():
dt_now = datetime.now()
dtn = dt_now.strftime("%a %d-%m-%Y, %H:%M:%S%p")
return dtn
print(date())
def find_ssid():
stt = time.time()
cdop = subprocess.run(["netsh", "WLAN", "show", "interfaces"], capture_output=True).stdout.decode()
ssid = (re.findall("SSID : (.*)\r", cdop))
for char in ssid:
ssid = f"Network Name: {char} \n"
sid = time.time() - stt
print(f'SSID found in: {sid:.2f}s')
print(ssid)
find_ssid()
def initialise():
print("Initialising network speed test... ")
st = speedtest.Speedtest()
print("Network speed test active.")
sta = time.time() - stt
print(f'Speed test activation time: {sta - sid:.2f}s')
def scan_servers():
print("Scanning for available servers...")
st.get_servers()
print("Found available servers.")
sft = time.time() - stt
print(f'Servers found in: {sft - sta:.2f}s')
def best_server():
print("Choosing best server...")
bserv = st.get_best_server()
print(f"Best server is: {bserv['sponsor']} - {bserv['host']} located in {bserv['name']}, {bserv['country']}")
bst = time.time() - stt
print(f'Best server found in: {bst - sft:.2f}s')
def ping_test():
print("Ping testing...")
p = st.results.ping
ph = f"Ping: {p:.2f}ms"
print("Ping test complete.")
ptt = time.time() - stt
print(f'Ping test completed in: {ptt - bst:.2f}s')
def download_speed_test():
print("Download speed testing...")
ds = st.download()
dsh = f"Download speed: {ds / 1024 / 1024:.2f}mb/s"
print("Download speed test complete.")
dst = time.time() - stt
print(f'Download speed test completed in: {dst - ptt:.2f}s')
def upload_speed_test():
print("Upload speed testing...")
us = st.upload()
ust = time.time() - stt
ush = f"Upload speed: {us / 1024 / 1024:.2f}mb/s \n"
print("Upload speed test complete. \n")
print(f'Upload speed test completed in: {ust - dst:.2f}s')
def result():
print("Speed test results are: \n")
print(ssid)
print(ph)
print(dsh)
print(ush)
ttn = datetime.now()
fdt = ttn.strftime("%a %d-%m-%Y, %H:%M:%S%p")
tt = time.time() - stt
print(f"Start Time: {dtn}")
print(f"Finish Time: {fdt}")
print(f'Total execution time: {tt:.2f}s')
results = [ssid, ph, dsh, ush, dtn]
txt = "Speedtest Results.txt"
with open(txt, 'a') as f:
f.write("\n")
f.write("\n".join(results))
f.write("\n")
f.close()
main()
You can run this on, one line i believe by
ssid = (re.findall("SSID : (.*)\r", cdop))
for char in ssid:
ssid = f"Network Name: {char} \n/
Which should make it quicker, have a look at list comprehension
Related
As the title states, I am trying to do a continual listening AI named Sapphire using the speech_recognition library. For about one minute after starting the code fresh it works fine, however, after it has been running for more than a minute, the speech_to_text() takes forever to run.
Any help would be appreciated, I am looking for some form of solution to this issue. Perhaps I am not understanding the functions well enough, or there may be a way to stop the speech_to_text() function after a certain time.
I am running a texting/email version of the bot as well in addition to the voice version using threading, but I was having this problem with speech_to_text() before threading was involved.
Thank you for your help!
Here is the output:
Me --> Sapphire what time is it
speech_to_text() Time = 5.611827599990647
Sapphire --> 16:46.
Listening...
Me --> ERROR
speech_to_text() Time = 3.4650153999973554
Listening...
Me --> ERROR
speech_to_text() Time = 6.241592899998068
Listening...
Me --> ERROR
speech_to_text() Time = 12.198483600004693
Listening...
Me --> ERROR
speech_to_text() Time = 3.7981161000061547
Listening...
Me --> shoe stamps
speech_to_text() Time = 51.52946890000021
Listening...
Me --> ERROR
speech_to_text() Time = 6.57019980000041
Listening...
Me --> ERROR
speech_to_text() Time = 46.647391800011974
Listening...
Here is my code to run the Sapphire AI:
class ChatBot():
def __init__(self, name):
print("----- Starting up", name, "-----")
self.name = name
def speech_to_text(self):
recognizer = sr.Recognizer()
# with sr.Microphone(device_index=3) as mic:
with sr.Microphone() as mic:
recognizer.adjust_for_ambient_noise(mic)
print("Listening...")
audio = recognizer.listen(mic)
self.text="ERROR"
try:
self.text = recognizer.recognize_google(audio)
print("Me --> ", self.text)
except:
print("Me --> ERROR")
#staticmethod
def text_to_speech(text):
if text == "":
print("ERROR")
else:
print((ai.name+" --> "), text)
speaker = gTTS(text=text, lang="en", slow=False)
speaker.save("res.mp3")
vlc_instance = vlc.Instance("--no-video")
player = vlc_instance.media_player_new()
media = vlc_instance.media_new("res.mp3")
player.set_media(media)
player.play()
def wake_up(self, text):
return True if (self.name).lower() in text.lower() else False
def parse_input(txt):
## action time
if "time" in txt and "is" in txt and "it" in txt:
res = action_time()
elif ai.name.lower() in txt:
res = np.random.choice(
["That's me!, Sapphire!", "Hello I am Sapphire the AI", "Yes I am Sapphire!", "My name is Sapphire, okay?!", "I am Sapphire and I am alive!",
"It's-a Me!, Sapphire!"])
## respond politely
elif any(i in txt for i in ["thank", "thanks"]):
res = np.random.choice(
["you're welcome!", "anytime!", "no problem!", "cool!", "I'm here if you need me!",
"mention not."])
elif any(i in txt for i in ["exit", "close"]):
res = np.random.choice(
["Tata!", "Have a good day!", "Bye!", "Goodbye!", "Hope to meet soon!", "peace out!"])
ex = False
## conversation
else:
if txt == "ERROR":
# res="Sorry, come again?"
res = ""
else:
starttime1 = timeit.default_timer()
chat = nlp(transformers.Conversation(txt), pad_token_id=50256)
endtime1 = timeit.default_timer()
print("Transformer Time = ", (endtime1 - starttime1))
res = str(chat)
res = res[res.find("bot >> ") + 6:].strip()
return res
def sapphire_audio():
ex = True
start = 0
while ex:
starttime1 = timeit.default_timer()
ai.speech_to_text()
endtime1 = timeit.default_timer()
print("speech_to_text() Time = ", (endtime1 - starttime1))
## wake up
if ai.wake_up(ai.text) is True:
#remove Sapphire from phrase
ai.text = ai.text.lower().replace(ai.name.lower(), "", 1)
if start == 0:
res = "Hello I am Sapphire the AI, what can I do for you?"
start = 1
else:
res = parse_input(ai.text)
ai.text_to_speech(res)
if __name__ == "__main__":
os.environ["TOKENIZERS_PARALLELISM"] = "true"
# sapphire_email()
threading.Thread(target=sapphire_email).start()
threading.Thread(target=sapphire_audio).start()
First of all - try to measure what method takes that long time to execute. Is it the listen() method or recognize_google()?
Try using function recognizer.adjust_for_ambient_noise(mic) just once in the beginning, and not every time when you use speech_to_text() function and see what will happen after that.
Function recognizer.listen(mic) waits for audio from your microphone to come down to some threshold set by recognizer.adjust_for_ambient_noise(mic).
I assume that sometimes threshold is set so low and to achieve that level of ambient noise you need to wait very long time. (Check your mic in audacity? Listen to that and analyze if ambient noise changes from time to time?)
Also you are sending that audio to Google server using public API key. It's only a guess but maybe some additional delay is provided with sending long length audio data using not that great home internet upload speed. And perhaps Google, since you are sending many requests on public API key, isn't prioritizing your requests which can lead to another delay.
But it's just a guess. Try to do what I wrote at the beginning and we will figure it out.
downloadStart = datetime.now()
while (True):
requestURL = transactionAPI.format(page = tempPage,limit = 5000)
response = requests.get(requestURL,headers=headers)
json_data = json.loads(response.content)
tempMomosTransactionHistory.extend(json_data["list"])
if(datetime.fromtimestamp(json_data["list"][-1]["crtime"]) < datetime(datetime.today().year,datetime.today().month,datetime.today().day - dateRange)):
break
tempPage += 1
downloadEnd = datetime.now()
Any suggestions please threading or something like that ?
Outputs here
downloadtime 0:00:02.056010
downloadtime 0:00:05.680806
downloadtime 0:00:05.447945
You need to improve it in two ways.
Optimise code within loop
Parallelize code execution
#1
By looking at your code I can see one improvement ie. create datetime.today object instead of doing 3 times. Check other methods like transactionAPI optimise further.
#2:
If you multi core CPU machine then you take advantage of machine by spanning thread per page. Refer to modified code of above.
import threading
def processRequest(tempPage):
requestURL = transactionAPI.format(page = tempPage,limit = 5000)
response = requests.get(requestURL,headers=headers)
json_data = json.loads(response.content)
tempMomosTransactionHistory.extend(json_data["list"])
downloadStart = datetime.now()
while (True):
#create thread per page
t1 = threading.Thread(target=processRequest, args=(tempPage, ))
t1.start()
#Fetch datetime today object once instaed 3 times
datetimetoday = datetime()
if(datetime.fromtimestamp(json_data["list"][-1]["crtime"]) < datetime(datetimetoday.year,datetimetoday.month,datetimetoday.day - dateRange)):
break
tempPage += 1
downloadEnd = datetime.now()
I am trying to leverage multi processing by using the Parallel library of python. However strangely I see that the sequential execution is taking longer compared to the parallel version. Below is the code which I am running for comparison
import time
from joblib import Parallel, delayed
def compute_features(summary, article):
feature_dict = {}
feature_dict["f1"] = summary
feature_dict["f2"] = article
return feature_dict
def construct_input(n):
summaries = []
articles = []
for i in range(n):
summaries.append("summary_" + str(i))
articles.append("articles_" + str(i))
return summaries, articles
def sequential_test(n):
print("Sequential test")
start_time = time.time()
summaries, articles = construct_input(n)
feature_list = []
for i in range(n):
feature_list.append(compute_features(summaries[i], articles[i]))
total_time = time.time() - start_time
print("Total Time Sequential : %s" % total_time)
# print(feature_list)
def parallel_test(n):
print("Parallel test")
start_time = time.time()
summaries, articles = construct_input(n)
feature_list = []
executor = Parallel(n_jobs=8, backend="multiprocessing", prefer="processes", verbose=True)
# executor = Parallel(n_jobs=4, prefer="threads")
tasks = (delayed(compute_features)(summaries[i], articles[i]) for i in range(n))
results = executor(tasks)
for result in results:
feature_list.append(result)
total_time = time.time() - start_time
print("Total Time Parallel : %s" % total_time)
# print(feature_list)
if __name__ == "__main__":
n = 500000
sequential_test(n)
parallel_test(n)
I get the following output when I run the code above
Sequential test
Total Time Sequential : 1.200118064880371
Parallel test
[Parallel(n_jobs=8)]: Using backend MultiprocessingBackend with 8 concurrent workers.
[Parallel(n_jobs=8)]: Done 56 tasks | elapsed: 0.0s
[Parallel(n_jobs=8)]: Done 49136 tasks | elapsed: 1.0s
[Parallel(n_jobs=8)]: Done 500000 out of 500000 | elapsed: 4.7s finished
Total Time Parallel : 5.427206039428711
I am running this code on a mac with the following configuration
Can you guys please help me understand why this is so? And if the hardware were to change say to use a GPU would the code be any faster? Appreciate your responses. Thanks in advance.
I'm doing a telemetry application using Azure IoT Hub, Azure IoT SDK in Python and a raspberry pi with temperature and humidity sensors.
Humidity + Temperature sensors => Rasperry Pi => Azure IoT Hub
In my first implementation thanks azure examples, I used one loop that collect data from the temperature sensor and the humidity sensor, and send them to Azure IoT Hub in the same time every 60 second.
>>> 1 Loop every 60s = Collect data & send data of temperature and humidity
Now I would like to send them with different frequencies, I mean :
One loop will collect the data of the temperature sensor and send it to Azure IoT Hub every 60 seconds;
Whereas a second loop will collect the data of the humidity sensor and send it to Azure IoT Hub every 600 seconds.
>>> 1 Loop every 60s= Collect data & send data of temperature
>>> 2 Loop every 600s= Collect data & send data of humidity
I think the tool I need is multi-threading, but I don't understand which library or structure I have to implement in my case.
Here is the code provided by Azure, including one loop that handles temperature and humidity at the same time. Reading the data and sending to Azure every 60 seconds.
import random
import time
import sys
# Using the Python Device SDK for IoT Hub:
from iothub_client import IoTHubClient, IoTHubClientError,
IoTHubTransportProvider, IoTHubClientResult
from iothub_client import IoTHubMessage, IoTHubMessageDispositionResult,
IoTHubError, DeviceMethodReturnValue
# The device connection string to authenticate the device with your IoT hub.
CONNECTION_STRING = "{Your IoT hub device connection string}"
# Using the MQTT protocol.
PROTOCOL = IoTHubTransportProvider.MQTT
MESSAGE_TIMEOUT = 10000
# Define the JSON message to send to IoT Hub.
TEMPERATURE = 20.0
HUMIDITY = 60
MSG_TXT = "{\"temperature\": %.2f,\"humidity\": %.2f}"
def send_confirmation_callback(message, result, user_context):
print ( "IoT Hub responded to message with status: %s" % (result) )
def iothub_client_init():
# Create an IoT Hub client
client = IoTHubClient(CONNECTION_STRING, PROTOCOL)
return client
def iothub_client_telemetry_sample_run():
try:
client = iothub_client_init()
print ( "IoT Hub device sending periodic messages, press Ctrl-C to exit" )
#******************LOOP*******************************
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted = MSG_TXT % (temperature, humidity)
message = IoTHubMessage(msg_txt_formatted)
# Send the message.
print( "Sending message: %s" % message.get_string() )
client.send_event_async(message, send_confirmation_callback, None)
time.sleep(60)
except IoTHubError as iothub_error:
print ( "Unexpected error %s from IoTHub" % iothub_error )
return
except KeyboardInterrupt:
print ( "IoTHubClient sample stopped" )
if __name__ == '__main__':
print ( "IoT Hub Quickstart #1 - Simulated device" )
print ( "Press Ctrl-C to exit" )
iothub_client_telemetry_sample_run()
I would like to use the same structure of functions, including two loops that handles temperature and humidity, one every 60s and one every 600s.
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
msg_txt_formatted1 = MSG_TXT1 % (temperature)
message1 = IoTHubMessage(msg_txt_formatted1)
# Send the message.
print( "Sending message: %s" % message1.get_string() )
client.send_event_async(message1, send_confirmation_callback, None)
time.sleep(60)
while True:
# Build the message with simulated telemetry values.
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted2 = MSG_TXT2 % (humidity)
message2 = IoTHubMessage(msg_txt_formatted2)
# Send the message.
print( "Sending message: %s" % message2.get_string() )
client.send_event_async(message2, send_confirmation_callback, None)
time.sleep(600)
How can I do that? How to call those loops with multi-threading or another method?
It may be simpler to do something like
while True:
loop_b()
for _ in range(10):
loop_a()
time.sleep(60)
or even
while True:
time.sleep(1)
now = time.time()
if now % 60 == 0:
loop_a()
if now % 600 == 0:
loop_b()
But if you really want to use threads, then:
import threading
class LoopAThread(threading.Thread):
def run(self):
loop_a()
class LoopBThread(threading.Thread):
def run(self):
loop_b()
...
thread_a = LoopAThread()
thread_b = LoopBThread()
thread_a.start()
thread_b.start()
thread_a.join()
thread_b.join()
Here are two competing approaches to consider
Don't bother with threads at all. Just have one loop that sleeps every 60 seconds like you have now. Keep track of the last time you sent humidity data. If 600 seconds has passed, then send it. Otherwise, skip it and go to sleep for 60 seconds. Something like this:
from datetime import datetime, timedelta
def iothub_client_telemetry_sample_run():
last_humidity_run = None
humidity_period = timedelta(seconds=600)
client = iothub_client_init()
while True:
now = datetime.now()
send_temperature_data(client)
if not last_humidity_run or now - last_humidity_run >= humidity_period:
send_humidity_data(client)
last_humidity_run = now
time.sleep(60)
Rename iothub_client_telemetry_sample_run to temperature_thread_func or something like it. Create a separate function that looks just like it for humidity. Spawn two threads from the main function of your program. Set them to daemon mode so they shutdown when the user exits
from threading import Thread
def temperature_thread_func():
client = iothub_client_init()
while True:
send_temperature_data(client)
time.sleep(60)
def humidity_thread_func():
client = iothub_client_init()
while True:
send_humidity_data(client)
time.sleep(600)
if __name__ == '__main__':
temp_thread = Thread(target=temperature_thread_func)
temp_thread.daemon = True
humidity_thread = Thread(target=humidity_thread_func)
humidity_thread.daemon = True
input('Polling for data. Press a key to exit')
Notes:
If you decide to use threads, consider using an
event
to terminate them cleanly.
time.sleep is not a precise way to keep
time. You might need a different timing mechanism if the samples need
to be taken at precise moments.
pub.py
import redis
import datetime
import time
import json
import sys
import threading
import gevent
from gevent import monkey
monkey.patch_all()
def main(chan):
redis_host = '10.235.13.29'
r = redis.client.StrictRedis(host=redis_host, port=6379)
while True:
def getpkg():
package = {'time': time.time(),
'signature' : 'content'
}
return package
#test 2: complex data
now = json.dumps(getpkg())
# send it
r.publish(chan, now)
print 'Sending {0}'.format(now)
print 'data type is %s' % type(now)
time.sleep(1)
def zerg_rush(n):
for x in range(n):
t = threading.Thread(target=main, args=(x,))
t.setDaemon(True)
t.start()
if __name__ == '__main__':
num_of_chan = 10
zerg_rush(num_of_chan)
cnt = 0
stop_cnt = 21
while True:
print 'Waiting'
cnt += 1
if cnt == stop_cnt:
sys.exit(0)
time.sleep(30)
sub.py
import redis
import threading
import time
import json
import gevent
from gevent import monkey
monkey.patch_all()
def callback(ind):
redis_host = '10.235.13.29'
r = redis.client.StrictRedis(host=redis_host, port=6379)
sub = r.pubsub()
sub.subscribe(str(ind))
start = False
avg = 0
tot = 0
sum = 0
while True:
for m in sub.listen():
if not start:
start = True
continue
got_time = time.time()
decoded = json.loads(m['data'])
sent_time = float(decoded['time'])
dur = got_time - sent_time
tot += 1
sum += dur
avg = sum / tot
print decoded #'Recieved: {0}'.format(m['data'])
file_name = 'logs/sub_%s' % ind
f = open(file_name, 'a')
f.write('processing no. %s' % tot)
f.write('it took %s' % dur)
f.write('current avg: %s\n' % avg)
f.close()
def zerg_rush(n):
for x in range(n):
t = threading.Thread(target=callback, args=(x,))
t.setDaemon(True)
t.start()
def main():
num_of_chan = 10
zerg_rush(num_of_chan)
while True:
print 'Waiting'
time.sleep(30)
if __name__ == '__main__':
main()
I am testing redis pubsub to replace the use of rsh to communicate with remote boxes.
One of the things I have tested for was the number of channels affecting latency of publish and pubsub.listen().
Test: One publisher and one subscriber per channel (publisher publish every one second). Incremented the number of channels from and observed the latency (The duration from the moment publisher publish a message to the moment subscriber got the message via listen)
num of chan--------------avg latency in seconds
10:----------------------------------0.004453
50:----------------------------------0.005246
100:---------------------------------0.0155
200:---------------------------------0.0221
300:---------------------------------0.0621
Note: tested on 2 CPU + 4GB RAM + 1 NICs RHEL6.4 VM.
What can I do to maintain low latency with high number of channels?
Redis is single-threaded so increasing more cpus wont help. maybe more RAM? if so, how much more?
Anything I can do code-wise or bottleneck is in Redis itself?
Maybe the limitation comes from the way my test codes are written with threading?
EDIT:
Redis Cluster vs ZeroMQ in Pub/Sub, for horizontally scaled distributed systems
Accepted answer says "You want to minimize latency, I guess. The number of channels is irrelevant. The key factors are the number of publishers and number of subscribers, message size, number of messages per second per publisher, number of messages received by each subscriber, roughly. ZeroMQ can do several million small messages per second from one node to another; your bottleneck will be the network long before it's the software. Most high-volume pubsub architectures therefore use something like PGM multicast, which ZeroMQ supports."
From my testings, i dont know if this is true. (The claim that the number of channels is irrelevant)
For example, i did a testing.
1) One channel. 100 publishers publishing to a channel with 1 subscriber listening. Publisher publishing one second at a time. latency was 0.00965 seconds
2) Same testing except 1000 publishers. latency was 0.00808 seconds
Now during my channel testing:
300 channels with 1 pub - 1 sub resulted in 0.0621 and this is only 600 connections which is less than above testing yet significantly slow in latency