We Keep Coding

How to apply time.sleep() in a lambda function of a pandas dataframe?

I am translating large amounts of text using googletrans. But I keep getting this error Unexpected status code "429" from ('translate.google.com',) as I am making a lot of requests. So what I am trying to do is call time.sleep(1) after every translation is done to try and see if this will solve it.
Now I know I could do this with a for-loop where I would just loop over the text in the column tweet_text, translate the word and then sleep for a second.
But I am trying to figure out how I would incorporate the time.sleep(1) within the lambda function that translates the words in the code below.
df_bdtu['translated_tweet_text'] = df_bdtu['tweet_text'].swifter.apply(lambda x: translator.translate(x, dest="en"))

So even with time.sleep() and removing the use of swifter the issue still persisted. I ended up trying 3 different libraries but they either had the same problem where I was making too many request or they were not able to process the emojis in the data I had.
I landed on this library deep-translator which was able to use the same code with swifter and translate about 5000 rows of tweets in roughly 15 minutes.
def translate_(x):
if x['lang'] != 'en':
text = x['tweet_text']
translated = GoogleTranslator(source='auto', target='en').translate(text)
return translated
df_bdtu['translated_tweet_text'] = df_bdtu.swifter.apply(translate_,axis=1)

Python code not fetching up to date sensor values

So I am trying to fetch for light sensor values using the light sensor that is part of GrovePi. When I run the sample script here (https://github.com/DexterInd/GrovePi/blob/master/Software/Python/grove_light_sensor.py) the code works perfectly so this leads me to conclude that the script I wrote is somehow only fetching for one value and just stores it and outputs it without fetching for the new updated value. Note: I am storing my values in a dictionary and posting to Dweet.io
If anyone can lead me to the right direction that would be great, my code is below :
import time
import grovepi
light_sensor = 0
grovepi.pinMode(light_sensor,"INPUT")
def getLight():
sensor_value = grovepi.analogRead(light_sensor)
s = sensor_value
#S seems to print just one number here?
print(s)
return s
def values():
dict = {}
dict["light"] = getLight()
return dict
while True:
dict = values()
print (dweepy.dweetfor(n, dict))
time.sleep(2)

Not an engineer, but why are you adding the whole dictionary part at all? Your sample code does not do that, and dweet certainly does not need it.
My guess is that is your issue, remove it.
While True:
print (dweepy.dweetfor(s))

I am stuck with an audio/text thing (using python)

I am trying to get all the info of a .wav file by interpreting it as a text file, but using the next code:
import wave
w = wave.open('C:/Users/jorge/Desktop/Programas/Python/Datos/Si_Canciones/NSYNC - Its Gonna Be Me.wav', 'r') # :P
for i in range(5000):#w.getnframes()):
frame = w.readframes(i)
print(frame)
It prints all as I want, but at the end I get something like this:
00\x00\x00\x00\x00\x00\x00\x0
b''
b''
b''
b''
#And the b''s continue for a while
I would like to add something like this in the for, so I don't get rid off those b''s:
if (something):
break
But I don't know what that "something" could be. Can someone help me with it? :/
(I stay tuned to your answers and wish you a nice week)

The most obvious answer would be
if frame==b"":
break
But as stated in the docs, there is also a method that gives you the amount of frames; so you might want to use that; enabling you to only iterate through the existing frames. I'm not familiar with the module though.

Extracting information from unconventional text files? (Python)

I am trying to extract some information from a set of files sent to me by a collaborator. Each file contains some python code which names a sequence of lists. They look something like this:
#PHASE = 0
x = np.array(1,2,...)
y = np.array(3,4,...)
z = np.array(5,6,...)
#PHASE = 30
x = np.array(1,4,...)
y = np.array(2,5,...)
z = np.array(3,6,...)
#PHASE = 40
...
And so on. There are 12 files in total, each with 7 phase sets. My goal is to convert each phase into it's own file which can then be read by ascii.read() as a Table object for manipulation in a different section of code.
My current method is extremely inefficient, both in terms of resources and time/energy required to assemble. It goes something like this: Start with a function
def makeTable(a,b,c):
output = Table()
output['x'] = a
output['y'] = b
output['z'] = c
return output
Then for each phase, I have manually copy-pasted the relevant part of the text file into a cell and appended a line of code
fileName_phase = makeTable(a,b,c)
Repeat ad nauseam. It would take 84 iterations of this to process all the data, and naturally each would need some minor adjustments to match the specific fileName and phase.
Finally, at the end of my code, I have a few lines of code set up to ascii.write each of the tables into .dat files for later manipulation.
This entire method is extremely exhausting to set up. If it's the only way to handle the data, I'll do it. I'm hoping I can find a quicker way to set it up, however. Is there one you can suggest?

If efficiency and code reuse instead of copy is the goal, I think that Classes might provide a good way. I'm going to sleep now, but I'll edit later. Here's my thoughts: create a class called FileWithArrays and use a parser to read the lines and put them inside the object FileWithArrays you will create using the class. Once that's done, you can then create a method to transform the object in a table.
P.S. A good idea for the parser is to store all the lines in a list and parse them one by one, using list.pop() to auto shrink the list. Hope it helps, tomorrow I'll look more on it if this doesn't help a lot. Try to rewrite/reformat the question if I misunderstood anything, it's not very easy to read.

I will suggest a way which will be scorned by many but will get your work done.
So apologies to every one.
The prerequisites for this method is that you absolutely trust the correctness of the input files. Which I guess you do. (After all he is your collaborator).
So the key point here is that the text in the file is code which means it can be executed.
So you can do something like this
import re
import numpy as np # this is for the actual code in the files. You might have to install numpy library for this to work.
file = open("xyz.txt")
content = file.read()
Now that you have all the content, you have to separate it by phase.
For this we will use the re.split function.
phase_data = re.split("#PHASE = .*\n", content)
Now we have the content of each phase in an array.
Now comes for the part of executing it.
for phase in phase_data:
if len(phase.strip()) == 0:
continue
exec(phase)
table = makeTable(x, y, z) # the x, y and z are defined by the exec.
# do whatever you want with the table.
I will reiterate that you have to absolutely trust the contents of the file. Since you are executing it as code.
But your work seems like a scripting one and I believe this will get your work done.
PS : The other "safer" alternative to exec is to have a sandboxing library which takes the string and executes it without affecting the parent scope.

To avoid the safety issue of using exec as suggested by #Ajay Brahmakshatriya, but keeping his first processing step, you can create your own minimal 'phase parser', something like:
VARS = 'xyz'
def makeTable(phase):
assert len(phase) >= 3
output = Table()
for i in range(3):
line = [s.strip() for s in phase[i].split('=')]
assert len(line) == 2
var, arr = line
assert var == VARS[i]
assert arr[:10]=='np.array([' and arr[-2:]=='])'
output[var] = np.fromstring(arr[10:-2], sep=',')
return output
and then call
table = makeTable(phase)
instead of
exec(phase)
table = makeTable(x, y, z)
You could also skip all these assert statements without compromising safety, if the file is corrupted or not formatted as expected the error that will be thrown might just be harder to understand...

Help with Python loop weirdness?

I'm learning Python as my second programming language (my first real one if you don't count HTML/CSS/Javascript). I'm trying to build something useful as my first real application - an IRC bot that alerts people via SMS when certain things happen in the channel. Per a request by someone, I'm (trying) to build in scheduling preferences where people can choose not to get alerts from between hours X and Y of the day.
Anyways, here's the code I'm having trouble with:
db = open("db.csv")
for line in db:
row = line.split(",") # storing stuff in a CSV, reading out of it
recipient = row[0] # who the SMS is going to
s = row[1] # gets the first hour of the "no alert" time range
f = row[2] # gets last hour of above
nrt = [] # empty array that will store hours
curtime = time.strftime("%H") # current hour
if s == "no":
print "They always want alerts, sending email" # start time will = "no" if they always want alerts
# send mail code goes here
else:
for hour in range(int(s), int(f)): #takes start, end hours, loops through to get hours in between, stores them in the above list
nrt.append(hour)
if curtime in nrt: # best way I could find of doing this, probably a better way, like I said I'm new
print "They don't want an alert during the current hour, not sending" # <== what it says
else:
# they do want an alert during the current hour, send an email
# send mail code here
The only problem I'm having is somehow the script only ends up looping through one of the lines (or something like that) because I only get one result every time, even if I have more than one entry in the CSV file.

If this is a regular CSV file you should not try to parse it yourself. Use the standard library csv module.
Here is a short example from the docs:
import csv
reader = csv.reader(open("some.csv", "rb"))
for row in reader:
print row

There are at least two bugs in your program:
curtime = time.strftime("%H")
...
for hour in range(int(s), int(f)):
nrt.append(hour)
# this is an inefficient synonym for
# nrt = range(int(s), int(f))
if curtime in nrt:
...
First, curtime is a string, whereas nrt is a list of integers. Python is strongly typed, so the two are not interchangeable, and won't compare equal:
'4' == 4 # False
'4' in [3, 4, 5] # False
This revised code addresses that issue, and is also more efficient than generating a list and searching for the current hour in it:
cur_hour = time.localtime().tm_hour
if int(s) <= cur_hour < int(f):
# You can "chain" comparison operators in Python
# so that a op1 b op2 c is equivalent to a op1 b and b op2c
...
A second issue that the above does not address is that your program will not behave properly if the hours wrap around midnight (e.g. s = 22 and f = 8).
Neither of these problems are necessarily related to "the script only ends up looping through one of the lines", but you haven't given us enough information to figure out why that might be. A more useful way to ask questions is to post a brief but complete code snippet that shows the behavior you are observing, along with sample input and the resulting error messages, if any (along with traceback).

Have you tried something more simple? Just to see how your file is actually read by Python:
db = open("db.csv")
for line in db:
print line
There can be problem with format of your csv-file. That happens, for instance, when you open Unix file in Windows environment. In that case the whole file looks like single string as Windows and Unix have different line separators. So, I don't know certain cause of your problem, but offer to think in that direction.
Update:
Your have multiple ways through the body of your loop:
when s is "no": "They always want alerts, sending email" will be printed.
when s is not "no" and curtime in nrt: "They don't want an alert during the current hour, not sending" will be printed.
when s is not "no" and curtime in nrt is false (the last else): nothing will be printed and no other action undertaken.
Shouldn't you place some print statement in the last else branch?
Also, what is exact output of your snippet? Is it "They always want alerts, sending email"?

I would check the logic in your conditionals. You looping construct should work.

You could go thro an existing well written IRC bot in Python Download

Be explicit with what's in a row. Using 0, 1, 2...n is actually your bug, and it makes code very hard to read in the future for yourself or others. So let's use the handy tuple to show what we're expecting from a row. This sort of works like code as documentation
db = open("db.csv")
for line in db.readlines():
recipient, start_hour, end_hour = line.split(",")
nrt = []
etc...
This shows the reader of your code what you're expecting a line to contain, and it would have shown your bug to you the first time you ran it :)