I am having quite a frustrating issue, and I'm quite new to Python/Raspberry pi
I have a script that is to match gps coordinates with a SSID filtered iwlist scan and then email the resulting file out whenever there is a wifi connection.
My issue is, the results are continuously buffered, so the gps coordinates could be significantly far away from the scanned SSIDs.
I have the serial line being flushed at the beginning of the while loop, but it doesn't seem to work. I also have python running un-buffered (-u, but that may only be for the text files..) I added Output.flush(), but I have not been able to test it yet. Figured I'd ask first.
So, my question is, is there a way to turn off the serial line buffer so every iteration is getting the GPS coordinate at the time the while loop is executing? All of my text files have file.flush() after they have been written. Does that need to be before the files are written, or would that not affect the serial buffer? What am I missing?
Any help would be appreciated
#!/usr/bin/python -u
import os
import gps
import time
import serial
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
username = 'gmail account'
password = 'account password'
port = serial.Serial("/dev/ttyAMA0", baudrate=9600)
#Setting GPS session, listen on port 2947 (gpsd) of localhost
session = gps.gps("localhost", "2947")
session.stream(gps.WATCH_ENABLE | gps.WATCH_NEWSTYLE)
#Begin GPS and WiFi Scan loop..
while True:
try:
port.flushInput()
#port.flushOutput() commented out because I haven't tested with this yet.
time.sleep(.5)
report = session.next()
#Uncomment below for report data
#print report
if report['class'] == 'TPV':
if hasattr(report, 'lat' and 'lon'):
#setting GPS variables and strings
latitude = report.lat
latString = "%f" %(latitude)
longitude = report.lon
lonString = "%f" %(longitude)
#WiFi scan and parse script. I don't think it is necessary to post,
#but if it is, I will. All text files are flushed before being closed
#Email when connected
ping = os.system('sudo ping -c4 8.8.8.8')
try:
if ping == 0:
msg = MIMEMultipart()
msg['Subject'] = "GPS/WiFi data from GPS PILOT"
msg['From'] = username
msg['To'] = username
body = "GPS/WiFi data attached.."
msg.attach(MIMEText(body, 'plain'))
part = MIMEBase('application', "octet-stream")
part.set_payload(open("/home/pi/gpsMaster/dataLog.csv", "rb").read())
Encoders.encode_base64(part)
part.add_header('Content-Disposition', 'attachment; filename="GPSWiFi_Scan.csv"')
msg.attach(part)
server = smtplib.SMTP('smtp.gmail.com:587')
server.ehlo()
server.starttls()
server.login(username, password)
server.sendmail(username, username, msg.as_string())
server.quit()
os.system("sudo rm /home/pi/gpsMaster/dataLog.csv")
else:
pass
except smtplib.SMTPException:
os.system('sudo reboot now')
#Adding loop delay
time.sleep(10)
#Loop ending exceptions
except KeyError:
pass
except IOError:
print ("ERROR")
#os.system('sudo reboot now')
except KeyboardInterrupt:
quit()
except StopIteration:
session = None
os.system("sudo reboot now")
I was having a similar issue. I fixed by flushing input and output.
port.flushInput()
port.flushOutput()
I appreciate that this is an old question, but it is still relevant as I hit the same problem. The issue is due to:
time.sleep(10)
in the main loop. The gpsd thread is adding received data to a queue which is accessed via:
report = session.next()
This mean that when you get to pull the next sample, over time it will drift further and further from reality (you are using GPS coordinates that lag behind your current location). We have a big queue, the consumer is pulling data much more slowly than it is being added due to the 10s sleep. I had much fun solving this and only by cutting the system to its bare essentials did I finally happen across the cause. The solution is to pull the data as soon as it is available, but discard as many data points as necessary until your delay time has been achieved. This stops the gpsd queue getting out of sync as the gpsd Python client is consuming them too slowly which causes coordinate drift over time.
Pseudo code (it can be polished, an exercise left for the reader):
import time
start = time.time()
while True:
# Consume the report from the gpsd
report = session.next()
# What time is it?
now = time.time()
# Magic number of 10s
if now > (start + 10):
# use the data contained in report
# Update with the latest time ready for the next iteration.
start = now
Related
I am trying to run a python script at bootup which will take a ~10 second video on applying an external input (such as push button, IR sensor etc, and in our case an ultrasonic sensor), and then mail this video to specified email addresses using the SMTPlib library of Python.
All of this is working fine. However, when using this input multiple times, the Raspberry Pi sends multiple videos (taken by pushing the button in the past) to the email address instead of the one initiated by the last input. Hence, pushing the button 1 time would send 1 video; pushing it 2 times would send a new one along with the last one; pushing it 3 times would send a new one and the last two as well, and so on.
I even tried inserting an os.remove() right after the mail is sent in the python script. After running the program, running an ls shows the file is indeed deleted. Yet somehow, these deleted videos make its way into the email. Could it be it is stored somewhere in the memory when smtplib is used?
The script in mention is below :
from email.mime.multipart import MIMEMultipart
from email import encoders
from email.message import Message
from email.mime.audio import MIMEAudio
from email.mime.base import MIMEBase
from email.mime.image import MIMEImage
from email.mime.text import MIMEText
emailfrom = "xyz#abc.com"
emailto = "xyz123#abc.com"
username = "xyz#abc.com"
password = "pass"
msg = MIMEMultipart()
msg["From"] = emailfrom
msg["To"] = emailto
msg["Subject"] = "Email Subject -- "
msg.preamble = "Email Body --"
while(True):
d=0
# Possibly the relevant section
d = pulse_d*17150
d= round(d, 2)
if(d<100):
with picamera.PiCamera() as camera:
camera.start_preview()
camera.start_recording('/home/pi/video.h264')
time.sleep(5)
camera.stop_recording()
camera.stop_preview()
time.sleep(5)
fileToSend = "/home/pi/video.h264"
ctype, encoding = mimetypes.guess_type(fileToSend)
if ctype is None or encoding is not None:
GPIO.output(test, True)
ctype = "application/octet-stream"
maintype, subtype = ctype.split("/", 1)
fp = open(fileToSend, "rb")
GPIO.output(test,False)
attachment = MIMEBase(maintype, subtype)
attachment.set_payload(fp.read())
fp.close()
encoders.encode_base64(attachment)
attachment.add_header("Content-Disposition", "attachment", filename=fileToSend)
msg.attach(attachment)
server = smtplib.SMTP("smtp.gmail.com:587")
server.starttls()
server.login(username,password)
server.sendmail(emailfrom, emailto, msg.as_string())
server.quit()
os.remove("/home/pi/video.h264")
I think you should create new message instead of reuse the old one.
Move the code where you create msg inside while loop
The problem is here:
msg.attach(attachment)
You attach files to the same message, but don't remove old attachments.
I am writing a script for a surveillance camera. I have programmed it to take a picture when motion is detected (PIR sensor) then attach the photo to an email, to a chosen recipient. However because I am running this without a screen, mouse or keyboard attached I had no way to turn the device off without pulling the plug! so I created a script to turn off the computer on a button press(this works fine on its own) however, because the rest of the code is in a loop I don't know where to place it. Keeping in mind I need to be able to turn it off any where in the code.
If you have any ideas they would be appreciated
Thanks
James
import os, re
import sys
import smtplib
import RPi.GPIO as GPIO
import time
import picamera
inport os
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import time
import RPi.GPIO as gpio
GPIO.setmode(GPIO.BCM)
GPIO_PIR = 4
print ("PIR Module Test (CTRL-C to exit)")
GPIO.setup(GPIO_PIR,GPIO.IN)
Current_State = 0
Previous_State = 0
try:
print ("Waiting for PIR to settle ...")
while GPIO.input(GPIO_PIR)==1:
Current_State = 0
print (" Ready")
while True :
Current_State = GPIO.input(GPIO_PIR)
surv_pic = open('/home/pi/Eaglecam/surveillance.jpg', 'wb')
if Current_State==1 and Previous_State==0:
print(" Motion detected!")
with picamera.PiCamera() as cam:
cam.capture(surv_pic)
surv_pic.close()
print(' Picture Taken')
SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = 587
sender = '**************'
password = "**********"
recipient = '**************'
subject = 'INTRUDER DETECTER!!'
message = 'INTRUDER ALLERT!! INTRUDER ALERT!! CHECK OUT THIS PICTURE OF THE INTRUDER! SAVE THIS PICTURE AS EVIDENCE!'
directory = "/home/pi/Eaglecam/"
def main():
msg = MIMEMultipart()
msg['Subject'] = 'INTRUDER ALERT'
msg['To'] = recipient
msg['From'] = sender
files = os.listdir(directory)
jpgsearch = re.compile(".jpg", re.IGNORECASE)
files = filter(jpgsearch.search, files)
for filename in files:
path = os.path.join(directory, filename)
if not os.path.isfile(path):
continue
img = MIMEImage(open(path, 'rb').read(), _subtype="jpg")
img.add_header('Content-Disposition', 'attachment', filename=filename)
msg.attach(img)
part = MIMEText('text', "plain")
part.set_payload(message)
msg.attach(part)
session = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
session.ehlo()
session.starttls()
session.ehlo
session.login(sender, password)
session.sendmail(sender, recipient, msg.as_string())
session.quit()
if __name__ == '__main__':
print(' Email Sent')
Previous_State=1
elif Current_State==0 and Previous_State==1:
print(" Ready")
Previous_State=0
time.sleep(0.01)
import time
import RPi.GPIO as gpio
except KeyboardInterrupt:
print('Quit')
GPIO.cleanup()
Here is the shutdown section of the script. Where would I place this in the loop?
gpio.setmode(gpio.BCM)
gpio.setup(7, gpio.IN, pull_up_down=gpio.PUD_UP)
buttonReleased = True
while buttonReleased:
gpio.wait_for_edge(7, gpio.FALLING)
buttonReleased = False
for i in range(1):
time.sleep(0.1)
if gpio.input(7):
buttonReleased = True
break
Two separate scripts would be unnecessary for this project.
You can do this a couple ways.
Create a global boolean variable called shutdownButton or whatever. Use a callback function for the GPIO pin, and set shutdownButton = True in the callback. And then change your main loop to be while not shutdownButton: instead of while True:.
You could just change your main loop to while gpio.input(7): instead of while True: (assuming you are pulling the pin up and then grounding it down with your switch).
And then finally just add a shutdown like os.system('sudo shutdown -h now') or call some other script you want to run to clean it up. The point is you just need a function to break out of your main loop when the button has been pressed and then shut down your pi at the end of the program.
There are other ways to do this also (I personally like the kernel patches from Adafruit that allow a power switch configuration to be added to /etc/modprobe.d...) but I only listed methods that applied directly to your question.
i've a python script, run on a raspeberry pi that monitoring my server room and periodically send me an e-mail and make other stuff.
First time script is running it send a notification and works fine, then wait for a time (5 or 30 minutues) depending if an alarm is triggered, so i've write a for loop to wait. when the loop is finish, script make a call of the function for sending e-mail and python return an error.
Could someone help me ?
i test it also on debian 7.7.0 box and i've the same error.
Below a sample script and the error.
Thanks in advance.
Dom.
File "/usr/lib/python2.7/smtplib.py", line 361, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed
# Import
from time import sleep
import time
import smtplib
import sys
import string
smtp_server = "10.0.0.4" # mail server address
mail_from = "server#domain.com"
mail_to = "user#gmail.com"
mail_subj = "Server room notify"
server = smtplib.SMTP(smtp_server)
valore = 24 # value for the funtion
def SendAlertEmail(t_value):
log_t = time.strftime("%d/%m/%A %H:%M:%S ")
Text = string.join("Server room notify \n " + str(log_t) + str(t_value))
Body_email = string.join((
"From: %s" % mail_from,
"To: %s" % mail_to,
"Subject: %s" % mail_subj,
"",
Text
), "\r\n")
server.sendmail(mail_from,mail_to,Body_email)
return
SendAlertEmail(valore) # calling from here works fine
for i in range(0,500):
sleep(1)
print(str(i))
SendAlertEmail(valore) # after 500 secs goes in error. Test on debian 7.7.0 python 2.7.3 gcc 4.7.2
smtplib opens a TCP connection at the creation of the SMTP object. You try to reuse it after 500 seconds of inactivity when it is likely that a timeout has already closed the connection.
You should keep only the server name, an create a new SMTP object for each connection in your use case, because the delay between sending messages is too high.
As Serge Ballesta mentions, you should open and close the connection to the server for each one the sent e-mails.
Some months ago I programmed a class to send e-mails in Python, which you can find here: https://github.com/kmundnic/Habitus/blob/master/data_sender.py You create a DataSender object and it handles the connections to the server. Also, it calls a JSON file where the accounts and password is stored, so you don't save it in your code.
Feel free to use it. It was adapted from http://kutuma.blogspot.com/2007/08/sending-emails-via-gmail-with-python.html, as it's mentioned in the comments at the beginning of the source code.
I have the script below which I'm using to send say 10 messages myself<->myself. However, I've noticed that Python really takes a while to do that. Last year I needed a system to send about 200 emails with attachments and text and I implemented it with msmtp + bash. As far as I remember it was much faster.
Moving the while loop inside (around the smtp_serv.sendmail(sender, recepient, msg) function yields similar results).
Am I doing something wrong? Surely it can't be slower than bash + msmtp (and I'm only sending a 'hi' message, no attachments).
#! /usr/bin/python3.1
def sendmail(recepient, msg):
import smtplib
# Parameters
sender = 'login#gmail.com'
password = 'password'
smtpStr = 'smtp.gmail.com'
smtpPort = 587
# /Parameters
smtp_serv = smtplib.SMTP(smtpStr, smtpPort)
smtp_serv.ehlo_or_helo_if_needed()
smtp_serv.starttls()
smtp_serv.ehlo()
recepientExists = smtp_serv.verify(recepient)
if recepientExists[0] == 250:
smtp_serv.login(sender, password)
try:
smtp_serv.sendmail(sender, recepient, msg)
except smtplib.SMTPException:
print(recepientExists[1])
else:
print('Error', recepientExists[0], ':', recepientExists[1])
smtp_serv.quit()
for in in range(10):
sendmail('receiver#gmail.com', 'hi')
In this script it takes five times more time to setup SMTP connection (5 seconds) than to send a e-mail (1 second) so it could make sense to setup a single connection and send several e-mails instead of creating the connection each time:
#!/usr/bin/env python3
import smtplib
from contextlib import contextmanager
from datetime import datetime
from email.mime.text import MIMEText
from netrc import netrc
from timeit import default_timer as timer
#contextmanager
def logined(sender, password, smtp_host='smtp.gmail.com', smtp_port=587):
start = timer(); smtp_serv = smtplib.SMTP(smtp_host, smtp_port, timeout=10)
try: # make smtp server and login
smtp_serv.ehlo_or_helo_if_needed()
smtp_serv.starttls()
smtp_serv.ehlo()
print('smtp setup took (%.2f seconds passed)' % (timer()-start,))
start = timer(); smtp_serv.login(sender, password)
print('login took %.2f seconds' % (timer()-start,))
start = timer(); yield smtp_serv
finally:
print('Operations with smtp_serv took %.2f seconds' % (timer()-start,))
start = timer(); smtp_serv.quit()
print('Quiting took %.2f seconds' % (timer()-start,))
smtp_host = 'smtp.gmail.com'
login, _, password = netrc().authenticators(smtp_host)
with logined(login, password, smtp_host) as smtp_serv:
for i in range(10):
msg = MIMEText('#%d timestamp %s' % (i, datetime.utcnow()))
msg['Subject'] = 'test #%d' % i
msg['From'] = login
msg['To'] = login
smtp_serv.send_message(msg)
Output
smtp setup took (5.43 seconds passed)
login took 0.40 seconds
Operations with smtp_serv took 9.84 seconds
Quiting took 0.05 seconds
If your Python version doesn't have .send_message() then you could use:
smtp_serv.sendmail(from, to, msg.as_string())
You are opening the connection to the SMTP server and then closing it for each email. It would be more efficient to keep the connection open while sending all of the emails.
Maybe this comes very late, but I think it is relevant for the matter.
I had the same issue recently and realized, by searching around, that the call to connect the SMTP server may be very time consuming due to issues with domain name resolution, since the SMTP server performs a reverse lookup to verify the connecting client.
In my case this call was taking around 1 minute!:
s = smtplib.SMTP(smtp_server)
Solution was to fix the domain name resolution on the Linux box. After that, connection became very quick.
Hope this may be of help.
The real answer here is "profile that code!". Time how long different parts of the code take so you know where most of the time is spent. That way you'll have a real answer without guesswork.
Still, my guess would be that it is the calls to smtp_serv.verify(recipient) may be the slow ones. Reasons might be that the server sometimes needs to ask other SMTP servers for info, or that it does throttling on these operations to avoid having spammers use them massively to gather email addresses.
Also, try pinging the SMTP server. If the ping-pong takes significant time, I would expect sending each email would take at least that long.
We are working on a project for school, and we have 2 PIR motion sensors running off an Arduino microcontroller. We are able to view the output of the serial port in both the Ardunio IDE and Python IDLE.
What we want to do next is, after about 30 seconds of motion being detected, sending out an email alert, seeing that we don't have Ethernet capability at this point, we figured the easiest way would be to grab the emailing through Python.
How to achieve this?
Update:
At this point we can send an email from Python, we can read the Arduino serial port in Python, but we just have an issue putting it all together.
This is what our Python code looks like, at the while 1: is where confusion happens:
import smtplib,serial
ser = serial.Serial(port=2, baudrate=9200)
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
import os
gmail_user = "usr#gmail.com"
gmail_pwd = "pw"
def mail(to, subject, text, attach):
msg = MIMEMultipart()
msg['From'] = gmail_user
msg['To'] = to
msg['Subject'] = subject
msg.attach(MIMEText(text))
part = MIMEBase('application', 'octet-stream')
part.set_payload(open(attach, 'rb').read())
Encoders.encode_base64(part)
part.add_header('Content-Disposition',
'attachment; filename="%s"' % os.path.basename(attach))
msg.attach(part)
mailServer = smtplib.SMTP("smtp.gmail.com", 587)
mailServer.ehlo()
mailServer.starttls()
mailServer.ehlo()
mailServer.login(gmail_user, gmail_pwd)
mailServer.sendmail(gmail_user, to, msg.as_string())
# Should be mailServer.quit(), but that crashes...
mailServer.close()
while 1: **// CONFUSION HAPPENS HERE //** <----------------------
ser.readline()
if ser.readline() = "motion"
do this mail sequence?
mail("usr2#gmail.com",
"Alarm Alert!",
"Both Motion Sensor A & B have been active for over # seconds",
"stor_fight.jpg")
Any tips would be much appreciated.
I'm not sure exactly which part of this process you're having trouble with, but here's a sketch of a solution:
You can use the pyserial library to communicate with the Arduino from python when the Arduino is plugged into the computer via USB.
On the python side, your code would look like this:
serial = serial.Serial("/dev/tty.usbserial-A6007btF", 38400) # the serial name you can see in the Arduino GUI - you might just need to say "COM1" on Windows
result = serial.readline(); # blocks until you get something from the Arduino
if result == "motion":
# send email here
On the Arduino side, you'd just do something like this:
void loop()
{
if(30 seconds have passed with motion)
Serial.println("motion");
}
Make sense?
ser.readline()
if ser.readline() = "motion"
This code reads a line from the serial port twice and discards the first one.
You also need a double-equals for comparison, and the line will (I assume...) have carriage-return characters on the end of it.
You might want to try something like
if ser.readline().starts_with("motion")
From the question, I think you already know how to read from the serial port.
So I'd suggest something like this
import time,smtplib
beginTime = time.time() + 86400 # stay one day ahead for now
while True:
if serial port has values : # ie. motion detected
beginTime = time.time()
if time.time() - beginTime > 30 :
mailObj = smtplib.SMTP('smtp_server_here', smtp_server_port)
mailObj.sendmail('from', 'to..', 'message')
beginTime = time.time() + 86400 # reset time
I hope that helps
You need to use the serial library in your Arduino sketch. See section Communication/Serial on this page:
http://arduino.cc/en/Reference/HomePage
There are examples for the serial library in folder 4.Communication of the Arduino environment.
On your host machine, use the pySerial module, as suggested by jder.
You might also find this page in the Arduino Playground to be a useful starting point:
http://www.arduino.cc/playground/Interfacing/Python
From what I can tell the board has a standard RS-232 port. You can make a listener Python script that accepts a signal from the board when the event occurs then fire off a method that sends the email. If you want a nicely designed piece look into Twisted, which has packages for serial port comm and mail. Otherwise check:
http://docs.python.org/library/email
http://www.varesano.net/blog/fabio/serial+rs232+connections+python