today I decided to transform my little script, based on gst-launch, into a real Python/GStreamer application, in order to add some features.
I developed a little program which send the audio from my microphone to both Icecast (shout2send) and local storage (filesink) thanks to tee.
Sometimes shout2send can halt, due to network problems. I would like to restart this element every N seconds until the connection is back, without stopping the pipeline, because the local audio file should not be impacted from network conditions.
Here's what I tried:
stopping/starting the pipeline one second after the network error (result: streaming works, local file is truncated)
unlink from tee, set shout2send state to NULL and removing it from pipeline (result: GStreamer critical errors like Trying to dispose element ... but it is in PLAYING instead of the NULL state)
Trying to understand how to use pads in this case (result: same as above, but with more code involved)
What should I do?
Here's how my code looks like :
import gi
gi.require_version("Gst", "1.0")
from gi.repository import GLib
from gi.repository import Gst
# [...]
def message_handler(bus, message):
if message.type == Gst.MessageType.ERROR:
if message.src == shout2send:
pass # TODO: restart the element
else:
print(message.parse_error())
pipeline.set_state(Gst.State.NULL)
exit(1)
else:
print(message.type)
pipeline = Gst.Pipeline()
message_bus = pipeline.get_bus()
message_bus.add_signal_watch()
message_bus.connect('message', message_handler)
# [...]
tee.link(queue0)
queue0.link(filesink)
tee.link(queue1)
queue1.link(shout2send)
Update (9/12/15): non-working code added + log
I tried to follow "Dynamically changing the pipeline" fro GStreamer doc, but my code doesn't work.
def event_probe(pad, info, *args):
Gst.Pad.remove_probe(pad, info)
queue1.unlink(shout2send)
tee.unlink(queue1)
pipeline.remove(shout2send)
pipeline.remove(queue1)
return Gst.PadProbeReturn.OK
def message_handler(bus, message):
if message.type == Gst.MessageType.ERROR:
if message.src == shout2send:
pad = queue1.get_static_pad('src')
pad.add_probe(Gst.PadProbeType.BLOCK_DOWNSTREAM, event_probe, None)
else:
print(message.parse_error())
pipeline.set_state(Gst.State.NULL)
exit(1)
else:
print(message.type)
Here's what I see if I run my script with GST_DEBUG=3 and I restart Icecast while streaming:
[...]
0:00:02.142033258 5462 0x55e414d900a0 WARN shout2 gstshout2.c:674:gst_shout2send_render:<shout2send> error: shout_send() failed: Socket error
0:00:02.658137998 5462 0x55e414d90140 WARN basesrc gstbasesrc.c:2943:gst_base_src_loop:<pulsesrc> error: Internal data flow error.
0:00:02.658169752 5462 0x55e414d90140 WARN basesrc gstbasesrc.c:2943:gst_base_src_loop:<pulsesrc> error: streaming task paused, reason error (-5)
(GLib.Error('Internal data flow error.', 'gst-stream-error-quark', 1), 'gstbasesrc.c(2943): gst_base_src_loop (): /GstPipeline:pipeline0/GstPulseSrc:pulsesrc:\nstreaming task paused, reason error (-5)')
0:00:02.658628129 5462 0x7f6ba8002a30 WARN audiosrc gstaudiosrc.c:244:audioringbuffer_thread_func:<pulsesrc> error reading data -1 (reason: Success), skipping segment
Thanks to otopolsky's comments I did it :)
What I did wrong:
elements must be set to NULL: this is very important
oggmux must stay after tee, on both sub-pipelines: otherwise Icecast will list the stream without be able to serve it. Do the same for opusenc
Advice:
Is not necessary to unlink every element you don't need: just break where needed
Is not necessary to remove from the pipeline every element you don't need: keep them if you think to reuse them
Final code (reconnection works correctly and independently from local encoding/recording):
def event_probe2(pad, info, *args):
Gst.Pad.remove_probe(pad, info.id)
tee.link(opusenc1)
opusenc1.set_state(Gst.State.PLAYING)
oggmux1.set_state(Gst.State.PLAYING)
queue1.set_state(Gst.State.PLAYING)
shout2send.set_state(Gst.State.PLAYING)
return Gst.PadProbeReturn.OK
def reconnect():
pad = tee.get_static_pad('src_1')
pad.add_probe(Gst.PadProbeType.BLOCK_DOWNSTREAM, event_probe2, None)
def event_probe(pad, info, *args):
Gst.Pad.remove_probe(pad, info.id)
tee.unlink(opusenc1)
opusenc1.set_state(Gst.State.NULL)
oggmux1.set_state(Gst.State.NULL)
queue1.set_state(Gst.State.NULL)
shout2send.set_state(Gst.State.NULL)
GLib.timeout_add_seconds(interval, reconnect)
return Gst.PadProbeReturn.OK
def message_handler(bus, message):
if message.type == Gst.MessageType.ERROR:
if message.src == shout2send:
pad = tee.get_static_pad('src_1')
pad.add_probe(Gst.PadProbeType.BLOCK_DOWNSTREAM, event_probe, None)
else:
print(message.parse_error())
pipeline.set_state(Gst.State.NULL)
exit(1)
else:
print(message.type)
Minor problems:
I use tee.get_static_pad('src_1'), but I think I could get the src id somewhere, instead of using a fixed value
Probably the whole thing could be written in a better form (but this is my first program with Python+Gstreamer and it works, so I'm fine with it)
In order to avoid data loss I call pipeline.set_state(Gst.State.NULL) one second after pipeline.send_event(Gst.Event.new_eos()), but I still get messages like WARN audiosrc gstaudiosrc.c:244:audioringbuffer_thread_func:<pulsesrc> error reading data -1 (reason: Success), skipping segment
Code: https://github.com/ViGLug/libre-streaming
Related
Please note: this is a self answered question for reference.
Using Python how does one display a notification:
a) with a timeout and
b) have the timeout visibly countdown.
Update: Just 2 weeks after posting this, I updated to Mint 19 (Ubuntu 18.04) the result, the timer function described below has vanished in the Standard Notification. I can only suppose that the move to GTK+3 has virtually hidden the timer. It is there but barely visible.
Choosing Notification style Nodoka or Coco in Control Centre --> Popup Notifications, does display the timer properly.
End Update
Using the standard notification modules notify2 and Notify in the gi.repository, one simply has to add an action, even though you have no intention of using it.
Note: there appears to be no documented reason for this, that I have found.
In addition to a close button being added to the notification, it also supplies a clockface that reduces based on the timeout supplied.
for notify2:
import notify2
class Notify():
def mess_callback():
pass
def __init__(self,parent,caption,msg,timeout=None,urgency=None):
if timeout != None: pass
else: timeout = 0 # message should not timeout
if urgency: pass
else: urgency = 0
img = '/home/rolf/MyApp.png'
caps = notify2.get_server_caps()
mess = notify2.Notification(caption,msg,img) # passing an image is optional
mess.set_timeout(timeout) #milliseconds
mess.set_urgency(urgency) #0-Low, 1-Normal, 2-Critical
# Without the following `add_action` option, No countdown to the time out is shown
if timeout != 0 and 'actions' in caps:
mess.add_action("close","Close",self.mess_callback,None) #Show the countdown to close
mess.show()
if __name__ == "__main__":
notify2.init("MyApp") #Register MyApp
Notify(None,"Error","This message is not timed and has to be manually cancelled")
Notify(None,"Error 2","This message will timeout after the default value",timeout=-1)
Notify(None,"Information","An Unimportant message",timeout=20000,urgency=0)
Notify(None,"Attention","An Important message",timeout=20000,urgency=1)
Notify(None,"Emergency","A Critical message",timeout=20000,urgency=2)
notify2.uninit() #Un-register
Using Notify in the gi.repository:
import gi
gi.require_version('Notify', '0.7')
from gi.repository import Notify
class Message():
def mess_callback():
pass
def __init__(self,parent,caption,msg,timeout=None,urgency=None):
if timeout != None: pass
else: timeout = 0 # message should not timeout
if urgency: pass
else: urgency = 0
img = '/home/rolf/MyApp.png'
caps = Notify.get_server_caps()
mess = Notify.Notification.new(caption, msg, img) # passing an image is optional
mess.set_timeout(timeout) #milliseconds
mess.set_urgency(urgency) #0-Low, 1-Normal, 2-Critical
# Without the following `add_action` option, No countdown to the time out is shown
if timeout != 0 and 'actions' in caps:
mess.add_action("close","Close",self.mess_callback,None) #Show the countdown to close
mess.show()
if __name__ == "__main__":
Notify.init("MyApp") #Register MyApp
Message(None,"Error","This message is not timed and has to be manually cancelled")
Message(None,"Error 2","This message will timeout after the default value",timeout=-1)
Message(None,"Information","An Unimportant message",timeout=20000,urgency=0)
Message(None,"Attention","An Important message",timeout=20000,urgency=1)
Message(None,"Emergency","A Critical message",timeout=20000,urgency=2)
I'm trying to use notifications for temperature updates on a Sensirion Smartgadget. So far everything works: connecting, reading data by polling.
I don't know how to enable the notifications on the Sensirion Smartgadget. What do I have to send to which characteristic?
The only documentation I found was this on page 10.
Since now I tried the following which did not work:
from bluepy import btle
from bluepy.btle import Peripheral
class MyDelegate(btle.DefaultDelegate):
def __init__(self):
btle.DefaultDelegate.__init__(self)
def handleNotification(self, cHandle, data):
print('notification arrived')
p = Peripheral(myAddress, "random")
p.setDelegate( MyDelegate() )
svc = p.getServiceByUUID( "00002234-b38d-4985-720e-0f993a68ee41" )
ch = svc.getCharacteristics( "00002235-b38d-4985-720e-0f993a68ee41" )[0]
ch.write((1).to_bytes(1, byteorder='little'))
ch.write((1).to_bytes(2, byteorder='big'))
ch.write((1).to_bytes(2, byteorder='little'))
ch.write((2).to_bytes(1, byteorder='big'))
ch.write((2).to_bytes(1, byteorder='little'))
ch.write((2).to_bytes(2, byteorder='big'))
ch.write((2).to_bytes(2, byteorder='little'))
while True:
if p.waitForNotifications(1.0):
continue
print("Waiting...")
Restarted investigating in the problem and found a brute force solution I didn't see back then...
Iterated over all the results of getCharacteristics and wrote 0x01 to them. Turned out I needed to write to the third characteristic the value to turn on notifications.
I started an API on GitHub for the Sensirion, which can be found here.
I'm a beginner programmer and my goal is to design a utility that can accept an sms command and run a corresponding function on my main pc remotely. I'm using a variety of tools to do this (IFTTT, Dropbox, Twilio, Task Scheduler), but I've run into a little trouble along the way.
Most recently I've tried running the main code through a function which I've named 'main()'. However, when I attempt to call the function, command line throws an error:
Traceback (most recent call last):
File "C:\Python_Files\WIP\Check.py", line 104, in <module>
main()
NameError: name 'main' is not defined
I will post my code here for context (some info is edited for security):
#DESCRIPTION:
#Check if C:/Users/Thermaltake/Dropbox/Remote_Control contains a .txt file
#set count to corresponding integer
#delete file
#call function of corresponding integer
#reset count
#Set Task Scheduler to run this program every 5 minutes when idle.
import os
import time
import subprocess
import sys
import collections
import webbrowser
import logging
from twilio.rest import Client
#Twilio Account Info
account_sid = 'id'#not included for security
auth_token = 'token'#not included for security
client = Client(account_sid, auth_token)
myTwilioNumber = '+myTwilioNumber'#not included for security
myCellPhone = '+MyCellPhone'#not included for security
##FUNCTIONS##
#shutdown command
def shutdown():
os.system('shutdown -s')
#restart command
def restart():
os.system('shutdown -r')
#wake up computer and launch core programs
def wakeup():
subprocess.Popen('C:/Users/Thermaltake/AppData/Roaming/Spotify/Spotify.exe')
webbrowser.open('https://us-mg6.mail.yahoo.com/neo/launch')
webbrowser.open('https://www.rescuetime.com/dashboard')
#Launch main coding applications
def code():
webbrowser.open('http://somafm.com/player/#/now-playing/groovesalad')
subprocess.Popen('C:\Windows\system32\cmd.exe')
subproces.Popen('C:\Python_Files\Sublime Text 3\sublime_text.exe')
#Launch Remote Desktop and automatically log in
def logmein():
def main(): #main code
x=0 #counter
#checks for txt file in Remote_Control folder and assigns value to x
if os.path.isfile('C:/Users/Thermaltake/Dropbox/Remote_Control/shutdown.txt')==True:
x=1
elif os.path.isfile('C:/Users/Thermaltake/Dropbox/Remote_Control/wakeup.txt')==True:
x=2
elif os.path.isfile('C:/Users/Thermaltake/Dropbox/Remote_Control/restart.txt')==True:
x=3
elif os.path.isfile('C:/Users/Thermaltake/Dropbox/Remote_Control/code.txt')==True:
x=4
else:
print('No File Found')
#Checks x value and executes function
if x==1:
os.remove('C:/Users/Thermaltake/Dropbox/Remote_Control/shutdown.txt')
shutdown()
#print('Shutdown')#Placeholder for testing
message = client.messages.create(body='Shutdown Initiated', from_=, to=)#not included for security
elif x==2:
os.remove('C:/Users/Thermaltake/Dropbox/Remote_Control/wakeup.txt')
wakeup()
#print ('Spotify') #Placeholder. Define function to wake up PC and launch core programs
message = client.messages.create(body='Waking Up', from_=, to=)#not included for security
elif x==3:
os.remove('C:/Users/Thermaltake/Dropbox/Remote_Control/restart.txt')
restart()
#print ('Restart') #Placeholder.
message = client.messages.create(body='Restart Initiated', from_=, to=)#not included for security
elif x==4:
os.remove('C:/Users/Thermaltake/Dropbox/Remote_Control/code.txt')
code()
print('Happy Coding!')
message = client.messages.create(body='Ready to Code!', from_=, to=)#not included for security
else:
print ('No command entered')
#End Sequence (prints value and resets counter)
print (x)
x=0
os.system('pause') #Placeholder for testing
if __name__ == '__main__': #Runs main function
main()
'''
TODO:
Twilio not yet working. try different iterations of body and message. try assigning it to its own function
subprocess failing to launch programs. research alternatives. SMTP is a possibility.
possibly need to add Enter keystroke to the end of shutdown() and restart(). maybe even log in to log off.
add 'send to-do list to phone' function.
cleanup indentation and remove unnecessary modules
add 'flvto' function
add 'remote in' function.
+ Version 1.1 7/6/17 +
WORKING:
subprocess.Popen('C:\Windows\system32\cmd.exe')
webbrowser.open('http://somafm.com/player/#/now-playing/groovesalad')
os.remove
wakeup()
NOT WORKING:
subproces.Popen('C:\Python_Files\Sublime Text 3\sublime_text.exe')
Twilio return message
Task Scheduler: "Check_Dropbox on Lock"
BUGS:
IFTTT is buggy and sometimes unresponsive
Task Scheduler to check file is too slow. Need something faster. (Possibly 'Watch 4 Folder')
shutdown() waits til logon to initialize.
restart() waits til logon to initialize.
'''
Any help would be enormously appreciated. Keep in mind I have no formal education in CS, but I'm just trying to get my feet wet before starting a CS major this fall. I've only been programming for about 3 months.
also, if anything in this code can be done more elegantly, I will happily take advice.
-Jake
remove main() from inside logmein() so its in the global scope. The error you are getting is because when you call main from the global scope it isn't defined.
Let's say I have a Service that I've written that's based on win32serviceutil.ServiceFramework like so:
class Launcher(win32serviceutil.ServiceFramework):
_svc_name_ = "QueueFolders"
_svc_display_name_ = "Queue Hot Folders"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
def SvcStop(self):
sys.stopservice = True
def SvcDoRun(self):
# A call to something Main() like here.
QueueFolders()
In this example calling QueueFolders instantiates an object that should be long-running if proper configuration has been done. If not, it raises an exception and I get this in my command prompt:
j:\EclipseWorkspace\Queue Folders\dist>net start queuefolders
The Queue Hot Folders service is starting.
The Queue Hot Folders service could not be started.
The service did not report an error.
More help is available by typing NET HELPMSG 3534.
I'm wondering, how would I report this error? I know I can try / catch my instantiation, but I've tried self.ReportServiceStatus(win32service.SERVICE_ERROR_CRITICAL) and still don't seem to be able to report anything meaningful back to the user (ideally, something in the form of "The service could not start, please check configuration."
If SvcDoRun has a problem, you can call:
self.ReportServiceStatus(win32service.SERVICE_STOPPED, win32ExitCode=exit_code)
Where exit_code is a Windows system error code. Maybe you can find one of those that matches the error situation well enough. For example, if exit_code is winerror.ERROR_STACK_OVERFLOW, you'd see the following from net start QueueFolders:
The Queue Hot Folders service is starting.
The Queue Hot Folders service could not be started.
A system error has occurred.
System error 1001 has occurred.
Recursion too deep; the stack overflowed.
A similar message is displayed using the GUI service manager.
Furthermore, you can write to the Windows event log, which is a typical place to report service messages. To do that, use the servicemanager module.
For example:
def SvcDoRun(self):
# oops something went wrong
import servicemanager
servicemanager.LogErrorMsg("Couldn't start - check your configuration")
Then, checking Event Viewer, in the Application section of the Windows Logs, you'd see your message.
The next script I'm using is used to listen to IMAP connection using IMAP IDLE and depends heavily on threads. What's the easiest way for me to eliminate the treads call and just use the main thread?
As a new python developer I tried editing def __init__(self, conn): method but just got more and more errors
A code sample would help me a lot
#!/usr/local/bin/python2.7
print "Content-type: text/html\r\n\r\n";
import socket, ssl, json, struct, re
import imaplib2, time
from threading import *
# enter gmail login details here
USER="username#gmail.com"
PASSWORD="password"
# enter device token here
deviceToken = 'my device token x x x x x'
deviceToken = deviceToken.replace(' ','').decode('hex')
currentBadgeNum = -1
def getUnseen():
(resp, data) = M.status("INBOX", '(UNSEEN)')
print data
return int(re.findall("UNSEEN (\d)*\)", data[0])[0])
def sendPushNotification(badgeNum):
global currentBadgeNum, deviceToken
if badgeNum != currentBadgeNum:
currentBadgeNum = badgeNum
thePayLoad = {
'aps': {
'alert':'Hello world!',
'sound':'',
'badge': badgeNum,
},
'test_data': { 'foo': 'bar' },
}
theCertfile = 'certif.pem'
theHost = ('gateway.push.apple.com', 2195)
data = json.dumps(thePayLoad)
theFormat = '!BH32sH%ds' % len(data)
theNotification = struct.pack(theFormat, 0, 32,
deviceToken, len(data), data)
ssl_sock = ssl.wrap_socket(socket.socket(socket.AF_INET,
socket.SOCK_STREAM), certfile=theCertfile)
ssl_sock.connect(theHost)
ssl_sock.write(theNotification)
ssl_sock.close()
print "Sent Push alert."
# This is the threading object that does all the waiting on
# the event
class Idler(object):
def __init__(self, conn):
self.thread = Thread(target=self.idle)
self.M = conn
self.event = Event()
def start(self):
self.thread.start()
def stop(self):
# This is a neat trick to make thread end. Took me a
# while to figure that one out!
self.event.set()
def join(self):
self.thread.join()
def idle(self):
# Starting an unending loop here
while True:
# This is part of the trick to make the loop stop
# when the stop() command is given
if self.event.isSet():
return
self.needsync = False
# A callback method that gets called when a new
# email arrives. Very basic, but that's good.
def callback(args):
if not self.event.isSet():
self.needsync = True
self.event.set()
# Do the actual idle call. This returns immediately,
# since it's asynchronous.
self.M.idle(callback=callback)
# This waits until the event is set. The event is
# set by the callback, when the server 'answers'
# the idle call and the callback function gets
# called.
self.event.wait()
# Because the function sets the needsync variable,
# this helps escape the loop without doing
# anything if the stop() is called. Kinda neat
# solution.
if self.needsync:
self.event.clear()
self.dosync()
# The method that gets called when a new email arrives.
# Replace it with something better.
def dosync(self):
print "Got an event!"
numUnseen = getUnseen()
sendPushNotification(numUnseen)
# Had to do this stuff in a try-finally, since some testing
# went a little wrong.....
while True:
try:
# Set the following two lines to your creds and server
M = imaplib2.IMAP4_SSL("imap.gmail.com")
M.login(USER, PASSWORD)
M.debug = 4
# We need to get out of the AUTH state, so we just select
# the INBOX.
M.select("INBOX")
numUnseen = getUnseen()
sendPushNotification(numUnseen)
typ, data = M.fetch(1, '(RFC822)')
raw_email = data[0][1]
import email
email_message = email.message_from_string(raw_email)
print email_message['Subject']
#print M.status("INBOX", '(UNSEEN)')
# Start the Idler thread
idler = Idler(M)
idler.start()
# Sleep forever, one minute at a time
while True:
time.sleep(60)
except imaplib2.IMAP4.abort:
print("Disconnected. Trying again.")
finally:
# Clean up.
#idler.stop() #Commented out to see the real error
#idler.join() #Commented out to see the real error
#M.close() #Commented out to see the real error
# This is important!
M.logout()
As far as I can tell, this code is hopelessly confused because the author used the "imaplib2" project library which forces a threading model which this code then never uses.
Only one thread is ever created, which wouldn't need to be a thread but for the choice of imaplib2. However, as the imaplib2 documentation notes:
This module presents an almost identical API as that provided by the standard python library module imaplib, the main difference being that this version allows parallel execution of commands on the IMAP4 server, and implements the IMAP4rev1 IDLE extension. (imaplib2 can be substituted for imaplib in existing clients with no changes in the code, but see the caveat below.)
Which makes it appear that you should be able to throw out much of class Idler and just use the connection M. I recommend that you look at Doug Hellman's excellent Python Module Of The Week for module imaplib prior to looking at the official documentation. You'll need to reverse engineer the code to find out its intent, but it looks to me like:
Open a connection to GMail
check for unseen messages in Inbox
count unseen messages from (2)
send a dummy message to some service at gateway.push.apple.com
Wait for notice, goto (2)
Perhaps the most interesting thing about the code is that it doesn't appear to do anything, although what sendPushNotification (step 4) does is a mystery, and the one line that uses an imaplib2 specific service:
self.M.idle(callback=callback)
uses a named argument that I don't see in the module documentation. Do you know if this code ever actually ran?
Aside from unneeded complexity, there's another reason to drop imaplib2: it exists independently on sourceforge and PyPi which one maintainer claimed two years ago "An attempt will be made to keep it up-to-date with the original". Which one do you have? Which would you install?
Don't do it
Since you are trying to remove the Thread usage solely because you didn't find how to handle the exceptions from the server, I don't recommend removing the Thread usage, because of the async nature of the library itself - the Idler handles it more smoothly than a one thread could.
Solution
You need to wrap the self.M.idle(callback=callback) with try-except and then re-raise it in the main thread. Then you handle the exception by re-running the code in the main thread to restart the connection.
You can find more details of the solution and possible reasons in this answer: https://stackoverflow.com/a/50163971/1544154
Complete solution is here: https://www.github.com/Elijas/email-notifier