Is it possible -- other than by using something like a .txt/dummy file -- to pass a value from one program to another?
I have a program that uses a .txt file to pass a starting value to another program. I update the value in the file in between starting the program each time I run it (ten times, essentially simultaneously). Doing this is fine, but I would like to have the 'child' program report back to the 'mother' program when it is finished, and also report back what files it found to download.
Is it possible to do this without using eleven files to do it (that's one for each instance of the 'child' to 'mother' reporting, and one file for the 'mother' to 'child')? I am talking about completely separate programs, not classes or functions or anything like that.
To operate efficently, and not be waiting around for hours for everything to complete, I need the 'child' program to run ten times and get things done MUCH faster. Thus I run the child program ten times and give each program a separate range to check through.
Both programs run fine, I but would like to get them to run/report back and forth with each other and hopefully not be using file 'transmission' to accomplish the task, especially on the child-mother side of the transferring of data.
'Mother' program...currently
import os
import sys
import subprocess
import time
os.chdir ('/media/')
#find highest download video
Hival = open("Highest.txt", "r")
Histr = Hival.read()
Hival.close()
HiNext = str(int(Histr)+1)
#setup download #1
NextVal = open("NextVal.txt","w")
NextVal.write(HiNext)
NextVal.close()
#call download #1
procs=[]
proc=subprocess.Popen(['python','test.py'])
procs.append(proc)
time.sleep(2)
#setup download #2-11
Histr2 = int(Histr)/10000
Histr2 = Histr2 + 1
for i in range(10):
Hiint = str(Histr2)+"0000"
NextVal = open("NextVal.txt","w")
NextVal.write(Hiint)
NextVal.close()
proc=subprocess.Popen(['python','test.py'])
procs.append(proc)
time.sleep(2)
Histr2 = Histr2 + 1
for proc in procs:
proc.wait()
'Child' program
import urllib
import os
from Tkinter import *
import time
root = Tk()
root.title("Audiodownloader")
root.geometry("200x200")
app = Frame(root)
app.grid()
os.chdir('/media/')
Fileval = open('NextVal.txt','r')
Fileupdate = Fileval.read()
Fileval.close()
Fileupdate = int(Fileupdate)
Filect = Fileupdate/10000
Filect2 = str(Filect)+"0009"
Filecount = int(Filect2)
while Fileupdate <= Filecount:
root.title(Fileupdate)
url = 'http://www.yourfavoritewebsite.com/audio/encoded/'+str(Fileupdate)+'.mp3'
urllib.urlretrieve(url,str(Fileupdate)+'.mp3')
statinfo = os.stat(str(Fileupdate)+'.mp3')
if statinfo.st_size<10000L:
os.remove(str(Fileupdate)+'.mp3')
time.sleep(.01)
Fileupdate = Fileupdate+1
root.update_idletasks()
I'm trying to convert the original VB6 program over to Linux and make it much easier to use at the same time. Hence the lack of .mainloop being missing. This was my first real attempt at anything in Python at all hence the lack of def or classes. I'm trying to come back and finish this up after 1.5 months of doing nothing with it mostly due to not knowing how to. In research a little while ago I found this is WAY over my head. I haven't ever did anything with threads/sockets/client/server interaction so I'm purely an idiot in this case. Google anything on it and I just get brought right back here to stackoverflow.
Yes, I want 10 running copies of the program at the same time, to save time. I could do without the gui interface if it was possible for the program to report back to 'mother' so the mother could print on the screen the current value that is being searched. As well as if the child could report back when its finished and if it had any file that it downloaded successfully(versus downloaded and then erased due to being to small). I would use the successful download information to update Highest.txt for the next time the program got ran.
I think this may clarify things MUCH better...that or I don't understand the nature of using server/client interaction:) Only reason time.sleep is in the program was due to try to make sure that the files could get written before the next instance of the child program got ran. I didn't know for sure what kind of timing issue I may run into so I included those lines for safety.
This can be implemented using a simple client/server topology using the multiprocessing library. Using your mother/child terminology:
server.py
from multiprocessing.connection import Listener
# client
def child(conn):
while True:
msg = conn.recv()
# this just echos the value back, replace with your custom logic
conn.send(msg)
# server
def mother(address):
serv = Listener(address)
while True:
client = serv.accept()
child(client)
mother(('', 5000))
client.py
from multiprocessing.connection import Client
c = Client(('localhost', 5000))
c.send('hello')
print('Got:', c.recv())
c.send({'a': 123})
print('Got:', c.recv())
Run with
$ python server.py
$ python client.py
When you talk about using txt to pass information between programs, we first need to know what language you're using.
Within my knowledge of Java and Python achi viable despite laborious depensendo the amount of information that wants to work.
In python, you can use the library that comes with it for reading and writing txt and schedule execution, you can use the apscheduler.
I have a simple Python program creates an RTSP stream using gst-rtsp-server. It works, but as-is there's no error handling. If the pipeline has a typo or there's some issue connecting to the video source, I don't see a stack trace or any logging. Where would I hook in code to handle problems like this?
I should mention that I'm a complete beginner to the GObject world. I suspect there is a standard way for these libraries to report errors but I haven't been able to find anything in the documentation I've read about how that's done.
In case it's helpful, here is my code as I have it right now:
from threading import Thread
from time import sleep
import signal
import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstRtsp", "1.0")
gi.require_version("GstRtspServer", "1.0")
from gi.repository import GLib, GObject, Gst, GstRtsp, GstRtspServer
PIPELINE = (
"( videotestsrc ! vp8enc ! rtpvp8pay name=pay0 pt=96 )")
def main():
GObject.threads_init()
Gst.init(None)
server = GstRtspServer.RTSPServer.new()
server.props.service = "3000"
server.attach(None)
loop = GLib.MainLoop.new(None, False)
def on_sigint(_sig, _frame):
print("Got a SIGINT, closing...")
loop.quit()
signal.signal(signal.SIGINT, on_sigint)
def run_main_loop():
loop.run()
main_loop_thread = Thread(target=run_main_loop)
main_loop_thread.start()
media_factory = GstRtspServer.RTSPMediaFactory.new()
media_factory.set_launch(PIPELINE)
media_factory.set_shared(True)
server.get_mount_points().add_factory("/test", media_factory)
print("Stream ready at rtsp://127.0.0.1:3000/test")
while loop.is_running():
sleep(0.1)
if __name__ == "__main__":
main()
So you can override do_handle_message in Gst.Bin in the following way:
import gi
gi.require_version("Gst", "1.0")
from gi.repository import Gst, GLib
Gst.init(None)
class SubclassBin(Gst.Bin):
def do_handle_message(self, message):
if message.type == Gst.MessageType.ERROR:
error, message = message.parse_error()
# TODO: Do something with the error
# Call the base Gst.Bin do_handle_message
super().do_handle_message(self, message)
subclass_bin = SubclassBin("mybin")
That said, I'm not sure how to tell GstRtspServer.RTSPMediaFactory to use SubclassBin instead of Gst.Bin because as far as I can tell, the only way to connect any Bin to RTSPMediaFactory is through the set_launch method which wants a pipeline string instead of a prebuilt bin. If there was a way of adding a prebuilt bin to RTSPMediaFactory, this would be a complete answer...but unfortunately this is as far as I can get.
Edit: I would recommend using bkanuka's solution (Subclassing Gst.Bin) instead of the below one, which has many drawbacks.
After some more experimentation with the GStreamer and the RTSP server library, the error handling situation is complicated.
The Normal Way
The canonical way to see errors on a GStreamer pipeline is to add a watcher to the pipeline's bus and listen for error messages.
def watcher(bus, message, *user_data);
if message.type == Gst.MessageType.ERROR:
error, message = message.parse_error()
# TODO: Do something with the error
my_pipeline.get_bus().add_watch(
GLib.PRIORITY_DEFAULT,
watcher,
None)
However, you cannot do this with pipelines you supply to a GstRtspServer. This is because GstRtspServer expects to be able to install its own watcher on the pipeline's bus and only one watcher can be attached to a bus at once. This is especially unfortunate because this prevents us from listening to any events on the pipeline, not just errors.
My Workaround
We can split the pipeline into two pieces: one that's in charge of the error-prone process of connecting to the source and decoding frames, and another that is in charge of encoding the resulting frames and payloading them for the GstRtspServer. We can then use the intervideo plugin to communicate between the two.
For example, let's say you're trying to stream from a file in the VP8 format. Our first pipeline that's in charge of reading and decoding frames would look like this:
filesrc location="{filepath}" ! decodebin ! intervideosink channel="file-channel"
... and our second pipeline that's in charge of encoding and payloading the frame would look like this:
intervideosrc channel="file-channel" ! videoconvert ! vp8enc deadline=1 ! rtpvp8pay name=pay0 pt=96
The key here is that only the second pipeline has to be managed by GstRtspServer, since it's the pipeline that provides the payloaded data. The first one is managed by us and we can attach our own watcher to it
that responds intelligently to errors and does whatever else we need. This of course is not a perfect solution because we aren't able to respond to errors related to encoding and payloading, but we've gained the ability to receive errors related to reading the file and decoding it. We're now also able to do other message-related tasks, like intercepting an end-of-stream message to loop the video file.
I am trying to save a stream from webcam as series of image using gstreamer. I have written this code so far...
#!/usr/bin/python
import sys, os
import pygtk, gtk, gobject
import pygst
pygst.require("0.10")
import gst
def __init__(self):
#....
# Code to create a gtk Window
#....
self.player = gst.Pipeline("player")
source = gst.element_factory_make("v4l2src", "video-source")
sink = gst.element_factory_make("xvimagesink", "video-output")
caps = gst.Caps("video/x-raw-yuv, width=640, height=480")
filter = gst.element_factory_make("capsfilter", "filter")
filter.set_property("caps", caps)
self.player.add(source, filter, sink)
gst.element_link_many(source, filter, sink)
After this, I am trying to create a signal over the bus to listen for any message from the source or the sink to indicate a new frame has been sent or received, so that it can be saved.
bus = self.player.get_bus()
bus.add_signal_watch()
bus.connect("message::any", self.save_file,"Save file")
where save_file is my callback, where I want to save the file.
def save_file(self, bus, msg):
print "SAVED A NEW FILE"
I have two questions,
How do I invoke this callback. The message::any is not working.
When this message is invoked, how do I get access to the image buffer.
UPDATE (4-12-2012):
Couple of links for reference
A python interface for v4l. But it has not been working for me. It seems to crash when i try to grab on 12.04 Ubuntu.
http://code.google.com/p/python-video4linux2/
A webcam viewer code for those interested. But this is not what I want since it uses gst-launch and does not provide the level of pipeline control I want to have. http://pygstdocs.berlios.de/pygst-tutorial/webcam-viewer.html
Gstreamer Bus is not intended to be used for this purpose. Messages that are put there signal rather some special event like end-of-stream, element state change and so on. Buffers (images) flowing through elements usualy don't generate any messages on the bus.
You may consider several possibilities:
make "tee" element before videosink and connect "multifilesink" in parallel to videosink (you may want to see some image encoders like pngenc or jpegenc and put one of them before multifilesink")
like before, but use "appsink" that allow you to handle buffers and do whatever-you-want with them
if you want to switch dumping on and off, consider using "valve" element
You may want to set "sync" property to false on your additional sink (Which cause buffers to be dumped as soon as possible without syncing to clock). Consider also adding some queues after tee (without this deadlock may occur during ready->paused transition).
I am not sure if my response after years will be useful for you. but hope, it will be useful for others.
to receive a message that you have received a buffer, you can use gstreamer probes.
It could be something similar:
def make_pipeline(self):
CLI2 = [
'v4l2src ! video/x-raw,format=RGB,width=640,height=480,framerate=30/1 ! ',
'videoconvert ! x264enc bitrate=128 ! mpegtsmux name="mux" ! hlssink name="sink"',
]
gcmd = ''.join(CLI2)
self.pipeline = Gst.parse_launch(gcmd)
self.hlssink = self.pipeline.get_by_name("sink")
self.hlssink.set_property("target-duration",2)
self.hlssink_pad = self.hlssink.get_static_pad("sink")
probe_id = self.hlssink_pad.add_probe(Gst.PadProbeType.EVENT_UPSTREAM,probe_callback)
and then, the probe callback function could be:
def probe_callback(hlssink_pad,info):
info_event = info.get_event()
info_structure = info_event.get_structure()
do_something_with_this_info
return Gst.PadProbeReturn.PASS
So, every time there is an event on either a source pad or sink pad, the probe callback function will be called in the main thread.
Hope this helps!
I have been trying to work with the standard GPS (gps.py) module in python 2.6. This is supposed to act as a client and read GPS Data from gpsd running in Ubuntu.
According to the documentation from GPSD webpage on client design (GPSD Client Howto), I should be able to use the following code (slightly modified from the example) for getting latest GPS Readings (lat long is what I am mainly interested in)
from gps import *
session = gps() # assuming gpsd running with default options on port 2947
session.stream(WATCH_ENABLE|WATCH_NEWSTYLE)
report = session.next()
print report
If I repeatedly use the next() it gives me buffered values from the bottom of the queue (from when the session was started), and not the LATEST Gps reading. Is there a way to get more recent values using this library? In a Way, seek the Stream to the latest values?
Has anyone got a code example using this library to poll the gps and get the value i am looking for ?
Here is what I am trying to do:
start the session
Wait for user to call the gps_poll() method in my code
Inside this method read the latest TPV (Time Position Velocity) report and return lat long
Go back to waiting for user to call gps_poll()
What you need to do is regularly poll 'session.next()' - the issue here is that you're dealing with a serial interface - you get results in the order they were received. Its up to you to maintain a 'current_value' that has the latest retrieved value.
If you don't poll the session object, eventually your UART FIFO will fill up and you won't get any new values anyway.
Consider using a thread for this, don't wait for the user to call gps_poll(), you should be polling and when the user wants a new value they use 'get_current_value()' which returns current_value.
Off the top of my head it could be something as simple as this:
import threading
import time
from gps import *
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.session = gps(mode=WATCH_ENABLE)
self.current_value = None
def get_current_value(self):
return self.current_value
def run(self):
try:
while True:
self.current_value = self.session.next()
time.sleep(0.2) # tune this, you might not get values that quickly
except StopIteration:
pass
if __name__ == '__main__':
gpsp = GpsPoller()
gpsp.start()
# gpsp now polls every .2 seconds for new data, storing it in self.current_value
while 1:
# In the main thread, every 5 seconds print the current value
time.sleep(5)
print gpsp.get_current_value()
The above answers are very inefficient and overly complex for anyone using modern versions of gpsd and needing data at only specific times, instead of streaming.
Most GPSes send their position information at least once per second. Presumably since many GPS-based applications desire real-time updates, the vast majority of gpsd client examples I've seen use the above method of watching a stream from gpsd and receiving realtime updates (more or less as often as the gps sends them).
However, if (as in the OP's case) you don't need streaming information but just need the last-reported position whenever it's requested (i.e. via user interaction or some other event), there's a much more efficient and simpler method: let gpsd cache the latest position information, and query it when needed.
The gpsd JSON protocol has a ?POLL; request, which returns the most recent GPS information that gpsd has seen. Instead of having to iterate over the backlog of gps messages, and continually read new messages to avoid full buffers, you can send a ?WATCH={"enable":true} message at the start of the gpsd session, and then query the latest position information whenever you need it with ?POLL;. The response is a single JSON object containing the most recent information that gpsd has seen from the GPS.
If you're using Python3, the easiest way I've found is to use the gpsd-py3 package available on pypi. To connect to gpsd, get the latest position information, and print the current position:
import gpsd
gpsd.connect()
packet = gpsd.get_current()
print(packet.position())
You can repeat the gpsd.get_current() call whenever you want new position information, and behind the scenes the gpsd package will execute the ?POLL; call to gpsd and return an object representing the response.
Doing this with the built-in gps module isn't terribly straightforward, but there are a number of other Python clients available, and it's also rather trivial to do with anything that can perform socket communication, including this example using telnet:
$ telnet localhost 2947
Trying ::1...
Connected to localhost.
Escape character is '^]'.
{"class":"VERSION","release":"3.16","rev":"3.16","proto_major":3,"proto_minor":11}
?WATCH={"enable":true}
{"class":"DEVICES","devices":[{"class":"DEVICE","path":"/dev/pts/10","driver":"SiRF","activated":"2018-03-02T21:14:52.687Z","flags":1,"native":1,"bps":4800,"parity":"N","stopbits":1,"cycle":1.00}]}
{"class":"WATCH","enable":true,"json":false,"nmea":false,"raw":0,"scaled":false,"timing":false,"split24":false,"pps":false}
?POLL;
{"class":"POLL","time":"2018-03-02T21:14:54.873Z","active":1,"tpv":[{"class":"TPV","device":"/dev/pts/10","mode":3,"time":"2005-06-09T14:34:53.280Z","ept":0.005,"lat":46.498332203,"lon":7.567403907,"alt":1343.165,"epx":24.829,"epy":25.326,"epv":78.615,"track":10.3788,"speed":0.091,"climb":-0.085,"eps":50.65,"epc":157.23}],"gst":[{"class":"GST","device":"/dev/pts/10","time":"1970-01-01T00:00:00.000Z","rms":0.000,"major":0.000,"minor":0.000,"orient":0.000,"lat":0.000,"lon":0.000,"alt":0.000}],"sky":[{"class":"SKY","device":"/dev/pts/10","time":"2005-06-09T14:34:53.280Z","xdop":1.66,"ydop":1.69,"vdop":3.42,"tdop":3.05,"hdop":2.40,"gdop":5.15,"pdop":4.16,"satellites":[{"PRN":23,"el":6,"az":84,"ss":0,"used":false},{"PRN":28,"el":7,"az":160,"ss":0,"used":false},{"PRN":8,"el":66,"az":189,"ss":45,"used":true},{"PRN":29,"el":13,"az":273,"ss":0,"used":false},{"PRN":10,"el":51,"az":304,"ss":29,"used":true},{"PRN":4,"el":15,"az":199,"ss":36,"used":true},{"PRN":2,"el":34,"az":241,"ss":41,"used":true},{"PRN":27,"el":71,"az":76,"ss":42,"used":true}]}]}
?POLL;
{"class":"POLL","time":"2018-03-02T21:14:58.856Z","active":1,"tpv":[{"class":"TPV","device":"/dev/pts/10","mode":3,"time":"2005-06-09T14:34:53.280Z","ept":0.005,"lat":46.498332203,"lon":7.567403907,"alt":1343.165,"epx":24.829,"epy":25.326,"epv":78.615,"track":10.3788,"speed":0.091,"climb":-0.085,"eps":50.65,"epc":157.23}],"gst":[{"class":"GST","device":"/dev/pts/10","time":"1970-01-01T00:00:00.000Z","rms":0.000,"major":0.000,"minor":0.000,"orient":0.000,"lat":0.000,"lon":0.000,"alt":0.000}],"sky":[{"class":"SKY","device":"/dev/pts/10","time":"2005-06-09T14:34:53.280Z","xdop":1.66,"ydop":1.69,"vdop":3.42,"tdop":3.05,"hdop":2.40,"gdop":5.15,"pdop":4.16,"satellites":[{"PRN":23,"el":6,"az":84,"ss":0,"used":false},{"PRN":28,"el":7,"az":160,"ss":0,"used":false},{"PRN":8,"el":66,"az":189,"ss":45,"used":true},{"PRN":29,"el":13,"az":273,"ss":0,"used":false},{"PRN":10,"el":51,"az":304,"ss":29,"used":true},{"PRN":4,"el":15,"az":199,"ss":36,"used":true},{"PRN":2,"el":34,"az":241,"ss":41,"used":true},{"PRN":27,"el":71,"az":76,"ss":42,"used":true}]}]}
Adding my two cents.
For whatever reason my raspberry pi would continue to execute a thread and I'd have to hard reset the pi.
So I've combined sysnthesizerpatel and an answer I found on Dan Mandel's blog here.
My gps_poller class looks like this:
import os
from gps import *
from time import *
import time
import threading
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.session = gps(mode=WATCH_ENABLE)
self.current_value = None
self.running = True
def get_current_value(self):
return self.current_value
def run(self):
try:
while self.running:
self.current_value = self.session.next()
except StopIteration:
pass
And the code in use looks like this:
from gps_poll import *
if __name__ == '__main__':
gpsp = GpsPoller()
try:
gpsp.start()
while True:
os.system('clear')
report = gpsp.get_current_value()
# print report
try:
if report.keys()[0] == 'epx':
print report['lat']
print report['lon']
time.sleep(.5)
except(AttributeError, KeyError):
pass
time.sleep(0.5)
except(KeyboardInterrupt, SystemExit):
print "\nKilling Thread.."
gpsp.running = False
gpsp.join()
print "Done.\nExiting."
You can also find the code here: Here and Here
I know its an old thread but just for everyone understanding, you can also use pyembedded python library for this.
pip install pyembedded
from pyembedded.gps_module.gps import GPS
import time
gps = GPS(port='COM3', baud_rate=9600)
while True:
print(gps.get_lat_long())
time.sleep(1)
https://pypi.org/project/pyembedded/
I've been playing around with the pybluez module recently to scan for nearby Bluetooth devices. What I want to do now is extend the program to also find nearby WiFi client devices.
The WiFi client scanner will have need to have a While True loop to continually monitor the airwaves. If I were to write this as a straight up, one file program, it would be easy.
import ...
while True:
client = scan()
print client['mac']
What I want, however, is to make this a module. I want to be able to reuse it later and, possible, have others use it too. What I can't figure out is how to handle the loop.
import mymodule
scan()
Assuming the first example code was 'mymodule', this program would simply print out the data to stdout. I would want to be able to use this data in my program instead of having the module print it out...
How should I code the module?
I think the best approach is going to be to have the scanner run on a separate thread from the main program. The module should have methods that start and stop the scanner, and another that returns the current access point list (using a lock to synchronize). See the threading module.
How about something pretty straightforward like:
mymodule.py
import ...
def scanner():
while True:
client = scan()
yield client['mac']
othermodule.py
import mymodule
for mac in mymodule.scanner():
print mac
If you want something more useful than that, I'd also suggest a background thread as #kindall did.
Two interfaces would be useful.
scan() itself, which returned a list of found devices, such that I could call it to get an instantaneous snapshot of available bluetooth. It might take a max_seconds_to_search or a max_num_to_return parameter.
A "notify on found" function that accepted a callback. For instance (maybe typos, i just wrote this off the cuff).
def find_bluetooth(callback_func, time_to_search = 5.0):
already_found = []
start_time = time.clock()
while 1:
if time.clock()-start_time > 5.0: break
found = scan()
for entry in found:
if entry not in already_found:
callback_func(entry)
already_found.append(entry)
which would be used by doing this:
def my_callback(new_entry):
print new_entry # or something more interesting...
find_bluetooth(my_callback)
If I get your question, you want scan() in a separate file, so that it can be reused later.
Create utils.py
def scan():
# write code for scan here.
Create WiFi.py
import utils
def scan_wifi():
while True:
cli = utils.scan()
...
return