I tried to read up on how to open files properly in python (concerning formatting of special characters, in this case carriage return) but still can't figure it out.
I'm writing this script in nano on my rpi (over ssh from my pc using putty).
The script collects data from sensors connected via I2C and SPI and prints them to logfiles. Any events, anomalies and errors are to be registered in an eventlog, with timestamp. that way, I dont need the ordinary print function and can make the program run in the backround and keep track of what it's doing by looking at the eventlog over FTP.
The following parts of the program gives different line handling
first occasion
second occasion
The first gives a file that gets a ^M at the beginning of each line except the first when i view it in nano, but it looks fine and dandy when i open it in notebook on my PC.
The second looks good in nano, but have no newlines or carriage returns when I open it in notepad and is impossible to read properly.
First: Why are they different? I have looked at it over and over again. Because one is inside a function and the other "raw" in the code (it is inside a while loop)
Second: What does it take to get the files to look proper in both nano and notepad?
Hopefully I've given enough details :)
Related
So this one is a doozie, and a little too specific to find an answer online.
I am writing to a file in C++ and reading that file in Python at the same time to move a robot. Or trying to.
When I try running both programs at the same time, the C++ one runs first and then the Python one runs.
Here's the command I use:
./ColorFollow & python fileToHex.py
This happens even if I switch the order of commands.
Even if I run them in different terminals (which is the same thing, just covering all bases).
Both the Python and C++ code read / write in 'infinite' loops, so these two should run until I say stop.
The code works fine; when the Python script finally runs the robot moves as intended. It's just that the code doesn't run at the same time.
Is there a way to make this happen, or is this impossible?
If you need more information, lemme know, but the code is pretty much what you'd expect it to be.
If you are using Linux, & will release bash session and in this case, CollorFlow and fileToXex.py will run in different bash sessions.
At the same time, composition ./ColorFollow | python fileToHex.py looks interesting, cause you redirect stdout of ColorFollow to fileToHex.py stdin - it can syncronize scripts by printing some code string upon exit, then reading it by fileToHex.py and exit as well.
I would create some empty file like /var/run/ColorFollow.flag and write there 1 when one of processes exit. Not a pipe - cause we do not care which process will start first. So, if next loop step of ColorFollow sees 1 in the file, it deletes it and exits (means that fileToHex already exited). The same - for fileToHex - check flag file each loop step and exit if it exists, after deleting flag file.
I'm developing a PyQt applications so there's a good possibility segfaults could happen.
I want to use the faulthandler module to catch these. Now instead of writing to stderr I want to do the following:
Make faulthandler write into a file with a known location.
When starting again (normally or after a crash), check if that file exists and has a crash log in it.
If it does, open a dialog which asks the user to open a bug report with the crash log, then delete it.
Now this works fine, except when I run multiple instances of my application.
Then I thought I could write into a random file with a known location (say, crash-XXXXX.log), and then when starting check for crash-*.log, and if it's non-empty do the same thing as above.
However when doing it that way, at least on Linux I'll be able to delete the file while another instance might still have it open, and then if that instance crashes the log gets lost.
I also can't just open() the file at the right time as faulthandler wants an open file.
I'm searching for a solution which:
Works with multiple instances
Catches crashes of all these instances correctly
Only opens the crash dialog one time, when starting a new instance after one crashed.
Doesn't leave any stale files behind after all instances are closed.
Works at least on Linux and Windows
I've thought about some different approaches, but all of them have one of these drawbacks.
I'm working on a script that should check on certain system events (like opening of a file, or changing of a registry key) and start further actions depending on that. But I haven't found a clean way to get the information into my script.
I'm looking for a way to get the output of Sysinternals Process Monitor into another program. This should happen without user interaction in close to real time; so saving into a CSV/XML and than using this doesn't work.
I've checked on using the backing file, but this is in the Process Monitor PML format, which i haven't found to be documented anywhere.
Does anybody know a way how I can get the output of Process Monitor into my script?
Or an other (not too messy) way to get a real time list of opened files, registry keys etc into a python program?
Thanks!
If you want to parse stdout or a file, and your ok with a 32 bit only solution, try Dr Strace or ntstrace.
YOu could also look into ospy or another ProcMon alternative. ospy is open source, so at the very least you could look at the source code for capturing events.
Here is a list of alternates to ProcMon.
I'm writing small bits of code inside org-mode files. This bits of code are slow (copy files from remote machines) and I wish to see how the copy progress (sometimes the connection to the remote machine fails and I wish to know). For that, I want to print the serial number of the currently accessed file.
Org-mode's code-block have two problems with this:
It places the either the printed messages or the returned variable in the results part of the block.
It does so only once the code ends.
Is there a way to get the printed output to a separated, live variable?
Reopening sys.stdout so that it is flushed should help.
See How to flush output of Python print?, and this blog post:
http://algorithmicallyrandom.blogspot.com.es/2009/10/python-tips-and-tricks-flushing-stdout.html
I'm working on a Python script to parse Squid(http://www.squid-cache.org/) log files. While the logs are rotated every day to stop them getting to big, they do reach between 40-90MB by the end of each day.
Essentially what I'm doing is reading the file line by line, parsing out the data I need(IP, Requested URL, Time) and adding it to an sqlite database. However this seems to be taking a very long time(It's been running over 20 minutes now)
So obviously, re-reading the file can't be done. What I would like to do is read the file and then detect all new lines written. Or even better, at the start of the day the script will simply read the data in real time as it is added so there will never be any long processing times.
How would I go about doing this?
One way to achieve this is by emulating tail -f. The script would constantly monitor the file and process each new line as it appears.
For a discussion and some recipes, see tail -f in python with no time.sleep
One way to do this is to use file system monitoring with py-inotify http://pyinotify.sourceforge.net/ - and set a callback function to be executed whenever
the log file size changed.
Another way to do it, without requiring external modules, is to record in the filesystem
(possibily on your sqlite database itself), the offset of the end of the lest line read on the log file, (which you get with with file.tell() ), and just read the newly added lines
from that offset onwards, which is done with a simple call to file.seek(offset) before looping through the lines.
The main difference of keeping track of the offset and the "tail" emulation described ont he other post is that this one allows your script to be run multiple times, i.e. - no need for it to be running continually, or to recover in case of a crash.